## 1. Project Overview & Quickstart (alexzhang13/rlm)

## File: README.md

---

<h1 align="center" style="font-size:2.8em">
Recursive Language Models (RLMs)
</h1>







## Overview
Recursive Language Models (RLMs) are a task-agnostic inference paradigm for language models (LMs) to handle near-infinite length contexts by enabling the LM to *programmatically* examine, decompose, and recursively call itself over its input. RLMs replace the canonical `llm.completion(prompt, model)` call with a `rlm.completion(prompt, model)` call, acting as a "language model". RLMs offload the context as a variable in a REPL environment that the LM can interact with and launch sub-LM calls inside of.

RLMs are a bet on future "language model" design choices. We argue for a [CodeAct](https://arxiv.org/abs/2402.01030)-style harness (i.e. all language models should have access to a code environment) with sub-(R)LM calls as functions in code, and context / prompts as objects in code. RLMs explicitly defer code execution with sub-calls as functions to the language model itself, which is incredibly flexible and lends itself well to scale if trained correctly. We want to move away from the JSON tool-calling standard for both sub-agents and generic tool calls. The naming comes from the fact that such a system is itself a "language model" (a probabilistic mapping from text to text) that builds around and relies on recursive sub-LLM calls.

This repository provides both an extensible inference engine and training environment for using RLMs around standard API-based and local LLMs. The initial experiments and idea were proposed in a [blogpost](https://alexzhang13.github.io/blog/2025/rlm/) in 2025, with expanded results in an [arXiv preprint](https://arxiv.org/abs/2512.24601).

We now also include a [verifiers](https://github.com/PrimeIntellect-ai/verifiers) training environment based on Prime Intellect's [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl) in the `training/` folder. Train your own RLMs, which directly can be plugged into our inference engine!

> [!NOTE]
> This repository contains inference code for RLMs with support for various sandbox environments. Open-source contributions are welcome. This repository is maintained by the authors of the paper from the MIT OASYS lab.

## Quick Setup
> [!NOTE]
> `rlms` requires **Python 3.11 or later**.

You can try out RLMs quickly by installing from PyPi:
```bash
pip install rlms
```

The default RLM client uses a REPL environment that runs on the host process through Python `exec` calls. It uses the same virtual environment as the host process (i.e. it will have access to the same dependencies), but with some limitations in its available global modules. As an example, we can call RLM completions using GPT-5-nano:
```python
from rlm import RLM

rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-5-nano"},
    verbose=True,  # For printing to console with rich, disabled by default.
)

print(rlm.completion("Print me the first 100 powers of two, each on a newline.").response)
```


**Manual Setup**

Set up the dependencies with `uv` (or your virtual environment of choice):
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
uv init && uv venv --python 3.12  # change version as needed
uv pip install -e .
```

This project includes a `Makefile` to simplify common tasks.

- `make install`: Install base dependencies.
- `make check`: Run linter, formatter, and tests.

To run a quick test, the following will run an RLM query with the OpenAI client using your environment variable `OPENAI_API_KEY` (feel free to change this). This will generate console output as well as a log which you can use with the visualizer to explore the trajectories.
```bash
make quickstart
```



## REPL Environments
We support two types of REPL environments -- isolated, and non-isolated. Non-isolated environments (default) run code execution on the same machine as the RLM (e.g. through `exec`), which is pretty reasonable for some local low-risk tasks, like simple benchmarking, but can be problematic if the prompts or tool calls can interact with malicious users. Fully isolated environments use cloud-based sandboxes (e.g. Prime Sandboxes, [Modal Sandboxes](https://modal.com/docs/guide/sandboxes)) to run code generated by the RLM, ensuring complete isolation from the host process. Environments can be added, but we natively support the following: `local` (default), `ipython`, `docker`, `modal`, `prime`, `daytona`, `e2b`.

```python
rlm = RLM(
    environment="...", # "local", "ipython", "docker", "modal", "prime", "daytona", "e2b"
    environment_kwargs={...},
)
```

### Local Environments
The default `local` environment `LocalREPL` runs in the same process as the RLM itself, with specified global and local namespaces for minimal security. Using this REPL is generally safe, but should not be used for production settings. It also shares the same virtual environment (e.g. Conda or uv) as the host process.

#### IPython (*requires `pip install 'rlms[ipython]'`*)
`IPythonREPL` runs cells inside a real IPython session — either in-process (default) or in a separate `ipykernel` subprocess. Subprocess mode adds hard `cell_timeout` enforcement and full namespace isolation from the RLM host. See the [IPythonREPL docs](https://alexzhang13.github.io/rlm/environments/ipython) for details.

#### Docker  (*requires [Docker installed](https://docs.docker.com/desktop/setup/install/)*)
We also support a Docker-based environment called `DockerREPL` that launches the REPL environment as a Docker image. By default, we use the `python:3.11-slim` image, but the user can specify custom images as well. The container runs fully isolated from the host; a lightweight host-side proxy bridges LM access back into the container.

`DockerREPL` supports the full feature set of the local environment: single LM calls (`llm_query` / `llm_query_batched`), recursive sub-RLM calls (`rlm_query` / `rlm_query_batched`, including parallel batched sub-calls bounded by `max_concurrent_subcalls`), `custom_tools` / `custom_sub_tools`, `persistent=True` multi-turn sessions (versioned `context_N` / `history_N` reused across `completion()` calls), and `compaction=True` auto-summarization of the running `history`. For isolated environments, custom tools should be passed as Python code strings or JSON-serializable values (host callables cannot cross the process boundary).

### Isolated Environments
We support several different REPL environments that run on separate, cloud-based machines. Whenever a recursive sub-call is made in these instances, it is requested from the host process.

#### Modal Sandboxes 
To use [Modal Sandboxes](https://modal.com/docs/guide/sandboxes) as the REPL environment, you need to install and authenticate your Modal account.
```bash
uv add modal  # add modal library
modal setup   # authenticate account
```

#### Prime Intellect Sandboxes 
> [!NOTE]
> **Prime Intellect Sandboxes** are currently a beta feature. See the [documentation](https://docs.primeintellect.ai/sandboxes/overview) for more information. We noticed slow runtimes when using these sandboxes, which is currently an open issue.


To use [Prime Sandboxes](https://docs.primeintellect.ai/sandboxes/sdk), install the SDK and set your API key:
```bash
uv pip install -e ".[prime]"
export PRIME_API_KEY=...
```


### Model Providers
We currently support most major clients (OpenAI, Anthropic), as well as the router platforms (OpenRouter, Portkey). For local models, we recommend using vLLM (which interfaces with the [OpenAI client](https://github.com/alexzhang13/rlm/blob/main/rlm/clients/openai.py)). To view or add support for more clients, start by looking at [`rlm/clients/`](https://github.com/alexzhang13/rlm/tree/main/rlm/clients).

## Training
We provide a simple RL training harness for training RLMs used in this repo (specifically the `local` REPL). The implementation uses no sandboxes for simplicity and slots easily your use case, but an ideal setup would use sandboxes for safety. Training logic is isolated to the [`training/`](https://github.com/alexzhang13/rlm/tree/main/training) folder, which exposes `rlm.RLM` as a [`verifiers`](https://github.com/willccbb/verifiers) `Environment` and plugs straight into [`prime-rl`](https://github.com/PrimeIntellect-ai/prime-rl). See the [training README](https://github.com/alexzhang13/rlm/tree/main/training#readme) for the launch command. The harness uses subprocess-isolated local REPL execution (no cloud sandboxes), matching the `local` environment above.

A worked example with an example `.toml` lives in [`training/environments/oolong/`](https://github.com/alexzhang13/rlm/tree/main/training/environments/oolong) (OOLONG long-context QA). New training environments can be added the same way — author a `verifiers` env that wraps your task (see the [verifiers docs](https://verifiers.readthedocs.io/)), then reference it from a config.

## Relevant Reading
* **[Dec '25]** [Recursive Language Models arXiv](https://arxiv.org/abs/2512.24601)
* **[Oct '25]** [Recursive Language Models Blogpost](https://alexzhang13.github.io/blog/2025/rlm/)

If you use this code or repository in your research, please cite:

```bibtex
@misc{zhang2026recursivelanguagemodels,
      title={Recursive Language Models},
      author={Alex L. Zhang and Tim Kraska and Omar Khattab},
      year={2026},
      eprint={2512.24601},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2512.24601},
}
```

## RLMs being used in the wild
There are many amazing demos and production-ready use cases of RLMs. We provide a list of notable examples that explicitly use RLMs as a central piece of their design.
*  [DSPy.RLM](https://github.com/stanfordnlp/dspy) [](https://github.com/stanfordnlp/dspy/stargazers)
*  [Prime Agent](https://github.com/PrimeIntellect-ai/prime-agent) [](https://github.com/PrimeIntellect-ai/prime-agent/stargazers)
*  [Ax](https://github.com/ax-llm/ax) [](https://github.com/ax-llm/ax/stargazers)
*  [context-labs/HALO: **RLM-based Automatic Agent Optimization Loop**](https://github.com/context-labs/halo) [](https://github.com/context-labs/halo/stargazers)
* [viplismism/rlm-cli: **CLI for Recursive Language Models**](https://github.com/viplismism/rlm-cli) [](https://github.com/viplismism/rlm-cli/stargazers)
*  [alphaXiv Official Blog. **Reinforcing Recursive Language Models**](https://www.alphaxiv.org/blog/reinforcement-learning-for-rlms#training)
*  [Daytona. **Building Deep Recursive Language Models**](https://www.daytona.io/docs/en/guides/rlm/recursive-language-models/)
*  [Symbolica. **SotA ARC-AGI-2 Results with REPL Agents**](https://www.symbolica.ai/blog/arcgentica)
*  [Google Cloud Community Articles. **RLMs in ADK**](https://discuss.google.dev/t/recursive-language-models-in-adk/)
*  [Prime Intellect Blog. **Recursive Language Models: *the* paradigm of 2026**](https://www.primeintellect.ai/blog/rlm)

## Optional: Trajectory metadata, logging, and debugging
`RLMChatCompletion` has an optional `metadata` field (default `None`) that holds the full trajectory (run config + all iterations and sub-calls) so you can reconstruct the run. Pass an `RLMLogger` to capture it:

- **In-memory only** (trajectory on `completion.metadata`): `logger=RLMLogger()` (no `log_dir`).
- **Also save to disk** (JSONL for the visualizer): `logger=RLMLogger(log_dir="./logs")`.

**Visualizing logs.** We also provide a simple visualizer to inspect code, sub-LM, and root-LM calls. Use `RLMLogger(log_dir="./logs")` so each completion writes a `.jsonl` file:
```python
from rlm.logger import RLMLogger
from rlm import RLM

logger = RLMLogger(log_dir="./logs")
rlm = RLM(..., logger=logger)
```

To run the visualizer locally, we use Node.js and shadcn/ui:
```
cd visualizer/
npm run dev        # default localhost:3001
```

---

## File: docs/api/rlm.md

---
layout: default
title: RLM Class
parent: API Reference
nav_order: 1
---

# RLM Class Reference
{: .no_toc }

Complete API documentation for the core RLM class.
{: .fs-6 .fw-300 }

## Table of Contents
{: .no_toc .text-delta }

1. TOC
{:toc}

---

## Overview

The `RLM` class is the main entry point for Recursive Language Model completions. It wraps an LM client and execution environment to enable iterative, code-augmented reasoning.

```python
from rlm import RLM

rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-5-nano"},
)
```

---

## Constructor

```python
RLM(
    backend: str = "openai",
    backend_kwargs: dict | None = None,
    environment: str = "local",
    environment_kwargs: dict | None = None,
    depth: int = 0,
    max_depth: int = 1,
    max_iterations: int = 30,
    max_budget: float | None = None,
    max_timeout: float | None = None,
    max_tokens: int | None = None,
    max_errors: int | None = None,
    custom_system_prompt: str | None = None,
    other_backends: list[str] | None = None,
    other_backend_kwargs: list[dict] | None = None,
    logger: RLMLogger | None = None,
    verbose: bool = False,
    persistent: bool = False,
    custom_tools: dict[str, Any] | None = None,
    custom_sub_tools: dict[str, Any] | None = None,
    compaction: bool = False,
    compaction_threshold_pct: float = 0.85,
    on_subcall_start: Callable | None = None,
    on_subcall_complete: Callable | None = None,
    on_iteration_start: Callable | None = None,
    on_iteration_complete: Callable | None = None,
)
```

### Parameters

#### `backend`
{: .no_toc }

**Type:** `Literal["openai", "portkey", "openrouter", "vllm", "anthropic"]`
**Default:** `"openai"`

The LM provider backend to use for the root model.

```python
# OpenAI
rlm = RLM(backend="openai", ...)

# Anthropic
rlm = RLM(backend="anthropic", ...)

# Local vLLM server
rlm = RLM(backend="vllm", ...)
```

---

#### `backend_kwargs`
{: .no_toc }

**Type:** `dict[str, Any] | None`
**Default:** `None`

Configuration passed to the LM client. Required fields vary by backend:

| Backend | Required | Optional |
|:--------|:---------|:---------|
| `openai` | `model_name` | `api_key`, `base_url` |
| `anthropic` | `model_name` | `api_key` |
| `portkey` | `model_name`, `api_key` | `base_url` |
| `openrouter` | `model_name` | `api_key` |
| `vllm` | `model_name`, `base_url` | — |


```python
backend_kwargs = {
    "api_key": "sk-...",
    "model_name": "gpt-4o",
    "base_url": "https://api.openai.com/v1",  # Optional
}
```

---

#### `environment`
{: .no_toc }

**Type:** `Literal["local", "docker", "modal", "prime", "daytona", "e2b"]`
**Default:** `"local"`

The execution environment for running generated code.

| Environment | Description |
|:------------|:------------|
| `local` | Same-process execution with sandboxed builtins (default) |
| `docker` | Containerized execution in Docker |
| `modal` | Cloud sandbox via Modal |
| `prime` | Cloud sandbox via Prime Intellect |
| `daytona` | Cloud sandbox via Daytona |
| `e2b` | Cloud sandbox via E2B |

---

#### `environment_kwargs`
{: .no_toc }

**Type:** `dict[str, Any] | None`
**Default:** `None`

Configuration for the execution environment:

**Local:**
```python
environment_kwargs = {
    "setup_code": "import numpy as np",  # Run before each completion
}
```

**Docker:**
```python
environment_kwargs = {
    "image": "python:3.11-slim",  # Docker image
}
```

**Modal:**
```python
environment_kwargs = {
    "app_name": "my-rlm-app",  # Modal app name
    "timeout": 600,            # Sandbox timeout in seconds
    "image": modal.Image...,   # Custom Modal image (optional)
}
```

---

#### `max_depth`
{: .no_toc }

**Type:** `int`
**Default:** `1`

Maximum recursion depth for nested RLM calls. When `max_depth > 1`, the REPL provides `rlm_query()` and `rlm_query_batched()` functions that spawn child RLMs with their own REPL environments.

When `depth >= max_depth`, `rlm_query()` falls back to a plain `llm_query()` call (no REPL, no iteration).

```python
# Enable one level of recursive sub-calls
rlm = RLM(..., max_depth=2)
```

---

#### `max_iterations`
{: .no_toc }

**Type:** `int`
**Default:** `30`

Maximum number of REPL iterations before forcing a final answer.

Each iteration consists of:
1. LM generates response (potentially with code blocks)
2. Code blocks are executed
3. Results are appended to conversation history

```python
# For complex tasks, allow more iterations
rlm = RLM(..., max_iterations=50)
```

---

#### `max_budget`
{: .no_toc }

**Type:** `float | None`
**Default:** `None`

Maximum total USD cost for a completion. If exceeded, raises `BudgetExceededError`. Requires a backend that reports cost.

---

#### `max_timeout`
{: .no_toc }

**Type:** `float | None`
**Default:** `None`

Maximum wall-clock seconds for a completion. If exceeded, raises `TimeoutExceededError`. The partial answer (if any) is available on the exception.

---

#### `max_tokens`
{: .no_toc }

**Type:** `int | None`
**Default:** `None`

Maximum total tokens (input + output) for a completion. If exceeded, raises `TokenLimitExceededError`.

---

#### `max_errors`
{: .no_toc }

**Type:** `int | None`
**Default:** `None`

Maximum consecutive REPL errors before aborting. The error counter resets on a successful execution. If exceeded, raises `ErrorThresholdExceededError`.

---

#### `custom_system_prompt`
{: .no_toc }

**Type:** `str | None`
**Default:** `None`

Override the default RLM system prompt. The default prompt instructs the LM on:
- How to use the `context` variable
- How to call `llm_query()` / `llm_query_batched()` for plain LM calls
- How to call `rlm_query()` / `rlm_query_batched()` for recursive sub-calls
- How to signal completion by setting `answer["content"]` and `answer["ready"] = True`

```python
custom_prompt = """You are a data analysis expert.
Use the REPL to analyze the context variable.
When done, run a ```repl``` block that does:
    answer["content"] = "your final answer here"
    answer["ready"] = True
"""

rlm = RLM(..., custom_system_prompt=custom_prompt)
```

---

#### `other_backends` / `other_backend_kwargs`
{: .no_toc }

**Type:** `list[str] | None` / `list[dict] | None`
**Default:** `None`

Register additional LM backends. The first `other_backend` is used as the default for depth-routed sub-calls (e.g. `llm_query()` calls from code at depth > 0 are routed to the other backend). Additional backends are registered by model name and can be selected explicitly.

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    other_backends=["anthropic"],
    other_backend_kwargs=[
        {"model_name": "claude-sonnet-4-20250514"},
    ],
)

# Inside REPL, code can call:
# llm_query(prompt)  # Routed to other_backend (Claude) at depth > 0
# llm_query(prompt, model="gpt-4o")  # Explicit model override
```

---

#### `logger`
{: .no_toc }

**Type:** `RLMLogger | None`
**Default:** `None`

Logger for capturing trajectory metadata. When provided, the returned `RLMChatCompletion.metadata` field contains the full trajectory (iterations, code blocks, sub-calls).

```python
from rlm.logger import RLMLogger

# In-memory only (trajectory on result.metadata)
logger = RLMLogger()

# Also save to disk (JSONL for the visualizer)
logger = RLMLogger(log_dir="./logs")

rlm = RLM(..., logger=logger)
```

---

#### `verbose`
{: .no_toc }

**Type:** `bool`
**Default:** `False`

Enable rich console output showing:
- Metadata at startup
- Each iteration's response
- Code execution results
- Final answer and statistics

---

#### `persistent`
{: .no_toc }

**Type:** `bool`
**Default:** `False`

When enabled, reuses the same environment across multiple `completion()` calls. This enables multi-turn conversations where each call adds a new context and the model retains all previous variables and state.

Contexts are versioned (`context_0`, `context_1`, ...) with `context` always aliasing `context_0`. Conversation histories from previous calls are available as `history_0`, `history_1`, etc.

Supports the context manager protocol for automatic cleanup:

```python
with RLM(..., persistent=True) as rlm:
    result1 = rlm.completion("First context")
    result2 = rlm.completion("Second context")  # Can access context_0 and context_1
```

---

#### `custom_tools`
{: .no_toc }

**Type:** `dict[str, Any] | None`
**Default:** `None`

Custom functions and data available in the REPL environment. Callable values are added to globals (callable by the model), non-callable values are added to locals (accessible as variables).

Two formats are supported:

```python
custom_tools = {
    # Plain value
    "fetch_data": my_fetch_function,
    "API_KEY": "sk-...",

    # With description (shown in system prompt)
    "calculator": {
        "tool": calc_function,
        "description": "Performs arithmetic calculations",
    },
}
```

Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `answer`, `SHOW_VARS`, and their batched variants) cannot be used as tool names.

---

#### `custom_sub_tools`
{: .no_toc }

**Type:** `dict[str, Any] | None`
**Default:** `None`

Separate set of custom tools for child RLMs spawned via `rlm_query()`. If `None`, children inherit the parent's `custom_tools`. Pass an empty dict `{}` to disable custom tools for children.

---

#### `compaction`
{: .no_toc }

**Type:** `bool`
**Default:** `False`

When enabled, automatically summarizes the conversation history when token usage exceeds `compaction_threshold_pct` of the model's context window. The full history (including summaries) is available in the REPL as the `history` variable.

---

#### `compaction_threshold_pct`
{: .no_toc }

**Type:** `float`
**Default:** `0.85`

Fraction of the model's context window that triggers compaction. Only used when `compaction=True`.

---

#### Event Callbacks
{: .no_toc }

Optional callbacks for monitoring execution progress:

| Callback | Signature | Triggered when |
|:---------|:----------|:---------------|
| `on_iteration_start` | `(depth: int, iteration_num: int)` | An iteration begins |
| `on_iteration_complete` | `(depth: int, iteration_num: int, duration: float)` | An iteration completes |
| `on_subcall_start` | `(depth: int, model: str, prompt_preview: str)` | A child RLM is spawned |
| `on_subcall_complete` | `(depth: int, model: str, duration: float, error: str \| None)` | A child RLM finishes |

---

## Methods

### `completion()`

Main entry point for RLM completions.

```python
def completion(
    self,
    prompt: str | dict[str, Any],
    root_prompt: str | None = None,
) -> RLMChatCompletion
```

#### Parameters

**`prompt`**
{: .no_toc }

The context/input to process. Becomes the `context` variable in the REPL.

```python
# String input
result = rlm.completion("Analyze this text...")

# Structured input (serialized to JSON)
result = rlm.completion({
    "documents": [...],
    "query": "Find relevant sections",
})

# List input
result = rlm.completion(["doc1", "doc2", "doc3"])
```

**`root_prompt`**
{: .no_toc }

Optional short prompt shown to the root LM on every iteration. Useful for Q&A tasks where the question should be visible throughout.

```python
# The context is the document, but the LM sees the question each iteration
result = rlm.completion(
    prompt=long_document,
    root_prompt="What is the main theme of this document?"
)
```

#### Returns

`RLMChatCompletion` dataclass:

```python
@dataclass
class RLMChatCompletion:
    root_model: str              # Model name used
    prompt: str | dict           # Original input
    response: str                # Final answer
    usage_summary: UsageSummary  # Token usage
    execution_time: float        # Total seconds
    metadata: dict | None        # Full trajectory when logger is provided
```

#### Example

```python
result = rlm.completion(
    "Calculate the factorial of 100 and return the number of digits."
)

print(result.response)          # "158"
print(result.execution_time)    # 12.34
print(result.metadata)          # Trajectory dict (if logger provided), else None
print(result.usage_summary.to_dict())
# {'model_usage_summaries': {'gpt-4o': {'total_calls': 5, ...}}}
```

### `close()`

Clean up persistent environment resources. Called automatically when using the context manager protocol (`with RLM(...) as rlm:`).

---

## Response Types

### `RLMChatCompletion`

```python
from rlm.core.types import RLMChatCompletion

result: RLMChatCompletion = rlm.completion(...)

result.root_model      # "gpt-4o"
result.prompt          # Original input
result.response        # Final answer string
result.execution_time  # Total time in seconds
result.usage_summary   # UsageSummary object
result.metadata        # Full trajectory dict (if logger provided)
```

### `UsageSummary`

```python
from rlm.core.types import UsageSummary

usage: UsageSummary = result.usage_summary
usage.to_dict()
# {
#     "model_usage_summaries": {
#         "gpt-4o": {
#             "total_calls": 5,
#             "total_input_tokens": 15000,
#             "total_output_tokens": 2000
#         }
#     }
# }
```

---

## REPL Functions

The following functions are available to model-generated code inside the REPL:

| Function | Description |
|:---------|:------------|
| `llm_query(prompt, model=None)` | Single plain LM completion. Fast, no REPL or iteration. |
| `llm_query_batched(prompts, model=None)` | Multiple plain LM completions concurrently. A single failed call doesn't fail the batch — that slot returns `"Error: llm() call failed - <msg>"`, the rest return normally. |
| `rlm_query(prompt, model=None)` | Spawn a child RLM with its own REPL for deeper thinking. Falls back to `llm_query` at max depth. |
| `rlm_query_batched(prompts, model=None)` | Spawn multiple child RLMs. Falls back to `llm_query_batched` at max depth. |
| `answer` | A dict (`{"content": "", "ready": False}`). Set `answer["content"]` to your final answer and `answer["ready"] = True` to terminate the run. |
| `SHOW_VARS()` | List all user-created variables in the REPL. |
| `print(...)` | Print output visible to the model in the next iteration. |

---

## Error Handling

RLM follows a "fail fast" philosophy:

```python
# Missing required argument
rlm = RLM(backend="vllm", backend_kwargs={"model_name": "llama"})
# Raises: AssertionError: base_url is required for vLLM

# Unknown backend
rlm = RLM(backend="unknown")
# Raises: ValueError: Unknown backend: unknown
```

If the RLM exhausts `max_iterations` without the model setting `answer["ready"] = True`, it prompts the LM one more time to provide a final answer based on the conversation history.

RLM raises explicit exceptions when limits are exceeded:

| Exception | Raised when | Key attributes |
|:----------|:------------|:---------------|
| `BudgetExceededError` | `max_budget` exceeded | `spent`, `budget` |
| `TimeoutExceededError` | `max_timeout` exceeded | `elapsed`, `timeout`, `partial_answer` |
| `TokenLimitExceededError` | `max_tokens` exceeded | `tokens_used`, `token_limit`, `partial_answer` |
| `ErrorThresholdExceededError` | `max_errors` consecutive errors | `error_count`, `threshold`, `last_error`, `partial_answer` |
| `CancellationError` | `KeyboardInterrupt` during completion | `partial_answer` |

All exceptions are importable from the top-level package:

```python
from rlm import RLM, TimeoutExceededError, CancellationError

try:
    result = rlm.completion(prompt)
except TimeoutExceededError as e:
    print(f"Timed out after {e.elapsed:.1f}s, partial: {e.partial_answer}")
except CancellationError as e:
    print(f"Cancelled, partial: {e.partial_answer}")
```

---

## Thread Safety

Each `completion()` call:
1. Spawns its own `LMHandler` socket server
2. Creates a fresh environment instance (unless persistent)
3. Cleans up both when done

This makes `completion()` calls independent, but the `RLM` instance itself should not be shared across threads without external synchronization.

---

## Example: Full Configuration

```python
import os
from rlm import RLM
from rlm.logger import RLMLogger

logger = RLMLogger(log_dir="./logs")

rlm = RLM(
    # Primary model
    backend="anthropic",
    backend_kwargs={
        "api_key": os.getenv("ANTHROPIC_API_KEY"),
        "model_name": "claude-sonnet-4-20250514",
    },

    # Execution environment
    environment="local",

    # Additional model for sub-calls (routed at depth > 0)
    other_backends=["openai"],
    other_backend_kwargs=[{
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o-mini",
    }],

    # Recursion: allow one level of child RLMs via rlm_query()
    max_depth=2,
    max_iterations=40,

    # Limits
    max_timeout=120.0,
    max_budget=1.0,
    max_errors=5,

    # Custom tools available in the REPL
    custom_tools={
        "fetch_data": {"tool": my_fetch_fn, "description": "Fetch data from API"},
    },

    # Compaction for long conversations
    compaction=True,
    compaction_threshold_pct=0.85,

    # Debugging
    logger=logger,
    verbose=True,
)

result = rlm.completion(
    prompt=massive_document,
    root_prompt="Summarize the key findings",
)

print(result.response)
print(result.metadata)  # Full trajectory (iterations, sub-calls, etc.)
```

---

## File: docs/architecture.md

---
layout: default
title: Architecture
nav_order: 3
---

# Architecture
{: .no_toc }

How the RLM runtime, LM handler, code execution, and recursive sub-calls fit together.
{: .fs-6 .fw-300 }

## Table of Contents
{: .no_toc .text-delta }

1. TOC
{:toc}

---

## Overview

An RLM completion involves three cooperating pieces:

1. **RLM** (`rlm/core/rlm.py`) — the main loop that drives iteration.
2. **LMHandler** (`rlm/core/lm_handler.py`) — a per-completion TCP server that routes LM API calls.
3. **LocalREPL** (`rlm/environments/local_repl.py`) — the Python execution environment where model-generated code runs.

```
┌────────────────────────────────────────────────────────────────┐
│  RLM.completion(prompt)                                        │
│                                                                │
│  1. Spawn LMHandler (TCP server on localhost, auto port)       │
│  2. Create LocalREPL (in-process exec() namespace)             │
│  3. Iterate:                                                   │
│     a. Send message history → LM backend → get response        │
│     b. Extract ```repl``` code blocks from response            │
│     c. Execute code in LocalREPL                               │
│     d. Append stdout/stderr to message history                 │
│     e. Repeat until answer["ready"] is True or limits exceeded │
│  4. Tear down handler and environment                          │
└────────────────────────────────────────────────────────────────┘
```

---

## LM Handler

### What it is

The LMHandler is a **multi-threaded TCP socket server** that sits between
the execution environment and the actual LM API backends. Every call to
`RLM.completion()` spins up a fresh handler (unless the environment is
persistent and already has one).

### Why a socket server?

The handler exists so that code running inside the execution environment
can make LM calls back to the host process without directly importing or
calling the LM client. This is essential for isolated environments (Docker,
Modal) that run in separate processes or machines — they communicate with
the handler over TCP. The local environment uses the same protocol for
consistency, even though it runs in-process.

### Lifecycle

```python
# Inside RLM._spawn_completion_context():
client = get_client(backend, backend_kwargs)            # 1. Create LM client
lm_handler = LMHandler(client, other_backend_client=…)  # 2. Wrap in handler
lm_handler.start()                                       # 3. Start TCP server (daemon thread)
# … run completion loop …
lm_handler.stop()                                        # 4. Shut down server
```

- The server binds to `127.0.0.1` with port `0` (OS auto-assigns an available port).
- It runs in a **daemon thread** so it doesn't block process exit.
- Each incoming connection is handled by a new thread (`ThreadingTCPServer`).

### Wire protocol

All messages use a simple framing: **4-byte big-endian length prefix + UTF-8 JSON payload**.

```
┌──────────┬─────────────────────────┐
│ 4 bytes  │  N bytes                │
│ len (BE) │  JSON payload (UTF-8)   │
└──────────┴─────────────────────────┘
```

Implemented in `socket_send()` / `socket_recv()` in `rlm/core/comms_utils.py`.

### Client routing

The handler can hold multiple LM clients and routes requests based on the
`model` and `depth` fields in the request:

```python
def get_client(self, model=None, depth=0):
    if model and model in self.clients:
        return self.clients[model]        # Explicit model override
    if depth == 1 and self.other_backend_client:
        return self.other_backend_client  # Depth-based routing
    return self.default_client            # Fallback
```

This lets you use a different (e.g. cheaper/faster) model for sub-LM calls
by specifying `other_backends` / `other_backend_kwargs` in the RLM constructor.

---

## Code Execution Environment (LocalREPL)

### In-process `exec()` — not a subprocess

LocalREPL executes model-generated code **in the same Python interpreter
process** as the RLM, using Python's built-in `exec()`. There is no
subprocess, no fork, and no IPC for code execution.

```python
# Simplified from LocalREPL.execute_code():
combined = {**self.globals, **self.locals}
exec(code, combined, combined)
```

### What this means in practice

- **Fast**: No process spawn overhead. Code execution is as fast as native Python.
- **Persistent namespace**: Variables created in one code block are visible in the next. The `self.locals` dict accumulates state across iterations.
- **Shared memory**: Helper functions like `llm_query()` and `rlm_query()` are plain Python closures in `self.globals`. When model code calls `llm_query("...")`, it's a direct function call within the same process.
- **Limited sandbox**: Dangerous builtins (`eval`, `exec`, `compile`, `input`) are removed from the namespace. This is a soft sandbox — it prevents accidental misuse but is not a security boundary.

### Namespace layout

```
globals (shared across all executions):
├── __builtins__        → _SAFE_BUILTINS (eval/exec/input removed)
├── llm_query()         → plain LM call via handler
├── llm_query_batched() → batched plain LM calls
├── rlm_query()         → recursive RLM sub-call (or fallback to llm_query)
├── rlm_query_batched() → batched recursive sub-calls
├── SHOW_VARS()         → list user-created variables
└── <custom_tools>      → user-provided callable tools

locals (accumulates user variables):
├── context             → alias for context_0
├── context_0           → first context payload
├── context_1, …        → additional contexts (persistent mode)
├── history             → conversation history (persistent/compaction mode)
├── answer              → {"content": "", "ready": False}; set ready=True to finish
└──     → anything created by model code
```

### Scaffold restoration

After each `exec()`, LocalREPL restores all reserved names to prevent model
code from corrupting the environment. If the model writes
`llm_query = "oops"` or `context = None`, the next execution will still
have the real functions and data. See `_restore_scaffold()`.

---

## How `llm_query()` and `rlm_query()` Work

These are the two functions available to model-generated code for making LM calls.
They have very different behaviors:

### `llm_query(prompt, model=None)` — Plain LM call

Always makes a single, direct LM completion. No REPL, no iteration — just
prompt in, text out. Fast and lightweight.

```
Model code: answer = llm_query("Summarize this text: ...")
    │
    ▼
LocalREPL._llm_query()
    │  Creates LMRequest(prompt=..., depth=self.depth)
    │  Opens TCP socket to handler
    ▼
LMHandler (TCP server)
    │  get_client(model, depth) → selects backend
    │  client.completion(prompt) → calls LM API
    ▼
Response flows back over socket
    │
    ▼
Returns response string to model code
```

### `rlm_query(prompt, model=None)` — Recursive RLM sub-call

Spawns a **child RLM** that gets its own REPL and can reason iteratively
over the prompt — just like the parent. Use this when the subtask needs
multi-step reasoning, code execution, or its own iterative problem-solving.

Falls back to `llm_query` when recursion is not available (i.e. the current
depth has reached `max_depth`).

```
Model code: answer = rlm_query("Solve this complex problem: ...")
    │
    ▼
LocalREPL._rlm_query()
    │  if self.subcall_fn is not None:    ← set when max_depth > 1
    │      calls self.subcall_fn(prompt, model)
    │  else:
    │      falls back to _llm_query()
    │
    ▼  (when subcall_fn exists)
RLM._subcall(prompt, model)
    │  next_depth = self.depth + 1
    │  if next_depth >= max_depth:
    │      → plain client.completion() (leaf call, no REPL)
    │  else:
    │      → create child RLM(depth=next_depth, ...)
    │      → child.completion(prompt)  ← full RLM with its own handler + REPL
    │
    ▼
Returns RLMChatCompletion to parent
```

### `llm_query_batched` / `rlm_query_batched`

Same semantics as above, but for multiple prompts. `llm_query_batched` sends
all prompts as a single batched request to the handler, which processes them
concurrently with `asyncio.gather`. `rlm_query_batched` calls `subcall_fn`
sequentially for each prompt (each child RLM is a blocking call).

**Failure handling is per-prompt.** A single failed call does not fail the
whole batch — that slot returns the string `"Error: llm() call failed - <msg>"`
(with the underlying error message) while the other prompts return their real
responses. Results stay aligned with the input prompts by index, so the list is
always the same length as `prompts`.

---

## Recursive Sub-Calls (Depth > 1)

### How depth works

```
max_depth=3

RLM (depth=0)
 └─ rlm_query() → child RLM (depth=1)
     └─ rlm_query() → child RLM (depth=2)
         └─ rlm_query() → plain LM call (depth=3 >= max_depth, no REPL)
```

- `depth=0` is the root RLM that the user calls.
- Each child increments depth by 1.
- When `next_depth >= max_depth`, `_subcall()` does a plain `client.completion()` instead of creating a child RLM. This is the leaf case — no REPL, no iteration.
- `llm_query()` always does a plain LM call regardless of depth. Only `rlm_query()` triggers recursion.

### Each child gets its own handler and environment

When a child RLM is created via `_subcall()`, its `completion()` method calls
`_spawn_completion_context()` which creates:

1. **A new `LMHandler`** listening on a **different auto-assigned port**.
2. **A new `LocalREPL`** with its own isolated namespace.

```
Parent RLM (depth=0)
├── LMHandler #1 on port 52301
├── LocalREPL #1 (depth=1)
│   ├── globals: {llm_query, rlm_query, ...}
│   ├── locals: {context: "parent prompt", ...}
│   └── subcall_fn = RLM._subcall  ← enables rlm_query()
│
└── When model code calls rlm_query("subtask"):
    │
    └── Child RLM (depth=1)
        ├── LMHandler #2 on port 52302  ← NEW handler, NEW port
        ├── LocalREPL #2 (depth=2)      ← NEW namespace
        │   ├── locals: {context: "subtask", ...}
        │   └── subcall_fn = child._subcall (or None if at max_depth-1)
        │
        └── Runs its own iteration loop, returns RLMChatCompletion
```

The child's handler and environment are torn down when the child's `completion()` finishes.

### Resource limits propagate

The parent passes **remaining** budget/timeout/tokens to the child, not the
original totals. This prevents a child from consuming all of the parent's resources:

```python
# In _subcall():
remaining_timeout = self.max_timeout - elapsed  # not self.max_timeout
remaining_budget = self.max_budget - spent       # not self.max_budget
child = RLM(..., max_timeout=remaining_timeout, max_budget=remaining_budget)
```

### Metadata flows back

Each child RLM can have its own `RLMLogger`. When the child completes, its
full trajectory metadata (iterations, code blocks, sub-calls) is captured in
the returned `RLMChatCompletion.metadata` dict. The parent's logger records
this as part of the REPL result's `rlm_calls` list, creating a nested
metadata tree.

---

## Putting It All Together

Here's the complete request flow for a depth-2 RLM call:

```
User: rlm.completion("Analyze this data")
 │
 ▼
RLM (depth=0)
 ├─ _spawn_completion_context()
 │   ├─ LMHandler #1 starts on port 52301
 │   └─ LocalREPL #1 created with context="Analyze this data"
 │
 ├─ Iteration 1: LM generates code
 │   │  ```repl
 │   │  answer = rlm_query("What patterns exist in: " + context[:5000])
 │   │  ```
 │   │
 │   └─ LocalREPL.execute_code() runs the code via exec()
 │       │
 │       ├─ rlm_query() → _rlm_query() → subcall_fn()
 │       │   │
 │       │   └─ RLM._subcall("What patterns exist in: ...")
 │       │       │
 │       │       ├─ depth=1 < max_depth=2, so create child RLM
 │       │       │
 │       │       └─ Child RLM (depth=1)
 │       │           ├─ LMHandler #2 on port 52302
 │       │           ├─ LocalREPL #2 with context="What patterns..."
 │       │           │
 │       │           ├─ Child iteration 1: LM generates code
 │       │           │   │  result = llm_query("Extract key metrics: " + context)
 │       │           │   │
 │       │           │   └─ llm_query() → TCP to Handler #2 → LM API → response
 │       │           │
 │       │           ├─ Child iteration 2: LM sets answer["content"]=result, answer["ready"]=True
 │       │           │
 │       │           └─ Returns RLMChatCompletion to parent
 │       │
 │       └─ child_response = child_completion.response
 │
 ├─ Iteration 2: LM uses child_response, sets answer["content"]=final, answer["ready"]=True
 │
 ├─ LMHandler #1 stops
 └─ Returns RLMChatCompletion to user
```

### Key takeaways

| Aspect | Detail |
|:-------|:-------|
| Code execution | In-process `exec()` in the same Python interpreter. Not a subprocess. |
| LM calls from code | Go through a local TCP socket server (LMHandler), even for in-process execution. |
| Handler per completion | Each `completion()` call gets its own handler on an auto-assigned port. |
| Child RLMs | Created by `_subcall()`, each with its own handler + LocalREPL. Fully independent. |
| `llm_query` vs `rlm_query` | `llm_query` = always plain LM call. `rlm_query` = recursive child RLM (or fallback). |
| Depth limit | At `max_depth`, `rlm_query` falls back to `llm_query`. No further recursion. |
| Resource isolation | Children get remaining budget/timeout, not the full amount. |
| Namespace isolation | Each LocalREPL has its own `globals`/`locals`. No shared state between parent and child. |

---

## File: docs/getting-started.md

---
layout: default
title: Getting Started
nav_order: 2
---

# Getting Started
{: .no_toc }

A complete guide to installing and configuring RLM for your projects.
{: .fs-6 .fw-300 }

## Table of Contents
{: .no_toc .text-delta }

1. TOC
{:toc}

---

## Installation

### Prerequisites

- Python 3.11 or higher
- An API key from a supported LLM provider (OpenAI, Anthropic, etc.)

### Using uv (Recommended)

```bash
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate virtual environment
uv init && uv venv --python 3.12
source .venv/bin/activate

# Install RLM in editable mode
uv pip install -e .
```

### Optional: Modal Support

For cloud-based sandboxed execution:

```bash
# Install Modal extra
uv pip install -e ".[modal]"

# Authenticate Modal
modal setup
```

### Optional: Docker Support

For containerized execution, ensure Docker is installed and running:

```bash
# Verify Docker is available
docker --version
```

---

## Your First RLM Call

### Step 1: Set Up API Keys

Create a `.env` file in your project root:

```bash
# .env
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
PORTKEY_API_KEY=...
```

### Step 2: Basic Usage

```python
import os
from dotenv import load_dotenv
from rlm import RLM

load_dotenv()

# Create RLM instance
rlm = RLM(
    backend="openai",
    backend_kwargs={
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o",
    },
)

# Make a completion call
result = rlm.completion("Calculate the 50th Fibonacci number using Python.")
print(result.response)
```

### Step 3: Enable Verbose Output

See what the RLM is doing step by step:

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o",
    },
    verbose=True,  # Enable rich console output
)
```

This will display:
- Each iteration's LM response
- Code blocks being executed
- Stdout/stderr from execution
- Final answer when reached

---

## Understanding the RLM Class

### Constructor Arguments

| Argument | Type | Default | Description |
|:---------|:-----|:--------|:------------|
| `backend` | `str` | `"openai"` | LM provider backend |
| `backend_kwargs` | `dict` | `None` | Backend-specific configuration |
| `environment` | `str` | `"local"` | Execution environment type |
| `environment_kwargs` | `dict` | `None` | Environment configuration |
| `max_depth` | `int` | `1` | Maximum recursion depth for `rlm_query()` |
| `max_iterations` | `int` | `30` | Max REPL iterations per call |
| `max_budget` | `float` | `None` | Max total USD cost (if provider reports cost) |
| `max_timeout` | `float` | `None` | Max wall-clock seconds per completion |
| `max_tokens` | `int` | `None` | Max total tokens (input + output) per completion |
| `max_errors` | `int` | `None` | Max consecutive REPL errors before abort |
| `custom_system_prompt` | `str` | `None` | Override default system prompt |
| `other_backends` | `list` | `None` | Additional backends for sub-calls |
| `other_backend_kwargs` | `list` | `None` | Configs for additional backends |
| `logger` | `RLMLogger` | `None` | Logger for trajectory tracking and `metadata` capture |
| `verbose` | `bool` | `False` | Enable console output |
| `persistent` | `bool` | `False` | Reuse environment across `completion()` calls |
| `custom_tools` | `dict` | `None` | Custom functions/data available in REPL |
| `custom_sub_tools` | `dict` | `None` | Custom tools for child RLMs (defaults to `custom_tools`) |
| `compaction` | `bool` | `False` | Auto-summarize history when context fills up |
| `compaction_threshold_pct` | `float` | `0.85` | Context usage fraction that triggers compaction |

### The `completion()` Method

```python
result = rlm.completion(
    prompt="Your input text or context",
    root_prompt="Optional: A short prompt visible to the root LM"
)
```

**Parameters:**
- `prompt`: The main context/input (string or dict). This becomes the `context` variable in the REPL.
- `root_prompt`: Optional hint shown to the root LM (useful for Q&A tasks).

**Returns:** `RLMChatCompletion` with:
- `response`: The final answer string
- `usage_summary`: Token usage statistics
- `execution_time`: Total time in seconds
- `root_model`: Model name used
- `prompt`: Original input
- `metadata`: Full trajectory dict (if `logger` was provided, else `None`)

### Depth>1 Recursion

Depth>1 recursive subcalls are supported. The REPL provides two LM call functions:

- **`llm_query(prompt)`** — Always makes a plain, single LM completion. Fast and lightweight. Use for simple extraction, summarization, or Q&A.
- **`rlm_query(prompt)`** — Spawns a child RLM with its own REPL and iterative reasoning. Use for subtasks that need multi-step thinking or code execution. Falls back to `llm_query` when `depth >= max_depth`.

Both have batched variants (`llm_query_batched`, `rlm_query_batched`) for processing multiple prompts concurrently.

See [Architecture](architecture.md) for details on how handlers, environments, and recursive sub-calls work.

### Limits and Exceptions

RLM raises explicit exceptions when limits are exceeded:
- `BudgetExceededError`
- `TimeoutExceededError`
- `TokenLimitExceededError`
- `ErrorThresholdExceededError`
- `CancellationError` (on `KeyboardInterrupt`)

All exceptions are importable from the top-level package (`from rlm import TimeoutExceededError, ...`).

---

## Choosing an Environment

RLM supports several execution environments:

### Local (Default)

Code runs in the same Python process with sandboxed builtins.

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    environment="local",
)
```

**Pros:** Fast, no setup required  
**Cons:** Less isolation from host process

### Docker

Code runs in a Docker container with full isolation. A lightweight host-side
proxy bridges LM access back into the container, so the container never needs
API keys or direct network access to the model provider.

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    environment="docker",
    environment_kwargs={
        "image": "python:3.11-slim",  # Custom image
    },
)
```

`DockerREPL` supports the same capabilities as the local environment:

- `llm_query` / `llm_query_batched` — single LM completions
- `rlm_query` / `rlm_query_batched` — recursive sub-RLM calls (with `max_depth > 1`),
  including parallel batched sub-calls bounded by `max_concurrent_subcalls`
- `custom_tools` / `custom_sub_tools` — injected functions/data (pass as Python
  code strings or JSON-serializable values, since host callables cannot cross
  the container boundary)
- `persistent=True` — multi-turn sessions; the container and its namespace are
  kept alive across `completion()` calls, with versioned `context_N` / `history_N`
- `compaction=True` — auto-summarizes the running `history` when context fills up

**Pros:** Containerized isolation, reproducible, full feature parity with local  
**Cons:** Requires Docker, slower startup

### Modal

Code runs in Modal's cloud sandboxes for full isolation.

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    environment="modal",
    environment_kwargs={
        "app_name": "my-rlm-app",
        "timeout": 600,
    },
)
```

**Pros:** Cloud-native, scalable, fully isolated  
**Cons:** Requires Modal account, network latency

---

## Choosing a Backend

### OpenAI

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o",
        # Optional: custom base URL
        # "base_url": "https://api.openai.com/v1",
    },
)
```

### Anthropic

```python
rlm = RLM(
    backend="anthropic",
    backend_kwargs={
        "api_key": os.getenv("ANTHROPIC_API_KEY"),
        "model_name": "claude-sonnet-4-20250514",
    },
)
```

### Portkey (Router)

```python
rlm = RLM(
    backend="portkey",
    backend_kwargs={
        "api_key": os.getenv("PORTKEY_API_KEY"),
        "model_name": "@openai/gpt-5-nano",  # Portkey model format
    },
)
```

### OpenRouter

```python
rlm = RLM(
    backend="openrouter",
    backend_kwargs={
        "api_key": os.getenv("OPENROUTER_API_KEY"),
        "model_name": "openai/gpt-4o",
    },
)
```

### vLLM (Local)

```python
rlm = RLM(
    backend="vllm",
    backend_kwargs={
        "base_url": "http://localhost:8000/v1",  # Required
        "model_name": "meta-llama/Llama-3-70b",
    },
)
```

---

## Custom Tools

You can provide custom functions and data that the RLM can use in its REPL environment. This allows you to give the model access to domain-specific tools, APIs, or helper functions.

### Basic Usage

```python
def fetch_weather(city: str) -> str:
    """Fetch weather data for a city."""
    # Your API call here
    return f"Weather in {city}: Sunny, 72°F"

def calculate_shipping(weight: float, distance: float) -> float:
    """Calculate shipping cost."""
    return weight * 0.5 + distance * 0.1

rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    custom_tools={
        "fetch_weather": fetch_weather,
        "calculate_shipping": calculate_shipping,
        "API_KEY": "your-api-key",  # Non-callable values become variables
    },
)

result = rlm.completion("What's the weather in Tokyo and calculate shipping for 10kg over 500km?")
```

Inside the REPL, the model can now call:
```python
weather = fetch_weather("Tokyo")
cost = calculate_shipping(10, 500)
```

### Tool Descriptions

You can provide descriptions for your tools that will be included in the system prompt, helping the model understand what each tool does:

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    custom_tools={
        # Dict format: {"tool": callable_or_value, "description": "..."}
        "fetch_weather": {"tool": fetch_weather, "description": "Fetch current weather data for a city name"},
        "calculate_shipping": {"tool": calculate_shipping, "description": "Calculate shipping cost given weight (kg) and distance (km)"},
        "API_KEY": {"tool": "your-api-key", "description": "API key for the weather service"},
    },
)
```

The descriptions are automatically added to the system prompt:
```
6. Custom tools and data available in the REPL:
- `fetch_weather`: Fetch current weather data for a city name
- `calculate_shipping`: Calculate shipping cost given weight (kg) and distance (km)
- `API_KEY`: API key for the weather service
```

### Isolated Environments (Modal, Daytona)

For isolated environments, custom tools must be serializable. You can provide:

1. **Code strings** - Python code that defines the function:
```python
custom_tools = {
    "helper": '''
def helper(x):
    return x * 2
''',
}
```

2. **Serializable data** - JSON-compatible values (strings, numbers, dicts, lists):
```python
custom_tools = {
    "CONFIG": {"api_url": "https://api.example.com", "timeout": 30},
    "ALLOWED_CITIES": ["Tokyo", "London", "New York"],
}
```

---

## Logging and Debugging

### Enable Logging

```python
from rlm import RLM
from rlm.logger import RLMLogger

# Create logger
logger = RLMLogger(log_dir="./logs")

rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    logger=logger,
    verbose=True,
)

result = rlm.completion("...")
# Logs saved to ./logs/rlm_TIMESTAMP_UUID.jsonl
```

### Log File Format

Logs are JSON-lines files with:

```json
{"type": "metadata", "root_model": "gpt-4o", "max_iterations": 30, ...}
{"type": "iteration", "iteration": 1, "response": "...", "code_blocks": [...]}
{"type": "iteration", "iteration": 2, "response": "...", "final_answer": "..."}
```

### Visualizer

Use the included visualizer to explore trajectories:

```bash
cd visualizer/
npm install
npm run dev  # Opens at localhost:3001
```

Upload `.jsonl` log files to visualize:
- Iteration timeline
- Code execution results
- Sub-LM call traces
- Token usage

---

## Next Steps

- [API Reference](api/rlm.md) - Complete RLM class documentation
- [Architecture](architecture.md) - How the handler, REPL, and recursive sub-calls work
- [Environments](environments/) - Deep dive into each environment
- [Backends](backends.md) - Detailed backend configuration

---

## File: visualizer/README.md

Vibe-coded visualizer with [shadcn](https://ui.shadcn.com) for viewing RLM trajectories.

This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).

## Getting Started

First, run the development server:

```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```

Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.

You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.

This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel.

## Learn More

To learn more about Next.js, take a look at the following resources:

- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.

You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!

## Deploy on Vercel

The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.

Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.

## 2. Official Technical Reference & Guides (alexzhang13/alexzhang13.github.io)

# al-folio



## User community

The vibrant community of **al-folio** users is growing!
Academics around the world use this theme for their homepages, blogs, lab pages, as well as webpages for courses, workshops, conferences, meetups, and more.
Check out the community webpages below.
Feel free to add your own page(s) by sending a PR.



| Academics | [★](https://martinbulla.github.io)
[★](https://maruan.alshedivat.com)
[★](https://www.cs.columbia.edu/~chen1ru/)
[★](https://maithraraghu.com)
[★](https://platanios.org)
[★](https://otiliastr.github.io)
[★](https://www.maths.dur.ac.uk/~sxwc62/)
[★](https://jessachandler.com/)
[★](https://mayankm96.github.io/)
[★](https://markdean.info/)
[★](https://kakodkar.github.io/)
[★](https://sahirbhatnagar.com/)
[★](https://spd.gr/)
[★](https://jay-sarkar.github.io/)
[★](https://aborowska.github.io/)
[★](https://aditisgh.github.io/)
[★](https://alexhaydock.co.uk/)
[★](https://alixkeener.net/)
[★](https://andreea7b.github.io/)
[★](https://rishabhjoshi.github.io/)
[★](https://sheelabhadra.github.io/)
[★](https://giograno.me/)
[★](https://immsrini.github.io/)
[★](https://apooladian.github.io/)
[★](https://chinmoy-dutta.github.io/)
[★](https://liamcli.com/)
[★](https://yoonholee.com/)
[★](https://zrqiao.github.io/)
[★](https://abstractgeek.github.io/)
[★](https://www.compphys.de/)
[★](https://julianstreyczek.github.io)
[★](https://sdaza.com)
[★](https://niweera.gq)
[★](https://www.alihkw.com)
[★](https://amirpourmand.ir)
[★](https://scottleechua.github.io)
[★](https://sk1y101.github.io)
[★](https://yyang768osu.github.io)
[★](https://veedata.github.io)
[★](https://K-Wu.github.io)
[★](https://amalawilson.com)
[★](https://tirtharajdash.github.io)
[★](https://carolinacarreira.github.io)
[★](https://manandey.github.io)
[★](https://johanneshoerner.github.io)
[★](https://ioannismavromatis.com)
[★](https://taidnguyen.github.io)
[★](https://lbugnon.github.io)
[★](https://joahannes.github.io)
[★](https://dominikstrb.github.io)
[★](https://tylerbarna.com)
[★](https://daviddmc.github.io/)
[★](https://andreaskuster.ch/)
[★](https://ellisbrown.github.io/)
[★](https://noman-bashir.github.io/)
[★](https://djherron.github.io/)
[★](https://rodosingh.github.io/)
[★](https://vdivakar.github.io/)
[★](https://george-gca.github.io/)
[★](https://bashirkazimi.github.io/)
[★](https://dohaison.github.io/)
[★](https://raphaaal.github.io/)
[★](https://varuniyer.info/)
[★](https://yukimasano.github.io/)
[★](https://hashe037.github.io/)
[★](https://wang-boyu.github.io/)
[★](https://qingqingchen.info)
[★](https://bajinsheng.github.io/)
[★](https://www.silviofanzon.com/)
[★](https://kaikaiyao.github.io/)
[★](https://alchemz.github.io/)
[★](https://samadamday.com/)
[★](https://fanpu.io/)
[★](https://abigalekim.github.io/)
[★](https://lucasresck.github.io/)
[★](https://users.wpi.edu/~lfichera/)
[★](https://anmspro.github.io/)
[★](https://berlyne.net/)
[★](https://filippomazzoli.github.io/)
[★](https://www.escontrela.me/)
[★](https://raffaem.github.io/)
[★](https://cbueth.de/)
[★](https://kyleaoman.github.io/)
[★](https://decwest.github.io/)
[★](https://www.jedburkat.com)
[★](https://hrzhang.me)
[★](https://kudhru.github.io/)
[★](https://mbarbetti.github.io/)
[★](https://www.zhivotenko.com/)
[★](https://giordanodaloisio.github.io/)
[★](https://aadityaura.github.io/)
[★](https://abhinav-mehta.github.io/)
[★](https://shubhashisroydipta.com/)
[★](https://astanziola.github.io)
[★](https://tinkerer.in)
[★](https://sam-bieberich.github.io/)
[★](https://afraniomelo.github.io/en/)
[★](https://jonaruthardt.github.io)
[★](https://www.zla.app/)
[★](https://stavros.github.io)
[★](https://ericslyman.com)
[★](https://ztjona.github.io/)
[★](https://chrischoi314.github.io)
[★](https://riccobelli.faculty.polimi.it)
[★](https://kishanved.tech/)
[★](https://abhilesh.github.io/)
[★](https://jackjburnett.github.io/)
[★](https://physics-morris.github.io/)
[★](https://sraf.ir)
[★](https://acad.garywei.dev/) |
| --- | --- |
| Labs | [★](https://www.haylab.caltech.edu/)
[★](https://sjkimlab.github.io/)
[★](https://systemconsultantgroup.github.io/scg-folio/)
[★](https://decisionlab.ucsf.edu/)
[★](https://programming-group.com/)
[★](https://sailing-lab.github.io/)
[★](https://inbt.jhu.edu/epidiagnostics/)
[★](https://www.nuesl.org/)
[★](https://big-culture.github.io/) |
| Courses | CMU PGM ([S-19](https://sailinglab.github.io/pgm-spring-2019/)) 
CMU DeepRL ([S-21](https://cmudeeprl.github.io/403_website/), [F-21](https://cmudeeprl.github.io/703website_f21/), [S-22](https://cmudeeprl.github.io/403website_s22/), [F-22](https://cmudeeprl.github.io/703website_f22/), [S-23](https://cmudeeprl.github.io/403website_s23/), [F-23](https://cmudeeprl.github.io/703website_f23/)) 
CMU MMML ([F-20](https://cmu-multicomp-lab.github.io/mmml-course/fall2020/), [F-22](https://cmu-multicomp-lab.github.io/mmml-course/fall2022/)) 
CMU AMMML ([S-22](https://cmu-multicomp-lab.github.io/adv-mmml-course/spring2022/), [S-23](https://cmu-multicomp-lab.github.io/adv-mmml-course/spring2023/)) 
CMU ASI ([S-23](https://cmu-multicomp-lab.github.io/asi-course/spring2023/)) 
CMU Distributed Systems ([S-24](https://andrew.cmu.edu/course/15-440/)) |
| Conferences & workshops | ICLR Blog Post Track ([2023](https://iclr-blogposts.github.io/2023/), [2024](https://iclr-blogposts.github.io/2024/about)) 
ML Retrospectives (NeurIPS: [2019](https://ml-retrospectives.github.io/neurips2019/), [2020](https://ml-retrospectives.github.io/neurips2020/); ICML: [2020](https://ml-retrospectives.github.io/icml2020/)) 
HAMLETS (NeurIPS: [2020](https://hamlets-workshop.github.io/)) 
ICBINB (NeurIPS: [2020](https://i-cant-believe-its-not-better.github.io/), [2021](https://i-cant-believe-its-not-better.github.io/neurips2021/)) 
Neural Compression (ICLR: [2021](https://neuralcompression.github.io/)) 
Score Based Methods (NeurIPS: [2022](https://score-based-methods-workshop.github.io/))
Images2Symbols (CogSci: [ 2022](https://images2symbols.github.io/)) 
Medical Robotics Junior Faculty Forum (ISMR: [ 2023](https://junior-forum-ismr.github.io/))
Beyond Vision: Physics meets AI (ICIAP: [2023](https://physicsmeetsai.github.io/beyond-vision/)) 
Workshop on Diffusion Models (NeurIPS: [2023](https://diffusionworkshop.github.io/)) |



## Lighthouse PageSpeed Insights

### Desktop

[](https://htmlpreview.github.io/?https://github.com/alshedivat/al-folio/blob/master/lighthouse_results/desktop/alshedivat_github_io_al_folio_.html)

Run the test yourself: [Google Lighthouse PageSpeed Insights](https://pagespeed.web.dev/report?url=https%3A%2F%2Falshedivat.github.io%2Fal-folio%2F&form_factor=desktop)

### Mobile

[](https://htmlpreview.github.io/?https://github.com/alshedivat/al-folio/blob/master/lighthouse_results/mobile/alshedivat_github_io_al_folio_.html)

Run the test yourself: [Google Lighthouse PageSpeed Insights](https://pagespeed.web.dev/report?url=https%3A%2F%2Falshedivat.github.io%2Fal-folio%2F&form_factor=mobile)

## Table Of Contents

- [al-folio](#al-folio)
  - [User community](#user-community)
  - [Lighthouse PageSpeed Insights](#lighthouse-pagespeed-insights)
    - [Desktop](#desktop)
    - [Mobile](#mobile)
  - [Table Of Contents](#table-of-contents)
  - [Getting started](#getting-started)
  - [Installing and Deploying](#installing-and-deploying)
  - [Customizing](#customizing)
  - [Features](#features)
    - [Light/Dark Mode](#lightdark-mode)
    - [CV](#cv)
    - [People](#people)
    - [Publications](#publications)
    - [Collections](#collections)
    - [Layouts](#layouts)
      - [The iconic style of Distill](#the-iconic-style-of-distill)
      - [Full support for math \& code](#full-support-for-math--code)
      - [Photos, Audio, Video and more](#photos-audio-video-and-more)
    - [Other features](#other-features)
      - [GitHub's repositories and user stats](#githubs-repositories-and-user-stats)
      - [Theming](#theming)
      - [Social media previews](#social-media-previews)
      - [Atom (RSS-like) Feed](#atom-rss-like-feed)
      - [Related posts](#related-posts)
      - [Code quality checks](#code-quality-checks)
  - [FAQ](#faq)
  - [Contributing](#contributing)
    - [Maintainers](#maintainers)
    - [All Contributors](#all-contributors)
  - [Star History](#star-history)
  - [License](#license)

## Getting started

Want to learn more about Jekyll? Check out [this tutorial](https://www.taniarascia.com/make-a-static-website-with-jekyll/). Why Jekyll? Read [Andrej Karpathy's blog post](https://karpathy.github.io/2014/07/01/switching-to-jekyll/)! Why write a blog? Read [Rachel Thomas blog post](https://medium.com/@racheltho/why-you-yes-you-should-blog-7d2544ac1045).

## Installing and Deploying

For installation and deployment details please refer to [INSTALL.md](INSTALL.md).

## Customizing

For customization details please refer to [CUSTOMIZE.md](CUSTOMIZE.md).

## Features

### Light/Dark Mode

This template has a built-in light/dark mode. It detects the user preferred color scheme and automatically switches to it. You can also manually switch between light and dark mode by clicking on the sun/moon icon in the top right corner of the page.



---

### CV

There are currently 2 different ways of generating the CV page content. The first one is by using a json file located in [assets/json/resume.json](assets/json/resume.json). It is a [known standard](https://jsonresume.org/) for creating a CV programmatically. The second one, currently used as a fallback when the json file is not found, is by using a yml file located in [\_data/cv.yml](_data/cv.yml). This was the original way of creating the CV page content and since it is more human readable than a json file we decided to keep it as an option.

What this means is, if there is no resume data defined in [\_config.yml](_config.yml) and loaded via a json file, it will load the contents of [\_data/cv.yml](_data/cv.yml) as fallback.

[](https://alshedivat.github.io/al-folio/cv/)

---

### People

You can create a people page if you want to feature more than one person. Each person can have its own short bio, profile picture, and you can also set if every person will appear at the same or opposite sides.

[](https://alshedivat.github.io/al-folio/people/)

---

### Publications

Your publications' page is generated automatically from your BibTex bibliography. Simply edit [\_bibliography/papers.bib](_bibliography/papers.bib). You can also add new `*.bib` files and customize the look of your publications however you like by editing [\_pages/publications.md](_pages/publications.md). By default, the publications will be sorted by year and the most recent will be displayed first. You can change this behavior and more in the `Jekyll Scholar` section in [\_config.yml](_config.yml) file.

You can add extra information to a publication, like a PDF file in the [assets/pdf/](assets/pdf/) directory and add the path to the PDF file in the BibTeX entry with the `pdf` field. Some of the supported fields are: `abstract`, `altmetric`, `arxiv`, `bibtex_show`, `blog`, `code`, `dimensions`, `doi`, `eprint`, `html`, `isbn`, `pdf`, `pmid`, `poster`, `slides`, `supp`, `video`, and `website`.

[](https://alshedivat.github.io/al-folio/publications/)

---

### Collections

This Jekyll theme implements `collections` to let you break up your work into categories. The theme comes with two default collections: `news` and `projects`. Items from the `news` collection are automatically displayed on the home page. Items from the `projects` collection are displayed on a responsive grid on projects page.

[](https://alshedivat.github.io/al-folio/projects/)

You can easily create your own collections, apps, short stories, courses, or whatever your creative work is. To do this, edit the collections in the [\_config.yml](_config.yml) file, create a corresponding folder, and create a landing page for your collection, similar to `_pages/projects.md`.

---

### Layouts

**al-folio** comes with stylish layouts for pages and blog posts.

#### The iconic style of Distill

The theme allows you to create blog posts in the [distill.pub](https://distill.pub/) style:

[](https://alshedivat.github.io/al-folio/blog/2021/distill/)

For more details on how to create distill-styled posts using `<d-*>` tags, please refer to [the example](https://alshedivat.github.io/al-folio/blog/2021/distill/).

#### Full support for math & code

**al-folio** supports fast math typesetting through [MathJax](https://www.mathjax.org/) and code syntax highlighting using [GitHub style](https://github.com/jwarby/jekyll-pygments-themes). Also supports [chartjs charts](https://www.chartjs.org/), [mermaid diagrams](https://mermaid-js.github.io/mermaid/#/), and [TikZ figures](https://tikzjax.com/).



#### Photos, Audio, Video and more

Photo formatting is made simple using [Bootstrap's grid system](https://getbootstrap.com/docs/4.4/layout/grid/). Easily create beautiful grids within your blog posts and project pages, also with support for [video](https://alshedivat.github.io/al-folio/blog/2023/videos/) and [audio](https://alshedivat.github.io/al-folio/blog/2023/audios/) embeds:



---

### Other features

#### GitHub's repositories and user stats

**al-folio** uses [github-readme-stats](https://github.com/anuraghazra/github-readme-stats) and [github-profile-trophy](https://github.com/ryo-ma/github-profile-trophy) to display GitHub repositories and user stats on the `/repositories/` page.

[](https://alshedivat.github.io/al-folio/repositories/)

Edit the `_data/repositories.yml` and change the `github_users` and `github_repos` lists to include your own GitHub profile and repositories to the `/repositories/` page.

You may also use the following codes for displaying this in any other pages.

```html

{% if site.data.repositories.github_users %}

  {% for user in site.data.repositories.github_users %} {% include repository/repo_user.liquid username=user %} {% endfor %}

{% endif %}


{% if site.repo_trophies.enabled %} {% for user in site.data.repositories.github_users %} {% if site.data.repositories.github_users.size > 1 %}
<h4>{{ user }}</h4>
{% endif %}

  {% include repository/repo_trophies.liquid username=user %}

{% endfor %} {% endif %}


{% if site.data.repositories.github_repos %}

  {% for repo in site.data.repositories.github_repos %} {% include repository/repo.liquid repository=repo %} {% endfor %}

{% endif %}
```

---

#### Theming

A variety of beautiful theme colors have been selected for you to choose from. The default is purple, but you can quickly change it by editing the `--global-theme-color` variable in the `_sass/_themes.scss` file. Other color variables are listed there as well. The stock theme color options available can be found at [\_sass/\_variables.scss](_sass/_variables.scss). You can also add your own colors to this file assigning each a name for ease of use across the template.

---

#### Social media previews

**al-folio** supports preview images on social media. To enable this functionality you will need to set `serve_og_meta` to `true` in your [\_config.yml](_config.yml). Once you have done so, all your site's pages will include Open Graph data in the HTML head element.

You will then need to configure what image to display in your site's social media previews. This can be configured on a per-page basis, by setting the `og_image` page variable. If for an individual page this variable is not set, then the theme will fall back to a site-wide `og_image` variable, configurable in your [\_config.yml](_config.yml). In both the page-specific and site-wide cases, the `og_image` variable needs to hold the URL for the image you wish to display in social media previews.

---

#### Atom (RSS-like) Feed

It generates an Atom (RSS-like) feed of your posts, useful for Atom and RSS readers. The feed is reachable simply by typing after your homepage `/feed.xml`. E.g. assuming your website mountpoint is the main folder, you can type `yourusername.github.io/feed.xml`

---

#### Related posts

By default, there will be a related posts section on the bottom of the blog posts. These are generated by selecting the `max_related` most recent posts that share at least `min_common_tags` tags with the current post. If you do not want to display related posts on a specific post, simply add `related_posts: false` to the front matter of the post. If you want to disable it for all posts, simply set `enabled` to false in the `related_blog_posts` section in [\_config.yml](_config.yml).

---

#### Code quality checks

Currently, we run some checks to ensure that the code quality and generated site are good. The checks are done using GitHub Actions and the following tools:

- [Prettier](https://prettier.io/) - check if the formatting of the code follows the style guide
- [lychee](https://lychee.cli.rs/) - check for broken links
- [Axe](https://github.com/dequelabs/axe-core) (need to run manually) - do some accessibility testing

We decided to keep `Axe` runs manual because fixing the issues are not straightforward and might be hard for people without web development knowledge.

## FAQ

For frequently asked questions, please refer to [FAQ.md](FAQ.md).

## Contributing

Contributions to al-folio are very welcome! Before you get started, please take a look at [the guidelines](CONTRIBUTING.md).

If you would like to improve documentation or fix a minor inconsistency or bug, please feel free to send a PR directly to `master`. For more complex issues/bugs or feature requests, please open an issue using the appropriate template.

### Maintainers

Our most active contributors are welcome to join the maintainers team. If you are interested, please reach out!






| [**Maruan**](http://maruan.alshedivat.com) | [**Rohan Deb Sarkar**](http://rohandebsarkar.github.io) | [**Amir Pourmand**](https://amirpourmand.ir) | [**George**](https://george-gca.github.io/) |
| --- | --- | --- | --- |








### All Contributors

<a href="https://contrib.rocks">
  
</a>

## Star History

<a href="https://star-history.com/#alshedivat/al-folio&Date">
  
    
    
    
  
</a>

## License

The theme is available as open source under the terms of the [MIT License](https://github.com/alshedivat/al-folio/blob/master/LICENSE).

Originally, **al-folio** was based on the [\*folio theme](https://github.com/bogoli/-folio) (published by [Lia Bogoev](https://liabogoev.com) and under the MIT license). Since then, it got a full re-write of the styles and many additional cool features.