### Getting Started

---
layout: default
title: Getting Started
nav_order: 2
---

# Getting Started
{: .no_toc }

A complete guide to installing and configuring RLM for your projects.
{: .fs-6 .fw-300 }

## Table of Contents
{: .no_toc .text-delta }

1. TOC
{:toc}

---

## Installation

### Prerequisites

- Python 3.11 or higher
- An API key from a supported LLM provider (OpenAI, Anthropic, etc.)

### Using uv (Recommended)

```bash
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate virtual environment
uv init && uv venv --python 3.12
source .venv/bin/activate

# Install RLM in editable mode
uv pip install -e .
```

### Optional: Modal Support

For cloud-based sandboxed execution:

```bash
# Install Modal extra
uv pip install -e ".[modal]"

# Authenticate Modal
modal setup
```

### Optional: Docker Support

For containerized execution, ensure Docker is installed and running:

```bash
# Verify Docker is available
docker --version
```

---

## Your First RLM Call

### Step 1: Set Up API Keys

Create a `.env` file in your project root:

```bash
# .env
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
PORTKEY_API_KEY=...
```

### Step 2: Basic Usage

```python
import os
from dotenv import load_dotenv
from rlm import RLM

load_dotenv()

# Create RLM instance
rlm = RLM(
    backend="openai",
    backend_kwargs={
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o",
    },
)

# Make a completion call
result = rlm.completion("Calculate the 50th Fibonacci number using Python.")
print(result.response)
```

### Step 3: Enable Verbose Output

See what the RLM is doing step by step:

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o",
    },
    verbose=True,  # Enable rich console output
)
```

This will display:
- Each iteration's LM response
- Code blocks being executed
- Stdout/stderr from execution
- Final answer when reached

---

## Understanding the RLM Class

### Constructor Arguments

| Argument | Type | Default | Description |
|:---------|:-----|:--------|:------------|
| `backend` | `str` | `"openai"` | LM provider backend |
| `backend_kwargs` | `dict` | `None` | Backend-specific configuration |
| `environment` | `str` | `"local"` | Execution environment type |
| `environment_kwargs` | `dict` | `None` | Environment configuration |
| `max_depth` | `int` | `1` | Maximum recursion depth for `rlm_query()` |
| `max_iterations` | `int` | `30` | Max REPL iterations per call |
| `max_budget` | `float` | `None` | Max total USD cost (if provider reports cost) |
| `max_timeout` | `float` | `None` | Max wall-clock seconds per completion |
| `max_tokens` | `int` | `None` | Max total tokens (input + output) per completion |
| `max_errors` | `int` | `None` | Max consecutive REPL errors before abort |
| `custom_system_prompt` | `str` | `None` | Override default system prompt |
| `other_backends` | `list` | `None` | Additional backends for sub-calls |
| `other_backend_kwargs` | `list` | `None` | Configs for additional backends |
| `logger` | `RLMLogger` | `None` | Logger for trajectory tracking and `metadata` capture |
| `verbose` | `bool` | `False` | Enable console output |
| `persistent` | `bool` | `False` | Reuse environment across `completion()` calls |
| `custom_tools` | `dict` | `None` | Custom functions/data available in REPL |
| `custom_sub_tools` | `dict` | `None` | Custom tools for child RLMs (defaults to `custom_tools`) |
| `compaction` | `bool` | `False` | Auto-summarize history when context fills up |
| `compaction_threshold_pct` | `float` | `0.85` | Context usage fraction that triggers compaction |

### The `completion()` Method

```python
result = rlm.completion(
    prompt="Your input text or context",
    root_prompt="Optional: A short prompt visible to the root LM"
)
```

**Parameters:**
- `prompt`: The main context/input (string or dict). This becomes the `context` variable in the REPL.
- `root_prompt`: Optional hint shown to the root LM (useful for Q&A tasks).

**Returns:** `RLMChatCompletion` with:
- `response`: The final answer string
- `usage_summary`: Token usage statistics
- `execution_time`: Total time in seconds
- `root_model`: Model name used
- `prompt`: Original input
- `metadata`: Full trajectory dict (if `logger` was provided, else `None`)

### Depth>1 Recursion

Depth>1 recursive subcalls are supported. The REPL provides two LM call functions:

- **`llm_query(prompt)`** — Always makes a plain, single LM completion. Fast and lightweight. Use for simple extraction, summarization, or Q&A.
- **`rlm_query(prompt)`** — Spawns a child RLM with its own REPL and iterative reasoning. Use for subtasks that need multi-step thinking or code execution. Falls back to `llm_query` when `depth >= max_depth`.

Both have batched variants (`llm_query_batched`, `rlm_query_batched`) for processing multiple prompts concurrently.

See [Architecture](architecture.md) for details on how handlers, environments, and recursive sub-calls work.

### Limits and Exceptions

RLM raises explicit exceptions when limits are exceeded:
- `BudgetExceededError`
- `TimeoutExceededError`
- `TokenLimitExceededError`
- `ErrorThresholdExceededError`
- `CancellationError` (on `KeyboardInterrupt`)

All exceptions are importable from the top-level package (`from rlm import TimeoutExceededError, ...`).

---

## Choosing an Environment

RLM supports several execution environments:

### Local (Default)

Code runs in the same Python process with sandboxed builtins.

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    environment="local",
)
```

**Pros:** Fast, no setup required  
**Cons:** Less isolation from host process

### Docker

Code runs in a Docker container with full isolation. A lightweight host-side
proxy bridges LM access back into the container, so the container never needs
API keys or direct network access to the model provider.

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    environment="docker",
    environment_kwargs={
        "image": "python:3.11-slim",  # Custom image
    },
)
```

`DockerREPL` supports the same capabilities as the local environment:

- `llm_query` / `llm_query_batched` — single LM completions
- `rlm_query` / `rlm_query_batched` — recursive sub-RLM calls (with `max_depth > 1`),
  including parallel batched sub-calls bounded by `max_concurrent_subcalls`
- `custom_tools` / `custom_sub_tools` — injected functions/data (pass as Python
  code strings or JSON-serializable values, since host callables cannot cross
  the container boundary)
- `persistent=True` — multi-turn sessions; the container and its namespace are
  kept alive across `completion()` calls, with versioned `context_N` / `history_N`
- `compaction=True` — auto-summarizes the running `history` when context fills up

**Pros:** Containerized isolation, reproducible, full feature parity with local  
**Cons:** Requires Docker, slower startup

### Modal

Code runs in Modal's cloud sandboxes for full isolation.

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    environment="modal",
    environment_kwargs={
        "app_name": "my-rlm-app",
        "timeout": 600,
    },
)
```

**Pros:** Cloud-native, scalable, fully isolated  
**Cons:** Requires Modal account, network latency

---

## Choosing a Backend

### OpenAI

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o",
        # Optional: custom base URL
        # "base_url": "https://api.openai.com/v1",
    },
)
```

### Anthropic

```python
rlm = RLM(
    backend="anthropic",
    backend_kwargs={
        "api_key": os.getenv("ANTHROPIC_API_KEY"),
        "model_name": "claude-sonnet-4-20250514",
    },
)
```

### Portkey (Router)

```python
rlm = RLM(
    backend="portkey",
    backend_kwargs={
        "api_key": os.getenv("PORTKEY_API_KEY"),
        "model_name": "@openai/gpt-5-nano",  # Portkey model format
    },
)
```

### OpenRouter

```python
rlm = RLM(
    backend="openrouter",
    backend_kwargs={
        "api_key": os.getenv("OPENROUTER_API_KEY"),
        "model_name": "openai/gpt-4o",
    },
)
```

### vLLM (Local)

```python
rlm = RLM(
    backend="vllm",
    backend_kwargs={
        "base_url": "http://localhost:8000/v1",  # Required
        "model_name": "meta-llama/Llama-3-70b",
    },
)
```

---

## Custom Tools

You can provide custom functions and data that the RLM can use in its REPL environment. This allows you to give the model access to domain-specific tools, APIs, or helper functions.

### Basic Usage

```python
def fetch_weather(city: str) -> str:
    """Fetch weather data for a city."""
    # Your API call here
    return f"Weather in {city}: Sunny, 72°F"

def calculate_shipping(weight: float, distance: float) -> float:
    """Calculate shipping cost."""
    return weight * 0.5 + distance * 0.1

rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    custom_tools={
        "fetch_weather": fetch_weather,
        "calculate_shipping": calculate_shipping,
        "API_KEY": "your-api-key",  # Non-callable values become variables
    },
)

result = rlm.completion("What's the weather in Tokyo and calculate shipping for 10kg over 500km?")
```

Inside the REPL, the model can now call:
```python
weather = fetch_weather("Tokyo")
cost = calculate_shipping(10, 500)
```

### Tool Descriptions

You can provide descriptions for your tools that will be included in the system prompt, helping the model understand what each tool does:

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    custom_tools={
        # Dict format: {"tool": callable_or_value, "description": "..."}
        "fetch_weather": {"tool": fetch_weather, "description": "Fetch current weather data for a city name"},
        "calculate_shipping": {"tool": calculate_shipping, "description": "Calculate shipping cost given weight (kg) and distance (km)"},
        "API_KEY": {"tool": "your-api-key", "description": "API key for the weather service"},
    },
)
```

The descriptions are automatically added to the system prompt:
```
6. Custom tools and data available in the REPL:
- `fetch_weather`: Fetch current weather data for a city name
- `calculate_shipping`: Calculate shipping cost given weight (kg) and distance (km)
- `API_KEY`: API key for the weather service
```

### Isolated Environments (Modal, Daytona)

For isolated environments, custom tools must be serializable. You can provide:

1. **Code strings** - Python code that defines the function:
```python
custom_tools = {
    "helper": '''
def helper(x):
    return x * 2
''',
}
```

2. **Serializable data** - JSON-compatible values (strings, numbers, dicts, lists):
```python
custom_tools = {
    "CONFIG": {"api_url": "https://api.example.com", "timeout": 30},
    "ALLOWED_CITIES": ["Tokyo", "London", "New York"],
}
```

---

## Logging and Debugging

### Enable Logging

```python
from rlm import RLM
from rlm.logger import RLMLogger

# Create logger
logger = RLMLogger(log_dir="./logs")

rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    logger=logger,
    verbose=True,
)

result = rlm.completion("...")
# Logs saved to ./logs/rlm_TIMESTAMP_UUID.jsonl
```

### Log File Format

Logs are JSON-lines files with:

```json
{"type": "metadata", "root_model": "gpt-4o", "max_iterations": 30, ...}
{"type": "iteration", "iteration": 1, "response": "...", "code_blocks": [...]}
{"type": "iteration", "iteration": 2, "response": "...", "final_answer": "..."}
```

### Visualizer

Use the included visualizer to explore trajectories:

```bash
cd visualizer/
npm install
npm run dev  # Opens at localhost:3001
```

Upload `.jsonl` log files to visualize:
- Iteration timeline
- Code execution results
- Sub-LM call traces
- Token usage

---

## Next Steps

- [API Reference](api/rlm.md) - Complete RLM class documentation
- [Architecture](architecture.md) - How the handler, REPL, and recursive sub-calls work
- [Environments](environments/) - Deep dive into each environment
- [Backends](backends.md) - Detailed backend configuration

---

### Api/Rlm

---
layout: default
title: RLM Class
parent: API Reference
nav_order: 1
---

# RLM Class Reference
{: .no_toc }

Complete API documentation for the core RLM class.
{: .fs-6 .fw-300 }

## Table of Contents
{: .no_toc .text-delta }

1. TOC
{:toc}

---

## Overview

The `RLM` class is the main entry point for Recursive Language Model completions. It wraps an LM client and execution environment to enable iterative, code-augmented reasoning.

```python
from rlm import RLM

rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-5-nano"},
)
```

---

## Constructor

```python
RLM(
    backend: str = "openai",
    backend_kwargs: dict | None = None,
    environment: str = "local",
    environment_kwargs: dict | None = None,
    depth: int = 0,
    max_depth: int = 1,
    max_iterations: int = 30,
    max_budget: float | None = None,
    max_timeout: float | None = None,
    max_tokens: int | None = None,
    max_errors: int | None = None,
    custom_system_prompt: str | None = None,
    other_backends: list[str] | None = None,
    other_backend_kwargs: list[dict] | None = None,
    logger: RLMLogger | None = None,
    verbose: bool = False,
    persistent: bool = False,
    custom_tools: dict[str, Any] | None = None,
    custom_sub_tools: dict[str, Any] | None = None,
    compaction: bool = False,
    compaction_threshold_pct: float = 0.85,
    on_subcall_start: Callable | None = None,
    on_subcall_complete: Callable | None = None,
    on_iteration_start: Callable | None = None,
    on_iteration_complete: Callable | None = None,
)
```

### Parameters

#### `backend`
{: .no_toc }

**Type:** `Literal["openai", "portkey", "openrouter", "vllm", "anthropic"]`
**Default:** `"openai"`

The LM provider backend to use for the root model.

```python
# OpenAI
rlm = RLM(backend="openai", ...)

# Anthropic
rlm = RLM(backend="anthropic", ...)

# Local vLLM server
rlm = RLM(backend="vllm", ...)
```

---

#### `backend_kwargs`
{: .no_toc }

**Type:** `dict[str, Any] | None`
**Default:** `None`

Configuration passed to the LM client. Required fields vary by backend:

| Backend | Required | Optional |
|:--------|:---------|:---------|
| `openai` | `model_name` | `api_key`, `base_url` |
| `anthropic` | `model_name` | `api_key` |
| `portkey` | `model_name`, `api_key` | `base_url` |
| `openrouter` | `model_name` | `api_key` |
| `vllm` | `model_name`, `base_url` | — |


```python
backend_kwargs = {
    "api_key": "sk-...",
    "model_name": "gpt-4o",
    "base_url": "https://api.openai.com/v1",  # Optional
}
```

---

#### `environment`
{: .no_toc }

**Type:** `Literal["local", "docker", "modal", "prime", "daytona", "e2b"]`
**Default:** `"local"`

The execution environment for running generated code.

| Environment | Description |
|:------------|:------------|
| `local` | Same-process execution with sandboxed builtins (default) |
| `docker` | Containerized execution in Docker |
| `modal` | Cloud sandbox via Modal |
| `prime` | Cloud sandbox via Prime Intellect |
| `daytona` | Cloud sandbox via Daytona |
| `e2b` | Cloud sandbox via E2B |

---

#### `environment_kwargs`
{: .no_toc }

**Type:** `dict[str, Any] | None`
**Default:** `None`

Configuration for the execution environment:

**Local:**
```python
environment_kwargs = {
    "setup_code": "import numpy as np",  # Run before each completion
}
```

**Docker:**
```python
environment_kwargs = {
    "image": "python:3.11-slim",  # Docker image
}
```

**Modal:**
```python
environment_kwargs = {
    "app_name": "my-rlm-app",  # Modal app name
    "timeout": 600,            # Sandbox timeout in seconds
    "image": modal.Image...,   # Custom Modal image (optional)
}
```

---

#### `max_depth`
{: .no_toc }

**Type:** `int`
**Default:** `1`

Maximum recursion depth for nested RLM calls. When `max_depth > 1`, the REPL provides `rlm_query()` and `rlm_query_batched()` functions that spawn child RLMs with their own REPL environments.

When `depth >= max_depth`, `rlm_query()` falls back to a plain `llm_query()` call (no REPL, no iteration).

```python
# Enable one level of recursive sub-calls
rlm = RLM(..., max_depth=2)
```

---

#### `max_iterations`
{: .no_toc }

**Type:** `int`
**Default:** `30`

Maximum number of REPL iterations before forcing a final answer.

Each iteration consists of:
1. LM generates response (potentially with code blocks)
2. Code blocks are executed
3. Results are appended to conversation history

```python
# For complex tasks, allow more iterations
rlm = RLM(..., max_iterations=50)
```

---

#### `max_budget`
{: .no_toc }

**Type:** `float | None`
**Default:** `None`

Maximum total USD cost for a completion. If exceeded, raises `BudgetExceededError`. Requires a backend that reports cost.

---

#### `max_timeout`
{: .no_toc }

**Type:** `float | None`
**Default:** `None`

Maximum wall-clock seconds for a completion. If exceeded, raises `TimeoutExceededError`. The partial answer (if any) is available on the exception.

---

#### `max_tokens`
{: .no_toc }

**Type:** `int | None`
**Default:** `None`

Maximum total tokens (input + output) for a completion. If exceeded, raises `TokenLimitExceededError`.

---

#### `max_errors`
{: .no_toc }

**Type:** `int | None`
**Default:** `None`

Maximum consecutive REPL errors before aborting. The error counter resets on a successful execution. If exceeded, raises `ErrorThresholdExceededError`.

---

#### `custom_system_prompt`
{: .no_toc }

**Type:** `str | None`
**Default:** `None`

Override the default RLM system prompt. The default prompt instructs the LM on:
- How to use the `context` variable
- How to call `llm_query()` / `llm_query_batched()` for plain LM calls
- How to call `rlm_query()` / `rlm_query_batched()` for recursive sub-calls
- How to signal completion by setting `answer["content"]` and `answer["ready"] = True`

```python
custom_prompt = """You are a data analysis expert.
Use the REPL to analyze the context variable.
When done, run a ```repl``` block that does:
    answer["content"] = "your final answer here"
    answer["ready"] = True
"""

rlm = RLM(..., custom_system_prompt=custom_prompt)
```

---

#### `other_backends` / `other_backend_kwargs`
{: .no_toc }

**Type:** `list[str] | None` / `list[dict] | None`
**Default:** `None`

Register additional LM backends. The first `other_backend` is used as the default for depth-routed sub-calls (e.g. `llm_query()` calls from code at depth > 0 are routed to the other backend). Additional backends are registered by model name and can be selected explicitly.

```python
rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-4o"},
    other_backends=["anthropic"],
    other_backend_kwargs=[
        {"model_name": "claude-sonnet-4-20250514"},
    ],
)

# Inside REPL, code can call:
# llm_query(prompt)  # Routed to other_backend (Claude) at depth > 0
# llm_query(prompt, model="gpt-4o")  # Explicit model override
```

---

#### `logger`
{: .no_toc }

**Type:** `RLMLogger | None`
**Default:** `None`

Logger for capturing trajectory metadata. When provided, the returned `RLMChatCompletion.metadata` field contains the full trajectory (iterations, code blocks, sub-calls).

```python
from rlm.logger import RLMLogger

# In-memory only (trajectory on result.metadata)
logger = RLMLogger()

# Also save to disk (JSONL for the visualizer)
logger = RLMLogger(log_dir="./logs")

rlm = RLM(..., logger=logger)
```

---

#### `verbose`
{: .no_toc }

**Type:** `bool`
**Default:** `False`

Enable rich console output showing:
- Metadata at startup
- Each iteration's response
- Code execution results
- Final answer and statistics

---

#### `persistent`
{: .no_toc }

**Type:** `bool`
**Default:** `False`

When enabled, reuses the same environment across multiple `completion()` calls. This enables multi-turn conversations where each call adds a new context and the model retains all previous variables and state.

Contexts are versioned (`context_0`, `context_1`, ...) with `context` always aliasing `context_0`. Conversation histories from previous calls are available as `history_0`, `history_1`, etc.

Supports the context manager protocol for automatic cleanup:

```python
with RLM(..., persistent=True) as rlm:
    result1 = rlm.completion("First context")
    result2 = rlm.completion("Second context")  # Can access context_0 and context_1
```

---

#### `custom_tools`
{: .no_toc }

**Type:** `dict[str, Any] | None`
**Default:** `None`

Custom functions and data available in the REPL environment. Callable values are added to globals (callable by the model), non-callable values are added to locals (accessible as variables).

Two formats are supported:

```python
custom_tools = {
    # Plain value
    "fetch_data": my_fetch_function,
    "API_KEY": "sk-...",

    # With description (shown in system prompt)
    "calculator": {
        "tool": calc_function,
        "description": "Performs arithmetic calculations",
    },
}
```

Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `answer`, `SHOW_VARS`, and their batched variants) cannot be used as tool names.

---

#### `custom_sub_tools`
{: .no_toc }

**Type:** `dict[str, Any] | None`
**Default:** `None`

Separate set of custom tools for child RLMs spawned via `rlm_query()`. If `None`, children inherit the parent's `custom_tools`. Pass an empty dict `{}` to disable custom tools for children.

---

#### `compaction`
{: .no_toc }

**Type:** `bool`
**Default:** `False`

When enabled, automatically summarizes the conversation history when token usage exceeds `compaction_threshold_pct` of the model's context window. The full history (including summaries) is available in the REPL as the `history` variable.

---

#### `compaction_threshold_pct`
{: .no_toc }

**Type:** `float`
**Default:** `0.85`

Fraction of the model's context window that triggers compaction. Only used when `compaction=True`.

---

#### Event Callbacks
{: .no_toc }

Optional callbacks for monitoring execution progress:

| Callback | Signature | Triggered when |
|:---------|:----------|:---------------|
| `on_iteration_start` | `(depth: int, iteration_num: int)` | An iteration begins |
| `on_iteration_complete` | `(depth: int, iteration_num: int, duration: float)` | An iteration completes |
| `on_subcall_start` | `(depth: int, model: str, prompt_preview: str)` | A child RLM is spawned |
| `on_subcall_complete` | `(depth: int, model: str, duration: float, error: str \| None)` | A child RLM finishes |

---

## Methods

### `completion()`

Main entry point for RLM completions.

```python
def completion(
    self,
    prompt: str | dict[str, Any],
    root_prompt: str | None = None,
) -> RLMChatCompletion
```

#### Parameters

**`prompt`**
{: .no_toc }

The context/input to process. Becomes the `context` variable in the REPL.

```python
# String input
result = rlm.completion("Analyze this text...")

# Structured input (serialized to JSON)
result = rlm.completion({
    "documents": [...],
    "query": "Find relevant sections",
})

# List input
result = rlm.completion(["doc1", "doc2", "doc3"])
```

**`root_prompt`**
{: .no_toc }

Optional short prompt shown to the root LM on every iteration. Useful for Q&A tasks where the question should be visible throughout.

```python
# The context is the document, but the LM sees the question each iteration
result = rlm.completion(
    prompt=long_document,
    root_prompt="What is the main theme of this document?"
)
```

#### Returns

`RLMChatCompletion` dataclass:

```python
@dataclass
class RLMChatCompletion:
    root_model: str              # Model name used
    prompt: str | dict           # Original input
    response: str                # Final answer
    usage_summary: UsageSummary  # Token usage
    execution_time: float        # Total seconds
    metadata: dict | None        # Full trajectory when logger is provided
```

#### Example

```python
result = rlm.completion(
    "Calculate the factorial of 100 and return the number of digits."
)

print(result.response)          # "158"
print(result.execution_time)    # 12.34
print(result.metadata)          # Trajectory dict (if logger provided), else None
print(result.usage_summary.to_dict())
# {'model_usage_summaries': {'gpt-4o': {'total_calls': 5, ...}}}
```

### `close()`

Clean up persistent environment resources. Called automatically when using the context manager protocol (`with RLM(...) as rlm:`).

---

## Response Types

### `RLMChatCompletion`

```python
from rlm.core.types import RLMChatCompletion

result: RLMChatCompletion = rlm.completion(...)

result.root_model      # "gpt-4o"
result.prompt          # Original input
result.response        # Final answer string
result.execution_time  # Total time in seconds
result.usage_summary   # UsageSummary object
result.metadata        # Full trajectory dict (if logger provided)
```

### `UsageSummary`

```python
from rlm.core.types import UsageSummary

usage: UsageSummary = result.usage_summary
usage.to_dict()
# {
#     "model_usage_summaries": {
#         "gpt-4o": {
#             "total_calls": 5,
#             "total_input_tokens": 15000,
#             "total_output_tokens": 2000
#         }
#     }
# }
```

---

## REPL Functions

The following functions are available to model-generated code inside the REPL:

| Function | Description |
|:---------|:------------|
| `llm_query(prompt, model=None)` | Single plain LM completion. Fast, no REPL or iteration. |
| `llm_query_batched(prompts, model=None)` | Multiple plain LM completions concurrently. A single failed call doesn't fail the batch — that slot returns `"Error: llm() call failed - <msg>"`, the rest return normally. |
| `rlm_query(prompt, model=None)` | Spawn a child RLM with its own REPL for deeper thinking. Falls back to `llm_query` at max depth. |
| `rlm_query_batched(prompts, model=None)` | Spawn multiple child RLMs. Falls back to `llm_query_batched` at max depth. |
| `answer` | A dict (`{"content": "", "ready": False}`). Set `answer["content"]` to your final answer and `answer["ready"] = True` to terminate the run. |
| `SHOW_VARS()` | List all user-created variables in the REPL. |
| `print(...)` | Print output visible to the model in the next iteration. |

---

## Error Handling

RLM follows a "fail fast" philosophy:

```python
# Missing required argument
rlm = RLM(backend="vllm", backend_kwargs={"model_name": "llama"})
# Raises: AssertionError: base_url is required for vLLM

# Unknown backend
rlm = RLM(backend="unknown")
# Raises: ValueError: Unknown backend: unknown
```

If the RLM exhausts `max_iterations` without the model setting `answer["ready"] = True`, it prompts the LM one more time to provide a final answer based on the conversation history.

RLM raises explicit exceptions when limits are exceeded:

| Exception | Raised when | Key attributes |
|:----------|:------------|:---------------|
| `BudgetExceededError` | `max_budget` exceeded | `spent`, `budget` |
| `TimeoutExceededError` | `max_timeout` exceeded | `elapsed`, `timeout`, `partial_answer` |
| `TokenLimitExceededError` | `max_tokens` exceeded | `tokens_used`, `token_limit`, `partial_answer` |
| `ErrorThresholdExceededError` | `max_errors` consecutive errors | `error_count`, `threshold`, `last_error`, `partial_answer` |
| `CancellationError` | `KeyboardInterrupt` during completion | `partial_answer` |

All exceptions are importable from the top-level package:

```python
from rlm import RLM, TimeoutExceededError, CancellationError

try:
    result = rlm.completion(prompt)
except TimeoutExceededError as e:
    print(f"Timed out after {e.elapsed:.1f}s, partial: {e.partial_answer}")
except CancellationError as e:
    print(f"Cancelled, partial: {e.partial_answer}")
```

---

## Thread Safety

Each `completion()` call:
1. Spawns its own `LMHandler` socket server
2. Creates a fresh environment instance (unless persistent)
3. Cleans up both when done

This makes `completion()` calls independent, but the `RLM` instance itself should not be shared across threads without external synchronization.

---

## Example: Full Configuration

```python
import os
from rlm import RLM
from rlm.logger import RLMLogger

logger = RLMLogger(log_dir="./logs")

rlm = RLM(
    # Primary model
    backend="anthropic",
    backend_kwargs={
        "api_key": os.getenv("ANTHROPIC_API_KEY"),
        "model_name": "claude-sonnet-4-20250514",
    },

    # Execution environment
    environment="local",

    # Additional model for sub-calls (routed at depth > 0)
    other_backends=["openai"],
    other_backend_kwargs=[{
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o-mini",
    }],

    # Recursion: allow one level of child RLMs via rlm_query()
    max_depth=2,
    max_iterations=40,

    # Limits
    max_timeout=120.0,
    max_budget=1.0,
    max_errors=5,

    # Custom tools available in the REPL
    custom_tools={
        "fetch_data": {"tool": my_fetch_fn, "description": "Fetch data from API"},
    },

    # Compaction for long conversations
    compaction=True,
    compaction_threshold_pct=0.85,

    # Debugging
    logger=logger,
    verbose=True,
)

result = rlm.completion(
    prompt=massive_document,
    root_prompt="Summarize the key findings",
)

print(result.response)
print(result.metadata)  # Full trajectory (iterations, sub-calls, etc.)
```

---

### Architecture

---
layout: default
title: Architecture
nav_order: 3
---

# Architecture
{: .no_toc }

How the RLM runtime, LM handler, code execution, and recursive sub-calls fit together.
{: .fs-6 .fw-300 }

## Table of Contents
{: .no_toc .text-delta }

1. TOC
{:toc}

---

## Overview

An RLM completion involves three cooperating pieces:

1. **RLM** (`rlm/core/rlm.py`) — the main loop that drives iteration.
2. **LMHandler** (`rlm/core/lm_handler.py`) — a per-completion TCP server that routes LM API calls.
3. **LocalREPL** (`rlm/environments/local_repl.py`) — the Python execution environment where model-generated code runs.

```
┌────────────────────────────────────────────────────────────────┐
│  RLM.completion(prompt)                                        │
│                                                                │
│  1. Spawn LMHandler (TCP server on localhost, auto port)       │
│  2. Create LocalREPL (in-process exec() namespace)             │
│  3. Iterate:                                                   │
│     a. Send message history → LM backend → get response        │
│     b. Extract ```repl``` code blocks from response            │
│     c. Execute code in LocalREPL                               │
│     d. Append stdout/stderr to message history                 │
│     e. Repeat until answer["ready"] is True or limits exceeded │
│  4. Tear down handler and environment                          │
└────────────────────────────────────────────────────────────────┘
```

---

## LM Handler

### What it is

The LMHandler is a **multi-threaded TCP socket server** that sits between
the execution environment and the actual LM API backends. Every call to
`RLM.completion()` spins up a fresh handler (unless the environment is
persistent and already has one).

### Why a socket server?

The handler exists so that code running inside the execution environment
can make LM calls back to the host process without directly importing or
calling the LM client. This is essential for isolated environments (Docker,
Modal) that run in separate processes or machines — they communicate with
the handler over TCP. The local environment uses the same protocol for
consistency, even though it runs in-process.

### Lifecycle

```python
# Inside RLM._spawn_completion_context():
client = get_client(backend, backend_kwargs)            # 1. Create LM client
lm_handler = LMHandler(client, other_backend_client=…)  # 2. Wrap in handler
lm_handler.start()                                       # 3. Start TCP server (daemon thread)
# … run completion loop …
lm_handler.stop()                                        # 4. Shut down server
```

- The server binds to `127.0.0.1` with port `0` (OS auto-assigns an available port).
- It runs in a **daemon thread** so it doesn't block process exit.
- Each incoming connection is handled by a new thread (`ThreadingTCPServer`).

### Wire protocol

All messages use a simple framing: **4-byte big-endian length prefix + UTF-8 JSON payload**.

```
┌──────────┬─────────────────────────┐
│ 4 bytes  │  N bytes                │
│ len (BE) │  JSON payload (UTF-8)   │
└──────────┴─────────────────────────┘
```

Implemented in `socket_send()` / `socket_recv()` in `rlm/core/comms_utils.py`.

### Client routing

The handler can hold multiple LM clients and routes requests based on the
`model` and `depth` fields in the request:

```python
def get_client(self, model=None, depth=0):
    if model and model in self.clients:
        return self.clients[model]        # Explicit model override
    if depth == 1 and self.other_backend_client:
        return self.other_backend_client  # Depth-based routing
    return self.default_client            # Fallback
```

This lets you use a different (e.g. cheaper/faster) model for sub-LM calls
by specifying `other_backends` / `other_backend_kwargs` in the RLM constructor.

---

## Code Execution Environment (LocalREPL)

### In-process `exec()` — not a subprocess

LocalREPL executes model-generated code **in the same Python interpreter
process** as the RLM, using Python's built-in `exec()`. There is no
subprocess, no fork, and no IPC for code execution.

```python
# Simplified from LocalREPL.execute_code():
combined = {**self.globals, **self.locals}
exec(code, combined, combined)
```

### What this means in practice

- **Fast**: No process spawn overhead. Code execution is as fast as native Python.
- **Persistent namespace**: Variables created in one code block are visible in the next. The `self.locals` dict accumulates state across iterations.
- **Shared memory**: Helper functions like `llm_query()` and `rlm_query()` are plain Python closures in `self.globals`. When model code calls `llm_query("...")`, it's a direct function call within the same process.
- **Limited sandbox**: Dangerous builtins (`eval`, `exec`, `compile`, `input`) are removed from the namespace. This is a soft sandbox — it prevents accidental misuse but is not a security boundary.

### Namespace layout

```
globals (shared across all executions):
├── __builtins__        → _SAFE_BUILTINS (eval/exec/input removed)
├── llm_query()         → plain LM call via handler
├── llm_query_batched() → batched plain LM calls
├── rlm_query()         → recursive RLM sub-call (or fallback to llm_query)
├── rlm_query_batched() → batched recursive sub-calls
├── SHOW_VARS()         → list user-created variables
└── <custom_tools>      → user-provided callable tools

locals (accumulates user variables):
├── context             → alias for context_0
├── context_0           → first context payload
├── context_1, …        → additional contexts (persistent mode)
├── history             → conversation history (persistent/compaction mode)
├── answer              → {"content": "", "ready": False}; set ready=True to finish
└── <user variables>    → anything created by model code
```

### Scaffold restoration

After each `exec()`, LocalREPL restores all reserved names to prevent model
code from corrupting the environment. If the model writes
`llm_query = "oops"` or `context = None`, the next execution will still
have the real functions and data. See `_restore_scaffold()`.

---

## How `llm_query()` and `rlm_query()` Work

These are the two functions available to model-generated code for making LM calls.
They have very different behaviors:

### `llm_query(prompt, model=None)` — Plain LM call

Always makes a single, direct LM completion. No REPL, no iteration — just
prompt in, text out. Fast and lightweight.

```
Model code: answer = llm_query("Summarize this text: ...")
    │
    ▼
LocalREPL._llm_query()
    │  Creates LMRequest(prompt=..., depth=self.depth)
    │  Opens TCP socket to handler
    ▼
LMHandler (TCP server)
    │  get_client(model, depth) → selects backend
    │  client.completion(prompt) → calls LM API
    ▼
Response flows back over socket
    │
    ▼
Returns response string to model code
```

### `rlm_query(prompt, model=None)` — Recursive RLM sub-call

Spawns a **child RLM** that gets its own REPL and can reason iteratively
over the prompt — just like the parent. Use this when the subtask needs
multi-step reasoning, code execution, or its own iterative problem-solving.

Falls back to `llm_query` when recursion is not available (i.e. the current
depth has reached `max_depth`).

```
Model code: answer = rlm_query("Solve this complex problem: ...")
    │
    ▼
LocalREPL._rlm_query()
    │  if self.subcall_fn is not None:    ← set when max_depth > 1
    │      calls self.subcall_fn(prompt, model)
    │  else:
    │      falls back to _llm_query()
    │
    ▼  (when subcall_fn exists)
RLM._subcall(prompt, model)
    │  next_depth = self.depth + 1
    │  if next_depth >= max_depth:
    │      → plain client.completion() (leaf call, no REPL)
    │  else:
    │      → create child RLM(depth=next_depth, ...)
    │      → child.completion(prompt)  ← full RLM with its own handler + REPL
    │
    ▼
Returns RLMChatCompletion to parent
```

### `llm_query_batched` / `rlm_query_batched`

Same semantics as above, but for multiple prompts. `llm_query_batched` sends
all prompts as a single batched request to the handler, which processes them
concurrently with `asyncio.gather`. `rlm_query_batched` calls `subcall_fn`
sequentially for each prompt (each child RLM is a blocking call).

**Failure handling is per-prompt.** A single failed call does not fail the
whole batch — that slot returns the string `"Error: llm() call failed - <msg>"`
(with the underlying error message) while the other prompts return their real
responses. Results stay aligned with the input prompts by index, so the list is
always the same length as `prompts`.

---

## Recursive Sub-Calls (Depth > 1)

### How depth works

```
max_depth=3

RLM (depth=0)
 └─ rlm_query() → child RLM (depth=1)
     └─ rlm_query() → child RLM (depth=2)
         └─ rlm_query() → plain LM call (depth=3 >= max_depth, no REPL)
```

- `depth=0` is the root RLM that the user calls.
- Each child increments depth by 1.
- When `next_depth >= max_depth`, `_subcall()` does a plain `client.completion()` instead of creating a child RLM. This is the leaf case — no REPL, no iteration.
- `llm_query()` always does a plain LM call regardless of depth. Only `rlm_query()` triggers recursion.

### Each child gets its own handler and environment

When a child RLM is created via `_subcall()`, its `completion()` method calls
`_spawn_completion_context()` which creates:

1. **A new `LMHandler`** listening on a **different auto-assigned port**.
2. **A new `LocalREPL`** with its own isolated namespace.

```
Parent RLM (depth=0)
├── LMHandler #1 on port 52301
├── LocalREPL #1 (depth=1)
│   ├── globals: {llm_query, rlm_query, ...}
│   ├── locals: {context: "parent prompt", ...}
│   └── subcall_fn = RLM._subcall  ← enables rlm_query()
│
└── When model code calls rlm_query("subtask"):
    │
    └── Child RLM (depth=1)
        ├── LMHandler #2 on port 52302  ← NEW handler, NEW port
        ├── LocalREPL #2 (depth=2)      ← NEW namespace
        │   ├── locals: {context: "subtask", ...}
        │   └── subcall_fn = child._subcall (or None if at max_depth-1)
        │
        └── Runs its own iteration loop, returns RLMChatCompletion
```

The child's handler and environment are torn down when the child's `completion()` finishes.

### Resource limits propagate

The parent passes **remaining** budget/timeout/tokens to the child, not the
original totals. This prevents a child from consuming all of the parent's resources:

```python
# In _subcall():
remaining_timeout = self.max_timeout - elapsed  # not self.max_timeout
remaining_budget = self.max_budget - spent       # not self.max_budget
child = RLM(..., max_timeout=remaining_timeout, max_budget=remaining_budget)
```

### Metadata flows back

Each child RLM can have its own `RLMLogger`. When the child completes, its
full trajectory metadata (iterations, code blocks, sub-calls) is captured in
the returned `RLMChatCompletion.metadata` dict. The parent's logger records
this as part of the REPL result's `rlm_calls` list, creating a nested
metadata tree.

---

## Putting It All Together

Here's the complete request flow for a depth-2 RLM call:

```
User: rlm.completion("Analyze this data")
 │
 ▼
RLM (depth=0)
 ├─ _spawn_completion_context()
 │   ├─ LMHandler #1 starts on port 52301
 │   └─ LocalREPL #1 created with context="Analyze this data"
 │
 ├─ Iteration 1: LM generates code
 │   │  ```repl
 │   │  answer = rlm_query("What patterns exist in: " + context[:5000])
 │   │  ```
 │   │
 │   └─ LocalREPL.execute_code() runs the code via exec()
 │       │
 │       ├─ rlm_query() → _rlm_query() → subcall_fn()
 │       │   │
 │       │   └─ RLM._subcall("What patterns exist in: ...")
 │       │       │
 │       │       ├─ depth=1 < max_depth=2, so create child RLM
 │       │       │
 │       │       └─ Child RLM (depth=1)
 │       │           ├─ LMHandler #2 on port 52302
 │       │           ├─ LocalREPL #2 with context="What patterns..."
 │       │           │
 │       │           ├─ Child iteration 1: LM generates code
 │       │           │   │  result = llm_query("Extract key metrics: " + context)
 │       │           │   │
 │       │           │   └─ llm_query() → TCP to Handler #2 → LM API → response
 │       │           │
 │       │           ├─ Child iteration 2: LM sets answer["content"]=result, answer["ready"]=True
 │       │           │
 │       │           └─ Returns RLMChatCompletion to parent
 │       │
 │       └─ child_response = child_completion.response
 │
 ├─ Iteration 2: LM uses child_response, sets answer["content"]=final, answer["ready"]=True
 │
 ├─ LMHandler #1 stops
 └─ Returns RLMChatCompletion to user
```

### Key takeaways

| Aspect | Detail |
|:-------|:-------|
| Code execution | In-process `exec()` in the same Python interpreter. Not a subprocess. |
| LM calls from code | Go through a local TCP socket server (LMHandler), even for in-process execution. |
| Handler per completion | Each `completion()` call gets its own handler on an auto-assigned port. |
| Child RLMs | Created by `_subcall()`, each with its own handler + LocalREPL. Fully independent. |
| `llm_query` vs `rlm_query` | `llm_query` = always plain LM call. `rlm_query` = recursive child RLM (or fallback). |
| Depth limit | At `max_depth`, `rlm_query` falls back to `llm_query`. No further recursion. |
| Resource isolation | Children get remaining budget/timeout, not the full amount. |
| Namespace isolation | Each LocalREPL has its own `globals`/`locals`. No shared state between parent and child. |

---

### CONTRIBUTING

I'm too lazy to write up a stricter set of rules for PRs, but generally I just ask that you avoid touching `core/` files unless necessary. I'd like to keep the repo as minimal as possible for as long as possible so it's still easy for users to read the entire repo in a short sitting.

Generally though, I'll outline the things we 1) need to implement; 2) want to implement; 3) can dream about implementing. The state of this repo is that it should be fully functional for most use cases, but it isn't super fast or anything.

There are likely more things we'll want to do, but here are some things I've been meaning to tackle. 

## Urgent TODOs
- [ ] **Additional Sandboxes**. Any more interesting, commonly used sandboxes (e.g. Prime Sandboxes are WIP atm).
- [ ] **Persistent REPL across the client.** Currently, the REPL is only persistent across an RLM completion call, but for multi-turn settings we may want a `flag` to handle persistence. There's some trickiness here though, which is that after every turn, the input context will change / be added onto. I haven't decided yet (open to suggestions), but we could add `context_{x}` and tell the model that it has a new context or something in the next completion step.
- [ ] **Finding interesting benchmarks / examples we can provide to get started**.
- [ ] **Improve documentation**. See `docs/`.

Low-hanging fruit of the urgent TODOs:
- [ ] **Add better unit tests.** I have a Mock LM class inspired by `verifiers`, but we need more comprehensive unit tests. Generally these should be made with most PRs.
- [ ] **Do more comprehensive bug finding**: Just find bugs and report them, we'll try to squash them all

## Would-be-nice TODOs
- [ ] **Multi-modal / arbitrary input support.** As it stands, we just support `str` / standard LM dict messages, but we should generally support any type of picklable-inputs. We might want to think of clever ways to do this lazily as well.
- [ ] **File-system based environments**. Beyond REPLs, we can also think about supporting filesystem + bash as a new type of environment. There seems to be a lot of interest in this.
- [ ] **Improved UI for visualization**.
- [ ] **Improvements to what data gets stored, useful for training and statistics about RLMs**/

## "If you can tackle these, thanks LOL" TODOs
- [ ] **Pipelining / asynchrony of LM calls**. This could be a paper of its own IMO, but how we deal with LM calls and how we actually implement these recursive calls can have big implications. I suspect this might happen when the repo has a massive overhaul, but something to think about.
- [ ] **Efficient prefix caching**. Another "would be nice" thing, but requires restructuring a lot of the core logic. Could also be a paper / entire research project of its own.
- [ ] **Training models to work as RLMs**. See the `verifiers` [rlm_env](https://github.com/PrimeIntellect-ai/verifiers/blob/main/verifiers/envs/experimental/rlm_env.py) as a starting point.

---

### README

---

<h1 align="center" style="font-size:2.8em">
<span>Recursive Language Models (<span style="color:orange">RLM</span>s)</span>
</h1>

<p align="center" style="font-size:1.3em">
  <a href="https://arxiv.org/abs/2512.24601">Full Paper</a> •
  <a href="https://alexzhang13.github.io/blog/2025/rlm/">Blogpost</a> •
  <a href="https://alexzhang13.github.io/rlm/">Documentation</a> •
  <a href="https://github.com/alexzhang13/rlm-minimal">RLM Minimal</a>
</p>

<p align="center">
  <a href="https://github.com/alexzhang13/rlm/actions/workflows/style.yml">
    <img src="https://github.com/alexzhang13/rlm/actions/workflows/style.yml/badge.svg" alt="Style" />
  </a>
  <a href="https://github.com/alexzhang13/rlm/actions/workflows/test.yml">
    <img src="https://github.com/alexzhang13/rlm/actions/workflows/test.yml/badge.svg" alt="Test" />
  </a>
</p>

<p align="center">
  <a href="https://arxiv.org/abs/2512.24601">
    <img src="media/paper_preview.png" alt="Paper Preview" width="300"/>
  </a>
</p>

## Overview
Recursive Language Models (RLMs) are a task-agnostic inference paradigm for language models (LMs) to handle near-infinite length contexts by enabling the LM to *programmatically* examine, decompose, and recursively call itself over its input. RLMs replace the canonical `llm.completion(prompt, model)` call with a `rlm.completion(prompt, model)` call, acting as a "language model". RLMs offload the context as a variable in a REPL environment that the LM can interact with and launch sub-LM calls inside of.

RLMs are a bet on future "language model" design choices. We argue for a [CodeAct](https://arxiv.org/abs/2402.01030)-style harness (i.e. all language models should have access to a code environment) with sub-(R)LM calls as functions in code, and context / prompts as objects in code. RLMs explicitly defer code execution with sub-calls as functions to the language model itself, which is incredibly flexible and lends itself well to scale if trained correctly. We want to move away from the JSON tool-calling standard for both sub-agents and generic tool calls. The naming comes from the fact that such a system is itself a "language model" (a probabilistic mapping from text to text) that builds around and relies on recursive sub-LLM calls.

This repository provides both an extensible inference engine and training environment for using RLMs around standard API-based and local LLMs. The initial experiments and idea were proposed in a [blogpost](https://alexzhang13.github.io/blog/2025/rlm/) in 2025, with expanded results in an [arXiv preprint](https://arxiv.org/abs/2512.24601).

We now also include a [verifiers](https://github.com/PrimeIntellect-ai/verifiers) training environment based on Prime Intellect's [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl) in the `training/` folder. Train your own RLMs, which directly can be plugged into our inference engine!

> [!NOTE]
> This repository contains inference code for RLMs with support for various sandbox environments. Open-source contributions are welcome. This repository is maintained by the authors of the paper from the MIT OASYS lab.

## Quick Setup
> [!NOTE]
> `rlms` requires **Python 3.11 or later**.

You can try out RLMs quickly by installing from PyPi:
```bash
pip install rlms
```

The default RLM client uses a REPL environment that runs on the host process through Python `exec` calls. It uses the same virtual environment as the host process (i.e. it will have access to the same dependencies), but with some limitations in its available global modules. As an example, we can call RLM completions using GPT-5-nano:
```python
from rlm import RLM

rlm = RLM(
    backend="openai",
    backend_kwargs={"model_name": "gpt-5-nano"},
    verbose=True,  # For printing to console with rich, disabled by default.
)

print(rlm.completion("Print me the first 100 powers of two, each on a newline.").response)
```

<details>
<summary><b>Manual Setup</b></summary>

Set up the dependencies with `uv` (or your virtual environment of choice):
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
uv init && uv venv --python 3.12  # change version as needed
uv pip install -e .
```

This project includes a `Makefile` to simplify common tasks.

- `make install`: Install base dependencies.
- `make check`: Run linter, formatter, and tests.

To run a quick test, the following will run an RLM query with the OpenAI client using your environment variable `OPENAI_API_KEY` (feel free to change this). This will generate console output as well as a log which you can use with the visualizer to explore the trajectories.
```bash
make quickstart
```

</details>

## REPL Environments
We support two types of REPL environments -- isolated, and non-isolated. Non-isolated environments (default) run code execution on the same machine as the RLM (e.g. through `exec`), which is pretty reasonable for some local low-risk tasks, like simple benchmarking, but can be problematic if the prompts or tool calls can interact with malicious users. Fully isolated environments use cloud-based sandboxes (e.g. Prime Sandboxes, [Modal Sandboxes](https://modal.com/docs/guide/sandboxes)) to run code generated by the RLM, ensuring complete isolation from the host process. Environments can be added, but we natively support the following: `local` (default), `ipython`, `docker`, `modal`, `prime`, `daytona`, `e2b`.

```python
rlm = RLM(
    environment="...", # "local", "ipython", "docker", "modal", "prime", "daytona", "e2b"
    environment_kwargs={...},
)
```

### Local Environments
The default `local` environment `LocalREPL` runs in the same process as the RLM itself, with specified global and local namespaces for minimal security. Using this REPL is generally safe, but should not be used for production settings. It also shares the same virtual environment (e.g. Conda or uv) as the host process.

#### IPython (*requires `pip install 'rlms[ipython]'`*)
`IPythonREPL` runs cells inside a real IPython session — either in-process (default) or in a separate `ipykernel` subprocess. Subprocess mode adds hard `cell_timeout` enforcement and full namespace isolation from the RLM host. See the [IPythonREPL docs](https://alexzhang13.github.io/rlm/environments/ipython) for details.

#### Docker <img src="https://github.com/docker.png" alt="Docker" height="20" style="vertical-align: middle;"/> (*requires [Docker installed](https://docs.docker.com/desktop/setup/install/)*)
We also support a Docker-based environment called `DockerREPL` that launches the REPL environment as a Docker image. By default, we use the `python:3.11-slim` image, but the user can specify custom images as well. The container runs fully isolated from the host; a lightweight host-side proxy bridges LM access back into the container.

`DockerREPL` supports the full feature set of the local environment: single LM calls (`llm_query` / `llm_query_batched`), recursive sub-RLM calls (`rlm_query` / `rlm_query_batched`, including parallel batched sub-calls bounded by `max_concurrent_subcalls`), `custom_tools` / `custom_sub_tools`, `persistent=True` multi-turn sessions (versioned `context_N` / `history_N` reused across `completion()` calls), and `compaction=True` auto-summarization of the running `history`. For isolated environments, custom tools should be passed as Python code strings or JSON-serializable values (host callables cannot cross the process boundary).

### Isolated Environments
We support several different REPL environments that run on separate, cloud-based machines. Whenever a recursive sub-call is made in these instances, it is requested from the host process.

#### Modal Sandboxes <img src="https://github.com/modal-labs.png" alt="Modal" height="20" style="vertical-align: middle;"/>
To use [Modal Sandboxes](https://modal.com/docs/guide/sandboxes) as the REPL environment, you need to install and authenticate your Modal account.
```bash
uv add modal  # add modal library
modal setup   # authenticate account
```

#### Prime Intellect Sandboxes <img src="https://github.com/PrimeIntellect-ai.png" alt="Prime Intellect" height="20" style="vertical-align: middle;"/>
> [!NOTE]
> **Prime Intellect Sandboxes** are currently a beta feature. See the [documentation](https://docs.primeintellect.ai/sandboxes/overview) for more information. We noticed slow runtimes when using these sandboxes, which is currently an open issue.


To use [Prime Sandboxes](https://docs.primeintellect.ai/sandboxes/sdk), install the SDK and set your API key:
```bash
uv pip install -e ".[prime]"
export PRIME_API_KEY=...
```


### Model Providers
We currently support most major clients (OpenAI, Anthropic), as well as the router platforms (OpenRouter, Portkey). For local models, we recommend using vLLM (which interfaces with the [OpenAI client](https://github.com/alexzhang13/rlm/blob/main/rlm/clients/openai.py)). To view or add support for more clients, start by looking at [`rlm/clients/`](https://github.com/alexzhang13/rlm/tree/main/rlm/clients).

## Training
We provide a simple RL training harness for training RLMs used in this repo (specifically the `local` REPL). The implementation uses no sandboxes for simplicity and slots easily your use case, but an ideal setup would use sandboxes for safety. Training logic is isolated to the [`training/`](https://github.com/alexzhang13/rlm/tree/main/training) folder, which exposes `rlm.RLM` as a [`verifiers`](https://github.com/willccbb/verifiers) `Environment` and plugs straight into [`prime-rl`](https://github.com/PrimeIntellect-ai/prime-rl). See the [training README](https://github.com/alexzhang13/rlm/tree/main/training#readme) for the launch command. The harness uses subprocess-isolated local REPL execution (no cloud sandboxes), matching the `local` environment above.

A worked example with an example `.toml` lives in [`training/environments/oolong/`](https://github.com/alexzhang13/rlm/tree/main/training/environments/oolong) (OOLONG long-context QA). New training environments can be added the same way — author a `verifiers` env that wraps your task (see the [verifiers docs](https://verifiers.readthedocs.io/)), then reference it from a config.

## Relevant Reading
* **[Dec '25]** [Recursive Language Models arXiv](https://arxiv.org/abs/2512.24601)
* **[Oct '25]** [Recursive Language Models Blogpost](https://alexzhang13.github.io/blog/2025/rlm/)

If you use this code or repository in your research, please cite:

```bibtex
@misc{zhang2026recursivelanguagemodels,
      title={Recursive Language Models},
      author={Alex L. Zhang and Tim Kraska and Omar Khattab},
      year={2026},
      eprint={2512.24601},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2512.24601},
}
```

## RLMs being used in the wild
There are many amazing demos and production-ready use cases of RLMs. We provide a list of notable examples that explicitly use RLMs as a central piece of their design.
* <img src="https://www.google.com/s2/favicons?domain=dspy.ai&sz=64" alt="DSPy" height="15" style="vertical-align: middle;"/> [DSPy.RLM](https://github.com/stanfordnlp/dspy) <a href="https://github.com/stanfordnlp/dspy/stargazers"><img src="https://badgen.net/github/stars/stanfordnlp/dspy?icon=github&label=Stars" height="15" alt="GitHub stars"></a>
* <img src="https://github.com/PrimeIntellect-ai.png" alt="Prime Intellect" height="15" style="vertical-align: middle;"/> [Prime Agent](https://github.com/PrimeIntellect-ai/prime-agent) <a href="https://github.com/PrimeIntellect-ai/prime-agent/stargazers"><img src="https://badgen.net/github/stars/PrimeIntellect-ai/prime-agent?icon=github&label=Stars" height="15" alt="GitHub stars"></a>
* <img src="https://github.com/ax-llm.png" alt="Ax" height="15" style="vertical-align: middle;"/> [Ax](https://github.com/ax-llm/ax) <a href="https://github.com/ax-llm/ax/stargazers"><img src="https://badgen.net/github/stars/ax-llm/ax?icon=github&label=Stars" height="15" alt="GitHub stars"></a>
* <img src="https://github.com/context-labs.png" alt="context-labs" height="15" style="vertical-align: middle;"/> [context-labs/HALO: **RLM-based Automatic Agent Optimization Loop**](https://github.com/context-labs/halo) <a href="https://github.com/context-labs/halo/stargazers"><img src="https://badgen.net/github/stars/context-labs/halo?icon=github&label=Stars" height="15" alt="GitHub stars"></a>
* [viplismism/rlm-cli: **CLI for Recursive Language Models**](https://github.com/viplismism/rlm-cli) <a href="https://github.com/viplismism/rlm-cli/stargazers"><img src="https://badgen.net/github/stars/viplismism/rlm-cli?icon=github&label=Stars" height="15" alt="GitHub stars"></a>
* <img src="https://www.google.com/s2/favicons?domain=alphaxiv.org&sz=64" alt="alphaXiv" height="15" style="vertical-align: middle;"/> [alphaXiv Official Blog. **Reinforcing Recursive Language Models**](https://www.alphaxiv.org/blog/reinforcement-learning-for-rlms#training)
* <img src="https://www.google.com/s2/favicons?domain=daytona.io&sz=64" alt="Daytona" height="15" style="vertical-align: middle;"/> [Daytona. **Building Deep Recursive Language Models**](https://www.daytona.io/docs/en/guides/rlm/recursive-language-models/)
* <img src="https://www.google.com/s2/favicons?domain=symbolica.ai&sz=64" alt="Symbolica" height="15" style="vertical-align: middle;"/> [Symbolica. **SotA ARC-AGI-2 Results with REPL Agents**](https://www.symbolica.ai/blog/arcgentica)
* <img src="https://www.google.com/s2/favicons?domain=cloud.google.com&sz=64" alt="Google Cloud" height="15" style="vertical-align: middle;"/> [Google Cloud Community Articles. **RLMs in ADK**](https://discuss.google.dev/t/recursive-language-models-in-adk/)
* <img src="https://github.com/PrimeIntellect-ai.png" alt="Prime Intellect" height="15" style="vertical-align: middle;"/> [Prime Intellect Blog. **Recursive Language Models: *the* paradigm of 2026**](https://www.primeintellect.ai/blog/rlm)

## Optional: Trajectory metadata, logging, and debugging
`RLMChatCompletion` has an optional `metadata` field (default `None`) that holds the full trajectory (run config + all iterations and sub-calls) so you can reconstruct the run. Pass an `RLMLogger` to capture it:

- **In-memory only** (trajectory on `completion.metadata`): `logger=RLMLogger()` (no `log_dir`).
- **Also save to disk** (JSONL for the visualizer): `logger=RLMLogger(log_dir="./logs")`.

**Visualizing logs.** We also provide a simple visualizer to inspect code, sub-LM, and root-LM calls. Use `RLMLogger(log_dir="./logs")` so each completion writes a `.jsonl` file:
```python
from rlm.logger import RLMLogger
from rlm import RLM

logger = RLMLogger(log_dir="./logs")
rlm = RLM(..., logger=logger)
```

To run the visualizer locally, we use Node.js and shadcn/ui:
```
cd visualizer/
npm run dev        # default localhost:3001
```

---