{"owner":"alexzhang13","repo":"rlm","hasSkills":true,"totalSkillsCount":2,"totalTokensCount":6743,"categories":["project-spec","root-instruction"],"hasMcp":false,"mcpConfig":null,"found":["docs/architecture.md","AGENTS.md"],"skills":{"docs/architecture.md":"---\nlayout: default\ntitle: Architecture\nnav_order: 3\n---\n\n# Architecture\n{: .no_toc }\n\nHow the RLM runtime, LM handler, code execution, and recursive sub-calls fit together.\n{: .fs-6 .fw-300 }\n\n## Table of Contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n\n## Overview\n\nAn RLM completion involves three cooperating pieces:\n\n1. **RLM** (`rlm/core/rlm.py`) — the main loop that drives iteration.\n2. **LMHandler** (`rlm/core/lm_handler.py`) — a per-completion TCP server that routes LM API calls.\n3. **LocalREPL** (`rlm/environments/local_repl.py`) — the Python execution environment where model-generated code runs.\n\n```\n┌────────────────────────────────────────────────────────────────┐\n│  RLM.completion(prompt)                                        │\n│                                                                │\n│  1. Spawn LMHandler (TCP server on localhost, auto port)       │\n│  2. Create LocalREPL (in-process exec() namespace)             │\n│  3. Iterate:                                                   │\n│     a. Send message history → LM backend → get response        │\n│     b. Extract ```repl``` code blocks from response            │\n│     c. Execute code in LocalREPL                               │\n│     d. Append stdout/stderr to message history                 │\n│     e. Repeat until answer[\"ready\"] is True or limits exceeded │\n│  4. Tear down handler and environment                          │\n└────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## LM Handler\n\n### What it is\n\nThe LMHandler is a **multi-threaded TCP socket server** that sits between\nthe execution environment and the actual LM API backends. Every call to\n`RLM.completion()` spins up a fresh handler (unless the environment is\npersistent and already has one).\n\n### Why a socket server?\n\nThe handler exists so that code running inside the execution environment\ncan make LM calls back to the host process without directly importing or\ncalling the LM client. This is essential for isolated environments (Docker,\nModal) that run in separate processes or machines — they communicate with\nthe handler over TCP. The local environment uses the same protocol for\nconsistency, even though it runs in-process.\n\n### Lifecycle\n\n```python\n# Inside RLM._spawn_completion_context():\nclient = get_client(backend, backend_kwargs)            # 1. Create LM client\nlm_handler = LMHandler(client, other_backend_client=…)  # 2. Wrap in handler\nlm_handler.start()                                       # 3. Start TCP server (daemon thread)\n# … run completion loop …\nlm_handler.stop()                                        # 4. Shut down server\n```\n\n- The server binds to `127.0.0.1` with port `0` (OS auto-assigns an available port).\n- It runs in a **daemon thread** so it doesn't block process exit.\n- Each incoming connection is handled by a new thread (`ThreadingTCPServer`).\n\n### Wire protocol\n\nAll messages use a simple framing: **4-byte big-endian length prefix + UTF-8 JSON payload**.\n\n```\n┌──────────┬─────────────────────────┐\n│ 4 bytes  │  N bytes                │\n│ len (BE) │  JSON payload (UTF-8)   │\n└──────────┴─────────────────────────┘\n```\n\nImplemented in `socket_send()` / `socket_recv()` in `rlm/core/comms_utils.py`.\n\n### Client routing\n\nThe handler can hold multiple LM clients and routes requests based on the\n`model` and `depth` fields in the request:\n\n```python\ndef get_client(self, model=None, depth=0):\n    if model and model in self.clients:\n        return self.clients[model]        # Explicit model override\n    if depth == 1 and self.other_backend_client:\n        return self.other_backend_client  # Depth-based routing\n    return self.default_client            # Fallback\n```\n\nThis lets you use a different (e.g. cheaper/faster) model for sub-LM calls\nby specifying `other_backends` / `other_backend_kwargs` in the RLM constructor.\n\n---\n\n## Code Execution Environment (LocalREPL)\n\n### In-process `exec()` — not a subprocess\n\nLocalREPL executes model-generated code **in the same Python interpreter\nprocess** as the RLM, using Python's built-in `exec()`. There is no\nsubprocess, no fork, and no IPC for code execution.\n\n```python\n# Simplified from LocalREPL.execute_code():\ncombined = {**self.globals, **self.locals}\nexec(code, combined, combined)\n```\n\n### What this means in practice\n\n- **Fast**: No process spawn overhead. Code execution is as fast as native Python.\n- **Persistent namespace**: Variables created in one code block are visible in the next. The `self.locals` dict accumulates state across iterations.\n- **Shared memory**: Helper functions like `llm_query()` and `rlm_query()` are plain Python closures in `self.globals`. When model code calls `llm_query(\"...\")`, it's a direct function call within the same process.\n- **Limited sandbox**: Dangerous builtins (`eval`, `exec`, `compile`, `input`) are removed from the namespace. This is a soft sandbox — it prevents accidental misuse but is not a security boundary.\n\n### Namespace layout\n\n```\nglobals (shared across all executions):\n├── __builtins__        → _SAFE_BUILTINS (eval/exec/input removed)\n├── llm_query()         → plain LM call via handler\n├── llm_query_batched() → batched plain LM calls\n├── rlm_query()         → recursive RLM sub-call (or fallback to llm_query)\n├── rlm_query_batched() → batched recursive sub-calls\n├── SHOW_VARS()         → list user-created variables\n└── <custom_tools>      → user-provided callable tools\n\nlocals (accumulates user variables):\n├── context             → alias for context_0\n├── context_0           → first context payload\n├── context_1, …        → additional contexts (persistent mode)\n├── history             → conversation history (persistent/compaction mode)\n├── answer              → {\"content\": \"\", \"ready\": False}; set ready=True to finish\n└── <user variables>    → anything created by model code\n```\n\n### Scaffold restoration\n\nAfter each `exec()`, LocalREPL restores all reserved names to prevent model\ncode from corrupting the environment. If the model writes\n`llm_query = \"oops\"` or `context = None`, the next execution will still\nhave the real functions and data. See `_restore_scaffold()`.\n\n---\n\n## How `llm_query()` and `rlm_query()` Work\n\nThese are the two functions available to model-generated code for making LM calls.\nThey have very different behaviors:\n\n### `llm_query(prompt, model=None)` — Plain LM call\n\nAlways makes a single, direct LM completion. No REPL, no iteration — just\nprompt in, text out. Fast and lightweight.\n\n```\nModel code: answer = llm_query(\"Summarize this text: ...\")\n    │\n    ▼\nLocalREPL._llm_query()\n    │  Creates LMRequest(prompt=..., depth=self.depth)\n    │  Opens TCP socket to handler\n    ▼\nLMHandler (TCP server)\n    │  get_client(model, depth) → selects backend\n    │  client.completion(prompt) → calls LM API\n    ▼\nResponse flows back over socket\n    │\n    ▼\nReturns response string to model code\n```\n\n### `rlm_query(prompt, model=None)` — Recursive RLM sub-call\n\nSpawns a **child RLM** that gets its own REPL and can reason iteratively\nover the prompt — just like the parent. Use this when the subtask needs\nmulti-step reasoning, code execution, or its own iterative problem-solving.\n\nFalls back to `llm_query` when recursion is not available (i.e. the current\ndepth has reached `max_depth`).\n\n```\nModel code: answer = rlm_query(\"Solve this complex problem: ...\")\n    │\n    ▼\nLocalREPL._rlm_query()\n    │  if self.subcall_fn is not None:    ← set when max_depth > 1\n    │      calls self.subcall_fn(prompt, model)\n    │  else:\n    │      falls back to _llm_query()\n    │\n    ▼  (when subcall_fn exists)\nRLM._subcall(prompt, model)\n    │  next_depth = self.depth + 1\n    │  if next_depth >= max_depth:\n    │      → plain client.completion() (leaf call, no REPL)\n    │  else:\n    │      → create child RLM(depth=next_depth, ...)\n    │      → child.completion(prompt)  ← full RLM with its own handler + REPL\n    │\n    ▼\nReturns RLMChatCompletion to parent\n```\n\n### `llm_query_batched` / `rlm_query_batched`\n\nSame semantics as above, but for multiple prompts. `llm_query_batched` sends\nall prompts as a single batched request to the handler, which processes them\nconcurrently with `asyncio.gather`. `rlm_query_batched` calls `subcall_fn`\nsequentially for each prompt (each child RLM is a blocking call).\n\n**Failure handling is per-prompt.** A single failed call does not fail the\nwhole batch — that slot returns the string `\"Error: llm() call failed - <msg>\"`\n(with the underlying error message) while the other prompts return their real\nresponses. Results stay aligned with the input prompts by index, so the list is\nalways the same length as `prompts`.\n\n---\n\n## Recursive Sub-Calls (Depth > 1)\n\n### How depth works\n\n```\nmax_depth=3\n\nRLM (depth=0)\n └─ rlm_query() → child RLM (depth=1)\n     └─ rlm_query() → child RLM (depth=2)\n         └─ rlm_query() → plain LM call (depth=3 >= max_depth, no REPL)\n```\n\n- `depth=0` is the root RLM that the user calls.\n- Each child increments depth by 1.\n- When `next_depth >= max_depth`, `_subcall()` does a plain `client.completion()` instead of creating a child RLM. This is the leaf case — no REPL, no iteration.\n- `llm_query()` always does a plain LM call regardless of depth. Only `rlm_query()` triggers recursion.\n\n### Each child gets its own handler and environment\n\nWhen a child RLM is created via `_subcall()`, its `completion()` method calls\n`_spawn_completion_context()` which creates:\n\n1. **A new `LMHandler`** listening on a **different auto-assigned port**.\n2. **A new `LocalREPL`** with its own isolated namespace.\n\n```\nParent RLM (depth=0)\n├── LMHandler #1 on port 52301\n├── LocalREPL #1 (depth=1)\n│   ├── globals: {llm_query, rlm_query, ...}\n│   ├── locals: {context: \"parent prompt\", ...}\n│   └── subcall_fn = RLM._subcall  ← enables rlm_query()\n│\n└── When model code calls rlm_query(\"subtask\"):\n    │\n    └── Child RLM (depth=1)\n        ├── LMHandler #2 on port 52302  ← NEW handler, NEW port\n        ├── LocalREPL #2 (depth=2)      ← NEW namespace\n        │   ├── locals: {context: \"subtask\", ...}\n        │   └── subcall_fn = child._subcall (or None if at max_depth-1)\n        │\n        └── Runs its own iteration loop, returns RLMChatCompletion\n```\n\nThe child's handler and environment are torn down when the child's `completion()` finishes.\n\n### Resource limits propagate\n\nThe parent passes **remaining** budget/timeout/tokens to the child, not the\noriginal totals. This prevents a child from consuming all of the parent's resources:\n\n```python\n# In _subcall():\nremaining_timeout = self.max_timeout - elapsed  # not self.max_timeout\nremaining_budget = self.max_budget - spent       # not self.max_budget\nchild = RLM(..., max_timeout=remaining_timeout, max_budget=remaining_budget)\n```\n\n### Metadata flows back\n\nEach child RLM can have its own `RLMLogger`. When the child completes, its\nfull trajectory metadata (iterations, code blocks, sub-calls) is captured in\nthe returned `RLMChatCompletion.metadata` dict. The parent's logger records\nthis as part of the REPL result's `rlm_calls` list, creating a nested\nmetadata tree.\n\n---\n\n## Putting It All Together\n\nHere's the complete request flow for a depth-2 RLM call:\n\n```\nUser: rlm.completion(\"Analyze this data\")\n │\n ▼\nRLM (depth=0)\n ├─ _spawn_completion_context()\n │   ├─ LMHandler #1 starts on port 52301\n │   └─ LocalREPL #1 created with context=\"Analyze this data\"\n │\n ├─ Iteration 1: LM generates code\n │   │  ```repl\n │   │  answer = rlm_query(\"What patterns exist in: \" + context[:5000])\n │   │  ```\n │   │\n │   └─ LocalREPL.execute_code() runs the code via exec()\n │       │\n │       ├─ rlm_query() → _rlm_query() → subcall_fn()\n │       │   │\n │       │   └─ RLM._subcall(\"What patterns exist in: ...\")\n │       │       │\n │       │       ├─ depth=1 < max_depth=2, so create child RLM\n │       │       │\n │       │       └─ Child RLM (depth=1)\n │       │           ├─ LMHandler #2 on port 52302\n │       │           ├─ LocalREPL #2 with context=\"What patterns...\"\n │       │           │\n │       │           ├─ Child iteration 1: LM generates code\n │       │           │   │  result = llm_query(\"Extract key metrics: \" + context)\n │       │           │   │\n │       │           │   └─ llm_query() → TCP to Handler #2 → LM API → response\n │       │           │\n │       │           ├─ Child iteration 2: LM sets answer[\"content\"]=result, answer[\"ready\"]=True\n │       │           │\n │       │           └─ Returns RLMChatCompletion to parent\n │       │\n │       └─ child_response = child_completion.response\n │\n ├─ Iteration 2: LM uses child_response, sets answer[\"content\"]=final, answer[\"ready\"]=True\n │\n ├─ LMHandler #1 stops\n └─ Returns RLMChatCompletion to user\n```\n\n### Key takeaways\n\n| Aspect | Detail |\n|:-------|:-------|\n| Code execution | In-process `exec()` in the same Python interpreter. Not a subprocess. |\n| LM calls from code | Go through a local TCP socket server (LMHandler), even for in-process execution. |\n| Handler per completion | Each `completion()` call gets its own handler on an auto-assigned port. |\n| Child RLMs | Created by `_subcall()`, each with its own handler + LocalREPL. Fully independent. |\n| `llm_query` vs `rlm_query` | `llm_query` = always plain LM call. `rlm_query` = recursive child RLM (or fallback). |\n| Depth limit | At `max_depth`, `rlm_query` falls back to `llm_query`. No further recursion. |\n| Resource isolation | Children get remaining budget/timeout, not the full amount. |\n| Namespace isolation | Each LocalREPL has its own `globals`/`locals`. No shared state between parent and child. |\n","AGENTS.md":"# AGENTS.md\n\nThis guide covers best practices for contributing to the core Recursive Language Models `rlm` library and developing new environments (in `rlm/environments/`) and LM clients (in `rlm/clients/`).\n\n## Setup\n\nWe use `uv` for developing `rlm`.\n```bash\n# Install uv (first time)\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Setup blank project if needed\nuv init && uv venv --python 3.12\nsource .venv/bin/activate\n\n# Install in editable mode\nuv pip install -e .\n\n# For Modal sandbox support\nuv pip install -e \".[modal]\"\n\n# For Prime sandbox support\nuv pip install -e \".[prime]\"\n```\n\n## General Guidelines\n\n### Code Style & Typing\n- **Formatting**: Strict `ruff` enforcement. All PRs must pass `ruff check --fix .`\n- **Typing**: Explicit types preferred\n  - **OK**: `cast(...)`, `assert ...` for type narrowing\n  - **SOMETIMES OK**: Untyped args for simple cases (e.g., prompt handlers)\n  - **NOT OK**: `# type: ignore` without strong justification\n\n### Naming Conventions\n- **Methods**: snake_case\n- **Classes**: PascalCase (e.g., `LocalREPL`, `PortkeyClient`)\n- **Variables**: snake_case\n- **Constants**: UPPER_CASE (e.g., `_SAFE_BUILTINS`, `RLM_SYSTEM_PROMPT`)\n\nDo NOT use `_` prefix for private methods unless explicitly requested.\n\n### Error Handling Philosophy\n- **Fail fast, fail loud** - No defensive programming or silent fallbacks\n- **Minimize branching** - Prefer single code paths; every `if`/`try` needs justification\n- **Example**: Missing API key → immediate `ValueError`, not graceful fallback\n\n## Core Repository Development\n\nFor PRs to `rlm` core:\n```bash\ngit clone https://github.com/alexzhang13/rlm.git\ncd rlm\n\n# Standard development:\nuv sync\n\n# Install dev + test dependencies:\nuv sync --group dev --group test\n\n# Install pre-commit hooks:\nuv run pre-commit install\n```\n\n### Dependencies\n- Avoid new core dependencies\n- Use optional extras for non-essential features (e.g., `modal` extra)\n- Exception: tiny deps that simplify widely-used code\n\n### Testing\n- `uv run pytest` with discovery under `tests/`\n- Write simple, deterministic unit tests\n- Update tests when changing functionality\n- For isolated environments, mock external services\n\n### Documentation\n- Keep concise and actionable\n- Update README when behavior changes\n- Avoid content duplication\n\n### Scope\n- Small, focused diffs\n- One change per PR\n- Backward compatibility is only desirable if it can be done without introducing excessive maintenance burden\n- Delete dead code (don't guard it)\n\n### Checklist\n\nBefore a PR:\n\n```bash\n# Run style + lint checks:\nuv run ruff check --fix .\nuv run ruff format .\nuv run pre-commit run --all-files\n\n# Run tests:\nuv run pytest\n```\n\nEnsure docs and tests are updated if necessary, and dead code is deleted. Strive for minimal, surgical diffs.\n\n## Developing LM Clients\n\nLM client implementations live in `rlm/clients/`. All clients must inherit from `BaseLM`.\n\n### Client Pattern\n\n| Base Class | When to Use | Key Methods |\n|------------|-------------|-------------|\n| `BaseLM` | All LM integrations | `completion`, `acompletion`, `get_usage_summary`, `get_last_usage` |\n\n### Requirements\n- Inherit from `BaseLM` in `rlm/clients/base_lm.py`\n- Implement all abstract methods: `completion`, `acompletion`, `get_usage_summary`, `get_last_usage`\n- Track per-model usage (calls, input/output tokens)\n- Handle both string and message list prompts\n- Register client in `rlm/clients/__init__.py`\n\n### Example Structure\n```python\nfrom rlm.clients.base_lm import BaseLM\nfrom rlm.core.types import ModelUsageSummary, UsageSummary\n\nclass MyClient(BaseLM):\n    def __init__(self, api_key: str, model_name: str, **kwargs):\n        super().__init__(model_name=model_name, **kwargs)\n        # Initialize your client\n        \n    def completion(self, prompt: str | list[dict[str, Any]], model: str | None = None) -> str:\n        # Handle both str and message list formats\n        # Track usage with _track_cost()\n        # Return response string\n        \n    def get_usage_summary(self) -> UsageSummary:\n        # Return aggregated usage across all calls\n```\n\n### Configuration Guidelines\n- **Environment variables**: ONLY for API keys (document in README)\n- **Hardcode**: Default base URLs, reasonable defaults\n- **Arguments**: Essential customization via `__init__()`\n\n## Developing Environments\n\nEnvironment implementations live in `rlm/environments/`. Choose the appropriate base class.\n\n### Environment Pattern\n\n| Pattern | Base Class | When to Use | Key Methods |\n|---------|------------|-------------|-------------|\n| **Non-isolated** | `NonIsolatedEnv` | Local execution, same machine | `setup`, `load_context`, `execute_code` |\n| **Isolated** | `IsolatedEnv` | Cloud sandboxes (Modal, Prime) | `setup`, `load_context`, `execute_code` |\n\n### Requirements\n- Inherit from `NonIsolatedEnv` or `IsolatedEnv` in `rlm/environments/base_env.py`\n- Implement all abstract methods: `setup`, `load_context`, `execute_code`\n- Return `REPLResult` from `execute_code`\n- Handle `lm_handler_address` for LM calls via `llm_query()` and `rlm_query()`\n- Implement `cleanup()` for resource management\n- Register environment in `rlm/environments/__init__.py`\n\n### Key Implementation Details\n- `setup()`: Initialize globals, locals, and helper functions\n- `load_context()`: Make context available as `context` variable\n- `execute_code()`: Execute code, capture stdout/stderr, return `REPLResult`\n- Always provide `llm_query`, `llm_query_batched`, `rlm_query`, and `rlm_query_batched` functions in environment globals\n\n### State Management\nEnvironments must provide these globals to executed code:\n- `context`: The loaded context payload\n- `llm_query(prompt, model=None)`: Plain single LM completion (no REPL, no iteration)\n- `llm_query_batched(prompts, model=None)`: Batched plain LM completions\n- `rlm_query(prompt, model=None)`: Recursive child RLM call (own REPL + iteration). Falls back to `llm_query` at max depth.\n- `rlm_query_batched(prompts, model=None)`: Batched recursive child RLM calls\n- `answer`: A dict (`{\"content\": \"\", \"ready\": False}`); the model sets `answer[\"content\"]` and `answer[\"ready\"] = True` to return a final answer. The env surfaces the content on `REPLResult.final_answer` once `ready` flips truthy.\n- `SHOW_VARS()`: For listing available variables\n\n### Example Structure\n```python\nfrom rlm.environments.base_env import NonIsolatedEnv\nfrom rlm.core.types import REPLResult\n\nclass MyEnvironment(NonIsolatedEnv):\n    def __init__(self, lm_handler_address: tuple[str, int] | None = None, \n                 context_payload: dict | list | str | None = None, **kwargs):\n        super().__init__(**kwargs)\n        self.lm_handler_address = lm_handler_address\n        self.setup()\n        if context_payload:\n            self.load_context(context_payload)\n            \n    def setup(self):\n        # Initialize execution namespace\n        \n    def load_context(self, context_payload: dict | list | str):\n        # Make context available to executed code\n        \n    def execute_code(self, code: str) -> REPLResult:\n        # Execute code and return REPLResult\n        \n    def cleanup(self):\n        # Clean up resources\n```\n\n### Checklist\n- Guidelines here are followed\n- Environment works with basic RLM completion calls\n- `cleanup()` properly releases all resources\n- Sub-LM calls work via `llm_query()` and `rlm_query()`\n- Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `answer`, `SHOW_VARS`) are restored after each execution\n\n## Architecture: Environment ↔ LM Handler Communication\n\nUnderstanding how environments communicate with the LM Handler is essential for developing new environments.\n\n### Overview\n\n```\n┌─────────────────────────────────────────────────────────────────────┐\n│  Host Machine                                                       │\n│  ┌─────────────┐       Socket (TCP)        ┌──────────────────────┐ │\n│  │   RLM       │◄──────────────────────────►  LMHandler           │ │\n│  │  (main)     │                           │  (ThreadingTCPServer)│ │\n│  └─────────────┘                           └──────────────────────┘ │\n│        │                                            ▲               │\n│        ▼                                            │               │\n│  ┌─────────────┐       Socket (TCP)                 │               │\n│  │ LocalREPL   │────────────────────────────────────┘               │\n│  │ (exec code) │  llm_query() / rlm_query() → LM calls               │\n│  └─────────────┘                                                    │\n└─────────────────────────────────────────────────────────────────────┘\n```\n\n### Socket Protocol (Non-Isolated Environments)\n\nNon-isolated environments like `LocalREPL` communicate directly with the `LMHandler` via TCP sockets using a length-prefixed JSON protocol:\n\n**Protocol Format**: `4-byte big-endian length prefix + UTF-8 JSON payload`\n\n```python\n# Sending a message (from rlm/core/comms_utils.py)\ndef socket_send(sock: socket.socket, data: dict) -> None:\n    payload = json.dumps(data).encode(\"utf-8\")\n    sock.sendall(struct.pack(\">I\", len(payload)) + payload)\n```\n\n**Request Flow**:\n1. Environment's `llm_query(prompt)` or `rlm_query(prompt)` is called during code execution\n2. For `llm_query`: creates `LMRequest` and calls `send_lm_request(address, request)`. For `rlm_query`: invokes `subcall_fn` to spawn a child RLM (or falls back to `llm_query` at max depth).\n3. Opens TCP connection to `LMHandler` at `(host, port)`\n4. Sends length-prefixed JSON request\n5. `LMHandler` processes via `LMRequestHandler.handle()`\n6. Returns `LMResponse` with `RLMChatCompletion` or error\n\n**Key Components**:\n- `LMHandler` (`rlm/core/lm_handler.py`): Multi-threaded TCP server wrapping LM clients\n- `LMRequest` / `LMResponse` (`rlm/core/comms_utils.py`): Typed request/response dataclasses\n- `send_lm_request()` / `send_lm_request_batched()`: Helper functions for socket communication\n\n### HTTP Broker Pattern (Isolated Environments)\n\nIsolated environments (Modal, Prime) cannot directly connect to the host's socket server. They use an HTTP broker pattern:\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│  Host Machine                                                               │\n│  ┌─────────┐    Socket    ┌────────────┐    HTTP Poll    ┌────────────────┐ │\n│  │   RLM   │◄────────────►│  LMHandler │◄────────────────│   ModalREPL    │ │\n│  └─────────┘              └────────────┘                 │  (poller)      │ │\n│                                                          └────────────────┘ │\n│                                                                  │          │\n│                                                          HTTP (tunnel)      │\n│                                                                  │          │\n└──────────────────────────────────────────────────────────────────┼──────────┘\n                                                                   │\n┌──────────────────────────────────────────────────────────────────┼──────────┐\n│  Cloud Sandbox (Modal/Prime)                                     ▼          │\n│  ┌─────────────┐     HTTP (localhost)     ┌─────────────────────────────┐   │\n│  │ Exec Script │◄────────────────────────►│   Broker Server (Flask)     │   │\n│  │ (exec code) │     /enqueue, etc.       │   - /enqueue (submit req)   │   │\n│  └─────────────┘                          │   - /pending (poll reqs)    │   │\n│                                           │   - /respond (return resp)  │   │\n│                                           └─────────────────────────────┘   │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n**How It Works**:\n\n1. **Sandbox Setup**: Environment creates a cloud sandbox with an HTTP broker server running inside\n2. **Tunnel Exposure**: Broker server is exposed via encrypted tunnel (e.g., Modal's `encrypted_ports`)\n3. **Code Execution**: When `llm_query()` is called inside sandbox, it POSTs to `http://localhost:8080/enqueue`\n4. **Request Queuing**: Broker queues the request and blocks waiting for response\n5. **Host Polling**: `ModalREPL` on host polls `{tunnel_url}/pending` for new requests\n6. **LM Forwarding**: Host forwards requests to `LMHandler` via socket, gets response\n7. **Response Delivery**: Host POSTs response to `{tunnel_url}/respond`\n8. **Unblocking**: Broker unblocks the original `/enqueue` call with the response\n\n**Broker Endpoints**:\n| Endpoint | Method | Purpose |\n|----------|--------|---------|\n| `/enqueue` | POST | Submit LLM request from sandbox code (blocks until response) |\n| `/pending` | GET | Get list of pending requests (called by host poller) |\n| `/respond` | POST | Submit response for a request ID (called by host poller) |\n| `/health` | GET | Health check |\n\n**Key Implementation Details**:\n- Broker runs as a Flask server inside the sandbox\n- Uses `threading.Event` for request/response synchronization\n- Poller thread on host runs in background with 100ms polling interval\n- State persistence via `dill` serialization to `/tmp/rlm_state.dill`\n\n### Implementing a New Isolated Environment\n\nWhen building a new isolated environment (e.g., for a new cloud provider):\n\n1. **Create broker server** - Flask/HTTP server with `/enqueue`, `/pending`, `/respond` endpoints\n2. **Expose tunnel** - Use provider's tunnel/port forwarding to expose broker to host\n3. **Implement poller** - Background thread on host to poll and forward requests\n4. **Build exec script** - Script that runs inside sandbox with `llm_query()` calling broker\n5. **Handle state** - Serialize/deserialize execution state between code blocks\n\nSee `rlm/environments/modal_repl.py` as the canonical reference implementation.\n\n"},"files":{"docs/architecture.md":"---\nlayout: default\ntitle: Architecture\nnav_order: 3\n---\n\n# Architecture\n{: .no_toc }\n\nHow the RLM runtime, LM handler, code execution, and recursive sub-calls fit together.\n{: .fs-6 .fw-300 }\n\n## Table of Contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n\n## Overview\n\nAn RLM completion involves three cooperating pieces:\n\n1. **RLM** (`rlm/core/rlm.py`) — the main loop that drives iteration.\n2. **LMHandler** (`rlm/core/lm_handler.py`) — a per-completion TCP server that routes LM API calls.\n3. **LocalREPL** (`rlm/environments/local_repl.py`) — the Python execution environment where model-generated code runs.\n\n```\n┌────────────────────────────────────────────────────────────────┐\n│  RLM.completion(prompt)                                        │\n│                                                                │\n│  1. Spawn LMHandler (TCP server on localhost, auto port)       │\n│  2. Create LocalREPL (in-process exec() namespace)             │\n│  3. Iterate:                                                   │\n│     a. Send message history → LM backend → get response        │\n│     b. Extract ```repl``` code blocks from response            │\n│     c. Execute code in LocalREPL                               │\n│     d. Append stdout/stderr to message history                 │\n│     e. Repeat until answer[\"ready\"] is True or limits exceeded │\n│  4. Tear down handler and environment                          │\n└────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## LM Handler\n\n### What it is\n\nThe LMHandler is a **multi-threaded TCP socket server** that sits between\nthe execution environment and the actual LM API backends. Every call to\n`RLM.completion()` spins up a fresh handler (unless the environment is\npersistent and already has one).\n\n### Why a socket server?\n\nThe handler exists so that code running inside the execution environment\ncan make LM calls back to the host process without directly importing or\ncalling the LM client. This is essential for isolated environments (Docker,\nModal) that run in separate processes or machines — they communicate with\nthe handler over TCP. The local environment uses the same protocol for\nconsistency, even though it runs in-process.\n\n### Lifecycle\n\n```python\n# Inside RLM._spawn_completion_context():\nclient = get_client(backend, backend_kwargs)            # 1. Create LM client\nlm_handler = LMHandler(client, other_backend_client=…)  # 2. Wrap in handler\nlm_handler.start()                                       # 3. Start TCP server (daemon thread)\n# … run completion loop …\nlm_handler.stop()                                        # 4. Shut down server\n```\n\n- The server binds to `127.0.0.1` with port `0` (OS auto-assigns an available port).\n- It runs in a **daemon thread** so it doesn't block process exit.\n- Each incoming connection is handled by a new thread (`ThreadingTCPServer`).\n\n### Wire protocol\n\nAll messages use a simple framing: **4-byte big-endian length prefix + UTF-8 JSON payload**.\n\n```\n┌──────────┬─────────────────────────┐\n│ 4 bytes  │  N bytes                │\n│ len (BE) │  JSON payload (UTF-8)   │\n└──────────┴─────────────────────────┘\n```\n\nImplemented in `socket_send()` / `socket_recv()` in `rlm/core/comms_utils.py`.\n\n### Client routing\n\nThe handler can hold multiple LM clients and routes requests based on the\n`model` and `depth` fields in the request:\n\n```python\ndef get_client(self, model=None, depth=0):\n    if model and model in self.clients:\n        return self.clients[model]        # Explicit model override\n    if depth == 1 and self.other_backend_client:\n        return self.other_backend_client  # Depth-based routing\n    return self.default_client            # Fallback\n```\n\nThis lets you use a different (e.g. cheaper/faster) model for sub-LM calls\nby specifying `other_backends` / `other_backend_kwargs` in the RLM constructor.\n\n---\n\n## Code Execution Environment (LocalREPL)\n\n### In-process `exec()` — not a subprocess\n\nLocalREPL executes model-generated code **in the same Python interpreter\nprocess** as the RLM, using Python's built-in `exec()`. There is no\nsubprocess, no fork, and no IPC for code execution.\n\n```python\n# Simplified from LocalREPL.execute_code():\ncombined = {**self.globals, **self.locals}\nexec(code, combined, combined)\n```\n\n### What this means in practice\n\n- **Fast**: No process spawn overhead. Code execution is as fast as native Python.\n- **Persistent namespace**: Variables created in one code block are visible in the next. The `self.locals` dict accumulates state across iterations.\n- **Shared memory**: Helper functions like `llm_query()` and `rlm_query()` are plain Python closures in `self.globals`. When model code calls `llm_query(\"...\")`, it's a direct function call within the same process.\n- **Limited sandbox**: Dangerous builtins (`eval`, `exec`, `compile`, `input`) are removed from the namespace. This is a soft sandbox — it prevents accidental misuse but is not a security boundary.\n\n### Namespace layout\n\n```\nglobals (shared across all executions):\n├── __builtins__        → _SAFE_BUILTINS (eval/exec/input removed)\n├── llm_query()         → plain LM call via handler\n├── llm_query_batched() → batched plain LM calls\n├── rlm_query()         → recursive RLM sub-call (or fallback to llm_query)\n├── rlm_query_batched() → batched recursive sub-calls\n├── SHOW_VARS()         → list user-created variables\n└── <custom_tools>      → user-provided callable tools\n\nlocals (accumulates user variables):\n├── context             → alias for context_0\n├── context_0           → first context payload\n├── context_1, …        → additional contexts (persistent mode)\n├── history             → conversation history (persistent/compaction mode)\n├── answer              → {\"content\": \"\", \"ready\": False}; set ready=True to finish\n└── <user variables>    → anything created by model code\n```\n\n### Scaffold restoration\n\nAfter each `exec()`, LocalREPL restores all reserved names to prevent model\ncode from corrupting the environment. If the model writes\n`llm_query = \"oops\"` or `context = None`, the next execution will still\nhave the real functions and data. See `_restore_scaffold()`.\n\n---\n\n## How `llm_query()` and `rlm_query()` Work\n\nThese are the two functions available to model-generated code for making LM calls.\nThey have very different behaviors:\n\n### `llm_query(prompt, model=None)` — Plain LM call\n\nAlways makes a single, direct LM completion. No REPL, no iteration — just\nprompt in, text out. Fast and lightweight.\n\n```\nModel code: answer = llm_query(\"Summarize this text: ...\")\n    │\n    ▼\nLocalREPL._llm_query()\n    │  Creates LMRequest(prompt=..., depth=self.depth)\n    │  Opens TCP socket to handler\n    ▼\nLMHandler (TCP server)\n    │  get_client(model, depth) → selects backend\n    │  client.completion(prompt) → calls LM API\n    ▼\nResponse flows back over socket\n    │\n    ▼\nReturns response string to model code\n```\n\n### `rlm_query(prompt, model=None)` — Recursive RLM sub-call\n\nSpawns a **child RLM** that gets its own REPL and can reason iteratively\nover the prompt — just like the parent. Use this when the subtask needs\nmulti-step reasoning, code execution, or its own iterative problem-solving.\n\nFalls back to `llm_query` when recursion is not available (i.e. the current\ndepth has reached `max_depth`).\n\n```\nModel code: answer = rlm_query(\"Solve this complex problem: ...\")\n    │\n    ▼\nLocalREPL._rlm_query()\n    │  if self.subcall_fn is not None:    ← set when max_depth > 1\n    │      calls self.subcall_fn(prompt, model)\n    │  else:\n    │      falls back to _llm_query()\n    │\n    ▼  (when subcall_fn exists)\nRLM._subcall(prompt, model)\n    │  next_depth = self.depth + 1\n    │  if next_depth >= max_depth:\n    │      → plain client.completion() (leaf call, no REPL)\n    │  else:\n    │      → create child RLM(depth=next_depth, ...)\n    │      → child.completion(prompt)  ← full RLM with its own handler + REPL\n    │\n    ▼\nReturns RLMChatCompletion to parent\n```\n\n### `llm_query_batched` / `rlm_query_batched`\n\nSame semantics as above, but for multiple prompts. `llm_query_batched` sends\nall prompts as a single batched request to the handler, which processes them\nconcurrently with `asyncio.gather`. `rlm_query_batched` calls `subcall_fn`\nsequentially for each prompt (each child RLM is a blocking call).\n\n**Failure handling is per-prompt.** A single failed call does not fail the\nwhole batch — that slot returns the string `\"Error: llm() call failed - <msg>\"`\n(with the underlying error message) while the other prompts return their real\nresponses. Results stay aligned with the input prompts by index, so the list is\nalways the same length as `prompts`.\n\n---\n\n## Recursive Sub-Calls (Depth > 1)\n\n### How depth works\n\n```\nmax_depth=3\n\nRLM (depth=0)\n └─ rlm_query() → child RLM (depth=1)\n     └─ rlm_query() → child RLM (depth=2)\n         └─ rlm_query() → plain LM call (depth=3 >= max_depth, no REPL)\n```\n\n- `depth=0` is the root RLM that the user calls.\n- Each child increments depth by 1.\n- When `next_depth >= max_depth`, `_subcall()` does a plain `client.completion()` instead of creating a child RLM. This is the leaf case — no REPL, no iteration.\n- `llm_query()` always does a plain LM call regardless of depth. Only `rlm_query()` triggers recursion.\n\n### Each child gets its own handler and environment\n\nWhen a child RLM is created via `_subcall()`, its `completion()` method calls\n`_spawn_completion_context()` which creates:\n\n1. **A new `LMHandler`** listening on a **different auto-assigned port**.\n2. **A new `LocalREPL`** with its own isolated namespace.\n\n```\nParent RLM (depth=0)\n├── LMHandler #1 on port 52301\n├── LocalREPL #1 (depth=1)\n│   ├── globals: {llm_query, rlm_query, ...}\n│   ├── locals: {context: \"parent prompt\", ...}\n│   └── subcall_fn = RLM._subcall  ← enables rlm_query()\n│\n└── When model code calls rlm_query(\"subtask\"):\n    │\n    └── Child RLM (depth=1)\n        ├── LMHandler #2 on port 52302  ← NEW handler, NEW port\n        ├── LocalREPL #2 (depth=2)      ← NEW namespace\n        │   ├── locals: {context: \"subtask\", ...}\n        │   └── subcall_fn = child._subcall (or None if at max_depth-1)\n        │\n        └── Runs its own iteration loop, returns RLMChatCompletion\n```\n\nThe child's handler and environment are torn down when the child's `completion()` finishes.\n\n### Resource limits propagate\n\nThe parent passes **remaining** budget/timeout/tokens to the child, not the\noriginal totals. This prevents a child from consuming all of the parent's resources:\n\n```python\n# In _subcall():\nremaining_timeout = self.max_timeout - elapsed  # not self.max_timeout\nremaining_budget = self.max_budget - spent       # not self.max_budget\nchild = RLM(..., max_timeout=remaining_timeout, max_budget=remaining_budget)\n```\n\n### Metadata flows back\n\nEach child RLM can have its own `RLMLogger`. When the child completes, its\nfull trajectory metadata (iterations, code blocks, sub-calls) is captured in\nthe returned `RLMChatCompletion.metadata` dict. The parent's logger records\nthis as part of the REPL result's `rlm_calls` list, creating a nested\nmetadata tree.\n\n---\n\n## Putting It All Together\n\nHere's the complete request flow for a depth-2 RLM call:\n\n```\nUser: rlm.completion(\"Analyze this data\")\n │\n ▼\nRLM (depth=0)\n ├─ _spawn_completion_context()\n │   ├─ LMHandler #1 starts on port 52301\n │   └─ LocalREPL #1 created with context=\"Analyze this data\"\n │\n ├─ Iteration 1: LM generates code\n │   │  ```repl\n │   │  answer = rlm_query(\"What patterns exist in: \" + context[:5000])\n │   │  ```\n │   │\n │   └─ LocalREPL.execute_code() runs the code via exec()\n │       │\n │       ├─ rlm_query() → _rlm_query() → subcall_fn()\n │       │   │\n │       │   └─ RLM._subcall(\"What patterns exist in: ...\")\n │       │       │\n │       │       ├─ depth=1 < max_depth=2, so create child RLM\n │       │       │\n │       │       └─ Child RLM (depth=1)\n │       │           ├─ LMHandler #2 on port 52302\n │       │           ├─ LocalREPL #2 with context=\"What patterns...\"\n │       │           │\n │       │           ├─ Child iteration 1: LM generates code\n │       │           │   │  result = llm_query(\"Extract key metrics: \" + context)\n │       │           │   │\n │       │           │   └─ llm_query() → TCP to Handler #2 → LM API → response\n │       │           │\n │       │           ├─ Child iteration 2: LM sets answer[\"content\"]=result, answer[\"ready\"]=True\n │       │           │\n │       │           └─ Returns RLMChatCompletion to parent\n │       │\n │       └─ child_response = child_completion.response\n │\n ├─ Iteration 2: LM uses child_response, sets answer[\"content\"]=final, answer[\"ready\"]=True\n │\n ├─ LMHandler #1 stops\n └─ Returns RLMChatCompletion to user\n```\n\n### Key takeaways\n\n| Aspect | Detail |\n|:-------|:-------|\n| Code execution | In-process `exec()` in the same Python interpreter. Not a subprocess. |\n| LM calls from code | Go through a local TCP socket server (LMHandler), even for in-process execution. |\n| Handler per completion | Each `completion()` call gets its own handler on an auto-assigned port. |\n| Child RLMs | Created by `_subcall()`, each with its own handler + LocalREPL. Fully independent. |\n| `llm_query` vs `rlm_query` | `llm_query` = always plain LM call. `rlm_query` = recursive child RLM (or fallback). |\n| Depth limit | At `max_depth`, `rlm_query` falls back to `llm_query`. No further recursion. |\n| Resource isolation | Children get remaining budget/timeout, not the full amount. |\n| Namespace isolation | Each LocalREPL has its own `globals`/`locals`. No shared state between parent and child. |\n","AGENTS.md":"# AGENTS.md\n\nThis guide covers best practices for contributing to the core Recursive Language Models `rlm` library and developing new environments (in `rlm/environments/`) and LM clients (in `rlm/clients/`).\n\n## Setup\n\nWe use `uv` for developing `rlm`.\n```bash\n# Install uv (first time)\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Setup blank project if needed\nuv init && uv venv --python 3.12\nsource .venv/bin/activate\n\n# Install in editable mode\nuv pip install -e .\n\n# For Modal sandbox support\nuv pip install -e \".[modal]\"\n\n# For Prime sandbox support\nuv pip install -e \".[prime]\"\n```\n\n## General Guidelines\n\n### Code Style & Typing\n- **Formatting**: Strict `ruff` enforcement. All PRs must pass `ruff check --fix .`\n- **Typing**: Explicit types preferred\n  - **OK**: `cast(...)`, `assert ...` for type narrowing\n  - **SOMETIMES OK**: Untyped args for simple cases (e.g., prompt handlers)\n  - **NOT OK**: `# type: ignore` without strong justification\n\n### Naming Conventions\n- **Methods**: snake_case\n- **Classes**: PascalCase (e.g., `LocalREPL`, `PortkeyClient`)\n- **Variables**: snake_case\n- **Constants**: UPPER_CASE (e.g., `_SAFE_BUILTINS`, `RLM_SYSTEM_PROMPT`)\n\nDo NOT use `_` prefix for private methods unless explicitly requested.\n\n### Error Handling Philosophy\n- **Fail fast, fail loud** - No defensive programming or silent fallbacks\n- **Minimize branching** - Prefer single code paths; every `if`/`try` needs justification\n- **Example**: Missing API key → immediate `ValueError`, not graceful fallback\n\n## Core Repository Development\n\nFor PRs to `rlm` core:\n```bash\ngit clone https://github.com/alexzhang13/rlm.git\ncd rlm\n\n# Standard development:\nuv sync\n\n# Install dev + test dependencies:\nuv sync --group dev --group test\n\n# Install pre-commit hooks:\nuv run pre-commit install\n```\n\n### Dependencies\n- Avoid new core dependencies\n- Use optional extras for non-essential features (e.g., `modal` extra)\n- Exception: tiny deps that simplify widely-used code\n\n### Testing\n- `uv run pytest` with discovery under `tests/`\n- Write simple, deterministic unit tests\n- Update tests when changing functionality\n- For isolated environments, mock external services\n\n### Documentation\n- Keep concise and actionable\n- Update README when behavior changes\n- Avoid content duplication\n\n### Scope\n- Small, focused diffs\n- One change per PR\n- Backward compatibility is only desirable if it can be done without introducing excessive maintenance burden\n- Delete dead code (don't guard it)\n\n### Checklist\n\nBefore a PR:\n\n```bash\n# Run style + lint checks:\nuv run ruff check --fix .\nuv run ruff format .\nuv run pre-commit run --all-files\n\n# Run tests:\nuv run pytest\n```\n\nEnsure docs and tests are updated if necessary, and dead code is deleted. Strive for minimal, surgical diffs.\n\n## Developing LM Clients\n\nLM client implementations live in `rlm/clients/`. All clients must inherit from `BaseLM`.\n\n### Client Pattern\n\n| Base Class | When to Use | Key Methods |\n|------------|-------------|-------------|\n| `BaseLM` | All LM integrations | `completion`, `acompletion`, `get_usage_summary`, `get_last_usage` |\n\n### Requirements\n- Inherit from `BaseLM` in `rlm/clients/base_lm.py`\n- Implement all abstract methods: `completion`, `acompletion`, `get_usage_summary`, `get_last_usage`\n- Track per-model usage (calls, input/output tokens)\n- Handle both string and message list prompts\n- Register client in `rlm/clients/__init__.py`\n\n### Example Structure\n```python\nfrom rlm.clients.base_lm import BaseLM\nfrom rlm.core.types import ModelUsageSummary, UsageSummary\n\nclass MyClient(BaseLM):\n    def __init__(self, api_key: str, model_name: str, **kwargs):\n        super().__init__(model_name=model_name, **kwargs)\n        # Initialize your client\n        \n    def completion(self, prompt: str | list[dict[str, Any]], model: str | None = None) -> str:\n        # Handle both str and message list formats\n        # Track usage with _track_cost()\n        # Return response string\n        \n    def get_usage_summary(self) -> UsageSummary:\n        # Return aggregated usage across all calls\n```\n\n### Configuration Guidelines\n- **Environment variables**: ONLY for API keys (document in README)\n- **Hardcode**: Default base URLs, reasonable defaults\n- **Arguments**: Essential customization via `__init__()`\n\n## Developing Environments\n\nEnvironment implementations live in `rlm/environments/`. Choose the appropriate base class.\n\n### Environment Pattern\n\n| Pattern | Base Class | When to Use | Key Methods |\n|---------|------------|-------------|-------------|\n| **Non-isolated** | `NonIsolatedEnv` | Local execution, same machine | `setup`, `load_context`, `execute_code` |\n| **Isolated** | `IsolatedEnv` | Cloud sandboxes (Modal, Prime) | `setup`, `load_context`, `execute_code` |\n\n### Requirements\n- Inherit from `NonIsolatedEnv` or `IsolatedEnv` in `rlm/environments/base_env.py`\n- Implement all abstract methods: `setup`, `load_context`, `execute_code`\n- Return `REPLResult` from `execute_code`\n- Handle `lm_handler_address` for LM calls via `llm_query()` and `rlm_query()`\n- Implement `cleanup()` for resource management\n- Register environment in `rlm/environments/__init__.py`\n\n### Key Implementation Details\n- `setup()`: Initialize globals, locals, and helper functions\n- `load_context()`: Make context available as `context` variable\n- `execute_code()`: Execute code, capture stdout/stderr, return `REPLResult`\n- Always provide `llm_query`, `llm_query_batched`, `rlm_query`, and `rlm_query_batched` functions in environment globals\n\n### State Management\nEnvironments must provide these globals to executed code:\n- `context`: The loaded context payload\n- `llm_query(prompt, model=None)`: Plain single LM completion (no REPL, no iteration)\n- `llm_query_batched(prompts, model=None)`: Batched plain LM completions\n- `rlm_query(prompt, model=None)`: Recursive child RLM call (own REPL + iteration). Falls back to `llm_query` at max depth.\n- `rlm_query_batched(prompts, model=None)`: Batched recursive child RLM calls\n- `answer`: A dict (`{\"content\": \"\", \"ready\": False}`); the model sets `answer[\"content\"]` and `answer[\"ready\"] = True` to return a final answer. The env surfaces the content on `REPLResult.final_answer` once `ready` flips truthy.\n- `SHOW_VARS()`: For listing available variables\n\n### Example Structure\n```python\nfrom rlm.environments.base_env import NonIsolatedEnv\nfrom rlm.core.types import REPLResult\n\nclass MyEnvironment(NonIsolatedEnv):\n    def __init__(self, lm_handler_address: tuple[str, int] | None = None, \n                 context_payload: dict | list | str | None = None, **kwargs):\n        super().__init__(**kwargs)\n        self.lm_handler_address = lm_handler_address\n        self.setup()\n        if context_payload:\n            self.load_context(context_payload)\n            \n    def setup(self):\n        # Initialize execution namespace\n        \n    def load_context(self, context_payload: dict | list | str):\n        # Make context available to executed code\n        \n    def execute_code(self, code: str) -> REPLResult:\n        # Execute code and return REPLResult\n        \n    def cleanup(self):\n        # Clean up resources\n```\n\n### Checklist\n- Guidelines here are followed\n- Environment works with basic RLM completion calls\n- `cleanup()` properly releases all resources\n- Sub-LM calls work via `llm_query()` and `rlm_query()`\n- Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `answer`, `SHOW_VARS`) are restored after each execution\n\n## Architecture: Environment ↔ LM Handler Communication\n\nUnderstanding how environments communicate with the LM Handler is essential for developing new environments.\n\n### Overview\n\n```\n┌─────────────────────────────────────────────────────────────────────┐\n│  Host Machine                                                       │\n│  ┌─────────────┐       Socket (TCP)        ┌──────────────────────┐ │\n│  │   RLM       │◄──────────────────────────►  LMHandler           │ │\n│  │  (main)     │                           │  (ThreadingTCPServer)│ │\n│  └─────────────┘                           └──────────────────────┘ │\n│        │                                            ▲               │\n│        ▼                                            │               │\n│  ┌─────────────┐       Socket (TCP)                 │               │\n│  │ LocalREPL   │────────────────────────────────────┘               │\n│  │ (exec code) │  llm_query() / rlm_query() → LM calls               │\n│  └─────────────┘                                                    │\n└─────────────────────────────────────────────────────────────────────┘\n```\n\n### Socket Protocol (Non-Isolated Environments)\n\nNon-isolated environments like `LocalREPL` communicate directly with the `LMHandler` via TCP sockets using a length-prefixed JSON protocol:\n\n**Protocol Format**: `4-byte big-endian length prefix + UTF-8 JSON payload`\n\n```python\n# Sending a message (from rlm/core/comms_utils.py)\ndef socket_send(sock: socket.socket, data: dict) -> None:\n    payload = json.dumps(data).encode(\"utf-8\")\n    sock.sendall(struct.pack(\">I\", len(payload)) + payload)\n```\n\n**Request Flow**:\n1. Environment's `llm_query(prompt)` or `rlm_query(prompt)` is called during code execution\n2. For `llm_query`: creates `LMRequest` and calls `send_lm_request(address, request)`. For `rlm_query`: invokes `subcall_fn` to spawn a child RLM (or falls back to `llm_query` at max depth).\n3. Opens TCP connection to `LMHandler` at `(host, port)`\n4. Sends length-prefixed JSON request\n5. `LMHandler` processes via `LMRequestHandler.handle()`\n6. Returns `LMResponse` with `RLMChatCompletion` or error\n\n**Key Components**:\n- `LMHandler` (`rlm/core/lm_handler.py`): Multi-threaded TCP server wrapping LM clients\n- `LMRequest` / `LMResponse` (`rlm/core/comms_utils.py`): Typed request/response dataclasses\n- `send_lm_request()` / `send_lm_request_batched()`: Helper functions for socket communication\n\n### HTTP Broker Pattern (Isolated Environments)\n\nIsolated environments (Modal, Prime) cannot directly connect to the host's socket server. They use an HTTP broker pattern:\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│  Host Machine                                                               │\n│  ┌─────────┐    Socket    ┌────────────┐    HTTP Poll    ┌────────────────┐ │\n│  │   RLM   │◄────────────►│  LMHandler │◄────────────────│   ModalREPL    │ │\n│  └─────────┘              └────────────┘                 │  (poller)      │ │\n│                                                          └────────────────┘ │\n│                                                                  │          │\n│                                                          HTTP (tunnel)      │\n│                                                                  │          │\n└──────────────────────────────────────────────────────────────────┼──────────┘\n                                                                   │\n┌──────────────────────────────────────────────────────────────────┼──────────┐\n│  Cloud Sandbox (Modal/Prime)                                     ▼          │\n│  ┌─────────────┐     HTTP (localhost)     ┌─────────────────────────────┐   │\n│  │ Exec Script │◄────────────────────────►│   Broker Server (Flask)     │   │\n│  │ (exec code) │     /enqueue, etc.       │   - /enqueue (submit req)   │   │\n│  └─────────────┘                          │   - /pending (poll reqs)    │   │\n│                                           │   - /respond (return resp)  │   │\n│                                           └─────────────────────────────┘   │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n**How It Works**:\n\n1. **Sandbox Setup**: Environment creates a cloud sandbox with an HTTP broker server running inside\n2. **Tunnel Exposure**: Broker server is exposed via encrypted tunnel (e.g., Modal's `encrypted_ports`)\n3. **Code Execution**: When `llm_query()` is called inside sandbox, it POSTs to `http://localhost:8080/enqueue`\n4. **Request Queuing**: Broker queues the request and blocks waiting for response\n5. **Host Polling**: `ModalREPL` on host polls `{tunnel_url}/pending` for new requests\n6. **LM Forwarding**: Host forwards requests to `LMHandler` via socket, gets response\n7. **Response Delivery**: Host POSTs response to `{tunnel_url}/respond`\n8. **Unblocking**: Broker unblocks the original `/enqueue` call with the response\n\n**Broker Endpoints**:\n| Endpoint | Method | Purpose |\n|----------|--------|---------|\n| `/enqueue` | POST | Submit LLM request from sandbox code (blocks until response) |\n| `/pending` | GET | Get list of pending requests (called by host poller) |\n| `/respond` | POST | Submit response for a request ID (called by host poller) |\n| `/health` | GET | Health check |\n\n**Key Implementation Details**:\n- Broker runs as a Flask server inside the sandbox\n- Uses `threading.Event` for request/response synchronization\n- Poller thread on host runs in background with 100ms polling interval\n- State persistence via `dill` serialization to `/tmp/rlm_state.dill`\n\n### Implementing a New Isolated Environment\n\nWhen building a new isolated environment (e.g., for a new cloud provider):\n\n1. **Create broker server** - Flask/HTTP server with `/enqueue`, `/pending`, `/respond` endpoints\n2. **Expose tunnel** - Use provider's tunnel/port forwarding to expose broker to host\n3. **Implement poller** - Background thread on host to poll and forward requests\n4. **Build exec script** - Script that runs inside sandbox with `llm_query()` calling broker\n5. **Handle state** - Serialize/deserialize execution state between code blocks\n\nSee `rlm/environments/modal_repl.py` as the canonical reference implementation.\n\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","rawUrl":"https://raw.githubusercontent.com/alexzhang13/rlm/HEAD/AGENTS.md","title":"AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"# AGENTS.md\n\nThis guide covers best practices for contributing to the core Recursive Language Models `rlm` library and developing new environments (in `rlm/environments/`) and LM clients (in `rlm/clients/`).\n\n## Setup\n\nWe use `uv` for developing `rlm`.\n```bash\n# Install uv (first time)\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Setup blank project if needed\nuv init && uv venv --python 3.12\nsource .venv/bin/activate\n\n# Install in editable mode\nuv pip install -e .\n\n# For Modal sandbox support\nuv pip install -e \".[modal]\"\n\n# For Prime sandbox support\nuv pip install -e \".[prime]\"\n```\n\n## General Guidelines\n\n### Code Style & Typing\n- **Formatting**: Strict `ruff` enforcement. All PRs must pass `ruff check --fix .`\n- **Typing**: Explicit types preferred\n  - **OK**: `cast(...)`, `assert ...` for type narrowing\n  - **SOMETIMES OK**: Untyped args for simple cases (e.g., prompt handlers)\n  - **NOT OK**: `# type: ignore` without strong justification\n\n### Naming Conventions\n- **Methods**: snake_case\n- **Classes**: PascalCase (e.g., `LocalREPL`, `PortkeyClient`)\n- **Variables**: snake_case\n- **Constants**: UPPER_CASE (e.g., `_SAFE_BUILTINS`, `RLM_SYSTEM_PROMPT`)\n\nDo NOT use `_` prefix for private methods unless explicitly requested.\n\n### Error Handling Philosophy\n- **Fail fast, fail loud** - No defensive programming or silent fallbacks\n- **Minimize branching** - Prefer single code paths; every `if`/`try` needs justification\n- **Example**: Missing API key → immediate `ValueError`, not graceful fallback\n\n## Core Repository Development\n\nFor PRs to `rlm` core:\n```bash\ngit clone https://github.com/alexzhang13/rlm.git\ncd rlm\n\n# Standard development:\nuv sync\n\n# Install dev + test dependencies:\nuv sync --group dev --group test\n\n# Install pre-commit hooks:\nuv run pre-commit install\n```\n\n### Dependencies\n- Avoid new core dependencies\n- Use optional extras for non-essential features (e.g., `modal` extra)\n- Exception: tiny deps that simplify widely-used code\n\n### Testing\n- `uv run pytest` with discovery under `tests/`\n- Write simple, deterministic unit tests\n- Update tests when changing functionality\n- For isolated environments, mock external services\n\n### Documentation\n- Keep concise and actionable\n- Update README when behavior changes\n- Avoid content duplication\n\n### Scope\n- Small, focused diffs\n- One change per PR\n- Backward compatibility is only desirable if it can be done without introducing excessive maintenance burden\n- Delete dead code (don't guard it)\n\n### Checklist\n\nBefore a PR:\n\n```bash\n# Run style + lint checks:\nuv run ruff check --fix .\nuv run ruff format .\nuv run pre-commit run --all-files\n\n# Run tests:\nuv run pytest\n```\n\nEnsure docs and tests are updated if necessary, and dead code is deleted. Strive for minimal, surgical diffs.\n\n## Developing LM Clients\n\nLM client implementations live in `rlm/clients/`. All clients must inherit from `BaseLM`.\n\n### Client Pattern\n\n| Base Class | When to Use | Key Methods |\n|------------|-------------|-------------|\n| `BaseLM` | All LM integrations | `completion`, `acompletion`, `get_usage_summary`, `get_last_usage` |\n\n### Requirements\n- Inherit from `BaseLM` in `rlm/clients/base_lm.py`\n- Implement all abstract methods: `completion`, `acompletion`, `get_usage_summary`, `get_last_usage`\n- Track per-model usage (calls, input/output tokens)\n- Handle both string and message list prompts\n- Register client in `rlm/clients/__init__.py`\n\n### Example Structure\n```python\nfrom rlm.clients.base_lm import BaseLM\nfrom rlm.core.types import ModelUsageSummary, UsageSummary\n\nclass MyClient(BaseLM):\n    def __init__(self, api_key: str, model_name: str, **kwargs):\n        super().__init__(model_name=model_name, **kwargs)\n        # Initialize your client\n        \n    def completion(self, prompt: str | list[dict[str, Any]], model: str | None = None) -> str:\n        # Handle both str and message list formats\n        # Track usage with _track_cost()\n        # Return response string\n        \n    def get_usage_summary(self) -> UsageSummary:\n        # Return aggregated usage across all calls\n```\n\n### Configuration Guidelines\n- **Environment variables**: ONLY for API keys (document in README)\n- **Hardcode**: Default base URLs, reasonable defaults\n- **Arguments**: Essential customization via `__init__()`\n\n## Developing Environments\n\nEnvironment implementations live in `rlm/environments/`. Choose the appropriate base class.\n\n### Environment Pattern\n\n| Pattern | Base Class | When to Use | Key Methods |\n|---------|------------|-------------|-------------|\n| **Non-isolated** | `NonIsolatedEnv` | Local execution, same machine | `setup`, `load_context`, `execute_code` |\n| **Isolated** | `IsolatedEnv` | Cloud sandboxes (Modal, Prime) | `setup`, `load_context`, `execute_code` |\n\n### Requirements\n- Inherit from `NonIsolatedEnv` or `IsolatedEnv` in `rlm/environments/base_env.py`\n- Implement all abstract methods: `setup`, `load_context`, `execute_code`\n- Return `REPLResult` from `execute_code`\n- Handle `lm_handler_address` for LM calls via `llm_query()` and `rlm_query()`\n- Implement `cleanup()` for resource management\n- Register environment in `rlm/environments/__init__.py`\n\n### Key Implementation Details\n- `setup()`: Initialize globals, locals, and helper functions\n- `load_context()`: Make context available as `context` variable\n- `execute_code()`: Execute code, capture stdout/stderr, return `REPLResult`\n- Always provide `llm_query`, `llm_query_batched`, `rlm_query`, and `rlm_query_batched` functions in environment globals\n\n### State Management\nEnvironments must provide these globals to executed code:\n- `context`: The loaded context payload\n- `llm_query(prompt, model=None)`: Plain single LM completion (no REPL, no iteration)\n- `llm_query_batched(prompts, model=None)`: Batched plain LM completions\n- `rlm_query(prompt, model=None)`: Recursive child RLM call (own REPL + iteration). Falls back to `llm_query` at max depth.\n- `rlm_query_batched(prompts, model=None)`: Batched recursive child RLM calls\n- `answer`: A dict (`{\"content\": \"\", \"ready\": False}`); the model sets `answer[\"content\"]` and `answer[\"ready\"] = True` to return a final answer. The env surfaces the content on `REPLResult.final_answer` once `ready` flips truthy.\n- `SHOW_VARS()`: For listing available variables\n\n### Example Structure\n```python\nfrom rlm.environments.base_env import NonIsolatedEnv\nfrom rlm.core.types import REPLResult\n\nclass MyEnvironment(NonIsolatedEnv):\n    def __init__(self, lm_handler_address: tuple[str, int] | None = None, \n                 context_payload: dict | list | str | None = None, **kwargs):\n        super().__init__(**kwargs)\n        self.lm_handler_address = lm_handler_address\n        self.setup()\n        if context_payload:\n            self.load_context(context_payload)\n            \n    def setup(self):\n        # Initialize execution namespace\n        \n    def load_context(self, context_payload: dict | list | str):\n        # Make context available to executed code\n        \n    def execute_code(self, code: str) -> REPLResult:\n        # Execute code and return REPLResult\n        \n    def cleanup(self):\n        # Clean up resources\n```\n\n### Checklist\n- Guidelines here are followed\n- Environment works with basic RLM completion calls\n- `cleanup()` properly releases all resources\n- Sub-LM calls work via `llm_query()` and `rlm_query()`\n- Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `answer`, `SHOW_VARS`) are restored after each execution\n\n## Architecture: Environment ↔ LM Handler Communication\n\nUnderstanding how environments communicate with the LM Handler is essential for developing new environments.\n\n### Overview\n\n```\n┌─────────────────────────────────────────────────────────────────────┐\n│  Host Machine                                                       │\n│  ┌─────────────┐       Socket (TCP)        ┌──────────────────────┐ │\n│  │   RLM       │◄──────────────────────────►  LMHandler           │ │\n│  │  (main)     │                           │  (ThreadingTCPServer)│ │\n│  └─────────────┘                           └──────────────────────┘ │\n│        │                                            ▲               │\n│        ▼                                            │               │\n│  ┌─────────────┐       Socket (TCP)                 │               │\n│  │ LocalREPL   │────────────────────────────────────┘               │\n│  │ (exec code) │  llm_query() / rlm_query() → LM calls               │\n│  └─────────────┘                                                    │\n└─────────────────────────────────────────────────────────────────────┘\n```\n\n### Socket Protocol (Non-Isolated Environments)\n\nNon-isolated environments like `LocalREPL` communicate directly with the `LMHandler` via TCP sockets using a length-prefixed JSON protocol:\n\n**Protocol Format**: `4-byte big-endian length prefix + UTF-8 JSON payload`\n\n```python\n# Sending a message (from rlm/core/comms_utils.py)\ndef socket_send(sock: socket.socket, data: dict) -> None:\n    payload = json.dumps(data).encode(\"utf-8\")\n    sock.sendall(struct.pack(\">I\", len(payload)) + payload)\n```\n\n**Request Flow**:\n1. Environment's `llm_query(prompt)` or `rlm_query(prompt)` is called during code execution\n2. For `llm_query`: creates `LMRequest` and calls `send_lm_request(address, request)`. For `rlm_query`: invokes `subcall_fn` to spawn a child RLM (or falls back to `llm_query` at max depth).\n3. Opens TCP connection to `LMHandler` at `(host, port)`\n4. Sends length-prefixed JSON request\n5. `LMHandler` processes via `LMRequestHandler.handle()`\n6. Returns `LMResponse` with `RLMChatCompletion` or error\n\n**Key Components**:\n- `LMHandler` (`rlm/core/lm_handler.py`): Multi-threaded TCP server wrapping LM clients\n- `LMRequest` / `LMResponse` (`rlm/core/comms_utils.py`): Typed request/response dataclasses\n- `send_lm_request()` / `send_lm_request_batched()`: Helper functions for socket communication\n\n### HTTP Broker Pattern (Isolated Environments)\n\nIsolated environments (Modal, Prime) cannot directly connect to the host's socket server. They use an HTTP broker pattern:\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│  Host Machine                                                               │\n│  ┌─────────┐    Socket    ┌────────────┐    HTTP Poll    ┌────────────────┐ │\n│  │   RLM   │◄────────────►│  LMHandler │◄────────────────│   ModalREPL    │ │\n│  └─────────┘              └────────────┘                 │  (poller)      │ │\n│                                                          └────────────────┘ │\n│                                                                  │          │\n│                                                          HTTP (tunnel)      │\n│                                                                  │          │\n└──────────────────────────────────────────────────────────────────┼──────────┘\n                                                                   │\n┌──────────────────────────────────────────────────────────────────┼──────────┐\n│  Cloud Sandbox (Modal/Prime)                                     ▼          │\n│  ┌─────────────┐     HTTP (localhost)     ┌─────────────────────────────┐   │\n│  │ Exec Script │◄────────────────────────►│   Broker Server (Flask)     │   │\n│  │ (exec code) │     /enqueue, etc.       │   - /enqueue (submit req)   │   │\n│  └─────────────┘                          │   - /pending (poll reqs)    │   │\n│                                           │   - /respond (return resp)  │   │\n│                                           └─────────────────────────────┘   │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n**How It Works**:\n\n1. **Sandbox Setup**: Environment creates a cloud sandbox with an HTTP broker server running inside\n2. **Tunnel Exposure**: Broker server is exposed via encrypted tunnel (e.g., Modal's `encrypted_ports`)\n3. **Code Execution**: When `llm_query()` is called inside sandbox, it POSTs to `http://localhost:8080/enqueue`\n4. **Request Queuing**: Broker queues the request and blocks waiting for response\n5. **Host Polling**: `ModalREPL` on host polls `{tunnel_url}/pending` for new requests\n6. **LM Forwarding**: Host forwards requests to `LMHandler` via socket, gets response\n7. **Response Delivery**: Host POSTs response to `{tunnel_url}/respond`\n8. **Unblocking**: Broker unblocks the original `/enqueue` call with the response\n\n**Broker Endpoints**:\n| Endpoint | Method | Purpose |\n|----------|--------|---------|\n| `/enqueue` | POST | Submit LLM request from sandbox code (blocks until response) |\n| `/pending` | GET | Get list of pending requests (called by host poller) |\n| `/respond` | POST | Submit response for a request ID (called by host poller) |\n| `/health` | GET | Health check |\n\n**Key Implementation Details**:\n- Broker runs as a Flask server inside the sandbox\n- Uses `threading.Event` for request/response synchronization\n- Poller thread on host runs in background with 100ms polling interval\n- State persistence via `dill` serialization to `/tmp/rlm_state.dill`\n\n### Implementing a New Isolated Environment\n\nWhen building a new isolated environment (e.g., for a new cloud provider):\n\n1. **Create broker server** - Flask/HTTP server with `/enqueue`, `/pending`, `/respond` endpoints\n2. **Expose tunnel** - Use provider's tunnel/port forwarding to expose broker to host\n3. **Implement poller** - Background thread on host to poll and forward requests\n4. **Build exec script** - Script that runs inside sandbox with `llm_query()` calling broker\n5. **Handle state** - Serialize/deserialize execution state between code blocks\n\nSee `rlm/environments/modal_repl.py` as the canonical reference implementation.\n\n","isInternal":false,"tokens":3253,"sizeBytes":15638},{"name":"architecture.md","path":"docs/architecture.md","rawUrl":"https://raw.githubusercontent.com/alexzhang13/rlm/HEAD/docs/architecture.md","title":"docs - System Architecture & Module Boundaries","category":"project-spec","format":"markdown","content":"---\nlayout: default\ntitle: Architecture\nnav_order: 3\n---\n\n# Architecture\n{: .no_toc }\n\nHow the RLM runtime, LM handler, code execution, and recursive sub-calls fit together.\n{: .fs-6 .fw-300 }\n\n## Table of Contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n\n## Overview\n\nAn RLM completion involves three cooperating pieces:\n\n1. **RLM** (`rlm/core/rlm.py`) — the main loop that drives iteration.\n2. **LMHandler** (`rlm/core/lm_handler.py`) — a per-completion TCP server that routes LM API calls.\n3. **LocalREPL** (`rlm/environments/local_repl.py`) — the Python execution environment where model-generated code runs.\n\n```\n┌────────────────────────────────────────────────────────────────┐\n│  RLM.completion(prompt)                                        │\n│                                                                │\n│  1. Spawn LMHandler (TCP server on localhost, auto port)       │\n│  2. Create LocalREPL (in-process exec() namespace)             │\n│  3. Iterate:                                                   │\n│     a. Send message history → LM backend → get response        │\n│     b. Extract ```repl``` code blocks from response            │\n│     c. Execute code in LocalREPL                               │\n│     d. Append stdout/stderr to message history                 │\n│     e. Repeat until answer[\"ready\"] is True or limits exceeded │\n│  4. Tear down handler and environment                          │\n└────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## LM Handler\n\n### What it is\n\nThe LMHandler is a **multi-threaded TCP socket server** that sits between\nthe execution environment and the actual LM API backends. Every call to\n`RLM.completion()` spins up a fresh handler (unless the environment is\npersistent and already has one).\n\n### Why a socket server?\n\nThe handler exists so that code running inside the execution environment\ncan make LM calls back to the host process without directly importing or\ncalling the LM client. This is essential for isolated environments (Docker,\nModal) that run in separate processes or machines — they communicate with\nthe handler over TCP. The local environment uses the same protocol for\nconsistency, even though it runs in-process.\n\n### Lifecycle\n\n```python\n# Inside RLM._spawn_completion_context():\nclient = get_client(backend, backend_kwargs)            # 1. Create LM client\nlm_handler = LMHandler(client, other_backend_client=…)  # 2. Wrap in handler\nlm_handler.start()                                       # 3. Start TCP server (daemon thread)\n# … run completion loop …\nlm_handler.stop()                                        # 4. Shut down server\n```\n\n- The server binds to `127.0.0.1` with port `0` (OS auto-assigns an available port).\n- It runs in a **daemon thread** so it doesn't block process exit.\n- Each incoming connection is handled by a new thread (`ThreadingTCPServer`).\n\n### Wire protocol\n\nAll messages use a simple framing: **4-byte big-endian length prefix + UTF-8 JSON payload**.\n\n```\n┌──────────┬─────────────────────────┐\n│ 4 bytes  │  N bytes                │\n│ len (BE) │  JSON payload (UTF-8)   │\n└──────────┴─────────────────────────┘\n```\n\nImplemented in `socket_send()` / `socket_recv()` in `rlm/core/comms_utils.py`.\n\n### Client routing\n\nThe handler can hold multiple LM clients and routes requests based on the\n`model` and `depth` fields in the request:\n\n```python\ndef get_client(self, model=None, depth=0):\n    if model and model in self.clients:\n        return self.clients[model]        # Explicit model override\n    if depth == 1 and self.other_backend_client:\n        return self.other_backend_client  # Depth-based routing\n    return self.default_client            # Fallback\n```\n\nThis lets you use a different (e.g. cheaper/faster) model for sub-LM calls\nby specifying `other_backends` / `other_backend_kwargs` in the RLM constructor.\n\n---\n\n## Code Execution Environment (LocalREPL)\n\n### In-process `exec()` — not a subprocess\n\nLocalREPL executes model-generated code **in the same Python interpreter\nprocess** as the RLM, using Python's built-in `exec()`. There is no\nsubprocess, no fork, and no IPC for code execution.\n\n```python\n# Simplified from LocalREPL.execute_code():\ncombined = {**self.globals, **self.locals}\nexec(code, combined, combined)\n```\n\n### What this means in practice\n\n- **Fast**: No process spawn overhead. Code execution is as fast as native Python.\n- **Persistent namespace**: Variables created in one code block are visible in the next. The `self.locals` dict accumulates state across iterations.\n- **Shared memory**: Helper functions like `llm_query()` and `rlm_query()` are plain Python closures in `self.globals`. When model code calls `llm_query(\"...\")`, it's a direct function call within the same process.\n- **Limited sandbox**: Dangerous builtins (`eval`, `exec`, `compile`, `input`) are removed from the namespace. This is a soft sandbox — it prevents accidental misuse but is not a security boundary.\n\n### Namespace layout\n\n```\nglobals (shared across all executions):\n├── __builtins__        → _SAFE_BUILTINS (eval/exec/input removed)\n├── llm_query()         → plain LM call via handler\n├── llm_query_batched() → batched plain LM calls\n├── rlm_query()         → recursive RLM sub-call (or fallback to llm_query)\n├── rlm_query_batched() → batched recursive sub-calls\n├── SHOW_VARS()         → list user-created variables\n└── <custom_tools>      → user-provided callable tools\n\nlocals (accumulates user variables):\n├── context             → alias for context_0\n├── context_0           → first context payload\n├── context_1, …        → additional contexts (persistent mode)\n├── history             → conversation history (persistent/compaction mode)\n├── answer              → {\"content\": \"\", \"ready\": False}; set ready=True to finish\n└── <user variables>    → anything created by model code\n```\n\n### Scaffold restoration\n\nAfter each `exec()`, LocalREPL restores all reserved names to prevent model\ncode from corrupting the environment. If the model writes\n`llm_query = \"oops\"` or `context = None`, the next execution will still\nhave the real functions and data. See `_restore_scaffold()`.\n\n---\n\n## How `llm_query()` and `rlm_query()` Work\n\nThese are the two functions available to model-generated code for making LM calls.\nThey have very different behaviors:\n\n### `llm_query(prompt, model=None)` — Plain LM call\n\nAlways makes a single, direct LM completion. No REPL, no iteration — just\nprompt in, text out. Fast and lightweight.\n\n```\nModel code: answer = llm_query(\"Summarize this text: ...\")\n    │\n    ▼\nLocalREPL._llm_query()\n    │  Creates LMRequest(prompt=..., depth=self.depth)\n    │  Opens TCP socket to handler\n    ▼\nLMHandler (TCP server)\n    │  get_client(model, depth) → selects backend\n    │  client.completion(prompt) → calls LM API\n    ▼\nResponse flows back over socket\n    │\n    ▼\nReturns response string to model code\n```\n\n### `rlm_query(prompt, model=None)` — Recursive RLM sub-call\n\nSpawns a **child RLM** that gets its own REPL and can reason iteratively\nover the prompt — just like the parent. Use this when the subtask needs\nmulti-step reasoning, code execution, or its own iterative problem-solving.\n\nFalls back to `llm_query` when recursion is not available (i.e. the current\ndepth has reached `max_depth`).\n\n```\nModel code: answer = rlm_query(\"Solve this complex problem: ...\")\n    │\n    ▼\nLocalREPL._rlm_query()\n    │  if self.subcall_fn is not None:    ← set when max_depth > 1\n    │      calls self.subcall_fn(prompt, model)\n    │  else:\n    │      falls back to _llm_query()\n    │\n    ▼  (when subcall_fn exists)\nRLM._subcall(prompt, model)\n    │  next_depth = self.depth + 1\n    │  if next_depth >= max_depth:\n    │      → plain client.completion() (leaf call, no REPL)\n    │  else:\n    │      → create child RLM(depth=next_depth, ...)\n    │      → child.completion(prompt)  ← full RLM with its own handler + REPL\n    │\n    ▼\nReturns RLMChatCompletion to parent\n```\n\n### `llm_query_batched` / `rlm_query_batched`\n\nSame semantics as above, but for multiple prompts. `llm_query_batched` sends\nall prompts as a single batched request to the handler, which processes them\nconcurrently with `asyncio.gather`. `rlm_query_batched` calls `subcall_fn`\nsequentially for each prompt (each child RLM is a blocking call).\n\n**Failure handling is per-prompt.** A single failed call does not fail the\nwhole batch — that slot returns the string `\"Error: llm() call failed - <msg>\"`\n(with the underlying error message) while the other prompts return their real\nresponses. Results stay aligned with the input prompts by index, so the list is\nalways the same length as `prompts`.\n\n---\n\n## Recursive Sub-Calls (Depth > 1)\n\n### How depth works\n\n```\nmax_depth=3\n\nRLM (depth=0)\n └─ rlm_query() → child RLM (depth=1)\n     └─ rlm_query() → child RLM (depth=2)\n         └─ rlm_query() → plain LM call (depth=3 >= max_depth, no REPL)\n```\n\n- `depth=0` is the root RLM that the user calls.\n- Each child increments depth by 1.\n- When `next_depth >= max_depth`, `_subcall()` does a plain `client.completion()` instead of creating a child RLM. This is the leaf case — no REPL, no iteration.\n- `llm_query()` always does a plain LM call regardless of depth. Only `rlm_query()` triggers recursion.\n\n### Each child gets its own handler and environment\n\nWhen a child RLM is created via `_subcall()`, its `completion()` method calls\n`_spawn_completion_context()` which creates:\n\n1. **A new `LMHandler`** listening on a **different auto-assigned port**.\n2. **A new `LocalREPL`** with its own isolated namespace.\n\n```\nParent RLM (depth=0)\n├── LMHandler #1 on port 52301\n├── LocalREPL #1 (depth=1)\n│   ├── globals: {llm_query, rlm_query, ...}\n│   ├── locals: {context: \"parent prompt\", ...}\n│   └── subcall_fn = RLM._subcall  ← enables rlm_query()\n│\n└── When model code calls rlm_query(\"subtask\"):\n    │\n    └── Child RLM (depth=1)\n        ├── LMHandler #2 on port 52302  ← NEW handler, NEW port\n        ├── LocalREPL #2 (depth=2)      ← NEW namespace\n        │   ├── locals: {context: \"subtask\", ...}\n        │   └── subcall_fn = child._subcall (or None if at max_depth-1)\n        │\n        └── Runs its own iteration loop, returns RLMChatCompletion\n```\n\nThe child's handler and environment are torn down when the child's `completion()` finishes.\n\n### Resource limits propagate\n\nThe parent passes **remaining** budget/timeout/tokens to the child, not the\noriginal totals. This prevents a child from consuming all of the parent's resources:\n\n```python\n# In _subcall():\nremaining_timeout = self.max_timeout - elapsed  # not self.max_timeout\nremaining_budget = self.max_budget - spent       # not self.max_budget\nchild = RLM(..., max_timeout=remaining_timeout, max_budget=remaining_budget)\n```\n\n### Metadata flows back\n\nEach child RLM can have its own `RLMLogger`. When the child completes, its\nfull trajectory metadata (iterations, code blocks, sub-calls) is captured in\nthe returned `RLMChatCompletion.metadata` dict. The parent's logger records\nthis as part of the REPL result's `rlm_calls` list, creating a nested\nmetadata tree.\n\n---\n\n## Putting It All Together\n\nHere's the complete request flow for a depth-2 RLM call:\n\n```\nUser: rlm.completion(\"Analyze this data\")\n │\n ▼\nRLM (depth=0)\n ├─ _spawn_completion_context()\n │   ├─ LMHandler #1 starts on port 52301\n │   └─ LocalREPL #1 created with context=\"Analyze this data\"\n │\n ├─ Iteration 1: LM generates code\n │   │  ```repl\n │   │  answer = rlm_query(\"What patterns exist in: \" + context[:5000])\n │   │  ```\n │   │\n │   └─ LocalREPL.execute_code() runs the code via exec()\n │       │\n │       ├─ rlm_query() → _rlm_query() → subcall_fn()\n │       │   │\n │       │   └─ RLM._subcall(\"What patterns exist in: ...\")\n │       │       │\n │       │       ├─ depth=1 < max_depth=2, so create child RLM\n │       │       │\n │       │       └─ Child RLM (depth=1)\n │       │           ├─ LMHandler #2 on port 52302\n │       │           ├─ LocalREPL #2 with context=\"What patterns...\"\n │       │           │\n │       │           ├─ Child iteration 1: LM generates code\n │       │           │   │  result = llm_query(\"Extract key metrics: \" + context)\n │       │           │   │\n │       │           │   └─ llm_query() → TCP to Handler #2 → LM API → response\n │       │           │\n │       │           ├─ Child iteration 2: LM sets answer[\"content\"]=result, answer[\"ready\"]=True\n │       │           │\n │       │           └─ Returns RLMChatCompletion to parent\n │       │\n │       └─ child_response = child_completion.response\n │\n ├─ Iteration 2: LM uses child_response, sets answer[\"content\"]=final, answer[\"ready\"]=True\n │\n ├─ LMHandler #1 stops\n └─ Returns RLMChatCompletion to user\n```\n\n### Key takeaways\n\n| Aspect | Detail |\n|:-------|:-------|\n| Code execution | In-process `exec()` in the same Python interpreter. Not a subprocess. |\n| LM calls from code | Go through a local TCP socket server (LMHandler), even for in-process execution. |\n| Handler per completion | Each `completion()` call gets its own handler on an auto-assigned port. |\n| Child RLMs | Created by `_subcall()`, each with its own handler + LocalREPL. Fully independent. |\n| `llm_query` vs `rlm_query` | `llm_query` = always plain LM call. `rlm_query` = recursive child RLM (or fallback). |\n| Depth limit | At `max_depth`, `rlm_query` falls back to `llm_query`. No further recursion. |\n| Resource isolation | Children get remaining budget/timeout, not the full amount. |\n| Namespace isolation | Each LocalREPL has its own `globals`/`locals`. No shared state between parent and child. |\n","frontmatter":{"layout":"default","title":"Architecture","nav_order":3},"isInternal":false,"tokens":3490,"sizeBytes":14563}],"systemPromptSnippet":"<yakaai_skills repo=\"alexzhang13/rlm\">\n<!-- File: AGENTS.md (Tokens: ~3253 | Category: root-instruction) -->\n# AGENTS.md\n\nThis guide covers best practices for contributing to the core Recursive Language Models `rlm` library and developing new environments (in `rlm/environments/`) and LM clients (in `rlm/clients/`).\n\n## Setup\n\nWe use `uv` for developing `rlm`.\n```bash\n# Install uv (first time)\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Setup blank project if needed\nuv init && uv venv --python 3.12\nsource .venv/bin/activate\n\n# Install in editable mode\nuv pip install -e .\n\n# For Modal sandbox support\nuv pip install -e \".[modal]\"\n\n# For Prime sandbox support\nuv pip install -e \".[prime]\"\n```\n\n## General Guidelines\n\n### Code Style & Typing\n- **Formatting**: Strict `ruff` enforcement. All PRs must pass `ruff check --fix .`\n- **Typing**: Explicit types preferred\n  - **OK**: `cast(...)`, `assert ...` for type narrowing\n  - **SOMETIMES OK**: Untyped args for simple cases (e.g., prompt handlers)\n  - **NOT OK**: `# type: ignore` without strong justification\n\n### Naming Conventions\n- **Methods**: snake_case\n- **Classes**: PascalCase (e.g., `LocalREPL`, `PortkeyClient`)\n- **Variables**: snake_case\n- **Constants**: UPPER_CASE (e.g., `_SAFE_BUILTINS`, `RLM_SYSTEM_PROMPT`)\n\nDo NOT use `_` prefix for private methods unless explicitly requested.\n\n### Error Handling Philosophy\n- **Fail fast, fail loud** - No defensive programming or silent fallbacks\n- **Minimize branching** - Prefer single code paths; every `if`/`try` needs justification\n- **Example**: Missing API key → immediate `ValueError`, not graceful fallback\n\n## Core Repository Development\n\nFor PRs to `rlm` core:\n```bash\ngit clone https://github.com/alexzhang13/rlm.git\ncd rlm\n\n# Standard development:\nuv sync\n\n# Install dev + test dependencies:\nuv sync --group dev --group test\n\n# Install pre-commit hooks:\nuv run pre-commit install\n```\n\n### Dependencies\n- Avoid new core dependencies\n- Use optional extras for non-essential features (e.g., `modal` extra)\n- Exception: tiny deps that simplify widely-used code\n\n### Testing\n- `uv run pytest` with discovery under `tests/`\n- Write simple, deterministic unit tests\n- Update tests when changing functionality\n- For isolated environments, mock external services\n\n### Documentation\n- Keep concise and actionable\n- Update README when behavior changes\n- Avoid content duplication\n\n### Scope\n- Small, focused diffs\n- One change per PR\n- Backward compatibility is only desirable if it can be done without introducing excessive maintenance burden\n- Delete dead code (don't guard it)\n\n### Checklist\n\nBefore a PR:\n\n```bash\n# Run style + lint checks:\nuv run ruff check --fix .\nuv run ruff format .\nuv run pre-commit run --all-files\n\n# Run tests:\nuv run pytest\n```\n\nEnsure docs and tests are updated if necessary, and dead code is deleted. Strive for minimal, surgical diffs.\n\n## Developing LM Clients\n\nLM client implementations live in `rlm/clients/`. All clients must inherit from `BaseLM`.\n\n### Client Pattern\n\n| Base Class | When to Use | Key Methods |\n|------------|-------------|-------------|\n| `BaseLM` | All LM integrations | `completion`, `acompletion`, `get_usage_summary`, `get_last_usage` |\n\n### Requirements\n- Inherit from `BaseLM` in `rlm/clients/base_lm.py`\n- Implement all abstract methods: `completion`, `acompletion`, `get_usage_summary`, `get_last_usage`\n- Track per-model usage (calls, input/output tokens)\n- Handle both string and message list prompts\n- Register client in `rlm/clients/__init__.py`\n\n### Example Structure\n```python\nfrom rlm.clients.base_lm import BaseLM\nfrom rlm.core.types import ModelUsageSummary, UsageSummary\n\nclass MyClient(BaseLM):\n    def __init__(self, api_key: str, model_name: str, **kwargs):\n        super().__init__(model_name=model_name, **kwargs)\n        # Initialize your client\n        \n    def completion(self, prompt: str | list[dict[str, Any]], model: str | None = None) -> str:\n        # Handle both str and message list formats\n        # Track usage with _track_cost()\n        # Return response string\n        \n    def get_usage_summary(self) -> UsageSummary:\n        # Return aggregated usage across all calls\n```\n\n### Configuration Guidelines\n- **Environment variables**: ONLY for API keys (document in README)\n- **Hardcode**: Default base URLs, reasonable defaults\n- **Arguments**: Essential customization via `__init__()`\n\n## Developing Environments\n\nEnvironment implementations live in `rlm/environments/`. Choose the appropriate base class.\n\n### Environment Pattern\n\n| Pattern | Base Class | When to Use | Key Methods |\n|---------|------------|-------------|-------------|\n| **Non-isolated** | `NonIsolatedEnv` | Local execution, same machine | `setup`, `load_context`, `execute_code` |\n| **Isolated** | `IsolatedEnv` | Cloud sandboxes (Modal, Prime) | `setup`, `load_context`, `execute_code` |\n\n### Requirements\n- Inherit from `NonIsolatedEnv` or `IsolatedEnv` in `rlm/environments/base_env.py`\n- Implement all abstract methods: `setup`, `load_context`, `execute_code`\n- Return `REPLResult` from `execute_code`\n- Handle `lm_handler_address` for LM calls via `llm_query()` and `rlm_query()`\n- Implement `cleanup()` for resource management\n- Register environment in `rlm/environments/__init__.py`\n\n### Key Implementation Details\n- `setup()`: Initialize globals, locals, and helper functions\n- `load_context()`: Make context available as `context` variable\n- `execute_code()`: Execute code, capture stdout/stderr, return `REPLResult`\n- Always provide `llm_query`, `llm_query_batched`, `rlm_query`, and `rlm_query_batched` functions in environment globals\n\n### State Management\nEnvironments must provide these globals to executed code:\n- `context`: The loaded context payload\n- `llm_query(prompt, model=None)`: Plain single LM completion (no REPL, no iteration)\n- `llm_query_batched(prompts, model=None)`: Batched plain LM completions\n- `rlm_query(prompt, model=None)`: Recursive child RLM call (own REPL + iteration). Falls back to `llm_query` at max depth.\n- `rlm_query_batched(prompts, model=None)`: Batched recursive child RLM calls\n- `answer`: A dict (`{\"content\": \"\", \"ready\": False}`); the model sets `answer[\"content\"]` and `answer[\"ready\"] = True` to return a final answer. The env surfaces the content on `REPLResult.final_answer` once `ready` flips truthy.\n- `SHOW_VARS()`: For listing available variables\n\n### Example Structure\n```python\nfrom rlm.environments.base_env import NonIsolatedEnv\nfrom rlm.core.types import REPLResult\n\nclass MyEnvironment(NonIsolatedEnv):\n    def __init__(self, lm_handler_address: tuple[str, int] | None = None, \n                 context_payload: dict | list | str | None = None, **kwargs):\n        super().__init__(**kwargs)\n        self.lm_handler_address = lm_handler_address\n        self.setup()\n        if context_payload:\n            self.load_context(context_payload)\n            \n    def setup(self):\n        # Initialize execution namespace\n        \n    def load_context(self, context_payload: dict | list | str):\n        # Make context available to executed code\n        \n    def execute_code(self, code: str) -> REPLResult:\n        # Execute code and return REPLResult\n        \n    def cleanup(self):\n        # Clean up resources\n```\n\n### Checklist\n- Guidelines here are followed\n- Environment works with basic RLM completion calls\n- `cleanup()` properly releases all resources\n- Sub-LM calls work via `llm_query()` and `rlm_query()`\n- Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `answer`, `SHOW_VARS`) are restored after each execution\n\n## Architecture: Environment ↔ LM Handler Communication\n\nUnderstanding how environments communicate with the LM Handler is essential for developing new environments.\n\n### Overview\n\n```\n┌─────────────────────────────────────────────────────────────────────┐\n│  Host Machine                                                       │\n│  ┌─────────────┐       Socket (TCP)        ┌──────────────────────┐ │\n│  │   RLM       │◄──────────────────────────►  LMHandler           │ │\n│  │  (main)     │                           │  (ThreadingTCPServer)│ │\n│  └─────────────┘                           └──────────────────────┘ │\n│        │                                            ▲               │\n│        ▼                                            │               │\n│  ┌─────────────┐       Socket (TCP)                 │               │\n│  │ LocalREPL   │────────────────────────────────────┘               │\n│  │ (exec code) │  llm_query() / rlm_query() → LM calls               │\n│  └─────────────┘                                                    │\n└─────────────────────────────────────────────────────────────────────┘\n```\n\n### Socket Protocol (Non-Isolated Environments)\n\nNon-isolated environments like `LocalREPL` communicate directly with the `LMHandler` via TCP sockets using a length-prefixed JSON protocol:\n\n**Protocol Format**: `4-byte big-endian length prefix + UTF-8 JSON payload`\n\n```python\n# Sending a message (from rlm/core/comms_utils.py)\ndef socket_send(sock: socket.socket, data: dict) -> None:\n    payload = json.dumps(data).encode(\"utf-8\")\n    sock.sendall(struct.pack(\">I\", len(payload)) + payload)\n```\n\n**Request Flow**:\n1. Environment's `llm_query(prompt)` or `rlm_query(prompt)` is called during code execution\n2. For `llm_query`: creates `LMRequest` and calls `send_lm_request(address, request)`. For `rlm_query`: invokes `subcall_fn` to spawn a child RLM (or falls back to `llm_query` at max depth).\n3. Opens TCP connection to `LMHandler` at `(host, port)`\n4. Sends length-prefixed JSON request\n5. `LMHandler` processes via `LMRequestHandler.handle()`\n6. Returns `LMResponse` with `RLMChatCompletion` or error\n\n**Key Components**:\n- `LMHandler` (`rlm/core/lm_handler.py`): Multi-threaded TCP server wrapping LM clients\n- `LMRequest` / `LMResponse` (`rlm/core/comms_utils.py`): Typed request/response dataclasses\n- `send_lm_request()` / `send_lm_request_batched()`: Helper functions for socket communication\n\n### HTTP Broker Pattern (Isolated Environments)\n\nIsolated environments (Modal, Prime) cannot directly connect to the host's socket server. They use an HTTP broker pattern:\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│  Host Machine                                                               │\n│  ┌─────────┐    Socket    ┌────────────┐    HTTP Poll    ┌────────────────┐ │\n│  │   RLM   │◄────────────►│  LMHandler │◄────────────────│   ModalREPL    │ │\n│  └─────────┘              └────────────┘                 │  (poller)      │ │\n│                                                          └────────────────┘ │\n│                                                                  │          │\n│                                                          HTTP (tunnel)      │\n│                                                                  │          │\n└──────────────────────────────────────────────────────────────────┼──────────┘\n                                                                   │\n┌──────────────────────────────────────────────────────────────────┼──────────┐\n│  Cloud Sandbox (Modal/Prime)                                     ▼          │\n│  ┌─────────────┐     HTTP (localhost)     ┌─────────────────────────────┐   │\n│  │ Exec Script │◄────────────────────────►│   Broker Server (Flask)     │   │\n│  │ (exec code) │     /enqueue, etc.       │   - /enqueue (submit req)   │   │\n│  └─────────────┘                          │   - /pending (poll reqs)    │   │\n│                                           │   - /respond (return resp)  │   │\n│                                           └─────────────────────────────┘   │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n**How It Works**:\n\n1. **Sandbox Setup**: Environment creates a cloud sandbox with an HTTP broker server running inside\n2. **Tunnel Exposure**: Broker server is exposed via encrypted tunnel (e.g., Modal's `encrypted_ports`)\n3. **Code Execution**: When `llm_query()` is called inside sandbox, it POSTs to `http://localhost:8080/enqueue`\n4. **Request Queuing**: Broker queues the request and blocks waiting for response\n5. **Host Polling**: `ModalREPL` on host polls `{tunnel_url}/pending` for new requests\n6. **LM Forwarding**: Host forwards requests to `LMHandler` via socket, gets response\n7. **Response Delivery**: Host POSTs response to `{tunnel_url}/respond`\n8. **Unblocking**: Broker unblocks the original `/enqueue` call with the response\n\n**Broker Endpoints**:\n| Endpoint | Method | Purpose |\n|----------|--------|---------|\n| `/enqueue` | POST | Submit LLM request from sandbox code (blocks until response) |\n| `/pending` | GET | Get list of pending requests (called by host poller) |\n| `/respond` | POST | Submit response for a request ID (called by host poller) |\n| `/health` | GET | Health check |\n\n**Key Implementation Details**:\n- Broker runs as a Flask server inside the sandbox\n- Uses `threading.Event` for request/response synchronization\n- Poller thread on host runs in background with 100ms polling interval\n- State persistence via `dill` serialization to `/tmp/rlm_state.dill`\n\n### Implementing a New Isolated Environment\n\nWhen building a new isolated environment (e.g., for a new cloud provider):\n\n1. **Create broker server** - Flask/HTTP server with `/enqueue`, `/pending`, `/respond` endpoints\n2. **Expose tunnel** - Use provider's tunnel/port forwarding to expose broker to host\n3. **Implement poller** - Background thread on host to poll and forward requests\n4. **Build exec script** - Script that runs inside sandbox with `llm_query()` calling broker\n5. **Handle state** - Serialize/deserialize execution state between code blocks\n\nSee `rlm/environments/modal_repl.py` as the canonical reference implementation.\n\n<!-- File: docs/architecture.md (Tokens: ~3490 | Category: project-spec) -->\n---\nlayout: default\ntitle: Architecture\nnav_order: 3\n---\n\n# Architecture\n{: .no_toc }\n\nHow the RLM runtime, LM handler, code execution, and recursive sub-calls fit together.\n{: .fs-6 .fw-300 }\n\n## Table of Contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n\n## Overview\n\nAn RLM completion involves three cooperating pieces:\n\n1. **RLM** (`rlm/core/rlm.py`) — the main loop that drives iteration.\n2. **LMHandler** (`rlm/core/lm_handler.py`) — a per-completion TCP server that routes LM API calls.\n3. **LocalREPL** (`rlm/environments/local_repl.py`) — the Python execution environment where model-generated code runs.\n\n```\n┌────────────────────────────────────────────────────────────────┐\n│  RLM.completion(prompt)                                        │\n│                                                                │\n│  1. Spawn LMHandler (TCP server on localhost, auto port)       │\n│  2. Create LocalREPL (in-process exec() namespace)             │\n│  3. Iterate:                                                   │\n│     a. Send message history → LM backend → get response        │\n│     b. Extract ```repl``` code blocks from response            │\n│     c. Execute code in LocalREPL                               │\n│     d. Append stdout/stderr to message history                 │\n│     e. Repeat until answer[\"ready\"] is True or limits exceeded │\n│  4. Tear down handler and environment                          │\n└────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## LM Handler\n\n### What it is\n\nThe LMHandler is a **multi-threaded TCP socket server** that sits between\nthe execution environment and the actual LM API backends. Every call to\n`RLM.completion()` spins up a fresh handler (unless the environment is\npersistent and already has one).\n\n### Why a socket server?\n\nThe handler exists so that code running inside the execution environment\ncan make LM calls back to the host process without directly importing or\ncalling the LM client. This is essential for isolated environments (Docker,\nModal) that run in separate processes or machines — they communicate with\nthe handler over TCP. The local environment uses the same protocol for\nconsistency, even though it runs in-process.\n\n### Lifecycle\n\n```python\n# Inside RLM._spawn_completion_context():\nclient = get_client(backend, backend_kwargs)            # 1. Create LM client\nlm_handler = LMHandler(client, other_backend_client=…)  # 2. Wrap in handler\nlm_handler.start()                                       # 3. Start TCP server (daemon thread)\n# … run completion loop …\nlm_handler.stop()                                        # 4. Shut down server\n```\n\n- The server binds to `127.0.0.1` with port `0` (OS auto-assigns an available port).\n- It runs in a **daemon thread** so it doesn't block process exit.\n- Each incoming connection is handled by a new thread (`ThreadingTCPServer`).\n\n### Wire protocol\n\nAll messages use a simple framing: **4-byte big-endian length prefix + UTF-8 JSON payload**.\n\n```\n┌──────────┬─────────────────────────┐\n│ 4 bytes  │  N bytes                │\n│ len (BE) │  JSON payload (UTF-8)   │\n└──────────┴─────────────────────────┘\n```\n\nImplemented in `socket_send()` / `socket_recv()` in `rlm/core/comms_utils.py`.\n\n### Client routing\n\nThe handler can hold multiple LM clients and routes requests based on the\n`model` and `depth` fields in the request:\n\n```python\ndef get_client(self, model=None, depth=0):\n    if model and model in self.clients:\n        return self.clients[model]        # Explicit model override\n    if depth == 1 and self.other_backend_client:\n        return self.other_backend_client  # Depth-based routing\n    return self.default_client            # Fallback\n```\n\nThis lets you use a different (e.g. cheaper/faster) model for sub-LM calls\nby specifying `other_backends` / `other_backend_kwargs` in the RLM constructor.\n\n---\n\n## Code Execution Environment (LocalREPL)\n\n### In-process `exec()` — not a subprocess\n\nLocalREPL executes model-generated code **in the same Python interpreter\nprocess** as the RLM, using Python's built-in `exec()`. There is no\nsubprocess, no fork, and no IPC for code execution.\n\n```python\n# Simplified from LocalREPL.execute_code():\ncombined = {**self.globals, **self.locals}\nexec(code, combined, combined)\n```\n\n### What this means in practice\n\n- **Fast**: No process spawn overhead. Code execution is as fast as native Python.\n- **Persistent namespace**: Variables created in one code block are visible in the next. The `self.locals` dict accumulates state across iterations.\n- **Shared memory**: Helper functions like `llm_query()` and `rlm_query()` are plain Python closures in `self.globals`. When model code calls `llm_query(\"...\")`, it's a direct function call within the same process.\n- **Limited sandbox**: Dangerous builtins (`eval`, `exec`, `compile`, `input`) are removed from the namespace. This is a soft sandbox — it prevents accidental misuse but is not a security boundary.\n\n### Namespace layout\n\n```\nglobals (shared across all executions):\n├── __builtins__        → _SAFE_BUILTINS (eval/exec/input removed)\n├── llm_query()         → plain LM call via handler\n├── llm_query_batched() → batched plain LM calls\n├── rlm_query()         → recursive RLM sub-call (or fallback to llm_query)\n├── rlm_query_batched() → batched recursive sub-calls\n├── SHOW_VARS()         → list user-created variables\n└── <custom_tools>      → user-provided callable tools\n\nlocals (accumulates user variables):\n├── context             → alias for context_0\n├── context_0           → first context payload\n├── context_1, …        → additional contexts (persistent mode)\n├── history             → conversation history (persistent/compaction mode)\n├── answer              → {\"content\": \"\", \"ready\": False}; set ready=True to finish\n└── <user variables>    → anything created by model code\n```\n\n### Scaffold restoration\n\nAfter each `exec()`, LocalREPL restores all reserved names to prevent model\ncode from corrupting the environment. If the model writes\n`llm_query = \"oops\"` or `context = None`, the next execution will still\nhave the real functions and data. See `_restore_scaffold()`.\n\n---\n\n## How `llm_query()` and `rlm_query()` Work\n\nThese are the two functions available to model-generated code for making LM calls.\nThey have very different behaviors:\n\n### `llm_query(prompt, model=None)` — Plain LM call\n\nAlways makes a single, direct LM completion. No REPL, no iteration — just\nprompt in, text out. Fast and lightweight.\n\n```\nModel code: answer = llm_query(\"Summarize this text: ...\")\n    │\n    ▼\nLocalREPL._llm_query()\n    │  Creates LMRequest(prompt=..., depth=self.depth)\n    │  Opens TCP socket to handler\n    ▼\nLMHandler (TCP server)\n    │  get_client(model, depth) → selects backend\n    │  client.completion(prompt) → calls LM API\n    ▼\nResponse flows back over socket\n    │\n    ▼\nReturns response string to model code\n```\n\n### `rlm_query(prompt, model=None)` — Recursive RLM sub-call\n\nSpawns a **child RLM** that gets its own REPL and can reason iteratively\nover the prompt — just like the parent. Use this when the subtask needs\nmulti-step reasoning, code execution, or its own iterative problem-solving.\n\nFalls back to `llm_query` when recursion is not available (i.e. the current\ndepth has reached `max_depth`).\n\n```\nModel code: answer = rlm_query(\"Solve this complex problem: ...\")\n    │\n    ▼\nLocalREPL._rlm_query()\n    │  if self.subcall_fn is not None:    ← set when max_depth > 1\n    │      calls self.subcall_fn(prompt, model)\n    │  else:\n    │      falls back to _llm_query()\n    │\n    ▼  (when subcall_fn exists)\nRLM._subcall(prompt, model)\n    │  next_depth = self.depth + 1\n    │  if next_depth >= max_depth:\n    │      → plain client.completion() (leaf call, no REPL)\n    │  else:\n    │      → create child RLM(depth=next_depth, ...)\n    │      → child.completion(prompt)  ← full RLM with its own handler + REPL\n    │\n    ▼\nReturns RLMChatCompletion to parent\n```\n\n### `llm_query_batched` / `rlm_query_batched`\n\nSame semantics as above, but for multiple prompts. `llm_query_batched` sends\nall prompts as a single batched request to the handler, which processes them\nconcurrently with `asyncio.gather`. `rlm_query_batched` calls `subcall_fn`\nsequentially for each prompt (each child RLM is a blocking call).\n\n**Failure handling is per-prompt.** A single failed call does not fail the\nwhole batch — that slot returns the string `\"Error: llm() call failed - <msg>\"`\n(with the underlying error message) while the other prompts return their real\nresponses. Results stay aligned with the input prompts by index, so the list is\nalways the same length as `prompts`.\n\n---\n\n## Recursive Sub-Calls (Depth > 1)\n\n### How depth works\n\n```\nmax_depth=3\n\nRLM (depth=0)\n └─ rlm_query() → child RLM (depth=1)\n     └─ rlm_query() → child RLM (depth=2)\n         └─ rlm_query() → plain LM call (depth=3 >= max_depth, no REPL)\n```\n\n- `depth=0` is the root RLM that the user calls.\n- Each child increments depth by 1.\n- When `next_depth >= max_depth`, `_subcall()` does a plain `client.completion()` instead of creating a child RLM. This is the leaf case — no REPL, no iteration.\n- `llm_query()` always does a plain LM call regardless of depth. Only `rlm_query()` triggers recursion.\n\n### Each child gets its own handler and environment\n\nWhen a child RLM is created via `_subcall()`, its `completion()` method calls\n`_spawn_completion_context()` which creates:\n\n1. **A new `LMHandler`** listening on a **different auto-assigned port**.\n2. **A new `LocalREPL`** with its own isolated namespace.\n\n```\nParent RLM (depth=0)\n├── LMHandler #1 on port 52301\n├── LocalREPL #1 (depth=1)\n│   ├── globals: {llm_query, rlm_query, ...}\n│   ├── locals: {context: \"parent prompt\", ...}\n│   └── subcall_fn = RLM._subcall  ← enables rlm_query()\n│\n└── When model code calls rlm_query(\"subtask\"):\n    │\n    └── Child RLM (depth=1)\n        ├── LMHandler #2 on port 52302  ← NEW handler, NEW port\n        ├── LocalREPL #2 (depth=2)      ← NEW namespace\n        │   ├── locals: {context: \"subtask\", ...}\n        │   └── subcall_fn = child._subcall (or None if at max_depth-1)\n        │\n        └── Runs its own iteration loop, returns RLMChatCompletion\n```\n\nThe child's handler and environment are torn down when the child's `completion()` finishes.\n\n### Resource limits propagate\n\nThe parent passes **remaining** budget/timeout/tokens to the child, not the\noriginal totals. This prevents a child from consuming all of the parent's resources:\n\n```python\n# In _subcall():\nremaining_timeout = self.max_timeout - elapsed  # not self.max_timeout\nremaining_budget = self.max_budget - spent       # not self.max_budget\nchild = RLM(..., max_timeout=remaining_timeout, max_budget=remaining_budget)\n```\n\n### Metadata flows back\n\nEach child RLM can have its own `RLMLogger`. When the child completes, its\nfull trajectory metadata (iterations, code blocks, sub-calls) is captured in\nthe returned `RLMChatCompletion.metadata` dict. The parent's logger records\nthis as part of the REPL result's `rlm_calls` list, creating a nested\nmetadata tree.\n\n---\n\n## Putting It All Together\n\nHere's the complete request flow for a depth-2 RLM call:\n\n```\nUser: rlm.completion(\"Analyze this data\")\n │\n ▼\nRLM (depth=0)\n ├─ _spawn_completion_context()\n │   ├─ LMHandler #1 starts on port 52301\n │   └─ LocalREPL #1 created with context=\"Analyze this data\"\n │\n ├─ Iteration 1: LM generates code\n │   │  ```repl\n │   │  answer = rlm_query(\"What patterns exist in: \" + context[:5000])\n │   │  ```\n │   │\n │   └─ LocalREPL.execute_code() runs the code via exec()\n │       │\n │       ├─ rlm_query() → _rlm_query() → subcall_fn()\n │       │   │\n │       │   └─ RLM._subcall(\"What patterns exist in: ...\")\n │       │       │\n │       │       ├─ depth=1 < max_depth=2, so create child RLM\n │       │       │\n │       │       └─ Child RLM (depth=1)\n │       │           ├─ LMHandler #2 on port 52302\n │       │           ├─ LocalREPL #2 with context=\"What patterns...\"\n │       │           │\n │       │           ├─ Child iteration 1: LM generates code\n │       │           │   │  result = llm_query(\"Extract key metrics: \" + context)\n │       │           │   │\n │       │           │   └─ llm_query() → TCP to Handler #2 → LM API → response\n │       │           │\n │       │           ├─ Child iteration 2: LM sets answer[\"content\"]=result, answer[\"ready\"]=True\n │       │           │\n │       │           └─ Returns RLMChatCompletion to parent\n │       │\n │       └─ child_response = child_completion.response\n │\n ├─ Iteration 2: LM uses child_response, sets answer[\"content\"]=final, answer[\"ready\"]=True\n │\n ├─ LMHandler #1 stops\n └─ Returns RLMChatCompletion to user\n```\n\n### Key takeaways\n\n| Aspect | Detail |\n|:-------|:-------|\n| Code execution | In-process `exec()` in the same Python interpreter. Not a subprocess. |\n| LM calls from code | Go through a local TCP socket server (LMHandler), even for in-process execution. |\n| Handler per completion | Each `completion()` call gets its own handler on an auto-assigned port. |\n| Child RLMs | Created by `_subcall()`, each with its own handler + LocalREPL. Fully independent. |\n| `llm_query` vs `rlm_query` | `llm_query` = always plain LM call. `rlm_query` = recursive child RLM (or fallback). |\n| Depth limit | At `max_depth`, `rlm_query` falls back to `llm_query`. No further recursion. |\n| Resource isolation | Children get remaining budget/timeout, not the full amount. |\n| Namespace isolation | Each LocalREPL has its own `globals`/`locals`. No shared state between parent and child. |\n</yakaai_skills>"}