# Technical Documentation: alibaba/OpenSandbox > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/alibaba/OpenSandbox) ### Overview (README.md)
OpenSandbox logo

OpenSandbox

Stars OpenSSF Best Practices CNCF Landscape Discord DingTalk E2E Status Kubernetes nightly build status

opensandbox-group%2FOpenSandbox | Trendshift

**Run AI agents in sandboxes on your own infrastructure.** OpenSandbox gives AI applications isolated environments to execute code, run commands, manage files, and operate browsers or desktops. Start locally with Docker and deploy on Kubernetes through a unified sandbox API. [Quick Start](#getting-started) · [Examples](#examples) · [Documentation](#documentation) · [Fast Sandbox](docs/architecture/fast-sandbox/index.md) ## Features | Feature | What it enables | Learn more | |---------|-----------------|------------| | **Fast Sandbox runtime** | Fast, high-density sandboxes on Kubernetes. Firecracker creation: **64 ms P50 (serial)** / **125.4 ms P99 (10 concurrent)**. Firecracker sandboxes support pause/resume with memory and disk state preserved. | [Integration](docs/architecture/fast-sandbox/index.md) · [Performance](docs/architecture/fast-sandbox/performance.md) · [Pause/resume](docs/architecture/fast-sandbox/checkpoints.md) | | **Agent working environments** | Execute commands, manage files, and run code with built-in APIs. Integration examples show how to run coding agents, browsers, and desktops inside sandboxes. | [Examples](docs/examples/index.md) | | **Network access control** | Route inbound traffic through a unified ingress gateway and control outbound access with per-sandbox egress policies. | [Ingress](docs/architecture/network/ingress.md) · [Egress](docs/architecture/network/egress.md) | | **Credential Vault** | Let agents call external services without exposing real credentials to sandbox workloads. | [Credential Vault](docs/guides/credential-vault.md) | | **Local to cluster** | Start with Docker and deploy on Kubernetes through a unified lifecycle API. Resource pools and batch creation support agent evaluation and RL training workloads. | [Kubernetes runtime](docs/architecture/control-plane/operator.md) | | **SDKs, CLI, and MCP** | Integrate with Python, Java/Kotlin, TypeScript/JavaScript, C#/.NET, or Go SDKs. Use `osb` from the terminal or connect agents through MCP. | [SDKs](#sdks) · [CLI](#cli) · [MCP](#mcp) | | **Extensible sandbox protocol** | Build custom runtime integrations against defined sandbox lifecycle and execution APIs. | [API specs](specs/README.md) | ## Getting Started Requirements: - Docker (required for local execution) - Python 3.10+ (required for examples and local runtime) ### Install and Configure the Sandbox Server ```bash uvx opensandbox-server init-config ~/.sandbox.toml --example docker uvx opensandbox-server # Show help # uvx opensandbox-server -h ``` ### Create a Sandbox and Execute Commands/Scripts Install the Sandbox SDK ```bash uv pip install opensandbox ``` Create a sandbox from an `alpine` image and execute commands and scripts. ```python import asyncio from opensandbox import Sandbox from opensandbox.models import WriteEntry async def main() -> None: # 1. Create a sandbox from the alpine image sandbox = await Sandbox.create("alpine") try: # 2. Execute a shell command execution = await sandbox.commands.run("echo 'Hello OpenSandbox!'") print(execution.logs.stdout[0].text) # 3. Write a script file await sandbox.files.write_files([ WriteEntry( path="/tmp/hello.sh", data="echo \"Hello $1\"\necho '2 + 2 =' $((2 + 2))", mode=755, ) ]) # 4. Read the file back content = await sandbox.files.read_file("/tmp/hello.sh") print(f"Content: {content}") # 5. Execute the script execution = await sandbox.commands.run("sh /tmp/hello.sh OpenSandbox") for log in execution.logs.stdout: print(log.text) finally: # 6. Cleanup the sandbox await sandbox.destroy() if __name__ == "__main__": asyncio.run(main()) ``` ## Examples Explore examples by what you want your agent to do. Runnable source code lives in [`examples/`](examples/). | Use case | What you can build | Examples | |----------|--------------------|----------| | **Coding agents** | Run coding agents in isolated environments to edit files, execute commands, and complete development tasks. | [Claude Code](docs/examples/claude-code.md) · [Codex CLI](docs/examples/codex-cli.md) · [DeerFlow](docs/examples/deer-flow.md) | | **Code execution and data analysis** | Execute model-generated code and work with results through the Code Interpreter SDK. | [Code Interpreter](docs/examples/code-interpreter.md) | | **Browser and desktop automation** | Automate web interactions and testing, or give agents access to a desktop environment. | [Playwright](docs/examples/playwright.md) · [Chrome](docs/examples/chrome.md) · [Desktop](docs/examples/desktop.md) | | **Agent evaluation** | Run evaluations with a separate sandbox for each trial. | [Harbor Evaluation](docs/examples/harbor-evaluation.md) | See the [full example catalog](docs/examples/index.md) for more coding agents, framework integrations, remote development environments, Kubernetes deployment, and storage patterns. ## SDKs Pick your language:
Python ```bash pip install opensandbox ```
Java/Kotlin (Gradle Kotlin DSL) ```kotlin dependencies { implementation("com.alibaba.opensandbox:sandbox:{latest_version}") } ```
Java/Kotlin (Maven) ```xml com.alibaba.opensandbox sandbox {latest_version} ```
JavaScript/TypeScript ```bash npm install @alibaba-group/opensandbox ```
C#/.NET ```bash dotnet add package Alibaba.OpenSandbox ```
Go ```bash go get github.com/alibaba/OpenSandbox/sdks/sandbox/go ```
## CLI OpenSandbox also provides `osb`, a terminal CLI for the common sandbox workflow: create sandboxes, run commands, move files, inspect diagnostics, and manage runtime egress policy. Install: ```bash pip install opensandbox-cli # or uv tool install opensandbox-cli ``` Quick start: ```bash osb config init osb config set connection.domain localhost:8080 osb config set connection.protocol http osb config set connection.api_key osb sandbox create --image python:3.12 --timeout 30m -o json osb command run -o raw -- python -c "print(1 + 1)" ``` See the [CLI README](cli/README.md) for the full command reference. ## MCP The OpenSandbox MCP server exposes sandbox creation, command execution, and text file operations to MCP-capable clients such as Claude Code and Cursor. Install and run: ```bash pip install opensandbox-mcp opensandbox-mcp --domain localhost:8080 --protocol http ``` Minimal stdio config: ```json { "mcpServers": { "opensandbox": { "command": "opensandbox-mcp", "args": ["--domain", "localhost:8080", "--protocol", "http"] } } } ``` See the [MCP README](sdks/mcp/sandbox/python/README.md) for client-specific setup. ## Official Container Images OpenSandbox release images are published under the same component name in three official registries: - Docker Hub: `docker.io/opensandbox/` - GitHub Container Registry: `ghcr.io/opensandbox-group/opensandbox/` - Alibaba Cloud Container Registry: `sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/` Tagged release images are signed keylessly with Cosign and include provenance attestations. Pin production images by digest and follow the [release verification guide](docs/community/release-verification.md) to verify the image against the OpenSandbox GitHub Actions identity before deployment. ## Documentation - [Website](https://open-sandbox.ai) — Project homepage and documentation site - [Architecture](docs/architecture/index.md) — System design and component responsibilities - [Deployment Guide](docs/deployment/index.md) — Kubernetes installation and operations - [Server Configuration](docs/getting-started/configuration.md) — Runtime and server settings - [SDK Reference](docs/sdks/index.md) — Language guides and capability coverage - [CLI Guide](cli/README.md) — Installation and command reference - [MCP Integration](sdks/mcp/sandbox/python/README.md) — Setup for MCP-capable clients - [API Reference](docs/api/index.md) — Sandbox lifecycle and execution contracts - [Credential Vault](docs/guides/credential-vault.md) — Outbound credential injection - [Release Verification](docs/community/release-verification.md) — Image signing and artifact verification - [Enhancement Proposals](oseps/README.md) — Design proposals and technical direction - [Roadmap](ROADMAP.md) — Project priorities and planning ## Contact and Discussion - Issues: Submit bugs, feature requests, or design discussions through GitHub Issues - Discord: Join the [OpenSandbox Discord community](https://discord.gg/g7FuPs8YeD) - DingTalk: Join the [OpenSandbox technical discussion group](https://qr.dingtalk.com/action/joingroup?code=v1,k1,A4Bgl5q1I1eNU/r33D18YFNrMY108aFF38V+r19RJOM=&_dt_no_comment=1&origin=11) ## License This project is open source under the [Apache 2.0 License](LICENSE). --- ### README (`docs/README.md`) # OpenSandbox Docs Site This directory hosts the VitePress site for OpenSandbox. ## Local development ```bash nvm use 22 cd docs pnpm install pnpm docs:dev ``` ## Build ```bash nvm use 22 cd docs pnpm install pnpm docs:build ``` ## Notes - Site content is maintained directly under `docs/` — not auto-generated from monorepo READMEs. - See `AGENTS.md` → "Documentation Rules" for content ownership and conventions. --- ### Index (`docs/index.md`) --- layout: home hero: name: OpenSandbox text: Universal Sandbox Infrastructure for AI Applications tagline: Securely run commands, code interpreters, browsers, and developer tools in isolated environments with multi-language SDKs. actions: - theme: brand text: Quick Start link: /getting-started/ - theme: alt text: Architecture link: /architecture/ - theme: alt text: SDKs link: /sdks/ features: - title: Sandbox Lifecycle Management details: Provision, monitor, renew, pause/resume, and terminate sandbox instances with Docker and Kubernetes runtimes. - title: Multi-Language SDKs details: Build with Python, Java/Kotlin, JavaScript/TypeScript, C#/.NET, and Go SDKs on top of standardized lifecycle and execution protocols. - title: In-Sandbox Execution details: Execute shell commands, manage files, run multi-language code interpreters, expose ports, and stream logs and metrics. - title: Built for AI Workloads details: Supports coding agents, browser automation, remote development, code execution, and reinforcement learning scenarios. --- ## Typical Scenarios OpenSandbox is listed in the [CNCF Landscape](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--opensandbox).

Coding Agents

Run Claude Code, Gemini CLI, Codex, and other coding agents in isolated sandboxes.

Browser Automation

Execute Chrome and Playwright workloads with controlled runtime, filesystem, and networking.

Remote Development

Host VS Code Web and desktop-like environments for secure cloud development workflows.

AI Code Execution

Run model-generated code safely, stream outputs, and iterate quickly with reproducible environments.
Explore all examples in [Examples](/examples/). ## Quick Install ::: code-group ```bash [Python SDK] pip install opensandbox ``` ```bash [JavaScript SDK] npm install @alibaba-group/opensandbox ``` ```kotlin [Kotlin (Gradle)] dependencies { implementation("com.alibaba.opensandbox:sandbox:{latest_version}") } ``` ```bash [Go SDK] go get github.com/alibaba/OpenSandbox/sdks/sandbox/go ``` ```bash [C# SDK] dotnet add package Alibaba.OpenSandbox ``` ::: See [Getting Started](/getting-started/) for the full setup guide. --- ### Api Index (`docs/api/index.md`) --- title: API Specifications description: OpenAPI specification documents defining the complete API interfaces and data models for OpenSandbox. --- # OpenSandbox API Specifications This section contains the OpenAPI specification documents for the OpenSandbox project, defining the complete API interfaces and data models. Use the server base URLs defined in each spec (for example, `http://localhost:8080/v1` for the lifecycle API, `http://localhost:44772` for execd, and `http://localhost:18080` for egress) when constructing requests. ## Specification Files Implementing an alternative backend or integrating a workload's own API? Read [Implementation compatibility](/api/implementation-compatibility) for the boundary between these contracts, the reference daemons, and current SDK expectations. ### 1. sandbox-lifecycle.yml [OpenAPI source](https://github.com/opensandbox-group/OpenSandbox/blob/main/specs/sandbox-lifecycle.yml) **Sandbox Lifecycle Management API** Defines the complete lifecycle interfaces for creating, managing, and destroying sandbox environments from container images or snapshots. **Core Features:** - **Sandbox Management**: Create, list, query, and delete sandbox instances with metadata filters and pagination - **State Control**: Pause and resume sandbox execution - **Lifecycle States**: Supports transitions across Pending -> Running -> Pausing -> Paused -> Stopping -> Terminated, and error handling with `Failed` - **Resource & Runtime Configuration**: Specify resource limits and optional Kubernetes `resourceRequests`, image startup `entrypoint`, `platform`, lifecycle hooks, optional `secureAccess`, volumes, environment variables, and opaque `extensions` - **Image Support**: Create sandboxes from public or private registries, including registry auth - **Timeout Management**: Optional `timeout` on creation (omit or set to `null` to disable automatic expiration) with explicit renewal via API - **Endpoint Access**: Retrieve public access endpoints for services running inside sandboxes, including required headers when secured access is enabled; endpoint lookups report the sandbox origin via the `OPEN-SANDBOX-ORIGIN` response header (`template` for fsb golden-image sandboxes) - **Template Management**: Create, list, inspect, and delete fsb golden-image templates; template builds are asynchronous (poll until `Succeeded`) - **Snapshot Management**: Create snapshots from sandboxes, list snapshots with source/name filters, and delete snapshots **Main Endpoints (base path `/v1`):** - `POST /sandboxes` - Create a sandbox from an image, snapshot, or template with timeout and resource limits - `GET /sandboxes` - List sandboxes with state/metadata filters and pagination - `GET /sandboxes/{sandboxId}` - Get full sandbox details (including startup source and entrypoint) - `DELETE /sandboxes/{sandboxId}` - Delete a sandbox - `POST /sandboxes/{sandboxId}/snapshots` - Create a snapshot from a sandbox - `GET /snapshots` - List snapshots with optional source sandbox, exact name, and state filtering plus pagination - `GET /snapshots/{snapshotId}` - Get snapshot state and metadata - `DELETE /snapshots/{snapshotId}` - Delete a snapshot - `POST /sandboxes/{sandboxId}/pause` - Pause a sandbox (asynchronous); returns `409 Conflict` if it is already paused - `POST /sandboxes/{sandboxId}/resume` - Resume a paused sandbox - `POST /sandboxes/{sandboxId}/renew-expiration` - Set the absolute sandbox expiration time - `PATCH /sandboxes/{sandboxId}/metadata` - Patch sandbox metadata (JSON Merge Patch, RFC 7396) - `GET /sandboxes/{sandboxId}/endpoints/{port}` - Get an access endpoint and required headers; supports `use_server_proxy` and signed-endpoint `expires` parameters - `GET/PUT/PATCH/DELETE /sandboxes/{sandboxId}/networkpolicy` - Inspect and manage the sandbox egress network policy (Fsb persists intent on the Sandbox CR; other backends proxy the sandbox-side egress service) - `POST/GET/PATCH/DELETE /sandboxes/{sandboxId}/credential-vault` (+ sanitized `credentials` / `bindings` reads) - Manage the sandbox Credential Vault (fsb relays to the egress-held vault over FastPath; other backends proxy the sandbox-side egress service; values are write-only and never returned) - `POST /templates` - Create a fsb template (asynchronous golden-image build) - `GET /templates` - List templates with metadata filters and pagination - `GET /templates/{templateId}` - Get template status and artifact references - `DELETE /templates/{templateId}` - Delete a template **Expiration updates:** `renew-expiration` sets `expiresAt` to the supplied future timestamp, which may be earlier than, equal to, or later than the current expiration. The configured `server.max_sandbox_timeout_seconds` applies only to `timeout` on creation; it does not cap expiration updates. See [Sandbox expiration](/getting-started/configuration#sandbox-expiration). **Optional `Sandbox.allocation` response field:** - Returned only when the runtime confirms the sandbox's current concrete Pool allocation. - Omitted for unconfirmed allocations, non-Pool sandboxes, and allocations being released. - This field is not a request echo, allocation history, or readiness signal, and does not expose Pod names or other Kubernetes-internal fields. **Authentication:** - HTTP Header: `OPEN-SANDBOX-API-KEY: your-api-key` - Environment Variable: `OPEN_SANDBOX_API_KEY` (for SDK clients) ### 2. diagnostic-api.yml {#diagnostics} [OpenAPI source](https://github.com/opensandbox-group/OpenSandbox/blob/main/specs/diagnostic-api.yml) **Sandbox Diagnostics API** Defines structured routing context and best-effort troubleshooting descriptors for sandbox diagnostic logs and events. Context identifies the runtime backend and its diagnostic identifiers. Log and event descriptors either embed plain-text content inline or return a download URL. This spec does not define a structured audit or observability model. **Main Endpoints (base path `/v1`):** - `GET /sandboxes/{sandboxId}/diagnostics/context` - Retrieve the runtime backend and backend-defined diagnostic identifiers - `GET /sandboxes/{sandboxId}/diagnostics/logs` - Retrieve a diagnostic log content descriptor; `scope` is required - `GET /sandboxes/{sandboxId}/diagnostics/events` - Retrieve a diagnostic event content descriptor; `scope` is required **Authentication:** - HTTP Header: `OPEN-SANDBOX-API-KEY: your-api-key` - Environment Variable: `OPEN_SANDBOX_API_KEY` (for SDK clients) `scope` is required. Docker and Kubernetes support `container`/`all` for logs and `runtime`/`all` for events. Fast Sandbox supports `runtime`/`all` events; log collection is not implemented there. Unsupported scopes return `DIAGNOSTICS_SCOPE_UNSUPPORTED`. Context does not take a scope and is available independently of execd readiness. The built-in server returns `docker` with container identifiers, `kubernetes` with namespace/workload identifiers and an allocated Pod name when available, or `fast-sandbox` with an empty identifier map. Kubernetes context can be queried before a Pod is allocated. Unknown sandboxes return 404; backends without context support return 501 `DIAGNOSTICS_NOT_IMPLEMENTED`. Responses use `delivery: inline` with `content`, or `delivery: url` with `contentUrl` and an optional expiry. The SDK does not download URL content; inspect `truncated` and `warnings` before treating results as complete. Python models and CLI JSON/YAML use snake_case fields such as `content_url`. CLI raw output prints the content or URL without following it. See [SDK diagnostics](/sdks/#diagnostics) for language support and [CLI diagnostics](/cli/#collect-diagnostics) for command examples. ### 3. execd-api.yaml [OpenAPI source](https://github.com/opensandbox-group/OpenSandbox/blob/main/specs/execd-api.yaml) **Code Execution API Inside Sandbox** Defines interfaces for executing code, commands, and file operations within sandbox environments, providing complete code interpreter and filesystem management capabilities. Forward the headers returned by lifecycle endpoint resolution, including `X-EXECD-ACCESS-TOKEN` when required. Do not hard-code an execd address or assume the lifecycle API key alone authorizes direct sandbox access. **Core Features:** - **Code Execution**: Stateful code execution supporting Python, JavaScript, and other languages with context lifecycle management - **Command Execution**: Shell command execution with foreground/background modes and polling endpoints for status/output - **File Operations**: Complete CRUD operations for files and directories - **Real-time Streaming**: Real-time output streaming via SSE (Server-Sent Events) - **System Monitoring**: Real-time monitoring of CPU and memory metrics - **Access Control**: Token-based API authentication via `X-EXECD-ACCESS-TOKEN` **Main Endpoint Categories:** **Health Check:** - `GET /ping` - Service health check **Code Interpreter:** - `GET /code/contexts` - List active code execution contexts (filterable by language) - `DELETE /code/contexts` - Delete all contexts for a language - `DELETE /code/contexts/{context_id}` - Delete a specific context - `POST /code/context` - Create a code execution context - `POST /code` - Execute code in a context (streaming output) - `DELETE /code` - Interrupt code execution **Command Execution:** - `POST /command` - Execute shell command (streaming output) - `DELETE /command` - Interrupt command execution - `GET /command/status/{id}` - Get foreground/background command status - `GET /command/{id}/logs` - Fetch accumulated stdout/stderr for a background command **Bash Session:** - `POST /session` - Create a bash session - `POST /session/{sessionId}/run` - Run command in a bash session (streaming output) - `DELETE /session/{sessionId}` - Delete a bash session **Filesystem:** - `GET /files/info` - Get metadata for files - `DELETE /files` - Delete files (not directories) - `POST /files/permissions` - Change file permissions - `POST /files/mv` - Move/rename files - `GET /files/search` - Search files (supports glob patterns) - `POST /files/replace` - Batch replace file content - `POST /files/upload` - Upload files (multipart) - `GET /files/download` - Download files (supports range requests) **Directory Operations:** - `GET /directories/list` - List directory contents with optional depth control - `POST /directories` - Create directories with permissions (mkdir -p semantics) - `DELETE /directories` - Recursively delete directories **System Metrics:** - `GET /metrics` - Get system resource metrics - `GET /metrics/watch` - Watch system metrics in real-time (SSE stream) **Isolated Execution (base path `/v1/isolated`):** - `POST /session` - Create an isolated bash session - `GET /sessions` - List isolated sessions - `GET /capabilities` - Get isolator capabilities - `GET /session/{sessionId}` - Get isolated session state - `DELETE /session/{sessionId}` - Delete an isolated session - `POST /session/{sessionId}/run` - Run a command in an isolated session (foreground SSE or background execution) - `GET/DELETE /session/{sessionId}/runs/{runId}` - Get status or interrupt an isolated run - `GET /session/{sessionId}/runs/{runId}/logs` - Retrieve isolated run logs - `GET /session/{sessionId}/files/info` - Get file information - `GET /session/{sessionId}/files/download` - Download a file - `POST /session/{sessionId}/files/upload` - Upload a file - `DELETE /session/{sessionId}/files` - Remove files - `POST /session/{sessionId}/files/mv` - Rename or move files - `POST /session/{sessionId}/files/permissions` - Change file permissions - `POST /session/{sessionId}/files/replace` - Replace file content - `GET /session/{sessionId}/files/search` - Search files - `GET /session/{sessionId}/directories/list` - List directory contents - `POST /session/{sessionId}/directories` - Create directories - `DELETE /session/{sessionId}/directories` - Delete directories `GET /session/{sessionId}/diff` and `POST /session/{sessionId}/commit` appear in the contract but are not implemented in the current execd: both return `503` with a not-supported error, and capabilities report `diff_supported: false` and `commit_supported: false`. Check `/v1/isolated/capabilities` before using runtime-dependent isolation features. ### 4. egress-api.yaml [OpenAPI source](https://github.com/opensandbox-group/OpenSandbox/blob/main/specs/egress-api.yaml) **Sandbox Egress Runtime API** Defines the runtime egress policy interface exposed directly by the [egress sidecar](/architecture/network/egress) inside a sandbox. Unlike lifecycle operations, this API is reached by first resolving the sandbox endpoint for the egress port and then calling the sidecar endpoint directly. **Core Features:** - **Policy Inspection**: Retrieve the currently enforced egress policy and derived runtime mode - **Policy Mutation**: Patch egress rules at runtime using sidecar merge semantics - **Direct Sidecar Access**: Access via sandbox endpoint resolution instead of server-side lifecycle forwarding - **Optional Sidecar Auth**: Supports endpoint-specific headers when the egress sidecar requires auth **Main Endpoints:** - `GET /policy` - Get the current egress policy - `PATCH /policy` - Merge new egress rules into the current policy - `DELETE /policy` - Remove specific egress rules from the current policy by target - `POST/GET/PATCH/DELETE /credential-vault` - Create, inspect, mutate, or delete vault state - `GET /credential-vault/credentials` and `GET /credential-vault/credentials/{credential_name}` - Read sanitized credential metadata - `GET /credential-vault/bindings` and `GET /credential-vault/bindings/{binding_name}` - Read binding metadata Credential values are write-only. Enable Credential Proxy and an egress policy before creating a vault. Template-backed (fsb) sandboxes use the server-routed `/sandboxes/{sandboxId}/credential-vault` endpoints above: the server relays revisions to the egress-held vault over the FastPath handler relay, the ack is synchronous, and values never appear on any read (see [Credential Vault](/guides/credential-vault)). ## Technical Features ### Streaming Output (Server-Sent Events) Code execution and command execution interfaces use SSE for real-time streaming output, supporting the following event types: - `init` - Initialization event - `status` - Status update - `stdout` / `stderr` - Standard output/error streams - `result` - Execution result - `execution_complete` - Execution completed - `execution_count` - Execution count - `error` - Error information ### Resource Limits Supports flexible resource configuration (similar to Kubernetes): ```json { "cpu": "500m", "memory": "512Mi", "gpu": "1" } ``` ### File Permissions Supports Unix-style file permission management: - Owner - Group - Permission mode values such as 644 or 755 --- ### Architecture Index (`docs/architecture/index.md`) --- title: Architecture description: OpenSandbox architecture overview covering client SDKs, lifecycle control plane, runtime backends, and data-plane components. --- # OpenSandbox Architecture OpenSandbox is a general-purpose sandbox platform for AI applications. It provides client SDKs and tools, protocol definitions, a lifecycle control plane, Docker and Kubernetes runtime backends (supporting both container workloads and template-backed microVMs), and in-sandbox execution components for commands, files, code interpreters, browser automation, desktop environments, and training workloads. This document describes the current repository architecture and the main boundaries between public contracts, server implementation, runtime providers, and sandbox data-plane components. ## Architecture Overview The architecture distinguishes three primary interaction paths: - **Lifecycle management** (orange arrows): Applications, agents, and operators send lifecycle requests (creating, listing, renewing, pausing, resuming, or deleting sandboxes, as well as managing snapshots and templates) to the FastAPI lifecycle server. The server persists metadata and delegates provisioning and orchestration to the configured runtime backend. - **Execution and service access** (blue arrows): For command, file, PTY, and Jupyter code execution, or reaching user workload services, clients bypass lifecycle orchestration and connect directly to resolved endpoints—via direct host/pod addresses, the lifecycle server proxy, or an optional Kubernetes ingress gateway. - **Outbound controls** (green arrows): Outbound requests from the sandbox to external APIs, model providers, or web services can be filtered and secured through egress controls (a per-sandbox sidecar for container workloads or a shared egress process per Fastlet pod for FastSandbox), with support for DNS/nftables policies and optional Credential Vault secret injection. Runtime and networking capabilities depend on the selected backend; the diagram shows the shared architecture, not a requirement to deploy every component. OpenSandbox is organized around six practical surfaces: 1. **Client surface** - SDKs, the `osb` CLI, and the MCP server used by applications, agents, and operators. 2. **Protocol surface** - OpenAPI contracts under `specs/` for lifecycle, diagnostics, in-sandbox execution, and egress policy. 3. **Lifecycle control plane** - the FastAPI server under `server/` that authenticates requests, validates config, persists server-managed records, and delegates lifecycle work to a configured runtime service. 4. **Runtime backends** - Docker for local and single-host deployments, and Kubernetes with container workload providers (BatchSandbox or `kubernetes-sigs/agent-sandbox`) alongside an optional FastSandbox integration for template-backed microVMs. 5. **Sandbox data plane** - the user workload and `execd` daemon inside a container or microVM, with optional Jupyter/code-interpreter support. Volume and egress integration depend on the runtime. 6. **Network and security plane** - endpoint resolution, server proxying, Kubernetes ingress gateway routing, secure endpoint access, egress policy enforcement, credential vault, resource limits, and secure container runtimes. The split is intentional: SDKs and tools should depend on the public contracts, the server should own lifecycle orchestration, runtime providers should own platform-specific resource creation, and `execd`/egress should own operations that happen from inside the sandbox network and filesystem namespace. The implementation is documented by architectural layer under `architecture/`: | Layer | Responsibility | Documentation | |---|---|---| | Control plane | Lifecycle orchestration, workload orchestration, snapshots | [Server](/architecture/control-plane/server), [Kubernetes Controller](/architecture/control-plane/operator) | | Data plane | In-sandbox execution, node-side collection | [Execd](/architecture/data-plane/execd), [Node Agent](/architecture/data-plane/node-agent) | | Network | Inbound access, outbound policy | [Ingress](/architecture/network/ingress), [Egress](/architecture/network/egress), [Single-Host Network](/architecture/network/single-host-network) | | Backend integration | Template-backed sandboxes on the fast-sandbox platform | [Fast Sandbox](/architecture/fast-sandbox/) | ## 1. Client Surface The client surface is the developer-facing entry point for OpenSandbox. ### 1.1 Sandbox SDKs The sandbox SDKs wrap lifecycle operations and in-sandbox operations behind language-native APIs: - Python: `sdks/sandbox/python` - JavaScript/TypeScript: `sdks/sandbox/javascript` - Java/Kotlin: `sdks/sandbox/kotlin` - C#/.NET: `sdks/sandbox/csharp` - Go: `sdks/sandbox/go` Common capabilities include: - Create, list, inspect, pause, resume, renew, and delete sandboxes. - Resolve service endpoints for sandbox ports. - Execute commands with streamed output and background status/log polling. - Manage files and directories. - Read resource metrics from `execd`. - Inspect or patch runtime egress policy when an egress sidecar is attached. Generated OpenAPI clients live beside handwritten adapters. Generated code handles ordinary request/response APIs; handwritten layers cover SDK ergonomics, streaming, transport lifecycle, error mapping, and high-level models. ### 1.2 Code Interpreter SDKs The code-interpreter SDKs build on the sandbox SDKs and `execd` code execution APIs. They manage code execution contexts and expose language-oriented code execution helpers. The official code-interpreter image is maintained in [opensandbox-group/sandbox-images](https://github.com/opensandbox-group/sandbox-images) (previously located under `sandboxes/code-interpreter/`; see [Migration Reference](/reference/code-interpreter-image-migration)). It provides Python, Java, Node.js, and Go runtimes, and Jupyter kernels for Python, Java, TypeScript/JavaScript, Go, and Bash. Exact language versions are image-controlled and selected through environment variables such as `PYTHON_VERSION`, `JAVA_VERSION`, `NODE_VERSION`, and `GO_VERSION`. ### 1.3 CLI and MCP The `osb` CLI under `cli/` is a terminal interface for day-to-day sandbox operations: - `osb sandbox`: lifecycle and endpoint management - `osb command`: command execution, background logs, and shell sessions - `osb file`: file and directory operations - `osb egress`: runtime egress policy inspection and mutation - `osb devops`: low-level diagnostics - `osb skills`: OpenSandbox-specific agent skill installation The MCP server under `sdks/mcp/sandbox/python` exposes focused sandbox lifecycle, command, and text-file tools to MCP-capable clients such as Claude Code and Cursor. ## 2. Protocol Surface OpenSandbox treats `specs/` as the public contract source of truth. ### 2.1 Lifecycle API `specs/sandbox-lifecycle.yml` defines the lifecycle API, served by the server with the base path `/v1`. Main resource groups: - **Sandboxes**: create from an image, snapshot, or template; list, get, delete, pause, resume, renew expiration, and resolve port endpoints. FastSandbox also supports inspecting and managing runtime network policy through `GET/PUT/PATCH/DELETE /sandboxes/{sandboxId}/networkpolicy`, and reports the sandbox origin to clients via the `OPEN-SANDBOX-ORIGIN` response header on endpoint lookups. - **Snapshots**: create a persistent snapshot from a sandbox, list snapshots, get snapshot state, and delete snapshots. - **Templates**: create, list, inspect, and delete template definitions in the catalog used by FastSandbox (`/templates`). Important request features: - `image` or `snapshotId` for container startup, or `templateId` for the FastSandbox path in Kubernetes mode. - `entrypoint`, environment variables, metadata, and opaque `extensions`. - `resourceLimits` for CPU, memory, GPU, and future resource keys. - `platform` constraints. - `volumes` for host paths, platform-managed named volumes/PVCs, and OSSFS. - `networkPolicy` for outbound network policy configuration. - `secureAccess` for Kubernetes ingress gateway deployments that require endpoint credentials. ### 2.2 Diagnostics API `specs/diagnostic-api.yml` defines best-effort diagnostic descriptors for sandbox logs and events. The server also exposes practical DevOps diagnostics routes that return plain text for operators and AI troubleshooting workflows. ### 2.3 Execd API `specs/execd-api.yaml` defines the in-sandbox execution API exposed by `components/execd/`. Main capabilities: - Health check: `GET /ping` - Code contexts and execution: `/code/contexts`, `/code/context`, `/code` - Bash sessions: `/session` - Commands: `/command`, command status, and background command logs - Files and directories: `/files/*`, `/directories` - Metrics: `/metrics`, `/metrics/watch` Command and code execution use Server-Sent Events for streaming output. The current `execd` implementation also includes interactive PTY WebSocket endpoints under `/pty` for long-lived shell sessions. ### 2.4 Egress API `specs/egress-api.yaml` defines the runtime policy and credential broker APIs exposed by `components/egress/`: - **Policy**: `GET /policy` to inspect the current policy; `PATCH /policy` to mutate rules at runtime. - **Credential Vault**: `GET`, `POST`, `PATCH`, and `DELETE` on `/credential-vault` to manage sandbox-local credential bindings and token injection without exposing raw secrets to the workload. For container backends, callers resolve the sandbox endpoint for the egress sidecar port and connect directly (forwarding endpoint authentication headers when sidecar auth is enabled). For FastSandbox, runtime network policy is managed through the lifecycle API (`/sandboxes/{sandboxId}/networkpolicy`), while the shared egress process (one per Fastlet pod) routes policy and per-subject credential vault operations using internal request headers. ## 3. Lifecycle Control Plane The lifecycle server under `server/` is a FastAPI application. It owns request validation, API-key authentication, server configuration, lifecycle orchestration, endpoint formatting, diagnostics, and server-managed persistence. ### 3.1 Server Structure Key packages: - `opensandbox_server/main.py`: app startup, middleware, router registration, runtime validation, and renew-intent startup. - `opensandbox_server/api/`: lifecycle routes, proxy routes, pool routes, diagnostics routes, and request/response schemas. - `opensandbox_server/services/`: lifecycle service interfaces and Docker/Kubernetes implementations. - `opensandbox_server/services/k8s/`: Kubernetes workload providers, endpoint resolution, volume/egress helpers, informer support, and provider-specific mapping. - `opensandbox_server/repositories/`: persistence adapters, used for server-managed snapshot records and the FastSandbox template catalog. - `opensandbox_server/integrations/renew_intent/`: optional auto-renew-on-access integration. - `opensandbox_server/middleware/`: API-key authentication and request ID middleware. ### 3.2 Runtime Service Selection The server selects its lifecycle implementation based on `[runtime].type`: - `docker` -> `DockerSandboxService` - `kubernetes` -> `CompositeSandboxService`, which combines `KubernetesSandboxService` (backed by BatchSandbox or agent-sandbox providers) with `FastSandboxService` for template-based microVM execution. The top-level `runtime.type` configuration option remains strictly `docker` or `kubernetes`. All implementations satisfy the same `SandboxService` interface, keeping API routes thin and delegating behavior to services. Under Kubernetes mode, `CompositeSandboxService` inspects creation requests: requests specifying a `templateId` dispatch to `FastSandboxService`, while container requests specifying an `image` or `snapshotId` dispatch to `KubernetesSandboxService`. Operations on existing sandboxes are routed by their ID prefix (IDs starting with `fsb-` route to `FastSandboxService`). Runtime-specific details stay behind the service boundary. ### 3.3 Server Persistence The `[store]` configuration selects SQLite (the default, at `~/.opensandbox/opensandbox.db`) or PostgreSQL. The server persists snapshot metadata and the FastSandbox template catalog through separate repository adapters (`repositories/snapshots/` and `repositories/templates/`). The template catalog stored in the database is the public source of truth; Kubernetes `SandboxTemplate` resources drive golden-image builds and provide asynchronous status updates. ### 3.4 Server Proxy and Endpoint Resolution The lifecycle endpoint API returns the reachable address for a service port inside a sandbox. Depending on runtime and configuration, the endpoint may be: - A Docker host/bridge mapped endpoint. - A Kubernetes ingress gateway endpoint. - A server-proxied URL under `/sandboxes/{sandboxId}/proxy/{port}` when `use_server_proxy=true`. Header-routed ingress endpoints include `OpenSandbox-Ingress-To` in their endpoint metadata. When the endpoint is rewritten to a server-proxied URL, the server removes only this ingress routing header and preserves other required endpoint headers. The server proxy supports HTTP and WebSocket traffic and is also integrated with optional renew-on-access behavior. For HTTP responses, it strips hop-by-hop headers and the backend `Server` header while preserving an origin `Date`; the server adds a current `Date` only when the response does not already contain one. A root-relative `Location` value that starts with a single `/` is rebased under the same sandbox proxy route, while absolute URLs, network-path references (`//host/path`), and ordinary path-relative values are forwarded unchanged. HTTP proxy responses preserve the sandbox service's status code, body, and `Content-Type`, including redirects and backend errors. The generated Server OpenAPI describes `200` and `default` responses with `*/*` and no fixed payload schema because the sandbox service controls the payload. Server-side validation and authentication still apply before forwarding; the explicit `422` validation response remains documented. These response declarations cover both root and `/v1` aliases, with and without a backend path, for every supported HTTP method. ## 4. Runtime Backends ### 4.1 Docker Runtime The Docker runtime is the local and single-host backend. It talks directly to the Docker daemon and manages containers, timers, labels, volumes, ports, optional sidecars, and snapshots. Core responsibilities: - Pull public or private images, including per-request registry authentication. - Create containers with CPU, memory, GPU, platform, capability, AppArmor, seccomp, PID, and secure-runtime settings. - Stage the `execd` binary from `[runtime].execd_image` into the sandbox and install a bootstrap launcher before starting the user entrypoint. - Support network modes `host`, `bridge`, `none`, `pasta`, `slirp4netns`, and custom user-defined networks. The user-mode modes (`pasta`, `slirp4netns`) expose no host-routable container IP and are reached through the published host ports. - Allocate host ports for `execd` and user service endpoints in non-host network modes. - Restore expiration timers for existing managed containers after server restart. - Support host bind mounts, Docker named volumes via the `pvc` volume model, and OSSFS-backed mounts. - Attach an egress sidecar when `networkPolicy` is requested and Docker networking is compatible. - Create Docker-backed persistent snapshots as local images and restore sandboxes from those snapshot images. Docker pause/resume uses container-level pause/resume. For sandboxes with an egress sidecar, pause freezes the sandbox container before the sidecar, while resume unfreezes the sidecar before the sandbox container. Docker snapshots are exposed through the public snapshot API. ### 4.2 Kubernetes Runtime The Kubernetes runtime delegates actual workload creation to a workload provider selected by `kubernetes.workload_provider`. Supported providers: - `batchsandbox` - the default provider backed by OpenSandbox's Kubernetes controller and `BatchSandbox` CRD. - `agent-sandbox` - a provider for `kubernetes-sigs/agent-sandbox`. The Kubernetes server path handles: - Kubernetes client initialization and informer-backed reads. - Workload creation from image requests; `snapshotId` startup resolves to a stored restorable image when the snapshot record supports restore. - Template merging for BatchSandbox and agent-sandbox manifests. - Per-request image pull secrets where the provider supports them. - Resource limits and GPU translation to Kubernetes extended resources. - Platform constraints and RuntimeClass integration for secure runtimes. - Volumes, egress sidecars, and secure endpoint access annotations. - Endpoint resolution through direct workload data or ingress gateway configuration. - Pause/resume delegation to providers. - Plain-text diagnostics from Kubernetes resources. ### 4.3 BatchSandbox Controller The Kubernetes controller under `kubernetes/` implements OpenSandbox-specific CRDs for high-throughput and pooled sandbox delivery: - `BatchSandbox`: create one or many sandbox replicas from a pod template. - `Pool`: maintain pre-warmed resources for fast allocation. - `SandboxSnapshot`: snapshot records used by BatchSandbox pause/resume and the public snapshot workflow. BatchSandbox supports both template-based creation and pool-based creation via `extensions.poolRef`. It also supports optional task orchestration for batch and RL-style workloads. For supported single-replica BatchSandbox workloads, the rootfs pause/resume path commits the sandbox filesystem to an OCI image and releases runtime resources; resume recreates the workload from that image while preserving the sandbox ID. Workloads configured with a QEMU snapshot contract use a VM-state checkpoint and restore path instead. The public snapshot API supports Docker and compatible `BatchSandbox` workloads. `DockerSnapshotRuntime` commits containers to local images, while `KubernetesSnapshotRuntime` creates a public `SandboxSnapshot` CR to commit and push images to an OCI registry. This is distinct from internal pause/resume, which releases compute while preserving sandbox continuity. Public snapshot support does not extend to every Kubernetes provider or VM runtime. ### 4.4 FastSandbox Integration In Kubernetes mode, `FastSandboxService` connects OpenSandbox to the external `fast-sandbox` platform for template-backed microVM provisioning: - **Routing**: `CompositeSandboxService` routes create requests with `templateId` to `FastSandboxService`, and routes operations on existing sandboxes by the `fsb-` ID prefix. FastSandbox is part of the Kubernetes composition, not a separate `runtime.type` value. - **Control Protocol**: Communicates with an external `fast-sandbox` control plane via gRPC FastPath v2 (port 9090). - **State**: FastPath owns mutations and live runtime operations; Kubernetes LIST/WATCH reads persisted `sandbox.fast.io/v1alpha2` resources and their observations. - **Pause/resume**: Delegates to FastPath, which restores the checkpoint on resume. Clients must resolve endpoints again after restoration. - **Networking**: The configured Fastlet pool supplies a shared egress process (one per Fastlet pod); the adapter sends network policy through FastPath Actions. This differs from attaching a per-sandbox container sidecar. The current server adapter rejects the `credentialProxy.enabled` creation flag and volume mounts on this path. The adapter and endpoint integration live in this repository; FastPath and the microVM runtime are provided by the external platform. See the [service dispatch](https://github.com/opensandbox-group/OpenSandbox/blob/main/server/opensandbox_server/services/composite_service.py) and [FastSandbox adapter](https://github.com/opensandbox-group/OpenSandbox/blob/main/server/opensandbox_server/services/fast_sandbox/service.py) for the implementation boundaries, and the [Fast Sandbox integration](/architecture/fast-sandbox/) documentation for the full design. ## 5. Sandbox Data Plane Each sandbox runs the user's image and entrypoint, with OpenSandbox control processes injected around it. ### 5.1 Execd `components/execd/` is a Go daemon built with Gin. It runs inside the sandbox and exposes the execution API. Responsibilities: - Shell command execution with SSE streaming. - Background command status and incremental log retrieval. - Persistent bash sessions. - Interactive PTY sessions over WebSocket. - File and directory operations. - Process isolation sessions for executing commands in isolated environments. - Jupyter-backed code contexts and code execution. - Local CPU/memory metrics and optional OpenTelemetry metrics export. - Optional shared access token enforcement through `X-EXECD-ACCESS-TOKEN`. In Docker, the server stages `execd` into the container and installs a bootstrap script. In Kubernetes BatchSandbox template mode, an init container copies `execd` and `bootstrap.sh` from the configured `execd_image` into an `emptyDir` volume mounted by the main sandbox container. For FastSandbox, the template build includes `execd` in the guest image before the microVM starts. ### 5.2 Code Interpreter Runtime The code-interpreter sandbox image starts Jupyter inside the sandbox. `execd` talks to Jupyter over HTTP/WebSocket and translates Jupyter kernel messages into OpenSandbox streaming events. The Code Interpreter SDKs are optional high-level clients. The lower-level execution API remains available through the sandbox SDKs and direct `execd` clients. ### 5.3 Volumes The lifecycle API exposes runtime-neutral volume models: - `host`: bind a permitted host path. - `pvc`: platform-managed named storage. Docker maps this to a Docker named volume; Kubernetes maps it to a PersistentVolumeClaim. - `ossfs`: mount Alibaba Cloud OSS through the server/runtime integration. Runtime providers validate and materialize these volume definitions differently, but the API shape stays shared. Volume mounts are supported for Docker and Kubernetes container workloads; the FastSandbox adapter currently rejects volume mounts. ### 5.4 Egress Sidecar `components/egress/` enforces outbound network policy and provides credential proxying for sandboxes. Capabilities: - FQDN and wildcard-domain allow/deny rules. - `dns` mode for DNS filtering. - `dns+nft` mode for DNS plus nftables enforcement of resolved IPs and CIDR/IP rules where supported. - Credential Vault / Proxy: transparent TLS MITM proxy (active in `dns+nft` mode) that provides scoped credential injection for outbound HTTPS requests based on host-managed secret bindings, keeping raw secrets out of the sandbox workload. - Runtime policy inspection and patching through `/policy`. - Optional sidecar authentication. - Optional platform-enforced always-allow and always-deny overlays. Docker starts the egress sidecar as a separate container and runs the main sandbox container in the sidecar network namespace. Kubernetes container providers append the egress sidecar to the pod spec and drop `NET_ADMIN` from the main sandbox container so only the sidecar mutates network rules. FastSandbox uses the shared egress process (one per Fastlet pod, outside the microVM guest), receiving policy actions through FastPath. ## 6. Networking and Access ### 6.1 Ingress `components/ingress/` is a Kubernetes-oriented HTTP/WebSocket reverse proxy. It watches sandbox resources and routes traffic to sandbox ports. Supported routing modes: - Header mode: `OpenSandbox-Ingress-To: -` or host parsing. - URI mode: `///`. - Wildcard host mode through server endpoint formatting. For `BatchSandbox`, ingress reads endpoint data from the `sandbox.opensandbox.io/endpoints` annotation. For `agent-sandbox`, it reads `status.serviceFQDN`. For FastSandbox routes, ingress asks FastPath to resolve the endpoint and forwards traffic through the returned Fastlet or central proxy address. ### 6.2 Secure Access `secureAccess` is currently supported for Kubernetes sandboxes exposed through ingress gateway mode. When enabled, the server provisions endpoint credentials and returns required headers with endpoint responses. Signed route tokens are also supported when gateway secure-access signing keys are configured. ### 6.3 Auto-Renew on Access The optional renew-intent integration extends sandbox TTL when access is observed. It can be triggered by server proxy requests or by ingress gateway events delivered through Redis. Per-sandbox opt-in is controlled by the `lifecycle.renewalPolicy` create field (OSEP-0009); the deprecated `extensions["access.renew.extend.seconds"]` key remains supported throughout 1.x. ## 7. Core Flows ### 7.1 Sandbox Creation ```text Client / SDK / CLI / MCP -> POST /v1/sandboxes -> FastAPI lifecycle server validates request and config -> selected service creates a Docker container, Kubernetes workload, or FastSandbox microVM -> runtime stages execd or uses a prepared guest image, with backend-specific networking and storage -> sandbox reaches Running or reports Failed with status reason/message ``` Creation is asynchronous from the API perspective. Clients should poll `GET /v1/sandboxes/{sandboxId}` or use SDK readiness helpers. ### 7.2 Command, File, and Code Execution ```text Client -> resolve execd endpoint from sandbox metadata or server proxy -> call execd API with X-EXECD-ACCESS-TOKEN when required -> execd runs command, file operation, session, PTY, or Jupyter code execution -> execd streams SSE/WebSocket output or returns structured responses ``` ### 7.3 Service Exposure ```text Client -> GET /v1/sandboxes/{sandboxId}/endpoints/{port} -> server returns Docker-mapped, ingress-gateway, or server-proxy endpoint -> client includes returned headers when secure access or sidecar auth requires them -> HTTP/WebSocket traffic reaches the target sandbox port ``` ### 7.4 Egress Policy ```text Create request with networkPolicy -> server validates runtime-specific network policy settings -> runtime attaches egress sidecar or binds policy actions to the shared Fastlet egress process -> sandbox outbound DNS/network traffic is filtered -> FastSandbox clients manage policy via GET/PUT/PATCH/DELETE /v1/sandboxes/{sandboxId}/networkpolicy -> container clients resolve the egress endpoint and PATCH /policy ``` ### 7.5 Pause, Resume, and Snapshots ```text Pause / resume -> lifecycle server delegates to runtime provider -> Docker pauses/resumes the sandbox container and its egress sidecar when present -> BatchSandbox uses rootfs commit/recreate or the configured QEMU VM-state restore path -> FastSandbox delegates pause/resume to external FastPath v2 service Public snapshot API -> server persists snapshot metadata -> Docker runtime commits the sandbox to a local OCI image -> Kubernetes runtime (scoped to BatchSandbox) creates a public SandboxSnapshot CR that commits and pushes to an OCI registry -> create-from-snapshot resolves that image and starts a new sandbox ``` ## 8. Design Principles ### Protocol First Public behavior starts from OpenAPI contracts in `specs/`. SDKs and clients should align to those contracts, and generated outputs should be regenerated from source specs rather than patched as the only fix. ### Control Plane vs Data Plane The lifecycle server should orchestrate and validate. Platform-specific provisioning belongs in runtime services/providers. In-sandbox operations belong in `execd` and egress sidecars. ### Runtime Neutral API, Runtime Specific Execution The lifecycle API uses shared concepts such as resource limits, volumes, endpoints, network policy, and metadata. Docker and Kubernetes can materialize those concepts differently while preserving the API contract. ### Secure Defaults with Explicit Escape Hatches The server supports API-key authentication, startup guardrails for unauthenticated mode, resource limits, capability drops, optional secure runtimes, egress controls, endpoint headers, and platform-specific network isolation. Less restrictive modes are intended for local development or explicit operator choice. ### Observable Failures Sandbox state includes `state`, `reason`, `message`, and transition time. `execd` exposes metrics, the server exposes diagnostics, ingress/egress/execd support logs and OpenTelemetry metrics where implemented, and request IDs are propagated for debugging. ## 9. Common Use Cases - **Coding agents**: run Claude Code, Gemini CLI, Codex CLI, Qwen Code, Kimi CLI, or other agent tools in isolated sandboxes. - **AI code execution**: execute model-generated code with command/file/code-interpreter APIs and streamed feedback. - **Browser automation**: run Chrome or Playwright with controlled filesystem and network behavior. - **Remote development**: expose VS Code Web, desktops, VNC, or development servers through sandbox endpoints. - **RL and evaluation workloads**: use Kubernetes BatchSandbox, Pool, and task orchestration for high-throughput sandbox delivery. - **Enterprise isolation**: combine secure runtimes, ingress, egress, endpoint access headers, and Kubernetes deployment controls. --- ### Cli Index (`docs/cli/index.md`) --- title: CLI description: The osb command-line interface for managing sandboxes, executing commands, handling files, and installing agent skills. --- # OpenSandbox CLI `osb` is the command-line interface for OpenSandbox. It is built for the common day-to-day flows: - create and manage sandboxes - run commands inside a sandbox - read and modify sandbox files - inspect runtime egress policy - manage sandbox-local Credential Vault state - collect low-level diagnostics - install OpenSandbox-specific skills for coding agents It uses the OpenSandbox Python SDK under the hood and is intended to be the shortest path from a terminal to a working sandbox workflow. ## Install ::: code-group ```bash [pip] pip install opensandbox-cli ``` ```bash [uv] uv tool install opensandbox-cli ``` ```bash [pipx] pipx install opensandbox-cli ``` ::: Confirm the install: ```bash osb --help osb --version ``` ## Before You Start Make sure an OpenSandbox server is reachable. If you are running locally, start the server first and then point the CLI at it. ```bash opensandbox-server ``` ## Quick Start ### 1. Initialize config ```bash osb config init osb config set connection.domain localhost:8080 osb config set connection.protocol http osb config set connection.api_key osb config show -o json ``` If you want a non-default config file, choose it at the root command level for the whole invocation: ```bash osb --config /tmp/dev.toml config init osb --config /tmp/dev.toml config set connection.domain localhost:8080 osb --config /tmp/dev.toml config show -o json ``` ### 2. Create a sandbox ```bash osb sandbox create --image python:3.12 --timeout 30m -o json ``` If you set defaults first, later create commands can be shorter: ```bash osb config set defaults.image python:3.12 osb config set defaults.timeout 30m osb sandbox create -o json ``` ### 3. Verify it is usable ```bash osb sandbox get -o json osb sandbox health -o json ``` ### 4. Run a command inside the sandbox Use `--` before the sandbox command payload. ```bash osb command run -o raw --argv -- python -c "print(1 + 1)" ``` ### 5. Read or write a file ```bash osb file write /tmp/hello.txt -c "hello" -o json osb file cat /tmp/hello.txt -o raw ``` ### 6. Clean up ```bash osb sandbox kill -o json ``` `kill` now returns `status: "deletion_requested"` instead of `terminated`; update scripts that match this value. It confirms request acceptance. To wait for deletion, see [cleanup completion](/architecture/control-plane/server#asynchronous-lifecycle-with-absolute-expiration). ## Common Tasks ### Create sandboxes Basic: ```bash osb sandbox create --image python:3.12 ``` Private image: ```bash osb sandbox create \ --image my-registry.example.com/team/app:latest \ --image-auth-username alice \ --image-auth-password ``` Manual cleanup mode: ```bash osb sandbox create --image python:3.12 --timeout none ``` Explicit entrypoint argv: ```bash osb sandbox create \ --image python:3.12 \ --entrypoint python \ --entrypoint -m \ --entrypoint http.server ``` Create with network policy and volumes: ```bash osb sandbox create \ --image python:3.12 \ --network-policy-file network-policy.json \ --volumes-file volumes.json ``` Create with Credential Vault proxy enabled: ```bash osb sandbox create --image python:3.12 --network-policy-file network-policy.json --credential-proxy -o json ``` Create from a template (golden image): ```bash osb sandbox create --template --timeout 30m -o json ``` Template mode fixes the workload shape on the server: `--image`, `--env`, `--resource`, `--entrypoint`, `--volumes-file`, and `--credential-proxy` cannot be combined with `--template`. Only `--timeout` (required), `--metadata`, `--extension`, and `--network-policy-file` may accompany it. Only templates in the `Succeeded` phase can be used. See [Manage templates](#manage-templates). Create from a snapshot: ```bash osb sandbox create --snapshot-id --timeout 30m -o json ``` `--image`, `--template`, and `--snapshot-id` are mutually exclusive. ### Manage templates Templates are golden-image builds; the build runs asynchronously, so poll `template get` until the status phase is `Succeeded`. Template management requires a Kubernetes-backed runtime. ```bash osb template create --image python:3.12 --publish s3://bucket/publish -o json osb template get -o json osb template list -o json osb template delete -o json ``` `template create` also accepts `--resource cpu=1 memory=512Mi disk=2Gi`, `--entrypoint` (repeat per argv item), `--env KEY=VALUE`, `--metadata KEY=VALUE`, `--format native|overlaybd`, `--readiness-probe`, and `--warmup-seconds`. ### Manage snapshots Snapshots capture a sandbox's state and can back new sandboxes. ```bash osb snapshot create --name golden -o json osb snapshot get -o json osb snapshot list --sandbox-id -o json osb snapshot delete -o json ``` ### List and inspect sandboxes ```bash osb sandbox list osb sandbox list -o json osb sandbox list --state running --state paused osb sandbox get -o json osb sandbox metrics osb sandbox metrics --watch -o raw ``` ### Pause, resume, and renew ```bash osb sandbox pause -o json osb sandbox get -o json # Repeat get with a deadline until status.state is Paused; stop on Failed. # Only then resume: osb sandbox resume --resume-timeout 60s -o json osb sandbox renew --timeout 30m -o json ``` Pause is asynchronous. A successful pause response means the request was accepted. See [Pause and Resume](/guides/pause-resume) for runtime-specific behavior. ### Expose a service ```bash osb sandbox endpoint --port 8080 -o json ``` Use the returned endpoint and headers together. The command resolves a route; it does not start a service or prove that the application is healthy. ### Run commands Foreground streaming: ```bash osb command run -o raw -- sh -lc 'echo ready' ``` Tracked background execution: ```bash osb command run --background -o json -- sh -c "sleep 10; echo done" osb command status -o json osb command logs -o json ``` By default the CLI shell-quotes each argument and joins them into a command string. To use pipelines, redirection, or sandbox-side variable expansion, pass an explicit shell and script: ```bash osb command run -o raw -- sh -c 'echo "$HOME"; printf "ready\n" | cat' ``` Use `--argv` for native execution without a shell. It requires execd support for the `argv` request field: ```bash osb command run -o raw --argv -- python3 -c "import sys; print(sys.argv[1:])" "a b" '$HOME' "x'y" "" osb command interrupt -o json ``` Foreground `command run` accepts `-o raw`; background execution accepts `table`, `json`, or `yaml`. Background execution metadata is returned only after execd confirms startup. An execution error or a stream that ends before this confirmation makes the CLI exit non-zero without printing success metadata. A successful background start does not mean the process has finished; use `command status` and `command logs` to check its eventual result. Persistent shell session: ```bash osb command session create --workdir /workspace -o json osb command session run -o raw -- pwd osb command session run -o raw -- export FOO=bar osb command session run -o raw -- sh -c 'echo $FOO' osb command session delete -o json ``` ### Open an interactive terminal `osb terminal` attaches your current terminal to a real PTY inside the sandbox (colors, `stty`, and resize all behave like a local shell). Exit the remote shell (or press Ctrl-D) to leave; the remote exit code becomes the command's exit code. ```bash osb terminal osb terminal -w /workspace ``` Requires an interactive TTY, and is supported on Unix, macOS, and Linux. If the connection drops while the shell is still running, the session is kept server-side and its id is printed so it can be inspected or removed via the execd API; a clean exit tears the session down. ### Work with files ```bash osb file upload ./local.txt /workspace/local.txt -o json osb file download /workspace/result.json ./result.json -o json osb file search /workspace --pattern "*.py" -o json osb file info /workspace/main.py -o json osb file replace /workspace/app.py --old old --new new -o json osb file chmod /workspace/script.sh --mode 755 -o json ``` For regular files, `file download` replaces the local destination only after the entire download succeeds. If the download fails or you interrupt it, an existing file stays unchanged and temporary download files are removed. The destination directory must be writable so the CLI can stage the download before replacing the file. Existing devices (such as `/dev/null`) and named pipes receive the download directly. They are not replaced, and bytes already written cannot be rolled back if the download fails or is interrupted. Destinations that refer to standard output, such as `/dev/stdout` and `/dev/fd/1`, stream directly even when stdout is redirected to a regular file. These downloads omit the success message in all output formats so stdout contains only file bytes; errors still go to stderr. A failed or interrupted stream can contain partial data. ### Manage runtime egress policy Inspect current policy: ```bash osb egress get -o json ``` Patch specific rules: ```bash osb egress patch --rule allow=pypi.org --rule deny=internal.example.com -o json ``` If you are debugging connectivity, verify behavior with an actual command: ```bash osb command run -o raw -- curl -I https://pypi.org ``` ### Manage Credential Vault Credential Vault operations call the sandbox egress sidecar through the Python SDK. Create the sandbox with `--credential-proxy` and an explicit network policy before writing vault state. Template-backed sandboxes do not support Credential Vault. See [Credential Vault](/guides/credential-vault) for payload examples and revision checks. ```bash osb credential-vault create --file vault.yaml -o json osb credential-vault get -o json osb credential-vault patch --file mutation.yaml -o json osb credential-vault credential list -o json osb credential-vault binding list -o json osb credential-vault delete -o json ``` ::: warning Use `--file -` to read a JSON/YAML payload from stdin. Do not pass plaintext credential values as command-line flags; keep them in the payload stream or file. ::: ### Collect diagnostics Use context to identify the runtime backend, then collect log and event descriptors. ```bash osb diagnostics context -o json osb diagnostics events --scope runtime -o raw osb diagnostics events --scope all -o raw osb diagnostics logs --scope container -o raw osb diagnostics logs --scope all -o json osb diagnostics events --scope runtime -o json osb diagnostics logs --scope container -o yaml ``` `context` returns backend routing information without connecting to execd and supports table, JSON, and YAML output. `--scope` is required for logs/events. The built-in server supports `container` and `all` for logs, and `runtime` and `all` for events. Docker/Kubernetes return `DIAGNOSTICS_SCOPE_UNSUPPORTED` for unavailable scopes, including lifecycle events. Best-effort scopes may include a `warnings` field when the backend can only provide a subset. Raw output prints inline diagnostic text, or the content URL when diagnostics are delivered as a temporary URL; raw output does not download that URL. JSON/YAML use Python model names such as `content_url`, `content_type`, and `expires_at` (the HTTP API uses camelCase). See [Diagnostics](/api/#diagnostics) for runtime differences. Older server builds may still return `DIAGNOSTICS_NOT_IMPLEMENTED` for scoped diagnostics. ::: info Legacy DevOps diagnostics remain experimental. Prefer `osb diagnostics logs/events` for stable API-backed log and event collection. ::: ```bash osb devops inspect -o raw osb devops summary -o raw ``` ## Output Formats Output selection is command-scoped, not global. - `table`: human-readable tables and panels - `json`: machine-readable JSON - `yaml`: machine-readable YAML - `raw`: unformatted text or streaming output Examples: ```bash osb sandbox list -o json osb sandbox list -o yaml osb file cat /workspace/hello.txt -o raw ``` Not every command supports every format. Use `--help` on the specific command when in doubt. ## Command Groups The main command groups are: - `osb sandbox`: lifecycle management - `osb template`: golden-image template builds - `osb snapshot`: snapshot management - `osb command`: command execution and persistent sessions - `osb terminal`: interactive PTY terminal into a sandbox - `osb file`: file and directory operations - `osb egress`: runtime egress policy - `osb credential-vault`: sandbox-local credentials and bindings - `osb diagnostics`: stable diagnostics logs and events - `osb devops`: experimental legacy diagnostics - `osb config`: local CLI configuration - `osb skills`: bundled skills for AI tools The CLI does not provide Client Pool or pool tracing commands; use the SDK/API for those workflows. A feature in the Python SDK is not automatically a CLI command. Explore them directly: ```bash osb sandbox --help osb template --help osb snapshot --help osb command --help osb file --help osb skills --help ``` ## Agent Skills The CLI ships with built-in OpenSandbox skills for coding agents and agent-oriented tools. Bundled skills: - `sandbox-lifecycle` - `command-execution` - `file-operations` - `network-egress` - `sandbox-troubleshooting` Supported targets: | Target | Install location | | --- | --- | | `claude` | `./.claude/skills//SKILL.md` or `~/.claude/skills//SKILL.md` | | `cursor` | `./.cursor/rules/` or `~/.cursor/rules/` | | `codex` | `./.codex/skills//SKILL.md` or `~/.codex/skills//SKILL.md` | | `copilot` | `./.github/copilot-instructions.md` or `~/.copilot/copilot-instructions.md` | | `windsurf` | `./.windsurfrules` or `~/.windsurfrules` | | `cline` | `./.clinerules` or `~/.cline/rules/opensandbox-.md` | | `opencode` | `./.agents/skills//SKILL.md` or `~/.agents/skills//SKILL.md` | Common flows: ```bash osb skills list osb skills show sandbox-lifecycle osb skills install sandbox-lifecycle --target codex --scope project osb skills install --all-builtins --target codex --scope global osb skills uninstall sandbox-troubleshooting --target claude --scope global ``` For scripts or agents, use structured output: ```bash osb skills install sandbox-lifecycle --target codex --scope project -o json ``` ## Configuration Model The CLI resolves configuration in this order: 1. root CLI flags such as `--api-key`, `--domain`, `--protocol`, `--request-timeout`, `--config` 2. environment variables such as `OPEN_SANDBOX_API_KEY` and `OPEN_SANDBOX_DOMAIN` 3. config file, defaulting to `~/.opensandbox/config.toml` 4. SDK defaults Config commands: ```bash osb config init osb config show osb config set connection.domain localhost:8080 osb config set connection.protocol http osb config set defaults.image python:3.12 osb config set defaults.timeout 30m ``` Example config file: ```toml [connection] api_key = "your-api-key" domain = "localhost:8080" protocol = "http" request_timeout = 30 use_server_proxy = false [output] color = true [defaults] image = "python:3.12" timeout = "30m" ``` ## Development For local development in this monorepo: ```bash cd cli uv sync uv run osb --help uv run pytest ``` This repository uses a local `uv` source override for the OpenSandbox Python SDK, so running from `cli/` will resolve against the checked-out SDK in the monorepo. --- ### Deployment Index (`docs/deployment/index.md`) --- title: Kubernetes Deployment description: Deploy OpenSandbox on Kubernetes — CRDs, controller, lifecycle server, and optional components, with Helm deployment order and configuration. --- # Kubernetes Deployment This guide covers deploying OpenSandbox on Kubernetes: the cluster foundation (CRDs), the controller, the lifecycle server, and the optional components (ingress gateway, node agent, fast-sandbox runtime). Charts are versioned sources in the [OpenSandbox repository](https://github.com/opensandbox-group/OpenSandbox/tree/main/manifests/charts). They are installed from a checkout of a `release-X.Y.Z` tag — standalone chart packages are not published. ## Prerequisites - Kubernetes 1.21.1+ - Helm 3.x - `kubectl` configured for your cluster ## Deployment Order Install the charts in this order: ```text base → opensandbox-controller → fast-sandbox* → ingress-gateway → opensandbox-server → optional components ``` \* `fast-sandbox` is optional, but when used it must be installed **before** the ingress gateway and the server. `ingress-gateway` is required for Kubernetes deployments — sandbox Pods are ClusterIP-only, so client traffic routes through the gateway. It is installed **before the server** so the server announces it from the first install. | Step | Chart | Why it comes here | |------|-------|-------------------| | 1 | `base` | Owns the `sandbox.opensandbox.io` CRDs, the fast-sandbox CRDs (`sandbox.fast.io`), and their RBAC. Everything else depends on these objects existing. Install once per cluster. | | 2 | `opensandbox-controller` | Reconciles `BatchSandbox`, `Pool`, and `SandboxSnapshot` objects created by you and by the server. Requires the CRDs from `base`. | | 3 | `fast-sandbox` (optional) | Firecracker runtime (`sandbox.fast.io`). Must be installed **before the server**: the server's `[runtime]`/fsb configuration points at the FastPath gRPC endpoint (`fast-sandbox-fastpath...svc:9090`) and watches `sandbox.fast.io` objects at startup. Also consumes the ServiceAccounts from `base` — keep the namespace values in sync with it. Skip if you only use the default Kubernetes runtime. | | 4 | `ingress-gateway` | Required for Kubernetes deployments: sandbox Pods are ClusterIP-only and client traffic routes through the gateway. Installed before the server so the announcement is configured from the first install. When serving sandboxes through the fsb runtime, point `gateway.fastpathEndpoint` at the FastPath service from the previous step. | | 5 | `opensandbox-server` | The lifecycle REST API that creates and deletes sandboxes. Requires the CRDs from `base`, a running controller, and — when serving sandboxes through the fsb runtime — the FastPath endpoint from the fast-sandbox release. Announces the ingress gateway through `server.gateway.*`. | | 6 | `opensandbox-node-agent` (optional) | Node-level sandbox data collection. Order-independent. | If you use the umbrella chart, this order is handled for you in a single release. ## Install Check out the version you want to deploy: ```bash git clone https://github.com/opensandbox-group/OpenSandbox.git cd OpenSandbox git checkout release-1.1.0 # or main for development ``` ### Option 1: Umbrella chart (recommended) One release installs every component with the correct ordering: ```bash cd manifests/charts # Package the sub-charts (charts/ is git-ignored, rebuilt every time) helm dependency build opensandbox helm install opensandbox opensandbox \ --namespace opensandbox-system \ --create-namespace ``` Optional components default to off. For Kubernetes deployments, enable the ingress gateway — sandbox Pods are ClusterIP-only and client traffic routes through it: ```bash helm install opensandbox opensandbox \ --namespace opensandbox-system \ --create-namespace \ --set ingress-gateway.enabled=true ``` ### Option 2: Per-component releases ```bash # 1. Cluster foundation: CRDs + RBAC helm install base manifests/charts/base # 2. Controller helm install opensandbox-controller manifests/charts/controller \ --namespace opensandbox-system \ --create-namespace # 3. Ingress gateway (required on Kubernetes; see Deployment Order) helm install ingress-gateway manifests/charts/ingress-gateway \ --namespace opensandbox-system # 4. Lifecycle server helm install opensandbox-server manifests/charts/server \ --namespace opensandbox-system \ --create-namespace ``` ### Configure API authentication By default, the server refuses to start without an API key in a non-interactive container. Create both the control-plane namespace and the default sandbox workload namespace, then store the key in a Kubernetes `Secret`: ```bash kubectl create namespace opensandbox-system --dry-run=client -o yaml | kubectl apply -f - kubectl create namespace opensandbox --dry-run=client -o yaml | kubectl apply -f - read -s OPENSANDBOX_API_KEY kubectl create secret generic opensandbox-api-key \ --namespace opensandbox-system \ --from-literal=api-key="${OPENSANDBOX_API_KEY}" \ --dry-run=client -o yaml | kubectl apply -f - unset OPENSANDBOX_API_KEY ``` Reference the Secret from a values file: ```yaml # values-server.yaml server: replicaCount: 1 env: - name: OPENSANDBOX_SERVER_API_KEY valueFrom: secretKeyRef: name: opensandbox-api-key key: api-key ``` Use an external secret manager instead of creating the Secret manually in production environments. The chart installs the server into `opensandbox-system`, while the default `configToml` creates sandbox and pool resources in `opensandbox`. If you change `[kubernetes].namespace` in `configToml`, create that namespace instead of `opensandbox` before submitting workloads. ::: warning Single-active Server default The chart defaults to `server.replicaCount: 1`. Keep one active Lifecycle Server unless you deliberately use the PostgreSQL-backed Kubernetes public snapshot topology documented below. That exception coordinates public snapshots only; it does not provide general multi-replica Server HA. The Server Deployment uses the `Recreate` strategy so an upgrade stops the active Server before starting its replacement; expect a brief API interruption during upgrades. ::: ### Use PostgreSQL for server persistence Create a Secret containing the PostgreSQL connection string: ```bash read -s OPENSANDBOX_POSTGRESQL_DSN kubectl create secret generic opensandbox-postgresql \ --namespace opensandbox-system \ --from-literal=dsn="${OPENSANDBOX_POSTGRESQL_DSN}" \ --dry-run=client -o yaml | kubectl apply -f - unset OPENSANDBOX_POSTGRESQL_DSN ``` In `values-server.yaml`, keep the default `server.replicaCount` at `1`, add the Secret-backed environment variable below, and add the shown `[store]` tables to the complete `configToml` value: ```yaml server: replicaCount: 1 env: - name: OPENSANDBOX_STORE_POSTGRESQL_DSN valueFrom: secretKeyRef: name: opensandbox-postgresql key: dsn configToml: | # Keep the rest of the chart's complete server configuration here. [store] type = "postgresql" [store.postgresql] min_pool_size = 1 max_pool_size = 10 snapshot_recovery_interval_seconds = 15 ``` ::: info The chart default remains one Server replica. You may explicitly set `server.replicaCount: 2` for multi-active public snapshot handling only when both replicas use the same PostgreSQL database and the Kubernetes runtime. SQLite and Docker snapshot execution do not support this multi-active topology. ::: ### Install the server with the API key and verify Install the server referencing your values file: ```bash helm install opensandbox-server manifests/charts/server \ --namespace opensandbox-system \ --create-namespace \ --values values-server.yaml ``` Wait for the Deployment and verify the API health endpoint: ```bash kubectl rollout status deployment/opensandbox-server \ --namespace opensandbox-system \ --timeout=180s kubectl port-forward \ --namespace opensandbox-system \ service/opensandbox-server 8080:80 ``` In another terminal: ```bash curl --fail http://127.0.0.1:8080/health ``` ### Important values | Value | Purpose | Notes | |-------|---------|-------| | `server.image.repository` | Server image registry and repository | Override for a private mirror or custom build. | | `server.image.tag` | Server image version | Defaults to the `release-` image published with the chart version; override only for custom builds. | | `server.replicaCount` | Number of server Pods | Defaults to `1`; general multi-replica Server HA is not supported yet. | | `server.env` | Additional container environment variables | Use it with `secretKeyRef` for `OPENSANDBOX_SERVER_API_KEY`. | | `configToml` | Complete server configuration | Mounted at `/etc/opensandbox/config.toml`; overriding it replaces the complete default TOML, including the workload namespace. | | `server.gateway.enabled` | Announce an ingress gateway to clients | Defaults to `false`. The gateway itself is deployed by the [ingress-gateway chart](https://github.com/opensandbox-group/OpenSandbox/tree/main/manifests/charts/ingress-gateway). | | `server.service.type` | Service type for the server | Defaults to `ClusterIP`. Use `NodePort` or `LoadBalancer` for access from outside the cluster; pin the port with `server.service.nodePort`. | | `namespaceOverride` | Namespace used by chart resources | Defaults to `opensandbox-system`. | The server container and its Service use port `80`. Keep `[server].port = 80` when replacing `configToml` unless the chart templates are also updated to use a different port. The Service is `ClusterIP` by default; set `server.service.type` to reach the server from outside the cluster. ### Configure egress sidecar resources When a create request includes `networkPolicy`, the lifecycle server adds an egress sidecar to each non-pooled sandbox Pod. Namespace `LimitRange` defaults apply to this container when it does not declare resources, which can reserve substantially more capacity than basic DNS/nft enforcement needs. Add optional resource settings to the `[egress]` section of `configToml`: ```toml [egress] image = "opensandbox/egress:release-1.1.0" requests = { cpu = "25m", memory = "64Mi" } limits = { cpu = "250m", memory = "256Mi" } ``` You can omit either `requests` or `limits`. Treat these values as a starting point and tune them from observed usage; Credential Vault and transparent mitmproxy generally need more headroom than basic DNS/nft enforcement. ### Ingress gateway (required on Kubernetes) Sandbox Pods on Kubernetes are ClusterIP-only — client traffic reaches sandboxes through the ingress gateway (`[ingress] mode = "gateway"`, the Kubernetes-oriented mode; the Docker runtime uses `direct` and does not need the gateway). Install the gateway **before the server**, then include the announcement in the server install: ```bash helm install ingress-gateway manifests/charts/ingress-gateway \ --namespace opensandbox-system \ --set gateway.fastpathEndpoint=fast-sandbox-fastpath.opensandbox-system.svc:9090 ``` Install the server with the announcement enabled (or `helm upgrade` an existing server release with the same flags): ```bash helm install opensandbox-server manifests/charts/server \ --namespace opensandbox-system \ --set server.gateway.enabled=true \ --set server.gateway.host=gateway.example.com \ --values values-server.yaml ``` Keep `server.gateway.gatewayRouteMode` in sync with `gateway.gatewayRouteMode` of the gateway chart. For signed, expiring sandbox routes, configure the shared secure-access key ring on both charts — see [secure-access keys](https://github.com/opensandbox-group/OpenSandbox/blob/main/manifests/HELM-DEPLOYMENT.md#secure-access-keys-osep-0011). ### Optional: fast-sandbox runtime The `fast-sandbox` chart adds the Firecracker (`sandbox.fast.io`) runtime. Install it **before the ingress gateway and the lifecycle server** (see [Deployment Order](#deployment-order)): the server's `[runtime]`/fsb configuration points at the FastPath gRPC endpoint this chart creates. It also requires `base` first, KVM-capable nodes, and companion images built from a pinned upstream commit — see the [fast-sandbox runtime deployment guide](https://github.com/opensandbox-group/OpenSandbox/blob/main/manifests/HELM-DEPLOYMENT.md#fast-sandbox-runtime-firecracker). ## Upgrade Upgrade the umbrella release from a newer checkout: ```bash git fetch --tags git checkout release-1.2.0 cd manifests/charts helm dependency build opensandbox helm upgrade opensandbox opensandbox --namespace opensandbox-system ``` For a per-component server release: ```bash git checkout release-1.2.0 helm upgrade opensandbox-server manifests/charts/server \ --namespace opensandbox-system \ --values values-server.yaml ``` For the complete values reference and local development installation, see the [`opensandbox-server` chart README](https://github.com/opensandbox-group/OpenSandbox/tree/main/manifests/charts/server). ## Operator Metrics The operator (controller-manager) exposes standard [controller-runtime](https://book.kubebuilder.io/reference/metrics) Prometheus metrics — reconcile rate and latency (`controller_runtime_reconcile_*`), work-queue depth, client-go request counts, and Go runtime stats. The endpoint is **disabled by default** (`--metrics-bind-address=0`). Enable it through the `opensandbox-controller` chart values: | Value | Default | Purpose | |-------|---------|---------| | `controller.metrics.enabled` | `false` | Expose the `/metrics` endpoint (sets `--metrics-bind-address`) | | `controller.metrics.port` | `8080` | Port for the metrics endpoint | | `controller.metrics.secure` | `false` | Serve over HTTPS with authn/authz (`--metrics-secure`); set `false` for plain HTTP scraping | ```yaml controller: metrics: enabled: true port: 8080 secure: false # plain HTTP, e.g. for a PodMonitoring/ServiceMonitor scrape ``` - With `secure: false` the endpoint is plain HTTP and can be scraped directly (no TLS or bearer token). - With `secure: true` the controller-runtime filter authenticates and authorizes each scrape via `TokenReview`/`SubjectAccessReview`. The chart then provisions two `ClusterRole`s automatically: - `opensandbox-metrics-auth-role` (bound to the manager) — lets the controller run the auth checks. - `opensandbox-metrics-reader` (**not** bound by the chart) — grants `get` on the `/metrics` non-resource URL. Bind it to your scraper's `ServiceAccount` (e.g. Prometheus) and have the scraper present that account's bearer token. Point your Prometheus stack at the `metrics` container port (for example via a `ServiceMonitor` or `PodMonitoring`). ### Business capacity metrics The elected controller also exports low-cardinality business capacity metrics over OTLP/HTTP when `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` or `OTEL_EXPORTER_OTLP_ENDPOINT` is set. This is independent of the controller-runtime Prometheus endpoint and remains disabled when neither variable is configured. ```yaml extraEnv: - name: OTEL_EXPORTER_OTLP_METRICS_ENDPOINT value: http://otel-collector.observability:4318/v1/metrics ``` | Metric | Unit | Attributes | Description | |--------|------|------------|-------------| | `controller.pool.pods` | `{pod}` | `namespace`, `pool_name`, `state` | Current Pool Pods, where `state` is `total`, `allocated`, `available`, or `updated` | | `controller.pool.cpu.requested` | `{cpu}` | `namespace`, `pool_name`, `state` | Scheduler-equivalent CPU requests represented by total, allocated, or available Pool Pods | | `controller.pool.memory.requested` | `By` | `namespace`, `pool_name`, `state` | Scheduler-equivalent memory requests represented by total, allocated, or available Pool Pods | | `controller.batchsandbox.count` | `{batchsandbox}` | `namespace`, `phase`, `allocation_mode` | Current BatchSandbox objects by lifecycle phase and pool/direct mode | | `controller.batchsandbox.pods` | `{pod}` | `namespace`, `state`, `allocation_mode` | Desired, current, allocated, and ready BatchSandbox Pod counts | | `controller.capacity.collect.duration` | `s` | None | Time spent reading cached objects and collecting one capacity snapshot | The metrics deliberately omit sandbox, BatchSandbox, and Pod identifiers. Only the leader exports them, so multiple controller replicas do not duplicate cluster totals. An unset initial BatchSandbox phase is exported as `Unknown`. Derive Pool utilization from `allocated / total` and calculate peak, valley, or percentile capacity in the telemetry backend. Actual CPU and memory usage remains available from kubelet/cAdvisor rather than being duplicated here. Each collection reads all Pools and BatchSandboxes from the controller manager's informer cache, then performs one cached, owner-UID-indexed Pod list for every non-deleting Pool. Collection CPU and memory therefore grow linearly with the number of cached Pools, BatchSandboxes, and Pool-owned Pods, without issuing one API-server list request per Pool. The OpenTelemetry periodic reader exports every 60 seconds by default; `OTEL_METRIC_EXPORT_INTERVAL` can change the interval in milliseconds. Monitor `controller.capacity.collect.duration` and validate the target cluster scale before shortening that interval. If OTLP setup fails after an endpoint is configured, the controller continues reconciling and logs the failed setup stage together with the endpoint environment variable and a credential-stripped endpoint. ## Configure the Server for Kubernetes Generate a Kubernetes-oriented server config: ```bash opensandbox-server init-config ~/.sandbox.toml --example k8s ``` Key Kubernetes-specific configuration sections: | Section | Purpose | |---------|---------| | `[kubernetes]` | Workload provider, BatchSandbox template file | | `[agent_sandbox]` | Agent sandbox settings | | `[ingress]` | Ingress gateway for sandbox traffic routing | | `[secure_runtime]` | Secure container runtime (gVisor, Kata) | See [Configuration](/getting-started/configuration) for the full reference. ## Components on Kubernetes | Component | Deployment | Purpose | |-----------|-----------|---------| | CRDs + RBAC (`base`) | Cluster-scoped | `BatchSandbox`, `Pool`, `SandboxSnapshot` and `sandbox.fast.io` API types | | Server | Deployment | Lifecycle control plane | | Controller (operator) | Deployment | Manages BatchSandbox/Pool CRDs | | Ingress gateway | Deployment | Routes traffic to sandboxes; required on Kubernetes deployments | | Egress | Sidecar | Per-sandbox egress policy enforcement | | Execd | Built into sandbox images | In-sandbox execution | | Node agent | DaemonSet | Optional node-level sandbox data collection | | fast-sandbox | Deployment + DaemonSet | Optional Firecracker control plane and node runtime | ## Uninstall ```bash helm uninstall opensandbox -n opensandbox-system ``` CRDs carry the `helm.sh/resource-policy: keep` annotation and are retained across uninstalls; the `opensandbox-dataplane` namespace (created by `base` for fast-sandbox) is kept as well. Delete them manually once their data is no longer needed: ```bash kubectl delete crd batchsandboxes.sandbox.opensandbox.io \ pools.sandbox.opensandbox.io \ sandboxsnapshots.sandbox.opensandbox.io kubectl delete namespace opensandbox-dataplane ``` ## Related - [Kubernetes Overview](/architecture/control-plane/operator) — Operator features and CRDs - [Pause & Resume](/guides/pause-resume) — Snapshot-based pause/resume on Kubernetes - [Secure Container](/guides/secure-container) — gVisor and Kata on Kubernetes - [Network Isolation on Kubernetes](/guides/network-isolation) — Egress policy design for Kubernetes - [Helm charts source](https://github.com/opensandbox-group/OpenSandbox/tree/main/manifests/charts) — Chart sources and values reference --- ### Examples Index (`docs/examples/index.md`) --- title: Examples description: Real-world usage examples for OpenSandbox covering coding agents, browser automation, desktop environments, and training workloads. --- # Examples OpenSandbox provides ready-to-run examples covering SDK usage, agent integrations, browser automation, and training workloads. ::: tip All example source code is available in the [`examples/`](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples) directory on GitHub. ::: ## Coding Agents Run coding CLIs and AI agent frameworks inside isolated sandboxes. | Example | Description | |---------|-------------| | [Claude Code](/examples/claude-code) | Run Claude Code CLI in a sandbox | | [Gemini CLI](/examples/gemini-cli) | Run Gemini CLI in a sandbox | | [Codex CLI](/examples/codex-cli) | Run OpenAI Codex CLI in a sandbox | | [Apply Model Patches](/examples/apply-patch) | Apply Codex-style file patches with a custom sandbox image | | [OpenCode](/examples/opencode) | Run the OpenCode coding agent in a sandbox | | [Qwen Code](/examples/qwen-code) | Run Qwen Code CLI in a sandbox | | [Kimi CLI](/examples/kimi-cli) | Run Kimi CLI (Moonshot AI) in a sandbox | | [LangGraph](/examples/langgraph) | LangGraph state-machine workflow with sandbox | | [Deep Agents](/examples/deep-agents) | Deep Agents file/shell tools running in a sandbox | | [Google ADK](/examples/google-adk) | Google ADK agent using OpenSandbox tools | | [OpenClaw](/examples/openclaw) | OpenClaw Gateway inside a sandbox | | [DeerFlow](/examples/deer-flow) | DeerFlow agent turns with tools running in a sandbox | ## Browser & Desktop Execute browser workloads and host desktop environments in sandboxes. | Example | Description | |---------|-------------| | [Chrome](/examples/chrome) | Chromium sandbox with VNC and DevTools | | [Playwright](/examples/playwright) | Playwright + Chromium headless testing | | [Desktop](/examples/desktop) | Full desktop environment with VNC | | [VS Code](/examples/vscode) | VS Code Web (code-server) in a sandbox | ## Core Usage Fundamental sandbox operations and SDK workflows. | Example | Description | |---------|-------------| | [Code Interpreter](/examples/code-interpreter) | End-to-end Code Interpreter SDK workflow | | [Scheduled Client Pool](/examples/client-pool-schedule) | Adjust idle capacity by business hours using Go's pool Resize API | | [AIO Sandbox](/examples/aio-sandbox) | All-in-One sandbox setup | | [Agent Sandbox](/examples/agent-sandbox) | Kubernetes agent-sandbox integration | | [AKS + Kata](/examples/aks-kata) | AKS deployment with Kata VM isolation, ingress, egress, and Credential Vault | | [Windows](/examples/windows) | Windows sandbox via KVM/QEMU | | [Harbor Evaluation](/examples/harbor-evaluation) | Run a Harbor agent evaluation, one sandbox per trial | ## Storage Persistent and shared storage patterns for sandboxes. | Example | Description | |---------|-------------| | [Host Volume Mount](/examples/host-volume-mount) | Mount host directories into sandboxes | | [Docker PVC Volume](/examples/docker-pvc-volume-mount) | Docker named volume mounts | | [Docker OSSFS Volume](/examples/docker-ossfs-volume-mount) | Docker OSSFS (OSS FUSE) mounts | | [rclone Volume Mount](/examples/rclone-volume-mount) | Remote storage through external Docker plugins or Kubernetes CSI volumes | | [Kubernetes PVC](/examples/kubernetes-pvc-volume-mount) | Kubernetes PersistentVolumeClaim mounts | ## How to Run 1. Start the OpenSandbox server (see [Quick Start](/getting-started/)) 2. Set environment variables: `export SANDBOX_DOMAIN=...`, `export SANDBOX_API_KEY=...` 3. Add provider-specific variables as needed (e.g., `ANTHROPIC_AUTH_TOKEN`, `OPENAI_API_KEY`) 4. Navigate to the example directory and run: `python main.py` ::: tip Most examples include a `main.py` entry point. Follow each example's run instructions for other languages. Some also include a `Dockerfile` for containerized execution. ::: --- ### Getting Started Index (`docs/getting-started/index.md`) --- title: Quick Start description: Get OpenSandbox running locally in minutes with Docker and Python. --- # Quick Start OpenSandbox is a general-purpose sandbox platform for AI applications, offering multi-language SDKs, unified sandbox APIs, and Docker/Kubernetes runtimes. ## Prerequisites - **Docker** Engine 20.10+ (for local execution) - **Python** 3.10+ (for the server and Python SDK) - **uv** (recommended) or pip ## 1. Start the Server ```bash # Generate a starter config uvx opensandbox-server init-config ~/.sandbox.toml --example docker # Start the server uvx opensandbox-server ``` Verify the server is running: ```bash curl http://127.0.0.1:8080/health # → {"status": "healthy"} ``` ## 2. Install an SDK ::: code-group ```bash [Python] pip install opensandbox ``` ```bash [JavaScript] npm install @alibaba-group/opensandbox ``` ```bash [Go] go get github.com/alibaba/OpenSandbox/sdks/sandbox/go ``` ```bash [C#] dotnet add package Alibaba.OpenSandbox ``` ::: For Kotlin/Java, see [Installation](/getting-started/installation). ## 3. Create and Use a Sandbox This example uses the sandbox SDK and a standard Python image. If the server has an API key configured, set `OPEN_SANDBOX_API_KEY` in your client environment. ```python import asyncio from datetime import timedelta from opensandbox import Sandbox from opensandbox.config import ConnectionConfig from opensandbox.models import WriteEntry async def main() -> None: sandbox = await Sandbox.create( "python:3.12", timeout=timedelta(minutes=10), connection_config=ConnectionConfig( domain="localhost:8080", use_server_proxy=True, ), ) try: execution = await sandbox.commands.run("python -c 'print(1 + 1)'") for output in execution.logs.stdout: print(output.text) await sandbox.files.write_files([ WriteEntry(path="/tmp/hello.txt", data="Hello World", mode=644) ]) print(await sandbox.files.read_file("/tmp/hello.txt")) finally: await sandbox.destroy() if __name__ == "__main__": asyncio.run(main()) ``` `use_server_proxy=True` routes sandbox traffic through the lifecycle server, which is useful when the client cannot reach container addresses directly. `destroy()` terminates the remote sandbox and closes local SDK resources. Closing the client alone does not terminate the remote sandbox. ## 4. Try the CLI ```bash pip install opensandbox-cli osb config init osb config set connection.domain localhost:8080 osb config set connection.protocol http osb config set connection.use_server_proxy true osb sandbox create --image python:3.12 --timeout 30m -o json osb command run -o raw --argv -- python -c "print(1 + 1)" osb sandbox kill -o json ``` ## Next Steps - [Installation](/getting-started/installation) — Detailed installation for all SDKs and runtimes - [Configuration](/getting-started/configuration) — Server configuration reference - [Architecture](/architecture/) — How OpenSandbox works under the hood - [Guides](/guides/) — Client Pool, tracing, diagnostics, and security features - [SDK capability matrix](/sdks/#capability-coverage) — Check support before choosing a language - [Examples](/examples/) — Real-world usage examples --- ### Guides Index (`docs/guides/index.md`) --- title: Feature Guides description: Guides to sandbox lifecycle, isolation, networking, and security. --- # Feature Guides Use these guides to add capabilities to a working [sandbox application](/getting-started/). Check the [SDK capability matrix](/sdks/#capability-coverage) for language support. ## Lifecycle and security - [Pause and resume](/guides/pause-resume) - [Lifecycle hooks](/guides/lifecycle-hooks) - [Isolation sessions](/guides/isolation-sessions) - [Credential Vault](/guides/credential-vault) - [Chained upstream proxy](/guides/egress-upstream-proxy) - [Secure access](/guides/secure-access) - [Secure container runtimes](/guides/secure-container) - [Network isolation on Kubernetes](/guides/network-isolation) - [Multi-tenancy](/guides/multi-tenancy) - [Windows sandboxes](/guides/windows-sandbox) ## Operations - [Telemetry](/guides/telemetry) --- ### Sdks Index (`docs/sdks/index.md`) --- title: SDKs description: Choose an OpenSandbox SDK and compare lifecycle, Client Pool, tracing, and diagnostics support. --- # SDKs OpenSandbox provides five sandbox SDKs for lifecycle management, command execution, file operations, and resource metrics. This site follows the repository's default branch; a feature merged here may require a newer package than the one you have installed. Check the [release index](https://github.com/opensandbox-group/OpenSandbox/releases) when upgrading. ## Installation | Language | Package | Install | | --- | --- | --- | | [Python](/sdks/python) | `opensandbox` | `pip install opensandbox` | | [JavaScript/TypeScript](/sdks/javascript) | `@alibaba-group/opensandbox` | `npm install @alibaba-group/opensandbox` | | [Kotlin/Java](/sdks/kotlin) | `com.alibaba.opensandbox:sandbox` | Gradle/Maven | | [Go](/sdks/go) | `github.com/alibaba/OpenSandbox/sdks/sandbox/go` | `go get github.com/alibaba/OpenSandbox/sdks/sandbox/go` | | [C#/.NET](/sdks/csharp) | `Alibaba.OpenSandbox` | `dotnet add package Alibaba.OpenSandbox` | ## Capability coverage “Yes” means a public SDK entry point exists. Availability also depends on the server runtime and the execd version installed in the sandbox. | Capability | Python async/sync | JavaScript/TypeScript | Kotlin/Java | Go | C#/.NET | | --- | --- | --- | --- | --- | --- | | Create, connect, renew, pause/resume, kill | Yes | Yes | Yes | Yes | Yes | | Snapshots and template management | Yes | Yes | Yes | Yes | Yes | | Resource requests, volumes, lifecycle hooks | Yes | Yes | Yes | Yes | Yes | | Commands, files, resource metrics | Yes | Yes | Yes | Yes | Yes | | Egress policy and Credential Vault | Yes | Yes | Yes | Yes | Yes | | Isolated sessions | Yes | Yes | Yes | Yes | Yes | | [Client Pool](/guides/client-pool), including Redis | Yes | Yes | Yes | Yes | No | | [Pool warmup tracing](/sdks/observability#pool-warmup-tracing) | Yes | Yes, fewer attributes | Yes | No | No | | [Diagnostic context, logs/events](/api/#diagnostics) | Yes | Yes | Yes | Yes | Yes | | [Create-latency telemetry](/sdks/observability#creation-metrics) | Yes | Yes | Yes | Yes | Yes | ### Differences that affect application code - **Pools:** Go exposes different warmup controls; the fields accepted by the default pool creator also differ across languages. See the [pool configuration matrix](/guides/client-pool#configuration). - **Go command control:** interrupt, command status, and accumulated command logs are exposed by `ExecdClient`, rather than the high-level `Sandbox` wrapper. Neither `Sandbox.CreateSession` nor `ExecdClient.CreateSession` accepts an initial working directory. Set `Cwd` when running a command in the session instead. - **Go connect/resume readiness:** pass `ReadyOptions` to request readiness checks. Without it, connecting resolves the endpoint without checking sandbox health. - **Metrics streaming:** Go exposes `ExecdClient.WatchMetrics`; the other SDKs' stable metrics services provide point-in-time reads. The [CLI](/cli/) also has a metrics stream via `osb sandbox metrics --watch`. - **Timeout units:** use Python `timedelta` and JVM `Duration`; JavaScript and C# command timeouts use seconds. Go `RunCommandRequest.Timeout` uses **milliseconds**. Template-backed sandboxes require an explicit TTL and inherit their workload configuration from the published template. Their egress policy is managed through the lifecycle API; they do not have a sandbox-side Credential Vault. ## Feature guides - [Client Pool](/guides/client-pool): keep a ready buffer, select an acquire policy, share state through Redis, and retire a pool namespace. - [Observability](/sdks/observability): configure pool warmup traces and creation metrics, understand the default settings, and locate slow startup phases. ## Diagnostics All sandbox SDKs expose diagnostic context, logs, and events on both `Sandbox` and `SandboxManager`. Use a manager when execd is not ready: these requests go to the lifecycle server without connecting to the sandbox. | Language | Context | Logs / events | | --- | --- | --- | | Python async/sync | `get_diagnostic_context` | `get_diagnostic_logs` / `get_diagnostic_events` | | Kotlin/Java, JavaScript | `getDiagnosticContext` | `getDiagnosticLogs` / `getDiagnosticEvents` | | Go | `GetDiagnosticContext` | `GetDiagnosticLogs` / `GetDiagnosticEvents` | | C# | `GetDiagnosticContextAsync` | `GetDiagnosticLogsAsync` / `GetDiagnosticEventsAsync` | Manager methods take a sandbox ID; sandbox methods use the current instance. Context returns the backend type, backend-defined identifiers, and lifecycle state when available. Log/event methods also require a scope such as `container` or `runtime`. Availability depends on the server backend. URL-delivered content is not downloaded automatically; inspect `truncated` and `warnings`. C# `SdkDiagnosticsOptions` separately controls local SDK logging. ## Lifecycle and cleanup High-level image/snapshot creation defaults to a 10-minute TTL. Configure the TTL explicitly for your workload. To disable expiration, use Python `timeout=None`, JavaScript `timeoutSeconds: null`, Kotlin `timeout(null)`, or the Go/C# `ManualCleanup` option. Template creation requires a TTL. `close()`, `DisposeAsync()`, and context-manager exit release local client resources; they do not kill the remote sandbox. Kill it in a `finally`/`defer` block, or use Python's `destroy()` helper. Pool acquisitions are consumed once and are not returned to the idle buffer. ## CLI Use the [CLI](/cli/) to manage sandboxes, run commands, and work with files from a terminal. ```bash uv tool install opensandbox-cli ``` ## MCP server The [MCP server](/sdks/mcp) exposes sandbox operations to MCP-capable clients: ```bash pip install opensandbox-mcp ``` --- ### Architecture Fast Sandbox Index (`docs/architecture/fast-sandbox/index.md`) --- title: Fast Sandbox description: OpenSandbox integration with the fast-sandbox platform — template-backed sandboxes delivered through Fastlet pools and the FastPath control plane. --- # Fast Sandbox Integration Fast Sandbox is a runtime backend that connects OpenSandbox to the external [fast-sandbox](https://github.com/opensandbox-group/fast-sandbox) platform. It provisions template-backed **Firecracker microVM** sandboxes inside pre-warmed **Fastlet** pods, removing the per-sandbox Kubernetes scheduler, watch propagation, and kubelet path from the creation hot path. The adapter and endpoint integration live in this repository. FastPath, the Fastlet runtime, and the `sandbox.fast.io` CRDs are owned by the fast-sandbox platform and consumed as-is. ## How it fits - **Backend selection**: Fast Sandbox is not a separate `runtime.type`. Under `runtime.type = "kubernetes"`, `CompositeSandboxService` routes create requests carrying `templateId` to `FastSandboxService`, and dispatches operations on existing `fsb-` prefixed sandbox IDs to it. Image- and snapshot-based requests keep using the Kubernetes workload providers. - **Control protocol**: the server and ingress talk to the fast-sandbox control plane (FastPath v2) over gRPC. gRPC ends at FastPath: the FastPath/Reconciler-to-Fastlet control protocol is JSON over HTTP (`/api/v2/fastlet/*`). FastPath owns mutations and live runtime operations; reads of persisted state use Kubernetes LIST/WATCH on `sandbox.fast.io` custom resources. - **Tenancy**: a tenant provider maps each authenticated tenant to a fast-sandbox namespace; request-scoped operations resolve it from context, background work falls back to a lookup by sandbox ID. 👉 Interactive architecture diagram — explore sandbox creation, template artifact flow, checkpoints, and deployment topology in a full-view explorable map. ## Core components | Component | Responsibility | Design goal | |---|---|---| | FastPath (gRPC control plane) | Serves every mutation and live operation: create, update, expiry, endpoint resolution, snapshots | Constant-time create via in-memory placement; restartable because durable state lives in CRDs, not here | | Reconcilers (controller) | Drive CRs to reality: build templates, run pools, place sandboxes, finish snapshots | Converge by observation — no leader election, replay from CRs after any restart | | Sandbox CRDs (`sandbox.fast.io`) | Durable record of intent, placement, policy bindings, and observations per sandbox | Intent survives every component failure; the CR is the source of truth | | Fastlet (pre-warmed pod) | Hosts many sandboxes in one network domain; provides runtime capacity and shared services | Density with isolation: pre-provisioned network slots, shared egress and DNS amortized across sandboxes | | Runtime agent (per node) | Owns Firecracker VM lifecycle over a unix socket; node readiness checks and self-labeling; installs assets | Node autonomy: the control plane delegates, the node verifies itself before admitting workloads | | [DART](https://github.com/data-accelerator/dart) (per node) | Peer-to-peer artifact delivery between node state roots | Amortize the origin: each block fetched from the store roughly once cluster-wide | | Egress process (per Fastlet) | Multi-sandbox outbound policy: subjects, Sandbox Actions handler, DNS and network enforcement | Fail-closed per-sandbox policy without per-sandbox sidecars — policy may be late, never open | | Template builder | Converts OCI images into golden-image artifacts (guest wiring, init, execd baked in) | Workload fixed before any request: creates express intent, not configuration | | Janitor | Sweeps orphaned resources of failed operations | Reconciliation finishes interrupted work; the janitor removes what should no longer exist | ## Design principles - **No new public API**: template-based creation and `fsb-` ID dispatch ride the existing lifecycle contract. SDKs work unchanged across backends, and no sandbox-to-backend registration store exists — the ID prefix is the dispatcher. - **CRD-first control**: durable per-sandbox intent lives in `sandbox.fast.io` resources; FastPath gRPC carries the mutation hot path. The Kubernetes scheduler, watch propagation, and kubelet leave the create path — the durable write does not. - **Fail-closed multi-tenant egress**: every subject denies traffic from registration until its data plane is ready — policy delivery can be late, never early-open. - **Authenticated routing**: sandbox routes are signed `f1.` scopes verified at ingress; endpoint issuance is decoupled from readiness. - **Firecracker by template**: golden images are built for Firecracker microVMs — the runtime shape is fixed at build time, not per request. ## Documentation | Page | Content | |---|---| | Interactive Architecture | Explorable component map: sandbox creation, template artifact flow, checkpoints, and deployment topology | | [Templates](/architecture/fast-sandbox/templates) | Template catalog, golden-image builds, artifact publication | | [Scheduling](/architecture/fast-sandbox/scheduling) | Pools and Fastlets, the create hot path, lifecycle and pause/resume | | [Networking](/architecture/fast-sandbox/networking) | Network slots, inbound route scopes and resolution, outbound egress enforcement | | [Pause, Resume, and Snapshots](/architecture/fast-sandbox/checkpoints) | Checkpoint artifacts, pause/resume across Fastlets, snapshot publication and restore | | [High Availability](/architecture/fast-sandbox/ha) | What survives which failure: control plane, nodes, sandboxes | | [Storage](/architecture/fast-sandbox/storage) | Artifact store, peer-to-peer block delivery, node state roots | | [Firecracker](/architecture/fast-sandbox/firecracker) | Host requirements, per-VM networking model, agent delegation | | [Performance](/architecture/fast-sandbox/performance) | ACK create, request, snapshot, and same-node pause/resume measurements | --- ### Console Docs 00 Design Principles (`console/docs/00-design-principles.md`) # Design principles ## Goals - Ship the console under `console/` with **minimal coupling** to lifecycle server internals; prefer a **BFF** over browser-held API keys. - **Tenant developers** use a tenant API key → see sandboxes in their namespace only. - **Platform admins** use a separate BFF admin token; the BFF aggregates Lifecycle calls **server-side** with per-tenant keys (optional read-only Kubernetes RBAC for platform pages). ## Avoid (large cross-cutting PRs) | Avoid | Why | |-------|-----| | Full server auth overhaul in the same PR as the UI | High review cost and conflict with `main` | | Browser storage of `OPEN-SANDBOX-API-KEY` | Use httpOnly BFF session instead | | Runtime-specific diagnostics in core UI | Stay on Diagnostics API | | Everything in one merge (history DB, Grafana, image catalog, K8s ops) | Split PRs; see [05-feature-matrix.md](./05-feature-matrix.md) | ## Do | Principle | Approach | |-----------|----------| | Decoupled delivery | `console/` + standalone Deployments (optional Helm subchart later) | | Single auth path to Lifecycle | Tenant key → BFF session; admin → `BFF_ADMIN_TOKEN` | | Runtime summary | BFF computes `runtimeSummary` from `createdAt` / `expiresAt` / status | | Pool | Call existing `/v1/pools`; UI states **platform-level, not tenant-isolated** | | Phased delivery | P0 sandbox CRUD; P1 snapshots/diagnostics; P2 admin + K8s views | ## Phasing ```text P0 Tenant: login, list/detail/create/renew/delete/endpoint, runtimeSummary P1 Snapshots, diagnostics, pools (admin) P2 Admin cross-tenant list/stats, K8s workloads/events, component health P3 Optional: PostgreSQL usage/history, OIDC (likely still at BFF layer) ``` ## References - [OSEP-0006: Developer Console](../../../oseps/0006-developer-console.md) — capability reference; this implementation uses **BFF session auth**, not OSEP Phase 1 server RBAC. - Optional server-side rows: `[store.lifecycle_audit]` in [server configuration](../../server/configuration.md). --- ### Console Docs 01 Architecture (`console/docs/01-architecture.md`) # Architecture ## Components ```mermaid flowchart LR Browser[Browser SPA] BFF[Console BFF] Life[Lifecycle Server /v1] K8s[K8s API read-only] Browser --> BFF BFF -->|tenant key proxy| Life BFF -->|admin optional| K8s ``` - **SPA**: static assets (nginx or same-origin reverse proxy). No Lifecycle API key in the bundle. - **BFF**: sessions, tenant resolution, admin multi-tenant list aggregation, `runtimeSummary`. - **Server**: unchanged Lifecycle behavior; multi-tenant isolation via `[tenants]` and API keys. ## Dual login | Role | Login | BFF session | Data scope | |------|-------|-------------|------------| | Developer | `POST /api/auth/tenant` + tenant API key | tenant session (tenant name only) | single tenant / namespace | | Admin | `POST /api/auth/admin` + `BFF_ADMIN_TOKEN` | admin session | all tenants via server-side key rotation | Admin mutations proxy to the owning tenant key where implemented. ## Shared `tenants.toml` Server and BFF should mount the **same** `tenants.toml` (ConfigMap or Secret): - Server validates keys and injects tenant context. - BFF validates keys for login; admin aggregation uses keys **only on the server**, never returned to the browser. ## Platform constraints (do not mislead in UI) - **Pool API (`/v1/pools`)** is not tenant-isolated. - With `[tenants]` enabled, there is no global `server.api_key`. - Proxy routes still require an API key (BFF can proxy on behalf of the tenant). ## Lifecycle proxy `LifecycleClient` calls the Lifecycle HTTP API with `httpx` and forwards the upstream status code and JSON body. The BFF then adds `runtimeSummary`. It does not use `opensandbox.SandboxManager`. That is a deliberate boundary: - The session picks the tenant API key per request. `ConnectionConfig.api_key` is fixed when a manager is constructed. A per-tenant manager cache would cover static tenant keys, but `/health` and `/version` are unauthenticated and sit outside `/v1`. - The SPA depends on the Lifecycle error body (`code` / `message`) and status code. The SDK returns typed models and raises `SandboxException`, which drops unknown fields and replaces the upstream error. - A short-lived `httpx` client does not reuse the SDK connection pool or retry policy. Pooling can be added on this client later without taking a dependency on SDK models. --- ### Console Docs 02 Bff Config And Secrets (`console/docs/02-bff-config-and-secrets.md`) # BFF configuration and secrets ## Required environment variables | Variable | Required | Description | |----------|----------|-------------| | `LIFECYCLE_API_BASE` | yes | Lifecycle root including `/v1`, e.g. `http://opensandbox-server.opensandbox-system.svc:8090/v1` | | `TENANTS_TOML_PATH` | yes | Path to **`tenants.toml`** (same content as server mount) | | `BFF_SESSION_SECRET` | yes | Session cookie signing secret (≥32 random chars) | | `BFF_ADMIN_TOKEN` | yes | Admin login password; **never** in frontend build | | `BFF_COOKIE_SECURE` | no | Set `true` behind HTTPS | | `BFF_CORS_ORIGINS` | no | Comma-separated SPA origins. Default `http://localhost:5173`. `*` is rejected because session cookies use credentials | | `BFF_HTTP_TIMEOUT_SECONDS` | no | Upstream timeout; default `30` | ### Optional: node-agent integration | Variable | Default | Description | |----------|---------|-------------| | `BFF_K8S_PROBE_ENABLED` | `false` | Platform probes, node-agent, workloads/events APIs | | `BFF_K8S_INSECURE_SKIP_TLS_VERIFY` | `false` | Skip Kubernetes API TLS verify (temporary; independent of server `[kubernetes] insecure_skip_tls_verify`) | | `BFF_K8S_SYSTEM_NAMESPACE` | `opensandbox-system` | Controller/ingress namespace | | `BFF_K8S_CONTROLLER_DEPLOYMENT` | `opensandbox-controller-manager` | Match `kubectl get deploy -n opensandbox-system` | | `BFF_K8S_INGRESS_DEPLOYMENT` | `opensandbox-ingress-gateway` | Same | | `BFF_K8S_NODE_AGENT_NAMESPACE` | `opensandbox-system` | DaemonSet namespace | | `BFF_K8S_NODE_AGENT_LABEL_SELECTOR` | `app.kubernetes.io/component=node-agent` | Pod selector | | `BFF_K8S_NODE_AGENT_PROBE_PORT` | `8080` | node-agent health port | Platform probes, the BFF ServiceAccount, and node-agent archive logs are not part of this PR. The deployment example uses the namespace default ServiceAccount. Future optional features (history PostgreSQL, Grafana monitor) are documented in [07-sandbox-history.md](./07-sandbox-history.md) and tracked on branch `feat/console-extended-scope`, not in the MVP PR. ## `tenants.toml` source 1. Maintain ConfigMap `opensandbox-tenants` `data.tenants.toml` (see [tenants-configmap.example.yaml](../k8s/tenants-configmap.example.yaml)). 2. Mount the same file on server and BFF pods (`subPath: tenants.toml`). `TENANTS_TOML_PATH` must point to a **`.toml` file**, not a Kubernetes YAML file. ## Secret structure See [console-bff-secret.example.yaml](../k8s/console-bff-secret.example.yaml): `BFF_SESSION_SECRET`, `BFF_ADMIN_TOKEN`. Tenant keys live only in mounted `tenants.toml`. ## Credentials separation | Credential | Storage | Use | |------------|---------|-----| | Tenant `api_keys[]` | `tenants.toml` | Developer login; BFF → Lifecycle | | `BFF_ADMIN_TOKEN` | BFF Secret | Admin BFF login only | ## Local `.env` Copy [bff/.env.example](../bff/.env.example) to `bff/.env` (gitignored). ## Security checklist - [ ] Tenant keys not in SPA bundle or `localStorage` - [ ] RBAC: only server + BFF service accounts read `tenants.toml` - [ ] Admin token rotation process - [ ] Ingress TLS; `BFF_COOKIE_SECURE=true` - [ ] Rate-limit or restrict admin login exposure --- ### Console Docs 03 Bff Api (`console/docs/03-bff-api.md`) # BFF HTTP API (SPA) Base path: `/api`. Cookie session required except login and health. ## Authentication ### `POST /api/auth/tenant` ```json { "apiKey": "..." } ``` - 200: `{ "role": "tenant", "tenant": "...", "namespace": "..." }` + Set-Cookie - 401: invalid key ### `POST /api/auth/admin` ```json { "adminToken": "..." } ``` - 200: `{ "role": "admin" }` + Set-Cookie ### `POST /api/auth/logout` Clears session. ### `GET /api/auth/me` `{ "role": "tenant"|"admin", "tenant"?, "namespace"? }` ## Tenant sandboxes Proxies Lifecycle; responses include BFF **`runtimeSummary`**. | Method | Path | Lifecycle | |--------|------|-----------| | GET | `/api/sandboxes` | `GET /sandboxes` | | GET | `/api/sandboxes/{id}` | `GET /sandboxes/{id}` | | POST | `/api/sandboxes` | `POST /sandboxes` | | DELETE | `/api/sandboxes/{id}` | DELETE | | POST | `/api/sandboxes/{id}/renew-expiration` | same | | POST | `/api/sandboxes/{id}/pause` | same (202) | | POST | `/api/sandboxes/{id}/resume` | same (202) | | POST | `/api/sandboxes/{id}/snapshots` | same (202) | | GET | `/api/sandboxes/{id}/diagnostics/logs` | query `scope` required | | GET | `/api/sandboxes/{id}/diagnostics/events` | same | | GET | `/api/sandboxes/{id}/endpoints/{port}` | same | ### Snapshots (tenant) | Method | Path | |--------|------| | GET | `/api/snapshots` | | GET | `/api/snapshots/{id}` | | DELETE | `/api/snapshots/{id}` | ### `runtimeSummary` ```json { "wallClockSeconds": 3842, "remainingSeconds": 900, "asOf": "2026-09-28T08:00:00Z", "basis": "createdAt" } ``` ## Admin ### `GET /api/admin/sandboxes` Aggregates per-tenant lists; optional `tenant=` filter. Pagination includes `tenantErrors` for partial failures. ### Admin sandbox ops Query **`tenant`** required for `{id}` read/write, diagnostics, etc. ### Admin snapshots `GET/DELETE /api/admin/snapshots[/{id}]?tenant=` ### Admin Kubernetes (read-only) `GET /api/admin/k8s/workloads`, `GET /api/admin/k8s/events` — requires `BFF_K8S_PROBE_ENABLED` and RBAC. ### `GET /api/admin/stats/runtime` Aggregated KPIs from admin sandbox list. ### Pools (admin) Proxies `/v1/pools` (**not tenant-isolated**; BFF uses first tenant key in `tenants.toml`). ### `GET /api/admin/platform/summary` Server health/version plus component placeholders. ## Health - `GET /health` → `{ "status": "ok" }` - `GET /api/version` → proxies server `/version` ## Errors ```json { "code": "UNAUTHORIZED", "message": "..." } ``` Lifecycle errors forward `code` / `message` when present. --- ### Api Implementation Compatibility (`docs/api/implementation-compatibility.md`) --- title: Implementation Compatibility description: Published OpenSandbox contracts, reference implementation boundaries, and current SDK expectations for independent implementations. --- # Implementation compatibility OpenSandbox separates sandbox lifecycle management, in-sandbox execution, and workload-specific services. This page describes the current repository's interfaces and client behavior. It does not define a new v1 conformance profile or a capability-negotiation protocol. Pin the spec revision and SDK version you target: an implementation of one API is not automatically compatible with every feature exposed by a sandbox SDK. ## Contracts and processes The [published OpenAPI files](/api/) are the starting point for implementation: | Contract | Interface and consumers | | --- | --- | | [`sandbox-lifecycle.yml`](https://github.com/opensandbox-group/OpenSandbox/blob/main/specs/sandbox-lifecycle.yml) | Control plane: creation, inspection, deletion, expiration, endpoint resolution, pause/resume, snapshots, templates, and lifecycle network-policy operations. Used by sandbox SDKs and management clients. | | [`diagnostic-api.yml`](https://github.com/opensandbox-group/OpenSandbox/blob/main/specs/diagnostic-api.yml) | Additional lifecycle-server routes for diagnostic log/event descriptors. Needed when clients use diagnostics; not the command-output API. | | [`execd-api.yaml`](https://github.com/opensandbox-group/OpenSandbox/blob/main/specs/execd-api.yaml) | Data plane: health, commands, files, directories, code contexts, metrics, and isolated-session operations. Used by the execution services in sandbox SDKs and Code Interpreter clients. | | [`egress-api.yaml`](https://github.com/opensandbox-group/OpenSandbox/blob/main/specs/egress-api.yaml) | Egress policy and Credential Vault management. A separate API from execd; the reference deployment normally serves it through an egress component. | The wire contract does not require an independent implementation to run the Go `execd` binary. A native execution service can serve the execd operations that its clients use. Compatibility requires their behavior as well as their route names: request/response schemas, authentication headers, file transfer semantics, SSE event payloads and completion, errors, and context/command lifetimes all matter. Implementing only the lifecycle contract does not supply SDK command, file, or code execution. Conversely, an execd-compatible service alone does not implement sandbox creation or endpoint resolution. The reference server's binary injection, container bootstrap, init/supervision, Jupyter integration, and proxy layout are implementation mechanisms. A backend can use different mechanisms, but must preserve the observable semantics of the operations it claims to support. OpenAPI coverage is not complete for every reference extension: for example, the current [execd router](https://github.com/opensandbox-group/OpenSandbox/blob/main/components/execd/pkg/web/router.go) exposes `/pty` HTTP/WebSocket routes which are not yet in `execd-api.yaml`. Do not infer support for such extensions from OpenAPI compatibility alone. ## Workload APIs and endpoint resolution `GET /v1/sandboxes/{sandboxId}/endpoints/{port}` resolves access to a service on the requested sandbox port. It does not convert that service into an execd API or impose execd's command/file schemas on it. In the [AIO example](https://github.com/opensandbox-group/OpenSandbox/blob/main/examples/aio-sandbox/main.py), OpenSandbox manages the sandbox and resolves port `8080`, then `AioSandboxClient` speaks the image's own shell/file/browser API. Endpoint resolution is agnostic to the workload's application API; transport support still depends on the selected network path. For example, the [Docker embedding proxy](/architecture/network/single-host-network) forwards HTTP, SSE, and WebSocket traffic. An arbitrary port number does not imply a generic TCP/UDP tunnel. Preserve the returned address (including proxy path) and required headers, and use a client suitable for the service and transport. Passing through execd's proxy does not make the destination process execd. ## Current Python SDK endpoint expectations The following describes [`SandboxSync`](https://github.com/opensandbox-group/OpenSandbox/blob/main/sdks/sandbox/python/src/opensandbox/sync/sandbox.py) in the current tree, rather than a portable protocol requirement for all clients: | Flow | Endpoint resolution before the health check | | --- | --- | | `create()` from an image or snapshot | Resolves execd port `44772`, then egress port `18080`. If the execd endpoint reports template origin, policy operations use the lifecycle service, but this create path has already resolved the egress endpoint. | | `create_from_template()` | Resolves `44772`; uses lifecycle network-policy operations without resolving `18080`. | | `connect()` | Resolves `44772` and reads the endpoint response's `OPEN-SANDBOX-ORIGIN` header. Template origin selects lifecycle network policy; other origins also resolve `18080`. | A custom `health_check` replaces the default execd `/ping` readiness probe. It does **not** remove these endpoint lookups or the construction of SDK service adapters. `skip_health_check=True` also still waits for endpoint publication. This explains why the AIO example's custom health check does not make it a lifecycle-only SDK client. Resolving an endpoint is distinct from invoking its API: service adapters are constructed before the application chooses operations. An independent backend targeting these high-level flows must account for these lookups, returned headers, and the operations it will expose. A client using only the lifecycle HTTP API and a workload-specific client can choose a narrower integration. Verify other SDK languages and versions separately; the table is not a promise that all SDKs have identical startup behavior. Template origin is an existing routing signal, not a general capability manifest. ## Backend-dependent features and unsupported operations The presence of an operation in a spec or SDK does not mean every runtime, image, or deployment enables it: | Feature | Current boundary | | --- | --- | | Pause/resume and snapshots | Depend on the workload provider, runtime, and storage configuration. In particular, the Kubernetes rootfs snapshot committer rejects gVisor RuntimeClasses. Do not infer snapshot support from basic create/execute support. | | Templates | Template management uses the Kubernetes Fast Sandbox integration; the reference Docker server rejects it with HTTP `501`. | | Egress policy and Credential Vault | Depend on egress configuration and backend routing. Template-backed Python sandboxes manage policy through the lifecycle service; `credential_vault` access raises an SDK exception for that origin. Policy support does not imply Vault support. | | Code execution | The reference execd uses Jupyter and installed kernels for code contexts. A shell-only image does not thereby support notebook-style code execution. A native implementation must reproduce the code API semantics if it claims that feature. | | PTY and isolated sessions | Depend on platform facilities and reference extensions. `/v1/isolated/capabilities` describes the isolator/hardening environment, not all lifecycle and execution features. Isolated diff/commit currently report unsupported and are not implemented. | There is no single general capability-negotiation endpoint in the current contracts. Document a backend's supported operations and prerequisites, and use feature-specific discovery where available. Unsupported requests should produce an explicit failure with an actionable error, not a fabricated success. Use the operation's specified error shape and distinguish an unsupported feature from missing authorization, missing resources, or a temporarily unready service. There is also no universal unsupported-operation status to substitute everywhere. Current reference examples include HTTP `501` for unsupported template management, HTTP `409` for the snapshot service's unsupported-runtime preflight, and HTTP `503` for unimplemented isolated diff/commit. See the [template handler](https://github.com/opensandbox-group/OpenSandbox/blob/main/server/opensandbox_server/api/templates.py), [snapshot service](https://github.com/opensandbox-group/OpenSandbox/blob/main/server/opensandbox_server/services/snapshot_service.py), and [isolated API notes](/api/). These describe present behavior, not a new uniform error convention. ## Choosing an initial compatibility target Start with a named client version and an explicit operation set. For a command and file integration, validate lifecycle create/inspect/delete and endpoint resolution, authentication and readiness, command SSE completion and errors, and the file operations the client needs. Add code contexts, snapshots, or policy/Vault operations only when the backend can provide their semantics. Use the repository's [SDK end-to-end tests](https://github.com/opensandbox-group/OpenSandbox/tree/main/tests) for the selected operations and document unimplemented features. This is a practical integration target, not a claim of full protocol conformance. --- ### Community Code Of Conduct (`docs/community/code-of-conduct.md`) --- title: Code of Conduct description: OpenSandbox community code of conduct. --- # Code of Conduct The code of conduct is maintained in the repository root as the single source of truth. Read the Code of Conduct on GitHub → --- ### Community Contributing (`docs/community/contributing.md`) --- title: Contributing description: Guide for contributing to the OpenSandbox project. --- # Contributing to OpenSandbox The contributing guide is maintained in the repository root to keep GitHub's contributor experience consistent. Read the Contributing Guide on GitHub → ## Contributor Advancement OpenSandbox maintains an explicit advancement process for contributors to become component maintainers or project maintainers. For nomination requirements, voting periods, and approval thresholds, see the project governance guide: Read the Maintainer Advancement Process on GitHub → --- ### Community Oseps (`docs/community/oseps.md`) --- title: Enhancement Proposals (OSEPs) description: Index of OpenSandbox Enhancement Proposals covering major features, architectural changes, and API modifications. --- # OpenSandbox Enhancement Proposals OpenSandbox Enhancement Proposals (OSEPs) are the mechanism for proposing major features, architectural changes, or modifications to the core API and security model. Small bug fixes and minor improvements do not require an OSEP. See the [OSEP contributing guide](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/CONTRIBUTING.md) for information on how to create and merge proposals. ## Proposal Index | OSEP | Title | Status | Last Updated | |:----:|:-----:|:------:|:------------:| | [OSEP-0001](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0001-fqdn-based-egress-control.md) | FQDN-based Egress Control | implemented | 2026-01-22 | | [OSEP-0002](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0002-kubernetes-sigs-agent-sandbox-support.md) | kubernetes-sigs/agent-sandbox Support | implemented | 2026-01-23 | | [OSEP-0003](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0003-volume-and-volumebinding-support.md) | Volume Support | implementing | 2026-02-11 | | [OSEP-0004](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0004-secure-container-runtime.md) | Pluggable Secure Container Runtime Support | implemented | 2026-07-27 | | [OSEP-0005](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0005-client-side-sandbox-pool.md) | Client-Side Sandbox Pool | implemented | 2026-07-27 | | [OSEP-0006](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0006-developer-console.md) | Developer Console for Sandbox Operations | implementable | 2026-03-06 | | [OSEP-0007](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0007-fast-sandbox-runtime-support.md) | Fast Sandbox Runtime Support | provisional | 2026-02-08 | | [OSEP-0008](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0008-pause-resume-rootfs-snapshot.md) | Pause and Resume via Rootfs Snapshot | implementing | 2026-03-13 | | [OSEP-0009](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0009-auto-renew-sandbox-on-ingress-access.md) | Auto-Renew Sandbox on Ingress Access | implemented | 2026-03-23 | | [OSEP-0010](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0010-opentelemetry-instrumentation.md) | OpenTelemetry Metrics and Logs (execd, egress, and ingress) | implementing | 2026-04-12 | | [OSEP-0011](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0011-secure-access-endpoint.md) | Secure Access on GetEndpoint and Signed Endpoint | implemented | 2026-04-25 | | [OSEP-0012](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0012-credential-vault.md) | Credential Vault and Credential Proxy | implemented | 2026-06-23 | | [OSEP-0013](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0013-isolated-execution-api.md) | Isolated Execution API | implementing | 2026-06-23 | | [OSEP-0014](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0014-multi-tenancy.md) | Multi-Tenancy Support for Kubernetes Runtime | implemented | 2026-07-27 | | [OSEP-0015](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0015-pod-snapshot.md) | Spec-Driven Pod Snapshot for Pause and Resume | draft | 2026-06-27 | | [OSEP-0016](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0016-unified-umbrella-release-governance.md) | Unified Umbrella Release Governance | implemented | 2026-09-21 | | [OSEP-0017](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0017-resilient-sdk-transport.md) | Resilient SDK Transport | implementing | 2026-07-22 | | [OSEP-0019](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0019-node-agent-sandbox-collection.md) | Node Agent for Node-Level Sandbox Collection | implementing | 2026-08-11 | ## Status Definitions | Status | Meaning | |--------|---------| | **draft** | Proposal is under initial development, not yet accepted | | **provisional** | Proposal accepted in principle, design may still evolve | | **implementable** | Design is final and ready for implementation | | **implementing** | Implementation is in progress | | **implemented** | Fully implemented and available | ## Submitting a Proposal ::: tip Before writing a full OSEP, consider opening a GitHub Issue or Discussion to gauge community interest and get early feedback on your idea. ::: To submit a new OSEP: 1. Fork the repository 2. Copy the OSEP template 3. Fill in the proposal details 4. Submit a pull request See the [OSEP contributing guide](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/CONTRIBUTING.md) for the full process. --- ### Community Release Automation (`docs/community/release-automation.md`) --- title: Release Automation description: How OpenSandbox releases work — one umbrella release, one tag, every artifact. --- # Release Automation OpenSandbox ships as a **unified umbrella release** (OSEP-0016): every artifact of a release — platform images, Helm charts (in-repo), CLI, and all SDKs — carries the same `X.Y.Z`, driven by a single workflow. Per-component and per-SDK release entry points have been removed; the umbrella is the only release path. See [Versioning](/community/versioning) for naming and cadence. ## Release Preparation (manual, on the release branch) ```bash # 1. one-command bump: chart versions, image references, SDK versions, # and dependency ranges rewritten to the target version in one commit manifests/release/create-umbrella-release.sh --version 1.1.0 --bump-only # 2. hand-author the release notes (copy docs/releases/TEMPLATE.md), # then push cp docs/releases/TEMPLATE.md docs/releases/1.1.0.md $EDITOR docs/releases/1.1.0.md git push ``` ## Prereleases (rc) An rc is always cut on the **line-birth version** (`X.Y.0-rc.N` — there is no `X.Y.Z-rc` for `Z > 0`). It validates that the platform deploys and the fan-out runs: images are published, **all packages are held (not published)**, and only the umbrella tag is minted (marked `--prerelease` on GitHub). rc runs are approval-free: no `release` environment gate, and the BOM PR merges without waiting for review (if the release-branch ruleset still mandates review, add a ruleset bypass for the release bot — the workflow fails fast instead of waiting). ```bash # 1. bump (channel=rc is derived from the suffix) manifests/release/create-umbrella-release.sh --version 1.1.0-rc.1 --bump-only # 2. notes, then push cp docs/releases/TEMPLATE.md docs/releases/1.1.0-rc.1.md $EDITOR docs/releases/1.1.0-rc.1.md git push # 3. dispatch gh workflow run release-umbrella.yml \ --ref release-1.1 \ -f version=1.1.0-rc.1 -f dry_run=false ``` Repeat with `-rc.2`, `-rc.3`, … as fixes land — every rc is a full rebuild and previous rc artifacts stay frozen in the registries. The stable release of the line (`1.1.0`, no suffix) then publishes packages for real. Prereleases are never resolved by default (PyPI / NuGet / Maven / Go hide them; npm publishes under the `rc` dist-tag). ## Release Execution (workflow) Dispatch `.github/workflows/release-umbrella.yml` (`dry_run=false`, `version`, `channel`, `release_branch`). Stable releases require a release approver other than the triggerer to approve the `release` environment; rc releases and dry runs run approval-free. The workflow then runs: | Stage | What happens | |---|---| | preflight | commit reachability + notes presence (`docs/releases/.md`) | | scan | version-consistency scan (release-blocking) | | build | 13 images built and pushed directly to `release-X.Y.Z` (run-scoped staging tags while the `UMBRELLA_PUBLISH_ENABLED` gate is closed); all packages built and held | | BOM | digests pinned into `docs/releases/.yaml`, pushed to a `release/` branch; the workflow opens a PR to the release branch and merges it (`C_bom`) — stable waits for a code-owner approval (required by the branch ruleset), rc merges immediately | | publish | once the BOM PR merges (all images green + code-owner approval), packages publish in parallel per registry (PyPI, npm, NuGet, Maven Central); images are already public at `release-X.Y.Z` from the build stage | | tag | `release-X.Y.Z` + `sdks/sandbox/go/vX.Y.Z` minted on `C_bom`, GitHub Release created with the notes and BOM | A failed release never mints its git tag or GitHub Release, so the release feed never shows a partial release; dispatch a dry run (`dry_run=true`) to keep the fan-out healthy between release windows. ### Dry runs `dry_run=true` exercises everything except publishing: images are built **locally** (single-arch, `--load` — no registry push, no credentials needed), packages are built and held as workflow artifacts, the BOM lands on the release branch through the BOM PR (one code-owner approval, as on any change to a protected branch) with local image IDs standing in for registry digests, and no git tags are minted. To rehearse a release in a fork, prepare the release branch (bump + notes) and dispatch the workflow with `dry_run=true` — the `UMBRELLA_PUBLISH_ENABLED` variable does not need to exist there. The fork must enable **Allow GitHub Actions to create and approve pull requests** (Settings → Actions → General → Workflow permissions); `pull-requests: write` in the workflow alone cannot override that setting being off. ## Release Artifacts - `docs/releases/.yaml` — BOM, authoritative for image digests - `docs/releases/.md` — hand-authored release notes (mirrored to the GitHub Release) - container images: `opensandbox/:release-X.Y.Z` on Docker Hub, GHCR, and ACR - packages: PyPI (server, CLI, 3 SDKs), npm (2), Maven Central (5 Kotlin/JVM coordinates), NuGet (2) — all at the umbrella version - Helm charts: **not published**; they ship in-repo at the release tag ## Release Runbook ### Publish and rollback Registry legs (PyPI, npm, NuGet, Maven Central) run in parallel and independently — there is no cross-ecosystem ordering. Publish steps are idempotent: a version already visible on its registry is skipped, so re-runs never collide with registry immutability (a released Maven version is immutable; the `sandbox-bom` POM's presence on Central is the skip marker). A successful upload is not re-verified against the public index — registries' simple indexes can lag an upload by minutes. On a failure, remediate already-published legs: | Ecosystem | Revocation | |---|---| | Maven Central | not released if its leg failed; after release, immutable — deprecate only | | NuGet | `dotnet nuget delete ` (72 h), else unlist | | npm | `npm unpublish @` (72 h), else `npm deprecate` | | PyPI | manual yank via project settings (no scriptable API) | | images | `crane delete :release-X.Y.Z` | Always file a P0 release incident with the run link before remediating. Global mirror caches may retain revoked artifacts; revocation is authoritative upstream only. ### N-1 emergency backport (CVSS ≥ 8.0, ≤ 72 h, one-shot per line) 1. Land the fix on `main` (standard PR review). 2. `git checkout release-X.(Y-1) && git cherry-pick && git push`. 3. Dispatch `release-umbrella.yml` with `version=X.(Y-1).Z`, `release_branch=release-X.(Y-1)`, `dry_run=false`. The full fan-out reruns; the release notes must link the CVE and name the equivalent current-line snapshot. Anything that is not a qualifying CVE waits for the next line birth. ### Staging cleanup Run-scoped staging image tags (`staging--`) exist only when a run does not publish release tags (dry runs, or real runs while the `UMBRELLA_PUBLISH_ENABLED` gate is still closed). They are namespaced and never referenced by user-facing surfaces; a scheduled job deletes staging tags whose run did not complete within 14 days. Held package artifacts use the default 14-day workflow-artifact retention. ## Legacy Releases Historical per-component tags (`server/v0.2.3`, `docker/execd/v1.1.0`, …) are frozen and keep resolving forever. Verification instructions for those older artifacts (and their workflow identities) are in [Release Verification](/community/release-verification). --- ### Community Release Verification (`docs/community/release-verification.md`) --- title: Release Verification description: Verify signed OpenSandbox releases for source archives, container images, packages, and Helm charts. --- # Release Verification OpenSandbox signs public release outputs without changing the normal install commands. Verification is optional for day-to-day use, but supported for users who need supply chain integrity checks. This process applies to releases produced after the signing workflows were introduced. Older releases may not have attestations or signatures. ## Signing Model OpenSandbox uses these signing paths: - Source code releases: the Generic Release workflow uploads an explicit `opensandbox-.tar.gz` source archive and `SHA256SUMS` file to the GitHub Release, then creates GitHub/Sigstore provenance attestations for both files. - Container images: the component and server image workflows sign Docker Hub, GitHub Container Registry (GHCR), and Alibaba Cloud Container Registry (ACR) image digests with `cosign` keyless signing, and publish provenance attestations to all three registries. - Python and CLI packages: wheels and source distributions are attested before `uv publish`. - JavaScript packages: the workflow runs `pnpm pack`, attests the generated npm tarball, and publishes that same tarball. - C# packages: NuGet `.nupkg` files are attested before publication. - Go SDK modules: the `sdks/sandbox/go/v` source release archive is attested by the Generic Release workflow. - Helm charts created by the current release workflow: packaged chart `.tgz` files are attested and published with an attested `SHA256SUMS` file so consumers can verify the exact release bytes. - Java/Kotlin packages: Maven Central publications are signed by the Gradle Maven publish signing configuration. Download the `.asc` signature next to the Maven artifact and verify it with OpenPGP tooling. Release tags may also be signed with `manifests/release/create-release.sh --sign-tag` when the release operator has a local git signing key configured. Do not rely on signed tags alone for generated deliverables; verify the artifact you are installing. ## Trust Roots and Keys Most OpenSandbox release signatures are keyless Sigstore signatures created by GitHub Actions OpenID Connect (OIDC). There is no long-lived OpenSandbox private key for these signatures, so there is no project public key file to download. The public certificates and signed bundles are retrieved by `gh` or `cosign` from GitHub's attestation service, OCI registries, and Sigstore transparency infrastructure. Expected identity values: - Repository: `opensandbox-group/OpenSandbox` - OIDC issuer: `https://token.actions.githubusercontent.com` ::: warning Legacy workflow identities (pre-umbrella releases only) The per-target publish workflows below were removed when OpenSandbox moved to unified umbrella releases. They remain the verification identity for artifacts published before that point. New umbrella releases are produced by `release-umbrella.yml` and `release-packages.yml`. ::: - Component image workflow: `opensandbox-group/OpenSandbox/.github/workflows/publish-components.yml` - Server image workflow: `opensandbox-group/OpenSandbox/.github/workflows/publish-server.yml` - CLI package workflow: `opensandbox-group/OpenSandbox/.github/workflows/publish-cli.yml` - Python package workflow: `opensandbox-group/OpenSandbox/.github/workflows/publish-python-sdks.yml` - JavaScript package workflow: `opensandbox-group/OpenSandbox/.github/workflows/publish-js-sdks.yml` - C# package workflow: `opensandbox-group/OpenSandbox/.github/workflows/publish-csharp-sdks.yml` - Helm chart workflow: `opensandbox-group/OpenSandbox/.github/workflows/publish-helm-chart.yml` Set the repository identity for the release you are verifying: ```bash REPOSITORY="opensandbox-group/OpenSandbox" WORKFLOW_REPOSITORY="${REPOSITORY}" WORKFLOW_REPOSITORY_URL="https://github.com/${WORKFLOW_REPOSITORY}" ``` For releases produced before the GitHub organization migration, use the historical identity instead: ```bash REPOSITORY="alibaba/OpenSandbox" WORKFLOW_REPOSITORY="${REPOSITORY}" WORKFLOW_REPOSITORY_URL="https://github.com/${WORKFLOW_REPOSITORY}" ``` If you run the release workflows from a downstream fork, replace `opensandbox-group/OpenSandbox` in the verification commands with that fork's `owner/repository` identity. Private signing material is not stored in GitHub Releases, Docker Hub, GHCR, ACR, PyPI, npm, Maven Central, NuGet, or Helm chart downloads. Java/Kotlin Maven Central signing keys are held only in GitHub Actions secrets. ## Verify Source Releases > Source archives are produced only for releases before 2026-08 (when the `release-generic.yml` workflow was removed). Newer releases have no source archive; skip this section for them. Set the release tag first: ```bash TAG="server/v0.1.13" SAFE_TAG="${TAG//\//-}" ``` Download the signed source archive and checksum file: ```bash gh release download "$TAG" \ --repo "$REPOSITORY" \ --pattern "opensandbox-${SAFE_TAG}.tar.gz" \ --pattern "SHA256SUMS" ``` Check the archive digest: ```bash sha256sum -c SHA256SUMS ``` Verify the source archive attestation: ```bash gh attestation verify "opensandbox-${SAFE_TAG}.tar.gz" \ --repo "$REPOSITORY" \ --signer-workflow "${WORKFLOW_REPOSITORY}/.github/workflows/release-generic.yml" ``` Verify the checksum file attestation: ```bash gh attestation verify SHA256SUMS \ --repo "$REPOSITORY" \ --signer-workflow "${WORKFLOW_REPOSITORY}/.github/workflows/release-generic.yml" ``` The Generic Release workflow is started with `workflow_dispatch`, so its provenance `source-ref` is the ref selected when the workflow was dispatched (normally `refs/heads/main`), not the release tag created by the job. ## Verify Container Images Install `cosign` and `gh`, then resolve the image digest. Always verify by digest, not by mutable tag alone. Release images are published with the same component name and digest in all three official registries: | Registry | Image name pattern | | --- | --- | | Docker Hub | `docker.io/opensandbox/` | | GitHub Container Registry | `ghcr.io/opensandbox-group/opensandbox/` | | Alibaba Cloud Container Registry | `sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/` | The component can be `execd`, `ingress`, `egress`, `controller`, `task-executor`, `image-committer`, or `nodeagent`. The server image uses the component name `server`. ::: tip Code Interpreter Image Verification The standalone `code-interpreter` environment image is published and maintained from [opensandbox-group/sandbox-images](https://github.com/opensandbox-group/sandbox-images). - **Existing image releases** (such as `v1.1.0` and earlier) continue using the OpenSandbox workflow identity (`.github/workflows/publish-components.yml`). - **New releases** from `opensandbox-group/sandbox-images` use its `release.yml` workflow identity. Users and operators verifying signatures or provenance for new releases must follow the verification documentation in [opensandbox-group/sandbox-images](https://github.com/opensandbox-group/sandbox-images). ::: ```bash IMAGE="docker.io/opensandbox/execd" TAG="v1.0.15" DIGEST="$(docker buildx imagetools inspect "${IMAGE}:${TAG}" --format '{{.Manifest.Digest}}')" IMAGE_REF="${IMAGE}@${DIGEST}" ``` Verify the cosign keyless signature: ```bash cosign verify "$IMAGE_REF" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ --certificate-identity-regexp "^${WORKFLOW_REPOSITORY_URL}/.github/workflows/publish-components.yml@refs/tags/(docker|k8s)/[^/]+/v?[0-9].*$" ``` Verify the registry provenance attestation: ```bash gh attestation verify "oci://${IMAGE_REF}" \ --repo "$REPOSITORY" \ --bundle-from-oci \ --signer-workflow "${WORKFLOW_REPOSITORY}/.github/workflows/publish-components.yml" ``` For the server image, use `docker.io/opensandbox/server` and the server workflow: ```bash IMAGE="docker.io/opensandbox/server" TAG="v0.1.13" DIGEST="$(docker buildx imagetools inspect "${IMAGE}:${TAG}" --format '{{.Manifest.Digest}}')" IMAGE_REF="${IMAGE}@${DIGEST}" cosign verify "$IMAGE_REF" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ --certificate-identity-regexp "^${WORKFLOW_REPOSITORY_URL}/.github/workflows/publish-server.yml@refs/tags/server/v[0-9].*$" ``` GHCR and ACR images use the same digest and identity checks with their respective image names, for example: ```bash IMAGE="ghcr.io/opensandbox-group/opensandbox/execd" # or IMAGE="sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/execd" ``` ## Verify Packages Download the package file from its normal package registry, then verify the attestation against the OpenSandbox repository and the expected release tag. Python and CLI packages: ```bash python -m pip download opensandbox-server==0.1.13 --no-deps gh attestation verify opensandbox_server-0.1.13*.whl \ --repo "$REPOSITORY" \ --signer-workflow "${WORKFLOW_REPOSITORY}/.github/workflows/publish-server.yml" \ --source-ref refs/tags/server/v0.1.13 ``` JavaScript packages: ```bash npm pack @alibaba-group/opensandbox@0.1.7 gh attestation verify alibaba-group-opensandbox-0.1.7.tgz \ --repo "$REPOSITORY" \ --signer-workflow "${WORKFLOW_REPOSITORY}/.github/workflows/publish-js-sdks.yml" \ --source-ref refs/tags/js/sandbox/v0.1.7 ``` C# packages: ```bash gh attestation verify Alibaba.OpenSandbox.1.0.0.nupkg \ --repo "$REPOSITORY" \ --signer-workflow "${WORKFLOW_REPOSITORY}/.github/workflows/publish-csharp-sdks.yml" \ --source-ref refs/tags/csharp/sandbox/v1.0.0 ``` Helm charts: ```bash HELM_CHART="opensandbox" HELM_VERSION="" HELM_TAG="helm/${HELM_CHART}/${HELM_VERSION}" HELM_PACKAGE="${HELM_CHART}-${HELM_VERSION}.tgz" gh release download "$HELM_TAG" \ --repo "$REPOSITORY" \ --pattern "$HELM_PACKAGE" \ --pattern SHA256SUMS sha256sum -c SHA256SUMS gh attestation verify "$HELM_PACKAGE" \ --repo "$REPOSITORY" \ --signer-workflow "${WORKFLOW_REPOSITORY}/.github/workflows/publish-helm-chart.yml" \ --source-ref "refs/tags/${HELM_TAG}" gh attestation verify SHA256SUMS \ --repo "$REPOSITORY" \ --signer-workflow "${WORKFLOW_REPOSITORY}/.github/workflows/publish-helm-chart.yml" \ --source-ref "refs/tags/${HELM_TAG}" ``` The checksum binds the downloaded package to the digest recorded by the release workflow. The attestation verifies that the package with that digest was produced by the expected OpenSandbox workflow. Change `HELM_CHART` and `HELM_VERSION` to verify a different Helm release. Manual Helm releases must select the exact existing release tag as their dispatch ref. Their provenance `source-ref`, like a tag-triggered Helm release, therefore identifies that protected Helm tag. ### Helm Release Status A stable `opensandbox` umbrella chart is marked `production-ready` only after the release workflow installs the exact packaged `.tgz` identified by the published digest into an ephemeral Kind cluster. The workflow does not rebuild the chart between the runtime gate and publication. The gate must verify: - The controller and server workloads become Ready. - The required OpenSandbox CRDs are established and usable. - Authenticated server access succeeds, while missing or invalid credentials are rejected. - A BatchSandbox completes the create, command-execution, and delete lifecycle. This status covers the default core profile named in the Release notes. It does not certify optional ingress, egress policy, snapshot/pause-resume, node-agent, upgrade, or multi-architecture lanes unless those profiles are listed separately. Standalone `opensandbox-controller`, `opensandbox-server`, and `opensandbox-node-agent` chart releases are `package-verified` only. Their validation covers the packaged chart and does not claim the integrated Kind runtime coverage of a stable umbrella release. Each Helm GitHub Release records the package SHA-256 digest, source ref and commit, validation workflow run, validation profile, and—when runtime-tested— the requested core image references, registry RepoDigests, and image IDs in its release notes. The profile distinguishes the stable umbrella Kind runtime gate from package-only verification. Because chart defaults remain documented image version tags, `production-ready` records the publication-time gate result; it does not claim that a registry tag can never be changed later. Java/Kotlin Maven artifacts: ```bash curl -O https://repo1.maven.org/maven2/com/alibaba/opensandbox/sandbox/1.0.10/sandbox-1.0.10.jar curl -O https://repo1.maven.org/maven2/com/alibaba/opensandbox/sandbox/1.0.10/sandbox-1.0.10.jar.asc KEY_ID="$(gpg --list-packets sandbox-1.0.10.jar.asc | awk '/keyid/ { print $NF; exit }')" gpg --keyserver hkps://keys.openpgp.org --recv-keys "$KEY_ID" gpg --verify sandbox-1.0.10.jar.asc sandbox-1.0.10.jar ``` If verification cannot find an attestation for a release that predates this process, use a newer release as the signed release evidence. --- ### Community Versioning (`docs/community/versioning.md`) --- title: Versioning and Releases description: Unified umbrella versioning for OpenSandbox — one version shared by every image, chart, CLI, and SDK. --- # Versioning and Releases OpenSandbox ships as a **unified umbrella release**: every artifact of a release — server and component images, Kubernetes controller and task-executor images, Helm chart, CLI, and all SDKs — carries the **same `X.Y.Z`**, cut from a single release commit and pinned in a signed BOM. Full design: [OSEP-0016](https://github.com/opensandbox-group/OpenSandbox/blob/main/oseps/0016-unified-umbrella-release-governance.md). ::: warning Rollout status Umbrella release tooling is being rolled out in phases (dry-run → rc → GA, see the OSEP's Migration section). Until GA, per-component tags remain the operative release mechanism ([Release Automation](/community/release-automation)). Historical per-component tags are frozen at that point — never deleted, never extended — and keep resolving forever. ::: ## First version: `1.1.0` The first umbrella release is **`release-1.1.0`** and doubles as the GA declaration. `1.0.0` is deliberately skipped: Maven Central (`com.alibaba.opensandbox:sandbox` up to `1.0.19`) and the Go module proxy (`sdks/sandbox/go` up to `v1.0.5`) already consumed those versions, and both registries are immutable. The umbrella starts at the lowest new **line** (`X.Y.0`) above every already-consumed version — `1.0.20` would clear the registries but is not a valid line birth (`Z > 0` is reserved for in-line snapshots), so `1.1.0` it is. ## Naming rules | Artifact | Format (example at `1.4.0`) | |---|---| | Git tag | `release-1.4.0` | | Container images | `opensandbox/{server,execd,ingress,egress,image-committer,controller,task-executor}:release-1.4.0` | | Go SDK (VCS tags) | `sdks/sandbox/go/v1.4.0`, `sdks/sandbox/go/poolredis/v1.4.0` — same commit as the umbrella tag | | CLI / SDKs / server on PyPI | bare `1.4.0` | Git and image tags share the same `release-` string: `git checkout release-1.4.0` and the image you pull are the same release. Package registries use the bare semver core because they reject prefixes. ## Scope - **Covered**: platform runtime images (server, execd, ingress, egress, image-committer, controller, task-executor; the fast-sandbox family ships as `opensandbox/fsb-*`), CLI, and all published SDKs (Python, JavaScript, Kotlin/JVM, .NET, Go). - **Helm charts are not published.** Charts live in-repo and are versioned at the release tag; render and deploy yourself: ``` git checkout release-1.1.0 helm template ./manifests/charts/opensandbox | kubectl apply -f - ``` GitOps platforms can point directly at the repo path and tag. - **Not covered**: sandbox template images such as `opensandbox/code-interpreter`. They are chosen by the user at sandbox-creation time and version independently in [opensandbox-group/sandbox-images](https://github.com/opensandbox-group/sandbox-images). ## Cadence and support - A line is born every 2 weeks (`X.Y.0`); in-line snapshots (`X.Y.Z`, `Z > 0`) ship on demand. Pre-releases look like `X.Y.0-rc.N` and publish images only — packages are held until the line's stable release. - **Latest line only, no LTS.** When `X.(Y+1).0` ships, `X.Y.*` is EOL except for a single emergency-CVE window (CVSS ≥ 8.0, ≤ 72h from disclosure, one-shot `X.(Y-1).Z` snapshot). - There are no per-component hotfixes: a backport is a full umbrella rebuild at the new `X.Y.Z`. To verify a release you installed, see [Release Verification](/community/release-verification). --- ### Deployment Ack (`docs/deployment/ack.md`) --- title: ACK 部署 description: 在阿里云容器服务 Kubernetes 版（ACK）上部署 OpenSandbox —— 创建集群、新增节点、渲染部署各组件及 ACK 环境注意事项。 --- # ACK 部署 OpenSandbox 可部署在任何标准 Kubernetes 集群上，ACK 完全兼容标准 Kubernetes API。组件安装细节见 [Kubernetes 部署](/deployment/)，本文介绍在 ACK 上部署的具体步骤与注意事项。 ## 前提条件 - 阿里云账号，已开通容器服务 ACK 并完成默认角色授权 - Kubernetes 1.21.1+ - Helm 3.x（仅用作模板渲染器）、`kubectl` - 集群 KubeConfig 凭证（见[第二步](#第二步-连接集群)） ## 第一步：创建 ACK 集群 推荐使用 **ACK 托管集群（Pro）**，参考[创建 ACK 托管集群](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-2/)完成创建，关键配置如下： | 配置项 | 建议 | |--------|------| | 集群版本 | ≥ 1.21 | | 网络插件 | Terway（沙箱 Pod 拥有 VPC 内可直接路由的 Pod IP） | | 初始节点池 | 2–3 台通用型实例，承载控制面组件 | | 安全组 | 放行 Worker 节点间 VPC 内互通 | ## 第二步：连接集群 参考[获取集群 KubeConfig 并通过 kubectl 工具连接集群](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/obtain-the-kubeconfig-file-of-a-cluster-and-use-kubectl-to-connect-to-the-cluster/)，配置 `~/.kube/config` 后验证： ```bash kubectl get nodes ``` ## 第三步：新增节点 为沙箱工作负载创建**独立节点池**（可开启自动伸缩），与控制面组件隔离，参考[创建和管理节点池](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/create-a-node-pool/)；已有 ECS 实例可通过[添加已有节点](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/add-existing-ecs-instances-to-an-ack-cluster/)接入集群。日常管理参考[节点管理](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/node-management/)。 ::: tip 污点隔离 沙箱节点池配置污点（Taint）时，需在 BatchSandbox Pod 模板中添加对应的 Toleration，否则沙箱 Pod 无法调度。 ::: ::: warning fast-sandbox 节点池要求 fast-sandbox（Firecracker）对节点有两个要求：Worker 节点必须是**裸金属服务器**（依赖 KVM）；节点池内**所有节点的 CPU 型号必须一致**。混用不同代际的机型会导致微虚拟机快照无法跨节点恢复，因此建议锁定单一实例规格并单独建节点池。 ::: ## 第四步：部署 OpenSandbox 采用 GitOps 渲染模式部署：`helm template` 将 chart 渲染为 YAML，经 `kubectl diff` 审阅后用 `kubectl apply --server-side` 应用到集群。渲染产物与 values 文件纳入 Git 管理。 按组件顺序部署（与 [Deployment Order](/deployment/#deployment-order) 一致）： ```text base → controller → fast-sandbox* → ingress-gateway → server ``` \* `fast-sandbox` 为可选组件；启用时需在 ingress-gateway 与 server 之前部署。 注意： - 不要与 `helm install` / `helm upgrade` 混用，两种方式会互相覆盖 - 使用 `--server-side`（Server-Side Apply）：CRD 体量超过 client-side apply 的注解上限 准备仓库与命名空间： ```bash git clone https://github.com/opensandbox-group/OpenSandbox.git cd OpenSandbox git checkout release-1.1.0 # 部署对应发布版本 mkdir -p out # 渲染产物目录 # 渲染部署不会自动创建 namespace，需提前创建 kubectl create namespace opensandbox-system --dry-run=client -o yaml \ | kubectl apply --server-side -f - kubectl create namespace opensandbox --dry-run=client -o yaml \ | kubectl apply --server-side -f - ``` ### 1. 部署 base（CRD + RBAC） ```bash helm template base manifests/charts/base > out/base.yaml kubectl diff -f out/base.yaml kubectl apply --server-side -f out/base.yaml ``` 应用成功后输出 21 行 `serverside-applied`（1 Namespace + 7 CRD + 11 ClusterRole + 2 ClusterRoleBinding）。验证 CRD： ```bash kubectl get crd | grep -E 'sandbox\.(opensandbox|fast)\.io' ``` ```text batchsandboxes.sandbox.opensandbox.io 2026-09-28T08:48:54Z pools.sandbox.opensandbox.io 2026-09-28T08:48:55Z sandboxes.sandbox.fast.io 2026-09-28T08:49:04Z sandboxpools.sandbox.fast.io 2026-09-28T08:48:59Z sandboxsnapshots.sandbox.fast.io 2026-09-28T08:49:06Z sandboxsnapshots.sandbox.opensandbox.io 2026-09-28T08:48:57Z sandboxtemplates.sandbox.fast.io 2026-09-28T08:49:07Z ``` ### 2. 部署 controller ```bash helm template opensandbox-controller manifests/charts/controller \ -n opensandbox-system \ --set controller.replicaCount=2 > out/controller.yaml kubectl diff -f out/controller.yaml kubectl apply --server-side -f out/controller.yaml kubectl rollout status deployment/opensandbox-controller-manager \ --namespace opensandbox-system --timeout=180s ``` 应用成功后输出 6 行 `serverside-applied`（SA + Role/Binding + ClusterRole/Binding + Deployment）。rollout 与 Pod 状态： ```text deployment "opensandbox-controller-manager" successfully rolled out opensandbox-controller-manager-865887c57-mvbr9 1/1 Running 0 6m25s 10.79.205.208 ap-southeast-1.10.79.205.204 opensandbox-controller-manager-865887c57-rhgxn 1/1 Running 0 37s 10.79.205.209 ap-southeast-1.10.79.205.203 ``` ### 3. 部署 fast-sandbox 必须在 server 之前部署：server 启动时会连接 FastPath gRPC 端点。 #### 准备制品仓库（OSS） fast-sandbox 的 golden image 与快照存放在 S3 兼容的制品仓库。ACK 上建议使用与集群同 region 的 OSS，并通过 VPC 内网域名访问（避免公网流量）： 1. 在与集群相同的 region 创建 OSS Bucket（如 `my-sandbox-images`） 2. 创建 RAM 子账号 AK/SK，授予该 Bucket 的读写权限 3. VPC 内网 Endpoint 格式为 `https://oss-.internal.aliyuncs.com`（如 `https://oss-ap-southeast-1.internal.aliyuncs.com`） #### 配置凭证 Secret ::: warning registry.json 的最终 schema 由 fast-sandbox 上游编译工具生成，以下为**占位符**示例，仅用于让 Pod 正常启动；实际拉取制品前需按工具产物补全。 ::: ```bash cat > /tmp/registry.json <<'EOF' { "endpoint": "https://oss-ap-southeast-1.internal.aliyuncs.com", "bucket": "my-sandbox-images", "accessKeyId": "", "accessKeySecret": "" } EOF kubectl create secret generic fast-sandbox-agent-registry \ -n opensandbox-system \ --from-file=registry.json=/tmp/registry.json \ --dry-run=client -o yaml | kubectl apply --server-side -f - rm /tmp/registry.json # AK/SK 不落盘、不进 Git ``` 渲染并部署（`artifactStore` 与 Bucket、Endpoint 保持一致）： ```bash helm template fast-sandbox manifests/charts/fast-sandbox \ -n opensandbox-system \ --set artifactStore.store=s3://my-sandbox-images/publish \ --set artifactStore.endpoint=https://oss-ap-southeast-1.internal.aliyuncs.com \ > out/fast-sandbox.yaml kubectl diff -f out/fast-sandbox.yaml kubectl apply --server-side -f out/fast-sandbox.yaml ``` 应用成功后输出 13 行 `serverside-applied`（PDB + SA + Secret/ConfigMap + RBAC + 2 Service + DaemonSet + Deployment）。验证 Pod 状态与节点标签： ```text fast-sandbox-controller-7d79b6447b-8q5bp 1/1 Running firecracker-runtime-hkg9d 2/2 Running firecracker-runtime-z8mq9 2/2 Running NAME STATUS KVM FIRECRACKER-NODE ap-southeast-1.10.79.205.203 Ready true true ap-southeast-1.10.79.205.204 Ready true true ``` runtime 就绪后会为节点自动打 4 个标签（`kvm`、`firecracker-node`、`cpu-identity`、`cpu-template`）。其中 `cpu-identity` 标识 CPU 型号（厂商-家族-型号），同一节点池内必须一致，否则微虚拟机快照无法跨节点恢复： ```bash kubectl get nodes -L sandbox.fast.io/kvm,fast-sandbox.io/cpu-identity,fast-sandbox.io/firecracker-node ``` ::: warning 节点需加载 kvm 内核模块 ECS 裸金属节点默认不加载 kvm 模块，readiness 会报 `/dev/kvm unusable: device does not exists`。此时需在节点上执行 `modprobe kvm kvm_amd`（可通过 privileged Pod 的 `nsenter -t 1 -m` 或节点 SSH），再执行 `kubectl rollout restart daemonset/firecracker-runtime -n opensandbox-system` 让 Pod 重建 `/dev`。模块加载在节点重启后失效，需重新执行；建议通过节点池 bootstrap 脚本或自定义镜像持久化。 ::: 模板构建、镜像准备等前置步骤见 [fast-sandbox runtime 部署指南](https://github.com/opensandbox-group/OpenSandbox/blob/main/manifests/HELM-DEPLOYMENT.md#fast-sandbox-runtime-firecracker)。 ### 4. 部署 ingress-gateway Kubernetes 环境中沙箱 Pod 只有 ClusterIP，客户端流量须经 ingress-gateway 路由。在 server 之前部署，这样 server 首次安装即可完成网关公告配置。 `providerType=fast-sandbox` 时网关强制要求 secure-access 签名密钥环（OSEP-0011）：server 用它签发路由令牌，网关用它验证。先生成密钥，部署 server 时复用： ```bash KEY=$(openssl rand -base64 32) # 保存好，部署 server 时复用 helm template ingress-gateway manifests/charts/ingress-gateway \ -n opensandbox-system \ --set gateway.providerType=fast-sandbox \ --set gateway.fastpathEndpoint=fast-sandbox-fastpath.opensandbox-system.svc:9090 \ --set "gateway.secureAccess.keys[0].key_id=a" \ --set "gateway.secureAccess.keys[0].key=$KEY" \ > out/ingress-gateway.yaml kubectl diff -f out/ingress-gateway.yaml kubectl apply --server-side -f out/ingress-gateway.yaml kubectl rollout status deployment/opensandbox-ingress-gateway \ --namespace opensandbox-system --timeout=180s ``` 验证： ```text deployment "opensandbox-ingress-gateway" successfully rolled out opensandbox-ingress-gateway-66bd4bf7b7-59nfz 1/1 Running 0 67s opensandbox-ingress-gateway-66bd4bf7b7-dbc2v 1/1 Running 0 56s ``` ### 5. 部署 lifecycle server 创建 API Key Secret（生产环境建议使用外部密钥管理系统）： ```bash read -s OPENSANDBOX_API_KEY kubectl create secret generic opensandbox-api-key \ --namespace opensandbox-system \ --from-literal=api-key="${OPENSANDBOX_API_KEY}" \ --dry-run=client -o yaml | kubectl apply -f - unset OPENSANDBOX_API_KEY ``` 在 values 文件中引用 Secret 并配置网关公告。注意 `secureAccess` 必须与 ingress-gateway 使用同一把密钥（`activeKey` 对应 key_id）；`gatewayRouteMode` 需与 ingress-gateway chart 的 `gateway.gatewayRouteMode` 一致（默认均为 `header`）： ```yaml # values-server.yaml server: env: - name: OPENSANDBOX_SERVER_API_KEY valueFrom: secretKeyRef: name: opensandbox-api-key key: api-key gateway: enabled: true host: opensandbox-ingress-gateway.opensandbox-system.svc secureAccess: activeKey: a keys: - key_id: a key: <部署网关时生成的 base64 密钥> ``` ::: info `configToml`（沙箱工作负载命名空间、镜像地址、运行时等）与 PostgreSQL 持久化等也在这个文件中配置，渲染前补齐。完整参考见 [Kubernetes 部署](/deployment/)。 ::: 渲染并部署： ```bash helm template opensandbox-server manifests/charts/server \ -n opensandbox-system \ -f values-server.yaml > out/server.yaml kubectl diff -f out/server.yaml kubectl apply --server-side -f out/server.yaml kubectl rollout status deployment/opensandbox-server \ --namespace opensandbox-system --timeout=180s kubectl port-forward --namespace opensandbox-system \ service/opensandbox-server 8080:80 curl --fail http://127.0.0.1:8080/health ``` 预期输出： ```text deployment "opensandbox-server" successfully rolled out {"status":"healthy"} ``` ### 可选：部署共享 SandboxPool SandboxPool（`sandbox.fast.io`）是 fast-sandbox 的容量与策略单元，定义 Fastlet 的预热规模（`capacity`）、运行时 profile（`runtime`）、每个沙箱的资源规格（`sandboxResources`）与调度约束（`fastletTemplate`），同一池中的沙箱共享这些设置。仓库提供了示例 `manifests/examples/sandboxpool-fast-sandbox.yaml`： ```yaml apiVersion: sandbox.fast.io/v1alpha2 kind: SandboxPool metadata: name: shared-pool namespace: opensandbox-dataplane spec: runtime: firecracker sandboxResources: cpu: "1" memory: 2Gi pids: 256 capacity: poolMin: 5 poolMax: 10 bufferMin: 0 bufferMax: 2 maxSandboxesPerPod: 4 fastletTemplate: spec: containers: - name: fastlet image: opensandbox/fsb-fastlet:release-1.1.1-rc.1 imagePullPolicy: IfNotPresent env: - name: FAST_SANDBOX_RUNTIME_AGENT_SOCKET value: /run/fast-sandbox/firecracker/runtime.sock volumeMounts: - name: agent-socket mountPath: /run/fast-sandbox/firecracker volumes: - name: agent-socket hostPath: path: /run/fast-sandbox/firecracker type: DirectoryOrCreate nodeSelector: fast-sandbox.io/firecracker-node: "true" affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: labelSelector: matchLabels: app: sandbox-fastlet topologyKey: kubernetes.io/hostname warmImages: [] ``` 注意：`fastletTemplate` 必须包含名为 `fastlet` 的容器（fast-sandbox companion 镜像），并挂载节点上 runtime-agent 的 UDS socket，否则 controller 会拒绝处理。需要 egress 网络策略时，参考集成环境的完整示例 `scripts/fast-sandbox-env/manifests/pool/firecracker-egress-pool.yaml`（含 egress sidecar 与 `infraComponents`/`actionHandlers` 声明）。 部署并验证： ```bash kubectl apply --server-side -f manifests/examples/sandboxpool-fast-sandbox.yaml kubectl get pods -n opensandbox-dataplane kubectl get sandboxpool shared-pool -n opensandbox-dataplane -o jsonpath='{.status.conditions[?(@.type=="RuntimeReady")].status}' ``` `poolMin=5` 表示预热 5 个 Fastlet（反亲和分散到各节点），全部就绪后 `RuntimeReady` 转为 `True`： ```text NAME READY STATUS RESTARTS AGE shared-pool-fastlet-5nh85 2/2 Running 0 54s shared-pool-fastlet-cxrln 2/2 Running 0 54s shared-pool-fastlet-gzhml 2/2 Running 0 54s shared-pool-fastlet-wnlsx 2/2 Running 0 54s shared-pool-fastlet-zdrd4 2/2 Running 0 54s True ``` ### 升级 切换到新版本标签，重新渲染全部组件，diff 审阅后逐个 apply： ```bash git fetch --tags git checkout release-1.2.0 helm template base manifests/charts/base > out/base.yaml helm template opensandbox-controller manifests/charts/controller \ -n opensandbox-system > out/controller.yaml helm template ingress-gateway manifests/charts/ingress-gateway \ -n opensandbox-system \ --set gateway.fastpathEndpoint=fast-sandbox-fastpath.opensandbox-system.svc:9090 \ > out/ingress-gateway.yaml helm template opensandbox-server manifests/charts/server \ -n opensandbox-system -f values-server.yaml > out/server.yaml kubectl diff -f out/base.yaml && kubectl apply --server-side -f out/base.yaml kubectl diff -f out/controller.yaml && kubectl apply --server-side -f out/controller.yaml kubectl diff -f out/server.yaml && kubectl apply --server-side -f out/server.yaml ``` `kubectl diff` 在存在差异时退出码为 `1`，脚本化时不应视为失败。 ### 对外暴露 Server 在 `values-server.yaml` 中设置 `server.service.type: LoadBalancer`（ACK 会自动创建 SLB），重新渲染并 apply。生产环境建议使用内网 SLB，沙箱流量经由 ingress gateway。 ## ACK 环境注意事项 | 主题 | 说明 | |------|------| | 网络插件 | 推荐 Terway；Flannel 模式下 Pod IP 为 overlay 地址，对外暴露沙箱前需确认路由可达 | | fast-sandbox | 必须使用裸金属节点（KVM）且 CPU 型号一致，否则跳过该组件，使用默认运行时。详见 [fast-sandbox runtime](https://github.com/opensandbox-group/OpenSandbox/blob/main/manifests/HELM-DEPLOYMENT.md#fast-sandbox-runtime-firecracker) | | 镜像拉取 | 默认镜像托管在 Docker Hub，VPC 内建议通过 ACR 同步后覆盖 `server.image.repository` 及 `configToml` 中的沙箱镜像地址 | | Server 持久化 | 生产环境建议使用 RDS for PostgreSQL（见 [PostgreSQL 持久化](/deployment/#use-postgresql-for-server-persistence)） | | 监控 | Controller 的 Prometheus 指标见 [Operator Metrics](/deployment/#operator-metrics)，可接入 ARMS Prometheus | ## 卸载 按部署的逆序删除渲染产物。注意：删除 `out/base.yaml` 会连同 CRD 一起删除（kubectl 不理会 `resource-policy: keep`），并级联删除所有自定义资源： ```bash kubectl delete -f out/server.yaml kubectl delete -f out/ingress-gateway.yaml kubectl delete -f out/fast-sandbox.yaml # 如部署 kubectl delete -f out/controller.yaml kubectl delete -f out/base.yaml ``` 如需保留 CRD、仅删除组件，跳过 `out/base.yaml` 即可；CRD 清理见 [Kubernetes 部署 → Uninstall](/deployment/#uninstall)。 ## 相关链接 - [Kubernetes 部署](/deployment/) — 组件完整安装与配置 - [Kubernetes Controller 概述](/architecture/control-plane/operator) — CRD 与控制器机制 - [ACK 快速入门](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/getting-started/) - [创建 ACK 托管集群](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-2/) - [创建和管理节点池](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/create-a-node-pool/) - [添加已有节点](https://help.aliyun.com/zh/ack/ack-managed-and-ack-dedicated/user-guide/add-existing-ecs-instances-to-an-ack-cluster/) --- ### Examples Agent Sandbox (`docs/examples/agent-sandbox.md`) --- title: Agent Sandbox description: Create a kubernetes-sigs/agent-sandbox instance and run a command using the OpenSandbox Python SDK. --- # Agent-Sandbox Example This example creates a sandbox backed by `kubernetes-sigs/agent-sandbox` and executes `echo hello world` via the OpenSandbox Python SDK. ## Prerequisites - A Kubernetes cluster with agent-sandbox >= v0.5.0 controller and CRDs installed, serving `agents.x-k8s.io/v1beta1`. - OpenSandbox server configured with Kubernetes runtime and `workload_provider = "agent-sandbox"`. - Sandbox image should include `bash` (default example uses `ubuntu:22.04`). ## Compatibility and migration The server provider and ingress informer use `agents.x-k8s.io/v1beta1` only. This API is available starting in agent-sandbox v0.5.0; v1.0.0 removes `v1alpha1`. Upgrade older controller and CRD installations before using this integration; there is no `v1alpha1` fallback. For an existing installation, first upgrade agent-sandbox to a release that serves both versions (such as v0.5.0), with its conversion webhook healthy, then upgrade the OpenSandbox server and ingress together. Complete the upstream resource/storage migration before installing an agent-sandbox release that removes the alpha API. Update custom AgentSandbox templates to use `apiVersion: agents.x-k8s.io/v1beta1` and remove `spec.replicas`. The server sets `spec.operatingMode: Running` and `spec.service: true`, including when merging a custom template. Service creation is required for ingress routing through `status.serviceFQDN`. The server applies these fields when creating new sandboxes; it does not rewrite existing resources. On v0.5.0, alpha-to-beta conversion preserves an omitted `service` field. If an existing sandbox has no Service/FQDN, explicitly enable `spec.service: true` on that resource and wait for `Ready=True` and a populated `status.serviceFQDN` before using ingress. This can restore routing without recreating the Sandbox. Keep intentionally service-free resources unchanged. The configured `shutdownPolicy`, pod template merging, and absolute `shutdownTime` semantics are preserved. Requests without an expiration omit `shutdownTime`, including any value supplied by a template. Status and endpoint resolution still use `Ready`, `selector`, `podIPs`, and `serviceFQDN`. OpenSandbox's own BatchSandbox API remains `sandbox.opensandbox.io/v1alpha1`. ## Pause and resume This provider supports the standard `POST /v1/sandboxes/{id}/pause` and `POST /v1/sandboxes/{id}/resume` endpoints. Both are implemented by patching the `Sandbox` CRD's `spec.operatingMode` field. Pause patches `spec.operatingMode: Suspended`. The agent-sandbox controller terminates and destroys the Pod but retains the `Sandbox` object, so the `sandboxId` stays stable across pause/resume cycles. The sandbox reports `Running` → `Pausing` → `Paused`; while the Pod is terminating, the status `reason` comes from the `Suspended` condition (e.g. `PodTerminating`). Resume patches `spec.operatingMode: Running`. The controller recreates the Pod from the Sandbox spec, so expect scheduling and image-pull latency before the sandbox is usable again. The sandbox reports `Paused` → `Resuming` → `Pending` → `Running`. | | Preserved? | |--|-----------| | Persistent volume (PVC) data | ✅ Yes — volumes are retained with the Sandbox object | | `sandboxId` and sandbox metadata | ✅ Yes | | Memory / running processes | ❌ No — the Pod is destroyed | | Root filesystem runtime state | ❌ No — the Pod is recreated from the original image on resume | Anything not written to a persistent volume is lost on pause. A paused sandbox still expires at its original timeout; a retained expired sandbox reports `Terminated` and can no longer be paused or resumed. ## Start OpenSandbox server 1. Install the server package and fetch the example config for agent-sandbox: ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker ``` 2. Update `~/.sandbox.toml` with the following sections: ```toml [runtime] type = "kubernetes" execd_image = "opensandbox/execd:v1.1.0" [kubernetes] namespace = "default" # kubeconfig_path = "/absolute/path/to/kubeconfig" # optional if running in-cluster workload_provider = "agent-sandbox" [agent_sandbox] shutdown_policy = "Delete" ``` 3. Start the server: ```shell opensandbox-server ``` ## Run the example ```shell uv pip install opensandbox uv run python examples/agent-sandbox/main.py ``` ## Expected output ```text command output: hello world ``` ## References - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/agent-sandbox) --- ### Examples Aio Sandbox (`docs/examples/aio-sandbox.md`) --- title: AIO Sandbox description: Create and access an All-in-One (AIO) Sandbox via OpenSandbox. --- # All-in-One (AIO) Sandbox Example This example demonstrates how to create and access an [All-in-One (AIO) Sandbox](https://github.com/agent-infra/sandbox) via OpenSandbox. ## Start OpenSandbox server [local] You can find the latest version [here](https://github.com/agent-infra/sandbox/pkgs/container/sandbox). You can pre-pull the target image which is used in the example. ::: info Docker runtime requirement The server is configured with `runtime.type = "docker"` by default, so it **must** be able to connect to a running Docker daemon. - **Docker Desktop**: ensure Docker Desktop is running, then verify with `docker version`. - **Colima (macOS)**: start it first (`colima start`) and export the socket before starting the server: ```shell export DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock" ``` ::: ```shell # pre-pull target image docker pull ghcr.io/agent-infra/sandbox:latest ``` Then, start the OpenSandbox server, you can obtain stdout log from terminal. ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker opensandbox-server ``` ::: tip `opensandbox-server` runs in the foreground and will keep the current terminal session busy. The example code lives in this repository -- clone it and, in a new terminal window/tab, `cd` into the project root before running the AIO sandbox creation steps below. If you see errors like `FileNotFoundError: [Errno 2] No such file or directory` from `docker/transport/unixconn.py`, it usually means the Docker unix socket is missing / Docker daemon is not running. ::: ## Create and Access the AIO Sandbox Instance This example uses a fixed configuration for quick start: - OpenSandbox server: `http://localhost:8080` - Image: `ghcr.io/agent-infra/sandbox:latest` - AIO port: `8080` - Timeout: `300s` Install dependencies with uv under project root: ```shell uv pip install opensandbox agent-sandbox==0.0.18 ``` Run the example (it will create a sandbox via OpenSandbox, wait until it's Running, then connect to it via agent-sandbox): ```shell uv run python examples/aio-sandbox/main.py ``` Subsequently, you will instantiate an AIO sandbox, navigate to Google, capture a screenshot, and download it to your local environment. ```text Creating AIO sandbox with image=ghcr.io/agent-infra/sandbox:latest on OpenSandbox server http://localhost:8080... [check] sandbox ready after 7.1s AIO portal endpoint: 127.0.0.1:56123 ... Screenshot saved to sandbox_screenshot.png ``` ## More examples For more examples of using the AIO Sandbox, refer to agent-infra/sandbox [examples](https://github.com/agent-infra/sandbox/tree/main/examples). ## References - [AIO Sandbox](https://github.com/agent-infra/sandbox/tree/main) - [AIO Sandbox Python SDK](https://github.com/agent-infra/sandbox/tree/main/sdk/python) - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/aio-sandbox) --- ### Examples Aks Kata (`docs/examples/aks-kata.md`) --- title: AKS + Kata description: Deploy OpenSandbox onto an AKS cluster with Kata VM isolation and interact with a Kata-isolated sandbox step by step, covering ingress, egress, and Credential Vault. --- # AKS Kata Example Deploy OpenSandbox onto an AKS cluster with `kata-vm-isolation`, then interact with a Kata-isolated sandbox step by step. ::: warning Demo credentials The Helm values in this example use **insecure, well-known demo credentials** (`api_key = "aks-kata-demo-key"` and a fixed `secureAccess` signing key) so the walkthrough works out of the box with local `kubectl port-forward`. **Change them before exposing the server or gateway beyond local port-forwarding.** See [Security Model](#security-model). ::: ## Prerequisites - AKS cluster with `kubectl get runtimeclass` showing `kata-vm-isolation` - Helm 3 - Python 3.10+ with `pip install opensandbox requests` - Azure OpenAI resource (endpoint + API key) ## Files The example lives in [`examples/aks-kata`](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/aks-kata): | File | Purpose | |------|---------| | `main.py` | CLI tool — create, inspect, and operate on sandboxes one step at a time | | `controller-values.yaml` | Helm values for the OpenSandbox controller | | `server-values.yaml` | Helm values for the lifecycle server and ingress gateway | | `batchsandbox-template-configmap.yaml` | BatchSandbox template that pins pods onto Kata nodes | ## 1. Install OpenSandbox From the repo root: ```bash kubectl create namespace opensandbox-system --dry-run=client -o yaml | kubectl apply -f - kubectl create namespace opensandbox --dry-run=client -o yaml | kubectl apply -f - kubectl apply -f examples/aks-kata/batchsandbox-template-configmap.yaml # The controller chart no longer ships the CRDs; install base first on a fresh cluster helm upgrade --install base ./manifests/charts/base helm upgrade --install opensandbox-controller ./manifests/charts/controller \ --namespace opensandbox-system \ -f examples/aks-kata/controller-values.yaml helm upgrade --install opensandbox-server ./manifests/charts/server \ --namespace opensandbox-system \ -f examples/aks-kata/server-values.yaml ``` Wait for the deployments: ```bash kubectl rollout status deploy/opensandbox-controller-manager -n opensandbox-system --timeout=180s kubectl rollout status deploy/opensandbox-server -n opensandbox-system --timeout=180s kubectl rollout status deploy/opensandbox-ingress-gateway -n opensandbox-system --timeout=180s ``` ## 2. Start port-forwards Open two separate terminals: ```bash # Terminal 1 — lifecycle server kubectl port-forward -n opensandbox-system svc/opensandbox-server 18080:80 # Terminal 2 — ingress gateway kubectl port-forward -n opensandbox-system svc/opensandbox-ingress-gateway 28080:80 ``` ## 3. Set environment variables ```bash export SANDBOX_DOMAIN=http://127.0.0.1:18080 export SANDBOX_API_KEY=aks-kata-demo-key export AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com export AZURE_OPENAI_API_KEY=your-real-key # Optional: # export AZURE_OPENAI_DEPLOYMENT=gpt-4o-mini ``` ## 4. Enable Pause/Resume (Optional) Pause commits the sandbox rootfs as an OCI image and pushes it to a registry. Resume recreates the sandbox from that image. Without a registry, the `pause` and `resume` steps will log an error and continue. The steps below use Azure Container Registry (ACR). Any OCI registry works — see the [Pause / Resume guide](/guides/pause-resume) for the full guide. ### Step 1: Grant AcrPush to the kubelet identity ```bash KUBELET_ID=$(az aks show \ -g -n \ --query "identityProfile.kubeletidentity.clientId" -o tsv) ACR_ID=$(az acr show --name --query id -o tsv) az role assignment create --assignee "$KUBELET_ID" --role AcrPush --scope "$ACR_ID" ``` ### Step 2: Create the push/pull secret ```bash ACR_PASSWORD=$(az acr credential show --name \ --query "passwords[0].value" -o tsv) kubectl create secret docker-registry acr-snapshot-push-secret \ --docker-server=.azurecr.io \ --docker-username= \ --docker-password="$ACR_PASSWORD" \ --namespace=opensandbox ``` ### Step 3: Upgrade the controller ```bash helm upgrade opensandbox-controller ./manifests/charts/controller \ --namespace opensandbox-system \ --reuse-values \ --set controller.snapshot.registry=.azurecr.io/opensandbox-snapshots \ --set controller.snapshot.snapshotPushSecret=acr-snapshot-push-secret \ --set controller.snapshot.resumePullSecret=acr-snapshot-push-secret kubectl rollout status deploy/opensandbox-controller-manager \ -n opensandbox-system --timeout=90s ``` ::: tip `controller.snapshot.containerdSocketPath` defaults to `""` in the chart, which allows the controller to use its built-in default (`/var/run/containerd/containerd.sock`) without passing the `--containerd-socket-path` flag unless explicitly configured. If your nodes use a non-default containerd socket and you deploy a controller image that supports the flag, set this value accordingly. ::: ## 5. Use `main.py` All `main.py` commands below are run from the example directory: ```bash cd examples/aks-kata ``` ### Run everything end to end ```bash python3 main.py all ``` This creates a sandbox, runs every demo step, and deletes it at the end. ### Run one step at a time #### Create a sandbox ```bash python3 main.py create ``` Expected output: ``` Creating Kata-isolated sandbox on AKS... OpenSandbox API: http://127.0.0.1:18080 Sandbox image: sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/code-interpreter:v1.1.0 SANDBOX_ID=96045ee2-6614-435c-9fa8-d4b6f9592598 Use --sandbox-id 96045ee2-6614-435c-9fa8-d4b6f9592598 for subsequent steps. ``` Save the ID for the following steps: ```bash export SANDBOX_ID=96045ee2-6614-435c-9fa8-d4b6f9592598 ``` Verify the pod is running on a Kata node: ```bash kubectl get pod -n opensandbox -o wide # NODE column should show aks-sandboxagent-* kubectl get pod -n opensandbox -o jsonpath='{.items[0].spec.runtimeClassName}' # Expected: kata-vm-isolation python3 main.py exec --sandbox-id $SANDBOX_ID -c "uname -r" # Expected: 6.6.137.mshv1-1.azl3 (mshv = Microsoft Hypervisor = Kata VM guest kernel) ``` #### Set up Credential Vault ```bash python3 main.py credentials --sandbox-id $SANDBOX_ID ``` Expected output: ``` [credentials] Credential Vault configured. ``` #### Ask the LLM a question ```bash python3 main.py llm --sandbox-id $SANDBOX_ID -q "What is the capital of France? Reply in one word." ``` Expected output: ``` [llm] Question: What is the capital of France? Reply in one word. [llm] Model: gpt-4o-mini [llm] Answer: Paris ``` The sandbox only has a fake API key — the real key is injected by the egress sidecar via Credential Vault. You can verify: ```bash python3 main.py exec --sandbox-id $SANDBOX_ID -c "echo \$AZURE_OPENAI_API_KEY" # Expected: fake-key-inside-sandbox ``` #### Run commands inside the sandbox ```bash python3 main.py exec --sandbox-id $SANDBOX_ID -c "uname -a" # [exec][stdout] Linux ...-0 6.6.137.mshv1-1.azl3 ... x86_64 GNU/Linux python3 main.py exec --sandbox-id $SANDBOX_ID -c "cat /etc/os-release | head -3" # [exec][stdout] PRETTY_NAME="Ubuntu 24.04.4 LTS" # [exec][stdout] NAME="Ubuntu" # [exec][stdout] VERSION_ID="24.04" python3 main.py exec --sandbox-id $SANDBOX_ID -c "ls -la /tmp/www/" # Shows the HTTP server root directory ``` #### Fetch from the sandbox HTTP server The sandbox runs `python3 -m http.server 8080` serving `/tmp/www/`. Traffic goes through the ingress gateway with secure-access headers. ```bash # Write a file into the sandbox, then fetch it via the gateway python3 main.py exec --sandbox-id $SANDBOX_ID -c "echo hello > /tmp/www/greeting.txt" python3 main.py http --sandbox-id $SANDBOX_ID -p /greeting.txt # [http] GET /greeting.txt -> 200 (6 bytes) # hello # Directory listing python3 main.py http --sandbox-id $SANDBOX_ID -p / # Shows HTML directory listing # 404 for missing file python3 main.py http --sandbox-id $SANDBOX_ID -p /nonexistent.txt # [http] GET /nonexistent.txt -> 404 ``` #### Check sandbox status ```bash python3 main.py status --sandbox-id $SANDBOX_ID # Sandbox: 96045ee2-6614-435c-9fa8-d4b6f9592598 # State: Running # Image: ...code-interpreter:v1.1.0 ``` #### Pause and resume Pause snapshots the container rootfs to the registry (1–5 min). Resume recreates the sandbox from that snapshot. Requires step 4 above. ```bash python3 main.py pause --sandbox-id $SANDBOX_ID # [lifecycle] pausing sandbox... # [lifecycle] sandbox is PAUSED python3 main.py status --sandbox-id $SANDBOX_ID # State: Paused # Pod is gone, but the BatchSandbox CR and snapshot remain kubectl get pods -n opensandbox # No resources found kubectl get sandboxsnapshot -n opensandbox # Shows snapshot with phase Succeed python3 main.py resume --sandbox-id $SANDBOX_ID # [lifecycle] resuming sandbox... # [setup] sandbox state: Resuming, waiting... # [lifecycle] state after resume: Running # [http] GET /index.html -> 200 (filesystem preserved!) python3 main.py status --sandbox-id $SANDBOX_ID # State: Running ``` #### Delete the sandbox ```bash python3 main.py delete --sandbox-id $SANDBOX_ID # [lifecycle] sandbox 96045ee2-... deleted. kubectl get pods -n opensandbox # No resources found ``` Without `-c`, `-q`, or `-p`, the `exec`, `llm`, and `http` steps run their built-in demo sequences. ## What gets installed | Component | Namespace | Purpose | |-----------|-----------|---------| | `opensandbox-controller` | `opensandbox-system` | Kubernetes operator — manages BatchSandbox CRDs, pools, and snapshots | | `opensandbox-server` | `opensandbox-system` | Lifecycle API — creates/deletes sandboxes, proxies SDK calls | | `opensandbox-ingress-gateway` | `opensandbox-system` | Routes external HTTP traffic into sandbox pods with auth | | BatchSandbox template | `opensandbox-system` | Pins sandbox pods onto Kata-capable AKS nodes via `nodeSelector` | The server is configured with: - `workload_provider = "batchsandbox"` - `secure_runtime.type = "kata"` / `k8s_runtime_class = "kata-vm-isolation"` - egress sidecar (`dns+nft` mode) - ingress gateway (`header` routing with `secureAccess`) ## Security Model ::: warning `server-values.yaml` ships with **demo-only** credentials: `api_key = "aks-kata-demo-key"` and a fixed base64 `secureAccess` signing key. They exist so the local port-forward walkthrough works without extra setup. Replace both with unique secret values (for example a random key and `openssl rand -base64 32` for the signing key) before exposing the server or gateway to anything other than `127.0.0.1`. ::: ### Kata VM isolation Each sandbox runs inside a dedicated Kata VM (`runtimeClassName: kata-vm-isolation`) on an AKS Kata node pool (`nodeSelector: kubernetes.azure.com/kata-vm-isolation: "true"`). The sandbox sees its own guest kernel, not the host kernel. ### Egress isolation The create request sets a deny-by-default outbound policy. Only explicitly allowed hosts (Azure OpenAI, pypi.org, files.pythonhosted.org) can be reached. ### Credential Vault The sandbox only sees a fake `AZURE_OPENAI_API_KEY`. The real key is stored in Credential Vault and injected by the egress sidecar as the `api-key` header only for matching outbound requests to `https://.openai.azure.com/openai/*`. ### Ingress gateway + secureAccess All external traffic to sandbox services flows through the ingress gateway. The gateway validates the `OpenSandbox-Secure-Access` token and routes via the `OpenSandbox-Ingress-To` header. Both are returned by the SDK's `get_endpoint()` call — callers never need to construct them manually. ### Per-service auth Even if someone bypasses the gateway and reaches the pod directly: - **execd** (port 44772) requires its own access token header - **egress sidecar** (port 18080) requires the `OPENSANDBOX-EGRESS-AUTH` header - Only the user's HTTP server (port 8080) has no built-in auth — that's what secureAccess protects ## Cleanup Remove everything installed by this example: ```bash # 1) Delete any running sandboxes kubectl delete batchsandbox --all -n opensandbox # 2) Uninstall Helm releases helm uninstall opensandbox-server -n opensandbox-system helm uninstall opensandbox-controller -n opensandbox-system # 3) Remove the BatchSandbox template kubectl delete configmap aks-kata-batchsandbox-template -n opensandbox-system # 4) Remove the snapshot push/pull secret (if created) kubectl delete secret acr-snapshot-push-secret -n opensandbox --ignore-not-found # 5) Delete the namespaces kubectl delete namespace opensandbox kubectl delete namespace opensandbox-system ``` --- ### Examples Apply Patch (`docs/examples/apply-patch.md`) --- title: Apply Model Patches description: Apply Codex-style file patches in an OpenSandbox sandbox with a custom image. --- # Apply Model Patches OpenSandbox exposes file operations and command execution, but the default `execd` image does not include a patch interpreter. The `/files/replace` API performs literal string replacement; it does not understand contextual hunks, file creation, deletion, or moves. If your coding agent emits the patch format used by OpenAI Codex, install the standalone `apply_patch` executable in a custom sandbox image and run it inside the sandbox. This keeps the patch dialect in the agent image instead of adding vendor-specific behavior to the OpenSandbox file API. ## Build a custom image The following example builds the executable from a pinned Codex revision and copies it into the OpenSandbox `execd` image. Pin `CODEX_REF` to a reviewed commit in production so image builds remain reproducible. ```dockerfile FROM rust:alpine AS apply-patch-builder ARG CODEX_REF= RUN apk add --no-cache build-base git musl-dev WORKDIR /src RUN git clone --depth 1 https://github.com/openai/codex.git . \ && git fetch --depth 1 origin ${CODEX_REF} \ && git checkout ${CODEX_REF} WORKDIR /src/codex-rs RUN cargo build --release --locked -p codex-apply-patch FROM opensandbox/execd:v1.1.0 COPY --from=apply-patch-builder /src/codex-rs/target/release/apply_patch /usr/local/bin/apply_patch RUN chmod 0755 /usr/local/bin/apply_patch ``` Build and publish the image, then use its name when creating the sandbox: ```shell docker build --build-arg CODEX_REF= \ -t my-registry/opensandbox-execd-with-patch:1 . docker push my-registry/opensandbox-execd-with-patch:1 ``` The builder and runtime stages must produce compatible binaries. If your base image is not Alpine, use a builder with the matching C library and toolchain. ## Apply a patch from the Python SDK Write the model output to a temporary file, then invoke `apply_patch` through the existing command API. The executable accepts either one patch argument or the patch on standard input; using a file avoids shell quoting and argument length problems. Run the following snippet inside an async function. It assumes `/workspace/hello.py` already exists and contains `print("hello")`. ```python import sys from opensandbox import Sandbox patch = """*** Begin Patch *** Update File: /workspace/hello.py @@ -print("hello") +print("hello from OpenSandbox") *** End Patch """ async with (await Sandbox.create(image="my-registry/opensandbox-execd-with-patch:1")) as sandbox: await sandbox.files.write_file("/tmp/change.patch", patch) result = await sandbox.commands.run( "cd /workspace && apply_patch < /tmp/change.patch" ) stdout = "".join(message.text for message in result.logs.stdout) stderr = "".join(message.text for message in result.logs.stderr) print(stdout, end="") print(stderr, end="", file=sys.stderr) if result.exit_code != 0: raise RuntimeError(f"apply_patch failed (exit code: {result.exit_code})") ``` Patch paths are resolved relative to the command working directory unless the patch uses an absolute path. Keep the working directory and patch paths inside the sandbox workspace, and treat patch content as untrusted input from the model. ## Operational notes - `apply_patch` can add, update, delete, and move files and matches update hunks using surrounding context. - A failed patch can leave earlier file changes committed. Check the command exit status and inspect the affected files before retrying. - The default OpenSandbox images remain unchanged. Use the custom image only for agents that need this patch format. - For agents that already manage their own editing tools, continue using the regular file and command APIs instead of installing this executable. See the [OpenAPI specifications](/api/) and the [Codex CLI example](/examples/codex-cli) for the surrounding sandbox setup. --- ### Examples Chrome (`docs/examples/chrome.md`) --- title: Chrome description: Run Chrome Browser with OpenSandbox runtime for remote debugging via DevTools. --- # Chrome Browser in OpenSandbox This example runs Chrome Browser with OpenSandbox runtime. The image starts a VNC server (`Xtigervnc :1`) and launches Chromium with remote debugging enabled on port `9222`. ## Getting Chrome image You can build the image from source or pull it from Docker Hub. ### Build from source ```shell cd examples/chrome docker build -t opensandbox/chrome . ``` ### Pull an existing image ```shell docker pull opensandbox/chrome:latest # use acr from china # docker pull sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/chrome:latest ``` ## Start OpenSandbox server Start the OpenSandbox server and tail stdout from the terminal: ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker opensandbox-server ``` ## Create and access a Chrome sandbox Build/pull the image above, then create a sandbox with image `opensandbox/chrome:latest` and an entrypoint that keeps it alive (e.g., `["/bin/sh", "-c", "sleep infinity"]`), or reuse `tail -f /dev/null`. Make sure the runtime exposes ports `5901` and `9222` for VNC/DevTools. ```shell uv pip install opensandbox uv run python examples/chrome/main.py ``` Then fetch endpoints for 5901/9222 to connect with a VNC client or DevTools, like: ```text execd daemon running with endpoint='127.0.0.1:48379/proxy/44772' VNC running with endpoint='127.0.0.1:48379/proxy/5901' DevTools running with endpoint='127.0.0.1:48379/proxy/9222'/json ``` ```text [ { "description": "", "devtoolsFrontendUrl": "https://chrome-devtools-frontend.appspot.com/serve_rev/@.../inspector.html?ws=127.0.0.1:52302/devtools/page/...", "faviconUrl": "https://www.gstatic.com/images/branding/searchlogo/ico/favicon.ico", "id": "2215AF60AC345E4BA6D822389CFC743B", "title": "Google", "type": "page", "url": "https://www.google.com.hk/", "webSocketDebuggerUrl": "ws://127.0.0.1:52302/devtools/page/2215AF60AC345E4BA6D822389CFC743B" } ] ``` Or you can use it by MCP client, more information please refer to: [chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp). ## References - [chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/chrome) --- ### Examples Claude Code (`docs/examples/claude-code.md`) --- title: Claude Code description: Access Claude via the claude-cli npm package in an OpenSandbox container. --- # Claude Code Example Access Claude via the `claude-cli` npm package in OpenSandbox. ## Start OpenSandbox server [local] Pre-pull the code-interpreter image (includes Node.js): ```shell docker pull sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/code-interpreter:v1.1.0 # use docker hub # docker pull opensandbox/code-interpreter:v1.1.0 ``` Then start the local OpenSandbox server, stdout logs will be visible in the terminal: ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker opensandbox-server ``` ## Create and Access the Claude Sandbox ```shell # Install OpenSandbox package uv pip install opensandbox # Run the example (requires SANDBOX_DOMAIN / SANDBOX_API_KEY / ANTHROPIC_AUTH_TOKEN) uv run python examples/claude-code/main.py ``` The script installs the Claude CLI (`npm i @anthropic-ai/claude-code@latest`) at runtime (Node.js is already in the code-interpreter image), then sends a simple request `claude "Compute 1+1=?."`. Auth is passed via `ANTHROPIC_AUTH_TOKEN`, and you can override endpoint/model with `ANTHROPIC_BASE_URL` / `ANTHROPIC_MODEL`. ## Headless Mode and Session Resume The example script also runs the two patterns used by coding-agent integrations: a headless run with structured output, and a follow-up turn that resumes the same conversation. ### Turn 1: headless run with structured output ```shell claude -p "Remember this for later: my favorite sandbox number is 42." --output-format json ``` With `-p` (print mode) the CLI answers the prompt and exits. `--output-format json` prints the reply, `session_id`, usage, and cost metadata as a single JSON object, so the script extracts `session_id` for the next turn. ### Turn 2: resume the session ```shell claude -p "What is my favorite sandbox number? Reply with just the number." \ --resume --output-format json ``` `--resume` continues a specific conversation with its context intact — the reply recalls "42" from turn 1. `--continue` resumes the most recent conversation instead; an explicit `--resume ` stays deterministic when several conversations live in the same sandbox. ::: tip Concurrent follow-ups Add `--fork-session` to `--resume` / `--continue` to create a new session ID instead of reusing the original, so parallel follow-ups on the same conversation don't overwrite each other. ::: ### Permission prompts in unattended runs There is no terminal to answer permission prompts in a `-p` run: requests that would prompt are denied, so a task that needs a tool (file edits, shell commands) can fail. Two common remedies for agent workloads: - `--dangerously-skip-permissions` (equivalent to `--permission-mode bypassPermissions`) runs tools without prompts — a common choice inside an ephemeral OpenSandbox container, where the sandbox itself is the isolation boundary. - `--permission-mode dontAsk` stays fail-closed: only tools pre-approved by your permission rules run — the standard pattern for locked-down CI. (`--permission-prompts none`, Claude Code v2.1.259+, is the print-mode equivalent.) ::: warning The interactive UI needs a TTY The interactive REPL (`claude "..."`) renders its UI, including permission prompts, on a terminal. Through a plain command pipe there is no way to answer a prompt, so the run can wait indefinitely. Use `-p` for scripted runs, or drive the interactive UI over a [PTY session](/architecture/data-plane/execd). ::: To receive output as it is generated (token-level events) instead of a single final JSON object, use `--output-format stream-json --verbose --include-partial-messages`. ## Environment Variables | Variable | Default | Description | |----------|---------|-------------| | `SANDBOX_DOMAIN` | `localhost:8080` | Sandbox service address | | `SANDBOX_API_KEY` | _(optional for local)_ | API key if your server requires authentication | | `SANDBOX_IMAGE` | `sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/code-interpreter:v1.1.0` | Sandbox image to use | | `ANTHROPIC_AUTH_TOKEN` | _(required)_ | Your Anthropic auth token | | `ANTHROPIC_BASE_URL` | _(optional)_ | Anthropic API endpoint (e.g., self-hosted proxy) | | `ANTHROPIC_MODEL` | `claude_sonnet4` | Model name | ## References - [claude-code](https://www.npmjs.com/package/claude-code) - NPM package for Claude Code CLI - [Claude Code headless mode](https://code.claude.com/docs/en/headless) - Print mode, output formats, and session resume - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/claude-code) --- ### Examples Client Pool Schedule (`docs/examples/client-pool-schedule.md`) --- title: Scheduled Client Pool Sizing description: Adjust a Go SandboxPool's idle target during business hours using an application-level controller and Resize. --- # Scheduled Client Pool Sizing Keep more sandboxes ready during business hours and reduce idle capacity at night or on weekends. This Go example evaluates a small application-owned schedule and calls the existing `SandboxPool.Resize` API. The SDK's reconciler handles creating and removing idle sandboxes. The schedule uses **Monday–Friday, 09:00 inclusive to 21:00 exclusive**, in an IANA time zone such as `Asia/Shanghai` or `America/New_York`. Its default targets are five idle sandboxes during business hours and zero outside them. Edit `desiredMaxIdle` in the example to implement your own demand or budget policy. Source: [`examples/client-pool-schedule/`](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/client-pool-schedule). ## Run Install Go 1.20 or newer, check out this repository, and start an [OpenSandbox server](/getting-started/). The example's Go module uses the SDK from the same checkout through a local `replace` directive. ```bash cd examples/client-pool-schedule export OPEN_SANDBOX_DOMAIN=localhost:8080 export OPEN_SANDBOX_API_KEY=your-api-key go run . -timezone Asia/Shanghai -peak-idle 5 -offpeak-idle 0 ``` Use `OPEN_SANDBOX_PROTOCOL=https` for an HTTPS server. The example routes sandbox health checks through the server proxy. It uses `ubuntu:22.04` with an entrypoint that keeps the sandbox alive; use `-image` to select another compatible image. Run `go run . -h` for all flags. ::: warning Sandbox resources This command creates real sandboxes when the scheduled target is positive. Use small targets while trying it out. Ctrl+C stops reconciliation, then drains the example's remaining idle sandboxes and waits for their best-effort kill requests to finish before exiting. Failed kills may leave sandboxes running until their TTL expires. Sandboxes already acquired by your application remain the caller's responsibility. ::: ## How resizing works - The pool starts with the target for the current time, avoiding an initial business-hours warmup when started at night. - Every 30 seconds, the controller compares the desired target with `Snapshot().MaxIdle`. It calls `Resize` only when those targets differ, so a temporary idle deficit does not trigger repeated resize calls. - Target updates are asynchronous. The reconcile interval, sandbox startup and readiness time, and warmup concurrency determine how quickly capacity converges. Move the business-hours start earlier if capacity must be ready before traffic arrives. - Scale-down removes surplus **idle** sandboxes. Acquired sandboxes are outside the idle buffer. A target of zero stops replenishment; it does not disable direct creation on acquire or act as a total sandbox quota. - The time zone is evaluated on each check, including daylight-saving changes. The example embeds Go's timezone database so it also works on hosts without system zoneinfo files. - A snapshot or resize error stops the example and runs cleanup. It does not call `Start` or clear namespace-destroy state to recover from an error. ## Multiple application processes This runnable example uses an **in-memory store**, so its pool belongs to one process. For a distributed pool, configure every participant with the same Redis store and `pool_name`, and a unique `owner_id`; see the [client pool guide](/guides/client-pool). Run only one application-level scheduling controller for each shared namespace, or elect one in your application. `Resize` updates the shared target, so you do not need a cron job on every worker. The pool's warmup leader election does not elect a leader for your external scheduling policy. Avoid competing manual and scheduled writers: this example reapplies the scheduled target on the next check. Coordinate worker startup and restarts with the controller, because starting a pool can write its configured initial target to the store. The example's shutdown drain is intended for its single-process pool. Do not copy that drain into the shutdown of one worker of a shared pool: it would drain the shared idle buffer while other workers may still replenish it. Use the documented namespace retirement procedure when retiring an entire distributed pool. ## Test without a server ```bash go test ./... go vet ./... ``` Tests cover weekday and weekend boundaries, time-zone conversion, daylight-saving transitions, and target updates through the real SDK's in-memory state store. A local HTTP test server verifies that shutdown waits for pending kill requests. --- ### Examples Code Interpreter (`docs/examples/code-interpreter.md`) --- title: Code Interpreter description: Complete demonstration of running Python code using the Code Interpreter SDK with OpenSandbox. --- # Code Interpreter Sandbox Complete demonstration of running Python code using the Code Interpreter SDK. ## Getting Code Interpreter image ::: info Image Source The container image source and environment definitions are maintained in [opensandbox-group/sandbox-images](https://github.com/opensandbox-group/sandbox-images). ::: Pull the prebuilt image from a registry: ```shell docker pull sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/code-interpreter:v1.1.0 # use docker hub # docker pull opensandbox/code-interpreter:v1.1.0 ``` ## Start OpenSandbox server [local] Start the local OpenSandbox server: ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker opensandbox-server ``` ## Create and access the Code Interpreter Sandbox ```shell # Install OpenSandbox packages uv pip install opensandbox opensandbox-code-interpreter # Run the example (requires SANDBOX_DOMAIN / SANDBOX_API_KEY) uv run python examples/code-interpreter/main.py ``` The script creates a Sandbox + CodeInterpreter, runs a Python code snippet and prints stdout/result, then terminates the remote instance. Cleanup is attempted in a `finally` block even if interpreter setup or code execution raises an exception. ## Environment variables | Variable | Default | Description | |----------|---------|-------------| | `SANDBOX_DOMAIN` | `localhost:8080` | Sandbox service address | | `SANDBOX_API_KEY` | _(optional)_ | API key if your server requires authentication | | `SANDBOX_IMAGE` | `sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/code-interpreter:v1.1.0` | Sandbox image to use | ## Example output ```text === Python example === [Python stdout] Hello from Python! [Python result] {'py': '3.14.2', 'sum': 4} === Java example === [Java stdout] Hello from Java! [Java stdout] 2 + 3 = 5 [Java result] 5 === Go example === [Go stdout] Hello from Go! 3 + 4 = 7 === TypeScript example === [TypeScript stdout] Hello from TypeScript! [TypeScript stdout] sum = 6 ``` ## Code Interpreter Sandbox from pool ### Start OpenSandbox server [k8s] Install the k8s OpenSandbox operator, and create a pool: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` #### How Pool entrypoint injection works The lifecycle API allocates an already-running Pod from the Pool, so it does not replace that Pod's `command`, `args`, or `env`. When a create request supplies an `entrypoint` or environment variables, the server records them in `BatchSandbox.spec.taskTemplate`. The controller then sends the task to the allocated Pod's IP on port `5758`. The Pool template must provide all parts of that execution path: - Install and run task-executor, listening on `0.0.0.0:5758`. Set its `-log-dir` explicitly so the troubleshooting path is deterministic; the example writes `/tmp/task-executor.log`. - Install execd and `bootstrap.sh` into the shared volume before the Pod starts. - Keep `bootstrap.sh` at `/opt/opensandbox/bootstrap.sh`. The server-generated task invokes that exact path. The execd binary can use another path only when the task-executor environment sets `EXECD` accordingly. - Start execd through `bootstrap.sh` after allocation so request-specific values such as `EXECD_ACCESS_TOKEN` are available. The example above leaves task-executor as the warm Pod's foreground process for this reason. The Pod YAML continuing to show the Pool template is therefore expected. Inspect the `BatchSandbox` resource and task-executor instead: ```shell # Confirm that the server injected the requested process and environment. kubectl get batchsandbox -n \ -o jsonpath='{.spec.taskTemplate}{"\n"}' # Find the allocated Pod. The annotation value contains a JSON `pods` array. kubectl get batchsandbox -n \ -o jsonpath='{.metadata.annotations.sandbox\.opensandbox\.io/alloc-status}{"\n"}' # Replace with the first Pod name from that array. kubectl exec -n -- \ sh -c 'test -x /opt/opensandbox/task-executor && test -x /opt/opensandbox/bootstrap.sh' kubectl exec -n -- \ tail -n 100 /tmp/task-executor.log # Check the executor health endpoint from a second terminal while this runs. kubectl port-forward pod/ -n 5758:5758 curl http://127.0.0.1:5758/health curl http://127.0.0.1:5758/getTasks # The lifecycle server uses -0 as the task name. Check the task's # captured output (adjust the path if task-executor uses a custom data directory). kubectl exec -n -- \ sh -c 'tail -n 100 /var/lib/sandbox/tasks/-0/stdout.log; tail -n 100 /var/lib/sandbox/tasks/-0/stderr.log' # Check controller logs for delivery failures between the controller and port 5758. kubectl logs -n opensandbox-system -l control-plane=controller-manager --tail=100 ``` If `taskTemplate` exists but the health check cannot reach port `5758`, verify that task-executor is installed and remains running. The generated task intentionally starts `bootstrap.sh` in the background, so its wrapper can report success even when `bootstrap.sh` is missing or the requested entrypoint later fails. Do not rely on `taskFailed` or `taskLastErrorMessage` alone for these failures; inspect the task's captured `stderr.log` and `stdout.log`, then verify the execd or application process directly. Start the k8s OpenSandbox server: ```shell uv pip install opensandbox-server # replace with your k8s cluster config, kubeconfig etc. opensandbox-server init-config ~/.sandbox.toml --example k8s curl -o ~/batchsandbox-template.yaml https://raw.githubusercontent.com/opensandbox-group/OpenSandbox/main/server/opensandbox_server/examples/example.batchsandbox-template.yaml opensandbox-server ``` ### Create and access the Code Interpreter Sandbox (pool) ```shell # Install OpenSandbox packages uv pip install opensandbox opensandbox-code-interpreter # Run the example (requires SANDBOX_DOMAIN / SANDBOX_API_KEY) uv run python examples/code-interpreter/main_use_pool.py ``` ### Pool example output ```text === Verify Environment Variable === [ENV Check] TEST_ENV value: test [ENV Result] 'test' === Java example === [Java stdout] Hello from Java! [Java stdout] 2 + 3 = 5 [Java result] 5 === Go example === [Go stdout] Hello from Go! 3 + 4 = 7 === TypeScript example === [TypeScript stdout] Hello from TypeScript! [TypeScript stdout] sum = 6 ``` ## References - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/code-interpreter) - [Code Interpreter Image Repository](https://github.com/opensandbox-group/sandbox-images) --- ### Examples Codex Cli (`docs/examples/codex-cli.md`) --- title: Codex CLI description: Use the official @openai/codex npm package to call OpenAI/Codex-like models in OpenSandbox. --- # Codex/OpenAI CLI Example Use the official `@openai/codex` npm package to call OpenAI/Codex-like models in OpenSandbox. ## Start OpenSandbox server [local] Pre-pull the code-interpreter image (includes Node.js): ```shell docker pull sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/code-interpreter:v1.1.0 # use docker hub # docker pull opensandbox/code-interpreter:v1.1.0 ``` Start the local OpenSandbox server, logs will be visible in the terminal: ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker opensandbox-server ``` ## Create and Access the Codex Sandbox ```shell # Install OpenSandbox package uv pip install opensandbox # Run the example (requires SANDBOX_DOMAIN / SANDBOX_API_KEY / OPENAI_API_KEY) uv run python examples/codex-cli/main.py ``` The script installs the Codex CLI (`npm install @openai/codex@latest`) at runtime (Node.js is already in the code-interpreter image), then executes a simple request `codex exec "Compute 1+1 and return JSON with keys result and reasoning." --skip-git-repo-check`. Auth is passed via `OPENAI_API_KEY`; you can override endpoint/model with `OPENAI_BASE_URL` / `OPENAI_MODEL`. ## Multi-Turn Sessions: Resume a Previous Run Each `codex exec` run is a session. For pipelines that need a follow-up turn (review → fix, plan → implement), resume the session instead of starting over with a fresh context. Turn 1 — run with `--json` and capture the session id. `--json` turns stdout into a JSON Lines (JSONL) event stream; the first event, `thread.started`, carries the `thread_id`, which is the session id that `codex exec resume` accepts: ```shell codex exec --json "Remember this for later: my favorite sandbox number is 42." --skip-git-repo-check ``` ```json {"type": "thread.started", "thread_id": "0199a213-81c0-7800-8aa1-bbab2a035a53"} ``` Turn 2 — resume that session with a follow-up prompt; the model recalls the context of the previous turns, so the reply is `42`: ```shell codex exec resume "0199a213-81c0-7800-8aa1-bbab2a035a53" "What is my favorite sandbox number? Reply with just the number." --skip-git-repo-check ``` ::: tip `codex exec resume --last "..."` continues the most recent session from the current working directory — convenient for quick two-stage pipelines. The example script captures the explicit id instead, which stays correct when several sessions interleave. ::: If you only need the final message for a downstream step, `-o ` / `--output-last-message ` writes it to a file, and `--output-schema ` validates the final response against a JSON Schema. ## Sandbox and Approval Modes Codex applies its own sandbox policy to model-generated shell commands; these flags control it inside the OpenSandbox container: - `--sandbox workspace-write` (or `-s workspace-write`) lets model-generated commands write inside the workspace — the usual choice for autonomous runs. - `--sandbox read-only` keeps the run strictly read-only. - `-a never` / `--ask-for-approval never` never pauses for human approval, so a non-interactive run cannot block on a person. - `--dangerously-bypass-approvals-and-sandbox` (alias `--yolo`) runs every command without approvals or sandboxing. The Codex CLI documents it as safe only "inside an externally hardened environment" — in OpenSandbox terms, that means a sandbox running a hardened runtime (gVisor, Kata), not the default runc; see [Secure Container Runtimes](/guides/secure-container). ::: warning `--full-auto` is deprecated — prefer `--sandbox workspace-write`; Codex prints a warning when the old flag is used. ::: ## Environment Variables | Variable | Default | Description | |----------|---------|-------------| | `SANDBOX_DOMAIN` | `localhost:8080` | Sandbox service address | | `SANDBOX_API_KEY` | _(optional for local)_ | API key if your server requires authentication | | `SANDBOX_IMAGE` | `sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/code-interpreter:v1.1.0` | Sandbox image to use | | `OPENAI_API_KEY` | _(required)_ | Your OpenAI API key | | `OPENAI_BASE_URL` | `https://api.openai.com/v1` | OpenAI API endpoint | | `OPENAI_MODEL` | `gpt-4o-mini` | Model to use | ## References - [@openai/codex](https://www.npmjs.com/package/@openai/codex) - Official OpenAI Codex CLI - [Codex non-interactive mode](https://developers.openai.com/codex/noninteractive) - `codex exec`, JSONL output, and session resume - [Codex CLI reference](https://developers.openai.com/codex/cli/reference) - Sandbox and approval flags - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/codex-cli) --- ### Examples Deep Agents (`docs/examples/deep-agents.md`) --- title: Deep Agents description: Run a Deep Agent with its file and shell tools executing inside an OpenSandbox sandbox. --- # Deep Agents + OpenSandbox Example Run a [Deep Agent](https://github.com/langchain-ai/deepagents) whose file and shell tools execute inside an OpenSandbox sandbox. The [`langchain-sandbox-opensandbox`](https://pypi.org/project/langchain-sandbox-opensandbox/) backend adapts the OpenSandbox Python SDK to the Deep Agents `BaseSandbox` interface, so every `ls` / `read_file` / `write_file` / `glob` / `grep` / command the agent performs is sandboxed. The backend passes the full `langchain-tests` `SandboxIntegrationTests` conformance suite (86/86) against a live server. ## Start OpenSandbox server [local] Start a local OpenSandbox server, logs will be visible in the terminal: ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker opensandbox-server ``` ## Run the example ```shell # Install Deep Agents + the OpenSandbox backend uv pip install deepagents langchain-sandbox-opensandbox # Run the example (requires SANDBOX_DOMAIN / ANTHROPIC_API_KEY) uv run python examples/deep-agents/main.py ``` The agent writes a Python script into the sandbox, runs it, and reports the output — all file and shell operations happen inside the sandbox rather than on the host. The sandbox is destroyed on exit (terminated and local resources closed). ## Environment Variables | Variable | Default | Description | |----------|---------|-------------| | `SANDBOX_DOMAIN` | `localhost:8080` | Sandbox service address (host and optional port) | | `SANDBOX_PROTOCOL` | `http` | Protocol used to reach the server (`http` or `https`) | | `SANDBOX_API_KEY` | _(optional for local)_ | API key if your server requires authentication | | `ANTHROPIC_API_KEY` | _(required)_ | Anthropic API key for the default Deep Agents model | ## References - [Deep Agents](https://github.com/langchain-ai/deepagents) - Agent framework - [langchain-sandbox-opensandbox](https://pypi.org/project/langchain-sandbox-opensandbox/) - OpenSandbox backend for Deep Agents - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/deep-agents) --- ### Examples Deer Flow (`docs/examples/deer-flow.md`) --- title: DeerFlow description: Run DeerFlow agent turns on OpenSandbox through the OpenSandboxProvider shipped in DeerFlow 2.1.0. --- # DeerFlow + OpenSandbox Example [DeerFlow](https://github.com/bytedance/deer-flow) is an open-source deep-research agent framework. Since 2.1.0 it ships an OpenSandbox sandbox provider, so the `bash`, file, and search tools an agent calls run inside a sandbox on your OpenSandbox server instead of on the gateway host. The provider lives in DeerFlow (`deerflow.community.opensandbox:OpenSandboxProvider`) and drives the OpenSandbox Python SDK. Nothing extra is needed on the OpenSandbox side: this example starts a server, points DeerFlow's `sandbox` section at it, and runs sandbox work through DeerFlow's own provider. ## Start OpenSandbox server [local] Start a local OpenSandbox server, logs will be visible in the terminal: ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker opensandbox-server ``` ::: info Docker runtime requirement The server uses `runtime.type = "docker"` by default, so it **must** be able to reach a running Docker daemon. Docker Desktop users should confirm with `docker version`; on Colima (macOS) export `DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock"` before starting the server. ::: ## Set up DeerFlow with the OpenSandbox provider DeerFlow is a cloned application rather than a PyPI library — the harness and its `deerflow-extension-api` companion are uv workspace members — so the provider extra is installed from a checkout. DeerFlow requires Python 3.12 or later. ```shell git clone https://github.com/bytedance/deer-flow.git cd deer-flow # optional: the full local workspace setup (backend, frontend, and pre-commit hooks) make install # backend dependencies plus the OpenSandbox provider cd backend && uv sync --all-packages --extra opensandbox && cd .. ``` The provider is imported only when `sandbox.use` selects it. ::: warning Re-add the extra after a plain sync `opensandbox` is an optional extra, and a plain `uv sync --locked` — which is what `make install` runs for the backend — prunes it from the environment. If a later `make install` or `uv sync` removes it, run `cd backend && uv sync --all-packages --extra opensandbox` again. The docker-compose deployment is unaffected: it installs the extra at image build time from `UV_EXTRAS=opensandbox` in `.env`. ::: ## Run the example The example drives DeerFlow's provider directly, so it needs no model configuration and no gateway process. It writes a minimal `config.yaml` containing only the `sandbox` section and points `DEER_FLOW_CONFIG_PATH` at it. Run it with the DeerFlow backend environment, from your OpenSandbox checkout: ```shell uv run --project ../deer-flow/backend python examples/deer-flow/main.py ``` Any interpreter that can `import deerflow` works — for example a shell where the DeerFlow checkout's `backend/.venv` is activated. The script creates a sandbox through `provider.acquire()`, runs a command, writes a Python script with DeerFlow's file API, executes it, then exercises `read_file`, `list_dir`, `glob`, `grep`, and the `/mnt/user-data`-restricted `download_file`. The sandbox is released to the warm pool and destroyed by `provider.shutdown()` on exit. You should see output similar to: ```text [config] DeerFlow config: /tmp/opensandbox-deerflow-xxxx/config.yaml [create] DeerFlow sandbox c37d0a7b8a926499 bound to remote b5419ac9-f80b-4263-b403-9bd74ee54c43 [command] Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.39 [file] Wrote /mnt/user-data/workspace/fib.py [command] [0, 1, 1, 2, 3, 5, 8, 13, 21, 34] [file] Read back: def fibonacci(count): [list_dir] ['/mnt/user-data/workspace', '/mnt/user-data/workspace/fib.py'] [glob] ['/mnt/user-data/workspace/fib.py'] [grep] /mnt/user-data/workspace/fib.py:1: def fibonacci(count): [download] 170 bytes fetched from /mnt/user-data/workspace/fib.py [cleanup] Sandbox destroyed ``` The sandbox id, remote id, and platform string differ on each run. ::: tip DeerFlow logs `No models are configured ...` while loading this generated config. That notice is expected on the provider-only path — the script never invokes a model. ::: ## Point the DeerFlow agent at OpenSandbox For agent runs, put the sandbox section in the DeerFlow checkout's `config.yaml` (`make config` creates one from `config.example.yaml`): ```yaml sandbox: use: deerflow.community.opensandbox:OpenSandboxProvider image: python:3.11 domain: localhost:8080 protocol: http ready_timeout: 120 # create/readiness deadline; covers a first-run image pull sandbox_timeout: 14400 # remote lifetime in seconds; 0 means explicit cleanup only bash_command_timeout: 600 # default deadline for commands run in the sandbox replicas: 3 # active + warm sandboxes per gateway process idle_timeout: 600 # destroy a warm sandbox after this many idle seconds; 0 disables ``` | Option | Default | Description | |--------|---------|-------------| | `image` | `python:3.11` | Sandbox image. Any image with a POSIX shell works. | | `domain` | `OPEN_SANDBOX_DOMAIN`, else `localhost:8080` | OpenSandbox server address | | `api_key` | `OPEN_SANDBOX_API_KEY` | API key, if the server requires authentication | | `protocol` | `http` | Use `https` for any non-loopback domain | | `request_timeout` | `30` | Management API request timeout in seconds | | `ready_timeout` | `30` | Create and readiness deadline in seconds | | `use_server_proxy` | `false` | Route execd and file traffic through the server | | `sandbox_timeout` | `14400` | Server-side lifetime in seconds; `0` disables renewal | | `bash_command_timeout` | `600` | Default command deadline in seconds | | `replicas` | `3` | Active plus warm sandbox cap per gateway process | | `idle_timeout` | `600` | Idle seconds before a warm sandbox is destroyed; `0` disables | | `environment` | `{}` | Environment variables injected into sandbox commands. A value starting with `$` is resolved from the gateway process environment. | `api_key` and `domain` may be omitted when `OPEN_SANDBOX_API_KEY` and `OPEN_SANDBOX_DOMAIN` are exported, which is how the example passes them. Set `use_server_proxy: true` when the gateway can reach the OpenSandbox management service but cannot reach sandbox `execd` endpoints directly. Then run one headless agent turn. The prompt makes the agent call its `bash` tool, which now executes inside OpenSandbox: ```shell deerflow --print "Run python3 -c 'print(2 ** 16)' and report the output" ``` `deerflow` opens the terminal workbench when run interactively, while `--print` and `--json` run one headless turn. The gateway and web UI start with `make dev`, and the docker-compose deployment reaches the provider through the `UV_EXTRAS=opensandbox` setting in `.env`. ## Behavior notes - The provider creates one sandbox per effective `(user_id, thread_id)` scope and parks it in an in-process warm pool after each turn. Only the same scope can reclaim it, and a reclaim runs a health check first. - Create returns only after the SDK readiness check and DeerFlow's `/mnt/user-data/{workspace,uploads,outputs}` bootstrap succeed. A bootstrap failure destroys the newly created remote sandbox. - Every operation renews the sandbox's server-side lifetime; commands that carry no explicit timeout use `bash_command_timeout`. Operations on one sandbox are serialized so a short renewal cannot shorten the horizon of an in-flight long command. - `execute_command` forwards per-call environment variables and timeouts, and preserves stdout, stderr, and non-zero exit status in the returned text. File reads and writes use OpenSandbox's native filesystem API. - `list_dir`, `glob`, and `grep` run portable `find`/`grep` commands inside the sandbox. All paths must be absolute and traversal-free; artifact downloads are restricted to `/mnt/user-data`. - A command-path HTTP 404 or 410, an unhealthy session, or a broken transport evicts the dead client so the next acquire cold-starts a replacement. - `replicas` is a per-gateway-process soft cap: DeerFlow does not coordinate sandbox ownership between processes yet, so one gateway process per OpenSandbox-backed deployment keeps the cap meaningful. ## Environment Variables | Variable | Default | Description | |----------|---------|-------------| | `SANDBOX_DOMAIN` | `localhost:8080` | Server address (host and optional port) | | `SANDBOX_PROTOCOL` | `http` | Protocol used to reach the server (`http` or `https`) | | `SANDBOX_API_KEY` | _(optional for local)_ | API key if your server requires authentication | | `SANDBOX_IMAGE` | `python:3.11` | Sandbox image used by this example | | `OPEN_SANDBOX_DOMAIN` | _(unset)_ | Server address when the DeerFlow config omits `domain` | | `OPEN_SANDBOX_API_KEY` | _(unset)_ | API key when the DeerFlow config omits `api_key` | ## All-in-one Docker image (server + DeerFlow Web UI) `examples/deer-flow/` ships an all-in-one lab image bundling both sides of the integration: the OpenSandbox server (docker runtime) and the DeerFlow backend with the OpenSandboxProvider extra preinstalled — plus the **pre-built Web UI**, served by the bundled nginx on `:2026`. Sandbox containers run on the **host** Docker daemon: the image mounts `docker.sock` at run time and drives the host daemon through it — the same topology as `server/docker-compose.example.yaml`. Build and run: ```shell cd examples/deer-flow docker build -t deerflow-opensandbox . docker run -d --name deerflow-opensandbox \ -p 2026:2026 \ -p 8080:8080 \ -v /var/run/docker.sock:/var/run/docker.sock \ --add-host host.docker.internal:host-gateway \ deerflow-opensandbox ``` `--add-host host.docker.internal:host-gateway` is required on Linux so the server can reach the host-published sandbox ports; Docker Desktop resolves that name already, and the flag is harmless there. Two entry points: - **DeerFlow Web UI**: — the teaching path. Complete the first-run setup in the page, configure a model under Settings → Models (API key stays in the Web UI), then chat: the agent's `bash` and file tools execute inside an OpenSandbox sandbox. - **OpenSandbox server API**: — for `curl` checks and the `osb` CLI. Verify from a shell on the host: ```shell curl -s http://127.0.0.1:8080/health # -> {"status":"healthy"} docker ps | grep -i sandbox # sandbox containers appear as you chat ``` Headless / model-free paths inside the container (`docker exec -it deerflow-opensandbox bash`): ```shell # 1) model-free: drive OpenSandboxProvider directly (no API key needed) cd /opt/OpenSandbox uv run --project /opt/deer-flow/backend python examples/deer-flow/main.py # 2) full agent turn via CLI (requires a configured model) deerflow --print "Run python3 -c 'print(2 ** 16)' and report the output" ``` ::: tip On first boot the entrypoint generates `/opt/deer-flow/config.yaml` from `config.example.yaml` and switches the sandbox section to `OpenSandboxProvider` via `enable-opensandbox` (on PATH, idempotent). If `make setup` or any wizard later overwrites the config, rerun `enable-opensandbox` to switch back. ::: ::: warning The image ships a fixed teaching API key (`deerflow-lab-key`, set as `server.api_key` and `OPEN_SANDBOX_API_KEY`) — required since the server refuses to start with an empty key in non-interactive mode. Fine for a classroom laptop; for any shared deployment mount your own config and override the env var. ::: Notes: - The Web UI stack starts in production mode (background `serve.sh --prod --daemon --skip-install --skip-frontend-build`), reusing the frontend build baked at image build time; the frontend build requires Node.js + pnpm, both installed in the image. - `UV_EXTRAS=opensandbox` is set in the image so DeerFlow's own start-time `uv sync --locked` (in `scripts/serve.sh`) keeps the `opensandbox` extra instead of pruning it. - Sandbox images default to the lab image itself (`opensandbox/deerflow-example:latest` on the ACR mirror — a python base with a rich preinstalled environment); any image with a POSIX shell works, e.g. Docker Hub `python:3.11`. Images are pulled by the **host** Docker; the provider's `ready_timeout` is set to 600s to cover the first pull of the larger default. - `DEERFLOW_REPO` / `DEERFLOW_REF` / `OPENSANDBOX_REPO` build args let network-restricted environments point the clones at mirrors; pip/uv honor `PIP_INDEX_URL` / `UV_INDEX_URL`, and pnpm uses the configured registry. ## References - [DeerFlow](https://github.com/bytedance/deer-flow) - Open-source deep-research agent framework - [DeerFlow OpenSandbox provider](https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/community/opensandbox/README.md) - Provider documentation - [deer-flow PR #4877](https://github.com/bytedance/deer-flow/pull/4877) - OpenSandbox provider contribution - [OpenSandbox Python SDK](https://pypi.org/project/opensandbox/) - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/deer-flow) --- ### Examples Desktop (`docs/examples/desktop.md`) --- title: Desktop (VNC) description: Launch Xvfb + x11vnc + fluxbox in OpenSandbox to provide a VNC-accessible desktop environment. --- # Desktop (VNC) Example Launch Xvfb + x11vnc + fluxbox in OpenSandbox to provide a VNC-accessible desktop environment. ## Build the Desktop Sandbox Image The Dockerfile in the example directory builds a sandbox image with desktop and VNC components pre-installed: ```shell cd examples/desktop docker build -t opensandbox/desktop:latest . ``` This image includes: - Xvfb (virtual framebuffer X server) - x11vnc (VNC server) - XFCE desktop (panel, file manager, terminal) - Non-root user (desktop) for security ## Start OpenSandbox server [local] Pre-pull the desktop image: ```shell docker pull opensandbox/desktop:latest ``` Start the local OpenSandbox server: ```shell uv pip install opensandbox-server opensandbox-server init-config ~/.sandbox.toml --example docker opensandbox-server ``` ## Create and Access the Desktop Sandbox ```shell # Install OpenSandbox package uv pip install opensandbox uv run python examples/desktop/main.py ``` The script starts the desktop stack (Xvfb + XFCE + x11vnc) and also launches noVNC/websockify. It prints: - VNC endpoint (`endpoint.endpoint`) for native VNC clients when direct endpoint mode is enabled, password from `VNC_PASSWORD` (default: `opensandbox`) - noVNC URL for browsers (`/vnc.html?host=...&port=...&path=...`) The sandbox stays alive for 5 minutes by default; interrupt sooner with Ctrl+C. Uses the prebuilt desktop image by default. ### Embed noVNC in an HTTPS page Browsers block an `http://` noVNC iframe inside an HTTPS page. Terminate TLS at a reverse proxy in front of the OpenSandbox server, then make the example use that HTTPS origin and the server proxy: ```shell export SANDBOX_DOMAIN=sandbox.example.com export SANDBOX_PROTOCOL=https export SANDBOX_USE_SERVER_PROXY=true export VNC_PASSWORD=opensandbox uv run python examples/desktop/main.py ``` The generated URL uses the same HTTPS origin for both the noVNC page and its WebSocket, for example: ```text https://sandbox.example.com/v1/sandboxes//proxy/6080/vnc.html?host=sandbox.example.com&port=443&path=v1/sandboxes//proxy/6080 ``` The TLS reverse proxy must forward normal HTTP requests and WebSocket upgrades to the OpenSandbox server. For Nginx, the relevant settings are: ```nginx map $http_upgrade $connection_upgrade { default upgrade; '' close; } server { listen 443 ssl; server_name sandbox.example.com; # Configure ssl_certificate and ssl_certificate_key here. location / { proxy_pass http://127.0.0.1:8080; proxy_http_version 1.1; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto https; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; } } ``` This direct browser URL works when the OpenSandbox proxy route does not require browser-supplied authentication headers, including the default single-tenant mode. In multi-tenant mode, the proxy route requires `OPEN-SANDBOX-API-KEY` on both the initial noVNC HTTP request and the WebSocket upgrade. The SDK adds this header to its own requests, but a browser navigation or WebSocket cannot add it. For multi-tenant browser access, put a trusted, browser-authenticated reverse proxy in front of OpenSandbox. It must authorize access to the requested sandbox, map the browser identity to the correct tenant, and inject that tenant's `OPEN-SANDBOX-API-KEY` header for both HTTP and WebSocket traffic. Do not put the API key in the noVNC URL or expose a reverse proxy that adds one shared key to unauthenticated traffic. ::: warning Do not expose a direct `http://:` URL in an HTTPS iframe. That endpoint has no TLS termination and the browser will reject it as mixed content. This SDK configuration uses one protocol for the management API and sandbox service requests, so the example requires `SANDBOX_USE_SERVER_PROXY=true` whenever the management API uses HTTPS. This keeps lifecycle, execd, noVNC page, and WebSocket traffic on the HTTPS server origin instead of incorrectly sending HTTPS to a direct sandbox endpoint. Server proxy mode does not support the raw TCP protocol used by native VNC clients, so the example omits the native VNC endpoint in this mode. ::: ## Environment Variables | Variable | Default | Description | |----------|---------|-------------| | `SANDBOX_DOMAIN` | `localhost:8080` | Sandbox service address; may include an `http://` or `https://` scheme | | `SANDBOX_PROTOCOL` | Domain scheme, otherwise `http` | Protocol used when `SANDBOX_DOMAIN` has no scheme (`http` or `https`) | | `SANDBOX_USE_SERVER_PROXY` | `false` | Route sandbox service, noVNC HTTP, and WebSocket traffic through the OpenSandbox server; required with HTTPS | | `SANDBOX_API_KEY` | _(optional for local)_ | Example-specific API key; takes precedence over `OPEN_SANDBOX_API_KEY` | | `OPEN_SANDBOX_API_KEY` | _(optional for local)_ | SDK-standard API key fallback when `SANDBOX_API_KEY` is unset | | `SANDBOX_IMAGE` | `opensandbox/desktop:latest` | Sandbox image to use | | `VNC_PASSWORD` | `opensandbox` | Password for VNC access | ## References - [noVNC](https://github.com/novnc/noVNC) - [x11vnc](https://github.com/LibVNC/x11vnc) - [Source code on GitHub](https://github.com/opensandbox-group/OpenSandbox/tree/main/examples/desktop) --- ### Examples Docker Ossfs Volume Mount (`docs/examples/docker-ossfs-volume-mount.md`) --- title: Docker OSSFS Volume description: Mount Alibaba Cloud OSS into sandboxes on Docker runtime using the OSSFS volume model. --- # Docker OSSFS Volume Mount Example This example demonstrates how to use the SDK `ossfs` volume model to mount Alibaba Cloud OSS into sandboxes on Docker runtime. ## What this example covers 1. **Basic read-write mount** on an OSSFS backend. 2. **Cross-sandbox sharing** on the same OSSFS backend path. 3. **Two mounts, different OSS prefixes via `subPath`**. ## Prerequisites ### 1. Start OpenSandbox server (Docker runtime) Make sure your server host has: - Linux host OS (OSSFS backend is not supported when OpenSandbox Server runs on Windows) - `ossfs` installed - FUSE support enabled - writable local mount root for OSSFS (default `storage.ossfs_mount_root=/mnt/ossfs`) `storage.ossfs_mount_root` is **optional** if you use the default `/mnt/ossfs`. Even with on-demand mounting, the runtime still needs a deterministic host-side base directory to place dynamic mounts (`//`). Optional config example: ```toml [runtime] type = "docker" [storage] ossfs_mount_root = "/mnt/ossfs" ``` Then start the server: ```bash opensandbox-server ``` ### 2. Install Python SDK ```bash uv pip install opensandbox ``` If your PyPI version does not include OSSFS volume models yet, install from source: ```bash pip install -e sdks/sandbox/python ``` ### 3. Prepare OSS credentials and target path ```bash export SANDBOX_DOMAIN=localhost:8080 export SANDBOX_API_KEY=your-api-key export SANDBOX_IMAGE=ubuntu export OSS_BUCKET=your-bucket export OSS_ENDPOINT=oss-cn-hangzhou.aliyuncs.com export OSS_ACCESS_KEY_ID=your-ak export OSS_ACCESS_KEY_SECRET=your-sk ``` ## Run ```bash uv run python examples/docker-ossfs-volume-mount/main.py ``` ## Minimal SDK usage snippet ```python from opensandbox import Sandbox from opensandbox.models.sandboxes import OSSFS, Volume sandbox = await Sandbox.create( image="ubuntu", volumes=[ Volume( name="oss-data", ossfs=OSSFS( bucket="your-bucket", endpoint="oss-cn-hangzhou.aliyuncs.com", # version="2.0", # optional, default is "2.0" accessKeyId="your-ak", accessKeySecret="your-sk", ), mountPath="/mnt/data", subPath="train", # optional readOnly=False, # optional ) ], ) ``` ## Notes ::: info Implementation details - Current implementation supports **inline credentials only** (`accessKeyId`/`accessKeySecret`). - Mounting is **on-demand** in Docker runtime (mount-or-reuse), not pre-mounted for all buckets. - `ossfs.version` exists in API/SDK with enum `"1.0" | "2.0"`, and defaults to `"2.0"` when omitted. - Docker runtime now applies **version-specific mount argument encoding**: - `1.0`: mounts via `ossfs ... -o