{"owner":"replicate","repo":"cog","hasSkills":true,"totalSkillsCount":20,"totalTokensCount":28613,"categories":["anthropic-skill","root-instruction","subagent-persona","plugin-manifest","project-spec"],"hasMcp":false,"mcpConfig":null,"found":[".agents/skills/release-cog/SKILL.md",".opencode/skills/cog-review/SKILL.md",".opencode/skills/python-review/SKILL.md",".opencode/skills/rust-review/SKILL.md",".opencode/skills/go-review/SKILL.md",".agents/skills/updating-architecture-docs/SKILL.md","AGENTS.md",".opencode/agents/reviewer.md","tools/test-harness/README.md","DESIGN.md","tools/test-harness/manifest.yaml","docs/llms.txt","tools/test-registry-util/README.md","tools/test-harness/fixtures/models/scalar-types/cog.yaml","tools/test-harness/fixtures/models/optional-types/cog.yaml","tools/test-harness/fixtures/models/optional-list-types/cog.yaml","tools/test-harness/fixtures/models/list-types/cog.yaml","tools/test-harness/fixtures/models/file-path-types/cog.yaml","tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml","tools/test-harness/fixtures/models/complex-output/cog.yaml"],"skills":{".agents/skills/release-cog/SKILL.md":"---\nname: release-cog\ndescription: Guide and automate the Cog release process\n---\n\n# Cog Release Skill\n\nThis skill helps you release new versions of Cog. Cog is a multi-language, multi-artifact project with a carefully orchestrated release process.\n\n## Overview\n\nCog releases include:\n\n- **CLI binaries** (Go) - macOS and Linux, x86_64 and ARM64\n- **coglet Python wheels** (Rust/PyO3) - Linux x86_64/ARM64, macOS ARM64\n- **cog SDK Python wheel** (Python) - Universal\n- **coglet Rust crate** - Published to crates.io\n\n## Release Types\n\n| Type | Format | Example | Branch | PyPI | crates.io | Homebrew |\n| --------------- | ------------------------------- | ------------ | ------ | ---- | --------- | -------- |\n| **Stable** | `v0.17.0` | v0.18.0 | main | ✓ | ✓ | ✓ |\n| **Pre-release** | `v0.17.0-alpha3`, `v0.17.0-rc1` | v0.18.0-rc1 | main | ✓ | ✓ | ✗ |\n| **Dev** | `v0.17.0-dev1` | v0.18.0-dev2 | any | ✗ | ✗ | ✗ |\n\n## Quick Release Commands\n\n### 1. Bump Version (if needed)\n\n```bash\n# Check current version\nmise run version\n\n# Bump to new version (updates VERSION.txt, Cargo.toml, Cargo.lock, commits)\nmise run version:bump 0.18.0\n```\n\n### 2. Create a release branch and push\n\n```bash\ngit checkout -b release/v0.18.0\ngit push origin release/v0.18.0\n```\n\n### 3. Create PR to main\n\n```bash\n# Open a pull request to main, get it reviewed and merged. Then you can create the release tag from main.\ngh pr create --base main --head release/v0.18.0 --title \"Release v0.18.0\" --body \"Release description and notes\"\n```\n\n### 4. Create and Push Tag\n\n```bash\ngit checkout main && git pull origin main\n\n# For stable release\ngit tag v0.18.0\ngit push origin v0.18.0\n\n# For pre-release\ngit tag v0.18.0-rc1\ngit push origin v0.18.0-rc1\n\n# For dev release (can be from any branch)\ngit tag v0.18.0-dev1\ngit push origin v0.18.0-dev1\n```\n\n### 5. Monitor Release Build\n\n```bash\n# Watch the release build workflow\ngh workflow view release-build.yaml\n\n# Or watch in real-time\ngh run watch\n```\n\n### 6. Write Release Notes\n\nAfter the draft release is created, update the release notes to follow the project's standard format. The release notes are not auto-generated from commit messages — they must be hand-written and categorized.\n\n**To see the previous release's format:**\n\n```bash\ngh release view v0.19.0 --json body\n```\n\n**To gather commits since the last release:**\n\n```bash\n# List commits between the previous release and this one\ngit log --oneline v0.19.0..v0.20.0 --no-merges\n```\n\n**Release notes structure:**\n\nGroup changes into three sections. Only include sections that have items.\n\n1. **New features** — New commands, new APIs, new annotations, new capabilities.\n2. **Improvements** — Performance, reliability, DX improvements, removals of legacy paths, build improvements.\n3. **Bug fixes** — User-visible bug fixes. Prefer \"Now does X correctly\" over \"Fixed X\".\n\n**Style guidelines:**\n\n- Lead each bullet with a **bold, user-facing summary sentence** (e.g., \"**`cog run` command.** ...\")\n- Follow with a short explanation of what changed and why it matters\n- Reference the PR number in parentheses at the end: `(#3015)`\n- Use backticks for commands, flags, and code references\n- Omit internal refactors, dependency bumps, and CI-only changes unless they are user-facing\n- Omit version bump commits\n\n**Example:**\n\n```markdown\n### New features\n\n- **`cog run` command.** The `cog predict` command has been renamed to `cog run` with full backward compatibility. `cog predict` still works as an alias. (#3015)\n- **Model refs for `cog push` and weights commands.** You can now reference models by name (e.g., `r8.im/user/model`) instead of full image URLs when pushing or managing weights. (#3018)\n\n### Improvements\n\n- **Runtime schema generation fully removed.** The legacy runtime Python schema generation path has been completely removed. Cog exclusively uses static schema generation, making builds faster and more reliable. (#3003)\n\n### Bug fixes\n\n- **Pushing a model with a version tag now emits a clean URL.** The Replicate model URL printed after `cog push` no longer includes the image tag (e.g., `:latest`), preventing 404 errors when users click the link. (#3020)\n```\n\n**To update the draft release:**\n\n```bash\ngh release edit v0.20.0 --notes \"$(cat <<'EOF'\n### New features\n\n- ...\n\n### Improvements\n\n- ...\n\n### Bug fixes\n\n- ...\nEOF\n)\"\n```\n\n### 7. Publish Release (stable/pre-release only)\n\n- Go to GitHub Releases page\n- Find the draft release\n- Review release notes\n- Click \"Publish release\"\n- This triggers `release-publish.yaml` which publishes to PyPI and crates.io\n\n## Release Process Details\n\n### Automated Workflows\n\n1. **`release-build.yaml`** - Triggered on version tags\n - Verifies tag matches VERSION.txt and Cargo.toml\n - Verifies stable/pre-release tags are on main branch\n - Builds SDK wheel (with updated coglet version constraint)\n - Builds coglet wheels for all platforms (Linux x64/ARM64, macOS ARM64)\n - Uses GoReleaser to build CLI binaries and create draft release\n - Uploads wheels to GitHub release\n - For dev releases: immediately publishes as pre-release\n\n2. **`release-publish.yaml`** - Triggered when release is published\n - Publishes coglet wheels to PyPI\n - Publishes coglet crate to crates.io\n - Publishes cog SDK to PyPI (depends on coglet)\n - Updates Homebrew tap (stable releases only)\n\n3. **`homebrew-tap.yaml`** - Updates Homebrew cask\n - Generates cask from `.github/cog.rb.tmpl`\n - Creates PR in `replicate/homebrew-tap`\n\n### Version Files\n\n| File | Purpose |\n| ------------------- | ------------------------------------------ |\n| `VERSION.txt` | Canonical version (single source of truth) |\n| `crates/Cargo.toml` | Rust workspace version |\n| `crates/Cargo.lock` | Locked dependency versions |\n\n### Version Constraints\n\nThe SDK (`pyproject.toml`) has a dependency on coglet:\n\n```toml\ncoglet>=0.1.0,<1.0\n```\n\nDuring release build, this is updated to:\n\n```toml\ncoglet>=0.18.0,<1.0\n```\n\nThis ensures the SDK depends on the matching coglet version.\n\n## Pre-Release Checklist\n\nBefore creating a release tag:\n\n- [ ] All tests pass: `mise run test`\n- [ ] Lint passes: `mise run lint`\n- [ ] Version is correct in `VERSION.txt`\n- [ ] `mise run version:check` passes\n- [ ] `crates/Cargo.toml` matches `VERSION.txt`\n- [ ] Changelog is updated (if applicable)\n- [ ] Documentation is updated (`mise run docs:llm`)\n\n## Troubleshooting\n\n### Version mismatch error\n\n```\nVersion mismatch! VERSION.txt has X but tag is vY\n```\n\nFix: Run `mise run version:bump Y`, push, then re-tag.\n\n### Tag not on main\n\n```\nRelease tags must be on the main branch\n```\n\nFix: Merge your changes to main, then tag from main.\n\n### Rebuilding a failed release\n\n1. Delete the GitHub release if it was created: `gh release delete v0.18.0 --yes`\n2. Delete the tag: `git push --delete origin v0.18.0 && git tag -d v0.18.0`\n3. Fix the issue\n4. Re-create and push the tag\n\n### Manual PyPI publish (emergency)\n\nIf the automated publish fails:\n\n```bash\n# Download wheels from GitHub release\ngh release download v0.18.0 -p \"coglet-*.whl\" -D dist\ngh release download v0.18.0 -p \"cog-*.whl\" -D dist\n\n# Publish with twine\n twine upload dist/coglet-*.whl # First!\ntwine upload dist/cog-*.whl # After coglet is uploaded\n```\n\n## Architecture Notes\n\n- **Trusted Publishing**: PyPI and crates.io use OIDC trusted publishing (no API tokens in secrets)\n- **Environments Required**: Configure `pypi`, `crates-io`, and `homebrew` environments in GitHub repo settings\n- **CGO**: Required for go-tree-sitter (static Python schema parser)\n- **Zig**: Used for Linux cross-compilation (CC=zig cc)\n- **macOS builds**: Use native compiler (zig lacks macOS SDK stubs)\n- **Wheel discovery**: CLI discovers wheels from `dist/` at Docker build time, not embedded in binary\n",".opencode/skills/cog-review/SKILL.md":"---\nname: cog-review\ndescription: Cog architecture and cross-cutting review guidelines\n---\n\n## Cog architecture review guidelines\n\nCog packages ML models into production-ready containers. Use this skill for\nchanges that cross language boundaries or touch core architecture.\n\n### Component overview\n\n- **CLI** (Go): `cmd/cog/` and `pkg/` -- builds, runs, and deploys models\n- **Python SDK**: `python/cog/` -- predictor interface, types, HTTP/queue server\n- **Coglet** (Rust): `crates/` -- prediction server inside containers (HTTP, worker management, IPC)\n\n### Key design patterns\n\n**Wheel resolution**: The CLI discovers SDK and coglet wheels from `dist/` at\nDocker build time. Wheels are NOT embedded in the binary. Changes to build\nartifacts need to account for this.\n\n**Dockerfile generation**: `pkg/dockerfile/` generates Dockerfiles from `cog.yaml`\nconfig. Template injection and escaping matter here.\n\n**Config parsing**: `pkg/config/config.go` parses `cog.yaml`. Schema is at\n`pkg/config/data/config_schema_v1.0.json`. Changes must keep schema and Go\ncode in sync.\n\n**Two-process coglet**: Parent process (HTTP server + orchestrator) and child\nworker process (Python predictor execution) communicate via IPC. Changes to\nthe IPC protocol affect both Rust and Python code.\n\n**Compatibility matrix**: CUDA/PyTorch/TensorFlow compatibility is managed in\n`tools/compatgen/`. Framework version changes have wide blast radius.\n\n### Cross-cutting concerns\n\n- **VERSION.txt** is the single source of truth for versioning. `Cargo.toml` must match.\n- Changes to `python/cog/base_predictor.py` affect all downstream model authors.\n- Changes to `docs/` require running `mise run docs:llm` to regenerate `docs/llms.txt`.\n- CLI reference docs are auto-generated -- changes to `cmd/` or `pkg/cli/` require `mise run docs:cli`.\n- Integration tests in `integration-tests/` use Go's testscript and need a built cog binary.\n\n### What to watch for\n\n- Breaking changes to the predictor interface or cog.yaml schema\n- Docker build changes that affect layer caching or build time\n- IPC protocol changes without updating both Rust and Python sides\n- Version bumps that miss one of the places VERSION.txt needs to match\n- New ML framework versions without compatibility testing\n",".opencode/skills/python-review/SKILL.md":"---\nname: python-review\ndescription: Python code review guidelines for the Cog SDK\n---\n\n## Python review guidelines\n\nThis project uses Python for the SDK (`python/cog/`) which defines the predictor\ninterface, type system, and HTTP/queue server.\n\n### What linters already catch (skip these)\n\nruff handles pycodestyle (E), Pyflakes (F), isort (I), warnings (W), bandit (S),\nbugbear (B), and annotations (ANN). Don't flag issues these would catch.\n\n### What to look for\n\n**Type annotations**\n\n- Required on all function signatures\n- Use `typing_extensions` for backward compatibility\n- Avoid `Any` where a concrete type is possible\n- Check that type annotations actually match runtime behavior\n\n**Compatibility**\n\n- Must support Python 3.10 through 3.13\n- Watch for syntax/stdlib features only available in newer versions\n- `from __future__ import annotations` if using newer annotation syntax\n\n**Error handling**\n\n- No bare `except:` or `except Exception:` that swallows everything\n- Exceptions should have descriptive messages\n- Resource cleanup with context managers, not try/finally when avoidable\n\n**Async patterns**\n\n- Tests use pytest-asyncio -- async tests need proper fixtures\n- Watch for blocking calls inside async functions\n- Proper cleanup of async resources (aclose, async context managers)\n\n**Predictor interface**\n\n- `base_predictor.py` is the core interface -- changes here affect all users\n- `types.py` defines input/output types -- check backward compatibility\n- Server code in `python/cog/server/` handles HTTP -- watch for request handling bugs\n\n**Testing**\n\n- Uses pytest with fixtures\n- tox runs tests across Python 3.10-3.13\n- Test isolation: don't rely on global state or test ordering\n",".opencode/skills/rust-review/SKILL.md":"---\nname: rust-review\ndescription: Rust code review guidelines for Coglet\n---\n\n## Rust review guidelines\n\nThis project uses Rust for Coglet (`crates/`), the prediction server that runs\ninside Cog containers. It handles HTTP requests, worker process management, and\nprediction execution.\n\n### What linters already catch (skip these)\n\nclippy runs in CI. cargo-deny audits dependencies for license/advisory issues.\nDon't flag issues these would catch.\n\n### What to look for\n\n**Error handling**\n\n- Use `thiserror` for typed errors in library code, `anyhow` for application errors\n- Don't use `.unwrap()` or `.expect()` in non-test code unless the invariant is documented\n- Error context: use `.context()` or `.with_context()` from anyhow, not bare `?`\n\n**Ownership and lifetimes**\n\n- Unnecessary cloning where a borrow would work\n- Lifetime issues that suggest a design problem (not just annotation noise)\n- Arc/Mutex when simpler patterns exist\n\n**Async (tokio)**\n\n- Blocking operations inside async contexts (use `spawn_blocking`)\n- Missing `.await` on futures (compiler catches some, but not all logical issues)\n- Proper cancellation handling and cleanup\n- Task/JoinHandle leaks\n\n**Safety**\n\n- Any `unsafe` block needs justification and a safety comment\n- FFI boundaries with Python (PyO3) -- check for panics across FFI, GIL handling\n- Memory safety in IPC between parent and worker processes\n\n**Architecture**\n\n- `crates/coglet/` is the core: HTTP server, worker orchestration, IPC\n- `crates/coglet-python/` is PyO3 bindings for Python predictor integration\n- Two-process architecture: parent (HTTP + orchestrator) and worker (Python execution)\n- Don't mix IPC concerns with HTTP handling\n- Snapshot tests use `insta` -- check if snapshots need updating\n\n**Dependencies**\n\n- New dependencies should be justified -- `crates/deny.toml` audits them\n- Prefer std or existing deps over adding new ones for small tasks\n",".opencode/skills/go-review/SKILL.md":"---\nname: go-review\ndescription: Go code review guidelines for the Cog codebase\n---\n\n## Go review guidelines\n\nThis project uses Go for the CLI (`cmd/cog/`, `pkg/`) and support tooling (`tools/`).\n\n### What linters already catch (skip these)\n\ngolangci-lint runs errcheck, gocritic, gosec, govet, ineffassign, misspell,\nrevive, staticcheck, and unused. Don't flag issues these would catch.\n\n### What to look for\n\n**Error handling**\n\n- Errors returned but not checked or silently discarded\n- User-facing errors should use `pkg/errors.CodedError` with error codes\n- Generic error wrapping that loses context (`fmt.Errorf(\"failed\")` with no `%w`)\n\n**Imports**\n\n- Should be three groups: stdlib, third-party, internal (`github.com/replicate/cog/pkg/...`)\n- Only flag if actually wrong, not cosmetic reordering\n\n**Testing**\n\n- Must use `testify/require` for fatal assertions and `testify/assert` for non-fatal\n- No raw `if` checks with `t.Fatal`/`t.Errorf`\n- Prefer table-driven tests for similar cases\n- Prefer specific assertions (`Equal`, `Contains`, `NoError`) over `True`/`False`\n\n**Concurrency**\n\n- Goroutine leaks (no cleanup path, missing context cancellation)\n- Shared state without synchronization\n- Channel misuse (sends on closed channels, unbuffered channels in wrong contexts)\n\n**Docker/container patterns**\n\n- The CLI uses the Docker Go SDK. Watch for leaked clients, unclosed response bodies\n- Dockerfile generation is in `pkg/dockerfile/` -- template injection risks\n\n**Architecture**\n\n- Commands belong in `pkg/cli/`, business logic in `pkg/`\n- Config parsing/validation in `pkg/config/`\n- Don't mix CLI concerns with library logic\n",".agents/skills/updating-architecture-docs/SKILL.md":"---\nname: updating-architecture-docs\ndescription: \"Use this skill when updating, reviewing, or creating architecture documentation in the architecture/ directory. This includes after refactors, feature additions, component changes, or when auditing docs for accuracy. Use it any time code changes affect how Cog's internals work -- new packages, changed IPC protocols, modified build pipeline, runtime behavior changes. Also use it proactively when reviewing PRs that touch core systems to check whether the architecture docs need updating.\"\n---\n\n# Updating Architecture Docs\n\n## Purpose\n\nThe `architecture/` directory is a bridge. It takes someone who already knows what Cog does and how to use it (that's what `docs/` is for) to the point where they can navigate the source code with confidence. It answers \"how does this work and why is it shaped this way\" -- the 10,000 foot view of the project and its ecosystem.\n\nThe goal is to move a reader from fuzzy concepts to clarity:\n\n- What are the components and what do they own?\n- What's the vocabulary? (What does \"slot\" mean? What's the \"envelope\"?)\n- Where are the boundaries between systems?\n- What are the key design decisions and why were they made?\n- Where do I look in the source to go deeper?\n\nThe architecture docs are **not** a substitute for reading code or a prose summary of the implementation. They give the reader enough context to make the code legible on first contact. Once someone understands that coglet uses a two-process model with slot-based IPC, reading `orchestrator.rs` makes sense. Without that context, it's just code.\n\n## Document structure\n\nThe docs are numbered to suggest a reading order. `00-overview.md` orients the reader and points to everything else. Each subsequent doc builds on concepts from the ones before it -- someone reading about the container runtime should already understand the predictor class and the HTTP API.\n\nThe current docs and their structure aren't fixed. New docs should be added when a topic becomes important enough to deserve its own section (testing philosophy, deployment model, etc.). Docs can be split, merged, or reorganized as the system evolves. The numbering is for reading order, not a permanent taxonomy. Check `architecture/` for the current set.\n\n## Principles\n\n### Bridge, don't summarize\n\nThe architecture docs should give a reader the mental model they need to read the code -- not replace the code. If a section starts reading like a prose restatement of what a module does line by line, it's gone too far. Pull back to the concepts and boundaries.\n\nGood: \"The orchestrator spawns a single worker subprocess and manages its lifecycle. Communication happens over two channels: a control channel for lifecycle events (stdin/stdout, JSON lines) and per-slot Unix sockets for prediction data.\"\n\nBad: \"The `spawn_worker` function in `orchestrator.rs` calls `Command::new('python')` with args `['-c', 'import coglet; coglet.server._run_worker()']` and sets up stdin/stdout pipes. It then reads from the control channel in a loop, matching on `ControlResponse` variants...\"\n\nThe first gives you the mental model. The second just restates the code. A reader with the first description can read `orchestrator.rs` and follow along. A reader with the second didn't need to read the doc at all.\n\n### Point at packages, not files\n\nReference source locations at the **package/directory level** with a description of what that package owns. Specific file paths and line numbers rot as code moves around. A pointer like \"`crates/coglet/src/bridge/` -- IPC protocol and transport\" stays accurate through refactors. \"`bridge/protocol.rs:69` -- ControlRequest enum\" doesn't.\n\nOnly document packages that matter for understanding the system's shape. Generic utility packages (`pkg/util/`, `pkg/path/`, etc.) don't need a mention -- their existence is obvious and they don't help a reader build a mental model. If someone needs them, they'll find them.\n\nWhen a specific file reference is genuinely useful (a key entry point, a non-obvious starting point for understanding a subsystem), include it -- but prefer \"the `PredictionService` in `service.rs`\" over a line number.\n\n### Document boundaries, not internals\n\nFocus on interfaces between components: what messages cross the IPC channel, what the HTTP API contract is, what labels a built image carries, what env vars control behavior. These are the things a reader needs to understand to reason about the system. Implementation details behind those boundaries belong in code comments and crate-level READMEs, not in architecture docs.\n\nThe practical test: would an internal refactor (changing how something works without changing its interface) require updating this doc? If yes, the doc is too detailed.\n\n### Explain the why\n\nDesign decisions are the most valuable content in architecture docs. They're the one thing you can't get from reading the code. \"Why two processes?\" has an answer (isolation, CUDA contexts, crash resilience) that makes the whole architecture make sense. Without it, a reader sees the complexity of IPC and wonders whether it's accidental.\n\nEvery major structural choice should have a short rationale. One or two sentences is enough.\n\n## Current state of the world\n\nThese facts should be reflected consistently across all docs. If the code changes and these become stale, the docs need updating.\n\n**One runtime:** Coglet (Rust/Axum + PyO3) is the sole runtime. There's no legacy Python/FastAPI runtime, no toggle, no \"experimental\" qualifier. Don't frame coglet as an alternative to something else.\n\n**Pydantic is not core:** Schema generation parses Python source in Go with tree-sitter and emits OpenAPI directly. `cog.BaseModel` is a dataclass wrapper. Pydantic BaseModel is supported in user code for compatibility but isn't part of Cog's own type system.\n\n**Static schema generation is the only path:** The tree-sitter-based static schema generator (Go side, `pkg/schema/`) runs on schema-generating `cog build` invocations. Parser failures stop the build before Docker starts; there is no Docker/Python runtime schema fallback or `COG_LEGACY_SCHEMA` toggle.\n\n**Wheels aren't embedded:** SDK and coglet wheels are resolved at Docker build time from PyPI, env vars, or local `dist/` directory. They're not compiled into the Go binary.\n\n**Three codebases:**\n\n- `cmd/cog/` + `pkg/` -- Go CLI and build tooling\n- `python/cog/` -- Python SDK (type definitions, predictor base class, thin server launcher)\n- `crates/coglet/` + `crates/coglet-python/` -- Rust prediction server with PyO3 bindings\n\n## How to update\n\n### When to update\n\nAfter code changes, ask: **did a boundary or interface change?**\n\nNeeds doc updates:\n\n- New or removed IPC message types\n- New or changed HTTP endpoints\n- New CLI commands\n- Changes to the build pipeline (new build steps, changed Dockerfile structure)\n- New top-level packages in `pkg/` or new crates in `crates/`\n- Changes to how components communicate\n- New design decisions or changed rationale for existing ones\n\nDoesn't need doc updates:\n\n- Bug fixes within a component\n- Internal refactors that don't change interfaces\n- Performance improvements\n- Test changes\n\n### Auditing for accuracy\n\nWhen checking docs against the codebase:\n\n1. **Read each doc** and identify claims about the system -- component names, package locations, protocol messages, env vars, build steps, CLI commands\n2. **Verify against source** -- check that things exist and work as described\n3. **Classify issues**:\n - **Structural** -- describes something that doesn't exist or works fundamentally differently (fix immediately)\n - **Misleading** -- technically true but gives the wrong mental model (fix immediately)\n - **Missing** -- a feature or component exists with no doc coverage (add if it's a boundary/interface concern)\n - **Stale reference** -- file/package renamed but concept is right (fix opportunistically)\n4. Fix structural and misleading issues first -- they actively harm readers\n\n### Writing conventions\n\nRead the existing architecture docs before writing. Match their tone and level of detail. New content should feel like it belongs -- if it reads like a different author wrote it, adjust.\n\nThe docs are technical writing. Clear, precise, no filler. Don't inflate -- avoid the kind of padded, formal language that agents default to (\"it should be noted that\", \"robust\", \"comprehensive\", \"this ensures that\"). Just say the thing.\n\nDiagrams use Mermaid or ASCII art. Both are fine. Use whichever communicates the structure more clearly. Keep each diagram focused on one concept.\n","AGENTS.md":"# AGENTS.md\n\nThis file provides guidance to coding agents when working with code in this repository.\n\n## Project Overview\n\nCog is a tool that packages machine learning models in production-ready containers.\n\nIt consists of:\n\n- **Cog CLI** (`cmd/cog/`) - Command-line interface for building, running, and deploying models, written in Go\n- **Python SDK** (`python/cog/`) - Python library for defining model predictors and training in Python\n- **Coglet** (`crates/`) - Rust-based prediction server that runs inside containers, with Python bindings via PyO3\n\nDocumentation for the CLI and SDK is available by reading ./docs/llms.txt.\n\n## Development Commands\n\nDevelopment tasks are managed with [mise](https://mise.jdx.dev/). Run `mise tasks` to see all available tasks.\n\n### Quick Reference\n\n| Task | Description |\n| ----------------------------- | ------------------------------------------------------------ |\n| `mise run fmt` | Check formatting (all languages) |\n| `mise run fmt:fix` | Fix formatting (all languages) |\n| `mise run lint` | Run linters (all languages) |\n| `mise run lint:fix` | Fix lint issues (all languages) |\n| `mise run test:go` | Run Go tests |\n| `mise run test:rust` | Run Rust tests |\n| `mise run test:python` | Run Python tests |\n| `mise run test:integration` | Run integration tests |\n| `mise run build:cog` | Build cog CLI binary |\n| `mise run build:coglet` | Build coglet wheel (dev) |\n| `mise run build:sdk` | Build SDK wheel |\n| `mise run install` | Build and symlink cog to /usr/local/bin |\n| `mise run docs:llm` | **IMPORTANT:** Regenerate `docs/llms.txt` after editing docs |\n| `mise run docs:cli` | Generate CLI reference docs from Go source code |\n| `mise run version` | Show current version from VERSION.txt |\n| `mise run version:bump ` | Bump version everywhere and commit |\n\n### Task Naming Convention\n\nTasks follow a consistent naming pattern:\n\n- **Language-based tasks** for fmt/lint/test/typecheck: `task:go`, `task:rust`, `task:python`\n- **Component-based tasks** for build: `build:cog`, `build:coglet`, `build:sdk`\n- **Check vs Fix**: `fmt` and `lint` default to check mode (non-destructive); use `:fix` suffix to auto-fix\n\n### All Tasks by Category\n\n**Format:**\n\n- `mise run fmt` / `mise run fmt:check` - Check all (alias)\n- `mise run fmt:fix` - Fix all\n- `mise run fmt:go` / `mise run fmt:rust` / `mise run fmt:python` - Per-language\n\n**Lint:**\n\n- `mise run lint` / `mise run lint:check` - Check all (alias)\n- `mise run lint:fix` - Fix all\n- `mise run lint:go` / `mise run lint:rust` / `mise run lint:python` - Per-language\n- `mise run lint:rust:deny` - Check Rust licenses/advisories\n\n**Test:**\n\n- `mise run test:go` - Go unit tests\n- `mise run test:rust` - Rust unit tests\n- `mise run test:python` - Python unit tests (via tox)\n- `mise run test:coglet:python` - Coglet Python binding tests\n- `mise run test:integration` - Integration tests\n\n**Build:**\n\n- `mise run build:cog` - Build cog CLI (development)\n- `mise run build:cog:release` - Build cog CLI (release)\n- `mise run build:coglet` - Build coglet wheel (dev install)\n- `mise run build:coglet:wheel` - Build coglet wheel (native platform)\n- `mise run build:coglet:wheel:linux-x64` - Build for Linux x86_64\n- `mise run build:coglet:wheel:linux-arm64` - Build for Linux ARM64\n- `mise run build:sdk` - Build SDK wheel\n\n**Install:**\n\n- `mise run install` - Symlink cog CLI to `/usr/local/bin` (requires `build:cog` first)\n- `PREFIX=/custom/path mise run install` - Symlink to custom location\n\n**Version:**\n\n- `mise run version` - Show current version from VERSION.txt\n- `mise run version:bump ` - Bump version everywhere and commit\n- `mise run version:check` - Verify VERSION.txt matches Cargo.toml\n\n**Other:**\n\n- `mise run typecheck` - Type check all languages\n- `mise run generate` - Run code generation\n- `mise run clean` - Clean all build artifacts\n- `mise run docs` - Build documentation\n- `mise run docs:serve` - Serve docs locally\n\n## Code Style Guidelines\n\n### Go\n\n- **Imports**: Organize in three groups separated by blank lines: (1) Standard library, (2) Third-party packages, (3) Internal packages (`github.com/replicate/cog/pkg/...`)\n- **Formatting**: Use `mise run fmt:go:fix`\n- **Linting**: Must pass golangci-lint with: errcheck, gocritic, gosec, govet, ineffassign, misspell, revive, staticcheck, unused\n- **Error Handling**: Return errors as values; use `pkg/errors.CodedError` for user-facing errors with error codes\n- **Naming**: CamelCase for exported, camelCase for unexported\n- **Testing**: Use `testify/require` for assertions; prefer table-driven tests\n\nExample import block:\n\n```go\nimport (\n \"fmt\"\n\n \"github.com/spf13/cobra\"\n\n \"github.com/replicate/cog/pkg/config\"\n)\n```\n\n### Python\n\n- **Imports**: Automatically organized by ruff/isort (stdlib → third-party → local)\n- **Formatting**: Use `mise run fmt:python:fix`\n- **Linting**: Must pass ruff checks: E (pycodestyle), F (Pyflakes), I (isort), W (warnings), S (bandit), B (bugbear), ANN (annotations)\n- **Type Annotations**: Required on all function signatures; use `typing_extensions` for compatibility; avoid `Any` where possible\n- **Error Handling**: Raise exceptions with descriptive messages; avoid generic exception catching\n- **Naming**: snake_case for functions/variables/modules, PascalCase for classes\n- **Testing**: Use pytest with fixtures; async tests with pytest-asyncio\n- **Compatibility**: Must support Python 3.10-3.13\n\n### Rust\n\n- **Formatting**: Use `mise run fmt:rust:fix`\n- **Linting**: Must pass `mise run lint:rust` (clippy)\n- **Dependencies**: Audited with `cargo-deny` (see `crates/deny.toml`); run `mise run lint:rust:deny`\n- **Error Handling**: Use `thiserror` for typed errors, `anyhow` for application errors\n- **Naming**: snake_case for functions/variables, PascalCase for types\n- **Testing**: Use `cargo test`; snapshot tests use `insta`\n- **Async**: tokio runtime; async/await patterns\n\n## Working on the CLI and support tooling\n\nThe CLI code is in the `cmd/cog/` and `pkg/` directories. Support tooling is in the `tools/` directory.\n\nThe main commands for working on the CLI are:\n\n- `go run ./cmd/cog` - Runs the Cog CLI directly from source (requires wheel to be built first)\n- `mise run build:cog` - Builds the Cog CLI binary\n- `mise run install` - Symlinks the built binary to `/usr/local/bin` (run `build:cog` first), or to a custom path with `PREFIX=/custom/path mise run install`\n- `mise run test:go` - Runs all Go unit tests\n- `go test ./pkg/...` - Runs tests directly with `go test`\n\n## Working on the Python SDK\n\nThe Python SDK is developed in the `python/cog/` directory. It uses `uv` for virtual environments and `tox` for testing across multiple Python versions.\n\nThe main commands for working on the SDK are:\n\n- `mise run build:sdk` - Builds the Python wheel\n- `mise run test:python` - Runs Python tests across all supported versions\n\n## Working on Coglet (Rust)\n\nCoglet is the Rust-based prediction server that runs inside Cog containers, handling HTTP requests, worker process management, and prediction execution.\n\nThe code is in the `crates/` directory:\n\n- `crates/coglet/` - Core Rust library (HTTP server, worker orchestration, IPC)\n- `crates/coglet-python/` - PyO3 bindings for Python predictor integration (requires Python 3.10+)\n\nFor detailed architecture documentation, see `crates/README.md` and `crates/coglet/README.md`.\n\nThe main commands for working on Coglet are:\n\n- `mise run build:coglet` - Build and install coglet wheel for development (macOS, for local Rust/Python tests)\n- `mise run build:coglet:wheel:linux-x64` - Build Linux x86_64 wheel (required to test Rust changes in Docker containers via `cog predict`/`cog train`)\n- `mise run test:rust` - Run Rust unit tests\n- `mise run lint:rust` - Run clippy linter\n- `mise run fmt:rust:fix` - Format code\n\n### Testing\n\nGo code is tested using the built-in `go test` framework:\n\n- `go test ./pkg/... -run ` - Runs specific Go tests by name\n- `mise run test:go` - Runs all Go unit tests\n\nPython code is tested using `tox`, which allows testing across multiple Python versions and configurations:\n\n- `mise run test:python` - Runs all Python unit tests\n- `uv run tox -e py312-tests -- python/tests/server/test_http.py::test_openapi_specification_with_yield` - Runs a specific Python test\n\nThe integration test suite in `integration-tests/` tests the end-to-end functionality of the Cog CLI and Python SDK using Go's testscript framework:\n\n- `mise run test:integration` - Runs the integration tests\n- `mise run test:integration string_predictor` - Runs a specific integration test\n\nThe integration tests require a built Cog binary, which defaults to the first `cog` in `PATH`. Run tests against a specific binary with the `COG_BINARY` environment variable:\n\n```bash\nmise run build:cog\nCOG_BINARY=dist/go/*/cog mise run test:integration\n```\n\n### Development Workflow\n\n1. Run `mise install` to set up the development environment\n2. Run `mise run build:sdk` after making changes to the `./python` directory\n3. Run `mise run build:coglet:wheel:linux-x64` after making changes to the `./crates` directory (needed for Docker testing)\n4. Run `mise run build:cog` to build the CLI (wheels are picked up from `dist/` at Docker build time, not embedded in the binary)\n5. Run `mise run fmt:fix` to format code\n6. Run `mise run lint` to check code quality\n7. Run `mise run docs:llm` to regenerate `docs/llms.txt` after changing `README.md` or any `docs/*.md` file\n8. Read the `./docs` directory and make sure the documentation is up to date\n\n**IMPORTANT:** Always run `mise run lint` (or the language-specific variant, e.g. `mise run lint:go`) before committing to catch linter errors early. CI will reject PRs that fail lint checks.\n\n## Architecture\n\n### CLI Architecture (Go)\n\nThe CLI follows a command pattern with subcommands. The main components are:\n\n- `pkg/cli/` - Command definitions (build, exec, predict, serve, etc.)\n- `pkg/docker/` - Docker client and container management\n- `pkg/dockerfile/` - Dockerfile generation and templating\n- `pkg/config/` - cog.yaml parsing and validation\n- `pkg/image/` - Image building and pushing logic\n\n### Python SDK Architecture\n\n- `python/cog/` - Core SDK\n - `base_predictor.py` - Base class for model predictors\n - `types.py` - Input/output type definitions\n - `server/` - HTTP/queue server implementation\n - `command/` - Runner implementations for predict/train\n\n### Coglet Architecture (Rust)\n\nThe prediction server that runs inside Cog containers. Uses a two-process architecture: a parent process (HTTP server + orchestrator) and a worker subprocess (Python predictor execution).\n\nSee `crates/README.md` for detailed architecture documentation.\n\n- `crates/coglet/` - Core Rust library (HTTP server, worker orchestration, IPC bridge)\n- `crates/coglet-python/` - PyO3 bindings for Python predictor integration\n\n### Key Design Patterns\n\n1. **Local Wheel Resolution**: The CLI discovers SDK and coglet wheels from `dist/` at Docker build time (not embedded in the binary)\n2. **Docker SDK Integration**: Uses Docker Go SDK for container operations\n3. **Type Safety**: Dataclasses for Python type validation, strongly typed Go interfaces\n4. **Compatibility Matrix**: Automated CUDA/PyTorch/TensorFlow compatibility management\n\nFor comprehensive architecture documentation, see [`architecture/`](./architecture/00-overview.md).\n\n## Common Tasks\n\n### Adding a new CLI command\n\n1. Create command file in `pkg/cli/`\n2. Add command to `pkg/cli/root.go`\n3. Implement business logic in appropriate `pkg/` subdirectory\n4. Add tests\n\n### Modifying Python SDK behavior\n\n1. Edit files in `python/cog/`\n2. Run `mise run build:sdk` to rebuild wheel\n3. Test with `mise run test:python`\n4. Integration test with `mise run test:integration`\n\n### Updating ML framework compatibility\n\n1. See `tools/compatgen/` for compatibility matrix generation\n2. Update framework versions in relevant Dockerfile templates\n3. Test with various framework combinations\n\n### Updating the docs\n\n- Documentation is in the `docs/` directory, written in Markdown and generated into HTML using `mkdocs`.\n- **IMPORTANT:** After editing any file in `docs/` or `README.md`, you MUST run `mise run docs:llm` to regenerate `docs/llms.txt`. This file is used by coding agents and should be kept in sync with the documentation.\n- **IMPORTANT:** CLI reference docs (`docs/cli.md`) are auto-generated from Go source code. After modifying CLI commands in `cmd/` or `pkg/cli/`, run `mise run docs:cli` to regenerate, and ensure `mise run docs:cli:check` passes before committing.\n\n## CI Tool Dependencies\n\nDevelopment tools are managed in **one place**: **`mise.toml`**. CI workflows use\n`jdx/mise-action@v4` (with caching enabled) to install the same tool versions that\ndevelopers use locally. This eliminates version drift between local dev and CI.\n\n**Exceptions:**\n\n| Tool | CI method | Why |\n| -------------------------- | ---------------------- | ---------------------------------------------------------------------- |\n| coglet wheel (maturin+zig) | `PyO3/maturin-action` | Builds inside a manylinux container with bundled maturin and zig |\n| Rust target cache | `Swatinem/rust-cache` | Caches `crates/target/` directory (separate from tool caching) |\n| `check-stubs` Python | `actions/setup-python` | Needs shared-library Python (libpython3.x.so) for PyO3 auto-initialize |\n\n**When updating a tool version**, update `mise.toml` only. CI picks it up automatically.\n\n## Agent Skills\n\nThe `.agents/skills/` directory contains shared skill definitions that provide specialized instructions and workflows for coding agents. Each skill is a subdirectory with a `SKILL.md` file that agents load on demand when the task matches.\n\nSkills are invoked automatically by agents when a matching task is detected. See individual `SKILL.md` files for details on when and how each skill applies.\n\n## Important Files\n\n- `VERSION.txt` - Canonical version (single source of truth)\n- `cog.yaml` - User-facing model configuration\n- `pkg/config/config.go` - Go code for parsing and validating `cog.yaml`\n- `pkg/config/data/config_schema_v1.0.json` - JSON schema for `cog.yaml`\n- `python/cog/base_predictor.py` - Predictor interface\n- `crates/Cargo.toml` - Rust workspace configuration (version must match VERSION.txt)\n- `crates/README.md` - Coglet architecture overview\n- `mise.toml` - Task definitions for development workflow\n\n## Testing Philosophy\n\n- Unit tests for individual components (Go and Python)\n- Integration tests for end-to-end workflows\n- Tests use real Docker operations (no mocking Docker API)\n- Always run `mise run build:sdk` after making Python changes before testing Go code\n- Python 3.10-3.13 compatibility is required\n\n### Go Test Conventions\n\nAll Go tests must use [testify](https://github.com/stretchr/testify) for assertions. Do **not** use raw `if` checks with `t.Fatal`/`t.Errorf` — use `require` and `assert` instead.\n\n- **`require`** — for fatal assertions that should stop the test (setup failures, preconditions):\n ```go\n require.NoError(t, err, \"failed to create client\")\n require.Equal(t, expected, actual)\n require.True(t, condition, \"server should be ready\")\n ```\n- **`assert`** — for non-fatal checks where the test should continue (e.g. validating multiple fields in a loop):\n ```go\n assert.Equal(t, http.StatusOK, resp.StatusCode)\n assert.Contains(t, output, \"expected substring\")\n assert.NoError(t, err, \"prediction %d failed\", i)\n ```\n- Use `require` for errors in setup/teardown and `assert` for the actual test expectations\n- Prefer specific assertions (`Equal`, `Contains`, `NoError`, `Len`, `Less`) over generic `True`/`False` — they produce better failure messages\n- Prefer table-driven tests for testing multiple similar cases\n",".opencode/agents/reviewer.md":"---\ndescription: Read-only code reviewer for pull requests\nmode: primary\nmodel: cf-gateway/deepseek-v4-flash-0731\ntemperature: 0.1\npermission:\n edit: deny\n bash:\n \"*\": deny\n \"git diff*\": allow\n \"git log*\": allow\n \"git show*\": allow\n webfetch: deny\n---\n\nYou are a code reviewer for the Cog project. Cog packages ML models into\nproduction-ready containers. The codebase has three languages: Go (CLI and\ntooling), Python (SDK), and Rust (coglet prediction server).\n\n## How to review\n\n1. Read the full diff. Understand what the PR is trying to do before commenting.\n2. Load the appropriate review skill(s) for the languages touched in the PR:\n - Go files: load `go-review`\n - Python files: load `python-review`\n - Rust files: load `rust-review`\n - Architecture/cross-cutting: load `cog-review`\n3. Focus on substance, not style. Linters handle formatting.\n4. Scope feedback to the PR's purpose. Don't demand unrelated refactors.\n5. Be direct. Say what the problem is, why it matters, and how to fix it.\n\n## Comment format\n\n- Reference specific files and lines: `path/to/file.go:42`\n- Use ```suggestion blocks when proposing concrete code changes\n- Categorize findings:\n - **Blocker**: Must fix before merge (bugs, security, data loss)\n - **Should fix**: Important but not blocking (error handling gaps, edge cases)\n - **Nit**: Minor improvement, take it or leave it\n- If the PR looks good, say so briefly. Don't invent problems.\n","tools/test-harness/README.md":"# Cog Model Test Harness\n\nAutomated test harness for validating Cog models against new CLI/SDK versions.\n\nThis is the Go implementation and replaces the previous Python harness.\n\n## Quick Start\n\n```bash\ncd tools/test-harness\n\n# List all models in the manifest\ngo run . list\n\n# Run all non-GPU models\ngo run . run --no-gpu\n\n# Run a specific model\ngo run . run --model hello-world\n\n# Run GPU models only (requires NVIDIA GPU + nvidia-docker)\ngo run . run --gpu-only\n\n# Output JSON report\ngo run . run --no-gpu --output json --output-file results/report.json\n\n# Build images only (no predictions)\ngo run . build --no-gpu\n\n# Use a locally-built cog binary\ngo run . run --no-gpu --cog-binary ../../dist/go/darwin-arm64/cog\n\n# Test from a git branch (builds CLI + SDK wheel automatically)\ngo run . run --no-gpu --cog-ref main\n```\n\nYou can also run from repo root:\n\n```bash\n(cd tools/test-harness && go run . list)\n(cd tools/test-harness && go run . run --no-gpu)\n```\n\n## Prerequisites\n\n- Go (see repo toolchain)\n- Docker\n- For `--cog-ref`: Go + uv on PATH (`mise install` sets this up)\n- For GPU models: NVIDIA GPU + nvidia-docker runtime\n\n## Manifest and Fixtures\n\n- Manifest: `tools/test-harness/manifest.yaml`\n- Fixtures: `tools/test-harness/fixtures/`\n\nInput values prefixed with `@` resolve to fixture paths. Example:\n\n```yaml\ninputs:\n image: \"@test_image.png\"\n```\n\n## CLI Reference\n\n```text\nusage: test-harness {run,build,list} [options]\n\nCommands:\n run Build and test models (full pipeline)\n build Build Docker images only (no predictions)\n list List models defined in the manifest\n\nCommon options:\n --manifest PATH Path to manifest.yaml\n --model NAME Run only this model (repeatable)\n --no-gpu Skip GPU models\n --gpu-only Only run GPU models\n --sdk-version VER SDK version override\n --cog-version TAG CLI version to download\n --cog-binary PATH Path to local cog binary\n --cog-ref REF Git ref to build cog from source\n --sdk-wheel PATH Local SDK wheel path\n --clean-images Remove Docker images after run (default: keep them)\n --keep-outputs Preserve prediction outputs and print the work directory path\n\nRun options:\n --output {console,json} Output format\n --output-file PATH Write JSON report to file\n```\n\n## Project Layout\n\n```text\ntools/test-harness/\n├── go.mod\n├── main.go\n├── README.md\n├── manifest.yaml\n├── fixtures/\n├── cmd/\n└── internal/\n```\n","DESIGN.md":"# Design\n\n## Background\n\nCog came from Andreas's experience at Spotify and Ben's experience at Docker.\n\nAt Spotify, Andreas noticed a cluster of related problems:\n\n- **It was hard to run open-source machine learning models.** All the advances in machine learning were locked up inside prose in PDFs, scraps of code on GitHub, weights on Google Drive (if you were lucky!). If you wanted to build upon this research, or apply it to real-world problems, you had to implement it all from scratch.\n- **It was hard to deploy machine learning models to production.** Andreas was the only person on the research team who was also an infrastructure engineer. Typically a researcher would have to sit down with Andreas to decide on an API, get a server written, package up dependencies, battle CUDA, get it running efficiently, get it deployed on the cluster, and so on and so forth. It would take weeks to get something running in production.\n\nBen connected this back to his experience at Docker. What Docker did was define a standard box that software could go in. You could put any kind of server software in there – Python, Java, Ruby on Rails, whatever – and you could then know that you could run it on your local machine or on any cloud, as long as it supported Docker. We wanted to do the same thing for machine learning.\n\n## Vision\n\nWe want Cog to be a standard artifact for what a model is and how that model is run.\n\n(More detail...)\n\n## Design principles\n\nThere are a few things driving Cog's design:\n\n- **Reproducible artifact**: When you've put your model in Cog, it'll run anywhere, and _keep_ on running. This is why it's a Docker image, with all of the model's dependencies. Docker images have a content-addressable SHA256 ID, which is the identifier for that model's behavior, byte-for-byte.\n- **Weights inside the image**: We encourage users to put model weights in images. If the weights are on cloud storage somewhere, then they might change or disappear, and the image will produce different results. There's nothing magical about Docker images – they're just a bundle of files. Docker moves around that bundle of files quite slowly, though, but we can optimize that process so it's as fast as reading weights directly from blob storage, or wherever.\n- **Models are just functions**: Models can be lots of things. We are of the opinion that [machine learning is just software](https://replicate.com/blog/machine-learning-needs-better-tools), and a model is just a function. It often needs to be attached to a GPU, but apart from that it's just a normal function that has some input and some output. This is the core difference between Docker's abstraction and Cog's abstraction: Docker packages up an executable, whereas Cog packages up a _function_.\n- **Standard interface**: When you run the Docker container, it serves an HTTP server, that is a standard API for running that function. You can think of it like a remote procedure call.\n- **Self-describing artifact**: A Cog model has it's schema (or type signature, if you're thinking of it as a function) attached to the image as a label. This means systems that work with Cog models can know what the model is and what requests to send to it. This is what powers the forms on Replicate, for example.\n- **Not just the model**: Before Cog, the typical standard packaging formats for machine learning models were at the network level. A way of taking a Tensorflow or PyTorch network and packaging up in a way that would run on lots of different types of accelerators. Things like [ONNX](https://onnx.ai/) or [TVM's IR](https://tvm.apache.org/). We realized that \"models\" are not just the network, but they are also pre- and post-processing, and are so diverse and the field is so fast-moving that you can't possible squeeze it into some high-level abstraction. It just needs to be code running on a computer.\n- **It's just Docker**: Cog models need to run anywhere, and they'll only run anywhere if it's vanilla Docker. We might optimize how Docker images get shipped around to make it faster, but we're not going to invent our own image format.\n- **The API is for software developers**: In the olden days, you have to pass tensors to TFServing and know how to generate a tensor from a JPEG. Cog's API intentionally just speaks JSON, strings, files, etc. It's intended to be the interface between the software developer and the ML engineer. Sort of like Docker was intended to be the interface between the software developer and the infrastructure engineer.\n- **Cog is the APIs and interfaces, not just the software**: The most important thing about Cog is that it defines a standard for what a model is and how to run it. It doesn't necessarily need to involve Cog the piece of software itself. For example, Replicate could serve a model from OpenAI with a Cog API and schema, but it's not packaged or running with Cog under the hood at all – it's just calling the OpenAI API directly.\n","tools/test-harness/manifest.yaml":"# Cog Model Test Manifest\n# =======================\n# Each entry defines a model to test, its inputs, and expected outputs.\n#\n# Input values prefixed with \"@\" are resolved as fixture file paths relative\n# to the fixtures/ directory (e.g. \"@test_image.png\" -> fixtures/test_image.png).\n#\n# Validation types:\n# exact - output string must equal `value` exactly\n# contains - output string must contain `value` as a substring\n# regex - output string must match `pattern`\n# file_exists - output is a file path; optionally check `mime` type\n# json_match - parse output as JSON, assert `match` is a subset\n# json_keys - parse output as JSON dict, assert it has entries\n# not_empty - output is non-empty (loose smoke test)\n#\n# Setup commands:\n# Optional `setup` list of shell commands to run after cloning the repo\n# but before building. Useful for models that generate cog.yaml from\n# templates or need other preparation steps.\n# Example:\n# setup:\n# - \"script/select.sh dev\"\n#\n# Required tools:\n# Optional `requires_tools` list of CLI tools that must be on PATH for\n# setup commands to work. The harness checks before running and prints\n# install instructions for known tools (yq, envsubst).\n# Example:\n# requires_tools: [\"yq\"]\n#\n# Model resolution (base_dir takes priority over repo):\n# - Set `base_dir` for local models -- `path` is resolved under it. A\n# relative `base_dir` is resolved against the repo root; an absolute path\n# is used as-is. Local only; never clones.\n# - Set `repo: owner/name` (with no `base_dir`) to clone from GitHub.\n# - Set neither to default to fixtures/models/ (local fixtures).\n# `repo: local` is no longer supported -- use `base_dir` instead.\n\ndefaults:\n sdk_version: \"latest\" # \"latest\" = newest stable from PyPI; or pin e.g. \"0.16.12\"\n cog_version: \"latest\" # \"latest\" = newest stable release; or pin e.g. \"v0.16.12\"\n\nmodels:\n # ── cog-examples (CPU) ──────────────────────────────────────────────\n\n - name: hello-world\n base_dir: examples\n path: hello-world\n gpu: false\n tests:\n - description: \"basic predict\"\n inputs:\n text: \"world\"\n expect:\n type: exact\n value: \"hello world\"\n\n - name: canary\n base_dir: examples\n path: canary\n gpu: false\n tests:\n - description: \"streaming concatenate iterator\"\n inputs:\n text: \"friend\"\n expect:\n type: contains\n value: \"friend\"\n\n - name: blur\n base_dir: examples\n path: blur\n gpu: false\n tests:\n - description: \"blur an image\"\n inputs:\n image: \"@test_image.png\"\n blur: 5\n expect:\n type: file_exists\n mime: \"image/png\"\n\n - name: hello-image\n base_dir: examples\n path: hello-image\n gpu: false\n tests:\n - description: \"return a static image\"\n inputs: {}\n expect:\n type: file_exists\n\n - name: hello-concurrency\n base_dir: examples\n path: hello-concurrency\n gpu: false\n tests:\n - description: \"async streaming output\"\n inputs:\n total: 3\n interval: 0\n expect:\n type: contains\n value: \"Apple\"\n\n - name: hello-context\n base_dir: examples\n path: hello-context\n gpu: false\n # NOTE: This model uses current_scope().context which may not be\n # available in coglet yet. A failure here is a real compatibility signal.\n tests:\n - description: \"returns input and context\"\n inputs:\n text: \"testing\"\n expect:\n type: json_match\n match:\n inputs:\n text: \"testing\"\n\n - name: hello-train\n base_dir: examples\n path: hello-train\n gpu: false\n train_tests:\n - description: \"train produces weights file\"\n inputs:\n prefix: \"custom\"\n expect:\n type: not_empty\n tests:\n - description: \"predict with default weights\"\n inputs:\n text: \"world\"\n expect:\n type: contains\n value: \"world\"\n\n - name: notebook\n base_dir: examples\n path: notebook\n gpu: false\n tests:\n - description: \"basic predict\"\n inputs:\n name: \"world\"\n expect:\n type: contains\n value: \"hello\"\n\n - name: managed-weights\n base_dir: examples\n path: managed-weights\n gpu: false\n requires_env:\n - COG_MANAGED_WEIGHTS_READY\n tests:\n - description: \"weight validation smoke test\"\n inputs: {}\n expect:\n type: not_empty\n\n - name: streaming-text\n base_dir: examples\n path: streaming-text\n gpu: false\n tests:\n - description: \"streaming text generation\"\n inputs:\n prompt: \"Hello\"\n expect:\n type: not_empty\n\n # ── cog-examples (GPU) ──────────────────────────────────────────────\n\n - name: resnet\n base_dir: examples\n path: resnet\n gpu: true\n tests:\n - description: \"classify hotdog image\"\n inputs:\n image: \"@hotdog.png\"\n expect:\n type: json_keys\n\n - name: z-image-turbo\n base_dir: examples\n path: z-image-turbo\n gpu: true\n timeout: 600\n tests:\n - description: \"generate image from prompt\"\n inputs:\n prompt: \"a cat sitting on a windowsill\"\n expect:\n type: file_exists\n mime: \"image/png\"\n\n # ── Local fixture models ─────────────────────────────────────────────\n # These set base_dir to the fixtures dir (relative to the repo root), so\n # path is relative to it. They cover the full input type matrix for build\n # and prediction testing.\n\n - name: fixture-scalar-types\n base_dir: tools/test-harness/fixtures/models\n path: scalar-types\n gpu: false\n tests:\n - description: \"basic scalar inputs\"\n inputs:\n text: \"hello\"\n expect:\n type: contains\n value: \"hello\"\n\n - name: fixture-optional-types\n base_dir: tools/test-harness/fixtures/models\n path: optional-types\n gpu: false\n tests:\n - description: \"optional inputs default to None\"\n inputs:\n text: \"hello\"\n expect:\n type: exact\n value: \"hello\"\n\n - name: fixture-list-types\n base_dir: tools/test-harness/fixtures/models\n path: list-types\n gpu: false\n\n - name: fixture-optional-list-types\n base_dir: tools/test-harness/fixtures/models\n path: optional-list-types\n gpu: false\n tests:\n - description: \"optional lists default to None\"\n inputs:\n text: \"hello\"\n expect:\n type: exact\n value: \"hello\"\n\n - name: fixture-constraints-and-choices\n base_dir: tools/test-harness/fixtures/models\n path: constraints-and-choices\n gpu: false\n tests:\n - description: \"defaults with constraints and choices\"\n inputs: {}\n expect:\n type: exact\n value: \"hello-0.7-50-balanced-1\"\n\n - name: fixture-file-path-types\n base_dir: tools/test-harness/fixtures/models\n path: file-path-types\n gpu: false\n\n - name: fixture-complex-output\n base_dir: tools/test-harness/fixtures/models\n path: complex-output\n gpu: false\n tests:\n - description: \"structured BaseModel output\"\n inputs:\n prompt: \"test\"\n expect:\n type: json_match\n match:\n text: \"generated: test\"\n score: 0.95\n\n # ── External models (add your own below) ────────────────────────────\n # - name: my-custom-model\n # repo: myorg/my-model-repo\n # path: \".\"\n # gpu: true\n # # sdk_version: \"0.16.12\" # optional per-model override\n # env:\n # HF_TOKEN: \"${HF_TOKEN}\"\n # timeout: 600\n # tests:\n # - description: \"smoke test\"\n # inputs:\n # prompt: \"hello\"\n # expect:\n # type: contains\n # value: \"result\"\n #\n # Models that need setup commands before building (e.g. generating cog.yaml):\n # - name: flux-dev\n # repo: replicate/cog-flux\n # path: \".\"\n # gpu: true\n # setup:\n # - \"script/select.sh dev\"\n # timeout: 900\n # tests:\n # - description: \"generate image\"\n # inputs:\n # prompt: \"a cat\"\n # expect:\n # type: file_exists\n","docs/llms.txt":"# Cog: Containers for machine learning\n\nCog is an open-source tool that lets you package machine learning models in a standard, production-ready container.\n\nYou can deploy your packaged model to your own infrastructure, or to [Replicate](https://replicate.com/).\n\n## Highlights\n\n- 📦 **Docker containers without the pain.** Writing your own `Dockerfile` can be a bewildering process. With Cog, you define your environment with a [simple configuration file](#how-it-works) and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.\n\n- 🤬️ **No more CUDA hell.** Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.\n\n- ✅ **Define the inputs and outputs for your model with standard Python.** Then, Cog generates an OpenAPI schema and validates the inputs and outputs.\n\n- 🎁 **Automatic HTTP inference server**: Your model's types are used to dynamically generate a RESTful HTTP API using a high-performance Rust/Axum server.\n\n- 🚀 **Ready for production.** Deploy your model anywhere that Docker images run. Your own infrastructure, or [Replicate](https://replicate.com).\n\n## How it works\n\nDefine the Docker environment your model runs in with `cog.yaml`:\n\n```yaml\nbuild:\n gpu: true\n system_packages:\n - \"libgl1\"\n - \"libglib2.0-0\"\n python_version: \"3.13\"\n python_requirements: requirements.txt\nrun: \"run.py:Runner\"\n```\n\nDefine how your model runs with `run.py`:\n\n```python\nfrom cog import BaseRunner, Input, Path\nimport torch\n\nclass Runner(BaseRunner):\n def setup(self):\n \"\"\"Load the model into memory to make running multiple inferences efficient\"\"\"\n self.model = torch.load(\"./weights.pth\")\n\n # The arguments and types the model takes as input\n def run(self,\n image: Path = Input(description=\"Grayscale input image\")\n ) -> Path:\n \"\"\"Run the model\"\"\"\n processed_image = preprocess(image)\n output = self.model(processed_image)\n return postprocess(output)\n```\n\nIn the above we accept a path to the image as an input, and return a path to our transformed image after running it through our model.\n\nNow, you can run the model:\n\n```console\n$ cog run -i image=@input.jpg\n--> Building Docker image...\n--> Running...\n--> Output written to output.jpg\n```\n\nOr, build a Docker image for deployment:\n\n```console\n$ cog build -t my-classification-model\n--> Building Docker image...\n--> Built my-classification-model:latest\n\n$ docker run -d -p 5000:5000 --gpus all my-classification-model\n\n$ curl http://localhost:5000/predictions -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"image\": \"https://.../input.jpg\"}}'\n```\n\nOr, combine build and run via the `serve` command:\n\n```console\n$ cog serve -p 8080\n\n$ curl http://localhost:8080/predictions -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"image\": \"https://.../input.jpg\"}}'\n```\n\n\n\n## Why are we building this?\n\nIt's really hard for researchers to ship machine learning models to production.\n\nPart of the solution is Docker, but it is so complex to get it to work: Dockerfiles, pre-/post-processing, Flask servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.\n\n[Andreas](https://github.com/andreasjansson) and [Ben](https://github.com/bfirsh) created Cog. Andreas used to work at Spotify, where he built tools for building and deploying ML models with Docker. Ben worked at Docker, where he created [Docker Compose](https://github.com/docker/compose).\n\nWe realized that, in addition to Spotify, other companies were also using Docker to build and deploy machine learning models. [Uber](https://eng.uber.com/michelangelo-pyml/) and others have built similar systems. So, we're making an open source version so other people can do this too.\n\nHit us up if you're interested in using it or want to collaborate with us. [We're on Discord](https://discord.gg/replicate) or email us at [team@replicate.com](mailto:team@replicate.com).\n\n## Prerequisites\n\n- **macOS, Linux or Windows 11**. Cog works on macOS, Linux and Windows 11 with [WSL 2](docs/wsl2/wsl2.md)\n- **Docker**. Cog uses Docker to create a container for your model. You'll need to [install Docker](https://docs.docker.com/get-docker/) before you can run Cog. If you install Docker Engine instead of Docker Desktop, you will need to [install Buildx](https://docs.docker.com/build/architecture/#buildx) as well.\n\n## Install\n\nChoose your platform for installation instructions.\n\n
\n macOS\n\nThe easiest way to install Cog on macOS is with Homebrew:\n\n```console\nbrew install replicate/tap/cog\n```\n\nYou can also use the install script:\n\n```sh\n# bash, zsh, and other shells\nsh <(curl -fsSL https://cog.run/install.sh)\n\n# fish shell\nsh (curl -fsSL https://cog.run/install.sh | psub)\n\n# download with wget and run in a separate command\nwget -qO- https://cog.run/install.sh\nsh ./install.sh\n```\n\nOr install manually:\n\n```console\nsudo curl -o /usr/local/bin/cog -L \"https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')\"\nsudo chmod +x /usr/local/bin/cog\nsudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true\n```\n\nIf you see a Gatekeeper warning saying the binary \"cannot be opened because the developer cannot be verified\", run:\n\n```console\nsudo xattr -d com.apple.quarantine /usr/local/bin/cog\n```\n\n
\n\n
\n Linux\n\nYou can install Cog using the install script:\n\n```sh\n# bash, zsh, and other shells\nsh <(curl -fsSL https://cog.run/install.sh)\n\n# fish shell\nsh (curl -fsSL https://cog.run/install.sh | psub)\n\n# download with wget and run in a separate command\nwget -qO- https://cog.run/install.sh\nsh ./install.sh\n```\n\nOr install manually:\n\n```console\nsudo curl -o /usr/local/bin/cog -L \"https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')\"\nsudo chmod +x /usr/local/bin/cog\n```\n\n
\n\n
\n Windows\n\nCog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above.\n\n
\n\n
\n Docker\n\nTo install Cog inside a Docker image:\n\n```dockerfile\nRUN sh -c \"INSTALL_DIR=\\\"/usr/local/bin\\\" SUDO=\\\"\\\" $(curl -fsSL https://cog.run/install.sh)\"\n```\n\n
\n\n## Upgrade\n\nIf you're using macOS and you previously installed Cog with Homebrew, run the following:\n\n```console\nbrew upgrade replicate/tap/cog\n```\n\nOtherwise, you can upgrade to the latest version by running the same commands you used to install it.\n\n## Development\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for how to set up a development environment and build from source.\n\n## Next steps\n\n- [Get started with an example model](docs/getting-started.md)\n- [Get started with your own model](docs/getting-started-own-model.md)\n- [Using Cog with notebooks](docs/notebooks.md)\n- [Using Cog with Windows 11](docs/wsl2/wsl2.md)\n- [Browse the example models in this repo](docs/examples.md)\n- [Deploy models with Cog](docs/deploy.md)\n- [`cog.yaml` reference](docs/yaml.md) to learn how to define your model's environment\n- [Run interface reference](docs/python.md) to learn how the `Runner` interface works\n- [Training interface reference](docs/training.md) to learn how to add a fine-tuning API to your model\n- [HTTP API reference](docs/http.md) to learn how to use the HTTP API that models serve\n\n## Need help?\n\n[Join us in #cog on Discord.](https://discord.gg/replicate)\n\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/replicate/cog)\n\n\n\n---\n\n# Base images\n\nCog builds your model into a Docker image. To speed up builds and reduce cold boot times, Cog uses **prebuilt base images** by default. These images contain the common dependencies that most models need, so Cog doesn't have to install them from scratch every time you build or push.\n\n## What is a Cog base image?\n\nA Cog base image is a Docker image maintained by Replicate that includes:\n\n- **Python runtime** — the Python version specified in your `cog.yaml`.\n- **System libraries** — common dependencies for machine learning and media processing, including `ffmpeg`, `git`, `curl`, `build-essential`, `cmake`, OpenCV libraries, audio libraries (`sox`, `libsndfile1`), and graphics libraries (`libgl1`, `libsm6`, `libxext6`).\n- **CUDA and PyTorch** (GPU images only) — the appropriate CUDA toolkit and PyTorch stack for your configuration.\n\nBase images are tagged by their configuration, for example:\n\n```\nr8.im/cog-base:cuda12.4-python3.13-torch2.6.0\nr8.im/cog-base:python3.13\n```\n\nWhen you run `cog build` or `cog push`, Cog selects the base image that matches your Python version, CUDA version, and PyTorch version. Because these images are pre-pulled on Replicate's infrastructure, models built on top of them start faster.\n\nThe entire list of available base images can be viewed at [https://r8.im/v2/cog-base/tags/list](https://r8.im/v2/cog-base/tags/list).\n\n## Using the Cog base image\n\nThe `--use-cog-base-image` flag controls whether Cog uses a prebuilt base image. It is **enabled by default** on the following commands:\n\n- [`cog build`](cli.md#cog-build)\n- [`cog push`](cli.md#cog-push)\n- [`cog run`](cli.md#cog-run)\n- [`cog serve`](cli.md#cog-serve)\n- [`cog exec`](cli.md#cog-exec)\n\nSince it's on by default, you don't need to pass any flags:\n\n```bash\ncog push r8.im/your-username/my-model\n```\n\nThis builds and pushes your model using a prebuilt Cog base image for faster cold boots.\n\n## Disabling the Cog base image\n\nIf you encounter build issues or need a custom base layer, you can disable the Cog base image:\n\n```bash\ncog build --use-cog-base-image=false\n```\n\nWhen disabled, Cog generates a Dockerfile from scratch using either an NVIDIA CUDA base image or a slim Python base image, depending on the `--use-cuda-base-image` flag.\n\n## Relationship to `--use-cuda-base-image`\n\nThe `--use-cuda-base-image` flag controls which underlying base image Cog uses **when the Cog base image is disabled**. It has no effect when `--use-cog-base-image` is enabled (the default), because the Cog base image already includes the appropriate CUDA and Python stack.\n\nWhen you disable the Cog base image with `--use-cog-base-image=false`, Cog chooses the base image automatically:\n\n- **GPU models** (`gpu: true` in `cog.yaml`): uses an NVIDIA CUDA base image.\n- **CPU models**: uses a slim Python base image.\n\nThese flags — along with `--dockerfile` — are **mutually exclusive**: you can only set one of them explicitly on a given command. To customize the base image, disable the Cog base image and let Cog choose between CUDA and Python automatically.\n\n## Troubleshooting\n\nIf `cog build` or `cog push` fails with the Cog base image enabled, try disabling it:\n\n```bash\ncog push --use-cog-base-image=false r8.im/your-username/my-model\n```\n\nThis falls back to building from a standard CUDA or Python base image, which can help diagnose whether the issue is with the base image or your model's configuration.\n\n\n---\n\n# CLI reference\n\n\n\n## `cog`\n\nContainers for machine learning.\n\nTo get started, take a look at the documentation:\nhttps://github.com/replicate/cog\n\n**Examples**\n\n```\n To execute a command inside a Docker environment defined with Cog:\n $ cog exec echo hello world\n```\n\n**Options**\n\n```\n --debug Show debugging output\n -h, --help help for cog\n --no-color Disable colored output\n --version Show version of Cog\n```\n\n## `cog build`\n\nBuild a Docker image from the cog.yaml in the current directory.\n\nThe generated image contains your model code, dependencies, and the Cog\nruntime. It can be run locally with 'cog run' or pushed to a registry\nwith 'cog push'.\n\n```\ncog build [flags]\n```\n\n**Examples**\n\n```\n # Build with default settings\n cog build\n\n # Build and tag the image\n cog build -t my-model:latest\n\n # Build without using the cache\n cog build --no-cache\n\n # Build with model weights in a separate layer\n cog build --separate-weights -t my-model:v1\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n -h, --help help for build\n --no-cache Do not use cache when building the image\n --openapi-schema string Load OpenAPI schema from a file\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'\n --separate-weights Separate model weights from code in image layers\n -t, --tag string A name for the built image in the form 'repository:tag'\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog doctor`\n\nDiagnose and fix common issues in your Cog project.\n\nNOTE: cog doctor is experimental. Behavior and checks may change in future versions.\n\nBy default, cog doctor reports problems without modifying any files.\nPass --fix to automatically apply safe fixes.\n\n```\ncog doctor [flags]\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --fix Automatically apply fixes\n -h, --help help for doctor\n```\n\n## `cog exec`\n\nExecute a command inside a Docker environment defined by cog.yaml.\n\nCog builds a temporary image from your cog.yaml configuration and runs the\ngiven command inside it. This is useful for debugging, running scripts, or\nexploring the environment your model will run in.\n\n```\ncog exec [arg...] [flags]\n```\n\n**Examples**\n\n```\n # Open a Python interpreter inside the model environment\n cog exec python\n\n # Run a script\n cog exec python train.py\n\n # Run with environment variables\n cog exec -e HUGGING_FACE_HUB_TOKEN=abc123 python download.py\n\n # Expose a port (e.g. for Jupyter)\n cog exec -p 8888 jupyter notebook\n```\n\n**Options**\n\n```\n -e, --env stringArray Environment variables, in the form name=value\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for exec\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n -p, --publish stringArray Publish a container's port to the host, e.g. -p 8000 or -p 0.0.0.0:8000\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog init`\n\nCreate a cog.yaml and run.py in the current directory.\n\nThese files provide a starting template for defining your model's environment\nand run interface. Edit them to match your model's requirements.\n\n```\ncog init [flags]\n```\n\n**Examples**\n\n```\n # Set up a new Cog project in the current directory\n cog init\n```\n\n**Options**\n\n```\n -h, --help help for init\n```\n\n## `cog login`\n\nLog in to a container registry.\n\nFor Replicate's registry (r8.im), this command handles authentication\nthrough Replicate's token-based flow.\n\nFor other registries, this command prompts for username and password,\nthen stores credentials using Docker's credential system.\n\n```\ncog login [flags]\n```\n\n**Options**\n\n```\n -h, --help help for login\n --token-stdin Pass login token on stdin instead of opening a browser. You can find your Replicate login token at https://replicate.com/auth/token\n```\n\n## `cog playground`\n\nOpen a browser playground for talking to a running model.\n\nStarts a local web server that serves a schema-driven UI (a Postman-like tool\nfor Cog models). Point it at any running Cog HTTP API -- for example one started\nwith 'cog serve' -- and the playground reflects that model's inputs and outputs\nfrom its OpenAPI schema in real time.\n\nRequests are reverse-proxied through this server, so the target API does not\nneed to set CORS headers. The server also hosts a webhook sink so async\npredictions can be observed in the browser.\n\nAsync/webhook testing against a containerized model requires the webhook URL to\nbe reachable from inside the container. On Docker Desktop the default\n'host.docker.internal' works once the server listens on a reachable interface\n(e.g. --host 0.0.0.0).\n\n```\ncog playground [flags]\n```\n\n**Examples**\n\n```\n # Start a model API in one terminal\n cog serve -p 8393\n\n # Open the playground pointing at it\n cog playground --target http://localhost:8393\n```\n\n**Options**\n\n```\n -h, --help help for playground\n --host string Address to bind (use 0.0.0.0 to receive webhooks from containers) (default \"127.0.0.1\")\n --no-open Do not open the browser automatically\n -p, --port int Port to listen on (0 picks a free port)\n --target string Default target model API URL (default \"http://localhost:8393\")\n --webhook-host string Hostname the model uses to reach this server for webhooks (default \"host.docker.internal\")\n```\n\n## `cog push`\n\nBuild from cog.yaml and push to an OCI-compliant registry. Run 'cog login'\nfirst when pushing to Replicate's registry (r8.im).\n\nTARGET overrides the configured destination and all COG_MODEL\\* environment\nvariables. It doesn't change the project format: projects configured with\n'image' push an image, while projects configured with 'model' push an OCI\nbundle. Push targets must use tags, not digests. Untagged image targets use\nDocker's default 'latest' tag; untagged bundle targets get a timestamp tag.\n\nWith --json, Cog writes one versioned JSON result to stdout after the entire\npush succeeds. Progress, warnings, and diagnostics continue on stderr. Every\nreference in the result is digest-pinned.\n\n```\ncog push [TARGET] [flags]\n```\n\n**Examples**\n\n```\n # Push an image to Replicate\n cog push r8.im/your-username/my-model\n\n # Push an image to any OCI registry\n cog push registry.example.com/your-username/model-name\n\n # Push a bundle project and print its immutable references as JSON\n cog push registry.example.com/your-username/model-name:v1 --json\n\n # Push with model weights in a separate image layer (Replicate only)\n cog push r8.im/your-username/my-model --separate-weights\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n -h, --help help for push\n --json Output the pushed references as JSON\n --no-cache Do not use cache when building the image\n --openapi-schema string Load OpenAPI schema from a file\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'\n --separate-weights Separate model weights from code in image layers\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog run`\n\nRun the model.\n\nIf 'image' is passed, it will run the model on that Docker image.\nIt must be an image that has been built by Cog.\n\nOtherwise, it will build the model in the current directory and run\nit.\n\n```\ncog run [image] [flags]\n```\n\n**Examples**\n\n```\n # Run the model with named inputs\n cog run -i prompt=\"a photo of a cat\"\n\n # Pass a file as input\n cog run -i image=@photo.jpg\n\n # Save output to a file\n cog run -i image=@input.jpg -o output.png\n\n # Pass multiple inputs\n cog run -i prompt=\"sunset\" -i width=1024 -i height=768\n\n # Run against a pre-built image\n cog run r8.im/your-username/my-model -i prompt=\"hello\"\n\n # Pass inputs as JSON\n echo '{\"prompt\": \"a cat\"}' | cog run --json @-\n```\n\n**Options**\n\n```\n -e, --env stringArray Environment variables, in the form name=value\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for run\n -i, --input stringArray Inputs, in the form name=value. if value is prefixed with @, then it is read from a file on disk. E.g. -i path=@image.jpg\n --json string Pass inputs as JSON object, read from file (@inputs.json) or via stdin (@-)\n -o, --output string Output path\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --setup-timeout uint32 The timeout for a container to setup (in seconds). (default 300)\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n --use-replicate-token Pass REPLICATE_API_TOKEN from local environment into the model context\n```\n\n## `cog serve`\n\nRun an HTTP server.\n\nBuilds the model and starts an HTTP server that exposes the model's inputs\nand outputs as a REST API. Compatible with the Cog HTTP protocol.\n\nBy default the container port is published on 127.0.0.1 (localhost), so the\nserver is only reachable from your local machine. The server process inside\nthe container binds to 0.0.0.0; use --host to control which host interface\nthe Docker port mapping is published on.\n\n```\ncog serve [flags]\n```\n\n**Examples**\n\n```\n # Start the server on the default port (8393)\n cog serve\n\n # Start on a custom port\n cog serve -p 5000\n\n # Start the playground alongside the model\n cog serve --playground\n\n # Listen on all interfaces (e.g. to expose to the network)\n cog serve --host 0.0.0.0\n\n # Test the server\n curl http://localhost:8393/predictions \\\n -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"prompt\": \"a cat\"}}'\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for serve\n --host string Host IP to publish the container port on. Use 0.0.0.0 to allow connections from other machines. (default \"127.0.0.1\")\n --playground Start the playground alongside the model\n --playground-port int Port for the playground (0 picks a free port) (default 9000)\n -p, --port int Port on which to listen (default 8393)\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --upload-url string Upload URL for file outputs (e.g. https://example.com/upload/)\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n\n---\n\n# Deploy models with Cog\n\nCog containers are Docker containers that serve an HTTP server\nfor running your model.\nYou can deploy them anywhere that Docker containers run.\n\nThe server inside Cog containers is **coglet**, a Rust-based inference server\nthat handles HTTP requests, worker process management, and run execution.\n\nThis guide assumes you have a model packaged with Cog.\nIf you don't, [follow our getting started guide](getting-started-own-model.md),\nor start from one of the [examples in the Cog repository](examples.md).\n\n## Build a Docker image\n\nBuild your model into a Docker image:\n\n```console\ncog build -t my-model\n```\n\nThe image contains your model code, dependencies, the Cog runtime, and everything in between.\nIt serves an HTTP server on port 5000 when run.\n\n## Push to a registry\n\n`cog push` builds the project and publishes it to an OCI-compliant registry:\n\n```console\ncog push registry.example.com/acme/my-model:v1\n```\n\nThe project configuration decides what Cog publishes. A project with `image` in\n`cog.yaml` publishes a container image. A project with `model` publishes an OCI\nbundle containing the image and any managed weights. Passing a target doesn't\nswitch between these formats.\n\nThe positional target is optional when `image` or `model` already supplies a\ndestination. When present, it overrides that configured destination and every\n`COG_MODEL*` environment variable. The target must be tag-addressable -- Cog\nrejects digest-pinned targets. An untagged image target uses Docker's `latest`\ntag, while an untagged bundle target gets a timestamp tag generated by Cog.\n\nFor a bundle with managed weights, use the same repository where `cog weights\nimport` published the weight manifests. The model tag may differ. If the\nrepository doesn't contain every required weight manifest, Cog fails before\npushing the image and tells you to run `cog weights import` for that repository.\n\nWithout `--json`, a successful push prints the published references as a\nhuman-readable tree on stderr.\n\n### Machine-readable output\n\nUse `--json` when another program needs the immutable references produced by a\npush. For example, this pushes a bundle and prints its model, image, and managed\nweight references:\n\n```console\ncog push registry.example.com/acme/my-model:v1 --json\n```\n\nVersion 1 output for a bundle with managed weights has this shape:\n\n```json\n{\n \"version\": 1,\n \"model\": \"registry.example.com/acme/my-model@sha256:1111111111111111111111111111111111111111111111111111111111111111\",\n \"image\": \"registry.example.com/acme/my-model@sha256:2222222222222222222222222222222222222222222222222222222222222222\",\n \"weights\": [\n {\n \"name\": \"transformer\",\n \"reference\": \"registry.example.com/acme/my-model@sha256:3333333333333333333333333333333333333333333333333333333333333333\"\n }\n ]\n}\n```\n\nThe version 1 fields are:\n\n- `version`: Always `1`.\n- `image`: Always present. The value is the published image's complete,\n digest-pinned reference.\n- `model`: Present only for bundle pushes. The value is the complete,\n digest-pinned bundle reference.\n- `weights`: Present only when the bundle has managed weights. Each entry has a\n `name` and a complete, digest-pinned `reference`. Entries keep their\n `cog.yaml` order.\n\nJSON whitespace and object key order aren't part of the contract.\n\nStdout stays empty while the command runs. Build and upload progress, warnings,\ndiagnostics, and registry-provider messages go to stderr. After the push and\nprovider post-processing both succeed, stdout receives exactly one JSON\ndocument. JSON mode suppresses the human-readable reference tree.\n\nAny build, push, digest-resolution, provider, or serialization failure exits\nnonzero without writing a JSON document to stdout. JSON mode requires Cog to\nresolve every published reference to an immutable digest, so it can report\nfailure even when an image was pushed successfully. Non-JSON image pushes keep\ntheir existing fallback when a registry can't resolve the pushed image's digest.\n\n## Run the model\n\nYou have several options for running a built image.\n\n### Docker\n\nRun the image directly with Docker.\nThis is the approach you'd use for production deployment.\n\n```shell\n# If your model uses a CPU:\ndocker run -d -p 5001:5000 my-model\n\n# If your model uses a GPU:\ndocker run -d -p 5001:5000 --gpus all my-model\n```\n\nThe server listens on port 5000 inside the container (mapped to 5001 above).\n\n### cog serve\n\nFor local development, `cog serve` builds the image and starts the server\nwith your project directory mounted in:\n\n```console\ncog serve\n```\n\nBy default the server runs on port 8393 and the container port is published on\n`127.0.0.1` (localhost), so it is only reachable from your local machine. The\nserver process inside the container binds to `0.0.0.0`; use `--host` to control\nwhich host interface the Docker port mapping is published on.\n\nUse `-p` to choose a different port:\n\n```console\ncog serve -p 5000\n```\n\n## Make a prediction\n\nOnce the server is running, make predictions by sending a POST request\nto the `/predictions` endpoint.\nInputs go inside an `\"input\"` object in the JSON body.\n\n> [!NOTE]\n> The examples below use `localhost:5001`, matching the Docker command above\n> (`-p 5001:5000`). If you used `cog serve`, use `localhost:8393` by default,\n> or the port you passed with `-p`.\n\n```console\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"input\": {\"prompt\": \"a photo of a cat\", \"steps\": 50}}'\n```\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"data:image/png;base64,...\",\n \"metrics\": {\n \"predict_time\": 4.52\n }\n}\n```\n\n> [!IMPORTANT]\n> Inputs **must** be wrapped in an `\"input\"` object.\n> `{\"input\": {\"scale\": 2.0}}` is correct; `{\"scale\": 2.0}` is not.\n\nTo discover what inputs your model accepts,\nview the OpenAPI schema:\n\n```console\ncurl http://localhost:5001/openapi.json\n```\n\n### Passing file inputs\n\nFile inputs (`cog.Path` or `cog.File` types) are passed as strings\ninside the `\"input\"` object.\nThere are two ways to do this:\n\n**1. HTTP/HTTPS URLs**\n\nPass a URL to a publicly accessible file.\nThe server downloads it inside the container:\n\n```console\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"input\": {\"image\": \"https://example.com/photo.jpg\"}}'\n```\n\n**2. Data URLs (base64)**\n\nTo pass a local file, encode it as a [data URL](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs):\n\n```bash\n# Construct a data URL from a local file\nDATA_URL=$(python3 -c \"\nimport base64, mimetypes\nwith open('input.jpg', 'rb') as f:\n data = base64.b64encode(f.read()).decode()\nmime = mimetypes.guess_type('input.jpg')[0] or 'application/octet-stream'\nprint(f'data:{mime};base64,{data}')\n\")\n\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d \"{\\\"input\\\": {\\\"image\\\": \\\"$DATA_URL\\\"}}\"\n```\n\n> [!NOTE]\n> The HTTP API only accepts JSON (`application/json`).\n> Multipart form uploads are not supported.\n> When you use `cog run -i image=@photo.jpg`,\n> the CLI handles the base64 encoding for you automatically.\n\n### Getting output files\n\nWhen a model returns a file output (`cog.Path` or `cog.File`),\nthe response contains a base64-encoded data URL by default:\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"data:image/png;base64,iVBORw0KGgo...\"\n}\n```\n\nTo have the server upload output files to external storage instead,\nstart the server with the `--upload-url` flag. The server then uploads each\nfile output to that URL prefix and returns the resulting URL in the response.\n\nWith `cog serve`:\n\n```console\ncog serve --upload-url https://example.com/upload/\n```\n\nWhen running the image directly with Docker, override the command to start the\nserver with `--upload-url`:\n\n```shell\ndocker run -d -p 5001:5000 my-model \\\n python -m cog.server.http --upload-url https://example.com/upload/\n```\n\nWith an upload URL configured, file outputs are uploaded and the response\ncontains the uploaded URL instead of a data URL:\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"https://example.com/upload/image.png\"\n}\n```\n\n## Run a one-off prediction\n\nThe Docker and `cog serve` commands above leave an HTTP server running.\nIf you instead want to run a single prediction and exit — without starting a\nserver — use `cog run`:\n\n```console\ncog run my-model -i image=@input.jpg\n```\n\nThis starts the container, runs one prediction, prints the result, and exits.\nFile inputs are passed with the `@` prefix (e.g. `-i image=@photo.jpg`),\nand the CLI handles base64 encoding for you.\n\n## Health checks\n\nThe server exposes a `GET /health-check` endpoint that returns the current status of the model container. Use this for readiness probes in orchestration systems like Kubernetes.\n\n```console\ncurl http://localhost:5001/health-check\n```\n\nThe response includes a `status` field with values like `STARTING`, `READY`, `BUSY`, `SETUP_FAILED`, or `DEFUNCT`. See the [HTTP API reference](http.md#get-health-check) for full details.\n\n## Stop the server\n\nIf you started the container with `docker run -d`, stop it with:\n\n```console\ndocker kill \n```\n\nIf you used `cog serve`, press `Ctrl+C` in the terminal.\n(`cog run` exits on its own once the prediction finishes, so there's nothing\nto stop.)\n\n## Concurrency\n\nBy default, the server processes one run at a time. To enable concurrent runs, make your `run()` method async and decorate it with `@cog.concurrent(max=N)`:\n\n```py\nimport cog\n\nclass Runner(cog.BaseRunner):\n @cog.concurrent(max=4)\n async def run(self) -> str:\n return \"hello world\"\n```\n\nThe deprecated [`concurrency.max`](yaml.md#concurrency) field in `cog.yaml` is still supported and takes precedence over the decorator by baking `COG_MAX_CONCURRENCY` into the image.\n\n## Environment variables\n\nYou can configure runtime behavior with environment variables:\n\n- `COG_SETUP_TIMEOUT`: Maximum time in seconds for the `setup()` method (default: no timeout).\n- `COG_MAX_CONCURRENCY`: Number of concurrent prediction slots (default: 1). Overrides both `@cog.concurrent` and deprecated `cog.yaml` concurrency.\n\nSee the [environment variables reference](environment.md) for the full list.\n\n## Next steps\n\n- [HTTP API reference](http.md) for full endpoint documentation\n- [Private registries](private-package-registry.md) for using private Python package registries\n- [`cog.yaml` reference](yaml.md) for configuration options\n\n\n---\n\n# Environment variables\n\nThis reference lists the public Cog-specific environment variables that change how Cog behaves.\n\n## Build-time variables\n\n### `COG_SDK_WHEEL`\n\nControls which Cog Python SDK wheel is installed in the Docker image during `cog build`. Takes precedence over `build.sdk_version` in `cog.yaml`.\n\n**Supported values:**\n\n| Value | Description |\n| -------------------- | ---------------------------------------------------- |\n| `pypi` | Install latest version from PyPI |\n| `pypi:0.12.0` | Install specific version from PyPI |\n| `dist` | Use wheel from `dist/` directory (requires git repo) |\n| `https://...` | Install from URL |\n| `/path/to/wheel.whl` | Install from local file path |\n\n**Default behaviour:**\n\n- Release builds install the latest Cog SDK from PyPI.\n- Development builds auto-detect a wheel in `dist/`, then fall back to the latest Cog SDK from PyPI.\n\n```console\n$ COG_SDK_WHEEL=pypi:0.11.0 cog build\n$ COG_SDK_WHEEL=dist cog build\n$ COG_SDK_WHEEL=https://example.com/cog-0.12.0-py3-none-any.whl cog build\n```\n\nThe `dist` option searches for wheels in:\n\n1. `./dist/` (current directory)\n2. `$REPO_ROOT/dist/` (if `REPO_ROOT` is set)\n3. `/dist/` (via `git rev-parse`, useful when running from subdirectories)\n\n### `COGLET_WHEEL`\n\nControls which coglet wheel is installed in the Docker image. Coglet is the Rust-based inference server.\n\n**Supported values:** Same as `COG_SDK_WHEEL`.\n\n**Default behaviour:** For development builds, auto-detects a wheel in `dist/`. For release builds, installs the latest version from PyPI.\n\n```console\n$ COGLET_WHEEL=dist cog build\n$ COGLET_WHEEL=pypi:0.1.0 cog build\n```\n\n### `COG_CA_CERT`\n\nInjects a custom CA certificate into the Docker image during `cog build`. This is useful when building behind a corporate proxy or VPN that uses custom certificate authorities (for example, Cloudflare WARP).\n\n**Supported values:**\n\n| Value | Description |\n| -------------------------------- | ----------------------------------------------------------- |\n| `/path/to/cert.crt` | Path to a single PEM certificate file |\n| `/path/to/certs/` | Directory of `.crt` and `.pem` files (all are concatenated) |\n| `-----BEGIN CERTIFICATE-----...` | Inline PEM certificate |\n| `LS0tLS1CRUdJTi...` | Base64-encoded PEM certificate |\n\nThe certificate is installed into the system CA store and the `SSL_CERT_FILE` and `REQUESTS_CA_BUNDLE` environment variables are set automatically in the built image.\n\n```console\n$ COG_CA_CERT=/usr/local/share/ca-certificates/corporate-ca.crt cog build\n$ COG_CA_CERT=/etc/custom-certs/ cog build\n$ COG_CA_CERT=\"$(cat /path/to/cert.pem)\" cog build\n```\n\n### `COG_OPENAPI_SCHEMA`\n\nUses a pre-built OpenAPI schema instead of generating one from the configured predict or train reference.\n\nThe value must be a path to a JSON schema file. Cog reads that file during schema generation and embeds it in the built image.\n\n```console\n$ COG_OPENAPI_SCHEMA=./openapi.json cog build\n```\n\n## CLI and local cache variables\n\n### `COG_NO_UPDATE_CHECK`\n\nDisables Cog's automatic update check. Set it to any non-empty value.\n\n```console\n$ COG_NO_UPDATE_CHECK=1 cog build\n```\n\n### `COG_NO_COLOR`\n\nDisables coloured CLI output. Set it to any non-empty value.\n\nCog also honours the standard `NO_COLOR` environment variable.\n\n```console\n$ COG_NO_COLOR=1 cog predict -i prompt=\"hello\"\n```\n\n### `COG_SKIP_DOCKER_CHECK`\n\nSkips the `cog doctor` Docker environment check. Set it to any non-empty value.\n\n```console\n$ COG_SKIP_DOCKER_CHECK=1 cog doctor\n```\n\n### `COG_SKIP_GPU_CHECK`\n\nSkips the `cog doctor` GPU compatibility check, which warns when the resolved torch/CUDA\nversions ship no kernels for the GPU in the current machine. Set it to any non-empty value.\n\nUse this when you are deliberately building for a different GPU than the one in this\nmachine, so the check should not warn about the local device.\n\n```console\n$ COG_SKIP_GPU_CHECK=1 cog doctor\n```\n\n### `COG_CACHE_DIR`\n\nOverrides Cog's local cache root.\n\nCog currently uses this cache for the content-addressed weights store. If unset, Cog uses `$XDG_CACHE_HOME/cog` when `XDG_CACHE_HOME` is set, otherwise `$HOME/.cache/cog`.\n\n```console\n$ COG_CACHE_DIR=/mnt/fast-cache cog weights pull\n```\n\n## Model reference and registry variables\n\n### `COG_MODEL`\n\nOverrides the full model reference used by commands that need a model destination, such as `cog push` and weights commands.\n\nThe value is parsed as a complete model reference (`registry/repo`, `registry/repo:tag`, or `registry/repo@digest`). If no tag is supplied, Cog generates a timestamp tag.\n\nWhen `COG_MODEL` is set, it takes precedence over `COG_MODEL_REGISTRY`, `COG_MODEL_REPO`, and `COG_MODEL_TAG`. A positional target passed to `cog push` takes precedence over all four variables. Although commands that inspect existing artifacts accept a digest, `cog push` rejects digest-pinned destinations because a push requires a tag.\n\n```console\n$ COG_MODEL=r8.im/acme/my-model:v1 cog push\n```\n\n### `COG_MODEL_REGISTRY`\n\nOverrides only the registry host of the model reference.\n\n```console\n$ COG_MODEL_REGISTRY=registry.example.com cog push\n```\n\n### `COG_MODEL_REPO`\n\nOverrides only the repository path of the model reference. The value must not include a registry host, tag, or digest.\n\n```console\n$ COG_MODEL_REPO=acme/my-model cog push\n```\n\n### `COG_MODEL_TAG`\n\nOverrides only the tag of the model reference.\n\nTags starting with `cog-` are reserved for tags that Cog generates internally and are rejected.\n\n```console\n$ COG_MODEL_TAG=staging cog push\n```\n\n### `COG_REGISTRY_HOST`\n\nChanges the default Replicate-compatible registry host used by commands such as `cog login`, base image resolution, and model reference resolution.\n\nThe default is `r8.im`.\n\n```console\n$ COG_REGISTRY_HOST=registry.example.com cog login\n```\n\n## Runtime server variables\n\nThese variables affect a running model server. Set them in `cog.yaml` under `environment`, pass them with `cog predict -e` or `cog serve -e`, or set them when running the built Docker image.\n\n### `COG_MAX_CONCURRENCY`\n\nControls how many predictions the model server can run concurrently. This overrides both `@cog.concurrent(max=N)` and the deprecated `concurrency.max` field in `cog.yaml`.\n\nBy default, Cog runs one prediction at a time unless the model uses `@cog.concurrent(max=N)`. Invalid values are ignored and the default of `1` is used.\n\nValues greater than `1` require an async `run()` method. This applies even when `COG_MAX_CONCURRENCY` is set as a runtime operator override.\n\nConcurrency is resolved in this order, from highest to lowest precedence:\n\n1. `COG_MAX_CONCURRENCY` set at runtime\n2. Deprecated `concurrency.max` in `cog.yaml`, which is baked into the image as `COG_MAX_CONCURRENCY`\n3. `@cog.concurrent(max=N)` on the async `run()` method\n4. Default: `1`\n\n```console\n$ COG_MAX_CONCURRENCY=4 docker run -p 5000:5000 my-model\n```\n\n### `COG_SETUP_TIMEOUT`\n\nControls the maximum time, in seconds, allowed for the model's `setup()` method to complete. If setup exceeds this timeout, the server reports setup failure.\n\nBy default, there is no timeout. Set to `0` to disable the timeout. Invalid values are ignored with a warning.\n\n```console\n$ COG_SETUP_TIMEOUT=300 docker run -p 5000:5000 my-model\n```\n\n### `COG_LOG_LEVEL`\n\nControls Coglet runtime log verbosity when `RUST_LOG` is not set.\n\nSupported values are `debug`, `info`, `warn`, `warning`, and `error`. The default is `info`.\n\n```console\n$ COG_LOG_LEVEL=debug docker run -p 5000:5000 my-model\n```\n\n### OpenTelemetry tracing\n\nTracing must first be enabled under `observability.traces` in `cog.yaml`. Runtime settings may disable an enabled image but cannot enable an image that did not opt in.\n\n| Variable | Purpose |\n| ----------------------------- | ---------------------------------------------- |\n| `COG_TRACE_ENABLED` | Set to `false` to disable tracing at runtime. |\n| `OTEL_SDK_DISABLED` | Hard-disable OpenTelemetry SDK initialization. |\n| `OTEL_EXPORTER_OTLP_ENDPOINT` | Collector endpoint for framework tracing. |\n| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf` or `grpc`. |\n| `OTEL_EXPORTER_OTLP_HEADERS` | Collector authentication headers. |\n| `OTEL_SERVICE_NAME` | Service name, default `cog`. |\n| `OTEL_TRACES_SAMPLER` | Runtime sampler override. |\n| `OTEL_TRACES_SAMPLER_ARG` | Ratio for ratio samplers. |\n\nWhen tracing is enabled without an endpoint, Cog logs one warning and continues without framework tracing. A custom Python provider configured by `observability.config` still runs and may use another exporter or no exporter. Delivery failures from Cog's built-in exporters never change prediction results; custom processors and exporters are model-owned code.\n\n`COG_OBSERVABILITY_*`, `COG_TRACE_*`, and `OTEL_*` are reserved from the general `cog.yaml` `environment` list. `COG_OBSERVABILITY_CONFIG` is internal and points to the validated file staged in the image. Supply supported `COG_TRACE_*` and `OTEL_*` settings to the running container instead.\n\n### `COG_THROTTLE_RESPONSE_INTERVAL`\n\nControls how often asynchronous webhook `output` and `logs` events are sent, in seconds.\n\nThe default is `0.5` seconds. Invalid values are ignored and the default is used. `start` and `completed` webhook events are always sent immediately.\n\n```console\n$ COG_THROTTLE_RESPONSE_INTERVAL=1 docker run -p 5000:5000 my-model\n```\n\n### `COG_STREAM_HISTORY_CAPACITY`\n\nControls how many server-sent event stream events are retained per prediction for replay when a client reconnects with `Accept: text/event-stream`.\n\nBy default, Cog retains the most recent 1024 events per prediction. Set to `0` to disable replay history while keeping live streaming enabled. Invalid values are ignored with a warning and the default is used.\n\n```console\n$ COG_STREAM_HISTORY_CAPACITY=0 docker run -p 5000:5000 my-model\n$ COG_STREAM_HISTORY_CAPACITY=4096 docker run -p 5000:5000 my-model\n```\n\n### `COG_WEIGHTS`\n\nProvides a weights path or URL to a model whose `setup()` method accepts a `weights` parameter.\n\n```console\n$ cog run -e COG_WEIGHTS=https://example.com/weights.tar -i prompt=\"hello\"\n```\n\n### `COG_USER_AGENT`\n\nSets the `User-Agent` header used by Cog when downloading URL-backed `File` inputs.\n\n```console\n$ COG_USER_AGENT=\"my-service/1.0\" docker run -p 5000:5000 my-model\n```\n\n## Push tuning variables\n\n### `COG_PUSH_OCI`\n\nEnables Cog's OCI chunked push path for container image layers when set to `1`. If the OCI push fails with a non-fatal error, Cog falls back to Docker's native push path.\n\n```console\n$ COG_PUSH_OCI=1 cog push\n```\n\n### `COG_PUSH_CONCURRENCY`\n\nControls how many image layers or weight blobs Cog uploads concurrently during push operations.\n\nThe default is `5`. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_CONCURRENCY=2 cog push\n```\n\n### `COG_PUSH_DEFAULT_CHUNK_SIZE`\n\nSets the default multipart upload chunk size, in bytes, when the registry does not advertise a maximum chunk size.\n\nThe default is 96 MiB. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_DEFAULT_CHUNK_SIZE=67108864 cog push\n```\n\n### `COG_PUSH_MULTIPART_THRESHOLD`\n\nSets the minimum blob size, in bytes, before Cog uses multipart upload.\n\nThe default is 128 MiB. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_MULTIPART_THRESHOLD=268435456 cog push\n```\n\n\n---\n\n# Examples\n\nCog's example models now live in the main Cog repository under\n[`examples/`](https://github.com/replicate/cog/tree/main/examples).\n\nIf you want a working project to copy from, start with one of these:\n\n- [`hello-world`](https://github.com/replicate/cog/tree/main/examples/hello-world):\n minimal string input/output example\n- [`hello-image`](https://github.com/replicate/cog/tree/main/examples/hello-image):\n returns a static image\n- [`hello-context`](https://github.com/replicate/cog/tree/main/examples/hello-context):\n shows how to read prediction context\n- [`hello-concurrency`](https://github.com/replicate/cog/tree/main/examples/hello-concurrency):\n demonstrates the `@cog.concurrent` decorator\n- [`hello-train`](https://github.com/replicate/cog/tree/main/examples/hello-train):\n defines a training interface\n- [`hello-replicate`](https://github.com/replicate/cog/tree/main/examples/hello-replicate):\n uses `cog.Secret` inputs and calls the Replicate API\n- [`blur`](https://github.com/replicate/cog/tree/main/examples/blur):\n simple image-to-image transformation\n- [`resnet`](https://github.com/replicate/cog/tree/main/examples/resnet):\n GPU image classification with ResNet50\n- [`streaming-text`](https://github.com/replicate/cog/tree/main/examples/streaming-text):\n streaming text generation with server-sent events\n- [`z-image-turbo`](https://github.com/replicate/cog/tree/main/examples/z-image-turbo):\n a generative image model example\n- [`notebook`](https://github.com/replicate/cog/tree/main/examples/notebook):\n runs a Jupyter notebook with Cog\n- [`canary`](https://github.com/replicate/cog/tree/main/examples/canary):\n small streaming output example used for validation\n- [`managed-weights`](https://github.com/replicate/cog/tree/main/examples/managed-weights):\n managed weights pipeline fixture\n\nThere is also an experimental example:\n\n- [`experimental/resnet-managed-weights`](https://github.com/replicate/cog/tree/main/examples/experimental/resnet-managed-weights):\n ResNet50 packaged with managed weights\n\nFor a quick summary of every example in the tree, see\n[`examples/README.md`](https://github.com/replicate/cog/blob/main/examples/README.md).\n\n\n---\n\n# Getting started with your own model\n\nThis guide will show you how to put your own machine learning model in a Docker image using Cog. If you haven't got a model to try out, you'll want to follow the [main getting started guide](getting-started.md).\n\n## Prerequisites\n\n- **Cog**. If you haven't already installed Cog, follow the [install instructions in the README](../README.md#install).\n- **macOS, Linux, or Windows 11**. Cog works on macOS and Linux. It also works on Windows 11 with [WSL 2](wsl2/wsl2.md).\n- **Docker**. Cog uses Docker to create a container for your model. You'll","tools/test-registry-util/README.md":"# test-registry-util Documentation\n\nPath: `tools/test-registry-util/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-registry-util/README.md)","tools/test-harness/fixtures/models/scalar-types/cog.yaml":"# Scalar-types Skill\n\nPath: `tools/test-harness/fixtures/models/scalar-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/scalar-types/cog.yaml)","tools/test-harness/fixtures/models/optional-types/cog.yaml":"# Optional-types Skill\n\nPath: `tools/test-harness/fixtures/models/optional-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-types/cog.yaml)","tools/test-harness/fixtures/models/optional-list-types/cog.yaml":"# Optional-list-types Skill\n\nPath: `tools/test-harness/fixtures/models/optional-list-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-list-types/cog.yaml)","tools/test-harness/fixtures/models/list-types/cog.yaml":"# List-types Skill\n\nPath: `tools/test-harness/fixtures/models/list-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/list-types/cog.yaml)","tools/test-harness/fixtures/models/file-path-types/cog.yaml":"# File-path-types Skill\n\nPath: `tools/test-harness/fixtures/models/file-path-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/file-path-types/cog.yaml)","tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml":"# Constraints-and-choices Skill\n\nPath: `tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml)","tools/test-harness/fixtures/models/complex-output/cog.yaml":"# Complex-output Skill\n\nPath: `tools/test-harness/fixtures/models/complex-output/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/complex-output/cog.yaml)"},"files":{".agents/skills/release-cog/SKILL.md":"---\nname: release-cog\ndescription: Guide and automate the Cog release process\n---\n\n# Cog Release Skill\n\nThis skill helps you release new versions of Cog. Cog is a multi-language, multi-artifact project with a carefully orchestrated release process.\n\n## Overview\n\nCog releases include:\n\n- **CLI binaries** (Go) - macOS and Linux, x86_64 and ARM64\n- **coglet Python wheels** (Rust/PyO3) - Linux x86_64/ARM64, macOS ARM64\n- **cog SDK Python wheel** (Python) - Universal\n- **coglet Rust crate** - Published to crates.io\n\n## Release Types\n\n| Type | Format | Example | Branch | PyPI | crates.io | Homebrew |\n| --------------- | ------------------------------- | ------------ | ------ | ---- | --------- | -------- |\n| **Stable** | `v0.17.0` | v0.18.0 | main | ✓ | ✓ | ✓ |\n| **Pre-release** | `v0.17.0-alpha3`, `v0.17.0-rc1` | v0.18.0-rc1 | main | ✓ | ✓ | ✗ |\n| **Dev** | `v0.17.0-dev1` | v0.18.0-dev2 | any | ✗ | ✗ | ✗ |\n\n## Quick Release Commands\n\n### 1. Bump Version (if needed)\n\n```bash\n# Check current version\nmise run version\n\n# Bump to new version (updates VERSION.txt, Cargo.toml, Cargo.lock, commits)\nmise run version:bump 0.18.0\n```\n\n### 2. Create a release branch and push\n\n```bash\ngit checkout -b release/v0.18.0\ngit push origin release/v0.18.0\n```\n\n### 3. Create PR to main\n\n```bash\n# Open a pull request to main, get it reviewed and merged. Then you can create the release tag from main.\ngh pr create --base main --head release/v0.18.0 --title \"Release v0.18.0\" --body \"Release description and notes\"\n```\n\n### 4. Create and Push Tag\n\n```bash\ngit checkout main && git pull origin main\n\n# For stable release\ngit tag v0.18.0\ngit push origin v0.18.0\n\n# For pre-release\ngit tag v0.18.0-rc1\ngit push origin v0.18.0-rc1\n\n# For dev release (can be from any branch)\ngit tag v0.18.0-dev1\ngit push origin v0.18.0-dev1\n```\n\n### 5. Monitor Release Build\n\n```bash\n# Watch the release build workflow\ngh workflow view release-build.yaml\n\n# Or watch in real-time\ngh run watch\n```\n\n### 6. Write Release Notes\n\nAfter the draft release is created, update the release notes to follow the project's standard format. The release notes are not auto-generated from commit messages — they must be hand-written and categorized.\n\n**To see the previous release's format:**\n\n```bash\ngh release view v0.19.0 --json body\n```\n\n**To gather commits since the last release:**\n\n```bash\n# List commits between the previous release and this one\ngit log --oneline v0.19.0..v0.20.0 --no-merges\n```\n\n**Release notes structure:**\n\nGroup changes into three sections. Only include sections that have items.\n\n1. **New features** — New commands, new APIs, new annotations, new capabilities.\n2. **Improvements** — Performance, reliability, DX improvements, removals of legacy paths, build improvements.\n3. **Bug fixes** — User-visible bug fixes. Prefer \"Now does X correctly\" over \"Fixed X\".\n\n**Style guidelines:**\n\n- Lead each bullet with a **bold, user-facing summary sentence** (e.g., \"**`cog run` command.** ...\")\n- Follow with a short explanation of what changed and why it matters\n- Reference the PR number in parentheses at the end: `(#3015)`\n- Use backticks for commands, flags, and code references\n- Omit internal refactors, dependency bumps, and CI-only changes unless they are user-facing\n- Omit version bump commits\n\n**Example:**\n\n```markdown\n### New features\n\n- **`cog run` command.** The `cog predict` command has been renamed to `cog run` with full backward compatibility. `cog predict` still works as an alias. (#3015)\n- **Model refs for `cog push` and weights commands.** You can now reference models by name (e.g., `r8.im/user/model`) instead of full image URLs when pushing or managing weights. (#3018)\n\n### Improvements\n\n- **Runtime schema generation fully removed.** The legacy runtime Python schema generation path has been completely removed. Cog exclusively uses static schema generation, making builds faster and more reliable. (#3003)\n\n### Bug fixes\n\n- **Pushing a model with a version tag now emits a clean URL.** The Replicate model URL printed after `cog push` no longer includes the image tag (e.g., `:latest`), preventing 404 errors when users click the link. (#3020)\n```\n\n**To update the draft release:**\n\n```bash\ngh release edit v0.20.0 --notes \"$(cat <<'EOF'\n### New features\n\n- ...\n\n### Improvements\n\n- ...\n\n### Bug fixes\n\n- ...\nEOF\n)\"\n```\n\n### 7. Publish Release (stable/pre-release only)\n\n- Go to GitHub Releases page\n- Find the draft release\n- Review release notes\n- Click \"Publish release\"\n- This triggers `release-publish.yaml` which publishes to PyPI and crates.io\n\n## Release Process Details\n\n### Automated Workflows\n\n1. **`release-build.yaml`** - Triggered on version tags\n - Verifies tag matches VERSION.txt and Cargo.toml\n - Verifies stable/pre-release tags are on main branch\n - Builds SDK wheel (with updated coglet version constraint)\n - Builds coglet wheels for all platforms (Linux x64/ARM64, macOS ARM64)\n - Uses GoReleaser to build CLI binaries and create draft release\n - Uploads wheels to GitHub release\n - For dev releases: immediately publishes as pre-release\n\n2. **`release-publish.yaml`** - Triggered when release is published\n - Publishes coglet wheels to PyPI\n - Publishes coglet crate to crates.io\n - Publishes cog SDK to PyPI (depends on coglet)\n - Updates Homebrew tap (stable releases only)\n\n3. **`homebrew-tap.yaml`** - Updates Homebrew cask\n - Generates cask from `.github/cog.rb.tmpl`\n - Creates PR in `replicate/homebrew-tap`\n\n### Version Files\n\n| File | Purpose |\n| ------------------- | ------------------------------------------ |\n| `VERSION.txt` | Canonical version (single source of truth) |\n| `crates/Cargo.toml` | Rust workspace version |\n| `crates/Cargo.lock` | Locked dependency versions |\n\n### Version Constraints\n\nThe SDK (`pyproject.toml`) has a dependency on coglet:\n\n```toml\ncoglet>=0.1.0,<1.0\n```\n\nDuring release build, this is updated to:\n\n```toml\ncoglet>=0.18.0,<1.0\n```\n\nThis ensures the SDK depends on the matching coglet version.\n\n## Pre-Release Checklist\n\nBefore creating a release tag:\n\n- [ ] All tests pass: `mise run test`\n- [ ] Lint passes: `mise run lint`\n- [ ] Version is correct in `VERSION.txt`\n- [ ] `mise run version:check` passes\n- [ ] `crates/Cargo.toml` matches `VERSION.txt`\n- [ ] Changelog is updated (if applicable)\n- [ ] Documentation is updated (`mise run docs:llm`)\n\n## Troubleshooting\n\n### Version mismatch error\n\n```\nVersion mismatch! VERSION.txt has X but tag is vY\n```\n\nFix: Run `mise run version:bump Y`, push, then re-tag.\n\n### Tag not on main\n\n```\nRelease tags must be on the main branch\n```\n\nFix: Merge your changes to main, then tag from main.\n\n### Rebuilding a failed release\n\n1. Delete the GitHub release if it was created: `gh release delete v0.18.0 --yes`\n2. Delete the tag: `git push --delete origin v0.18.0 && git tag -d v0.18.0`\n3. Fix the issue\n4. Re-create and push the tag\n\n### Manual PyPI publish (emergency)\n\nIf the automated publish fails:\n\n```bash\n# Download wheels from GitHub release\ngh release download v0.18.0 -p \"coglet-*.whl\" -D dist\ngh release download v0.18.0 -p \"cog-*.whl\" -D dist\n\n# Publish with twine\n twine upload dist/coglet-*.whl # First!\ntwine upload dist/cog-*.whl # After coglet is uploaded\n```\n\n## Architecture Notes\n\n- **Trusted Publishing**: PyPI and crates.io use OIDC trusted publishing (no API tokens in secrets)\n- **Environments Required**: Configure `pypi`, `crates-io`, and `homebrew` environments in GitHub repo settings\n- **CGO**: Required for go-tree-sitter (static Python schema parser)\n- **Zig**: Used for Linux cross-compilation (CC=zig cc)\n- **macOS builds**: Use native compiler (zig lacks macOS SDK stubs)\n- **Wheel discovery**: CLI discovers wheels from `dist/` at Docker build time, not embedded in binary\n",".opencode/skills/cog-review/SKILL.md":"---\nname: cog-review\ndescription: Cog architecture and cross-cutting review guidelines\n---\n\n## Cog architecture review guidelines\n\nCog packages ML models into production-ready containers. Use this skill for\nchanges that cross language boundaries or touch core architecture.\n\n### Component overview\n\n- **CLI** (Go): `cmd/cog/` and `pkg/` -- builds, runs, and deploys models\n- **Python SDK**: `python/cog/` -- predictor interface, types, HTTP/queue server\n- **Coglet** (Rust): `crates/` -- prediction server inside containers (HTTP, worker management, IPC)\n\n### Key design patterns\n\n**Wheel resolution**: The CLI discovers SDK and coglet wheels from `dist/` at\nDocker build time. Wheels are NOT embedded in the binary. Changes to build\nartifacts need to account for this.\n\n**Dockerfile generation**: `pkg/dockerfile/` generates Dockerfiles from `cog.yaml`\nconfig. Template injection and escaping matter here.\n\n**Config parsing**: `pkg/config/config.go` parses `cog.yaml`. Schema is at\n`pkg/config/data/config_schema_v1.0.json`. Changes must keep schema and Go\ncode in sync.\n\n**Two-process coglet**: Parent process (HTTP server + orchestrator) and child\nworker process (Python predictor execution) communicate via IPC. Changes to\nthe IPC protocol affect both Rust and Python code.\n\n**Compatibility matrix**: CUDA/PyTorch/TensorFlow compatibility is managed in\n`tools/compatgen/`. Framework version changes have wide blast radius.\n\n### Cross-cutting concerns\n\n- **VERSION.txt** is the single source of truth for versioning. `Cargo.toml` must match.\n- Changes to `python/cog/base_predictor.py` affect all downstream model authors.\n- Changes to `docs/` require running `mise run docs:llm` to regenerate `docs/llms.txt`.\n- CLI reference docs are auto-generated -- changes to `cmd/` or `pkg/cli/` require `mise run docs:cli`.\n- Integration tests in `integration-tests/` use Go's testscript and need a built cog binary.\n\n### What to watch for\n\n- Breaking changes to the predictor interface or cog.yaml schema\n- Docker build changes that affect layer caching or build time\n- IPC protocol changes without updating both Rust and Python sides\n- Version bumps that miss one of the places VERSION.txt needs to match\n- New ML framework versions without compatibility testing\n",".opencode/skills/python-review/SKILL.md":"---\nname: python-review\ndescription: Python code review guidelines for the Cog SDK\n---\n\n## Python review guidelines\n\nThis project uses Python for the SDK (`python/cog/`) which defines the predictor\ninterface, type system, and HTTP/queue server.\n\n### What linters already catch (skip these)\n\nruff handles pycodestyle (E), Pyflakes (F), isort (I), warnings (W), bandit (S),\nbugbear (B), and annotations (ANN). Don't flag issues these would catch.\n\n### What to look for\n\n**Type annotations**\n\n- Required on all function signatures\n- Use `typing_extensions` for backward compatibility\n- Avoid `Any` where a concrete type is possible\n- Check that type annotations actually match runtime behavior\n\n**Compatibility**\n\n- Must support Python 3.10 through 3.13\n- Watch for syntax/stdlib features only available in newer versions\n- `from __future__ import annotations` if using newer annotation syntax\n\n**Error handling**\n\n- No bare `except:` or `except Exception:` that swallows everything\n- Exceptions should have descriptive messages\n- Resource cleanup with context managers, not try/finally when avoidable\n\n**Async patterns**\n\n- Tests use pytest-asyncio -- async tests need proper fixtures\n- Watch for blocking calls inside async functions\n- Proper cleanup of async resources (aclose, async context managers)\n\n**Predictor interface**\n\n- `base_predictor.py` is the core interface -- changes here affect all users\n- `types.py` defines input/output types -- check backward compatibility\n- Server code in `python/cog/server/` handles HTTP -- watch for request handling bugs\n\n**Testing**\n\n- Uses pytest with fixtures\n- tox runs tests across Python 3.10-3.13\n- Test isolation: don't rely on global state or test ordering\n",".opencode/skills/rust-review/SKILL.md":"---\nname: rust-review\ndescription: Rust code review guidelines for Coglet\n---\n\n## Rust review guidelines\n\nThis project uses Rust for Coglet (`crates/`), the prediction server that runs\ninside Cog containers. It handles HTTP requests, worker process management, and\nprediction execution.\n\n### What linters already catch (skip these)\n\nclippy runs in CI. cargo-deny audits dependencies for license/advisory issues.\nDon't flag issues these would catch.\n\n### What to look for\n\n**Error handling**\n\n- Use `thiserror` for typed errors in library code, `anyhow` for application errors\n- Don't use `.unwrap()` or `.expect()` in non-test code unless the invariant is documented\n- Error context: use `.context()` or `.with_context()` from anyhow, not bare `?`\n\n**Ownership and lifetimes**\n\n- Unnecessary cloning where a borrow would work\n- Lifetime issues that suggest a design problem (not just annotation noise)\n- Arc/Mutex when simpler patterns exist\n\n**Async (tokio)**\n\n- Blocking operations inside async contexts (use `spawn_blocking`)\n- Missing `.await` on futures (compiler catches some, but not all logical issues)\n- Proper cancellation handling and cleanup\n- Task/JoinHandle leaks\n\n**Safety**\n\n- Any `unsafe` block needs justification and a safety comment\n- FFI boundaries with Python (PyO3) -- check for panics across FFI, GIL handling\n- Memory safety in IPC between parent and worker processes\n\n**Architecture**\n\n- `crates/coglet/` is the core: HTTP server, worker orchestration, IPC\n- `crates/coglet-python/` is PyO3 bindings for Python predictor integration\n- Two-process architecture: parent (HTTP + orchestrator) and worker (Python execution)\n- Don't mix IPC concerns with HTTP handling\n- Snapshot tests use `insta` -- check if snapshots need updating\n\n**Dependencies**\n\n- New dependencies should be justified -- `crates/deny.toml` audits them\n- Prefer std or existing deps over adding new ones for small tasks\n",".opencode/skills/go-review/SKILL.md":"---\nname: go-review\ndescription: Go code review guidelines for the Cog codebase\n---\n\n## Go review guidelines\n\nThis project uses Go for the CLI (`cmd/cog/`, `pkg/`) and support tooling (`tools/`).\n\n### What linters already catch (skip these)\n\ngolangci-lint runs errcheck, gocritic, gosec, govet, ineffassign, misspell,\nrevive, staticcheck, and unused. Don't flag issues these would catch.\n\n### What to look for\n\n**Error handling**\n\n- Errors returned but not checked or silently discarded\n- User-facing errors should use `pkg/errors.CodedError` with error codes\n- Generic error wrapping that loses context (`fmt.Errorf(\"failed\")` with no `%w`)\n\n**Imports**\n\n- Should be three groups: stdlib, third-party, internal (`github.com/replicate/cog/pkg/...`)\n- Only flag if actually wrong, not cosmetic reordering\n\n**Testing**\n\n- Must use `testify/require` for fatal assertions and `testify/assert` for non-fatal\n- No raw `if` checks with `t.Fatal`/`t.Errorf`\n- Prefer table-driven tests for similar cases\n- Prefer specific assertions (`Equal`, `Contains`, `NoError`) over `True`/`False`\n\n**Concurrency**\n\n- Goroutine leaks (no cleanup path, missing context cancellation)\n- Shared state without synchronization\n- Channel misuse (sends on closed channels, unbuffered channels in wrong contexts)\n\n**Docker/container patterns**\n\n- The CLI uses the Docker Go SDK. Watch for leaked clients, unclosed response bodies\n- Dockerfile generation is in `pkg/dockerfile/` -- template injection risks\n\n**Architecture**\n\n- Commands belong in `pkg/cli/`, business logic in `pkg/`\n- Config parsing/validation in `pkg/config/`\n- Don't mix CLI concerns with library logic\n",".agents/skills/updating-architecture-docs/SKILL.md":"---\nname: updating-architecture-docs\ndescription: \"Use this skill when updating, reviewing, or creating architecture documentation in the architecture/ directory. This includes after refactors, feature additions, component changes, or when auditing docs for accuracy. Use it any time code changes affect how Cog's internals work -- new packages, changed IPC protocols, modified build pipeline, runtime behavior changes. Also use it proactively when reviewing PRs that touch core systems to check whether the architecture docs need updating.\"\n---\n\n# Updating Architecture Docs\n\n## Purpose\n\nThe `architecture/` directory is a bridge. It takes someone who already knows what Cog does and how to use it (that's what `docs/` is for) to the point where they can navigate the source code with confidence. It answers \"how does this work and why is it shaped this way\" -- the 10,000 foot view of the project and its ecosystem.\n\nThe goal is to move a reader from fuzzy concepts to clarity:\n\n- What are the components and what do they own?\n- What's the vocabulary? (What does \"slot\" mean? What's the \"envelope\"?)\n- Where are the boundaries between systems?\n- What are the key design decisions and why were they made?\n- Where do I look in the source to go deeper?\n\nThe architecture docs are **not** a substitute for reading code or a prose summary of the implementation. They give the reader enough context to make the code legible on first contact. Once someone understands that coglet uses a two-process model with slot-based IPC, reading `orchestrator.rs` makes sense. Without that context, it's just code.\n\n## Document structure\n\nThe docs are numbered to suggest a reading order. `00-overview.md` orients the reader and points to everything else. Each subsequent doc builds on concepts from the ones before it -- someone reading about the container runtime should already understand the predictor class and the HTTP API.\n\nThe current docs and their structure aren't fixed. New docs should be added when a topic becomes important enough to deserve its own section (testing philosophy, deployment model, etc.). Docs can be split, merged, or reorganized as the system evolves. The numbering is for reading order, not a permanent taxonomy. Check `architecture/` for the current set.\n\n## Principles\n\n### Bridge, don't summarize\n\nThe architecture docs should give a reader the mental model they need to read the code -- not replace the code. If a section starts reading like a prose restatement of what a module does line by line, it's gone too far. Pull back to the concepts and boundaries.\n\nGood: \"The orchestrator spawns a single worker subprocess and manages its lifecycle. Communication happens over two channels: a control channel for lifecycle events (stdin/stdout, JSON lines) and per-slot Unix sockets for prediction data.\"\n\nBad: \"The `spawn_worker` function in `orchestrator.rs` calls `Command::new('python')` with args `['-c', 'import coglet; coglet.server._run_worker()']` and sets up stdin/stdout pipes. It then reads from the control channel in a loop, matching on `ControlResponse` variants...\"\n\nThe first gives you the mental model. The second just restates the code. A reader with the first description can read `orchestrator.rs` and follow along. A reader with the second didn't need to read the doc at all.\n\n### Point at packages, not files\n\nReference source locations at the **package/directory level** with a description of what that package owns. Specific file paths and line numbers rot as code moves around. A pointer like \"`crates/coglet/src/bridge/` -- IPC protocol and transport\" stays accurate through refactors. \"`bridge/protocol.rs:69` -- ControlRequest enum\" doesn't.\n\nOnly document packages that matter for understanding the system's shape. Generic utility packages (`pkg/util/`, `pkg/path/`, etc.) don't need a mention -- their existence is obvious and they don't help a reader build a mental model. If someone needs them, they'll find them.\n\nWhen a specific file reference is genuinely useful (a key entry point, a non-obvious starting point for understanding a subsystem), include it -- but prefer \"the `PredictionService` in `service.rs`\" over a line number.\n\n### Document boundaries, not internals\n\nFocus on interfaces between components: what messages cross the IPC channel, what the HTTP API contract is, what labels a built image carries, what env vars control behavior. These are the things a reader needs to understand to reason about the system. Implementation details behind those boundaries belong in code comments and crate-level READMEs, not in architecture docs.\n\nThe practical test: would an internal refactor (changing how something works without changing its interface) require updating this doc? If yes, the doc is too detailed.\n\n### Explain the why\n\nDesign decisions are the most valuable content in architecture docs. They're the one thing you can't get from reading the code. \"Why two processes?\" has an answer (isolation, CUDA contexts, crash resilience) that makes the whole architecture make sense. Without it, a reader sees the complexity of IPC and wonders whether it's accidental.\n\nEvery major structural choice should have a short rationale. One or two sentences is enough.\n\n## Current state of the world\n\nThese facts should be reflected consistently across all docs. If the code changes and these become stale, the docs need updating.\n\n**One runtime:** Coglet (Rust/Axum + PyO3) is the sole runtime. There's no legacy Python/FastAPI runtime, no toggle, no \"experimental\" qualifier. Don't frame coglet as an alternative to something else.\n\n**Pydantic is not core:** Schema generation parses Python source in Go with tree-sitter and emits OpenAPI directly. `cog.BaseModel` is a dataclass wrapper. Pydantic BaseModel is supported in user code for compatibility but isn't part of Cog's own type system.\n\n**Static schema generation is the only path:** The tree-sitter-based static schema generator (Go side, `pkg/schema/`) runs on schema-generating `cog build` invocations. Parser failures stop the build before Docker starts; there is no Docker/Python runtime schema fallback or `COG_LEGACY_SCHEMA` toggle.\n\n**Wheels aren't embedded:** SDK and coglet wheels are resolved at Docker build time from PyPI, env vars, or local `dist/` directory. They're not compiled into the Go binary.\n\n**Three codebases:**\n\n- `cmd/cog/` + `pkg/` -- Go CLI and build tooling\n- `python/cog/` -- Python SDK (type definitions, predictor base class, thin server launcher)\n- `crates/coglet/` + `crates/coglet-python/` -- Rust prediction server with PyO3 bindings\n\n## How to update\n\n### When to update\n\nAfter code changes, ask: **did a boundary or interface change?**\n\nNeeds doc updates:\n\n- New or removed IPC message types\n- New or changed HTTP endpoints\n- New CLI commands\n- Changes to the build pipeline (new build steps, changed Dockerfile structure)\n- New top-level packages in `pkg/` or new crates in `crates/`\n- Changes to how components communicate\n- New design decisions or changed rationale for existing ones\n\nDoesn't need doc updates:\n\n- Bug fixes within a component\n- Internal refactors that don't change interfaces\n- Performance improvements\n- Test changes\n\n### Auditing for accuracy\n\nWhen checking docs against the codebase:\n\n1. **Read each doc** and identify claims about the system -- component names, package locations, protocol messages, env vars, build steps, CLI commands\n2. **Verify against source** -- check that things exist and work as described\n3. **Classify issues**:\n - **Structural** -- describes something that doesn't exist or works fundamentally differently (fix immediately)\n - **Misleading** -- technically true but gives the wrong mental model (fix immediately)\n - **Missing** -- a feature or component exists with no doc coverage (add if it's a boundary/interface concern)\n - **Stale reference** -- file/package renamed but concept is right (fix opportunistically)\n4. Fix structural and misleading issues first -- they actively harm readers\n\n### Writing conventions\n\nRead the existing architecture docs before writing. Match their tone and level of detail. New content should feel like it belongs -- if it reads like a different author wrote it, adjust.\n\nThe docs are technical writing. Clear, precise, no filler. Don't inflate -- avoid the kind of padded, formal language that agents default to (\"it should be noted that\", \"robust\", \"comprehensive\", \"this ensures that\"). Just say the thing.\n\nDiagrams use Mermaid or ASCII art. Both are fine. Use whichever communicates the structure more clearly. Keep each diagram focused on one concept.\n","AGENTS.md":"# AGENTS.md\n\nThis file provides guidance to coding agents when working with code in this repository.\n\n## Project Overview\n\nCog is a tool that packages machine learning models in production-ready containers.\n\nIt consists of:\n\n- **Cog CLI** (`cmd/cog/`) - Command-line interface for building, running, and deploying models, written in Go\n- **Python SDK** (`python/cog/`) - Python library for defining model predictors and training in Python\n- **Coglet** (`crates/`) - Rust-based prediction server that runs inside containers, with Python bindings via PyO3\n\nDocumentation for the CLI and SDK is available by reading ./docs/llms.txt.\n\n## Development Commands\n\nDevelopment tasks are managed with [mise](https://mise.jdx.dev/). Run `mise tasks` to see all available tasks.\n\n### Quick Reference\n\n| Task | Description |\n| ----------------------------- | ------------------------------------------------------------ |\n| `mise run fmt` | Check formatting (all languages) |\n| `mise run fmt:fix` | Fix formatting (all languages) |\n| `mise run lint` | Run linters (all languages) |\n| `mise run lint:fix` | Fix lint issues (all languages) |\n| `mise run test:go` | Run Go tests |\n| `mise run test:rust` | Run Rust tests |\n| `mise run test:python` | Run Python tests |\n| `mise run test:integration` | Run integration tests |\n| `mise run build:cog` | Build cog CLI binary |\n| `mise run build:coglet` | Build coglet wheel (dev) |\n| `mise run build:sdk` | Build SDK wheel |\n| `mise run install` | Build and symlink cog to /usr/local/bin |\n| `mise run docs:llm` | **IMPORTANT:** Regenerate `docs/llms.txt` after editing docs |\n| `mise run docs:cli` | Generate CLI reference docs from Go source code |\n| `mise run version` | Show current version from VERSION.txt |\n| `mise run version:bump ` | Bump version everywhere and commit |\n\n### Task Naming Convention\n\nTasks follow a consistent naming pattern:\n\n- **Language-based tasks** for fmt/lint/test/typecheck: `task:go`, `task:rust`, `task:python`\n- **Component-based tasks** for build: `build:cog`, `build:coglet`, `build:sdk`\n- **Check vs Fix**: `fmt` and `lint` default to check mode (non-destructive); use `:fix` suffix to auto-fix\n\n### All Tasks by Category\n\n**Format:**\n\n- `mise run fmt` / `mise run fmt:check` - Check all (alias)\n- `mise run fmt:fix` - Fix all\n- `mise run fmt:go` / `mise run fmt:rust` / `mise run fmt:python` - Per-language\n\n**Lint:**\n\n- `mise run lint` / `mise run lint:check` - Check all (alias)\n- `mise run lint:fix` - Fix all\n- `mise run lint:go` / `mise run lint:rust` / `mise run lint:python` - Per-language\n- `mise run lint:rust:deny` - Check Rust licenses/advisories\n\n**Test:**\n\n- `mise run test:go` - Go unit tests\n- `mise run test:rust` - Rust unit tests\n- `mise run test:python` - Python unit tests (via tox)\n- `mise run test:coglet:python` - Coglet Python binding tests\n- `mise run test:integration` - Integration tests\n\n**Build:**\n\n- `mise run build:cog` - Build cog CLI (development)\n- `mise run build:cog:release` - Build cog CLI (release)\n- `mise run build:coglet` - Build coglet wheel (dev install)\n- `mise run build:coglet:wheel` - Build coglet wheel (native platform)\n- `mise run build:coglet:wheel:linux-x64` - Build for Linux x86_64\n- `mise run build:coglet:wheel:linux-arm64` - Build for Linux ARM64\n- `mise run build:sdk` - Build SDK wheel\n\n**Install:**\n\n- `mise run install` - Symlink cog CLI to `/usr/local/bin` (requires `build:cog` first)\n- `PREFIX=/custom/path mise run install` - Symlink to custom location\n\n**Version:**\n\n- `mise run version` - Show current version from VERSION.txt\n- `mise run version:bump ` - Bump version everywhere and commit\n- `mise run version:check` - Verify VERSION.txt matches Cargo.toml\n\n**Other:**\n\n- `mise run typecheck` - Type check all languages\n- `mise run generate` - Run code generation\n- `mise run clean` - Clean all build artifacts\n- `mise run docs` - Build documentation\n- `mise run docs:serve` - Serve docs locally\n\n## Code Style Guidelines\n\n### Go\n\n- **Imports**: Organize in three groups separated by blank lines: (1) Standard library, (2) Third-party packages, (3) Internal packages (`github.com/replicate/cog/pkg/...`)\n- **Formatting**: Use `mise run fmt:go:fix`\n- **Linting**: Must pass golangci-lint with: errcheck, gocritic, gosec, govet, ineffassign, misspell, revive, staticcheck, unused\n- **Error Handling**: Return errors as values; use `pkg/errors.CodedError` for user-facing errors with error codes\n- **Naming**: CamelCase for exported, camelCase for unexported\n- **Testing**: Use `testify/require` for assertions; prefer table-driven tests\n\nExample import block:\n\n```go\nimport (\n \"fmt\"\n\n \"github.com/spf13/cobra\"\n\n \"github.com/replicate/cog/pkg/config\"\n)\n```\n\n### Python\n\n- **Imports**: Automatically organized by ruff/isort (stdlib → third-party → local)\n- **Formatting**: Use `mise run fmt:python:fix`\n- **Linting**: Must pass ruff checks: E (pycodestyle), F (Pyflakes), I (isort), W (warnings), S (bandit), B (bugbear), ANN (annotations)\n- **Type Annotations**: Required on all function signatures; use `typing_extensions` for compatibility; avoid `Any` where possible\n- **Error Handling**: Raise exceptions with descriptive messages; avoid generic exception catching\n- **Naming**: snake_case for functions/variables/modules, PascalCase for classes\n- **Testing**: Use pytest with fixtures; async tests with pytest-asyncio\n- **Compatibility**: Must support Python 3.10-3.13\n\n### Rust\n\n- **Formatting**: Use `mise run fmt:rust:fix`\n- **Linting**: Must pass `mise run lint:rust` (clippy)\n- **Dependencies**: Audited with `cargo-deny` (see `crates/deny.toml`); run `mise run lint:rust:deny`\n- **Error Handling**: Use `thiserror` for typed errors, `anyhow` for application errors\n- **Naming**: snake_case for functions/variables, PascalCase for types\n- **Testing**: Use `cargo test`; snapshot tests use `insta`\n- **Async**: tokio runtime; async/await patterns\n\n## Working on the CLI and support tooling\n\nThe CLI code is in the `cmd/cog/` and `pkg/` directories. Support tooling is in the `tools/` directory.\n\nThe main commands for working on the CLI are:\n\n- `go run ./cmd/cog` - Runs the Cog CLI directly from source (requires wheel to be built first)\n- `mise run build:cog` - Builds the Cog CLI binary\n- `mise run install` - Symlinks the built binary to `/usr/local/bin` (run `build:cog` first), or to a custom path with `PREFIX=/custom/path mise run install`\n- `mise run test:go` - Runs all Go unit tests\n- `go test ./pkg/...` - Runs tests directly with `go test`\n\n## Working on the Python SDK\n\nThe Python SDK is developed in the `python/cog/` directory. It uses `uv` for virtual environments and `tox` for testing across multiple Python versions.\n\nThe main commands for working on the SDK are:\n\n- `mise run build:sdk` - Builds the Python wheel\n- `mise run test:python` - Runs Python tests across all supported versions\n\n## Working on Coglet (Rust)\n\nCoglet is the Rust-based prediction server that runs inside Cog containers, handling HTTP requests, worker process management, and prediction execution.\n\nThe code is in the `crates/` directory:\n\n- `crates/coglet/` - Core Rust library (HTTP server, worker orchestration, IPC)\n- `crates/coglet-python/` - PyO3 bindings for Python predictor integration (requires Python 3.10+)\n\nFor detailed architecture documentation, see `crates/README.md` and `crates/coglet/README.md`.\n\nThe main commands for working on Coglet are:\n\n- `mise run build:coglet` - Build and install coglet wheel for development (macOS, for local Rust/Python tests)\n- `mise run build:coglet:wheel:linux-x64` - Build Linux x86_64 wheel (required to test Rust changes in Docker containers via `cog predict`/`cog train`)\n- `mise run test:rust` - Run Rust unit tests\n- `mise run lint:rust` - Run clippy linter\n- `mise run fmt:rust:fix` - Format code\n\n### Testing\n\nGo code is tested using the built-in `go test` framework:\n\n- `go test ./pkg/... -run ` - Runs specific Go tests by name\n- `mise run test:go` - Runs all Go unit tests\n\nPython code is tested using `tox`, which allows testing across multiple Python versions and configurations:\n\n- `mise run test:python` - Runs all Python unit tests\n- `uv run tox -e py312-tests -- python/tests/server/test_http.py::test_openapi_specification_with_yield` - Runs a specific Python test\n\nThe integration test suite in `integration-tests/` tests the end-to-end functionality of the Cog CLI and Python SDK using Go's testscript framework:\n\n- `mise run test:integration` - Runs the integration tests\n- `mise run test:integration string_predictor` - Runs a specific integration test\n\nThe integration tests require a built Cog binary, which defaults to the first `cog` in `PATH`. Run tests against a specific binary with the `COG_BINARY` environment variable:\n\n```bash\nmise run build:cog\nCOG_BINARY=dist/go/*/cog mise run test:integration\n```\n\n### Development Workflow\n\n1. Run `mise install` to set up the development environment\n2. Run `mise run build:sdk` after making changes to the `./python` directory\n3. Run `mise run build:coglet:wheel:linux-x64` after making changes to the `./crates` directory (needed for Docker testing)\n4. Run `mise run build:cog` to build the CLI (wheels are picked up from `dist/` at Docker build time, not embedded in the binary)\n5. Run `mise run fmt:fix` to format code\n6. Run `mise run lint` to check code quality\n7. Run `mise run docs:llm` to regenerate `docs/llms.txt` after changing `README.md` or any `docs/*.md` file\n8. Read the `./docs` directory and make sure the documentation is up to date\n\n**IMPORTANT:** Always run `mise run lint` (or the language-specific variant, e.g. `mise run lint:go`) before committing to catch linter errors early. CI will reject PRs that fail lint checks.\n\n## Architecture\n\n### CLI Architecture (Go)\n\nThe CLI follows a command pattern with subcommands. The main components are:\n\n- `pkg/cli/` - Command definitions (build, exec, predict, serve, etc.)\n- `pkg/docker/` - Docker client and container management\n- `pkg/dockerfile/` - Dockerfile generation and templating\n- `pkg/config/` - cog.yaml parsing and validation\n- `pkg/image/` - Image building and pushing logic\n\n### Python SDK Architecture\n\n- `python/cog/` - Core SDK\n - `base_predictor.py` - Base class for model predictors\n - `types.py` - Input/output type definitions\n - `server/` - HTTP/queue server implementation\n - `command/` - Runner implementations for predict/train\n\n### Coglet Architecture (Rust)\n\nThe prediction server that runs inside Cog containers. Uses a two-process architecture: a parent process (HTTP server + orchestrator) and a worker subprocess (Python predictor execution).\n\nSee `crates/README.md` for detailed architecture documentation.\n\n- `crates/coglet/` - Core Rust library (HTTP server, worker orchestration, IPC bridge)\n- `crates/coglet-python/` - PyO3 bindings for Python predictor integration\n\n### Key Design Patterns\n\n1. **Local Wheel Resolution**: The CLI discovers SDK and coglet wheels from `dist/` at Docker build time (not embedded in the binary)\n2. **Docker SDK Integration**: Uses Docker Go SDK for container operations\n3. **Type Safety**: Dataclasses for Python type validation, strongly typed Go interfaces\n4. **Compatibility Matrix**: Automated CUDA/PyTorch/TensorFlow compatibility management\n\nFor comprehensive architecture documentation, see [`architecture/`](./architecture/00-overview.md).\n\n## Common Tasks\n\n### Adding a new CLI command\n\n1. Create command file in `pkg/cli/`\n2. Add command to `pkg/cli/root.go`\n3. Implement business logic in appropriate `pkg/` subdirectory\n4. Add tests\n\n### Modifying Python SDK behavior\n\n1. Edit files in `python/cog/`\n2. Run `mise run build:sdk` to rebuild wheel\n3. Test with `mise run test:python`\n4. Integration test with `mise run test:integration`\n\n### Updating ML framework compatibility\n\n1. See `tools/compatgen/` for compatibility matrix generation\n2. Update framework versions in relevant Dockerfile templates\n3. Test with various framework combinations\n\n### Updating the docs\n\n- Documentation is in the `docs/` directory, written in Markdown and generated into HTML using `mkdocs`.\n- **IMPORTANT:** After editing any file in `docs/` or `README.md`, you MUST run `mise run docs:llm` to regenerate `docs/llms.txt`. This file is used by coding agents and should be kept in sync with the documentation.\n- **IMPORTANT:** CLI reference docs (`docs/cli.md`) are auto-generated from Go source code. After modifying CLI commands in `cmd/` or `pkg/cli/`, run `mise run docs:cli` to regenerate, and ensure `mise run docs:cli:check` passes before committing.\n\n## CI Tool Dependencies\n\nDevelopment tools are managed in **one place**: **`mise.toml`**. CI workflows use\n`jdx/mise-action@v4` (with caching enabled) to install the same tool versions that\ndevelopers use locally. This eliminates version drift between local dev and CI.\n\n**Exceptions:**\n\n| Tool | CI method | Why |\n| -------------------------- | ---------------------- | ---------------------------------------------------------------------- |\n| coglet wheel (maturin+zig) | `PyO3/maturin-action` | Builds inside a manylinux container with bundled maturin and zig |\n| Rust target cache | `Swatinem/rust-cache` | Caches `crates/target/` directory (separate from tool caching) |\n| `check-stubs` Python | `actions/setup-python` | Needs shared-library Python (libpython3.x.so) for PyO3 auto-initialize |\n\n**When updating a tool version**, update `mise.toml` only. CI picks it up automatically.\n\n## Agent Skills\n\nThe `.agents/skills/` directory contains shared skill definitions that provide specialized instructions and workflows for coding agents. Each skill is a subdirectory with a `SKILL.md` file that agents load on demand when the task matches.\n\nSkills are invoked automatically by agents when a matching task is detected. See individual `SKILL.md` files for details on when and how each skill applies.\n\n## Important Files\n\n- `VERSION.txt` - Canonical version (single source of truth)\n- `cog.yaml` - User-facing model configuration\n- `pkg/config/config.go` - Go code for parsing and validating `cog.yaml`\n- `pkg/config/data/config_schema_v1.0.json` - JSON schema for `cog.yaml`\n- `python/cog/base_predictor.py` - Predictor interface\n- `crates/Cargo.toml` - Rust workspace configuration (version must match VERSION.txt)\n- `crates/README.md` - Coglet architecture overview\n- `mise.toml` - Task definitions for development workflow\n\n## Testing Philosophy\n\n- Unit tests for individual components (Go and Python)\n- Integration tests for end-to-end workflows\n- Tests use real Docker operations (no mocking Docker API)\n- Always run `mise run build:sdk` after making Python changes before testing Go code\n- Python 3.10-3.13 compatibility is required\n\n### Go Test Conventions\n\nAll Go tests must use [testify](https://github.com/stretchr/testify) for assertions. Do **not** use raw `if` checks with `t.Fatal`/`t.Errorf` — use `require` and `assert` instead.\n\n- **`require`** — for fatal assertions that should stop the test (setup failures, preconditions):\n ```go\n require.NoError(t, err, \"failed to create client\")\n require.Equal(t, expected, actual)\n require.True(t, condition, \"server should be ready\")\n ```\n- **`assert`** — for non-fatal checks where the test should continue (e.g. validating multiple fields in a loop):\n ```go\n assert.Equal(t, http.StatusOK, resp.StatusCode)\n assert.Contains(t, output, \"expected substring\")\n assert.NoError(t, err, \"prediction %d failed\", i)\n ```\n- Use `require` for errors in setup/teardown and `assert` for the actual test expectations\n- Prefer specific assertions (`Equal`, `Contains`, `NoError`, `Len`, `Less`) over generic `True`/`False` — they produce better failure messages\n- Prefer table-driven tests for testing multiple similar cases\n",".opencode/agents/reviewer.md":"---\ndescription: Read-only code reviewer for pull requests\nmode: primary\nmodel: cf-gateway/deepseek-v4-flash-0731\ntemperature: 0.1\npermission:\n edit: deny\n bash:\n \"*\": deny\n \"git diff*\": allow\n \"git log*\": allow\n \"git show*\": allow\n webfetch: deny\n---\n\nYou are a code reviewer for the Cog project. Cog packages ML models into\nproduction-ready containers. The codebase has three languages: Go (CLI and\ntooling), Python (SDK), and Rust (coglet prediction server).\n\n## How to review\n\n1. Read the full diff. Understand what the PR is trying to do before commenting.\n2. Load the appropriate review skill(s) for the languages touched in the PR:\n - Go files: load `go-review`\n - Python files: load `python-review`\n - Rust files: load `rust-review`\n - Architecture/cross-cutting: load `cog-review`\n3. Focus on substance, not style. Linters handle formatting.\n4. Scope feedback to the PR's purpose. Don't demand unrelated refactors.\n5. Be direct. Say what the problem is, why it matters, and how to fix it.\n\n## Comment format\n\n- Reference specific files and lines: `path/to/file.go:42`\n- Use ```suggestion blocks when proposing concrete code changes\n- Categorize findings:\n - **Blocker**: Must fix before merge (bugs, security, data loss)\n - **Should fix**: Important but not blocking (error handling gaps, edge cases)\n - **Nit**: Minor improvement, take it or leave it\n- If the PR looks good, say so briefly. Don't invent problems.\n","tools/test-harness/README.md":"# Cog Model Test Harness\n\nAutomated test harness for validating Cog models against new CLI/SDK versions.\n\nThis is the Go implementation and replaces the previous Python harness.\n\n## Quick Start\n\n```bash\ncd tools/test-harness\n\n# List all models in the manifest\ngo run . list\n\n# Run all non-GPU models\ngo run . run --no-gpu\n\n# Run a specific model\ngo run . run --model hello-world\n\n# Run GPU models only (requires NVIDIA GPU + nvidia-docker)\ngo run . run --gpu-only\n\n# Output JSON report\ngo run . run --no-gpu --output json --output-file results/report.json\n\n# Build images only (no predictions)\ngo run . build --no-gpu\n\n# Use a locally-built cog binary\ngo run . run --no-gpu --cog-binary ../../dist/go/darwin-arm64/cog\n\n# Test from a git branch (builds CLI + SDK wheel automatically)\ngo run . run --no-gpu --cog-ref main\n```\n\nYou can also run from repo root:\n\n```bash\n(cd tools/test-harness && go run . list)\n(cd tools/test-harness && go run . run --no-gpu)\n```\n\n## Prerequisites\n\n- Go (see repo toolchain)\n- Docker\n- For `--cog-ref`: Go + uv on PATH (`mise install` sets this up)\n- For GPU models: NVIDIA GPU + nvidia-docker runtime\n\n## Manifest and Fixtures\n\n- Manifest: `tools/test-harness/manifest.yaml`\n- Fixtures: `tools/test-harness/fixtures/`\n\nInput values prefixed with `@` resolve to fixture paths. Example:\n\n```yaml\ninputs:\n image: \"@test_image.png\"\n```\n\n## CLI Reference\n\n```text\nusage: test-harness {run,build,list} [options]\n\nCommands:\n run Build and test models (full pipeline)\n build Build Docker images only (no predictions)\n list List models defined in the manifest\n\nCommon options:\n --manifest PATH Path to manifest.yaml\n --model NAME Run only this model (repeatable)\n --no-gpu Skip GPU models\n --gpu-only Only run GPU models\n --sdk-version VER SDK version override\n --cog-version TAG CLI version to download\n --cog-binary PATH Path to local cog binary\n --cog-ref REF Git ref to build cog from source\n --sdk-wheel PATH Local SDK wheel path\n --clean-images Remove Docker images after run (default: keep them)\n --keep-outputs Preserve prediction outputs and print the work directory path\n\nRun options:\n --output {console,json} Output format\n --output-file PATH Write JSON report to file\n```\n\n## Project Layout\n\n```text\ntools/test-harness/\n├── go.mod\n├── main.go\n├── README.md\n├── manifest.yaml\n├── fixtures/\n├── cmd/\n└── internal/\n```\n","DESIGN.md":"# Design\n\n## Background\n\nCog came from Andreas's experience at Spotify and Ben's experience at Docker.\n\nAt Spotify, Andreas noticed a cluster of related problems:\n\n- **It was hard to run open-source machine learning models.** All the advances in machine learning were locked up inside prose in PDFs, scraps of code on GitHub, weights on Google Drive (if you were lucky!). If you wanted to build upon this research, or apply it to real-world problems, you had to implement it all from scratch.\n- **It was hard to deploy machine learning models to production.** Andreas was the only person on the research team who was also an infrastructure engineer. Typically a researcher would have to sit down with Andreas to decide on an API, get a server written, package up dependencies, battle CUDA, get it running efficiently, get it deployed on the cluster, and so on and so forth. It would take weeks to get something running in production.\n\nBen connected this back to his experience at Docker. What Docker did was define a standard box that software could go in. You could put any kind of server software in there – Python, Java, Ruby on Rails, whatever – and you could then know that you could run it on your local machine or on any cloud, as long as it supported Docker. We wanted to do the same thing for machine learning.\n\n## Vision\n\nWe want Cog to be a standard artifact for what a model is and how that model is run.\n\n(More detail...)\n\n## Design principles\n\nThere are a few things driving Cog's design:\n\n- **Reproducible artifact**: When you've put your model in Cog, it'll run anywhere, and _keep_ on running. This is why it's a Docker image, with all of the model's dependencies. Docker images have a content-addressable SHA256 ID, which is the identifier for that model's behavior, byte-for-byte.\n- **Weights inside the image**: We encourage users to put model weights in images. If the weights are on cloud storage somewhere, then they might change or disappear, and the image will produce different results. There's nothing magical about Docker images – they're just a bundle of files. Docker moves around that bundle of files quite slowly, though, but we can optimize that process so it's as fast as reading weights directly from blob storage, or wherever.\n- **Models are just functions**: Models can be lots of things. We are of the opinion that [machine learning is just software](https://replicate.com/blog/machine-learning-needs-better-tools), and a model is just a function. It often needs to be attached to a GPU, but apart from that it's just a normal function that has some input and some output. This is the core difference between Docker's abstraction and Cog's abstraction: Docker packages up an executable, whereas Cog packages up a _function_.\n- **Standard interface**: When you run the Docker container, it serves an HTTP server, that is a standard API for running that function. You can think of it like a remote procedure call.\n- **Self-describing artifact**: A Cog model has it's schema (or type signature, if you're thinking of it as a function) attached to the image as a label. This means systems that work with Cog models can know what the model is and what requests to send to it. This is what powers the forms on Replicate, for example.\n- **Not just the model**: Before Cog, the typical standard packaging formats for machine learning models were at the network level. A way of taking a Tensorflow or PyTorch network and packaging up in a way that would run on lots of different types of accelerators. Things like [ONNX](https://onnx.ai/) or [TVM's IR](https://tvm.apache.org/). We realized that \"models\" are not just the network, but they are also pre- and post-processing, and are so diverse and the field is so fast-moving that you can't possible squeeze it into some high-level abstraction. It just needs to be code running on a computer.\n- **It's just Docker**: Cog models need to run anywhere, and they'll only run anywhere if it's vanilla Docker. We might optimize how Docker images get shipped around to make it faster, but we're not going to invent our own image format.\n- **The API is for software developers**: In the olden days, you have to pass tensors to TFServing and know how to generate a tensor from a JPEG. Cog's API intentionally just speaks JSON, strings, files, etc. It's intended to be the interface between the software developer and the ML engineer. Sort of like Docker was intended to be the interface between the software developer and the infrastructure engineer.\n- **Cog is the APIs and interfaces, not just the software**: The most important thing about Cog is that it defines a standard for what a model is and how to run it. It doesn't necessarily need to involve Cog the piece of software itself. For example, Replicate could serve a model from OpenAI with a Cog API and schema, but it's not packaged or running with Cog under the hood at all – it's just calling the OpenAI API directly.\n","tools/test-harness/manifest.yaml":"# Cog Model Test Manifest\n# =======================\n# Each entry defines a model to test, its inputs, and expected outputs.\n#\n# Input values prefixed with \"@\" are resolved as fixture file paths relative\n# to the fixtures/ directory (e.g. \"@test_image.png\" -> fixtures/test_image.png).\n#\n# Validation types:\n# exact - output string must equal `value` exactly\n# contains - output string must contain `value` as a substring\n# regex - output string must match `pattern`\n# file_exists - output is a file path; optionally check `mime` type\n# json_match - parse output as JSON, assert `match` is a subset\n# json_keys - parse output as JSON dict, assert it has entries\n# not_empty - output is non-empty (loose smoke test)\n#\n# Setup commands:\n# Optional `setup` list of shell commands to run after cloning the repo\n# but before building. Useful for models that generate cog.yaml from\n# templates or need other preparation steps.\n# Example:\n# setup:\n# - \"script/select.sh dev\"\n#\n# Required tools:\n# Optional `requires_tools` list of CLI tools that must be on PATH for\n# setup commands to work. The harness checks before running and prints\n# install instructions for known tools (yq, envsubst).\n# Example:\n# requires_tools: [\"yq\"]\n#\n# Model resolution (base_dir takes priority over repo):\n# - Set `base_dir` for local models -- `path` is resolved under it. A\n# relative `base_dir` is resolved against the repo root; an absolute path\n# is used as-is. Local only; never clones.\n# - Set `repo: owner/name` (with no `base_dir`) to clone from GitHub.\n# - Set neither to default to fixtures/models/ (local fixtures).\n# `repo: local` is no longer supported -- use `base_dir` instead.\n\ndefaults:\n sdk_version: \"latest\" # \"latest\" = newest stable from PyPI; or pin e.g. \"0.16.12\"\n cog_version: \"latest\" # \"latest\" = newest stable release; or pin e.g. \"v0.16.12\"\n\nmodels:\n # ── cog-examples (CPU) ──────────────────────────────────────────────\n\n - name: hello-world\n base_dir: examples\n path: hello-world\n gpu: false\n tests:\n - description: \"basic predict\"\n inputs:\n text: \"world\"\n expect:\n type: exact\n value: \"hello world\"\n\n - name: canary\n base_dir: examples\n path: canary\n gpu: false\n tests:\n - description: \"streaming concatenate iterator\"\n inputs:\n text: \"friend\"\n expect:\n type: contains\n value: \"friend\"\n\n - name: blur\n base_dir: examples\n path: blur\n gpu: false\n tests:\n - description: \"blur an image\"\n inputs:\n image: \"@test_image.png\"\n blur: 5\n expect:\n type: file_exists\n mime: \"image/png\"\n\n - name: hello-image\n base_dir: examples\n path: hello-image\n gpu: false\n tests:\n - description: \"return a static image\"\n inputs: {}\n expect:\n type: file_exists\n\n - name: hello-concurrency\n base_dir: examples\n path: hello-concurrency\n gpu: false\n tests:\n - description: \"async streaming output\"\n inputs:\n total: 3\n interval: 0\n expect:\n type: contains\n value: \"Apple\"\n\n - name: hello-context\n base_dir: examples\n path: hello-context\n gpu: false\n # NOTE: This model uses current_scope().context which may not be\n # available in coglet yet. A failure here is a real compatibility signal.\n tests:\n - description: \"returns input and context\"\n inputs:\n text: \"testing\"\n expect:\n type: json_match\n match:\n inputs:\n text: \"testing\"\n\n - name: hello-train\n base_dir: examples\n path: hello-train\n gpu: false\n train_tests:\n - description: \"train produces weights file\"\n inputs:\n prefix: \"custom\"\n expect:\n type: not_empty\n tests:\n - description: \"predict with default weights\"\n inputs:\n text: \"world\"\n expect:\n type: contains\n value: \"world\"\n\n - name: notebook\n base_dir: examples\n path: notebook\n gpu: false\n tests:\n - description: \"basic predict\"\n inputs:\n name: \"world\"\n expect:\n type: contains\n value: \"hello\"\n\n - name: managed-weights\n base_dir: examples\n path: managed-weights\n gpu: false\n requires_env:\n - COG_MANAGED_WEIGHTS_READY\n tests:\n - description: \"weight validation smoke test\"\n inputs: {}\n expect:\n type: not_empty\n\n - name: streaming-text\n base_dir: examples\n path: streaming-text\n gpu: false\n tests:\n - description: \"streaming text generation\"\n inputs:\n prompt: \"Hello\"\n expect:\n type: not_empty\n\n # ── cog-examples (GPU) ──────────────────────────────────────────────\n\n - name: resnet\n base_dir: examples\n path: resnet\n gpu: true\n tests:\n - description: \"classify hotdog image\"\n inputs:\n image: \"@hotdog.png\"\n expect:\n type: json_keys\n\n - name: z-image-turbo\n base_dir: examples\n path: z-image-turbo\n gpu: true\n timeout: 600\n tests:\n - description: \"generate image from prompt\"\n inputs:\n prompt: \"a cat sitting on a windowsill\"\n expect:\n type: file_exists\n mime: \"image/png\"\n\n # ── Local fixture models ─────────────────────────────────────────────\n # These set base_dir to the fixtures dir (relative to the repo root), so\n # path is relative to it. They cover the full input type matrix for build\n # and prediction testing.\n\n - name: fixture-scalar-types\n base_dir: tools/test-harness/fixtures/models\n path: scalar-types\n gpu: false\n tests:\n - description: \"basic scalar inputs\"\n inputs:\n text: \"hello\"\n expect:\n type: contains\n value: \"hello\"\n\n - name: fixture-optional-types\n base_dir: tools/test-harness/fixtures/models\n path: optional-types\n gpu: false\n tests:\n - description: \"optional inputs default to None\"\n inputs:\n text: \"hello\"\n expect:\n type: exact\n value: \"hello\"\n\n - name: fixture-list-types\n base_dir: tools/test-harness/fixtures/models\n path: list-types\n gpu: false\n\n - name: fixture-optional-list-types\n base_dir: tools/test-harness/fixtures/models\n path: optional-list-types\n gpu: false\n tests:\n - description: \"optional lists default to None\"\n inputs:\n text: \"hello\"\n expect:\n type: exact\n value: \"hello\"\n\n - name: fixture-constraints-and-choices\n base_dir: tools/test-harness/fixtures/models\n path: constraints-and-choices\n gpu: false\n tests:\n - description: \"defaults with constraints and choices\"\n inputs: {}\n expect:\n type: exact\n value: \"hello-0.7-50-balanced-1\"\n\n - name: fixture-file-path-types\n base_dir: tools/test-harness/fixtures/models\n path: file-path-types\n gpu: false\n\n - name: fixture-complex-output\n base_dir: tools/test-harness/fixtures/models\n path: complex-output\n gpu: false\n tests:\n - description: \"structured BaseModel output\"\n inputs:\n prompt: \"test\"\n expect:\n type: json_match\n match:\n text: \"generated: test\"\n score: 0.95\n\n # ── External models (add your own below) ────────────────────────────\n # - name: my-custom-model\n # repo: myorg/my-model-repo\n # path: \".\"\n # gpu: true\n # # sdk_version: \"0.16.12\" # optional per-model override\n # env:\n # HF_TOKEN: \"${HF_TOKEN}\"\n # timeout: 600\n # tests:\n # - description: \"smoke test\"\n # inputs:\n # prompt: \"hello\"\n # expect:\n # type: contains\n # value: \"result\"\n #\n # Models that need setup commands before building (e.g. generating cog.yaml):\n # - name: flux-dev\n # repo: replicate/cog-flux\n # path: \".\"\n # gpu: true\n # setup:\n # - \"script/select.sh dev\"\n # timeout: 900\n # tests:\n # - description: \"generate image\"\n # inputs:\n # prompt: \"a cat\"\n # expect:\n # type: file_exists\n","docs/llms.txt":"# Cog: Containers for machine learning\n\nCog is an open-source tool that lets you package machine learning models in a standard, production-ready container.\n\nYou can deploy your packaged model to your own infrastructure, or to [Replicate](https://replicate.com/).\n\n## Highlights\n\n- 📦 **Docker containers without the pain.** Writing your own `Dockerfile` can be a bewildering process. With Cog, you define your environment with a [simple configuration file](#how-it-works) and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.\n\n- 🤬️ **No more CUDA hell.** Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.\n\n- ✅ **Define the inputs and outputs for your model with standard Python.** Then, Cog generates an OpenAPI schema and validates the inputs and outputs.\n\n- 🎁 **Automatic HTTP inference server**: Your model's types are used to dynamically generate a RESTful HTTP API using a high-performance Rust/Axum server.\n\n- 🚀 **Ready for production.** Deploy your model anywhere that Docker images run. Your own infrastructure, or [Replicate](https://replicate.com).\n\n## How it works\n\nDefine the Docker environment your model runs in with `cog.yaml`:\n\n```yaml\nbuild:\n gpu: true\n system_packages:\n - \"libgl1\"\n - \"libglib2.0-0\"\n python_version: \"3.13\"\n python_requirements: requirements.txt\nrun: \"run.py:Runner\"\n```\n\nDefine how your model runs with `run.py`:\n\n```python\nfrom cog import BaseRunner, Input, Path\nimport torch\n\nclass Runner(BaseRunner):\n def setup(self):\n \"\"\"Load the model into memory to make running multiple inferences efficient\"\"\"\n self.model = torch.load(\"./weights.pth\")\n\n # The arguments and types the model takes as input\n def run(self,\n image: Path = Input(description=\"Grayscale input image\")\n ) -> Path:\n \"\"\"Run the model\"\"\"\n processed_image = preprocess(image)\n output = self.model(processed_image)\n return postprocess(output)\n```\n\nIn the above we accept a path to the image as an input, and return a path to our transformed image after running it through our model.\n\nNow, you can run the model:\n\n```console\n$ cog run -i image=@input.jpg\n--> Building Docker image...\n--> Running...\n--> Output written to output.jpg\n```\n\nOr, build a Docker image for deployment:\n\n```console\n$ cog build -t my-classification-model\n--> Building Docker image...\n--> Built my-classification-model:latest\n\n$ docker run -d -p 5000:5000 --gpus all my-classification-model\n\n$ curl http://localhost:5000/predictions -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"image\": \"https://.../input.jpg\"}}'\n```\n\nOr, combine build and run via the `serve` command:\n\n```console\n$ cog serve -p 8080\n\n$ curl http://localhost:8080/predictions -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"image\": \"https://.../input.jpg\"}}'\n```\n\n\n\n## Why are we building this?\n\nIt's really hard for researchers to ship machine learning models to production.\n\nPart of the solution is Docker, but it is so complex to get it to work: Dockerfiles, pre-/post-processing, Flask servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.\n\n[Andreas](https://github.com/andreasjansson) and [Ben](https://github.com/bfirsh) created Cog. Andreas used to work at Spotify, where he built tools for building and deploying ML models with Docker. Ben worked at Docker, where he created [Docker Compose](https://github.com/docker/compose).\n\nWe realized that, in addition to Spotify, other companies were also using Docker to build and deploy machine learning models. [Uber](https://eng.uber.com/michelangelo-pyml/) and others have built similar systems. So, we're making an open source version so other people can do this too.\n\nHit us up if you're interested in using it or want to collaborate with us. [We're on Discord](https://discord.gg/replicate) or email us at [team@replicate.com](mailto:team@replicate.com).\n\n## Prerequisites\n\n- **macOS, Linux or Windows 11**. Cog works on macOS, Linux and Windows 11 with [WSL 2](docs/wsl2/wsl2.md)\n- **Docker**. Cog uses Docker to create a container for your model. You'll need to [install Docker](https://docs.docker.com/get-docker/) before you can run Cog. If you install Docker Engine instead of Docker Desktop, you will need to [install Buildx](https://docs.docker.com/build/architecture/#buildx) as well.\n\n## Install\n\nChoose your platform for installation instructions.\n\n
\n macOS\n\nThe easiest way to install Cog on macOS is with Homebrew:\n\n```console\nbrew install replicate/tap/cog\n```\n\nYou can also use the install script:\n\n```sh\n# bash, zsh, and other shells\nsh <(curl -fsSL https://cog.run/install.sh)\n\n# fish shell\nsh (curl -fsSL https://cog.run/install.sh | psub)\n\n# download with wget and run in a separate command\nwget -qO- https://cog.run/install.sh\nsh ./install.sh\n```\n\nOr install manually:\n\n```console\nsudo curl -o /usr/local/bin/cog -L \"https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')\"\nsudo chmod +x /usr/local/bin/cog\nsudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true\n```\n\nIf you see a Gatekeeper warning saying the binary \"cannot be opened because the developer cannot be verified\", run:\n\n```console\nsudo xattr -d com.apple.quarantine /usr/local/bin/cog\n```\n\n
\n\n
\n Linux\n\nYou can install Cog using the install script:\n\n```sh\n# bash, zsh, and other shells\nsh <(curl -fsSL https://cog.run/install.sh)\n\n# fish shell\nsh (curl -fsSL https://cog.run/install.sh | psub)\n\n# download with wget and run in a separate command\nwget -qO- https://cog.run/install.sh\nsh ./install.sh\n```\n\nOr install manually:\n\n```console\nsudo curl -o /usr/local/bin/cog -L \"https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')\"\nsudo chmod +x /usr/local/bin/cog\n```\n\n
\n\n
\n Windows\n\nCog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above.\n\n
\n\n
\n Docker\n\nTo install Cog inside a Docker image:\n\n```dockerfile\nRUN sh -c \"INSTALL_DIR=\\\"/usr/local/bin\\\" SUDO=\\\"\\\" $(curl -fsSL https://cog.run/install.sh)\"\n```\n\n
\n\n## Upgrade\n\nIf you're using macOS and you previously installed Cog with Homebrew, run the following:\n\n```console\nbrew upgrade replicate/tap/cog\n```\n\nOtherwise, you can upgrade to the latest version by running the same commands you used to install it.\n\n## Development\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for how to set up a development environment and build from source.\n\n## Next steps\n\n- [Get started with an example model](docs/getting-started.md)\n- [Get started with your own model](docs/getting-started-own-model.md)\n- [Using Cog with notebooks](docs/notebooks.md)\n- [Using Cog with Windows 11](docs/wsl2/wsl2.md)\n- [Browse the example models in this repo](docs/examples.md)\n- [Deploy models with Cog](docs/deploy.md)\n- [`cog.yaml` reference](docs/yaml.md) to learn how to define your model's environment\n- [Run interface reference](docs/python.md) to learn how the `Runner` interface works\n- [Training interface reference](docs/training.md) to learn how to add a fine-tuning API to your model\n- [HTTP API reference](docs/http.md) to learn how to use the HTTP API that models serve\n\n## Need help?\n\n[Join us in #cog on Discord.](https://discord.gg/replicate)\n\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/replicate/cog)\n\n\n\n---\n\n# Base images\n\nCog builds your model into a Docker image. To speed up builds and reduce cold boot times, Cog uses **prebuilt base images** by default. These images contain the common dependencies that most models need, so Cog doesn't have to install them from scratch every time you build or push.\n\n## What is a Cog base image?\n\nA Cog base image is a Docker image maintained by Replicate that includes:\n\n- **Python runtime** — the Python version specified in your `cog.yaml`.\n- **System libraries** — common dependencies for machine learning and media processing, including `ffmpeg`, `git`, `curl`, `build-essential`, `cmake`, OpenCV libraries, audio libraries (`sox`, `libsndfile1`), and graphics libraries (`libgl1`, `libsm6`, `libxext6`).\n- **CUDA and PyTorch** (GPU images only) — the appropriate CUDA toolkit and PyTorch stack for your configuration.\n\nBase images are tagged by their configuration, for example:\n\n```\nr8.im/cog-base:cuda12.4-python3.13-torch2.6.0\nr8.im/cog-base:python3.13\n```\n\nWhen you run `cog build` or `cog push`, Cog selects the base image that matches your Python version, CUDA version, and PyTorch version. Because these images are pre-pulled on Replicate's infrastructure, models built on top of them start faster.\n\nThe entire list of available base images can be viewed at [https://r8.im/v2/cog-base/tags/list](https://r8.im/v2/cog-base/tags/list).\n\n## Using the Cog base image\n\nThe `--use-cog-base-image` flag controls whether Cog uses a prebuilt base image. It is **enabled by default** on the following commands:\n\n- [`cog build`](cli.md#cog-build)\n- [`cog push`](cli.md#cog-push)\n- [`cog run`](cli.md#cog-run)\n- [`cog serve`](cli.md#cog-serve)\n- [`cog exec`](cli.md#cog-exec)\n\nSince it's on by default, you don't need to pass any flags:\n\n```bash\ncog push r8.im/your-username/my-model\n```\n\nThis builds and pushes your model using a prebuilt Cog base image for faster cold boots.\n\n## Disabling the Cog base image\n\nIf you encounter build issues or need a custom base layer, you can disable the Cog base image:\n\n```bash\ncog build --use-cog-base-image=false\n```\n\nWhen disabled, Cog generates a Dockerfile from scratch using either an NVIDIA CUDA base image or a slim Python base image, depending on the `--use-cuda-base-image` flag.\n\n## Relationship to `--use-cuda-base-image`\n\nThe `--use-cuda-base-image` flag controls which underlying base image Cog uses **when the Cog base image is disabled**. It has no effect when `--use-cog-base-image` is enabled (the default), because the Cog base image already includes the appropriate CUDA and Python stack.\n\nWhen you disable the Cog base image with `--use-cog-base-image=false`, Cog chooses the base image automatically:\n\n- **GPU models** (`gpu: true` in `cog.yaml`): uses an NVIDIA CUDA base image.\n- **CPU models**: uses a slim Python base image.\n\nThese flags — along with `--dockerfile` — are **mutually exclusive**: you can only set one of them explicitly on a given command. To customize the base image, disable the Cog base image and let Cog choose between CUDA and Python automatically.\n\n## Troubleshooting\n\nIf `cog build` or `cog push` fails with the Cog base image enabled, try disabling it:\n\n```bash\ncog push --use-cog-base-image=false r8.im/your-username/my-model\n```\n\nThis falls back to building from a standard CUDA or Python base image, which can help diagnose whether the issue is with the base image or your model's configuration.\n\n\n---\n\n# CLI reference\n\n\n\n## `cog`\n\nContainers for machine learning.\n\nTo get started, take a look at the documentation:\nhttps://github.com/replicate/cog\n\n**Examples**\n\n```\n To execute a command inside a Docker environment defined with Cog:\n $ cog exec echo hello world\n```\n\n**Options**\n\n```\n --debug Show debugging output\n -h, --help help for cog\n --no-color Disable colored output\n --version Show version of Cog\n```\n\n## `cog build`\n\nBuild a Docker image from the cog.yaml in the current directory.\n\nThe generated image contains your model code, dependencies, and the Cog\nruntime. It can be run locally with 'cog run' or pushed to a registry\nwith 'cog push'.\n\n```\ncog build [flags]\n```\n\n**Examples**\n\n```\n # Build with default settings\n cog build\n\n # Build and tag the image\n cog build -t my-model:latest\n\n # Build without using the cache\n cog build --no-cache\n\n # Build with model weights in a separate layer\n cog build --separate-weights -t my-model:v1\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n -h, --help help for build\n --no-cache Do not use cache when building the image\n --openapi-schema string Load OpenAPI schema from a file\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'\n --separate-weights Separate model weights from code in image layers\n -t, --tag string A name for the built image in the form 'repository:tag'\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog doctor`\n\nDiagnose and fix common issues in your Cog project.\n\nNOTE: cog doctor is experimental. Behavior and checks may change in future versions.\n\nBy default, cog doctor reports problems without modifying any files.\nPass --fix to automatically apply safe fixes.\n\n```\ncog doctor [flags]\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --fix Automatically apply fixes\n -h, --help help for doctor\n```\n\n## `cog exec`\n\nExecute a command inside a Docker environment defined by cog.yaml.\n\nCog builds a temporary image from your cog.yaml configuration and runs the\ngiven command inside it. This is useful for debugging, running scripts, or\nexploring the environment your model will run in.\n\n```\ncog exec [arg...] [flags]\n```\n\n**Examples**\n\n```\n # Open a Python interpreter inside the model environment\n cog exec python\n\n # Run a script\n cog exec python train.py\n\n # Run with environment variables\n cog exec -e HUGGING_FACE_HUB_TOKEN=abc123 python download.py\n\n # Expose a port (e.g. for Jupyter)\n cog exec -p 8888 jupyter notebook\n```\n\n**Options**\n\n```\n -e, --env stringArray Environment variables, in the form name=value\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for exec\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n -p, --publish stringArray Publish a container's port to the host, e.g. -p 8000 or -p 0.0.0.0:8000\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog init`\n\nCreate a cog.yaml and run.py in the current directory.\n\nThese files provide a starting template for defining your model's environment\nand run interface. Edit them to match your model's requirements.\n\n```\ncog init [flags]\n```\n\n**Examples**\n\n```\n # Set up a new Cog project in the current directory\n cog init\n```\n\n**Options**\n\n```\n -h, --help help for init\n```\n\n## `cog login`\n\nLog in to a container registry.\n\nFor Replicate's registry (r8.im), this command handles authentication\nthrough Replicate's token-based flow.\n\nFor other registries, this command prompts for username and password,\nthen stores credentials using Docker's credential system.\n\n```\ncog login [flags]\n```\n\n**Options**\n\n```\n -h, --help help for login\n --token-stdin Pass login token on stdin instead of opening a browser. You can find your Replicate login token at https://replicate.com/auth/token\n```\n\n## `cog playground`\n\nOpen a browser playground for talking to a running model.\n\nStarts a local web server that serves a schema-driven UI (a Postman-like tool\nfor Cog models). Point it at any running Cog HTTP API -- for example one started\nwith 'cog serve' -- and the playground reflects that model's inputs and outputs\nfrom its OpenAPI schema in real time.\n\nRequests are reverse-proxied through this server, so the target API does not\nneed to set CORS headers. The server also hosts a webhook sink so async\npredictions can be observed in the browser.\n\nAsync/webhook testing against a containerized model requires the webhook URL to\nbe reachable from inside the container. On Docker Desktop the default\n'host.docker.internal' works once the server listens on a reachable interface\n(e.g. --host 0.0.0.0).\n\n```\ncog playground [flags]\n```\n\n**Examples**\n\n```\n # Start a model API in one terminal\n cog serve -p 8393\n\n # Open the playground pointing at it\n cog playground --target http://localhost:8393\n```\n\n**Options**\n\n```\n -h, --help help for playground\n --host string Address to bind (use 0.0.0.0 to receive webhooks from containers) (default \"127.0.0.1\")\n --no-open Do not open the browser automatically\n -p, --port int Port to listen on (0 picks a free port)\n --target string Default target model API URL (default \"http://localhost:8393\")\n --webhook-host string Hostname the model uses to reach this server for webhooks (default \"host.docker.internal\")\n```\n\n## `cog push`\n\nBuild from cog.yaml and push to an OCI-compliant registry. Run 'cog login'\nfirst when pushing to Replicate's registry (r8.im).\n\nTARGET overrides the configured destination and all COG_MODEL\\* environment\nvariables. It doesn't change the project format: projects configured with\n'image' push an image, while projects configured with 'model' push an OCI\nbundle. Push targets must use tags, not digests. Untagged image targets use\nDocker's default 'latest' tag; untagged bundle targets get a timestamp tag.\n\nWith --json, Cog writes one versioned JSON result to stdout after the entire\npush succeeds. Progress, warnings, and diagnostics continue on stderr. Every\nreference in the result is digest-pinned.\n\n```\ncog push [TARGET] [flags]\n```\n\n**Examples**\n\n```\n # Push an image to Replicate\n cog push r8.im/your-username/my-model\n\n # Push an image to any OCI registry\n cog push registry.example.com/your-username/model-name\n\n # Push a bundle project and print its immutable references as JSON\n cog push registry.example.com/your-username/model-name:v1 --json\n\n # Push with model weights in a separate image layer (Replicate only)\n cog push r8.im/your-username/my-model --separate-weights\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n -h, --help help for push\n --json Output the pushed references as JSON\n --no-cache Do not use cache when building the image\n --openapi-schema string Load OpenAPI schema from a file\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'\n --separate-weights Separate model weights from code in image layers\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog run`\n\nRun the model.\n\nIf 'image' is passed, it will run the model on that Docker image.\nIt must be an image that has been built by Cog.\n\nOtherwise, it will build the model in the current directory and run\nit.\n\n```\ncog run [image] [flags]\n```\n\n**Examples**\n\n```\n # Run the model with named inputs\n cog run -i prompt=\"a photo of a cat\"\n\n # Pass a file as input\n cog run -i image=@photo.jpg\n\n # Save output to a file\n cog run -i image=@input.jpg -o output.png\n\n # Pass multiple inputs\n cog run -i prompt=\"sunset\" -i width=1024 -i height=768\n\n # Run against a pre-built image\n cog run r8.im/your-username/my-model -i prompt=\"hello\"\n\n # Pass inputs as JSON\n echo '{\"prompt\": \"a cat\"}' | cog run --json @-\n```\n\n**Options**\n\n```\n -e, --env stringArray Environment variables, in the form name=value\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for run\n -i, --input stringArray Inputs, in the form name=value. if value is prefixed with @, then it is read from a file on disk. E.g. -i path=@image.jpg\n --json string Pass inputs as JSON object, read from file (@inputs.json) or via stdin (@-)\n -o, --output string Output path\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --setup-timeout uint32 The timeout for a container to setup (in seconds). (default 300)\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n --use-replicate-token Pass REPLICATE_API_TOKEN from local environment into the model context\n```\n\n## `cog serve`\n\nRun an HTTP server.\n\nBuilds the model and starts an HTTP server that exposes the model's inputs\nand outputs as a REST API. Compatible with the Cog HTTP protocol.\n\nBy default the container port is published on 127.0.0.1 (localhost), so the\nserver is only reachable from your local machine. The server process inside\nthe container binds to 0.0.0.0; use --host to control which host interface\nthe Docker port mapping is published on.\n\n```\ncog serve [flags]\n```\n\n**Examples**\n\n```\n # Start the server on the default port (8393)\n cog serve\n\n # Start on a custom port\n cog serve -p 5000\n\n # Start the playground alongside the model\n cog serve --playground\n\n # Listen on all interfaces (e.g. to expose to the network)\n cog serve --host 0.0.0.0\n\n # Test the server\n curl http://localhost:8393/predictions \\\n -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"prompt\": \"a cat\"}}'\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for serve\n --host string Host IP to publish the container port on. Use 0.0.0.0 to allow connections from other machines. (default \"127.0.0.1\")\n --playground Start the playground alongside the model\n --playground-port int Port for the playground (0 picks a free port) (default 9000)\n -p, --port int Port on which to listen (default 8393)\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --upload-url string Upload URL for file outputs (e.g. https://example.com/upload/)\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n\n---\n\n# Deploy models with Cog\n\nCog containers are Docker containers that serve an HTTP server\nfor running your model.\nYou can deploy them anywhere that Docker containers run.\n\nThe server inside Cog containers is **coglet**, a Rust-based inference server\nthat handles HTTP requests, worker process management, and run execution.\n\nThis guide assumes you have a model packaged with Cog.\nIf you don't, [follow our getting started guide](getting-started-own-model.md),\nor start from one of the [examples in the Cog repository](examples.md).\n\n## Build a Docker image\n\nBuild your model into a Docker image:\n\n```console\ncog build -t my-model\n```\n\nThe image contains your model code, dependencies, the Cog runtime, and everything in between.\nIt serves an HTTP server on port 5000 when run.\n\n## Push to a registry\n\n`cog push` builds the project and publishes it to an OCI-compliant registry:\n\n```console\ncog push registry.example.com/acme/my-model:v1\n```\n\nThe project configuration decides what Cog publishes. A project with `image` in\n`cog.yaml` publishes a container image. A project with `model` publishes an OCI\nbundle containing the image and any managed weights. Passing a target doesn't\nswitch between these formats.\n\nThe positional target is optional when `image` or `model` already supplies a\ndestination. When present, it overrides that configured destination and every\n`COG_MODEL*` environment variable. The target must be tag-addressable -- Cog\nrejects digest-pinned targets. An untagged image target uses Docker's `latest`\ntag, while an untagged bundle target gets a timestamp tag generated by Cog.\n\nFor a bundle with managed weights, use the same repository where `cog weights\nimport` published the weight manifests. The model tag may differ. If the\nrepository doesn't contain every required weight manifest, Cog fails before\npushing the image and tells you to run `cog weights import` for that repository.\n\nWithout `--json`, a successful push prints the published references as a\nhuman-readable tree on stderr.\n\n### Machine-readable output\n\nUse `--json` when another program needs the immutable references produced by a\npush. For example, this pushes a bundle and prints its model, image, and managed\nweight references:\n\n```console\ncog push registry.example.com/acme/my-model:v1 --json\n```\n\nVersion 1 output for a bundle with managed weights has this shape:\n\n```json\n{\n \"version\": 1,\n \"model\": \"registry.example.com/acme/my-model@sha256:1111111111111111111111111111111111111111111111111111111111111111\",\n \"image\": \"registry.example.com/acme/my-model@sha256:2222222222222222222222222222222222222222222222222222222222222222\",\n \"weights\": [\n {\n \"name\": \"transformer\",\n \"reference\": \"registry.example.com/acme/my-model@sha256:3333333333333333333333333333333333333333333333333333333333333333\"\n }\n ]\n}\n```\n\nThe version 1 fields are:\n\n- `version`: Always `1`.\n- `image`: Always present. The value is the published image's complete,\n digest-pinned reference.\n- `model`: Present only for bundle pushes. The value is the complete,\n digest-pinned bundle reference.\n- `weights`: Present only when the bundle has managed weights. Each entry has a\n `name` and a complete, digest-pinned `reference`. Entries keep their\n `cog.yaml` order.\n\nJSON whitespace and object key order aren't part of the contract.\n\nStdout stays empty while the command runs. Build and upload progress, warnings,\ndiagnostics, and registry-provider messages go to stderr. After the push and\nprovider post-processing both succeed, stdout receives exactly one JSON\ndocument. JSON mode suppresses the human-readable reference tree.\n\nAny build, push, digest-resolution, provider, or serialization failure exits\nnonzero without writing a JSON document to stdout. JSON mode requires Cog to\nresolve every published reference to an immutable digest, so it can report\nfailure even when an image was pushed successfully. Non-JSON image pushes keep\ntheir existing fallback when a registry can't resolve the pushed image's digest.\n\n## Run the model\n\nYou have several options for running a built image.\n\n### Docker\n\nRun the image directly with Docker.\nThis is the approach you'd use for production deployment.\n\n```shell\n# If your model uses a CPU:\ndocker run -d -p 5001:5000 my-model\n\n# If your model uses a GPU:\ndocker run -d -p 5001:5000 --gpus all my-model\n```\n\nThe server listens on port 5000 inside the container (mapped to 5001 above).\n\n### cog serve\n\nFor local development, `cog serve` builds the image and starts the server\nwith your project directory mounted in:\n\n```console\ncog serve\n```\n\nBy default the server runs on port 8393 and the container port is published on\n`127.0.0.1` (localhost), so it is only reachable from your local machine. The\nserver process inside the container binds to `0.0.0.0`; use `--host` to control\nwhich host interface the Docker port mapping is published on.\n\nUse `-p` to choose a different port:\n\n```console\ncog serve -p 5000\n```\n\n## Make a prediction\n\nOnce the server is running, make predictions by sending a POST request\nto the `/predictions` endpoint.\nInputs go inside an `\"input\"` object in the JSON body.\n\n> [!NOTE]\n> The examples below use `localhost:5001`, matching the Docker command above\n> (`-p 5001:5000`). If you used `cog serve`, use `localhost:8393` by default,\n> or the port you passed with `-p`.\n\n```console\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"input\": {\"prompt\": \"a photo of a cat\", \"steps\": 50}}'\n```\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"data:image/png;base64,...\",\n \"metrics\": {\n \"predict_time\": 4.52\n }\n}\n```\n\n> [!IMPORTANT]\n> Inputs **must** be wrapped in an `\"input\"` object.\n> `{\"input\": {\"scale\": 2.0}}` is correct; `{\"scale\": 2.0}` is not.\n\nTo discover what inputs your model accepts,\nview the OpenAPI schema:\n\n```console\ncurl http://localhost:5001/openapi.json\n```\n\n### Passing file inputs\n\nFile inputs (`cog.Path` or `cog.File` types) are passed as strings\ninside the `\"input\"` object.\nThere are two ways to do this:\n\n**1. HTTP/HTTPS URLs**\n\nPass a URL to a publicly accessible file.\nThe server downloads it inside the container:\n\n```console\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"input\": {\"image\": \"https://example.com/photo.jpg\"}}'\n```\n\n**2. Data URLs (base64)**\n\nTo pass a local file, encode it as a [data URL](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs):\n\n```bash\n# Construct a data URL from a local file\nDATA_URL=$(python3 -c \"\nimport base64, mimetypes\nwith open('input.jpg', 'rb') as f:\n data = base64.b64encode(f.read()).decode()\nmime = mimetypes.guess_type('input.jpg')[0] or 'application/octet-stream'\nprint(f'data:{mime};base64,{data}')\n\")\n\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d \"{\\\"input\\\": {\\\"image\\\": \\\"$DATA_URL\\\"}}\"\n```\n\n> [!NOTE]\n> The HTTP API only accepts JSON (`application/json`).\n> Multipart form uploads are not supported.\n> When you use `cog run -i image=@photo.jpg`,\n> the CLI handles the base64 encoding for you automatically.\n\n### Getting output files\n\nWhen a model returns a file output (`cog.Path` or `cog.File`),\nthe response contains a base64-encoded data URL by default:\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"data:image/png;base64,iVBORw0KGgo...\"\n}\n```\n\nTo have the server upload output files to external storage instead,\nstart the server with the `--upload-url` flag. The server then uploads each\nfile output to that URL prefix and returns the resulting URL in the response.\n\nWith `cog serve`:\n\n```console\ncog serve --upload-url https://example.com/upload/\n```\n\nWhen running the image directly with Docker, override the command to start the\nserver with `--upload-url`:\n\n```shell\ndocker run -d -p 5001:5000 my-model \\\n python -m cog.server.http --upload-url https://example.com/upload/\n```\n\nWith an upload URL configured, file outputs are uploaded and the response\ncontains the uploaded URL instead of a data URL:\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"https://example.com/upload/image.png\"\n}\n```\n\n## Run a one-off prediction\n\nThe Docker and `cog serve` commands above leave an HTTP server running.\nIf you instead want to run a single prediction and exit — without starting a\nserver — use `cog run`:\n\n```console\ncog run my-model -i image=@input.jpg\n```\n\nThis starts the container, runs one prediction, prints the result, and exits.\nFile inputs are passed with the `@` prefix (e.g. `-i image=@photo.jpg`),\nand the CLI handles base64 encoding for you.\n\n## Health checks\n\nThe server exposes a `GET /health-check` endpoint that returns the current status of the model container. Use this for readiness probes in orchestration systems like Kubernetes.\n\n```console\ncurl http://localhost:5001/health-check\n```\n\nThe response includes a `status` field with values like `STARTING`, `READY`, `BUSY`, `SETUP_FAILED`, or `DEFUNCT`. See the [HTTP API reference](http.md#get-health-check) for full details.\n\n## Stop the server\n\nIf you started the container with `docker run -d`, stop it with:\n\n```console\ndocker kill \n```\n\nIf you used `cog serve`, press `Ctrl+C` in the terminal.\n(`cog run` exits on its own once the prediction finishes, so there's nothing\nto stop.)\n\n## Concurrency\n\nBy default, the server processes one run at a time. To enable concurrent runs, make your `run()` method async and decorate it with `@cog.concurrent(max=N)`:\n\n```py\nimport cog\n\nclass Runner(cog.BaseRunner):\n @cog.concurrent(max=4)\n async def run(self) -> str:\n return \"hello world\"\n```\n\nThe deprecated [`concurrency.max`](yaml.md#concurrency) field in `cog.yaml` is still supported and takes precedence over the decorator by baking `COG_MAX_CONCURRENCY` into the image.\n\n## Environment variables\n\nYou can configure runtime behavior with environment variables:\n\n- `COG_SETUP_TIMEOUT`: Maximum time in seconds for the `setup()` method (default: no timeout).\n- `COG_MAX_CONCURRENCY`: Number of concurrent prediction slots (default: 1). Overrides both `@cog.concurrent` and deprecated `cog.yaml` concurrency.\n\nSee the [environment variables reference](environment.md) for the full list.\n\n## Next steps\n\n- [HTTP API reference](http.md) for full endpoint documentation\n- [Private registries](private-package-registry.md) for using private Python package registries\n- [`cog.yaml` reference](yaml.md) for configuration options\n\n\n---\n\n# Environment variables\n\nThis reference lists the public Cog-specific environment variables that change how Cog behaves.\n\n## Build-time variables\n\n### `COG_SDK_WHEEL`\n\nControls which Cog Python SDK wheel is installed in the Docker image during `cog build`. Takes precedence over `build.sdk_version` in `cog.yaml`.\n\n**Supported values:**\n\n| Value | Description |\n| -------------------- | ---------------------------------------------------- |\n| `pypi` | Install latest version from PyPI |\n| `pypi:0.12.0` | Install specific version from PyPI |\n| `dist` | Use wheel from `dist/` directory (requires git repo) |\n| `https://...` | Install from URL |\n| `/path/to/wheel.whl` | Install from local file path |\n\n**Default behaviour:**\n\n- Release builds install the latest Cog SDK from PyPI.\n- Development builds auto-detect a wheel in `dist/`, then fall back to the latest Cog SDK from PyPI.\n\n```console\n$ COG_SDK_WHEEL=pypi:0.11.0 cog build\n$ COG_SDK_WHEEL=dist cog build\n$ COG_SDK_WHEEL=https://example.com/cog-0.12.0-py3-none-any.whl cog build\n```\n\nThe `dist` option searches for wheels in:\n\n1. `./dist/` (current directory)\n2. `$REPO_ROOT/dist/` (if `REPO_ROOT` is set)\n3. `/dist/` (via `git rev-parse`, useful when running from subdirectories)\n\n### `COGLET_WHEEL`\n\nControls which coglet wheel is installed in the Docker image. Coglet is the Rust-based inference server.\n\n**Supported values:** Same as `COG_SDK_WHEEL`.\n\n**Default behaviour:** For development builds, auto-detects a wheel in `dist/`. For release builds, installs the latest version from PyPI.\n\n```console\n$ COGLET_WHEEL=dist cog build\n$ COGLET_WHEEL=pypi:0.1.0 cog build\n```\n\n### `COG_CA_CERT`\n\nInjects a custom CA certificate into the Docker image during `cog build`. This is useful when building behind a corporate proxy or VPN that uses custom certificate authorities (for example, Cloudflare WARP).\n\n**Supported values:**\n\n| Value | Description |\n| -------------------------------- | ----------------------------------------------------------- |\n| `/path/to/cert.crt` | Path to a single PEM certificate file |\n| `/path/to/certs/` | Directory of `.crt` and `.pem` files (all are concatenated) |\n| `-----BEGIN CERTIFICATE-----...` | Inline PEM certificate |\n| `LS0tLS1CRUdJTi...` | Base64-encoded PEM certificate |\n\nThe certificate is installed into the system CA store and the `SSL_CERT_FILE` and `REQUESTS_CA_BUNDLE` environment variables are set automatically in the built image.\n\n```console\n$ COG_CA_CERT=/usr/local/share/ca-certificates/corporate-ca.crt cog build\n$ COG_CA_CERT=/etc/custom-certs/ cog build\n$ COG_CA_CERT=\"$(cat /path/to/cert.pem)\" cog build\n```\n\n### `COG_OPENAPI_SCHEMA`\n\nUses a pre-built OpenAPI schema instead of generating one from the configured predict or train reference.\n\nThe value must be a path to a JSON schema file. Cog reads that file during schema generation and embeds it in the built image.\n\n```console\n$ COG_OPENAPI_SCHEMA=./openapi.json cog build\n```\n\n## CLI and local cache variables\n\n### `COG_NO_UPDATE_CHECK`\n\nDisables Cog's automatic update check. Set it to any non-empty value.\n\n```console\n$ COG_NO_UPDATE_CHECK=1 cog build\n```\n\n### `COG_NO_COLOR`\n\nDisables coloured CLI output. Set it to any non-empty value.\n\nCog also honours the standard `NO_COLOR` environment variable.\n\n```console\n$ COG_NO_COLOR=1 cog predict -i prompt=\"hello\"\n```\n\n### `COG_SKIP_DOCKER_CHECK`\n\nSkips the `cog doctor` Docker environment check. Set it to any non-empty value.\n\n```console\n$ COG_SKIP_DOCKER_CHECK=1 cog doctor\n```\n\n### `COG_SKIP_GPU_CHECK`\n\nSkips the `cog doctor` GPU compatibility check, which warns when the resolved torch/CUDA\nversions ship no kernels for the GPU in the current machine. Set it to any non-empty value.\n\nUse this when you are deliberately building for a different GPU than the one in this\nmachine, so the check should not warn about the local device.\n\n```console\n$ COG_SKIP_GPU_CHECK=1 cog doctor\n```\n\n### `COG_CACHE_DIR`\n\nOverrides Cog's local cache root.\n\nCog currently uses this cache for the content-addressed weights store. If unset, Cog uses `$XDG_CACHE_HOME/cog` when `XDG_CACHE_HOME` is set, otherwise `$HOME/.cache/cog`.\n\n```console\n$ COG_CACHE_DIR=/mnt/fast-cache cog weights pull\n```\n\n## Model reference and registry variables\n\n### `COG_MODEL`\n\nOverrides the full model reference used by commands that need a model destination, such as `cog push` and weights commands.\n\nThe value is parsed as a complete model reference (`registry/repo`, `registry/repo:tag`, or `registry/repo@digest`). If no tag is supplied, Cog generates a timestamp tag.\n\nWhen `COG_MODEL` is set, it takes precedence over `COG_MODEL_REGISTRY`, `COG_MODEL_REPO`, and `COG_MODEL_TAG`. A positional target passed to `cog push` takes precedence over all four variables. Although commands that inspect existing artifacts accept a digest, `cog push` rejects digest-pinned destinations because a push requires a tag.\n\n```console\n$ COG_MODEL=r8.im/acme/my-model:v1 cog push\n```\n\n### `COG_MODEL_REGISTRY`\n\nOverrides only the registry host of the model reference.\n\n```console\n$ COG_MODEL_REGISTRY=registry.example.com cog push\n```\n\n### `COG_MODEL_REPO`\n\nOverrides only the repository path of the model reference. The value must not include a registry host, tag, or digest.\n\n```console\n$ COG_MODEL_REPO=acme/my-model cog push\n```\n\n### `COG_MODEL_TAG`\n\nOverrides only the tag of the model reference.\n\nTags starting with `cog-` are reserved for tags that Cog generates internally and are rejected.\n\n```console\n$ COG_MODEL_TAG=staging cog push\n```\n\n### `COG_REGISTRY_HOST`\n\nChanges the default Replicate-compatible registry host used by commands such as `cog login`, base image resolution, and model reference resolution.\n\nThe default is `r8.im`.\n\n```console\n$ COG_REGISTRY_HOST=registry.example.com cog login\n```\n\n## Runtime server variables\n\nThese variables affect a running model server. Set them in `cog.yaml` under `environment`, pass them with `cog predict -e` or `cog serve -e`, or set them when running the built Docker image.\n\n### `COG_MAX_CONCURRENCY`\n\nControls how many predictions the model server can run concurrently. This overrides both `@cog.concurrent(max=N)` and the deprecated `concurrency.max` field in `cog.yaml`.\n\nBy default, Cog runs one prediction at a time unless the model uses `@cog.concurrent(max=N)`. Invalid values are ignored and the default of `1` is used.\n\nValues greater than `1` require an async `run()` method. This applies even when `COG_MAX_CONCURRENCY` is set as a runtime operator override.\n\nConcurrency is resolved in this order, from highest to lowest precedence:\n\n1. `COG_MAX_CONCURRENCY` set at runtime\n2. Deprecated `concurrency.max` in `cog.yaml`, which is baked into the image as `COG_MAX_CONCURRENCY`\n3. `@cog.concurrent(max=N)` on the async `run()` method\n4. Default: `1`\n\n```console\n$ COG_MAX_CONCURRENCY=4 docker run -p 5000:5000 my-model\n```\n\n### `COG_SETUP_TIMEOUT`\n\nControls the maximum time, in seconds, allowed for the model's `setup()` method to complete. If setup exceeds this timeout, the server reports setup failure.\n\nBy default, there is no timeout. Set to `0` to disable the timeout. Invalid values are ignored with a warning.\n\n```console\n$ COG_SETUP_TIMEOUT=300 docker run -p 5000:5000 my-model\n```\n\n### `COG_LOG_LEVEL`\n\nControls Coglet runtime log verbosity when `RUST_LOG` is not set.\n\nSupported values are `debug`, `info`, `warn`, `warning`, and `error`. The default is `info`.\n\n```console\n$ COG_LOG_LEVEL=debug docker run -p 5000:5000 my-model\n```\n\n### OpenTelemetry tracing\n\nTracing must first be enabled under `observability.traces` in `cog.yaml`. Runtime settings may disable an enabled image but cannot enable an image that did not opt in.\n\n| Variable | Purpose |\n| ----------------------------- | ---------------------------------------------- |\n| `COG_TRACE_ENABLED` | Set to `false` to disable tracing at runtime. |\n| `OTEL_SDK_DISABLED` | Hard-disable OpenTelemetry SDK initialization. |\n| `OTEL_EXPORTER_OTLP_ENDPOINT` | Collector endpoint for framework tracing. |\n| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf` or `grpc`. |\n| `OTEL_EXPORTER_OTLP_HEADERS` | Collector authentication headers. |\n| `OTEL_SERVICE_NAME` | Service name, default `cog`. |\n| `OTEL_TRACES_SAMPLER` | Runtime sampler override. |\n| `OTEL_TRACES_SAMPLER_ARG` | Ratio for ratio samplers. |\n\nWhen tracing is enabled without an endpoint, Cog logs one warning and continues without framework tracing. A custom Python provider configured by `observability.config` still runs and may use another exporter or no exporter. Delivery failures from Cog's built-in exporters never change prediction results; custom processors and exporters are model-owned code.\n\n`COG_OBSERVABILITY_*`, `COG_TRACE_*`, and `OTEL_*` are reserved from the general `cog.yaml` `environment` list. `COG_OBSERVABILITY_CONFIG` is internal and points to the validated file staged in the image. Supply supported `COG_TRACE_*` and `OTEL_*` settings to the running container instead.\n\n### `COG_THROTTLE_RESPONSE_INTERVAL`\n\nControls how often asynchronous webhook `output` and `logs` events are sent, in seconds.\n\nThe default is `0.5` seconds. Invalid values are ignored and the default is used. `start` and `completed` webhook events are always sent immediately.\n\n```console\n$ COG_THROTTLE_RESPONSE_INTERVAL=1 docker run -p 5000:5000 my-model\n```\n\n### `COG_STREAM_HISTORY_CAPACITY`\n\nControls how many server-sent event stream events are retained per prediction for replay when a client reconnects with `Accept: text/event-stream`.\n\nBy default, Cog retains the most recent 1024 events per prediction. Set to `0` to disable replay history while keeping live streaming enabled. Invalid values are ignored with a warning and the default is used.\n\n```console\n$ COG_STREAM_HISTORY_CAPACITY=0 docker run -p 5000:5000 my-model\n$ COG_STREAM_HISTORY_CAPACITY=4096 docker run -p 5000:5000 my-model\n```\n\n### `COG_WEIGHTS`\n\nProvides a weights path or URL to a model whose `setup()` method accepts a `weights` parameter.\n\n```console\n$ cog run -e COG_WEIGHTS=https://example.com/weights.tar -i prompt=\"hello\"\n```\n\n### `COG_USER_AGENT`\n\nSets the `User-Agent` header used by Cog when downloading URL-backed `File` inputs.\n\n```console\n$ COG_USER_AGENT=\"my-service/1.0\" docker run -p 5000:5000 my-model\n```\n\n## Push tuning variables\n\n### `COG_PUSH_OCI`\n\nEnables Cog's OCI chunked push path for container image layers when set to `1`. If the OCI push fails with a non-fatal error, Cog falls back to Docker's native push path.\n\n```console\n$ COG_PUSH_OCI=1 cog push\n```\n\n### `COG_PUSH_CONCURRENCY`\n\nControls how many image layers or weight blobs Cog uploads concurrently during push operations.\n\nThe default is `5`. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_CONCURRENCY=2 cog push\n```\n\n### `COG_PUSH_DEFAULT_CHUNK_SIZE`\n\nSets the default multipart upload chunk size, in bytes, when the registry does not advertise a maximum chunk size.\n\nThe default is 96 MiB. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_DEFAULT_CHUNK_SIZE=67108864 cog push\n```\n\n### `COG_PUSH_MULTIPART_THRESHOLD`\n\nSets the minimum blob size, in bytes, before Cog uses multipart upload.\n\nThe default is 128 MiB. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_MULTIPART_THRESHOLD=268435456 cog push\n```\n\n\n---\n\n# Examples\n\nCog's example models now live in the main Cog repository under\n[`examples/`](https://github.com/replicate/cog/tree/main/examples).\n\nIf you want a working project to copy from, start with one of these:\n\n- [`hello-world`](https://github.com/replicate/cog/tree/main/examples/hello-world):\n minimal string input/output example\n- [`hello-image`](https://github.com/replicate/cog/tree/main/examples/hello-image):\n returns a static image\n- [`hello-context`](https://github.com/replicate/cog/tree/main/examples/hello-context):\n shows how to read prediction context\n- [`hello-concurrency`](https://github.com/replicate/cog/tree/main/examples/hello-concurrency):\n demonstrates the `@cog.concurrent` decorator\n- [`hello-train`](https://github.com/replicate/cog/tree/main/examples/hello-train):\n defines a training interface\n- [`hello-replicate`](https://github.com/replicate/cog/tree/main/examples/hello-replicate):\n uses `cog.Secret` inputs and calls the Replicate API\n- [`blur`](https://github.com/replicate/cog/tree/main/examples/blur):\n simple image-to-image transformation\n- [`resnet`](https://github.com/replicate/cog/tree/main/examples/resnet):\n GPU image classification with ResNet50\n- [`streaming-text`](https://github.com/replicate/cog/tree/main/examples/streaming-text):\n streaming text generation with server-sent events\n- [`z-image-turbo`](https://github.com/replicate/cog/tree/main/examples/z-image-turbo):\n a generative image model example\n- [`notebook`](https://github.com/replicate/cog/tree/main/examples/notebook):\n runs a Jupyter notebook with Cog\n- [`canary`](https://github.com/replicate/cog/tree/main/examples/canary):\n small streaming output example used for validation\n- [`managed-weights`](https://github.com/replicate/cog/tree/main/examples/managed-weights):\n managed weights pipeline fixture\n\nThere is also an experimental example:\n\n- [`experimental/resnet-managed-weights`](https://github.com/replicate/cog/tree/main/examples/experimental/resnet-managed-weights):\n ResNet50 packaged with managed weights\n\nFor a quick summary of every example in the tree, see\n[`examples/README.md`](https://github.com/replicate/cog/blob/main/examples/README.md).\n\n\n---\n\n# Getting started with your own model\n\nThis guide will show you how to put your own machine learning model in a Docker image using Cog. If you haven't got a model to try out, you'll want to follow the [main getting started guide](getting-started.md).\n\n## Prerequisites\n\n- **Cog**. If you haven't already installed Cog, follow the [install instructions in the README](../README.md#install).\n- **macOS, Linux, or Windows 11**. Cog works on macOS and Linux. It also works on Windows 11 with [WSL 2](wsl2/wsl2.md).\n- **Docker**. Cog uses Docker to create a container for your model. You'll","tools/test-registry-util/README.md":"# test-registry-util Documentation\n\nPath: `tools/test-registry-util/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-registry-util/README.md)","tools/test-harness/fixtures/models/scalar-types/cog.yaml":"# Scalar-types Skill\n\nPath: `tools/test-harness/fixtures/models/scalar-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/scalar-types/cog.yaml)","tools/test-harness/fixtures/models/optional-types/cog.yaml":"# Optional-types Skill\n\nPath: `tools/test-harness/fixtures/models/optional-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-types/cog.yaml)","tools/test-harness/fixtures/models/optional-list-types/cog.yaml":"# Optional-list-types Skill\n\nPath: `tools/test-harness/fixtures/models/optional-list-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-list-types/cog.yaml)","tools/test-harness/fixtures/models/list-types/cog.yaml":"# List-types Skill\n\nPath: `tools/test-harness/fixtures/models/list-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/list-types/cog.yaml)","tools/test-harness/fixtures/models/file-path-types/cog.yaml":"# File-path-types Skill\n\nPath: `tools/test-harness/fixtures/models/file-path-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/file-path-types/cog.yaml)","tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml":"# Constraints-and-choices Skill\n\nPath: `tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml)","tools/test-harness/fixtures/models/complex-output/cog.yaml":"# Complex-output Skill\n\nPath: `tools/test-harness/fixtures/models/complex-output/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/complex-output/cog.yaml)"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/AGENTS.md","title":"AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"# AGENTS.md\n\nThis file provides guidance to coding agents when working with code in this repository.\n\n## Project Overview\n\nCog is a tool that packages machine learning models in production-ready containers.\n\nIt consists of:\n\n- **Cog CLI** (`cmd/cog/`) - Command-line interface for building, running, and deploying models, written in Go\n- **Python SDK** (`python/cog/`) - Python library for defining model predictors and training in Python\n- **Coglet** (`crates/`) - Rust-based prediction server that runs inside containers, with Python bindings via PyO3\n\nDocumentation for the CLI and SDK is available by reading ./docs/llms.txt.\n\n## Development Commands\n\nDevelopment tasks are managed with [mise](https://mise.jdx.dev/). Run `mise tasks` to see all available tasks.\n\n### Quick Reference\n\n| Task | Description |\n| ----------------------------- | ------------------------------------------------------------ |\n| `mise run fmt` | Check formatting (all languages) |\n| `mise run fmt:fix` | Fix formatting (all languages) |\n| `mise run lint` | Run linters (all languages) |\n| `mise run lint:fix` | Fix lint issues (all languages) |\n| `mise run test:go` | Run Go tests |\n| `mise run test:rust` | Run Rust tests |\n| `mise run test:python` | Run Python tests |\n| `mise run test:integration` | Run integration tests |\n| `mise run build:cog` | Build cog CLI binary |\n| `mise run build:coglet` | Build coglet wheel (dev) |\n| `mise run build:sdk` | Build SDK wheel |\n| `mise run install` | Build and symlink cog to /usr/local/bin |\n| `mise run docs:llm` | **IMPORTANT:** Regenerate `docs/llms.txt` after editing docs |\n| `mise run docs:cli` | Generate CLI reference docs from Go source code |\n| `mise run version` | Show current version from VERSION.txt |\n| `mise run version:bump ` | Bump version everywhere and commit |\n\n### Task Naming Convention\n\nTasks follow a consistent naming pattern:\n\n- **Language-based tasks** for fmt/lint/test/typecheck: `task:go`, `task:rust`, `task:python`\n- **Component-based tasks** for build: `build:cog`, `build:coglet`, `build:sdk`\n- **Check vs Fix**: `fmt` and `lint` default to check mode (non-destructive); use `:fix` suffix to auto-fix\n\n### All Tasks by Category\n\n**Format:**\n\n- `mise run fmt` / `mise run fmt:check` - Check all (alias)\n- `mise run fmt:fix` - Fix all\n- `mise run fmt:go` / `mise run fmt:rust` / `mise run fmt:python` - Per-language\n\n**Lint:**\n\n- `mise run lint` / `mise run lint:check` - Check all (alias)\n- `mise run lint:fix` - Fix all\n- `mise run lint:go` / `mise run lint:rust` / `mise run lint:python` - Per-language\n- `mise run lint:rust:deny` - Check Rust licenses/advisories\n\n**Test:**\n\n- `mise run test:go` - Go unit tests\n- `mise run test:rust` - Rust unit tests\n- `mise run test:python` - Python unit tests (via tox)\n- `mise run test:coglet:python` - Coglet Python binding tests\n- `mise run test:integration` - Integration tests\n\n**Build:**\n\n- `mise run build:cog` - Build cog CLI (development)\n- `mise run build:cog:release` - Build cog CLI (release)\n- `mise run build:coglet` - Build coglet wheel (dev install)\n- `mise run build:coglet:wheel` - Build coglet wheel (native platform)\n- `mise run build:coglet:wheel:linux-x64` - Build for Linux x86_64\n- `mise run build:coglet:wheel:linux-arm64` - Build for Linux ARM64\n- `mise run build:sdk` - Build SDK wheel\n\n**Install:**\n\n- `mise run install` - Symlink cog CLI to `/usr/local/bin` (requires `build:cog` first)\n- `PREFIX=/custom/path mise run install` - Symlink to custom location\n\n**Version:**\n\n- `mise run version` - Show current version from VERSION.txt\n- `mise run version:bump ` - Bump version everywhere and commit\n- `mise run version:check` - Verify VERSION.txt matches Cargo.toml\n\n**Other:**\n\n- `mise run typecheck` - Type check all languages\n- `mise run generate` - Run code generation\n- `mise run clean` - Clean all build artifacts\n- `mise run docs` - Build documentation\n- `mise run docs:serve` - Serve docs locally\n\n## Code Style Guidelines\n\n### Go\n\n- **Imports**: Organize in three groups separated by blank lines: (1) Standard library, (2) Third-party packages, (3) Internal packages (`github.com/replicate/cog/pkg/...`)\n- **Formatting**: Use `mise run fmt:go:fix`\n- **Linting**: Must pass golangci-lint with: errcheck, gocritic, gosec, govet, ineffassign, misspell, revive, staticcheck, unused\n- **Error Handling**: Return errors as values; use `pkg/errors.CodedError` for user-facing errors with error codes\n- **Naming**: CamelCase for exported, camelCase for unexported\n- **Testing**: Use `testify/require` for assertions; prefer table-driven tests\n\nExample import block:\n\n```go\nimport (\n \"fmt\"\n\n \"github.com/spf13/cobra\"\n\n \"github.com/replicate/cog/pkg/config\"\n)\n```\n\n### Python\n\n- **Imports**: Automatically organized by ruff/isort (stdlib → third-party → local)\n- **Formatting**: Use `mise run fmt:python:fix`\n- **Linting**: Must pass ruff checks: E (pycodestyle), F (Pyflakes), I (isort), W (warnings), S (bandit), B (bugbear), ANN (annotations)\n- **Type Annotations**: Required on all function signatures; use `typing_extensions` for compatibility; avoid `Any` where possible\n- **Error Handling**: Raise exceptions with descriptive messages; avoid generic exception catching\n- **Naming**: snake_case for functions/variables/modules, PascalCase for classes\n- **Testing**: Use pytest with fixtures; async tests with pytest-asyncio\n- **Compatibility**: Must support Python 3.10-3.13\n\n### Rust\n\n- **Formatting**: Use `mise run fmt:rust:fix`\n- **Linting**: Must pass `mise run lint:rust` (clippy)\n- **Dependencies**: Audited with `cargo-deny` (see `crates/deny.toml`); run `mise run lint:rust:deny`\n- **Error Handling**: Use `thiserror` for typed errors, `anyhow` for application errors\n- **Naming**: snake_case for functions/variables, PascalCase for types\n- **Testing**: Use `cargo test`; snapshot tests use `insta`\n- **Async**: tokio runtime; async/await patterns\n\n## Working on the CLI and support tooling\n\nThe CLI code is in the `cmd/cog/` and `pkg/` directories. Support tooling is in the `tools/` directory.\n\nThe main commands for working on the CLI are:\n\n- `go run ./cmd/cog` - Runs the Cog CLI directly from source (requires wheel to be built first)\n- `mise run build:cog` - Builds the Cog CLI binary\n- `mise run install` - Symlinks the built binary to `/usr/local/bin` (run `build:cog` first), or to a custom path with `PREFIX=/custom/path mise run install`\n- `mise run test:go` - Runs all Go unit tests\n- `go test ./pkg/...` - Runs tests directly with `go test`\n\n## Working on the Python SDK\n\nThe Python SDK is developed in the `python/cog/` directory. It uses `uv` for virtual environments and `tox` for testing across multiple Python versions.\n\nThe main commands for working on the SDK are:\n\n- `mise run build:sdk` - Builds the Python wheel\n- `mise run test:python` - Runs Python tests across all supported versions\n\n## Working on Coglet (Rust)\n\nCoglet is the Rust-based prediction server that runs inside Cog containers, handling HTTP requests, worker process management, and prediction execution.\n\nThe code is in the `crates/` directory:\n\n- `crates/coglet/` - Core Rust library (HTTP server, worker orchestration, IPC)\n- `crates/coglet-python/` - PyO3 bindings for Python predictor integration (requires Python 3.10+)\n\nFor detailed architecture documentation, see `crates/README.md` and `crates/coglet/README.md`.\n\nThe main commands for working on Coglet are:\n\n- `mise run build:coglet` - Build and install coglet wheel for development (macOS, for local Rust/Python tests)\n- `mise run build:coglet:wheel:linux-x64` - Build Linux x86_64 wheel (required to test Rust changes in Docker containers via `cog predict`/`cog train`)\n- `mise run test:rust` - Run Rust unit tests\n- `mise run lint:rust` - Run clippy linter\n- `mise run fmt:rust:fix` - Format code\n\n### Testing\n\nGo code is tested using the built-in `go test` framework:\n\n- `go test ./pkg/... -run ` - Runs specific Go tests by name\n- `mise run test:go` - Runs all Go unit tests\n\nPython code is tested using `tox`, which allows testing across multiple Python versions and configurations:\n\n- `mise run test:python` - Runs all Python unit tests\n- `uv run tox -e py312-tests -- python/tests/server/test_http.py::test_openapi_specification_with_yield` - Runs a specific Python test\n\nThe integration test suite in `integration-tests/` tests the end-to-end functionality of the Cog CLI and Python SDK using Go's testscript framework:\n\n- `mise run test:integration` - Runs the integration tests\n- `mise run test:integration string_predictor` - Runs a specific integration test\n\nThe integration tests require a built Cog binary, which defaults to the first `cog` in `PATH`. Run tests against a specific binary with the `COG_BINARY` environment variable:\n\n```bash\nmise run build:cog\nCOG_BINARY=dist/go/*/cog mise run test:integration\n```\n\n### Development Workflow\n\n1. Run `mise install` to set up the development environment\n2. Run `mise run build:sdk` after making changes to the `./python` directory\n3. Run `mise run build:coglet:wheel:linux-x64` after making changes to the `./crates` directory (needed for Docker testing)\n4. Run `mise run build:cog` to build the CLI (wheels are picked up from `dist/` at Docker build time, not embedded in the binary)\n5. Run `mise run fmt:fix` to format code\n6. Run `mise run lint` to check code quality\n7. Run `mise run docs:llm` to regenerate `docs/llms.txt` after changing `README.md` or any `docs/*.md` file\n8. Read the `./docs` directory and make sure the documentation is up to date\n\n**IMPORTANT:** Always run `mise run lint` (or the language-specific variant, e.g. `mise run lint:go`) before committing to catch linter errors early. CI will reject PRs that fail lint checks.\n\n## Architecture\n\n### CLI Architecture (Go)\n\nThe CLI follows a command pattern with subcommands. The main components are:\n\n- `pkg/cli/` - Command definitions (build, exec, predict, serve, etc.)\n- `pkg/docker/` - Docker client and container management\n- `pkg/dockerfile/` - Dockerfile generation and templating\n- `pkg/config/` - cog.yaml parsing and validation\n- `pkg/image/` - Image building and pushing logic\n\n### Python SDK Architecture\n\n- `python/cog/` - Core SDK\n - `base_predictor.py` - Base class for model predictors\n - `types.py` - Input/output type definitions\n - `server/` - HTTP/queue server implementation\n - `command/` - Runner implementations for predict/train\n\n### Coglet Architecture (Rust)\n\nThe prediction server that runs inside Cog containers. Uses a two-process architecture: a parent process (HTTP server + orchestrator) and a worker subprocess (Python predictor execution).\n\nSee `crates/README.md` for detailed architecture documentation.\n\n- `crates/coglet/` - Core Rust library (HTTP server, worker orchestration, IPC bridge)\n- `crates/coglet-python/` - PyO3 bindings for Python predictor integration\n\n### Key Design Patterns\n\n1. **Local Wheel Resolution**: The CLI discovers SDK and coglet wheels from `dist/` at Docker build time (not embedded in the binary)\n2. **Docker SDK Integration**: Uses Docker Go SDK for container operations\n3. **Type Safety**: Dataclasses for Python type validation, strongly typed Go interfaces\n4. **Compatibility Matrix**: Automated CUDA/PyTorch/TensorFlow compatibility management\n\nFor comprehensive architecture documentation, see [`architecture/`](./architecture/00-overview.md).\n\n## Common Tasks\n\n### Adding a new CLI command\n\n1. Create command file in `pkg/cli/`\n2. Add command to `pkg/cli/root.go`\n3. Implement business logic in appropriate `pkg/` subdirectory\n4. Add tests\n\n### Modifying Python SDK behavior\n\n1. Edit files in `python/cog/`\n2. Run `mise run build:sdk` to rebuild wheel\n3. Test with `mise run test:python`\n4. Integration test with `mise run test:integration`\n\n### Updating ML framework compatibility\n\n1. See `tools/compatgen/` for compatibility matrix generation\n2. Update framework versions in relevant Dockerfile templates\n3. Test with various framework combinations\n\n### Updating the docs\n\n- Documentation is in the `docs/` directory, written in Markdown and generated into HTML using `mkdocs`.\n- **IMPORTANT:** After editing any file in `docs/` or `README.md`, you MUST run `mise run docs:llm` to regenerate `docs/llms.txt`. This file is used by coding agents and should be kept in sync with the documentation.\n- **IMPORTANT:** CLI reference docs (`docs/cli.md`) are auto-generated from Go source code. After modifying CLI commands in `cmd/` or `pkg/cli/`, run `mise run docs:cli` to regenerate, and ensure `mise run docs:cli:check` passes before committing.\n\n## CI Tool Dependencies\n\nDevelopment tools are managed in **one place**: **`mise.toml`**. CI workflows use\n`jdx/mise-action@v4` (with caching enabled) to install the same tool versions that\ndevelopers use locally. This eliminates version drift between local dev and CI.\n\n**Exceptions:**\n\n| Tool | CI method | Why |\n| -------------------------- | ---------------------- | ---------------------------------------------------------------------- |\n| coglet wheel (maturin+zig) | `PyO3/maturin-action` | Builds inside a manylinux container with bundled maturin and zig |\n| Rust target cache | `Swatinem/rust-cache` | Caches `crates/target/` directory (separate from tool caching) |\n| `check-stubs` Python | `actions/setup-python` | Needs shared-library Python (libpython3.x.so) for PyO3 auto-initialize |\n\n**When updating a tool version**, update `mise.toml` only. CI picks it up automatically.\n\n## Agent Skills\n\nThe `.agents/skills/` directory contains shared skill definitions that provide specialized instructions and workflows for coding agents. Each skill is a subdirectory with a `SKILL.md` file that agents load on demand when the task matches.\n\nSkills are invoked automatically by agents when a matching task is detected. See individual `SKILL.md` files for details on when and how each skill applies.\n\n## Important Files\n\n- `VERSION.txt` - Canonical version (single source of truth)\n- `cog.yaml` - User-facing model configuration\n- `pkg/config/config.go` - Go code for parsing and validating `cog.yaml`\n- `pkg/config/data/config_schema_v1.0.json` - JSON schema for `cog.yaml`\n- `python/cog/base_predictor.py` - Predictor interface\n- `crates/Cargo.toml` - Rust workspace configuration (version must match VERSION.txt)\n- `crates/README.md` - Coglet architecture overview\n- `mise.toml` - Task definitions for development workflow\n\n## Testing Philosophy\n\n- Unit tests for individual components (Go and Python)\n- Integration tests for end-to-end workflows\n- Tests use real Docker operations (no mocking Docker API)\n- Always run `mise run build:sdk` after making Python changes before testing Go code\n- Python 3.10-3.13 compatibility is required\n\n### Go Test Conventions\n\nAll Go tests must use [testify](https://github.com/stretchr/testify) for assertions. Do **not** use raw `if` checks with `t.Fatal`/`t.Errorf` — use `require` and `assert` instead.\n\n- **`require`** — for fatal assertions that should stop the test (setup failures, preconditions):\n ```go\n require.NoError(t, err, \"failed to create client\")\n require.Equal(t, expected, actual)\n require.True(t, condition, \"server should be ready\")\n ```\n- **`assert`** — for non-fatal checks where the test should continue (e.g. validating multiple fields in a loop):\n ```go\n assert.Equal(t, http.StatusOK, resp.StatusCode)\n assert.Contains(t, output, \"expected substring\")\n assert.NoError(t, err, \"prediction %d failed\", i)\n ```\n- Use `require` for errors in setup/teardown and `assert` for the actual test expectations\n- Prefer specific assertions (`Equal`, `Contains`, `NoError`, `Len`, `Less`) over generic `True`/`False` — they produce better failure messages\n- Prefer table-driven tests for testing multiple similar cases\n","isInternal":false,"tokens":4014,"sizeBytes":16626},{"name":"llms.txt","path":"docs/llms.txt","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/docs/llms.txt","title":"docs - LLM Index & Context Digest","category":"root-instruction","format":"text","content":"# Cog: Containers for machine learning\n\nCog is an open-source tool that lets you package machine learning models in a standard, production-ready container.\n\nYou can deploy your packaged model to your own infrastructure, or to [Replicate](https://replicate.com/).\n\n## Highlights\n\n- 📦 **Docker containers without the pain.** Writing your own `Dockerfile` can be a bewildering process. With Cog, you define your environment with a [simple configuration file](#how-it-works) and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.\n\n- 🤬️ **No more CUDA hell.** Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.\n\n- ✅ **Define the inputs and outputs for your model with standard Python.** Then, Cog generates an OpenAPI schema and validates the inputs and outputs.\n\n- 🎁 **Automatic HTTP inference server**: Your model's types are used to dynamically generate a RESTful HTTP API using a high-performance Rust/Axum server.\n\n- 🚀 **Ready for production.** Deploy your model anywhere that Docker images run. Your own infrastructure, or [Replicate](https://replicate.com).\n\n## How it works\n\nDefine the Docker environment your model runs in with `cog.yaml`:\n\n```yaml\nbuild:\n gpu: true\n system_packages:\n - \"libgl1\"\n - \"libglib2.0-0\"\n python_version: \"3.13\"\n python_requirements: requirements.txt\nrun: \"run.py:Runner\"\n```\n\nDefine how your model runs with `run.py`:\n\n```python\nfrom cog import BaseRunner, Input, Path\nimport torch\n\nclass Runner(BaseRunner):\n def setup(self):\n \"\"\"Load the model into memory to make running multiple inferences efficient\"\"\"\n self.model = torch.load(\"./weights.pth\")\n\n # The arguments and types the model takes as input\n def run(self,\n image: Path = Input(description=\"Grayscale input image\")\n ) -> Path:\n \"\"\"Run the model\"\"\"\n processed_image = preprocess(image)\n output = self.model(processed_image)\n return postprocess(output)\n```\n\nIn the above we accept a path to the image as an input, and return a path to our transformed image after running it through our model.\n\nNow, you can run the model:\n\n```console\n$ cog run -i image=@input.jpg\n--> Building Docker image...\n--> Running...\n--> Output written to output.jpg\n```\n\nOr, build a Docker image for deployment:\n\n```console\n$ cog build -t my-classification-model\n--> Building Docker image...\n--> Built my-classification-model:latest\n\n$ docker run -d -p 5000:5000 --gpus all my-classification-model\n\n$ curl http://localhost:5000/predictions -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"image\": \"https://.../input.jpg\"}}'\n```\n\nOr, combine build and run via the `serve` command:\n\n```console\n$ cog serve -p 8080\n\n$ curl http://localhost:8080/predictions -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"image\": \"https://.../input.jpg\"}}'\n```\n\n\n\n## Why are we building this?\n\nIt's really hard for researchers to ship machine learning models to production.\n\nPart of the solution is Docker, but it is so complex to get it to work: Dockerfiles, pre-/post-processing, Flask servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.\n\n[Andreas](https://github.com/andreasjansson) and [Ben](https://github.com/bfirsh) created Cog. Andreas used to work at Spotify, where he built tools for building and deploying ML models with Docker. Ben worked at Docker, where he created [Docker Compose](https://github.com/docker/compose).\n\nWe realized that, in addition to Spotify, other companies were also using Docker to build and deploy machine learning models. [Uber](https://eng.uber.com/michelangelo-pyml/) and others have built similar systems. So, we're making an open source version so other people can do this too.\n\nHit us up if you're interested in using it or want to collaborate with us. [We're on Discord](https://discord.gg/replicate) or email us at [team@replicate.com](mailto:team@replicate.com).\n\n## Prerequisites\n\n- **macOS, Linux or Windows 11**. Cog works on macOS, Linux and Windows 11 with [WSL 2](docs/wsl2/wsl2.md)\n- **Docker**. Cog uses Docker to create a container for your model. You'll need to [install Docker](https://docs.docker.com/get-docker/) before you can run Cog. If you install Docker Engine instead of Docker Desktop, you will need to [install Buildx](https://docs.docker.com/build/architecture/#buildx) as well.\n\n## Install\n\nChoose your platform for installation instructions.\n\n
\n macOS\n\nThe easiest way to install Cog on macOS is with Homebrew:\n\n```console\nbrew install replicate/tap/cog\n```\n\nYou can also use the install script:\n\n```sh\n# bash, zsh, and other shells\nsh <(curl -fsSL https://cog.run/install.sh)\n\n# fish shell\nsh (curl -fsSL https://cog.run/install.sh | psub)\n\n# download with wget and run in a separate command\nwget -qO- https://cog.run/install.sh\nsh ./install.sh\n```\n\nOr install manually:\n\n```console\nsudo curl -o /usr/local/bin/cog -L \"https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')\"\nsudo chmod +x /usr/local/bin/cog\nsudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true\n```\n\nIf you see a Gatekeeper warning saying the binary \"cannot be opened because the developer cannot be verified\", run:\n\n```console\nsudo xattr -d com.apple.quarantine /usr/local/bin/cog\n```\n\n
\n\n
\n Linux\n\nYou can install Cog using the install script:\n\n```sh\n# bash, zsh, and other shells\nsh <(curl -fsSL https://cog.run/install.sh)\n\n# fish shell\nsh (curl -fsSL https://cog.run/install.sh | psub)\n\n# download with wget and run in a separate command\nwget -qO- https://cog.run/install.sh\nsh ./install.sh\n```\n\nOr install manually:\n\n```console\nsudo curl -o /usr/local/bin/cog -L \"https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')\"\nsudo chmod +x /usr/local/bin/cog\n```\n\n
\n\n
\n Windows\n\nCog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above.\n\n
\n\n
\n Docker\n\nTo install Cog inside a Docker image:\n\n```dockerfile\nRUN sh -c \"INSTALL_DIR=\\\"/usr/local/bin\\\" SUDO=\\\"\\\" $(curl -fsSL https://cog.run/install.sh)\"\n```\n\n
\n\n## Upgrade\n\nIf you're using macOS and you previously installed Cog with Homebrew, run the following:\n\n```console\nbrew upgrade replicate/tap/cog\n```\n\nOtherwise, you can upgrade to the latest version by running the same commands you used to install it.\n\n## Development\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for how to set up a development environment and build from source.\n\n## Next steps\n\n- [Get started with an example model](docs/getting-started.md)\n- [Get started with your own model](docs/getting-started-own-model.md)\n- [Using Cog with notebooks](docs/notebooks.md)\n- [Using Cog with Windows 11](docs/wsl2/wsl2.md)\n- [Browse the example models in this repo](docs/examples.md)\n- [Deploy models with Cog](docs/deploy.md)\n- [`cog.yaml` reference](docs/yaml.md) to learn how to define your model's environment\n- [Run interface reference](docs/python.md) to learn how the `Runner` interface works\n- [Training interface reference](docs/training.md) to learn how to add a fine-tuning API to your model\n- [HTTP API reference](docs/http.md) to learn how to use the HTTP API that models serve\n\n## Need help?\n\n[Join us in #cog on Discord.](https://discord.gg/replicate)\n\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/replicate/cog)\n\n\n\n---\n\n# Base images\n\nCog builds your model into a Docker image. To speed up builds and reduce cold boot times, Cog uses **prebuilt base images** by default. These images contain the common dependencies that most models need, so Cog doesn't have to install them from scratch every time you build or push.\n\n## What is a Cog base image?\n\nA Cog base image is a Docker image maintained by Replicate that includes:\n\n- **Python runtime** — the Python version specified in your `cog.yaml`.\n- **System libraries** — common dependencies for machine learning and media processing, including `ffmpeg`, `git`, `curl`, `build-essential`, `cmake`, OpenCV libraries, audio libraries (`sox`, `libsndfile1`), and graphics libraries (`libgl1`, `libsm6`, `libxext6`).\n- **CUDA and PyTorch** (GPU images only) — the appropriate CUDA toolkit and PyTorch stack for your configuration.\n\nBase images are tagged by their configuration, for example:\n\n```\nr8.im/cog-base:cuda12.4-python3.13-torch2.6.0\nr8.im/cog-base:python3.13\n```\n\nWhen you run `cog build` or `cog push`, Cog selects the base image that matches your Python version, CUDA version, and PyTorch version. Because these images are pre-pulled on Replicate's infrastructure, models built on top of them start faster.\n\nThe entire list of available base images can be viewed at [https://r8.im/v2/cog-base/tags/list](https://r8.im/v2/cog-base/tags/list).\n\n## Using the Cog base image\n\nThe `--use-cog-base-image` flag controls whether Cog uses a prebuilt base image. It is **enabled by default** on the following commands:\n\n- [`cog build`](cli.md#cog-build)\n- [`cog push`](cli.md#cog-push)\n- [`cog run`](cli.md#cog-run)\n- [`cog serve`](cli.md#cog-serve)\n- [`cog exec`](cli.md#cog-exec)\n\nSince it's on by default, you don't need to pass any flags:\n\n```bash\ncog push r8.im/your-username/my-model\n```\n\nThis builds and pushes your model using a prebuilt Cog base image for faster cold boots.\n\n## Disabling the Cog base image\n\nIf you encounter build issues or need a custom base layer, you can disable the Cog base image:\n\n```bash\ncog build --use-cog-base-image=false\n```\n\nWhen disabled, Cog generates a Dockerfile from scratch using either an NVIDIA CUDA base image or a slim Python base image, depending on the `--use-cuda-base-image` flag.\n\n## Relationship to `--use-cuda-base-image`\n\nThe `--use-cuda-base-image` flag controls which underlying base image Cog uses **when the Cog base image is disabled**. It has no effect when `--use-cog-base-image` is enabled (the default), because the Cog base image already includes the appropriate CUDA and Python stack.\n\nWhen you disable the Cog base image with `--use-cog-base-image=false`, Cog chooses the base image automatically:\n\n- **GPU models** (`gpu: true` in `cog.yaml`): uses an NVIDIA CUDA base image.\n- **CPU models**: uses a slim Python base image.\n\nThese flags — along with `--dockerfile` — are **mutually exclusive**: you can only set one of them explicitly on a given command. To customize the base image, disable the Cog base image and let Cog choose between CUDA and Python automatically.\n\n## Troubleshooting\n\nIf `cog build` or `cog push` fails with the Cog base image enabled, try disabling it:\n\n```bash\ncog push --use-cog-base-image=false r8.im/your-username/my-model\n```\n\nThis falls back to building from a standard CUDA or Python base image, which can help diagnose whether the issue is with the base image or your model's configuration.\n\n\n---\n\n# CLI reference\n\n\n\n## `cog`\n\nContainers for machine learning.\n\nTo get started, take a look at the documentation:\nhttps://github.com/replicate/cog\n\n**Examples**\n\n```\n To execute a command inside a Docker environment defined with Cog:\n $ cog exec echo hello world\n```\n\n**Options**\n\n```\n --debug Show debugging output\n -h, --help help for cog\n --no-color Disable colored output\n --version Show version of Cog\n```\n\n## `cog build`\n\nBuild a Docker image from the cog.yaml in the current directory.\n\nThe generated image contains your model code, dependencies, and the Cog\nruntime. It can be run locally with 'cog run' or pushed to a registry\nwith 'cog push'.\n\n```\ncog build [flags]\n```\n\n**Examples**\n\n```\n # Build with default settings\n cog build\n\n # Build and tag the image\n cog build -t my-model:latest\n\n # Build without using the cache\n cog build --no-cache\n\n # Build with model weights in a separate layer\n cog build --separate-weights -t my-model:v1\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n -h, --help help for build\n --no-cache Do not use cache when building the image\n --openapi-schema string Load OpenAPI schema from a file\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'\n --separate-weights Separate model weights from code in image layers\n -t, --tag string A name for the built image in the form 'repository:tag'\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog doctor`\n\nDiagnose and fix common issues in your Cog project.\n\nNOTE: cog doctor is experimental. Behavior and checks may change in future versions.\n\nBy default, cog doctor reports problems without modifying any files.\nPass --fix to automatically apply safe fixes.\n\n```\ncog doctor [flags]\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --fix Automatically apply fixes\n -h, --help help for doctor\n```\n\n## `cog exec`\n\nExecute a command inside a Docker environment defined by cog.yaml.\n\nCog builds a temporary image from your cog.yaml configuration and runs the\ngiven command inside it. This is useful for debugging, running scripts, or\nexploring the environment your model will run in.\n\n```\ncog exec [arg...] [flags]\n```\n\n**Examples**\n\n```\n # Open a Python interpreter inside the model environment\n cog exec python\n\n # Run a script\n cog exec python train.py\n\n # Run with environment variables\n cog exec -e HUGGING_FACE_HUB_TOKEN=abc123 python download.py\n\n # Expose a port (e.g. for Jupyter)\n cog exec -p 8888 jupyter notebook\n```\n\n**Options**\n\n```\n -e, --env stringArray Environment variables, in the form name=value\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for exec\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n -p, --publish stringArray Publish a container's port to the host, e.g. -p 8000 or -p 0.0.0.0:8000\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog init`\n\nCreate a cog.yaml and run.py in the current directory.\n\nThese files provide a starting template for defining your model's environment\nand run interface. Edit them to match your model's requirements.\n\n```\ncog init [flags]\n```\n\n**Examples**\n\n```\n # Set up a new Cog project in the current directory\n cog init\n```\n\n**Options**\n\n```\n -h, --help help for init\n```\n\n## `cog login`\n\nLog in to a container registry.\n\nFor Replicate's registry (r8.im), this command handles authentication\nthrough Replicate's token-based flow.\n\nFor other registries, this command prompts for username and password,\nthen stores credentials using Docker's credential system.\n\n```\ncog login [flags]\n```\n\n**Options**\n\n```\n -h, --help help for login\n --token-stdin Pass login token on stdin instead of opening a browser. You can find your Replicate login token at https://replicate.com/auth/token\n```\n\n## `cog playground`\n\nOpen a browser playground for talking to a running model.\n\nStarts a local web server that serves a schema-driven UI (a Postman-like tool\nfor Cog models). Point it at any running Cog HTTP API -- for example one started\nwith 'cog serve' -- and the playground reflects that model's inputs and outputs\nfrom its OpenAPI schema in real time.\n\nRequests are reverse-proxied through this server, so the target API does not\nneed to set CORS headers. The server also hosts a webhook sink so async\npredictions can be observed in the browser.\n\nAsync/webhook testing against a containerized model requires the webhook URL to\nbe reachable from inside the container. On Docker Desktop the default\n'host.docker.internal' works once the server listens on a reachable interface\n(e.g. --host 0.0.0.0).\n\n```\ncog playground [flags]\n```\n\n**Examples**\n\n```\n # Start a model API in one terminal\n cog serve -p 8393\n\n # Open the playground pointing at it\n cog playground --target http://localhost:8393\n```\n\n**Options**\n\n```\n -h, --help help for playground\n --host string Address to bind (use 0.0.0.0 to receive webhooks from containers) (default \"127.0.0.1\")\n --no-open Do not open the browser automatically\n -p, --port int Port to listen on (0 picks a free port)\n --target string Default target model API URL (default \"http://localhost:8393\")\n --webhook-host string Hostname the model uses to reach this server for webhooks (default \"host.docker.internal\")\n```\n\n## `cog push`\n\nBuild from cog.yaml and push to an OCI-compliant registry. Run 'cog login'\nfirst when pushing to Replicate's registry (r8.im).\n\nTARGET overrides the configured destination and all COG_MODEL\\* environment\nvariables. It doesn't change the project format: projects configured with\n'image' push an image, while projects configured with 'model' push an OCI\nbundle. Push targets must use tags, not digests. Untagged image targets use\nDocker's default 'latest' tag; untagged bundle targets get a timestamp tag.\n\nWith --json, Cog writes one versioned JSON result to stdout after the entire\npush succeeds. Progress, warnings, and diagnostics continue on stderr. Every\nreference in the result is digest-pinned.\n\n```\ncog push [TARGET] [flags]\n```\n\n**Examples**\n\n```\n # Push an image to Replicate\n cog push r8.im/your-username/my-model\n\n # Push an image to any OCI registry\n cog push registry.example.com/your-username/model-name\n\n # Push a bundle project and print its immutable references as JSON\n cog push registry.example.com/your-username/model-name:v1 --json\n\n # Push with model weights in a separate image layer (Replicate only)\n cog push r8.im/your-username/my-model --separate-weights\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n -h, --help help for push\n --json Output the pushed references as JSON\n --no-cache Do not use cache when building the image\n --openapi-schema string Load OpenAPI schema from a file\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'\n --separate-weights Separate model weights from code in image layers\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog run`\n\nRun the model.\n\nIf 'image' is passed, it will run the model on that Docker image.\nIt must be an image that has been built by Cog.\n\nOtherwise, it will build the model in the current directory and run\nit.\n\n```\ncog run [image] [flags]\n```\n\n**Examples**\n\n```\n # Run the model with named inputs\n cog run -i prompt=\"a photo of a cat\"\n\n # Pass a file as input\n cog run -i image=@photo.jpg\n\n # Save output to a file\n cog run -i image=@input.jpg -o output.png\n\n # Pass multiple inputs\n cog run -i prompt=\"sunset\" -i width=1024 -i height=768\n\n # Run against a pre-built image\n cog run r8.im/your-username/my-model -i prompt=\"hello\"\n\n # Pass inputs as JSON\n echo '{\"prompt\": \"a cat\"}' | cog run --json @-\n```\n\n**Options**\n\n```\n -e, --env stringArray Environment variables, in the form name=value\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for run\n -i, --input stringArray Inputs, in the form name=value. if value is prefixed with @, then it is read from a file on disk. E.g. -i path=@image.jpg\n --json string Pass inputs as JSON object, read from file (@inputs.json) or via stdin (@-)\n -o, --output string Output path\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --setup-timeout uint32 The timeout for a container to setup (in seconds). (default 300)\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n --use-replicate-token Pass REPLICATE_API_TOKEN from local environment into the model context\n```\n\n## `cog serve`\n\nRun an HTTP server.\n\nBuilds the model and starts an HTTP server that exposes the model's inputs\nand outputs as a REST API. Compatible with the Cog HTTP protocol.\n\nBy default the container port is published on 127.0.0.1 (localhost), so the\nserver is only reachable from your local machine. The server process inside\nthe container binds to 0.0.0.0; use --host to control which host interface\nthe Docker port mapping is published on.\n\n```\ncog serve [flags]\n```\n\n**Examples**\n\n```\n # Start the server on the default port (8393)\n cog serve\n\n # Start on a custom port\n cog serve -p 5000\n\n # Start the playground alongside the model\n cog serve --playground\n\n # Listen on all interfaces (e.g. to expose to the network)\n cog serve --host 0.0.0.0\n\n # Test the server\n curl http://localhost:8393/predictions \\\n -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"prompt\": \"a cat\"}}'\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for serve\n --host string Host IP to publish the container port on. Use 0.0.0.0 to allow connections from other machines. (default \"127.0.0.1\")\n --playground Start the playground alongside the model\n --playground-port int Port for the playground (0 picks a free port) (default 9000)\n -p, --port int Port on which to listen (default 8393)\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --upload-url string Upload URL for file outputs (e.g. https://example.com/upload/)\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n\n---\n\n# Deploy models with Cog\n\nCog containers are Docker containers that serve an HTTP server\nfor running your model.\nYou can deploy them anywhere that Docker containers run.\n\nThe server inside Cog containers is **coglet**, a Rust-based inference server\nthat handles HTTP requests, worker process management, and run execution.\n\nThis guide assumes you have a model packaged with Cog.\nIf you don't, [follow our getting started guide](getting-started-own-model.md),\nor start from one of the [examples in the Cog repository](examples.md).\n\n## Build a Docker image\n\nBuild your model into a Docker image:\n\n```console\ncog build -t my-model\n```\n\nThe image contains your model code, dependencies, the Cog runtime, and everything in between.\nIt serves an HTTP server on port 5000 when run.\n\n## Push to a registry\n\n`cog push` builds the project and publishes it to an OCI-compliant registry:\n\n```console\ncog push registry.example.com/acme/my-model:v1\n```\n\nThe project configuration decides what Cog publishes. A project with `image` in\n`cog.yaml` publishes a container image. A project with `model` publishes an OCI\nbundle containing the image and any managed weights. Passing a target doesn't\nswitch between these formats.\n\nThe positional target is optional when `image` or `model` already supplies a\ndestination. When present, it overrides that configured destination and every\n`COG_MODEL*` environment variable. The target must be tag-addressable -- Cog\nrejects digest-pinned targets. An untagged image target uses Docker's `latest`\ntag, while an untagged bundle target gets a timestamp tag generated by Cog.\n\nFor a bundle with managed weights, use the same repository where `cog weights\nimport` published the weight manifests. The model tag may differ. If the\nrepository doesn't contain every required weight manifest, Cog fails before\npushing the image and tells you to run `cog weights import` for that repository.\n\nWithout `--json`, a successful push prints the published references as a\nhuman-readable tree on stderr.\n\n### Machine-readable output\n\nUse `--json` when another program needs the immutable references produced by a\npush. For example, this pushes a bundle and prints its model, image, and managed\nweight references:\n\n```console\ncog push registry.example.com/acme/my-model:v1 --json\n```\n\nVersion 1 output for a bundle with managed weights has this shape:\n\n```json\n{\n \"version\": 1,\n \"model\": \"registry.example.com/acme/my-model@sha256:1111111111111111111111111111111111111111111111111111111111111111\",\n \"image\": \"registry.example.com/acme/my-model@sha256:2222222222222222222222222222222222222222222222222222222222222222\",\n \"weights\": [\n {\n \"name\": \"transformer\",\n \"reference\": \"registry.example.com/acme/my-model@sha256:3333333333333333333333333333333333333333333333333333333333333333\"\n }\n ]\n}\n```\n\nThe version 1 fields are:\n\n- `version`: Always `1`.\n- `image`: Always present. The value is the published image's complete,\n digest-pinned reference.\n- `model`: Present only for bundle pushes. The value is the complete,\n digest-pinned bundle reference.\n- `weights`: Present only when the bundle has managed weights. Each entry has a\n `name` and a complete, digest-pinned `reference`. Entries keep their\n `cog.yaml` order.\n\nJSON whitespace and object key order aren't part of the contract.\n\nStdout stays empty while the command runs. Build and upload progress, warnings,\ndiagnostics, and registry-provider messages go to stderr. After the push and\nprovider post-processing both succeed, stdout receives exactly one JSON\ndocument. JSON mode suppresses the human-readable reference tree.\n\nAny build, push, digest-resolution, provider, or serialization failure exits\nnonzero without writing a JSON document to stdout. JSON mode requires Cog to\nresolve every published reference to an immutable digest, so it can report\nfailure even when an image was pushed successfully. Non-JSON image pushes keep\ntheir existing fallback when a registry can't resolve the pushed image's digest.\n\n## Run the model\n\nYou have several options for running a built image.\n\n### Docker\n\nRun the image directly with Docker.\nThis is the approach you'd use for production deployment.\n\n```shell\n# If your model uses a CPU:\ndocker run -d -p 5001:5000 my-model\n\n# If your model uses a GPU:\ndocker run -d -p 5001:5000 --gpus all my-model\n```\n\nThe server listens on port 5000 inside the container (mapped to 5001 above).\n\n### cog serve\n\nFor local development, `cog serve` builds the image and starts the server\nwith your project directory mounted in:\n\n```console\ncog serve\n```\n\nBy default the server runs on port 8393 and the container port is published on\n`127.0.0.1` (localhost), so it is only reachable from your local machine. The\nserver process inside the container binds to `0.0.0.0`; use `--host` to control\nwhich host interface the Docker port mapping is published on.\n\nUse `-p` to choose a different port:\n\n```console\ncog serve -p 5000\n```\n\n## Make a prediction\n\nOnce the server is running, make predictions by sending a POST request\nto the `/predictions` endpoint.\nInputs go inside an `\"input\"` object in the JSON body.\n\n> [!NOTE]\n> The examples below use `localhost:5001`, matching the Docker command above\n> (`-p 5001:5000`). If you used `cog serve`, use `localhost:8393` by default,\n> or the port you passed with `-p`.\n\n```console\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"input\": {\"prompt\": \"a photo of a cat\", \"steps\": 50}}'\n```\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"data:image/png;base64,...\",\n \"metrics\": {\n \"predict_time\": 4.52\n }\n}\n```\n\n> [!IMPORTANT]\n> Inputs **must** be wrapped in an `\"input\"` object.\n> `{\"input\": {\"scale\": 2.0}}` is correct; `{\"scale\": 2.0}` is not.\n\nTo discover what inputs your model accepts,\nview the OpenAPI schema:\n\n```console\ncurl http://localhost:5001/openapi.json\n```\n\n### Passing file inputs\n\nFile inputs (`cog.Path` or `cog.File` types) are passed as strings\ninside the `\"input\"` object.\nThere are two ways to do this:\n\n**1. HTTP/HTTPS URLs**\n\nPass a URL to a publicly accessible file.\nThe server downloads it inside the container:\n\n```console\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"input\": {\"image\": \"https://example.com/photo.jpg\"}}'\n```\n\n**2. Data URLs (base64)**\n\nTo pass a local file, encode it as a [data URL](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs):\n\n```bash\n# Construct a data URL from a local file\nDATA_URL=$(python3 -c \"\nimport base64, mimetypes\nwith open('input.jpg', 'rb') as f:\n data = base64.b64encode(f.read()).decode()\nmime = mimetypes.guess_type('input.jpg')[0] or 'application/octet-stream'\nprint(f'data:{mime};base64,{data}')\n\")\n\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d \"{\\\"input\\\": {\\\"image\\\": \\\"$DATA_URL\\\"}}\"\n```\n\n> [!NOTE]\n> The HTTP API only accepts JSON (`application/json`).\n> Multipart form uploads are not supported.\n> When you use `cog run -i image=@photo.jpg`,\n> the CLI handles the base64 encoding for you automatically.\n\n### Getting output files\n\nWhen a model returns a file output (`cog.Path` or `cog.File`),\nthe response contains a base64-encoded data URL by default:\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"data:image/png;base64,iVBORw0KGgo...\"\n}\n```\n\nTo have the server upload output files to external storage instead,\nstart the server with the `--upload-url` flag. The server then uploads each\nfile output to that URL prefix and returns the resulting URL in the response.\n\nWith `cog serve`:\n\n```console\ncog serve --upload-url https://example.com/upload/\n```\n\nWhen running the image directly with Docker, override the command to start the\nserver with `--upload-url`:\n\n```shell\ndocker run -d -p 5001:5000 my-model \\\n python -m cog.server.http --upload-url https://example.com/upload/\n```\n\nWith an upload URL configured, file outputs are uploaded and the response\ncontains the uploaded URL instead of a data URL:\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"https://example.com/upload/image.png\"\n}\n```\n\n## Run a one-off prediction\n\nThe Docker and `cog serve` commands above leave an HTTP server running.\nIf you instead want to run a single prediction and exit — without starting a\nserver — use `cog run`:\n\n```console\ncog run my-model -i image=@input.jpg\n```\n\nThis starts the container, runs one prediction, prints the result, and exits.\nFile inputs are passed with the `@` prefix (e.g. `-i image=@photo.jpg`),\nand the CLI handles base64 encoding for you.\n\n## Health checks\n\nThe server exposes a `GET /health-check` endpoint that returns the current status of the model container. Use this for readiness probes in orchestration systems like Kubernetes.\n\n```console\ncurl http://localhost:5001/health-check\n```\n\nThe response includes a `status` field with values like `STARTING`, `READY`, `BUSY`, `SETUP_FAILED`, or `DEFUNCT`. See the [HTTP API reference](http.md#get-health-check) for full details.\n\n## Stop the server\n\nIf you started the container with `docker run -d`, stop it with:\n\n```console\ndocker kill \n```\n\nIf you used `cog serve`, press `Ctrl+C` in the terminal.\n(`cog run` exits on its own once the prediction finishes, so there's nothing\nto stop.)\n\n## Concurrency\n\nBy default, the server processes one run at a time. To enable concurrent runs, make your `run()` method async and decorate it with `@cog.concurrent(max=N)`:\n\n```py\nimport cog\n\nclass Runner(cog.BaseRunner):\n @cog.concurrent(max=4)\n async def run(self) -> str:\n return \"hello world\"\n```\n\nThe deprecated [`concurrency.max`](yaml.md#concurrency) field in `cog.yaml` is still supported and takes precedence over the decorator by baking `COG_MAX_CONCURRENCY` into the image.\n\n## Environment variables\n\nYou can configure runtime behavior with environment variables:\n\n- `COG_SETUP_TIMEOUT`: Maximum time in seconds for the `setup()` method (default: no timeout).\n- `COG_MAX_CONCURRENCY`: Number of concurrent prediction slots (default: 1). Overrides both `@cog.concurrent` and deprecated `cog.yaml` concurrency.\n\nSee the [environment variables reference](environment.md) for the full list.\n\n## Next steps\n\n- [HTTP API reference](http.md) for full endpoint documentation\n- [Private registries](private-package-registry.md) for using private Python package registries\n- [`cog.yaml` reference](yaml.md) for configuration options\n\n\n---\n\n# Environment variables\n\nThis reference lists the public Cog-specific environment variables that change how Cog behaves.\n\n## Build-time variables\n\n### `COG_SDK_WHEEL`\n\nControls which Cog Python SDK wheel is installed in the Docker image during `cog build`. Takes precedence over `build.sdk_version` in `cog.yaml`.\n\n**Supported values:**\n\n| Value | Description |\n| -------------------- | ---------------------------------------------------- |\n| `pypi` | Install latest version from PyPI |\n| `pypi:0.12.0` | Install specific version from PyPI |\n| `dist` | Use wheel from `dist/` directory (requires git repo) |\n| `https://...` | Install from URL |\n| `/path/to/wheel.whl` | Install from local file path |\n\n**Default behaviour:**\n\n- Release builds install the latest Cog SDK from PyPI.\n- Development builds auto-detect a wheel in `dist/`, then fall back to the latest Cog SDK from PyPI.\n\n```console\n$ COG_SDK_WHEEL=pypi:0.11.0 cog build\n$ COG_SDK_WHEEL=dist cog build\n$ COG_SDK_WHEEL=https://example.com/cog-0.12.0-py3-none-any.whl cog build\n```\n\nThe `dist` option searches for wheels in:\n\n1. `./dist/` (current directory)\n2. `$REPO_ROOT/dist/` (if `REPO_ROOT` is set)\n3. `/dist/` (via `git rev-parse`, useful when running from subdirectories)\n\n### `COGLET_WHEEL`\n\nControls which coglet wheel is installed in the Docker image. Coglet is the Rust-based inference server.\n\n**Supported values:** Same as `COG_SDK_WHEEL`.\n\n**Default behaviour:** For development builds, auto-detects a wheel in `dist/`. For release builds, installs the latest version from PyPI.\n\n```console\n$ COGLET_WHEEL=dist cog build\n$ COGLET_WHEEL=pypi:0.1.0 cog build\n```\n\n### `COG_CA_CERT`\n\nInjects a custom CA certificate into the Docker image during `cog build`. This is useful when building behind a corporate proxy or VPN that uses custom certificate authorities (for example, Cloudflare WARP).\n\n**Supported values:**\n\n| Value | Description |\n| -------------------------------- | ----------------------------------------------------------- |\n| `/path/to/cert.crt` | Path to a single PEM certificate file |\n| `/path/to/certs/` | Directory of `.crt` and `.pem` files (all are concatenated) |\n| `-----BEGIN CERTIFICATE-----...` | Inline PEM certificate |\n| `LS0tLS1CRUdJTi...` | Base64-encoded PEM certificate |\n\nThe certificate is installed into the system CA store and the `SSL_CERT_FILE` and `REQUESTS_CA_BUNDLE` environment variables are set automatically in the built image.\n\n```console\n$ COG_CA_CERT=/usr/local/share/ca-certificates/corporate-ca.crt cog build\n$ COG_CA_CERT=/etc/custom-certs/ cog build\n$ COG_CA_CERT=\"$(cat /path/to/cert.pem)\" cog build\n```\n\n### `COG_OPENAPI_SCHEMA`\n\nUses a pre-built OpenAPI schema instead of generating one from the configured predict or train reference.\n\nThe value must be a path to a JSON schema file. Cog reads that file during schema generation and embeds it in the built image.\n\n```console\n$ COG_OPENAPI_SCHEMA=./openapi.json cog build\n```\n\n## CLI and local cache variables\n\n### `COG_NO_UPDATE_CHECK`\n\nDisables Cog's automatic update check. Set it to any non-empty value.\n\n```console\n$ COG_NO_UPDATE_CHECK=1 cog build\n```\n\n### `COG_NO_COLOR`\n\nDisables coloured CLI output. Set it to any non-empty value.\n\nCog also honours the standard `NO_COLOR` environment variable.\n\n```console\n$ COG_NO_COLOR=1 cog predict -i prompt=\"hello\"\n```\n\n### `COG_SKIP_DOCKER_CHECK`\n\nSkips the `cog doctor` Docker environment check. Set it to any non-empty value.\n\n```console\n$ COG_SKIP_DOCKER_CHECK=1 cog doctor\n```\n\n### `COG_SKIP_GPU_CHECK`\n\nSkips the `cog doctor` GPU compatibility check, which warns when the resolved torch/CUDA\nversions ship no kernels for the GPU in the current machine. Set it to any non-empty value.\n\nUse this when you are deliberately building for a different GPU than the one in this\nmachine, so the check should not warn about the local device.\n\n```console\n$ COG_SKIP_GPU_CHECK=1 cog doctor\n```\n\n### `COG_CACHE_DIR`\n\nOverrides Cog's local cache root.\n\nCog currently uses this cache for the content-addressed weights store. If unset, Cog uses `$XDG_CACHE_HOME/cog` when `XDG_CACHE_HOME` is set, otherwise `$HOME/.cache/cog`.\n\n```console\n$ COG_CACHE_DIR=/mnt/fast-cache cog weights pull\n```\n\n## Model reference and registry variables\n\n### `COG_MODEL`\n\nOverrides the full model reference used by commands that need a model destination, such as `cog push` and weights commands.\n\nThe value is parsed as a complete model reference (`registry/repo`, `registry/repo:tag`, or `registry/repo@digest`). If no tag is supplied, Cog generates a timestamp tag.\n\nWhen `COG_MODEL` is set, it takes precedence over `COG_MODEL_REGISTRY`, `COG_MODEL_REPO`, and `COG_MODEL_TAG`. A positional target passed to `cog push` takes precedence over all four variables. Although commands that inspect existing artifacts accept a digest, `cog push` rejects digest-pinned destinations because a push requires a tag.\n\n```console\n$ COG_MODEL=r8.im/acme/my-model:v1 cog push\n```\n\n### `COG_MODEL_REGISTRY`\n\nOverrides only the registry host of the model reference.\n\n```console\n$ COG_MODEL_REGISTRY=registry.example.com cog push\n```\n\n### `COG_MODEL_REPO`\n\nOverrides only the repository path of the model reference. The value must not include a registry host, tag, or digest.\n\n```console\n$ COG_MODEL_REPO=acme/my-model cog push\n```\n\n### `COG_MODEL_TAG`\n\nOverrides only the tag of the model reference.\n\nTags starting with `cog-` are reserved for tags that Cog generates internally and are rejected.\n\n```console\n$ COG_MODEL_TAG=staging cog push\n```\n\n### `COG_REGISTRY_HOST`\n\nChanges the default Replicate-compatible registry host used by commands such as `cog login`, base image resolution, and model reference resolution.\n\nThe default is `r8.im`.\n\n```console\n$ COG_REGISTRY_HOST=registry.example.com cog login\n```\n\n## Runtime server variables\n\nThese variables affect a running model server. Set them in `cog.yaml` under `environment`, pass them with `cog predict -e` or `cog serve -e`, or set them when running the built Docker image.\n\n### `COG_MAX_CONCURRENCY`\n\nControls how many predictions the model server can run concurrently. This overrides both `@cog.concurrent(max=N)` and the deprecated `concurrency.max` field in `cog.yaml`.\n\nBy default, Cog runs one prediction at a time unless the model uses `@cog.concurrent(max=N)`. Invalid values are ignored and the default of `1` is used.\n\nValues greater than `1` require an async `run()` method. This applies even when `COG_MAX_CONCURRENCY` is set as a runtime operator override.\n\nConcurrency is resolved in this order, from highest to lowest precedence:\n\n1. `COG_MAX_CONCURRENCY` set at runtime\n2. Deprecated `concurrency.max` in `cog.yaml`, which is baked into the image as `COG_MAX_CONCURRENCY`\n3. `@cog.concurrent(max=N)` on the async `run()` method\n4. Default: `1`\n\n```console\n$ COG_MAX_CONCURRENCY=4 docker run -p 5000:5000 my-model\n```\n\n### `COG_SETUP_TIMEOUT`\n\nControls the maximum time, in seconds, allowed for the model's `setup()` method to complete. If setup exceeds this timeout, the server reports setup failure.\n\nBy default, there is no timeout. Set to `0` to disable the timeout. Invalid values are ignored with a warning.\n\n```console\n$ COG_SETUP_TIMEOUT=300 docker run -p 5000:5000 my-model\n```\n\n### `COG_LOG_LEVEL`\n\nControls Coglet runtime log verbosity when `RUST_LOG` is not set.\n\nSupported values are `debug`, `info`, `warn`, `warning`, and `error`. The default is `info`.\n\n```console\n$ COG_LOG_LEVEL=debug docker run -p 5000:5000 my-model\n```\n\n### OpenTelemetry tracing\n\nTracing must first be enabled under `observability.traces` in `cog.yaml`. Runtime settings may disable an enabled image but cannot enable an image that did not opt in.\n\n| Variable | Purpose |\n| ----------------------------- | ---------------------------------------------- |\n| `COG_TRACE_ENABLED` | Set to `false` to disable tracing at runtime. |\n| `OTEL_SDK_DISABLED` | Hard-disable OpenTelemetry SDK initialization. |\n| `OTEL_EXPORTER_OTLP_ENDPOINT` | Collector endpoint for framework tracing. |\n| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf` or `grpc`. |\n| `OTEL_EXPORTER_OTLP_HEADERS` | Collector authentication headers. |\n| `OTEL_SERVICE_NAME` | Service name, default `cog`. |\n| `OTEL_TRACES_SAMPLER` | Runtime sampler override. |\n| `OTEL_TRACES_SAMPLER_ARG` | Ratio for ratio samplers. |\n\nWhen tracing is enabled without an endpoint, Cog logs one warning and continues without framework tracing. A custom Python provider configured by `observability.config` still runs and may use another exporter or no exporter. Delivery failures from Cog's built-in exporters never change prediction results; custom processors and exporters are model-owned code.\n\n`COG_OBSERVABILITY_*`, `COG_TRACE_*`, and `OTEL_*` are reserved from the general `cog.yaml` `environment` list. `COG_OBSERVABILITY_CONFIG` is internal and points to the validated file staged in the image. Supply supported `COG_TRACE_*` and `OTEL_*` settings to the running container instead.\n\n### `COG_THROTTLE_RESPONSE_INTERVAL`\n\nControls how often asynchronous webhook `output` and `logs` events are sent, in seconds.\n\nThe default is `0.5` seconds. Invalid values are ignored and the default is used. `start` and `completed` webhook events are always sent immediately.\n\n```console\n$ COG_THROTTLE_RESPONSE_INTERVAL=1 docker run -p 5000:5000 my-model\n```\n\n### `COG_STREAM_HISTORY_CAPACITY`\n\nControls how many server-sent event stream events are retained per prediction for replay when a client reconnects with `Accept: text/event-stream`.\n\nBy default, Cog retains the most recent 1024 events per prediction. Set to `0` to disable replay history while keeping live streaming enabled. Invalid values are ignored with a warning and the default is used.\n\n```console\n$ COG_STREAM_HISTORY_CAPACITY=0 docker run -p 5000:5000 my-model\n$ COG_STREAM_HISTORY_CAPACITY=4096 docker run -p 5000:5000 my-model\n```\n\n### `COG_WEIGHTS`\n\nProvides a weights path or URL to a model whose `setup()` method accepts a `weights` parameter.\n\n```console\n$ cog run -e COG_WEIGHTS=https://example.com/weights.tar -i prompt=\"hello\"\n```\n\n### `COG_USER_AGENT`\n\nSets the `User-Agent` header used by Cog when downloading URL-backed `File` inputs.\n\n```console\n$ COG_USER_AGENT=\"my-service/1.0\" docker run -p 5000:5000 my-model\n```\n\n## Push tuning variables\n\n### `COG_PUSH_OCI`\n\nEnables Cog's OCI chunked push path for container image layers when set to `1`. If the OCI push fails with a non-fatal error, Cog falls back to Docker's native push path.\n\n```console\n$ COG_PUSH_OCI=1 cog push\n```\n\n### `COG_PUSH_CONCURRENCY`\n\nControls how many image layers or weight blobs Cog uploads concurrently during push operations.\n\nThe default is `5`. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_CONCURRENCY=2 cog push\n```\n\n### `COG_PUSH_DEFAULT_CHUNK_SIZE`\n\nSets the default multipart upload chunk size, in bytes, when the registry does not advertise a maximum chunk size.\n\nThe default is 96 MiB. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_DEFAULT_CHUNK_SIZE=67108864 cog push\n```\n\n### `COG_PUSH_MULTIPART_THRESHOLD`\n\nSets the minimum blob size, in bytes, before Cog uses multipart upload.\n\nThe default is 128 MiB. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_MULTIPART_THRESHOLD=268435456 cog push\n```\n\n\n---\n\n# Examples\n\nCog's example models now live in the main Cog repository under\n[`examples/`](https://github.com/replicate/cog/tree/main/examples).\n\nIf you want a working project to copy from, start with one of these:\n\n- [`hello-world`](https://github.com/replicate/cog/tree/main/examples/hello-world):\n minimal string input/output example\n- [`hello-image`](https://github.com/replicate/cog/tree/main/examples/hello-image):\n returns a static image\n- [`hello-context`](https://github.com/replicate/cog/tree/main/examples/hello-context):\n shows how to read prediction context\n- [`hello-concurrency`](https://github.com/replicate/cog/tree/main/examples/hello-concurrency):\n demonstrates the `@cog.concurrent` decorator\n- [`hello-train`](https://github.com/replicate/cog/tree/main/examples/hello-train):\n defines a training interface\n- [`hello-replicate`](https://github.com/replicate/cog/tree/main/examples/hello-replicate):\n uses `cog.Secret` inputs and calls the Replicate API\n- [`blur`](https://github.com/replicate/cog/tree/main/examples/blur):\n simple image-to-image transformation\n- [`resnet`](https://github.com/replicate/cog/tree/main/examples/resnet):\n GPU image classification with ResNet50\n- [`streaming-text`](https://github.com/replicate/cog/tree/main/examples/streaming-text):\n streaming text generation with server-sent events\n- [`z-image-turbo`](https://github.com/replicate/cog/tree/main/examples/z-image-turbo):\n a generative image model example\n- [`notebook`](https://github.com/replicate/cog/tree/main/examples/notebook):\n runs a Jupyter notebook with Cog\n- [`canary`](https://github.com/replicate/cog/tree/main/examples/canary):\n small streaming output example used for validation\n- [`managed-weights`](https://github.com/replicate/cog/tree/main/examples/managed-weights):\n managed weights pipeline fixture\n\nThere is also an experimental example:\n\n- [`experimental/resnet-managed-weights`](https://github.com/replicate/cog/tree/main/examples/experimental/resnet-managed-weights):\n ResNet50 packaged with managed weights\n\nFor a quick summary of every example in the tree, see\n[`examples/README.md`](https://github.com/replicate/cog/blob/main/examples/README.md).\n\n\n---\n\n# Getting started with your own model\n\nThis guide will show you how to put your own machine learning model in a Docker image using Cog. If you haven't got a model to try out, you'll want to follow the [main getting started guide](getting-started.md).\n\n## Prerequisites\n\n- **Cog**. If you haven't already installed Cog, follow the [install instructions in the README](../README.md#install).\n- **macOS, Linux, or Windows 11**. Cog works on macOS and Linux. It also works on Windows 11 with [WSL 2](wsl2/wsl2.md).\n- **Docker**. Cog uses Docker to create a container for your model. You'll","isInternal":false,"tokens":12307,"sizeBytes":50026},{"name":"DESIGN.md","path":"DESIGN.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/DESIGN.md","title":"UI Design System & Style Tokens","category":"project-spec","format":"markdown","content":"# Design\n\n## Background\n\nCog came from Andreas's experience at Spotify and Ben's experience at Docker.\n\nAt Spotify, Andreas noticed a cluster of related problems:\n\n- **It was hard to run open-source machine learning models.** All the advances in machine learning were locked up inside prose in PDFs, scraps of code on GitHub, weights on Google Drive (if you were lucky!). If you wanted to build upon this research, or apply it to real-world problems, you had to implement it all from scratch.\n- **It was hard to deploy machine learning models to production.** Andreas was the only person on the research team who was also an infrastructure engineer. Typically a researcher would have to sit down with Andreas to decide on an API, get a server written, package up dependencies, battle CUDA, get it running efficiently, get it deployed on the cluster, and so on and so forth. It would take weeks to get something running in production.\n\nBen connected this back to his experience at Docker. What Docker did was define a standard box that software could go in. You could put any kind of server software in there – Python, Java, Ruby on Rails, whatever – and you could then know that you could run it on your local machine or on any cloud, as long as it supported Docker. We wanted to do the same thing for machine learning.\n\n## Vision\n\nWe want Cog to be a standard artifact for what a model is and how that model is run.\n\n(More detail...)\n\n## Design principles\n\nThere are a few things driving Cog's design:\n\n- **Reproducible artifact**: When you've put your model in Cog, it'll run anywhere, and _keep_ on running. This is why it's a Docker image, with all of the model's dependencies. Docker images have a content-addressable SHA256 ID, which is the identifier for that model's behavior, byte-for-byte.\n- **Weights inside the image**: We encourage users to put model weights in images. If the weights are on cloud storage somewhere, then they might change or disappear, and the image will produce different results. There's nothing magical about Docker images – they're just a bundle of files. Docker moves around that bundle of files quite slowly, though, but we can optimize that process so it's as fast as reading weights directly from blob storage, or wherever.\n- **Models are just functions**: Models can be lots of things. We are of the opinion that [machine learning is just software](https://replicate.com/blog/machine-learning-needs-better-tools), and a model is just a function. It often needs to be attached to a GPU, but apart from that it's just a normal function that has some input and some output. This is the core difference between Docker's abstraction and Cog's abstraction: Docker packages up an executable, whereas Cog packages up a _function_.\n- **Standard interface**: When you run the Docker container, it serves an HTTP server, that is a standard API for running that function. You can think of it like a remote procedure call.\n- **Self-describing artifact**: A Cog model has it's schema (or type signature, if you're thinking of it as a function) attached to the image as a label. This means systems that work with Cog models can know what the model is and what requests to send to it. This is what powers the forms on Replicate, for example.\n- **Not just the model**: Before Cog, the typical standard packaging formats for machine learning models were at the network level. A way of taking a Tensorflow or PyTorch network and packaging up in a way that would run on lots of different types of accelerators. Things like [ONNX](https://onnx.ai/) or [TVM's IR](https://tvm.apache.org/). We realized that \"models\" are not just the network, but they are also pre- and post-processing, and are so diverse and the field is so fast-moving that you can't possible squeeze it into some high-level abstraction. It just needs to be code running on a computer.\n- **It's just Docker**: Cog models need to run anywhere, and they'll only run anywhere if it's vanilla Docker. We might optimize how Docker images get shipped around to make it faster, but we're not going to invent our own image format.\n- **The API is for software developers**: In the olden days, you have to pass tensors to TFServing and know how to generate a tensor from a JPEG. Cog's API intentionally just speaks JSON, strings, files, etc. It's intended to be the interface between the software developer and the ML engineer. Sort of like Docker was intended to be the interface between the software developer and the infrastructure engineer.\n- **Cog is the APIs and interfaces, not just the software**: The most important thing about Cog is that it defines a standard for what a model is and how to run it. It doesn't necessarily need to involve Cog the piece of software itself. For example, Replicate could serve a model from OpenAI with a Cog API and schema, but it's not packaged or running with Cog under the hood at all – it's just calling the OpenAI API directly.\n","isInternal":false,"tokens":1077,"sizeBytes":4958},{"name":"SKILL.md","path":".agents/skills/release-cog/SKILL.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/.agents/skills/release-cog/SKILL.md","title":"release-cog","category":"anthropic-skill","format":"markdown","content":"---\nname: release-cog\ndescription: Guide and automate the Cog release process\n---\n\n# Cog Release Skill\n\nThis skill helps you release new versions of Cog. Cog is a multi-language, multi-artifact project with a carefully orchestrated release process.\n\n## Overview\n\nCog releases include:\n\n- **CLI binaries** (Go) - macOS and Linux, x86_64 and ARM64\n- **coglet Python wheels** (Rust/PyO3) - Linux x86_64/ARM64, macOS ARM64\n- **cog SDK Python wheel** (Python) - Universal\n- **coglet Rust crate** - Published to crates.io\n\n## Release Types\n\n| Type | Format | Example | Branch | PyPI | crates.io | Homebrew |\n| --------------- | ------------------------------- | ------------ | ------ | ---- | --------- | -------- |\n| **Stable** | `v0.17.0` | v0.18.0 | main | ✓ | ✓ | ✓ |\n| **Pre-release** | `v0.17.0-alpha3`, `v0.17.0-rc1` | v0.18.0-rc1 | main | ✓ | ✓ | ✗ |\n| **Dev** | `v0.17.0-dev1` | v0.18.0-dev2 | any | ✗ | ✗ | ✗ |\n\n## Quick Release Commands\n\n### 1. Bump Version (if needed)\n\n```bash\n# Check current version\nmise run version\n\n# Bump to new version (updates VERSION.txt, Cargo.toml, Cargo.lock, commits)\nmise run version:bump 0.18.0\n```\n\n### 2. Create a release branch and push\n\n```bash\ngit checkout -b release/v0.18.0\ngit push origin release/v0.18.0\n```\n\n### 3. Create PR to main\n\n```bash\n# Open a pull request to main, get it reviewed and merged. Then you can create the release tag from main.\ngh pr create --base main --head release/v0.18.0 --title \"Release v0.18.0\" --body \"Release description and notes\"\n```\n\n### 4. Create and Push Tag\n\n```bash\ngit checkout main && git pull origin main\n\n# For stable release\ngit tag v0.18.0\ngit push origin v0.18.0\n\n# For pre-release\ngit tag v0.18.0-rc1\ngit push origin v0.18.0-rc1\n\n# For dev release (can be from any branch)\ngit tag v0.18.0-dev1\ngit push origin v0.18.0-dev1\n```\n\n### 5. Monitor Release Build\n\n```bash\n# Watch the release build workflow\ngh workflow view release-build.yaml\n\n# Or watch in real-time\ngh run watch\n```\n\n### 6. Write Release Notes\n\nAfter the draft release is created, update the release notes to follow the project's standard format. The release notes are not auto-generated from commit messages — they must be hand-written and categorized.\n\n**To see the previous release's format:**\n\n```bash\ngh release view v0.19.0 --json body\n```\n\n**To gather commits since the last release:**\n\n```bash\n# List commits between the previous release and this one\ngit log --oneline v0.19.0..v0.20.0 --no-merges\n```\n\n**Release notes structure:**\n\nGroup changes into three sections. Only include sections that have items.\n\n1. **New features** — New commands, new APIs, new annotations, new capabilities.\n2. **Improvements** — Performance, reliability, DX improvements, removals of legacy paths, build improvements.\n3. **Bug fixes** — User-visible bug fixes. Prefer \"Now does X correctly\" over \"Fixed X\".\n\n**Style guidelines:**\n\n- Lead each bullet with a **bold, user-facing summary sentence** (e.g., \"**`cog run` command.** ...\")\n- Follow with a short explanation of what changed and why it matters\n- Reference the PR number in parentheses at the end: `(#3015)`\n- Use backticks for commands, flags, and code references\n- Omit internal refactors, dependency bumps, and CI-only changes unless they are user-facing\n- Omit version bump commits\n\n**Example:**\n\n```markdown\n### New features\n\n- **`cog run` command.** The `cog predict` command has been renamed to `cog run` with full backward compatibility. `cog predict` still works as an alias. (#3015)\n- **Model refs for `cog push` and weights commands.** You can now reference models by name (e.g., `r8.im/user/model`) instead of full image URLs when pushing or managing weights. (#3018)\n\n### Improvements\n\n- **Runtime schema generation fully removed.** The legacy runtime Python schema generation path has been completely removed. Cog exclusively uses static schema generation, making builds faster and more reliable. (#3003)\n\n### Bug fixes\n\n- **Pushing a model with a version tag now emits a clean URL.** The Replicate model URL printed after `cog push` no longer includes the image tag (e.g., `:latest`), preventing 404 errors when users click the link. (#3020)\n```\n\n**To update the draft release:**\n\n```bash\ngh release edit v0.20.0 --notes \"$(cat <<'EOF'\n### New features\n\n- ...\n\n### Improvements\n\n- ...\n\n### Bug fixes\n\n- ...\nEOF\n)\"\n```\n\n### 7. Publish Release (stable/pre-release only)\n\n- Go to GitHub Releases page\n- Find the draft release\n- Review release notes\n- Click \"Publish release\"\n- This triggers `release-publish.yaml` which publishes to PyPI and crates.io\n\n## Release Process Details\n\n### Automated Workflows\n\n1. **`release-build.yaml`** - Triggered on version tags\n - Verifies tag matches VERSION.txt and Cargo.toml\n - Verifies stable/pre-release tags are on main branch\n - Builds SDK wheel (with updated coglet version constraint)\n - Builds coglet wheels for all platforms (Linux x64/ARM64, macOS ARM64)\n - Uses GoReleaser to build CLI binaries and create draft release\n - Uploads wheels to GitHub release\n - For dev releases: immediately publishes as pre-release\n\n2. **`release-publish.yaml`** - Triggered when release is published\n - Publishes coglet wheels to PyPI\n - Publishes coglet crate to crates.io\n - Publishes cog SDK to PyPI (depends on coglet)\n - Updates Homebrew tap (stable releases only)\n\n3. **`homebrew-tap.yaml`** - Updates Homebrew cask\n - Generates cask from `.github/cog.rb.tmpl`\n - Creates PR in `replicate/homebrew-tap`\n\n### Version Files\n\n| File | Purpose |\n| ------------------- | ------------------------------------------ |\n| `VERSION.txt` | Canonical version (single source of truth) |\n| `crates/Cargo.toml` | Rust workspace version |\n| `crates/Cargo.lock` | Locked dependency versions |\n\n### Version Constraints\n\nThe SDK (`pyproject.toml`) has a dependency on coglet:\n\n```toml\ncoglet>=0.1.0,<1.0\n```\n\nDuring release build, this is updated to:\n\n```toml\ncoglet>=0.18.0,<1.0\n```\n\nThis ensures the SDK depends on the matching coglet version.\n\n## Pre-Release Checklist\n\nBefore creating a release tag:\n\n- [ ] All tests pass: `mise run test`\n- [ ] Lint passes: `mise run lint`\n- [ ] Version is correct in `VERSION.txt`\n- [ ] `mise run version:check` passes\n- [ ] `crates/Cargo.toml` matches `VERSION.txt`\n- [ ] Changelog is updated (if applicable)\n- [ ] Documentation is updated (`mise run docs:llm`)\n\n## Troubleshooting\n\n### Version mismatch error\n\n```\nVersion mismatch! VERSION.txt has X but tag is vY\n```\n\nFix: Run `mise run version:bump Y`, push, then re-tag.\n\n### Tag not on main\n\n```\nRelease tags must be on the main branch\n```\n\nFix: Merge your changes to main, then tag from main.\n\n### Rebuilding a failed release\n\n1. Delete the GitHub release if it was created: `gh release delete v0.18.0 --yes`\n2. Delete the tag: `git push --delete origin v0.18.0 && git tag -d v0.18.0`\n3. Fix the issue\n4. Re-create and push the tag\n\n### Manual PyPI publish (emergency)\n\nIf the automated publish fails:\n\n```bash\n# Download wheels from GitHub release\ngh release download v0.18.0 -p \"coglet-*.whl\" -D dist\ngh release download v0.18.0 -p \"cog-*.whl\" -D dist\n\n# Publish with twine\n twine upload dist/coglet-*.whl # First!\ntwine upload dist/cog-*.whl # After coglet is uploaded\n```\n\n## Architecture Notes\n\n- **Trusted Publishing**: PyPI and crates.io use OIDC trusted publishing (no API tokens in secrets)\n- **Environments Required**: Configure `pypi`, `crates-io`, and `homebrew` environments in GitHub repo settings\n- **CGO**: Required for go-tree-sitter (static Python schema parser)\n- **Zig**: Used for Linux cross-compilation (CC=zig cc)\n- **macOS builds**: Use native compiler (zig lacks macOS SDK stubs)\n- **Wheel discovery**: CLI discovers wheels from `dist/` at Docker build time, not embedded in binary\n","frontmatter":{"name":"release-cog","description":"Guide and automate the Cog release process"},"isInternal":false,"tokens":2134,"sizeBytes":8045},{"name":"SKILL.md","path":".agents/skills/updating-architecture-docs/SKILL.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/.agents/skills/updating-architecture-docs/SKILL.md","title":"updating-architecture-docs","category":"anthropic-skill","format":"markdown","content":"---\nname: updating-architecture-docs\ndescription: \"Use this skill when updating, reviewing, or creating architecture documentation in the architecture/ directory. This includes after refactors, feature additions, component changes, or when auditing docs for accuracy. Use it any time code changes affect how Cog's internals work -- new packages, changed IPC protocols, modified build pipeline, runtime behavior changes. Also use it proactively when reviewing PRs that touch core systems to check whether the architecture docs need updating.\"\n---\n\n# Updating Architecture Docs\n\n## Purpose\n\nThe `architecture/` directory is a bridge. It takes someone who already knows what Cog does and how to use it (that's what `docs/` is for) to the point where they can navigate the source code with confidence. It answers \"how does this work and why is it shaped this way\" -- the 10,000 foot view of the project and its ecosystem.\n\nThe goal is to move a reader from fuzzy concepts to clarity:\n\n- What are the components and what do they own?\n- What's the vocabulary? (What does \"slot\" mean? What's the \"envelope\"?)\n- Where are the boundaries between systems?\n- What are the key design decisions and why were they made?\n- Where do I look in the source to go deeper?\n\nThe architecture docs are **not** a substitute for reading code or a prose summary of the implementation. They give the reader enough context to make the code legible on first contact. Once someone understands that coglet uses a two-process model with slot-based IPC, reading `orchestrator.rs` makes sense. Without that context, it's just code.\n\n## Document structure\n\nThe docs are numbered to suggest a reading order. `00-overview.md` orients the reader and points to everything else. Each subsequent doc builds on concepts from the ones before it -- someone reading about the container runtime should already understand the predictor class and the HTTP API.\n\nThe current docs and their structure aren't fixed. New docs should be added when a topic becomes important enough to deserve its own section (testing philosophy, deployment model, etc.). Docs can be split, merged, or reorganized as the system evolves. The numbering is for reading order, not a permanent taxonomy. Check `architecture/` for the current set.\n\n## Principles\n\n### Bridge, don't summarize\n\nThe architecture docs should give a reader the mental model they need to read the code -- not replace the code. If a section starts reading like a prose restatement of what a module does line by line, it's gone too far. Pull back to the concepts and boundaries.\n\nGood: \"The orchestrator spawns a single worker subprocess and manages its lifecycle. Communication happens over two channels: a control channel for lifecycle events (stdin/stdout, JSON lines) and per-slot Unix sockets for prediction data.\"\n\nBad: \"The `spawn_worker` function in `orchestrator.rs` calls `Command::new('python')` with args `['-c', 'import coglet; coglet.server._run_worker()']` and sets up stdin/stdout pipes. It then reads from the control channel in a loop, matching on `ControlResponse` variants...\"\n\nThe first gives you the mental model. The second just restates the code. A reader with the first description can read `orchestrator.rs` and follow along. A reader with the second didn't need to read the doc at all.\n\n### Point at packages, not files\n\nReference source locations at the **package/directory level** with a description of what that package owns. Specific file paths and line numbers rot as code moves around. A pointer like \"`crates/coglet/src/bridge/` -- IPC protocol and transport\" stays accurate through refactors. \"`bridge/protocol.rs:69` -- ControlRequest enum\" doesn't.\n\nOnly document packages that matter for understanding the system's shape. Generic utility packages (`pkg/util/`, `pkg/path/`, etc.) don't need a mention -- their existence is obvious and they don't help a reader build a mental model. If someone needs them, they'll find them.\n\nWhen a specific file reference is genuinely useful (a key entry point, a non-obvious starting point for understanding a subsystem), include it -- but prefer \"the `PredictionService` in `service.rs`\" over a line number.\n\n### Document boundaries, not internals\n\nFocus on interfaces between components: what messages cross the IPC channel, what the HTTP API contract is, what labels a built image carries, what env vars control behavior. These are the things a reader needs to understand to reason about the system. Implementation details behind those boundaries belong in code comments and crate-level READMEs, not in architecture docs.\n\nThe practical test: would an internal refactor (changing how something works without changing its interface) require updating this doc? If yes, the doc is too detailed.\n\n### Explain the why\n\nDesign decisions are the most valuable content in architecture docs. They're the one thing you can't get from reading the code. \"Why two processes?\" has an answer (isolation, CUDA contexts, crash resilience) that makes the whole architecture make sense. Without it, a reader sees the complexity of IPC and wonders whether it's accidental.\n\nEvery major structural choice should have a short rationale. One or two sentences is enough.\n\n## Current state of the world\n\nThese facts should be reflected consistently across all docs. If the code changes and these become stale, the docs need updating.\n\n**One runtime:** Coglet (Rust/Axum + PyO3) is the sole runtime. There's no legacy Python/FastAPI runtime, no toggle, no \"experimental\" qualifier. Don't frame coglet as an alternative to something else.\n\n**Pydantic is not core:** Schema generation parses Python source in Go with tree-sitter and emits OpenAPI directly. `cog.BaseModel` is a dataclass wrapper. Pydantic BaseModel is supported in user code for compatibility but isn't part of Cog's own type system.\n\n**Static schema generation is the only path:** The tree-sitter-based static schema generator (Go side, `pkg/schema/`) runs on schema-generating `cog build` invocations. Parser failures stop the build before Docker starts; there is no Docker/Python runtime schema fallback or `COG_LEGACY_SCHEMA` toggle.\n\n**Wheels aren't embedded:** SDK and coglet wheels are resolved at Docker build time from PyPI, env vars, or local `dist/` directory. They're not compiled into the Go binary.\n\n**Three codebases:**\n\n- `cmd/cog/` + `pkg/` -- Go CLI and build tooling\n- `python/cog/` -- Python SDK (type definitions, predictor base class, thin server launcher)\n- `crates/coglet/` + `crates/coglet-python/` -- Rust prediction server with PyO3 bindings\n\n## How to update\n\n### When to update\n\nAfter code changes, ask: **did a boundary or interface change?**\n\nNeeds doc updates:\n\n- New or removed IPC message types\n- New or changed HTTP endpoints\n- New CLI commands\n- Changes to the build pipeline (new build steps, changed Dockerfile structure)\n- New top-level packages in `pkg/` or new crates in `crates/`\n- Changes to how components communicate\n- New design decisions or changed rationale for existing ones\n\nDoesn't need doc updates:\n\n- Bug fixes within a component\n- Internal refactors that don't change interfaces\n- Performance improvements\n- Test changes\n\n### Auditing for accuracy\n\nWhen checking docs against the codebase:\n\n1. **Read each doc** and identify claims about the system -- component names, package locations, protocol messages, env vars, build steps, CLI commands\n2. **Verify against source** -- check that things exist and work as described\n3. **Classify issues**:\n - **Structural** -- describes something that doesn't exist or works fundamentally differently (fix immediately)\n - **Misleading** -- technically true but gives the wrong mental model (fix immediately)\n - **Missing** -- a feature or component exists with no doc coverage (add if it's a boundary/interface concern)\n - **Stale reference** -- file/package renamed but concept is right (fix opportunistically)\n4. Fix structural and misleading issues first -- they actively harm readers\n\n### Writing conventions\n\nRead the existing architecture docs before writing. Match their tone and level of detail. New content should feel like it belongs -- if it reads like a different author wrote it, adjust.\n\nThe docs are technical writing. Clear, precise, no filler. Don't inflate -- avoid the kind of padded, formal language that agents default to (\"it should be noted that\", \"robust\", \"comprehensive\", \"this ensures that\"). Just say the thing.\n\nDiagrams use Mermaid or ASCII art. Both are fine. Use whichever communicates the structure more clearly. Keep each diagram focused on one concept.\n","frontmatter":{"name":"updating-architecture-docs","description":"Use this skill when updating, reviewing, or creating architecture documentation in the architecture/ directory. This includes after refactors, feature additions, component changes, or when auditing docs for accuracy. Use it any time code changes affect how Cog's internals work -- new packages, changed IPC protocols, modified build pipeline, runtime behavior changes. Also use it proactively when reviewing PRs that touch core systems to check whether the architecture docs need updating."},"isInternal":false,"tokens":1845,"sizeBytes":8603},{"name":"SKILL.md","path":".opencode/skills/cog-review/SKILL.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/.opencode/skills/cog-review/SKILL.md","title":"cog-review","category":"anthropic-skill","format":"markdown","content":"---\nname: cog-review\ndescription: Cog architecture and cross-cutting review guidelines\n---\n\n## Cog architecture review guidelines\n\nCog packages ML models into production-ready containers. Use this skill for\nchanges that cross language boundaries or touch core architecture.\n\n### Component overview\n\n- **CLI** (Go): `cmd/cog/` and `pkg/` -- builds, runs, and deploys models\n- **Python SDK**: `python/cog/` -- predictor interface, types, HTTP/queue server\n- **Coglet** (Rust): `crates/` -- prediction server inside containers (HTTP, worker management, IPC)\n\n### Key design patterns\n\n**Wheel resolution**: The CLI discovers SDK and coglet wheels from `dist/` at\nDocker build time. Wheels are NOT embedded in the binary. Changes to build\nartifacts need to account for this.\n\n**Dockerfile generation**: `pkg/dockerfile/` generates Dockerfiles from `cog.yaml`\nconfig. Template injection and escaping matter here.\n\n**Config parsing**: `pkg/config/config.go` parses `cog.yaml`. Schema is at\n`pkg/config/data/config_schema_v1.0.json`. Changes must keep schema and Go\ncode in sync.\n\n**Two-process coglet**: Parent process (HTTP server + orchestrator) and child\nworker process (Python predictor execution) communicate via IPC. Changes to\nthe IPC protocol affect both Rust and Python code.\n\n**Compatibility matrix**: CUDA/PyTorch/TensorFlow compatibility is managed in\n`tools/compatgen/`. Framework version changes have wide blast radius.\n\n### Cross-cutting concerns\n\n- **VERSION.txt** is the single source of truth for versioning. `Cargo.toml` must match.\n- Changes to `python/cog/base_predictor.py` affect all downstream model authors.\n- Changes to `docs/` require running `mise run docs:llm` to regenerate `docs/llms.txt`.\n- CLI reference docs are auto-generated -- changes to `cmd/` or `pkg/cli/` require `mise run docs:cli`.\n- Integration tests in `integration-tests/` use Go's testscript and need a built cog binary.\n\n### What to watch for\n\n- Breaking changes to the predictor interface or cog.yaml schema\n- Docker build changes that affect layer caching or build time\n- IPC protocol changes without updating both Rust and Python sides\n- Version bumps that miss one of the places VERSION.txt needs to match\n- New ML framework versions without compatibility testing\n","frontmatter":{"name":"cog-review","description":"Cog architecture and cross-cutting review guidelines"},"isInternal":false,"tokens":515,"sizeBytes":2259},{"name":"SKILL.md","path":".opencode/skills/go-review/SKILL.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/.opencode/skills/go-review/SKILL.md","title":"go-review","category":"anthropic-skill","format":"markdown","content":"---\nname: go-review\ndescription: Go code review guidelines for the Cog codebase\n---\n\n## Go review guidelines\n\nThis project uses Go for the CLI (`cmd/cog/`, `pkg/`) and support tooling (`tools/`).\n\n### What linters already catch (skip these)\n\ngolangci-lint runs errcheck, gocritic, gosec, govet, ineffassign, misspell,\nrevive, staticcheck, and unused. Don't flag issues these would catch.\n\n### What to look for\n\n**Error handling**\n\n- Errors returned but not checked or silently discarded\n- User-facing errors should use `pkg/errors.CodedError` with error codes\n- Generic error wrapping that loses context (`fmt.Errorf(\"failed\")` with no `%w`)\n\n**Imports**\n\n- Should be three groups: stdlib, third-party, internal (`github.com/replicate/cog/pkg/...`)\n- Only flag if actually wrong, not cosmetic reordering\n\n**Testing**\n\n- Must use `testify/require` for fatal assertions and `testify/assert` for non-fatal\n- No raw `if` checks with `t.Fatal`/`t.Errorf`\n- Prefer table-driven tests for similar cases\n- Prefer specific assertions (`Equal`, `Contains`, `NoError`) over `True`/`False`\n\n**Concurrency**\n\n- Goroutine leaks (no cleanup path, missing context cancellation)\n- Shared state without synchronization\n- Channel misuse (sends on closed channels, unbuffered channels in wrong contexts)\n\n**Docker/container patterns**\n\n- The CLI uses the Docker Go SDK. Watch for leaked clients, unclosed response bodies\n- Dockerfile generation is in `pkg/dockerfile/` -- template injection risks\n\n**Architecture**\n\n- Commands belong in `pkg/cli/`, business logic in `pkg/`\n- Config parsing/validation in `pkg/config/`\n- Don't mix CLI concerns with library logic\n","frontmatter":{"name":"go-review","description":"Go code review guidelines for the Cog codebase"},"isInternal":false,"tokens":395,"sizeBytes":1643},{"name":"SKILL.md","path":".opencode/skills/python-review/SKILL.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/.opencode/skills/python-review/SKILL.md","title":"python-review","category":"anthropic-skill","format":"markdown","content":"---\nname: python-review\ndescription: Python code review guidelines for the Cog SDK\n---\n\n## Python review guidelines\n\nThis project uses Python for the SDK (`python/cog/`) which defines the predictor\ninterface, type system, and HTTP/queue server.\n\n### What linters already catch (skip these)\n\nruff handles pycodestyle (E), Pyflakes (F), isort (I), warnings (W), bandit (S),\nbugbear (B), and annotations (ANN). Don't flag issues these would catch.\n\n### What to look for\n\n**Type annotations**\n\n- Required on all function signatures\n- Use `typing_extensions` for backward compatibility\n- Avoid `Any` where a concrete type is possible\n- Check that type annotations actually match runtime behavior\n\n**Compatibility**\n\n- Must support Python 3.10 through 3.13\n- Watch for syntax/stdlib features only available in newer versions\n- `from __future__ import annotations` if using newer annotation syntax\n\n**Error handling**\n\n- No bare `except:` or `except Exception:` that swallows everything\n- Exceptions should have descriptive messages\n- Resource cleanup with context managers, not try/finally when avoidable\n\n**Async patterns**\n\n- Tests use pytest-asyncio -- async tests need proper fixtures\n- Watch for blocking calls inside async functions\n- Proper cleanup of async resources (aclose, async context managers)\n\n**Predictor interface**\n\n- `base_predictor.py` is the core interface -- changes here affect all users\n- `types.py` defines input/output types -- check backward compatibility\n- Server code in `python/cog/server/` handles HTTP -- watch for request handling bugs\n\n**Testing**\n\n- Uses pytest with fixtures\n- tox runs tests across Python 3.10-3.13\n- Test isolation: don't rely on global state or test ordering\n","frontmatter":{"name":"python-review","description":"Python code review guidelines for the Cog SDK"},"isInternal":false,"tokens":383,"sizeBytes":1708},{"name":"SKILL.md","path":".opencode/skills/rust-review/SKILL.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/.opencode/skills/rust-review/SKILL.md","title":"rust-review","category":"anthropic-skill","format":"markdown","content":"---\nname: rust-review\ndescription: Rust code review guidelines for Coglet\n---\n\n## Rust review guidelines\n\nThis project uses Rust for Coglet (`crates/`), the prediction server that runs\ninside Cog containers. It handles HTTP requests, worker process management, and\nprediction execution.\n\n### What linters already catch (skip these)\n\nclippy runs in CI. cargo-deny audits dependencies for license/advisory issues.\nDon't flag issues these would catch.\n\n### What to look for\n\n**Error handling**\n\n- Use `thiserror` for typed errors in library code, `anyhow` for application errors\n- Don't use `.unwrap()` or `.expect()` in non-test code unless the invariant is documented\n- Error context: use `.context()` or `.with_context()` from anyhow, not bare `?`\n\n**Ownership and lifetimes**\n\n- Unnecessary cloning where a borrow would work\n- Lifetime issues that suggest a design problem (not just annotation noise)\n- Arc/Mutex when simpler patterns exist\n\n**Async (tokio)**\n\n- Blocking operations inside async contexts (use `spawn_blocking`)\n- Missing `.await` on futures (compiler catches some, but not all logical issues)\n- Proper cancellation handling and cleanup\n- Task/JoinHandle leaks\n\n**Safety**\n\n- Any `unsafe` block needs justification and a safety comment\n- FFI boundaries with Python (PyO3) -- check for panics across FFI, GIL handling\n- Memory safety in IPC between parent and worker processes\n\n**Architecture**\n\n- `crates/coglet/` is the core: HTTP server, worker orchestration, IPC\n- `crates/coglet-python/` is PyO3 bindings for Python predictor integration\n- Two-process architecture: parent (HTTP + orchestrator) and worker (Python execution)\n- Don't mix IPC concerns with HTTP handling\n- Snapshot tests use `insta` -- check if snapshots need updating\n\n**Dependencies**\n\n- New dependencies should be justified -- `crates/deny.toml` audits them\n- Prefer std or existing deps over adding new ones for small tasks\n","frontmatter":{"name":"rust-review","description":"Rust code review guidelines for Coglet"},"isInternal":false,"tokens":435,"sizeBytes":1914},{"name":"cog.yaml","path":"tools/test-harness/fixtures/models/complex-output/cog.yaml","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/complex-output/cog.yaml","title":"Complex-output Skill","category":"anthropic-skill","format":"yaml","content":"# Complex-output Skill\n\nPath: `tools/test-harness/fixtures/models/complex-output/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/complex-output/cog.yaml)","tokens":300,"sizeBytes":233},{"name":"cog.yaml","path":"tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml","title":"Constraints-and-choices Skill","category":"anthropic-skill","format":"yaml","content":"# Constraints-and-choices Skill\n\nPath: `tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml)","tokens":300,"sizeBytes":260},{"name":"cog.yaml","path":"tools/test-harness/fixtures/models/file-path-types/cog.yaml","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/file-path-types/cog.yaml","title":"File-path-types Skill","category":"anthropic-skill","format":"yaml","content":"# File-path-types Skill\n\nPath: `tools/test-harness/fixtures/models/file-path-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/file-path-types/cog.yaml)","tokens":300,"sizeBytes":236},{"name":"cog.yaml","path":"tools/test-harness/fixtures/models/list-types/cog.yaml","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/list-types/cog.yaml","title":"List-types Skill","category":"anthropic-skill","format":"yaml","content":"# List-types Skill\n\nPath: `tools/test-harness/fixtures/models/list-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/list-types/cog.yaml)","tokens":300,"sizeBytes":221},{"name":"cog.yaml","path":"tools/test-harness/fixtures/models/optional-list-types/cog.yaml","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-list-types/cog.yaml","title":"Optional-list-types Skill","category":"anthropic-skill","format":"yaml","content":"# Optional-list-types Skill\n\nPath: `tools/test-harness/fixtures/models/optional-list-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-list-types/cog.yaml)","tokens":300,"sizeBytes":248},{"name":"cog.yaml","path":"tools/test-harness/fixtures/models/optional-types/cog.yaml","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-types/cog.yaml","title":"Optional-types Skill","category":"anthropic-skill","format":"yaml","content":"# Optional-types Skill\n\nPath: `tools/test-harness/fixtures/models/optional-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-types/cog.yaml)","tokens":300,"sizeBytes":233},{"name":"cog.yaml","path":"tools/test-harness/fixtures/models/scalar-types/cog.yaml","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/scalar-types/cog.yaml","title":"Scalar-types Skill","category":"anthropic-skill","format":"yaml","content":"# Scalar-types Skill\n\nPath: `tools/test-harness/fixtures/models/scalar-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/scalar-types/cog.yaml)","tokens":300,"sizeBytes":227},{"name":"manifest.yaml","path":"tools/test-harness/manifest.yaml","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/manifest.yaml","title":"Test-harness Skill","category":"anthropic-skill","format":"yaml","content":"# Cog Model Test Manifest\n# =======================\n# Each entry defines a model to test, its inputs, and expected outputs.\n#\n# Input values prefixed with \"@\" are resolved as fixture file paths relative\n# to the fixtures/ directory (e.g. \"@test_image.png\" -> fixtures/test_image.png).\n#\n# Validation types:\n# exact - output string must equal `value` exactly\n# contains - output string must contain `value` as a substring\n# regex - output string must match `pattern`\n# file_exists - output is a file path; optionally check `mime` type\n# json_match - parse output as JSON, assert `match` is a subset\n# json_keys - parse output as JSON dict, assert it has entries\n# not_empty - output is non-empty (loose smoke test)\n#\n# Setup commands:\n# Optional `setup` list of shell commands to run after cloning the repo\n# but before building. Useful for models that generate cog.yaml from\n# templates or need other preparation steps.\n# Example:\n# setup:\n# - \"script/select.sh dev\"\n#\n# Required tools:\n# Optional `requires_tools` list of CLI tools that must be on PATH for\n# setup commands to work. The harness checks before running and prints\n# install instructions for known tools (yq, envsubst).\n# Example:\n# requires_tools: [\"yq\"]\n#\n# Model resolution (base_dir takes priority over repo):\n# - Set `base_dir` for local models -- `path` is resolved under it. A\n# relative `base_dir` is resolved against the repo root; an absolute path\n# is used as-is. Local only; never clones.\n# - Set `repo: owner/name` (with no `base_dir`) to clone from GitHub.\n# - Set neither to default to fixtures/models/ (local fixtures).\n# `repo: local` is no longer supported -- use `base_dir` instead.\n\ndefaults:\n sdk_version: \"latest\" # \"latest\" = newest stable from PyPI; or pin e.g. \"0.16.12\"\n cog_version: \"latest\" # \"latest\" = newest stable release; or pin e.g. \"v0.16.12\"\n\nmodels:\n # ── cog-examples (CPU) ──────────────────────────────────────────────\n\n - name: hello-world\n base_dir: examples\n path: hello-world\n gpu: false\n tests:\n - description: \"basic predict\"\n inputs:\n text: \"world\"\n expect:\n type: exact\n value: \"hello world\"\n\n - name: canary\n base_dir: examples\n path: canary\n gpu: false\n tests:\n - description: \"streaming concatenate iterator\"\n inputs:\n text: \"friend\"\n expect:\n type: contains\n value: \"friend\"\n\n - name: blur\n base_dir: examples\n path: blur\n gpu: false\n tests:\n - description: \"blur an image\"\n inputs:\n image: \"@test_image.png\"\n blur: 5\n expect:\n type: file_exists\n mime: \"image/png\"\n\n - name: hello-image\n base_dir: examples\n path: hello-image\n gpu: false\n tests:\n - description: \"return a static image\"\n inputs: {}\n expect:\n type: file_exists\n\n - name: hello-concurrency\n base_dir: examples\n path: hello-concurrency\n gpu: false\n tests:\n - description: \"async streaming output\"\n inputs:\n total: 3\n interval: 0\n expect:\n type: contains\n value: \"Apple\"\n\n - name: hello-context\n base_dir: examples\n path: hello-context\n gpu: false\n # NOTE: This model uses current_scope().context which may not be\n # available in coglet yet. A failure here is a real compatibility signal.\n tests:\n - description: \"returns input and context\"\n inputs:\n text: \"testing\"\n expect:\n type: json_match\n match:\n inputs:\n text: \"testing\"\n\n - name: hello-train\n base_dir: examples\n path: hello-train\n gpu: false\n train_tests:\n - description: \"train produces weights file\"\n inputs:\n prefix: \"custom\"\n expect:\n type: not_empty\n tests:\n - description: \"predict with default weights\"\n inputs:\n text: \"world\"\n expect:\n type: contains\n value: \"world\"\n\n - name: notebook\n base_dir: examples\n path: notebook\n gpu: false\n tests:\n - description: \"basic predict\"\n inputs:\n name: \"world\"\n expect:\n type: contains\n value: \"hello\"\n\n - name: managed-weights\n base_dir: examples\n path: managed-weights\n gpu: false\n requires_env:\n - COG_MANAGED_WEIGHTS_READY\n tests:\n - description: \"weight validation smoke test\"\n inputs: {}\n expect:\n type: not_empty\n\n - name: streaming-text\n base_dir: examples\n path: streaming-text\n gpu: false\n tests:\n - description: \"streaming text generation\"\n inputs:\n prompt: \"Hello\"\n expect:\n type: not_empty\n\n # ── cog-examples (GPU) ──────────────────────────────────────────────\n\n - name: resnet\n base_dir: examples\n path: resnet\n gpu: true\n tests:\n - description: \"classify hotdog image\"\n inputs:\n image: \"@hotdog.png\"\n expect:\n type: json_keys\n\n - name: z-image-turbo\n base_dir: examples\n path: z-image-turbo\n gpu: true\n timeout: 600\n tests:\n - description: \"generate image from prompt\"\n inputs:\n prompt: \"a cat sitting on a windowsill\"\n expect:\n type: file_exists\n mime: \"image/png\"\n\n # ── Local fixture models ─────────────────────────────────────────────\n # These set base_dir to the fixtures dir (relative to the repo root), so\n # path is relative to it. They cover the full input type matrix for build\n # and prediction testing.\n\n - name: fixture-scalar-types\n base_dir: tools/test-harness/fixtures/models\n path: scalar-types\n gpu: false\n tests:\n - description: \"basic scalar inputs\"\n inputs:\n text: \"hello\"\n expect:\n type: contains\n value: \"hello\"\n\n - name: fixture-optional-types\n base_dir: tools/test-harness/fixtures/models\n path: optional-types\n gpu: false\n tests:\n - description: \"optional inputs default to None\"\n inputs:\n text: \"hello\"\n expect:\n type: exact\n value: \"hello\"\n\n - name: fixture-list-types\n base_dir: tools/test-harness/fixtures/models\n path: list-types\n gpu: false\n\n - name: fixture-optional-list-types\n base_dir: tools/test-harness/fixtures/models\n path: optional-list-types\n gpu: false\n tests:\n - description: \"optional lists default to None\"\n inputs:\n text: \"hello\"\n expect:\n type: exact\n value: \"hello\"\n\n - name: fixture-constraints-and-choices\n base_dir: tools/test-harness/fixtures/models\n path: constraints-and-choices\n gpu: false\n tests:\n - description: \"defaults with constraints and choices\"\n inputs: {}\n expect:\n type: exact\n value: \"hello-0.7-50-balanced-1\"\n\n - name: fixture-file-path-types\n base_dir: tools/test-harness/fixtures/models\n path: file-path-types\n gpu: false\n\n - name: fixture-complex-output\n base_dir: tools/test-harness/fixtures/models\n path: complex-output\n gpu: false\n tests:\n - description: \"structured BaseModel output\"\n inputs:\n prompt: \"test\"\n expect:\n type: json_match\n match:\n text: \"generated: test\"\n score: 0.95\n\n # ── External models (add your own below) ────────────────────────────\n # - name: my-custom-model\n # repo: myorg/my-model-repo\n # path: \".\"\n # gpu: true\n # # sdk_version: \"0.16.12\" # optional per-model override\n # env:\n # HF_TOKEN: \"${HF_TOKEN}\"\n # timeout: 600\n # tests:\n # - description: \"smoke test\"\n # inputs:\n # prompt: \"hello\"\n # expect:\n # type: contains\n # value: \"result\"\n #\n # Models that need setup commands before building (e.g. generating cog.yaml):\n # - name: flux-dev\n # repo: replicate/cog-flux\n # path: \".\"\n # gpu: true\n # setup:\n # - \"script/select.sh dev\"\n # timeout: 900\n # tests:\n # - description: \"generate image\"\n # inputs:\n # prompt: \"a cat\"\n # expect:\n # type: file_exists\n","isInternal":false,"tokens":2095,"sizeBytes":8629},{"name":"README.md","path":"tools/test-harness/README.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/README.md","title":"test-harness Documentation","category":"plugin-manifest","format":"markdown","content":"# Cog Model Test Harness\n\nAutomated test harness for validating Cog models against new CLI/SDK versions.\n\nThis is the Go implementation and replaces the previous Python harness.\n\n## Quick Start\n\n```bash\ncd tools/test-harness\n\n# List all models in the manifest\ngo run . list\n\n# Run all non-GPU models\ngo run . run --no-gpu\n\n# Run a specific model\ngo run . run --model hello-world\n\n# Run GPU models only (requires NVIDIA GPU + nvidia-docker)\ngo run . run --gpu-only\n\n# Output JSON report\ngo run . run --no-gpu --output json --output-file results/report.json\n\n# Build images only (no predictions)\ngo run . build --no-gpu\n\n# Use a locally-built cog binary\ngo run . run --no-gpu --cog-binary ../../dist/go/darwin-arm64/cog\n\n# Test from a git branch (builds CLI + SDK wheel automatically)\ngo run . run --no-gpu --cog-ref main\n```\n\nYou can also run from repo root:\n\n```bash\n(cd tools/test-harness && go run . list)\n(cd tools/test-harness && go run . run --no-gpu)\n```\n\n## Prerequisites\n\n- Go (see repo toolchain)\n- Docker\n- For `--cog-ref`: Go + uv on PATH (`mise install` sets this up)\n- For GPU models: NVIDIA GPU + nvidia-docker runtime\n\n## Manifest and Fixtures\n\n- Manifest: `tools/test-harness/manifest.yaml`\n- Fixtures: `tools/test-harness/fixtures/`\n\nInput values prefixed with `@` resolve to fixture paths. Example:\n\n```yaml\ninputs:\n image: \"@test_image.png\"\n```\n\n## CLI Reference\n\n```text\nusage: test-harness {run,build,list} [options]\n\nCommands:\n run Build and test models (full pipeline)\n build Build Docker images only (no predictions)\n list List models defined in the manifest\n\nCommon options:\n --manifest PATH Path to manifest.yaml\n --model NAME Run only this model (repeatable)\n --no-gpu Skip GPU models\n --gpu-only Only run GPU models\n --sdk-version VER SDK version override\n --cog-version TAG CLI version to download\n --cog-binary PATH Path to local cog binary\n --cog-ref REF Git ref to build cog from source\n --sdk-wheel PATH Local SDK wheel path\n --clean-images Remove Docker images after run (default: keep them)\n --keep-outputs Preserve prediction outputs and print the work directory path\n\nRun options:\n --output {console,json} Output format\n --output-file PATH Write JSON report to file\n```\n\n## Project Layout\n\n```text\ntools/test-harness/\n├── go.mod\n├── main.go\n├── README.md\n├── manifest.yaml\n├── fixtures/\n├── cmd/\n└── internal/\n```\n","isInternal":false,"tokens":641,"sizeBytes":2533},{"name":"README.md","path":"tools/test-registry-util/README.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-registry-util/README.md","title":"test-registry-util Documentation","category":"plugin-manifest","format":"markdown","content":"# test-registry-util Documentation\n\nPath: `tools/test-registry-util/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-registry-util/README.md)","tokens":300,"sizeBytes":197},{"name":"reviewer.md","path":".opencode/agents/reviewer.md","rawUrl":"https://raw.githubusercontent.com/replicate/cog/HEAD/.opencode/agents/reviewer.md","title":"Subagent: reviewer","category":"subagent-persona","format":"markdown","content":"---\ndescription: Read-only code reviewer for pull requests\nmode: primary\nmodel: cf-gateway/deepseek-v4-flash-0731\ntemperature: 0.1\npermission:\n edit: deny\n bash:\n \"*\": deny\n \"git diff*\": allow\n \"git log*\": allow\n \"git show*\": allow\n webfetch: deny\n---\n\nYou are a code reviewer for the Cog project. Cog packages ML models into\nproduction-ready containers. The codebase has three languages: Go (CLI and\ntooling), Python (SDK), and Rust (coglet prediction server).\n\n## How to review\n\n1. Read the full diff. Understand what the PR is trying to do before commenting.\n2. Load the appropriate review skill(s) for the languages touched in the PR:\n - Go files: load `go-review`\n - Python files: load `python-review`\n - Rust files: load `rust-review`\n - Architecture/cross-cutting: load `cog-review`\n3. Focus on substance, not style. Linters handle formatting.\n4. Scope feedback to the PR's purpose. Don't demand unrelated refactors.\n5. Be direct. Say what the problem is, why it matters, and how to fix it.\n\n## Comment format\n\n- Reference specific files and lines: `path/to/file.go:42`\n- Use ```suggestion blocks when proposing concrete code changes\n- Categorize findings:\n - **Blocker**: Must fix before merge (bugs, security, data loss)\n - **Should fix**: Important but not blocking (error handling gaps, edge cases)\n - **Nit**: Minor improvement, take it or leave it\n- If the PR looks good, say so briefly. Don't invent problems.\n","frontmatter":{"description":"Read-only code reviewer for pull requests","mode":"primary","model":"cf-gateway/deepseek-v4-flash-0731","temperature":0.1,"permission":{"edit":"deny","bash":{"*":"deny","git diff*":"allow","git log*":"allow","git show*":"allow"},"webfetch":"deny"}},"isInternal":false,"tokens":372,"sizeBytes":1450}],"systemPromptSnippet":"\n\n# AGENTS.md\n\nThis file provides guidance to coding agents when working with code in this repository.\n\n## Project Overview\n\nCog is a tool that packages machine learning models in production-ready containers.\n\nIt consists of:\n\n- **Cog CLI** (`cmd/cog/`) - Command-line interface for building, running, and deploying models, written in Go\n- **Python SDK** (`python/cog/`) - Python library for defining model predictors and training in Python\n- **Coglet** (`crates/`) - Rust-based prediction server that runs inside containers, with Python bindings via PyO3\n\nDocumentation for the CLI and SDK is available by reading ./docs/llms.txt.\n\n## Development Commands\n\nDevelopment tasks are managed with [mise](https://mise.jdx.dev/). Run `mise tasks` to see all available tasks.\n\n### Quick Reference\n\n| Task | Description |\n| ----------------------------- | ------------------------------------------------------------ |\n| `mise run fmt` | Check formatting (all languages) |\n| `mise run fmt:fix` | Fix formatting (all languages) |\n| `mise run lint` | Run linters (all languages) |\n| `mise run lint:fix` | Fix lint issues (all languages) |\n| `mise run test:go` | Run Go tests |\n| `mise run test:rust` | Run Rust tests |\n| `mise run test:python` | Run Python tests |\n| `mise run test:integration` | Run integration tests |\n| `mise run build:cog` | Build cog CLI binary |\n| `mise run build:coglet` | Build coglet wheel (dev) |\n| `mise run build:sdk` | Build SDK wheel |\n| `mise run install` | Build and symlink cog to /usr/local/bin |\n| `mise run docs:llm` | **IMPORTANT:** Regenerate `docs/llms.txt` after editing docs |\n| `mise run docs:cli` | Generate CLI reference docs from Go source code |\n| `mise run version` | Show current version from VERSION.txt |\n| `mise run version:bump ` | Bump version everywhere and commit |\n\n### Task Naming Convention\n\nTasks follow a consistent naming pattern:\n\n- **Language-based tasks** for fmt/lint/test/typecheck: `task:go`, `task:rust`, `task:python`\n- **Component-based tasks** for build: `build:cog`, `build:coglet`, `build:sdk`\n- **Check vs Fix**: `fmt` and `lint` default to check mode (non-destructive); use `:fix` suffix to auto-fix\n\n### All Tasks by Category\n\n**Format:**\n\n- `mise run fmt` / `mise run fmt:check` - Check all (alias)\n- `mise run fmt:fix` - Fix all\n- `mise run fmt:go` / `mise run fmt:rust` / `mise run fmt:python` - Per-language\n\n**Lint:**\n\n- `mise run lint` / `mise run lint:check` - Check all (alias)\n- `mise run lint:fix` - Fix all\n- `mise run lint:go` / `mise run lint:rust` / `mise run lint:python` - Per-language\n- `mise run lint:rust:deny` - Check Rust licenses/advisories\n\n**Test:**\n\n- `mise run test:go` - Go unit tests\n- `mise run test:rust` - Rust unit tests\n- `mise run test:python` - Python unit tests (via tox)\n- `mise run test:coglet:python` - Coglet Python binding tests\n- `mise run test:integration` - Integration tests\n\n**Build:**\n\n- `mise run build:cog` - Build cog CLI (development)\n- `mise run build:cog:release` - Build cog CLI (release)\n- `mise run build:coglet` - Build coglet wheel (dev install)\n- `mise run build:coglet:wheel` - Build coglet wheel (native platform)\n- `mise run build:coglet:wheel:linux-x64` - Build for Linux x86_64\n- `mise run build:coglet:wheel:linux-arm64` - Build for Linux ARM64\n- `mise run build:sdk` - Build SDK wheel\n\n**Install:**\n\n- `mise run install` - Symlink cog CLI to `/usr/local/bin` (requires `build:cog` first)\n- `PREFIX=/custom/path mise run install` - Symlink to custom location\n\n**Version:**\n\n- `mise run version` - Show current version from VERSION.txt\n- `mise run version:bump ` - Bump version everywhere and commit\n- `mise run version:check` - Verify VERSION.txt matches Cargo.toml\n\n**Other:**\n\n- `mise run typecheck` - Type check all languages\n- `mise run generate` - Run code generation\n- `mise run clean` - Clean all build artifacts\n- `mise run docs` - Build documentation\n- `mise run docs:serve` - Serve docs locally\n\n## Code Style Guidelines\n\n### Go\n\n- **Imports**: Organize in three groups separated by blank lines: (1) Standard library, (2) Third-party packages, (3) Internal packages (`github.com/replicate/cog/pkg/...`)\n- **Formatting**: Use `mise run fmt:go:fix`\n- **Linting**: Must pass golangci-lint with: errcheck, gocritic, gosec, govet, ineffassign, misspell, revive, staticcheck, unused\n- **Error Handling**: Return errors as values; use `pkg/errors.CodedError` for user-facing errors with error codes\n- **Naming**: CamelCase for exported, camelCase for unexported\n- **Testing**: Use `testify/require` for assertions; prefer table-driven tests\n\nExample import block:\n\n```go\nimport (\n \"fmt\"\n\n \"github.com/spf13/cobra\"\n\n \"github.com/replicate/cog/pkg/config\"\n)\n```\n\n### Python\n\n- **Imports**: Automatically organized by ruff/isort (stdlib → third-party → local)\n- **Formatting**: Use `mise run fmt:python:fix`\n- **Linting**: Must pass ruff checks: E (pycodestyle), F (Pyflakes), I (isort), W (warnings), S (bandit), B (bugbear), ANN (annotations)\n- **Type Annotations**: Required on all function signatures; use `typing_extensions` for compatibility; avoid `Any` where possible\n- **Error Handling**: Raise exceptions with descriptive messages; avoid generic exception catching\n- **Naming**: snake_case for functions/variables/modules, PascalCase for classes\n- **Testing**: Use pytest with fixtures; async tests with pytest-asyncio\n- **Compatibility**: Must support Python 3.10-3.13\n\n### Rust\n\n- **Formatting**: Use `mise run fmt:rust:fix`\n- **Linting**: Must pass `mise run lint:rust` (clippy)\n- **Dependencies**: Audited with `cargo-deny` (see `crates/deny.toml`); run `mise run lint:rust:deny`\n- **Error Handling**: Use `thiserror` for typed errors, `anyhow` for application errors\n- **Naming**: snake_case for functions/variables, PascalCase for types\n- **Testing**: Use `cargo test`; snapshot tests use `insta`\n- **Async**: tokio runtime; async/await patterns\n\n## Working on the CLI and support tooling\n\nThe CLI code is in the `cmd/cog/` and `pkg/` directories. Support tooling is in the `tools/` directory.\n\nThe main commands for working on the CLI are:\n\n- `go run ./cmd/cog` - Runs the Cog CLI directly from source (requires wheel to be built first)\n- `mise run build:cog` - Builds the Cog CLI binary\n- `mise run install` - Symlinks the built binary to `/usr/local/bin` (run `build:cog` first), or to a custom path with `PREFIX=/custom/path mise run install`\n- `mise run test:go` - Runs all Go unit tests\n- `go test ./pkg/...` - Runs tests directly with `go test`\n\n## Working on the Python SDK\n\nThe Python SDK is developed in the `python/cog/` directory. It uses `uv` for virtual environments and `tox` for testing across multiple Python versions.\n\nThe main commands for working on the SDK are:\n\n- `mise run build:sdk` - Builds the Python wheel\n- `mise run test:python` - Runs Python tests across all supported versions\n\n## Working on Coglet (Rust)\n\nCoglet is the Rust-based prediction server that runs inside Cog containers, handling HTTP requests, worker process management, and prediction execution.\n\nThe code is in the `crates/` directory:\n\n- `crates/coglet/` - Core Rust library (HTTP server, worker orchestration, IPC)\n- `crates/coglet-python/` - PyO3 bindings for Python predictor integration (requires Python 3.10+)\n\nFor detailed architecture documentation, see `crates/README.md` and `crates/coglet/README.md`.\n\nThe main commands for working on Coglet are:\n\n- `mise run build:coglet` - Build and install coglet wheel for development (macOS, for local Rust/Python tests)\n- `mise run build:coglet:wheel:linux-x64` - Build Linux x86_64 wheel (required to test Rust changes in Docker containers via `cog predict`/`cog train`)\n- `mise run test:rust` - Run Rust unit tests\n- `mise run lint:rust` - Run clippy linter\n- `mise run fmt:rust:fix` - Format code\n\n### Testing\n\nGo code is tested using the built-in `go test` framework:\n\n- `go test ./pkg/... -run ` - Runs specific Go tests by name\n- `mise run test:go` - Runs all Go unit tests\n\nPython code is tested using `tox`, which allows testing across multiple Python versions and configurations:\n\n- `mise run test:python` - Runs all Python unit tests\n- `uv run tox -e py312-tests -- python/tests/server/test_http.py::test_openapi_specification_with_yield` - Runs a specific Python test\n\nThe integration test suite in `integration-tests/` tests the end-to-end functionality of the Cog CLI and Python SDK using Go's testscript framework:\n\n- `mise run test:integration` - Runs the integration tests\n- `mise run test:integration string_predictor` - Runs a specific integration test\n\nThe integration tests require a built Cog binary, which defaults to the first `cog` in `PATH`. Run tests against a specific binary with the `COG_BINARY` environment variable:\n\n```bash\nmise run build:cog\nCOG_BINARY=dist/go/*/cog mise run test:integration\n```\n\n### Development Workflow\n\n1. Run `mise install` to set up the development environment\n2. Run `mise run build:sdk` after making changes to the `./python` directory\n3. Run `mise run build:coglet:wheel:linux-x64` after making changes to the `./crates` directory (needed for Docker testing)\n4. Run `mise run build:cog` to build the CLI (wheels are picked up from `dist/` at Docker build time, not embedded in the binary)\n5. Run `mise run fmt:fix` to format code\n6. Run `mise run lint` to check code quality\n7. Run `mise run docs:llm` to regenerate `docs/llms.txt` after changing `README.md` or any `docs/*.md` file\n8. Read the `./docs` directory and make sure the documentation is up to date\n\n**IMPORTANT:** Always run `mise run lint` (or the language-specific variant, e.g. `mise run lint:go`) before committing to catch linter errors early. CI will reject PRs that fail lint checks.\n\n## Architecture\n\n### CLI Architecture (Go)\n\nThe CLI follows a command pattern with subcommands. The main components are:\n\n- `pkg/cli/` - Command definitions (build, exec, predict, serve, etc.)\n- `pkg/docker/` - Docker client and container management\n- `pkg/dockerfile/` - Dockerfile generation and templating\n- `pkg/config/` - cog.yaml parsing and validation\n- `pkg/image/` - Image building and pushing logic\n\n### Python SDK Architecture\n\n- `python/cog/` - Core SDK\n - `base_predictor.py` - Base class for model predictors\n - `types.py` - Input/output type definitions\n - `server/` - HTTP/queue server implementation\n - `command/` - Runner implementations for predict/train\n\n### Coglet Architecture (Rust)\n\nThe prediction server that runs inside Cog containers. Uses a two-process architecture: a parent process (HTTP server + orchestrator) and a worker subprocess (Python predictor execution).\n\nSee `crates/README.md` for detailed architecture documentation.\n\n- `crates/coglet/` - Core Rust library (HTTP server, worker orchestration, IPC bridge)\n- `crates/coglet-python/` - PyO3 bindings for Python predictor integration\n\n### Key Design Patterns\n\n1. **Local Wheel Resolution**: The CLI discovers SDK and coglet wheels from `dist/` at Docker build time (not embedded in the binary)\n2. **Docker SDK Integration**: Uses Docker Go SDK for container operations\n3. **Type Safety**: Dataclasses for Python type validation, strongly typed Go interfaces\n4. **Compatibility Matrix**: Automated CUDA/PyTorch/TensorFlow compatibility management\n\nFor comprehensive architecture documentation, see [`architecture/`](./architecture/00-overview.md).\n\n## Common Tasks\n\n### Adding a new CLI command\n\n1. Create command file in `pkg/cli/`\n2. Add command to `pkg/cli/root.go`\n3. Implement business logic in appropriate `pkg/` subdirectory\n4. Add tests\n\n### Modifying Python SDK behavior\n\n1. Edit files in `python/cog/`\n2. Run `mise run build:sdk` to rebuild wheel\n3. Test with `mise run test:python`\n4. Integration test with `mise run test:integration`\n\n### Updating ML framework compatibility\n\n1. See `tools/compatgen/` for compatibility matrix generation\n2. Update framework versions in relevant Dockerfile templates\n3. Test with various framework combinations\n\n### Updating the docs\n\n- Documentation is in the `docs/` directory, written in Markdown and generated into HTML using `mkdocs`.\n- **IMPORTANT:** After editing any file in `docs/` or `README.md`, you MUST run `mise run docs:llm` to regenerate `docs/llms.txt`. This file is used by coding agents and should be kept in sync with the documentation.\n- **IMPORTANT:** CLI reference docs (`docs/cli.md`) are auto-generated from Go source code. After modifying CLI commands in `cmd/` or `pkg/cli/`, run `mise run docs:cli` to regenerate, and ensure `mise run docs:cli:check` passes before committing.\n\n## CI Tool Dependencies\n\nDevelopment tools are managed in **one place**: **`mise.toml`**. CI workflows use\n`jdx/mise-action@v4` (with caching enabled) to install the same tool versions that\ndevelopers use locally. This eliminates version drift between local dev and CI.\n\n**Exceptions:**\n\n| Tool | CI method | Why |\n| -------------------------- | ---------------------- | ---------------------------------------------------------------------- |\n| coglet wheel (maturin+zig) | `PyO3/maturin-action` | Builds inside a manylinux container with bundled maturin and zig |\n| Rust target cache | `Swatinem/rust-cache` | Caches `crates/target/` directory (separate from tool caching) |\n| `check-stubs` Python | `actions/setup-python` | Needs shared-library Python (libpython3.x.so) for PyO3 auto-initialize |\n\n**When updating a tool version**, update `mise.toml` only. CI picks it up automatically.\n\n## Agent Skills\n\nThe `.agents/skills/` directory contains shared skill definitions that provide specialized instructions and workflows for coding agents. Each skill is a subdirectory with a `SKILL.md` file that agents load on demand when the task matches.\n\nSkills are invoked automatically by agents when a matching task is detected. See individual `SKILL.md` files for details on when and how each skill applies.\n\n## Important Files\n\n- `VERSION.txt` - Canonical version (single source of truth)\n- `cog.yaml` - User-facing model configuration\n- `pkg/config/config.go` - Go code for parsing and validating `cog.yaml`\n- `pkg/config/data/config_schema_v1.0.json` - JSON schema for `cog.yaml`\n- `python/cog/base_predictor.py` - Predictor interface\n- `crates/Cargo.toml` - Rust workspace configuration (version must match VERSION.txt)\n- `crates/README.md` - Coglet architecture overview\n- `mise.toml` - Task definitions for development workflow\n\n## Testing Philosophy\n\n- Unit tests for individual components (Go and Python)\n- Integration tests for end-to-end workflows\n- Tests use real Docker operations (no mocking Docker API)\n- Always run `mise run build:sdk` after making Python changes before testing Go code\n- Python 3.10-3.13 compatibility is required\n\n### Go Test Conventions\n\nAll Go tests must use [testify](https://github.com/stretchr/testify) for assertions. Do **not** use raw `if` checks with `t.Fatal`/`t.Errorf` — use `require` and `assert` instead.\n\n- **`require`** — for fatal assertions that should stop the test (setup failures, preconditions):\n ```go\n require.NoError(t, err, \"failed to create client\")\n require.Equal(t, expected, actual)\n require.True(t, condition, \"server should be ready\")\n ```\n- **`assert`** — for non-fatal checks where the test should continue (e.g. validating multiple fields in a loop):\n ```go\n assert.Equal(t, http.StatusOK, resp.StatusCode)\n assert.Contains(t, output, \"expected substring\")\n assert.NoError(t, err, \"prediction %d failed\", i)\n ```\n- Use `require` for errors in setup/teardown and `assert` for the actual test expectations\n- Prefer specific assertions (`Equal`, `Contains`, `NoError`, `Len`, `Less`) over generic `True`/`False` — they produce better failure messages\n- Prefer table-driven tests for testing multiple similar cases\n\n\n# Cog: Containers for machine learning\n\nCog is an open-source tool that lets you package machine learning models in a standard, production-ready container.\n\nYou can deploy your packaged model to your own infrastructure, or to [Replicate](https://replicate.com/).\n\n## Highlights\n\n- 📦 **Docker containers without the pain.** Writing your own `Dockerfile` can be a bewildering process. With Cog, you define your environment with a [simple configuration file](#how-it-works) and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.\n\n- 🤬️ **No more CUDA hell.** Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.\n\n- ✅ **Define the inputs and outputs for your model with standard Python.** Then, Cog generates an OpenAPI schema and validates the inputs and outputs.\n\n- 🎁 **Automatic HTTP inference server**: Your model's types are used to dynamically generate a RESTful HTTP API using a high-performance Rust/Axum server.\n\n- 🚀 **Ready for production.** Deploy your model anywhere that Docker images run. Your own infrastructure, or [Replicate](https://replicate.com).\n\n## How it works\n\nDefine the Docker environment your model runs in with `cog.yaml`:\n\n```yaml\nbuild:\n gpu: true\n system_packages:\n - \"libgl1\"\n - \"libglib2.0-0\"\n python_version: \"3.13\"\n python_requirements: requirements.txt\nrun: \"run.py:Runner\"\n```\n\nDefine how your model runs with `run.py`:\n\n```python\nfrom cog import BaseRunner, Input, Path\nimport torch\n\nclass Runner(BaseRunner):\n def setup(self):\n \"\"\"Load the model into memory to make running multiple inferences efficient\"\"\"\n self.model = torch.load(\"./weights.pth\")\n\n # The arguments and types the model takes as input\n def run(self,\n image: Path = Input(description=\"Grayscale input image\")\n ) -> Path:\n \"\"\"Run the model\"\"\"\n processed_image = preprocess(image)\n output = self.model(processed_image)\n return postprocess(output)\n```\n\nIn the above we accept a path to the image as an input, and return a path to our transformed image after running it through our model.\n\nNow, you can run the model:\n\n```console\n$ cog run -i image=@input.jpg\n--> Building Docker image...\n--> Running...\n--> Output written to output.jpg\n```\n\nOr, build a Docker image for deployment:\n\n```console\n$ cog build -t my-classification-model\n--> Building Docker image...\n--> Built my-classification-model:latest\n\n$ docker run -d -p 5000:5000 --gpus all my-classification-model\n\n$ curl http://localhost:5000/predictions -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"image\": \"https://.../input.jpg\"}}'\n```\n\nOr, combine build and run via the `serve` command:\n\n```console\n$ cog serve -p 8080\n\n$ curl http://localhost:8080/predictions -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"image\": \"https://.../input.jpg\"}}'\n```\n\n\n\n## Why are we building this?\n\nIt's really hard for researchers to ship machine learning models to production.\n\nPart of the solution is Docker, but it is so complex to get it to work: Dockerfiles, pre-/post-processing, Flask servers, CUDA versions. More often than not the researcher has to sit down with an engineer to get the damn thing deployed.\n\n[Andreas](https://github.com/andreasjansson) and [Ben](https://github.com/bfirsh) created Cog. Andreas used to work at Spotify, where he built tools for building and deploying ML models with Docker. Ben worked at Docker, where he created [Docker Compose](https://github.com/docker/compose).\n\nWe realized that, in addition to Spotify, other companies were also using Docker to build and deploy machine learning models. [Uber](https://eng.uber.com/michelangelo-pyml/) and others have built similar systems. So, we're making an open source version so other people can do this too.\n\nHit us up if you're interested in using it or want to collaborate with us. [We're on Discord](https://discord.gg/replicate) or email us at [team@replicate.com](mailto:team@replicate.com).\n\n## Prerequisites\n\n- **macOS, Linux or Windows 11**. Cog works on macOS, Linux and Windows 11 with [WSL 2](docs/wsl2/wsl2.md)\n- **Docker**. Cog uses Docker to create a container for your model. You'll need to [install Docker](https://docs.docker.com/get-docker/) before you can run Cog. If you install Docker Engine instead of Docker Desktop, you will need to [install Buildx](https://docs.docker.com/build/architecture/#buildx) as well.\n\n## Install\n\nChoose your platform for installation instructions.\n\n
\n macOS\n\nThe easiest way to install Cog on macOS is with Homebrew:\n\n```console\nbrew install replicate/tap/cog\n```\n\nYou can also use the install script:\n\n```sh\n# bash, zsh, and other shells\nsh <(curl -fsSL https://cog.run/install.sh)\n\n# fish shell\nsh (curl -fsSL https://cog.run/install.sh | psub)\n\n# download with wget and run in a separate command\nwget -qO- https://cog.run/install.sh\nsh ./install.sh\n```\n\nOr install manually:\n\n```console\nsudo curl -o /usr/local/bin/cog -L \"https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')\"\nsudo chmod +x /usr/local/bin/cog\nsudo xattr -d com.apple.quarantine /usr/local/bin/cog 2>/dev/null || true\n```\n\nIf you see a Gatekeeper warning saying the binary \"cannot be opened because the developer cannot be verified\", run:\n\n```console\nsudo xattr -d com.apple.quarantine /usr/local/bin/cog\n```\n\n
\n\n
\n Linux\n\nYou can install Cog using the install script:\n\n```sh\n# bash, zsh, and other shells\nsh <(curl -fsSL https://cog.run/install.sh)\n\n# fish shell\nsh (curl -fsSL https://cog.run/install.sh | psub)\n\n# download with wget and run in a separate command\nwget -qO- https://cog.run/install.sh\nsh ./install.sh\n```\n\nOr install manually:\n\n```console\nsudo curl -o /usr/local/bin/cog -L \"https://github.com/replicate/cog/releases/latest/download/cog_$(uname -s)_$(uname -m | sed 's/aarch64/arm64/')\"\nsudo chmod +x /usr/local/bin/cog\n```\n\n
\n\n
\n Windows\n\nCog does not natively support Windows, but you can run it on Windows 11 using [WSL 2](docs/wsl2/wsl2.md). Once WSL 2 is set up, follow the Linux installation instructions above.\n\n
\n\n
\n Docker\n\nTo install Cog inside a Docker image:\n\n```dockerfile\nRUN sh -c \"INSTALL_DIR=\\\"/usr/local/bin\\\" SUDO=\\\"\\\" $(curl -fsSL https://cog.run/install.sh)\"\n```\n\n
\n\n## Upgrade\n\nIf you're using macOS and you previously installed Cog with Homebrew, run the following:\n\n```console\nbrew upgrade replicate/tap/cog\n```\n\nOtherwise, you can upgrade to the latest version by running the same commands you used to install it.\n\n## Development\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for how to set up a development environment and build from source.\n\n## Next steps\n\n- [Get started with an example model](docs/getting-started.md)\n- [Get started with your own model](docs/getting-started-own-model.md)\n- [Using Cog with notebooks](docs/notebooks.md)\n- [Using Cog with Windows 11](docs/wsl2/wsl2.md)\n- [Browse the example models in this repo](docs/examples.md)\n- [Deploy models with Cog](docs/deploy.md)\n- [`cog.yaml` reference](docs/yaml.md) to learn how to define your model's environment\n- [Run interface reference](docs/python.md) to learn how the `Runner` interface works\n- [Training interface reference](docs/training.md) to learn how to add a fine-tuning API to your model\n- [HTTP API reference](docs/http.md) to learn how to use the HTTP API that models serve\n\n## Need help?\n\n[Join us in #cog on Discord.](https://discord.gg/replicate)\n\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/replicate/cog)\n\n\n\n---\n\n# Base images\n\nCog builds your model into a Docker image. To speed up builds and reduce cold boot times, Cog uses **prebuilt base images** by default. These images contain the common dependencies that most models need, so Cog doesn't have to install them from scratch every time you build or push.\n\n## What is a Cog base image?\n\nA Cog base image is a Docker image maintained by Replicate that includes:\n\n- **Python runtime** — the Python version specified in your `cog.yaml`.\n- **System libraries** — common dependencies for machine learning and media processing, including `ffmpeg`, `git`, `curl`, `build-essential`, `cmake`, OpenCV libraries, audio libraries (`sox`, `libsndfile1`), and graphics libraries (`libgl1`, `libsm6`, `libxext6`).\n- **CUDA and PyTorch** (GPU images only) — the appropriate CUDA toolkit and PyTorch stack for your configuration.\n\nBase images are tagged by their configuration, for example:\n\n```\nr8.im/cog-base:cuda12.4-python3.13-torch2.6.0\nr8.im/cog-base:python3.13\n```\n\nWhen you run `cog build` or `cog push`, Cog selects the base image that matches your Python version, CUDA version, and PyTorch version. Because these images are pre-pulled on Replicate's infrastructure, models built on top of them start faster.\n\nThe entire list of available base images can be viewed at [https://r8.im/v2/cog-base/tags/list](https://r8.im/v2/cog-base/tags/list).\n\n## Using the Cog base image\n\nThe `--use-cog-base-image` flag controls whether Cog uses a prebuilt base image. It is **enabled by default** on the following commands:\n\n- [`cog build`](cli.md#cog-build)\n- [`cog push`](cli.md#cog-push)\n- [`cog run`](cli.md#cog-run)\n- [`cog serve`](cli.md#cog-serve)\n- [`cog exec`](cli.md#cog-exec)\n\nSince it's on by default, you don't need to pass any flags:\n\n```bash\ncog push r8.im/your-username/my-model\n```\n\nThis builds and pushes your model using a prebuilt Cog base image for faster cold boots.\n\n## Disabling the Cog base image\n\nIf you encounter build issues or need a custom base layer, you can disable the Cog base image:\n\n```bash\ncog build --use-cog-base-image=false\n```\n\nWhen disabled, Cog generates a Dockerfile from scratch using either an NVIDIA CUDA base image or a slim Python base image, depending on the `--use-cuda-base-image` flag.\n\n## Relationship to `--use-cuda-base-image`\n\nThe `--use-cuda-base-image` flag controls which underlying base image Cog uses **when the Cog base image is disabled**. It has no effect when `--use-cog-base-image` is enabled (the default), because the Cog base image already includes the appropriate CUDA and Python stack.\n\nWhen you disable the Cog base image with `--use-cog-base-image=false`, Cog chooses the base image automatically:\n\n- **GPU models** (`gpu: true` in `cog.yaml`): uses an NVIDIA CUDA base image.\n- **CPU models**: uses a slim Python base image.\n\nThese flags — along with `--dockerfile` — are **mutually exclusive**: you can only set one of them explicitly on a given command. To customize the base image, disable the Cog base image and let Cog choose between CUDA and Python automatically.\n\n## Troubleshooting\n\nIf `cog build` or `cog push` fails with the Cog base image enabled, try disabling it:\n\n```bash\ncog push --use-cog-base-image=false r8.im/your-username/my-model\n```\n\nThis falls back to building from a standard CUDA or Python base image, which can help diagnose whether the issue is with the base image or your model's configuration.\n\n\n---\n\n# CLI reference\n\n\n\n## `cog`\n\nContainers for machine learning.\n\nTo get started, take a look at the documentation:\nhttps://github.com/replicate/cog\n\n**Examples**\n\n```\n To execute a command inside a Docker environment defined with Cog:\n $ cog exec echo hello world\n```\n\n**Options**\n\n```\n --debug Show debugging output\n -h, --help help for cog\n --no-color Disable colored output\n --version Show version of Cog\n```\n\n## `cog build`\n\nBuild a Docker image from the cog.yaml in the current directory.\n\nThe generated image contains your model code, dependencies, and the Cog\nruntime. It can be run locally with 'cog run' or pushed to a registry\nwith 'cog push'.\n\n```\ncog build [flags]\n```\n\n**Examples**\n\n```\n # Build with default settings\n cog build\n\n # Build and tag the image\n cog build -t my-model:latest\n\n # Build without using the cache\n cog build --no-cache\n\n # Build with model weights in a separate layer\n cog build --separate-weights -t my-model:v1\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n -h, --help help for build\n --no-cache Do not use cache when building the image\n --openapi-schema string Load OpenAPI schema from a file\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'\n --separate-weights Separate model weights from code in image layers\n -t, --tag string A name for the built image in the form 'repository:tag'\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog doctor`\n\nDiagnose and fix common issues in your Cog project.\n\nNOTE: cog doctor is experimental. Behavior and checks may change in future versions.\n\nBy default, cog doctor reports problems without modifying any files.\nPass --fix to automatically apply safe fixes.\n\n```\ncog doctor [flags]\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --fix Automatically apply fixes\n -h, --help help for doctor\n```\n\n## `cog exec`\n\nExecute a command inside a Docker environment defined by cog.yaml.\n\nCog builds a temporary image from your cog.yaml configuration and runs the\ngiven command inside it. This is useful for debugging, running scripts, or\nexploring the environment your model will run in.\n\n```\ncog exec [arg...] [flags]\n```\n\n**Examples**\n\n```\n # Open a Python interpreter inside the model environment\n cog exec python\n\n # Run a script\n cog exec python train.py\n\n # Run with environment variables\n cog exec -e HUGGING_FACE_HUB_TOKEN=abc123 python download.py\n\n # Expose a port (e.g. for Jupyter)\n cog exec -p 8888 jupyter notebook\n```\n\n**Options**\n\n```\n -e, --env stringArray Environment variables, in the form name=value\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for exec\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n -p, --publish stringArray Publish a container's port to the host, e.g. -p 8000 or -p 0.0.0.0:8000\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog init`\n\nCreate a cog.yaml and run.py in the current directory.\n\nThese files provide a starting template for defining your model's environment\nand run interface. Edit them to match your model's requirements.\n\n```\ncog init [flags]\n```\n\n**Examples**\n\n```\n # Set up a new Cog project in the current directory\n cog init\n```\n\n**Options**\n\n```\n -h, --help help for init\n```\n\n## `cog login`\n\nLog in to a container registry.\n\nFor Replicate's registry (r8.im), this command handles authentication\nthrough Replicate's token-based flow.\n\nFor other registries, this command prompts for username and password,\nthen stores credentials using Docker's credential system.\n\n```\ncog login [flags]\n```\n\n**Options**\n\n```\n -h, --help help for login\n --token-stdin Pass login token on stdin instead of opening a browser. You can find your Replicate login token at https://replicate.com/auth/token\n```\n\n## `cog playground`\n\nOpen a browser playground for talking to a running model.\n\nStarts a local web server that serves a schema-driven UI (a Postman-like tool\nfor Cog models). Point it at any running Cog HTTP API -- for example one started\nwith 'cog serve' -- and the playground reflects that model's inputs and outputs\nfrom its OpenAPI schema in real time.\n\nRequests are reverse-proxied through this server, so the target API does not\nneed to set CORS headers. The server also hosts a webhook sink so async\npredictions can be observed in the browser.\n\nAsync/webhook testing against a containerized model requires the webhook URL to\nbe reachable from inside the container. On Docker Desktop the default\n'host.docker.internal' works once the server listens on a reachable interface\n(e.g. --host 0.0.0.0).\n\n```\ncog playground [flags]\n```\n\n**Examples**\n\n```\n # Start a model API in one terminal\n cog serve -p 8393\n\n # Open the playground pointing at it\n cog playground --target http://localhost:8393\n```\n\n**Options**\n\n```\n -h, --help help for playground\n --host string Address to bind (use 0.0.0.0 to receive webhooks from containers) (default \"127.0.0.1\")\n --no-open Do not open the browser automatically\n -p, --port int Port to listen on (0 picks a free port)\n --target string Default target model API URL (default \"http://localhost:8393\")\n --webhook-host string Hostname the model uses to reach this server for webhooks (default \"host.docker.internal\")\n```\n\n## `cog push`\n\nBuild from cog.yaml and push to an OCI-compliant registry. Run 'cog login'\nfirst when pushing to Replicate's registry (r8.im).\n\nTARGET overrides the configured destination and all COG_MODEL\\* environment\nvariables. It doesn't change the project format: projects configured with\n'image' push an image, while projects configured with 'model' push an OCI\nbundle. Push targets must use tags, not digests. Untagged image targets use\nDocker's default 'latest' tag; untagged bundle targets get a timestamp tag.\n\nWith --json, Cog writes one versioned JSON result to stdout after the entire\npush succeeds. Progress, warnings, and diagnostics continue on stderr. Every\nreference in the result is digest-pinned.\n\n```\ncog push [TARGET] [flags]\n```\n\n**Examples**\n\n```\n # Push an image to Replicate\n cog push r8.im/your-username/my-model\n\n # Push an image to any OCI registry\n cog push registry.example.com/your-username/model-name\n\n # Push a bundle project and print its immutable references as JSON\n cog push registry.example.com/your-username/model-name:v1 --json\n\n # Push with model weights in a separate image layer (Replicate only)\n cog push r8.im/your-username/my-model --separate-weights\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n -h, --help help for push\n --json Output the pushed references as JSON\n --no-cache Do not use cache when building the image\n --openapi-schema string Load OpenAPI schema from a file\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --secret stringArray Secrets to pass to the build environment in the form 'id=foo,src=/path/to/file'\n --separate-weights Separate model weights from code in image layers\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n## `cog run`\n\nRun the model.\n\nIf 'image' is passed, it will run the model on that Docker image.\nIt must be an image that has been built by Cog.\n\nOtherwise, it will build the model in the current directory and run\nit.\n\n```\ncog run [image] [flags]\n```\n\n**Examples**\n\n```\n # Run the model with named inputs\n cog run -i prompt=\"a photo of a cat\"\n\n # Pass a file as input\n cog run -i image=@photo.jpg\n\n # Save output to a file\n cog run -i image=@input.jpg -o output.png\n\n # Pass multiple inputs\n cog run -i prompt=\"sunset\" -i width=1024 -i height=768\n\n # Run against a pre-built image\n cog run r8.im/your-username/my-model -i prompt=\"hello\"\n\n # Pass inputs as JSON\n echo '{\"prompt\": \"a cat\"}' | cog run --json @-\n```\n\n**Options**\n\n```\n -e, --env stringArray Environment variables, in the form name=value\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for run\n -i, --input stringArray Inputs, in the form name=value. if value is prefixed with @, then it is read from a file on disk. E.g. -i path=@image.jpg\n --json string Pass inputs as JSON object, read from file (@inputs.json) or via stdin (@-)\n -o, --output string Output path\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --setup-timeout uint32 The timeout for a container to setup (in seconds). (default 300)\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n --use-replicate-token Pass REPLICATE_API_TOKEN from local environment into the model context\n```\n\n## `cog serve`\n\nRun an HTTP server.\n\nBuilds the model and starts an HTTP server that exposes the model's inputs\nand outputs as a REST API. Compatible with the Cog HTTP protocol.\n\nBy default the container port is published on 127.0.0.1 (localhost), so the\nserver is only reachable from your local machine. The server process inside\nthe container binds to 0.0.0.0; use --host to control which host interface\nthe Docker port mapping is published on.\n\n```\ncog serve [flags]\n```\n\n**Examples**\n\n```\n # Start the server on the default port (8393)\n cog serve\n\n # Start on a custom port\n cog serve -p 5000\n\n # Start the playground alongside the model\n cog serve --playground\n\n # Listen on all interfaces (e.g. to expose to the network)\n cog serve --host 0.0.0.0\n\n # Test the server\n curl http://localhost:8393/predictions \\\n -X POST \\\n -H 'Content-Type: application/json' \\\n -d '{\"input\": {\"prompt\": \"a cat\"}}'\n```\n\n**Options**\n\n```\n -f, --file string The name of the config file. (default \"cog.yaml\")\n --gpus docker run --gpus GPU devices to add to the container, in the same format as docker run --gpus.\n -h, --help help for serve\n --host string Host IP to publish the container port on. Use 0.0.0.0 to allow connections from other machines. (default \"127.0.0.1\")\n --playground Start the playground alongside the model\n --playground-port int Port for the playground (0 picks a free port) (default 9000)\n -p, --port int Port on which to listen (default 8393)\n --progress string Set type of build progress output, 'auto' (default), 'tty', 'plain', or 'quiet' (default \"auto\")\n --upload-url string Upload URL for file outputs (e.g. https://example.com/upload/)\n --use-cog-base-image Use pre-built Cog base image for faster cold boots (default true)\n --use-cuda-base-image string Use Nvidia CUDA base image, 'true' (default) or 'false' (use python base image). False results in a smaller image but may cause problems for non-torch projects (default \"auto\")\n```\n\n\n---\n\n# Deploy models with Cog\n\nCog containers are Docker containers that serve an HTTP server\nfor running your model.\nYou can deploy them anywhere that Docker containers run.\n\nThe server inside Cog containers is **coglet**, a Rust-based inference server\nthat handles HTTP requests, worker process management, and run execution.\n\nThis guide assumes you have a model packaged with Cog.\nIf you don't, [follow our getting started guide](getting-started-own-model.md),\nor start from one of the [examples in the Cog repository](examples.md).\n\n## Build a Docker image\n\nBuild your model into a Docker image:\n\n```console\ncog build -t my-model\n```\n\nThe image contains your model code, dependencies, the Cog runtime, and everything in between.\nIt serves an HTTP server on port 5000 when run.\n\n## Push to a registry\n\n`cog push` builds the project and publishes it to an OCI-compliant registry:\n\n```console\ncog push registry.example.com/acme/my-model:v1\n```\n\nThe project configuration decides what Cog publishes. A project with `image` in\n`cog.yaml` publishes a container image. A project with `model` publishes an OCI\nbundle containing the image and any managed weights. Passing a target doesn't\nswitch between these formats.\n\nThe positional target is optional when `image` or `model` already supplies a\ndestination. When present, it overrides that configured destination and every\n`COG_MODEL*` environment variable. The target must be tag-addressable -- Cog\nrejects digest-pinned targets. An untagged image target uses Docker's `latest`\ntag, while an untagged bundle target gets a timestamp tag generated by Cog.\n\nFor a bundle with managed weights, use the same repository where `cog weights\nimport` published the weight manifests. The model tag may differ. If the\nrepository doesn't contain every required weight manifest, Cog fails before\npushing the image and tells you to run `cog weights import` for that repository.\n\nWithout `--json`, a successful push prints the published references as a\nhuman-readable tree on stderr.\n\n### Machine-readable output\n\nUse `--json` when another program needs the immutable references produced by a\npush. For example, this pushes a bundle and prints its model, image, and managed\nweight references:\n\n```console\ncog push registry.example.com/acme/my-model:v1 --json\n```\n\nVersion 1 output for a bundle with managed weights has this shape:\n\n```json\n{\n \"version\": 1,\n \"model\": \"registry.example.com/acme/my-model@sha256:1111111111111111111111111111111111111111111111111111111111111111\",\n \"image\": \"registry.example.com/acme/my-model@sha256:2222222222222222222222222222222222222222222222222222222222222222\",\n \"weights\": [\n {\n \"name\": \"transformer\",\n \"reference\": \"registry.example.com/acme/my-model@sha256:3333333333333333333333333333333333333333333333333333333333333333\"\n }\n ]\n}\n```\n\nThe version 1 fields are:\n\n- `version`: Always `1`.\n- `image`: Always present. The value is the published image's complete,\n digest-pinned reference.\n- `model`: Present only for bundle pushes. The value is the complete,\n digest-pinned bundle reference.\n- `weights`: Present only when the bundle has managed weights. Each entry has a\n `name` and a complete, digest-pinned `reference`. Entries keep their\n `cog.yaml` order.\n\nJSON whitespace and object key order aren't part of the contract.\n\nStdout stays empty while the command runs. Build and upload progress, warnings,\ndiagnostics, and registry-provider messages go to stderr. After the push and\nprovider post-processing both succeed, stdout receives exactly one JSON\ndocument. JSON mode suppresses the human-readable reference tree.\n\nAny build, push, digest-resolution, provider, or serialization failure exits\nnonzero without writing a JSON document to stdout. JSON mode requires Cog to\nresolve every published reference to an immutable digest, so it can report\nfailure even when an image was pushed successfully. Non-JSON image pushes keep\ntheir existing fallback when a registry can't resolve the pushed image's digest.\n\n## Run the model\n\nYou have several options for running a built image.\n\n### Docker\n\nRun the image directly with Docker.\nThis is the approach you'd use for production deployment.\n\n```shell\n# If your model uses a CPU:\ndocker run -d -p 5001:5000 my-model\n\n# If your model uses a GPU:\ndocker run -d -p 5001:5000 --gpus all my-model\n```\n\nThe server listens on port 5000 inside the container (mapped to 5001 above).\n\n### cog serve\n\nFor local development, `cog serve` builds the image and starts the server\nwith your project directory mounted in:\n\n```console\ncog serve\n```\n\nBy default the server runs on port 8393 and the container port is published on\n`127.0.0.1` (localhost), so it is only reachable from your local machine. The\nserver process inside the container binds to `0.0.0.0`; use `--host` to control\nwhich host interface the Docker port mapping is published on.\n\nUse `-p` to choose a different port:\n\n```console\ncog serve -p 5000\n```\n\n## Make a prediction\n\nOnce the server is running, make predictions by sending a POST request\nto the `/predictions` endpoint.\nInputs go inside an `\"input\"` object in the JSON body.\n\n> [!NOTE]\n> The examples below use `localhost:5001`, matching the Docker command above\n> (`-p 5001:5000`). If you used `cog serve`, use `localhost:8393` by default,\n> or the port you passed with `-p`.\n\n```console\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"input\": {\"prompt\": \"a photo of a cat\", \"steps\": 50}}'\n```\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"data:image/png;base64,...\",\n \"metrics\": {\n \"predict_time\": 4.52\n }\n}\n```\n\n> [!IMPORTANT]\n> Inputs **must** be wrapped in an `\"input\"` object.\n> `{\"input\": {\"scale\": 2.0}}` is correct; `{\"scale\": 2.0}` is not.\n\nTo discover what inputs your model accepts,\nview the OpenAPI schema:\n\n```console\ncurl http://localhost:5001/openapi.json\n```\n\n### Passing file inputs\n\nFile inputs (`cog.Path` or `cog.File` types) are passed as strings\ninside the `\"input\"` object.\nThere are two ways to do this:\n\n**1. HTTP/HTTPS URLs**\n\nPass a URL to a publicly accessible file.\nThe server downloads it inside the container:\n\n```console\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d '{\"input\": {\"image\": \"https://example.com/photo.jpg\"}}'\n```\n\n**2. Data URLs (base64)**\n\nTo pass a local file, encode it as a [data URL](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs):\n\n```bash\n# Construct a data URL from a local file\nDATA_URL=$(python3 -c \"\nimport base64, mimetypes\nwith open('input.jpg', 'rb') as f:\n data = base64.b64encode(f.read()).decode()\nmime = mimetypes.guess_type('input.jpg')[0] or 'application/octet-stream'\nprint(f'data:{mime};base64,{data}')\n\")\n\ncurl http://localhost:5001/predictions -X POST \\\n -H \"Content-Type: application/json\" \\\n -d \"{\\\"input\\\": {\\\"image\\\": \\\"$DATA_URL\\\"}}\"\n```\n\n> [!NOTE]\n> The HTTP API only accepts JSON (`application/json`).\n> Multipart form uploads are not supported.\n> When you use `cog run -i image=@photo.jpg`,\n> the CLI handles the base64 encoding for you automatically.\n\n### Getting output files\n\nWhen a model returns a file output (`cog.Path` or `cog.File`),\nthe response contains a base64-encoded data URL by default:\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"data:image/png;base64,iVBORw0KGgo...\"\n}\n```\n\nTo have the server upload output files to external storage instead,\nstart the server with the `--upload-url` flag. The server then uploads each\nfile output to that URL prefix and returns the resulting URL in the response.\n\nWith `cog serve`:\n\n```console\ncog serve --upload-url https://example.com/upload/\n```\n\nWhen running the image directly with Docker, override the command to start the\nserver with `--upload-url`:\n\n```shell\ndocker run -d -p 5001:5000 my-model \\\n python -m cog.server.http --upload-url https://example.com/upload/\n```\n\nWith an upload URL configured, file outputs are uploaded and the response\ncontains the uploaded URL instead of a data URL:\n\n```json\n{\n \"status\": \"succeeded\",\n \"output\": \"https://example.com/upload/image.png\"\n}\n```\n\n## Run a one-off prediction\n\nThe Docker and `cog serve` commands above leave an HTTP server running.\nIf you instead want to run a single prediction and exit — without starting a\nserver — use `cog run`:\n\n```console\ncog run my-model -i image=@input.jpg\n```\n\nThis starts the container, runs one prediction, prints the result, and exits.\nFile inputs are passed with the `@` prefix (e.g. `-i image=@photo.jpg`),\nand the CLI handles base64 encoding for you.\n\n## Health checks\n\nThe server exposes a `GET /health-check` endpoint that returns the current status of the model container. Use this for readiness probes in orchestration systems like Kubernetes.\n\n```console\ncurl http://localhost:5001/health-check\n```\n\nThe response includes a `status` field with values like `STARTING`, `READY`, `BUSY`, `SETUP_FAILED`, or `DEFUNCT`. See the [HTTP API reference](http.md#get-health-check) for full details.\n\n## Stop the server\n\nIf you started the container with `docker run -d`, stop it with:\n\n```console\ndocker kill \n```\n\nIf you used `cog serve`, press `Ctrl+C` in the terminal.\n(`cog run` exits on its own once the prediction finishes, so there's nothing\nto stop.)\n\n## Concurrency\n\nBy default, the server processes one run at a time. To enable concurrent runs, make your `run()` method async and decorate it with `@cog.concurrent(max=N)`:\n\n```py\nimport cog\n\nclass Runner(cog.BaseRunner):\n @cog.concurrent(max=4)\n async def run(self) -> str:\n return \"hello world\"\n```\n\nThe deprecated [`concurrency.max`](yaml.md#concurrency) field in `cog.yaml` is still supported and takes precedence over the decorator by baking `COG_MAX_CONCURRENCY` into the image.\n\n## Environment variables\n\nYou can configure runtime behavior with environment variables:\n\n- `COG_SETUP_TIMEOUT`: Maximum time in seconds for the `setup()` method (default: no timeout).\n- `COG_MAX_CONCURRENCY`: Number of concurrent prediction slots (default: 1). Overrides both `@cog.concurrent` and deprecated `cog.yaml` concurrency.\n\nSee the [environment variables reference](environment.md) for the full list.\n\n## Next steps\n\n- [HTTP API reference](http.md) for full endpoint documentation\n- [Private registries](private-package-registry.md) for using private Python package registries\n- [`cog.yaml` reference](yaml.md) for configuration options\n\n\n---\n\n# Environment variables\n\nThis reference lists the public Cog-specific environment variables that change how Cog behaves.\n\n## Build-time variables\n\n### `COG_SDK_WHEEL`\n\nControls which Cog Python SDK wheel is installed in the Docker image during `cog build`. Takes precedence over `build.sdk_version` in `cog.yaml`.\n\n**Supported values:**\n\n| Value | Description |\n| -------------------- | ---------------------------------------------------- |\n| `pypi` | Install latest version from PyPI |\n| `pypi:0.12.0` | Install specific version from PyPI |\n| `dist` | Use wheel from `dist/` directory (requires git repo) |\n| `https://...` | Install from URL |\n| `/path/to/wheel.whl` | Install from local file path |\n\n**Default behaviour:**\n\n- Release builds install the latest Cog SDK from PyPI.\n- Development builds auto-detect a wheel in `dist/`, then fall back to the latest Cog SDK from PyPI.\n\n```console\n$ COG_SDK_WHEEL=pypi:0.11.0 cog build\n$ COG_SDK_WHEEL=dist cog build\n$ COG_SDK_WHEEL=https://example.com/cog-0.12.0-py3-none-any.whl cog build\n```\n\nThe `dist` option searches for wheels in:\n\n1. `./dist/` (current directory)\n2. `$REPO_ROOT/dist/` (if `REPO_ROOT` is set)\n3. `/dist/` (via `git rev-parse`, useful when running from subdirectories)\n\n### `COGLET_WHEEL`\n\nControls which coglet wheel is installed in the Docker image. Coglet is the Rust-based inference server.\n\n**Supported values:** Same as `COG_SDK_WHEEL`.\n\n**Default behaviour:** For development builds, auto-detects a wheel in `dist/`. For release builds, installs the latest version from PyPI.\n\n```console\n$ COGLET_WHEEL=dist cog build\n$ COGLET_WHEEL=pypi:0.1.0 cog build\n```\n\n### `COG_CA_CERT`\n\nInjects a custom CA certificate into the Docker image during `cog build`. This is useful when building behind a corporate proxy or VPN that uses custom certificate authorities (for example, Cloudflare WARP).\n\n**Supported values:**\n\n| Value | Description |\n| -------------------------------- | ----------------------------------------------------------- |\n| `/path/to/cert.crt` | Path to a single PEM certificate file |\n| `/path/to/certs/` | Directory of `.crt` and `.pem` files (all are concatenated) |\n| `-----BEGIN CERTIFICATE-----...` | Inline PEM certificate |\n| `LS0tLS1CRUdJTi...` | Base64-encoded PEM certificate |\n\nThe certificate is installed into the system CA store and the `SSL_CERT_FILE` and `REQUESTS_CA_BUNDLE` environment variables are set automatically in the built image.\n\n```console\n$ COG_CA_CERT=/usr/local/share/ca-certificates/corporate-ca.crt cog build\n$ COG_CA_CERT=/etc/custom-certs/ cog build\n$ COG_CA_CERT=\"$(cat /path/to/cert.pem)\" cog build\n```\n\n### `COG_OPENAPI_SCHEMA`\n\nUses a pre-built OpenAPI schema instead of generating one from the configured predict or train reference.\n\nThe value must be a path to a JSON schema file. Cog reads that file during schema generation and embeds it in the built image.\n\n```console\n$ COG_OPENAPI_SCHEMA=./openapi.json cog build\n```\n\n## CLI and local cache variables\n\n### `COG_NO_UPDATE_CHECK`\n\nDisables Cog's automatic update check. Set it to any non-empty value.\n\n```console\n$ COG_NO_UPDATE_CHECK=1 cog build\n```\n\n### `COG_NO_COLOR`\n\nDisables coloured CLI output. Set it to any non-empty value.\n\nCog also honours the standard `NO_COLOR` environment variable.\n\n```console\n$ COG_NO_COLOR=1 cog predict -i prompt=\"hello\"\n```\n\n### `COG_SKIP_DOCKER_CHECK`\n\nSkips the `cog doctor` Docker environment check. Set it to any non-empty value.\n\n```console\n$ COG_SKIP_DOCKER_CHECK=1 cog doctor\n```\n\n### `COG_SKIP_GPU_CHECK`\n\nSkips the `cog doctor` GPU compatibility check, which warns when the resolved torch/CUDA\nversions ship no kernels for the GPU in the current machine. Set it to any non-empty value.\n\nUse this when you are deliberately building for a different GPU than the one in this\nmachine, so the check should not warn about the local device.\n\n```console\n$ COG_SKIP_GPU_CHECK=1 cog doctor\n```\n\n### `COG_CACHE_DIR`\n\nOverrides Cog's local cache root.\n\nCog currently uses this cache for the content-addressed weights store. If unset, Cog uses `$XDG_CACHE_HOME/cog` when `XDG_CACHE_HOME` is set, otherwise `$HOME/.cache/cog`.\n\n```console\n$ COG_CACHE_DIR=/mnt/fast-cache cog weights pull\n```\n\n## Model reference and registry variables\n\n### `COG_MODEL`\n\nOverrides the full model reference used by commands that need a model destination, such as `cog push` and weights commands.\n\nThe value is parsed as a complete model reference (`registry/repo`, `registry/repo:tag`, or `registry/repo@digest`). If no tag is supplied, Cog generates a timestamp tag.\n\nWhen `COG_MODEL` is set, it takes precedence over `COG_MODEL_REGISTRY`, `COG_MODEL_REPO`, and `COG_MODEL_TAG`. A positional target passed to `cog push` takes precedence over all four variables. Although commands that inspect existing artifacts accept a digest, `cog push` rejects digest-pinned destinations because a push requires a tag.\n\n```console\n$ COG_MODEL=r8.im/acme/my-model:v1 cog push\n```\n\n### `COG_MODEL_REGISTRY`\n\nOverrides only the registry host of the model reference.\n\n```console\n$ COG_MODEL_REGISTRY=registry.example.com cog push\n```\n\n### `COG_MODEL_REPO`\n\nOverrides only the repository path of the model reference. The value must not include a registry host, tag, or digest.\n\n```console\n$ COG_MODEL_REPO=acme/my-model cog push\n```\n\n### `COG_MODEL_TAG`\n\nOverrides only the tag of the model reference.\n\nTags starting with `cog-` are reserved for tags that Cog generates internally and are rejected.\n\n```console\n$ COG_MODEL_TAG=staging cog push\n```\n\n### `COG_REGISTRY_HOST`\n\nChanges the default Replicate-compatible registry host used by commands such as `cog login`, base image resolution, and model reference resolution.\n\nThe default is `r8.im`.\n\n```console\n$ COG_REGISTRY_HOST=registry.example.com cog login\n```\n\n## Runtime server variables\n\nThese variables affect a running model server. Set them in `cog.yaml` under `environment`, pass them with `cog predict -e` or `cog serve -e`, or set them when running the built Docker image.\n\n### `COG_MAX_CONCURRENCY`\n\nControls how many predictions the model server can run concurrently. This overrides both `@cog.concurrent(max=N)` and the deprecated `concurrency.max` field in `cog.yaml`.\n\nBy default, Cog runs one prediction at a time unless the model uses `@cog.concurrent(max=N)`. Invalid values are ignored and the default of `1` is used.\n\nValues greater than `1` require an async `run()` method. This applies even when `COG_MAX_CONCURRENCY` is set as a runtime operator override.\n\nConcurrency is resolved in this order, from highest to lowest precedence:\n\n1. `COG_MAX_CONCURRENCY` set at runtime\n2. Deprecated `concurrency.max` in `cog.yaml`, which is baked into the image as `COG_MAX_CONCURRENCY`\n3. `@cog.concurrent(max=N)` on the async `run()` method\n4. Default: `1`\n\n```console\n$ COG_MAX_CONCURRENCY=4 docker run -p 5000:5000 my-model\n```\n\n### `COG_SETUP_TIMEOUT`\n\nControls the maximum time, in seconds, allowed for the model's `setup()` method to complete. If setup exceeds this timeout, the server reports setup failure.\n\nBy default, there is no timeout. Set to `0` to disable the timeout. Invalid values are ignored with a warning.\n\n```console\n$ COG_SETUP_TIMEOUT=300 docker run -p 5000:5000 my-model\n```\n\n### `COG_LOG_LEVEL`\n\nControls Coglet runtime log verbosity when `RUST_LOG` is not set.\n\nSupported values are `debug`, `info`, `warn`, `warning`, and `error`. The default is `info`.\n\n```console\n$ COG_LOG_LEVEL=debug docker run -p 5000:5000 my-model\n```\n\n### OpenTelemetry tracing\n\nTracing must first be enabled under `observability.traces` in `cog.yaml`. Runtime settings may disable an enabled image but cannot enable an image that did not opt in.\n\n| Variable | Purpose |\n| ----------------------------- | ---------------------------------------------- |\n| `COG_TRACE_ENABLED` | Set to `false` to disable tracing at runtime. |\n| `OTEL_SDK_DISABLED` | Hard-disable OpenTelemetry SDK initialization. |\n| `OTEL_EXPORTER_OTLP_ENDPOINT` | Collector endpoint for framework tracing. |\n| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf` or `grpc`. |\n| `OTEL_EXPORTER_OTLP_HEADERS` | Collector authentication headers. |\n| `OTEL_SERVICE_NAME` | Service name, default `cog`. |\n| `OTEL_TRACES_SAMPLER` | Runtime sampler override. |\n| `OTEL_TRACES_SAMPLER_ARG` | Ratio for ratio samplers. |\n\nWhen tracing is enabled without an endpoint, Cog logs one warning and continues without framework tracing. A custom Python provider configured by `observability.config` still runs and may use another exporter or no exporter. Delivery failures from Cog's built-in exporters never change prediction results; custom processors and exporters are model-owned code.\n\n`COG_OBSERVABILITY_*`, `COG_TRACE_*`, and `OTEL_*` are reserved from the general `cog.yaml` `environment` list. `COG_OBSERVABILITY_CONFIG` is internal and points to the validated file staged in the image. Supply supported `COG_TRACE_*` and `OTEL_*` settings to the running container instead.\n\n### `COG_THROTTLE_RESPONSE_INTERVAL`\n\nControls how often asynchronous webhook `output` and `logs` events are sent, in seconds.\n\nThe default is `0.5` seconds. Invalid values are ignored and the default is used. `start` and `completed` webhook events are always sent immediately.\n\n```console\n$ COG_THROTTLE_RESPONSE_INTERVAL=1 docker run -p 5000:5000 my-model\n```\n\n### `COG_STREAM_HISTORY_CAPACITY`\n\nControls how many server-sent event stream events are retained per prediction for replay when a client reconnects with `Accept: text/event-stream`.\n\nBy default, Cog retains the most recent 1024 events per prediction. Set to `0` to disable replay history while keeping live streaming enabled. Invalid values are ignored with a warning and the default is used.\n\n```console\n$ COG_STREAM_HISTORY_CAPACITY=0 docker run -p 5000:5000 my-model\n$ COG_STREAM_HISTORY_CAPACITY=4096 docker run -p 5000:5000 my-model\n```\n\n### `COG_WEIGHTS`\n\nProvides a weights path or URL to a model whose `setup()` method accepts a `weights` parameter.\n\n```console\n$ cog run -e COG_WEIGHTS=https://example.com/weights.tar -i prompt=\"hello\"\n```\n\n### `COG_USER_AGENT`\n\nSets the `User-Agent` header used by Cog when downloading URL-backed `File` inputs.\n\n```console\n$ COG_USER_AGENT=\"my-service/1.0\" docker run -p 5000:5000 my-model\n```\n\n## Push tuning variables\n\n### `COG_PUSH_OCI`\n\nEnables Cog's OCI chunked push path for container image layers when set to `1`. If the OCI push fails with a non-fatal error, Cog falls back to Docker's native push path.\n\n```console\n$ COG_PUSH_OCI=1 cog push\n```\n\n### `COG_PUSH_CONCURRENCY`\n\nControls how many image layers or weight blobs Cog uploads concurrently during push operations.\n\nThe default is `5`. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_CONCURRENCY=2 cog push\n```\n\n### `COG_PUSH_DEFAULT_CHUNK_SIZE`\n\nSets the default multipart upload chunk size, in bytes, when the registry does not advertise a maximum chunk size.\n\nThe default is 96 MiB. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_DEFAULT_CHUNK_SIZE=67108864 cog push\n```\n\n### `COG_PUSH_MULTIPART_THRESHOLD`\n\nSets the minimum blob size, in bytes, before Cog uses multipart upload.\n\nThe default is 128 MiB. Invalid values and values less than `1` are ignored.\n\n```console\n$ COG_PUSH_MULTIPART_THRESHOLD=268435456 cog push\n```\n\n\n---\n\n# Examples\n\nCog's example models now live in the main Cog repository under\n[`examples/`](https://github.com/replicate/cog/tree/main/examples).\n\nIf you want a working project to copy from, start with one of these:\n\n- [`hello-world`](https://github.com/replicate/cog/tree/main/examples/hello-world):\n minimal string input/output example\n- [`hello-image`](https://github.com/replicate/cog/tree/main/examples/hello-image):\n returns a static image\n- [`hello-context`](https://github.com/replicate/cog/tree/main/examples/hello-context):\n shows how to read prediction context\n- [`hello-concurrency`](https://github.com/replicate/cog/tree/main/examples/hello-concurrency):\n demonstrates the `@cog.concurrent` decorator\n- [`hello-train`](https://github.com/replicate/cog/tree/main/examples/hello-train):\n defines a training interface\n- [`hello-replicate`](https://github.com/replicate/cog/tree/main/examples/hello-replicate):\n uses `cog.Secret` inputs and calls the Replicate API\n- [`blur`](https://github.com/replicate/cog/tree/main/examples/blur):\n simple image-to-image transformation\n- [`resnet`](https://github.com/replicate/cog/tree/main/examples/resnet):\n GPU image classification with ResNet50\n- [`streaming-text`](https://github.com/replicate/cog/tree/main/examples/streaming-text):\n streaming text generation with server-sent events\n- [`z-image-turbo`](https://github.com/replicate/cog/tree/main/examples/z-image-turbo):\n a generative image model example\n- [`notebook`](https://github.com/replicate/cog/tree/main/examples/notebook):\n runs a Jupyter notebook with Cog\n- [`canary`](https://github.com/replicate/cog/tree/main/examples/canary):\n small streaming output example used for validation\n- [`managed-weights`](https://github.com/replicate/cog/tree/main/examples/managed-weights):\n managed weights pipeline fixture\n\nThere is also an experimental example:\n\n- [`experimental/resnet-managed-weights`](https://github.com/replicate/cog/tree/main/examples/experimental/resnet-managed-weights):\n ResNet50 packaged with managed weights\n\nFor a quick summary of every example in the tree, see\n[`examples/README.md`](https://github.com/replicate/cog/blob/main/examples/README.md).\n\n\n---\n\n# Getting started with your own model\n\nThis guide will show you how to put your own machine learning model in a Docker image using Cog. If you haven't got a model to try out, you'll want to follow the [main getting started guide](getting-started.md).\n\n## Prerequisites\n\n- **Cog**. If you haven't already installed Cog, follow the [install instructions in the README](../README.md#install).\n- **macOS, Linux, or Windows 11**. Cog works on macOS and Linux. It also works on Windows 11 with [WSL 2](wsl2/wsl2.md).\n- **Docker**. Cog uses Docker to create a container for your model. You'll\n\n\n# Design\n\n## Background\n\nCog came from Andreas's experience at Spotify and Ben's experience at Docker.\n\nAt Spotify, Andreas noticed a cluster of related problems:\n\n- **It was hard to run open-source machine learning models.** All the advances in machine learning were locked up inside prose in PDFs, scraps of code on GitHub, weights on Google Drive (if you were lucky!). If you wanted to build upon this research, or apply it to real-world problems, you had to implement it all from scratch.\n- **It was hard to deploy machine learning models to production.** Andreas was the only person on the research team who was also an infrastructure engineer. Typically a researcher would have to sit down with Andreas to decide on an API, get a server written, package up dependencies, battle CUDA, get it running efficiently, get it deployed on the cluster, and so on and so forth. It would take weeks to get something running in production.\n\nBen connected this back to his experience at Docker. What Docker did was define a standard box that software could go in. You could put any kind of server software in there – Python, Java, Ruby on Rails, whatever – and you could then know that you could run it on your local machine or on any cloud, as long as it supported Docker. We wanted to do the same thing for machine learning.\n\n## Vision\n\nWe want Cog to be a standard artifact for what a model is and how that model is run.\n\n(More detail...)\n\n## Design principles\n\nThere are a few things driving Cog's design:\n\n- **Reproducible artifact**: When you've put your model in Cog, it'll run anywhere, and _keep_ on running. This is why it's a Docker image, with all of the model's dependencies. Docker images have a content-addressable SHA256 ID, which is the identifier for that model's behavior, byte-for-byte.\n- **Weights inside the image**: We encourage users to put model weights in images. If the weights are on cloud storage somewhere, then they might change or disappear, and the image will produce different results. There's nothing magical about Docker images – they're just a bundle of files. Docker moves around that bundle of files quite slowly, though, but we can optimize that process so it's as fast as reading weights directly from blob storage, or wherever.\n- **Models are just functions**: Models can be lots of things. We are of the opinion that [machine learning is just software](https://replicate.com/blog/machine-learning-needs-better-tools), and a model is just a function. It often needs to be attached to a GPU, but apart from that it's just a normal function that has some input and some output. This is the core difference between Docker's abstraction and Cog's abstraction: Docker packages up an executable, whereas Cog packages up a _function_.\n- **Standard interface**: When you run the Docker container, it serves an HTTP server, that is a standard API for running that function. You can think of it like a remote procedure call.\n- **Self-describing artifact**: A Cog model has it's schema (or type signature, if you're thinking of it as a function) attached to the image as a label. This means systems that work with Cog models can know what the model is and what requests to send to it. This is what powers the forms on Replicate, for example.\n- **Not just the model**: Before Cog, the typical standard packaging formats for machine learning models were at the network level. A way of taking a Tensorflow or PyTorch network and packaging up in a way that would run on lots of different types of accelerators. Things like [ONNX](https://onnx.ai/) or [TVM's IR](https://tvm.apache.org/). We realized that \"models\" are not just the network, but they are also pre- and post-processing, and are so diverse and the field is so fast-moving that you can't possible squeeze it into some high-level abstraction. It just needs to be code running on a computer.\n- **It's just Docker**: Cog models need to run anywhere, and they'll only run anywhere if it's vanilla Docker. We might optimize how Docker images get shipped around to make it faster, but we're not going to invent our own image format.\n- **The API is for software developers**: In the olden days, you have to pass tensors to TFServing and know how to generate a tensor from a JPEG. Cog's API intentionally just speaks JSON, strings, files, etc. It's intended to be the interface between the software developer and the ML engineer. Sort of like Docker was intended to be the interface between the software developer and the infrastructure engineer.\n- **Cog is the APIs and interfaces, not just the software**: The most important thing about Cog is that it defines a standard for what a model is and how to run it. It doesn't necessarily need to involve Cog the piece of software itself. For example, Replicate could serve a model from OpenAI with a Cog API and schema, but it's not packaged or running with Cog under the hood at all – it's just calling the OpenAI API directly.\n\n\n---\nname: release-cog\ndescription: Guide and automate the Cog release process\n---\n\n# Cog Release Skill\n\nThis skill helps you release new versions of Cog. Cog is a multi-language, multi-artifact project with a carefully orchestrated release process.\n\n## Overview\n\nCog releases include:\n\n- **CLI binaries** (Go) - macOS and Linux, x86_64 and ARM64\n- **coglet Python wheels** (Rust/PyO3) - Linux x86_64/ARM64, macOS ARM64\n- **cog SDK Python wheel** (Python) - Universal\n- **coglet Rust crate** - Published to crates.io\n\n## Release Types\n\n| Type | Format | Example | Branch | PyPI | crates.io | Homebrew |\n| --------------- | ------------------------------- | ------------ | ------ | ---- | --------- | -------- |\n| **Stable** | `v0.17.0` | v0.18.0 | main | ✓ | ✓ | ✓ |\n| **Pre-release** | `v0.17.0-alpha3`, `v0.17.0-rc1` | v0.18.0-rc1 | main | ✓ | ✓ | ✗ |\n| **Dev** | `v0.17.0-dev1` | v0.18.0-dev2 | any | ✗ | ✗ | ✗ |\n\n## Quick Release Commands\n\n### 1. Bump Version (if needed)\n\n```bash\n# Check current version\nmise run version\n\n# Bump to new version (updates VERSION.txt, Cargo.toml, Cargo.lock, commits)\nmise run version:bump 0.18.0\n```\n\n### 2. Create a release branch and push\n\n```bash\ngit checkout -b release/v0.18.0\ngit push origin release/v0.18.0\n```\n\n### 3. Create PR to main\n\n```bash\n# Open a pull request to main, get it reviewed and merged. Then you can create the release tag from main.\ngh pr create --base main --head release/v0.18.0 --title \"Release v0.18.0\" --body \"Release description and notes\"\n```\n\n### 4. Create and Push Tag\n\n```bash\ngit checkout main && git pull origin main\n\n# For stable release\ngit tag v0.18.0\ngit push origin v0.18.0\n\n# For pre-release\ngit tag v0.18.0-rc1\ngit push origin v0.18.0-rc1\n\n# For dev release (can be from any branch)\ngit tag v0.18.0-dev1\ngit push origin v0.18.0-dev1\n```\n\n### 5. Monitor Release Build\n\n```bash\n# Watch the release build workflow\ngh workflow view release-build.yaml\n\n# Or watch in real-time\ngh run watch\n```\n\n### 6. Write Release Notes\n\nAfter the draft release is created, update the release notes to follow the project's standard format. The release notes are not auto-generated from commit messages — they must be hand-written and categorized.\n\n**To see the previous release's format:**\n\n```bash\ngh release view v0.19.0 --json body\n```\n\n**To gather commits since the last release:**\n\n```bash\n# List commits between the previous release and this one\ngit log --oneline v0.19.0..v0.20.0 --no-merges\n```\n\n**Release notes structure:**\n\nGroup changes into three sections. Only include sections that have items.\n\n1. **New features** — New commands, new APIs, new annotations, new capabilities.\n2. **Improvements** — Performance, reliability, DX improvements, removals of legacy paths, build improvements.\n3. **Bug fixes** — User-visible bug fixes. Prefer \"Now does X correctly\" over \"Fixed X\".\n\n**Style guidelines:**\n\n- Lead each bullet with a **bold, user-facing summary sentence** (e.g., \"**`cog run` command.** ...\")\n- Follow with a short explanation of what changed and why it matters\n- Reference the PR number in parentheses at the end: `(#3015)`\n- Use backticks for commands, flags, and code references\n- Omit internal refactors, dependency bumps, and CI-only changes unless they are user-facing\n- Omit version bump commits\n\n**Example:**\n\n```markdown\n### New features\n\n- **`cog run` command.** The `cog predict` command has been renamed to `cog run` with full backward compatibility. `cog predict` still works as an alias. (#3015)\n- **Model refs for `cog push` and weights commands.** You can now reference models by name (e.g., `r8.im/user/model`) instead of full image URLs when pushing or managing weights. (#3018)\n\n### Improvements\n\n- **Runtime schema generation fully removed.** The legacy runtime Python schema generation path has been completely removed. Cog exclusively uses static schema generation, making builds faster and more reliable. (#3003)\n\n### Bug fixes\n\n- **Pushing a model with a version tag now emits a clean URL.** The Replicate model URL printed after `cog push` no longer includes the image tag (e.g., `:latest`), preventing 404 errors when users click the link. (#3020)\n```\n\n**To update the draft release:**\n\n```bash\ngh release edit v0.20.0 --notes \"$(cat <<'EOF'\n### New features\n\n- ...\n\n### Improvements\n\n- ...\n\n### Bug fixes\n\n- ...\nEOF\n)\"\n```\n\n### 7. Publish Release (stable/pre-release only)\n\n- Go to GitHub Releases page\n- Find the draft release\n- Review release notes\n- Click \"Publish release\"\n- This triggers `release-publish.yaml` which publishes to PyPI and crates.io\n\n## Release Process Details\n\n### Automated Workflows\n\n1. **`release-build.yaml`** - Triggered on version tags\n - Verifies tag matches VERSION.txt and Cargo.toml\n - Verifies stable/pre-release tags are on main branch\n - Builds SDK wheel (with updated coglet version constraint)\n - Builds coglet wheels for all platforms (Linux x64/ARM64, macOS ARM64)\n - Uses GoReleaser to build CLI binaries and create draft release\n - Uploads wheels to GitHub release\n - For dev releases: immediately publishes as pre-release\n\n2. **`release-publish.yaml`** - Triggered when release is published\n - Publishes coglet wheels to PyPI\n - Publishes coglet crate to crates.io\n - Publishes cog SDK to PyPI (depends on coglet)\n - Updates Homebrew tap (stable releases only)\n\n3. **`homebrew-tap.yaml`** - Updates Homebrew cask\n - Generates cask from `.github/cog.rb.tmpl`\n - Creates PR in `replicate/homebrew-tap`\n\n### Version Files\n\n| File | Purpose |\n| ------------------- | ------------------------------------------ |\n| `VERSION.txt` | Canonical version (single source of truth) |\n| `crates/Cargo.toml` | Rust workspace version |\n| `crates/Cargo.lock` | Locked dependency versions |\n\n### Version Constraints\n\nThe SDK (`pyproject.toml`) has a dependency on coglet:\n\n```toml\ncoglet>=0.1.0,<1.0\n```\n\nDuring release build, this is updated to:\n\n```toml\ncoglet>=0.18.0,<1.0\n```\n\nThis ensures the SDK depends on the matching coglet version.\n\n## Pre-Release Checklist\n\nBefore creating a release tag:\n\n- [ ] All tests pass: `mise run test`\n- [ ] Lint passes: `mise run lint`\n- [ ] Version is correct in `VERSION.txt`\n- [ ] `mise run version:check` passes\n- [ ] `crates/Cargo.toml` matches `VERSION.txt`\n- [ ] Changelog is updated (if applicable)\n- [ ] Documentation is updated (`mise run docs:llm`)\n\n## Troubleshooting\n\n### Version mismatch error\n\n```\nVersion mismatch! VERSION.txt has X but tag is vY\n```\n\nFix: Run `mise run version:bump Y`, push, then re-tag.\n\n### Tag not on main\n\n```\nRelease tags must be on the main branch\n```\n\nFix: Merge your changes to main, then tag from main.\n\n### Rebuilding a failed release\n\n1. Delete the GitHub release if it was created: `gh release delete v0.18.0 --yes`\n2. Delete the tag: `git push --delete origin v0.18.0 && git tag -d v0.18.0`\n3. Fix the issue\n4. Re-create and push the tag\n\n### Manual PyPI publish (emergency)\n\nIf the automated publish fails:\n\n```bash\n# Download wheels from GitHub release\ngh release download v0.18.0 -p \"coglet-*.whl\" -D dist\ngh release download v0.18.0 -p \"cog-*.whl\" -D dist\n\n# Publish with twine\n twine upload dist/coglet-*.whl # First!\ntwine upload dist/cog-*.whl # After coglet is uploaded\n```\n\n## Architecture Notes\n\n- **Trusted Publishing**: PyPI and crates.io use OIDC trusted publishing (no API tokens in secrets)\n- **Environments Required**: Configure `pypi`, `crates-io`, and `homebrew` environments in GitHub repo settings\n- **CGO**: Required for go-tree-sitter (static Python schema parser)\n- **Zig**: Used for Linux cross-compilation (CC=zig cc)\n- **macOS builds**: Use native compiler (zig lacks macOS SDK stubs)\n- **Wheel discovery**: CLI discovers wheels from `dist/` at Docker build time, not embedded in binary\n\n\n---\nname: updating-architecture-docs\ndescription: \"Use this skill when updating, reviewing, or creating architecture documentation in the architecture/ directory. This includes after refactors, feature additions, component changes, or when auditing docs for accuracy. Use it any time code changes affect how Cog's internals work -- new packages, changed IPC protocols, modified build pipeline, runtime behavior changes. Also use it proactively when reviewing PRs that touch core systems to check whether the architecture docs need updating.\"\n---\n\n# Updating Architecture Docs\n\n## Purpose\n\nThe `architecture/` directory is a bridge. It takes someone who already knows what Cog does and how to use it (that's what `docs/` is for) to the point where they can navigate the source code with confidence. It answers \"how does this work and why is it shaped this way\" -- the 10,000 foot view of the project and its ecosystem.\n\nThe goal is to move a reader from fuzzy concepts to clarity:\n\n- What are the components and what do they own?\n- What's the vocabulary? (What does \"slot\" mean? What's the \"envelope\"?)\n- Where are the boundaries between systems?\n- What are the key design decisions and why were they made?\n- Where do I look in the source to go deeper?\n\nThe architecture docs are **not** a substitute for reading code or a prose summary of the implementation. They give the reader enough context to make the code legible on first contact. Once someone understands that coglet uses a two-process model with slot-based IPC, reading `orchestrator.rs` makes sense. Without that context, it's just code.\n\n## Document structure\n\nThe docs are numbered to suggest a reading order. `00-overview.md` orients the reader and points to everything else. Each subsequent doc builds on concepts from the ones before it -- someone reading about the container runtime should already understand the predictor class and the HTTP API.\n\nThe current docs and their structure aren't fixed. New docs should be added when a topic becomes important enough to deserve its own section (testing philosophy, deployment model, etc.). Docs can be split, merged, or reorganized as the system evolves. The numbering is for reading order, not a permanent taxonomy. Check `architecture/` for the current set.\n\n## Principles\n\n### Bridge, don't summarize\n\nThe architecture docs should give a reader the mental model they need to read the code -- not replace the code. If a section starts reading like a prose restatement of what a module does line by line, it's gone too far. Pull back to the concepts and boundaries.\n\nGood: \"The orchestrator spawns a single worker subprocess and manages its lifecycle. Communication happens over two channels: a control channel for lifecycle events (stdin/stdout, JSON lines) and per-slot Unix sockets for prediction data.\"\n\nBad: \"The `spawn_worker` function in `orchestrator.rs` calls `Command::new('python')` with args `['-c', 'import coglet; coglet.server._run_worker()']` and sets up stdin/stdout pipes. It then reads from the control channel in a loop, matching on `ControlResponse` variants...\"\n\nThe first gives you the mental model. The second just restates the code. A reader with the first description can read `orchestrator.rs` and follow along. A reader with the second didn't need to read the doc at all.\n\n### Point at packages, not files\n\nReference source locations at the **package/directory level** with a description of what that package owns. Specific file paths and line numbers rot as code moves around. A pointer like \"`crates/coglet/src/bridge/` -- IPC protocol and transport\" stays accurate through refactors. \"`bridge/protocol.rs:69` -- ControlRequest enum\" doesn't.\n\nOnly document packages that matter for understanding the system's shape. Generic utility packages (`pkg/util/`, `pkg/path/`, etc.) don't need a mention -- their existence is obvious and they don't help a reader build a mental model. If someone needs them, they'll find them.\n\nWhen a specific file reference is genuinely useful (a key entry point, a non-obvious starting point for understanding a subsystem), include it -- but prefer \"the `PredictionService` in `service.rs`\" over a line number.\n\n### Document boundaries, not internals\n\nFocus on interfaces between components: what messages cross the IPC channel, what the HTTP API contract is, what labels a built image carries, what env vars control behavior. These are the things a reader needs to understand to reason about the system. Implementation details behind those boundaries belong in code comments and crate-level READMEs, not in architecture docs.\n\nThe practical test: would an internal refactor (changing how something works without changing its interface) require updating this doc? If yes, the doc is too detailed.\n\n### Explain the why\n\nDesign decisions are the most valuable content in architecture docs. They're the one thing you can't get from reading the code. \"Why two processes?\" has an answer (isolation, CUDA contexts, crash resilience) that makes the whole architecture make sense. Without it, a reader sees the complexity of IPC and wonders whether it's accidental.\n\nEvery major structural choice should have a short rationale. One or two sentences is enough.\n\n## Current state of the world\n\nThese facts should be reflected consistently across all docs. If the code changes and these become stale, the docs need updating.\n\n**One runtime:** Coglet (Rust/Axum + PyO3) is the sole runtime. There's no legacy Python/FastAPI runtime, no toggle, no \"experimental\" qualifier. Don't frame coglet as an alternative to something else.\n\n**Pydantic is not core:** Schema generation parses Python source in Go with tree-sitter and emits OpenAPI directly. `cog.BaseModel` is a dataclass wrapper. Pydantic BaseModel is supported in user code for compatibility but isn't part of Cog's own type system.\n\n**Static schema generation is the only path:** The tree-sitter-based static schema generator (Go side, `pkg/schema/`) runs on schema-generating `cog build` invocations. Parser failures stop the build before Docker starts; there is no Docker/Python runtime schema fallback or `COG_LEGACY_SCHEMA` toggle.\n\n**Wheels aren't embedded:** SDK and coglet wheels are resolved at Docker build time from PyPI, env vars, or local `dist/` directory. They're not compiled into the Go binary.\n\n**Three codebases:**\n\n- `cmd/cog/` + `pkg/` -- Go CLI and build tooling\n- `python/cog/` -- Python SDK (type definitions, predictor base class, thin server launcher)\n- `crates/coglet/` + `crates/coglet-python/` -- Rust prediction server with PyO3 bindings\n\n## How to update\n\n### When to update\n\nAfter code changes, ask: **did a boundary or interface change?**\n\nNeeds doc updates:\n\n- New or removed IPC message types\n- New or changed HTTP endpoints\n- New CLI commands\n- Changes to the build pipeline (new build steps, changed Dockerfile structure)\n- New top-level packages in `pkg/` or new crates in `crates/`\n- Changes to how components communicate\n- New design decisions or changed rationale for existing ones\n\nDoesn't need doc updates:\n\n- Bug fixes within a component\n- Internal refactors that don't change interfaces\n- Performance improvements\n- Test changes\n\n### Auditing for accuracy\n\nWhen checking docs against the codebase:\n\n1. **Read each doc** and identify claims about the system -- component names, package locations, protocol messages, env vars, build steps, CLI commands\n2. **Verify against source** -- check that things exist and work as described\n3. **Classify issues**:\n - **Structural** -- describes something that doesn't exist or works fundamentally differently (fix immediately)\n - **Misleading** -- technically true but gives the wrong mental model (fix immediately)\n - **Missing** -- a feature or component exists with no doc coverage (add if it's a boundary/interface concern)\n - **Stale reference** -- file/package renamed but concept is right (fix opportunistically)\n4. Fix structural and misleading issues first -- they actively harm readers\n\n### Writing conventions\n\nRead the existing architecture docs before writing. Match their tone and level of detail. New content should feel like it belongs -- if it reads like a different author wrote it, adjust.\n\nThe docs are technical writing. Clear, precise, no filler. Don't inflate -- avoid the kind of padded, formal language that agents default to (\"it should be noted that\", \"robust\", \"comprehensive\", \"this ensures that\"). Just say the thing.\n\nDiagrams use Mermaid or ASCII art. Both are fine. Use whichever communicates the structure more clearly. Keep each diagram focused on one concept.\n\n\n---\nname: cog-review\ndescription: Cog architecture and cross-cutting review guidelines\n---\n\n## Cog architecture review guidelines\n\nCog packages ML models into production-ready containers. Use this skill for\nchanges that cross language boundaries or touch core architecture.\n\n### Component overview\n\n- **CLI** (Go): `cmd/cog/` and `pkg/` -- builds, runs, and deploys models\n- **Python SDK**: `python/cog/` -- predictor interface, types, HTTP/queue server\n- **Coglet** (Rust): `crates/` -- prediction server inside containers (HTTP, worker management, IPC)\n\n### Key design patterns\n\n**Wheel resolution**: The CLI discovers SDK and coglet wheels from `dist/` at\nDocker build time. Wheels are NOT embedded in the binary. Changes to build\nartifacts need to account for this.\n\n**Dockerfile generation**: `pkg/dockerfile/` generates Dockerfiles from `cog.yaml`\nconfig. Template injection and escaping matter here.\n\n**Config parsing**: `pkg/config/config.go` parses `cog.yaml`. Schema is at\n`pkg/config/data/config_schema_v1.0.json`. Changes must keep schema and Go\ncode in sync.\n\n**Two-process coglet**: Parent process (HTTP server + orchestrator) and child\nworker process (Python predictor execution) communicate via IPC. Changes to\nthe IPC protocol affect both Rust and Python code.\n\n**Compatibility matrix**: CUDA/PyTorch/TensorFlow compatibility is managed in\n`tools/compatgen/`. Framework version changes have wide blast radius.\n\n### Cross-cutting concerns\n\n- **VERSION.txt** is the single source of truth for versioning. `Cargo.toml` must match.\n- Changes to `python/cog/base_predictor.py` affect all downstream model authors.\n- Changes to `docs/` require running `mise run docs:llm` to regenerate `docs/llms.txt`.\n- CLI reference docs are auto-generated -- changes to `cmd/` or `pkg/cli/` require `mise run docs:cli`.\n- Integration tests in `integration-tests/` use Go's testscript and need a built cog binary.\n\n### What to watch for\n\n- Breaking changes to the predictor interface or cog.yaml schema\n- Docker build changes that affect layer caching or build time\n- IPC protocol changes without updating both Rust and Python sides\n- Version bumps that miss one of the places VERSION.txt needs to match\n- New ML framework versions without compatibility testing\n\n\n---\nname: go-review\ndescription: Go code review guidelines for the Cog codebase\n---\n\n## Go review guidelines\n\nThis project uses Go for the CLI (`cmd/cog/`, `pkg/`) and support tooling (`tools/`).\n\n### What linters already catch (skip these)\n\ngolangci-lint runs errcheck, gocritic, gosec, govet, ineffassign, misspell,\nrevive, staticcheck, and unused. Don't flag issues these would catch.\n\n### What to look for\n\n**Error handling**\n\n- Errors returned but not checked or silently discarded\n- User-facing errors should use `pkg/errors.CodedError` with error codes\n- Generic error wrapping that loses context (`fmt.Errorf(\"failed\")` with no `%w`)\n\n**Imports**\n\n- Should be three groups: stdlib, third-party, internal (`github.com/replicate/cog/pkg/...`)\n- Only flag if actually wrong, not cosmetic reordering\n\n**Testing**\n\n- Must use `testify/require` for fatal assertions and `testify/assert` for non-fatal\n- No raw `if` checks with `t.Fatal`/`t.Errorf`\n- Prefer table-driven tests for similar cases\n- Prefer specific assertions (`Equal`, `Contains`, `NoError`) over `True`/`False`\n\n**Concurrency**\n\n- Goroutine leaks (no cleanup path, missing context cancellation)\n- Shared state without synchronization\n- Channel misuse (sends on closed channels, unbuffered channels in wrong contexts)\n\n**Docker/container patterns**\n\n- The CLI uses the Docker Go SDK. Watch for leaked clients, unclosed response bodies\n- Dockerfile generation is in `pkg/dockerfile/` -- template injection risks\n\n**Architecture**\n\n- Commands belong in `pkg/cli/`, business logic in `pkg/`\n- Config parsing/validation in `pkg/config/`\n- Don't mix CLI concerns with library logic\n\n\n---\nname: python-review\ndescription: Python code review guidelines for the Cog SDK\n---\n\n## Python review guidelines\n\nThis project uses Python for the SDK (`python/cog/`) which defines the predictor\ninterface, type system, and HTTP/queue server.\n\n### What linters already catch (skip these)\n\nruff handles pycodestyle (E), Pyflakes (F), isort (I), warnings (W), bandit (S),\nbugbear (B), and annotations (ANN). Don't flag issues these would catch.\n\n### What to look for\n\n**Type annotations**\n\n- Required on all function signatures\n- Use `typing_extensions` for backward compatibility\n- Avoid `Any` where a concrete type is possible\n- Check that type annotations actually match runtime behavior\n\n**Compatibility**\n\n- Must support Python 3.10 through 3.13\n- Watch for syntax/stdlib features only available in newer versions\n- `from __future__ import annotations` if using newer annotation syntax\n\n**Error handling**\n\n- No bare `except:` or `except Exception:` that swallows everything\n- Exceptions should have descriptive messages\n- Resource cleanup with context managers, not try/finally when avoidable\n\n**Async patterns**\n\n- Tests use pytest-asyncio -- async tests need proper fixtures\n- Watch for blocking calls inside async functions\n- Proper cleanup of async resources (aclose, async context managers)\n\n**Predictor interface**\n\n- `base_predictor.py` is the core interface -- changes here affect all users\n- `types.py` defines input/output types -- check backward compatibility\n- Server code in `python/cog/server/` handles HTTP -- watch for request handling bugs\n\n**Testing**\n\n- Uses pytest with fixtures\n- tox runs tests across Python 3.10-3.13\n- Test isolation: don't rely on global state or test ordering\n\n\n---\nname: rust-review\ndescription: Rust code review guidelines for Coglet\n---\n\n## Rust review guidelines\n\nThis project uses Rust for Coglet (`crates/`), the prediction server that runs\ninside Cog containers. It handles HTTP requests, worker process management, and\nprediction execution.\n\n### What linters already catch (skip these)\n\nclippy runs in CI. cargo-deny audits dependencies for license/advisory issues.\nDon't flag issues these would catch.\n\n### What to look for\n\n**Error handling**\n\n- Use `thiserror` for typed errors in library code, `anyhow` for application errors\n- Don't use `.unwrap()` or `.expect()` in non-test code unless the invariant is documented\n- Error context: use `.context()` or `.with_context()` from anyhow, not bare `?`\n\n**Ownership and lifetimes**\n\n- Unnecessary cloning where a borrow would work\n- Lifetime issues that suggest a design problem (not just annotation noise)\n- Arc/Mutex when simpler patterns exist\n\n**Async (tokio)**\n\n- Blocking operations inside async contexts (use `spawn_blocking`)\n- Missing `.await` on futures (compiler catches some, but not all logical issues)\n- Proper cancellation handling and cleanup\n- Task/JoinHandle leaks\n\n**Safety**\n\n- Any `unsafe` block needs justification and a safety comment\n- FFI boundaries with Python (PyO3) -- check for panics across FFI, GIL handling\n- Memory safety in IPC between parent and worker processes\n\n**Architecture**\n\n- `crates/coglet/` is the core: HTTP server, worker orchestration, IPC\n- `crates/coglet-python/` is PyO3 bindings for Python predictor integration\n- Two-process architecture: parent (HTTP + orchestrator) and worker (Python execution)\n- Don't mix IPC concerns with HTTP handling\n- Snapshot tests use `insta` -- check if snapshots need updating\n\n**Dependencies**\n\n- New dependencies should be justified -- `crates/deny.toml` audits them\n- Prefer std or existing deps over adding new ones for small tasks\n\n\n# Complex-output Skill\n\nPath: `tools/test-harness/fixtures/models/complex-output/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/complex-output/cog.yaml)\n\n\n# Constraints-and-choices Skill\n\nPath: `tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/constraints-and-choices/cog.yaml)\n\n\n# File-path-types Skill\n\nPath: `tools/test-harness/fixtures/models/file-path-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/file-path-types/cog.yaml)\n\n\n# List-types Skill\n\nPath: `tools/test-harness/fixtures/models/list-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/list-types/cog.yaml)\n\n\n# Optional-list-types Skill\n\nPath: `tools/test-harness/fixtures/models/optional-list-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-list-types/cog.yaml)\n\n\n# Optional-types Skill\n\nPath: `tools/test-harness/fixtures/models/optional-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/optional-types/cog.yaml)\n\n\n# Scalar-types Skill\n\nPath: `tools/test-harness/fixtures/models/scalar-types/cog.yaml`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-harness/fixtures/models/scalar-types/cog.yaml)\n\n\n# Cog Model Test Manifest\n# =======================\n# Each entry defines a model to test, its inputs, and expected outputs.\n#\n# Input values prefixed with \"@\" are resolved as fixture file paths relative\n# to the fixtures/ directory (e.g. \"@test_image.png\" -> fixtures/test_image.png).\n#\n# Validation types:\n# exact - output string must equal `value` exactly\n# contains - output string must contain `value` as a substring\n# regex - output string must match `pattern`\n# file_exists - output is a file path; optionally check `mime` type\n# json_match - parse output as JSON, assert `match` is a subset\n# json_keys - parse output as JSON dict, assert it has entries\n# not_empty - output is non-empty (loose smoke test)\n#\n# Setup commands:\n# Optional `setup` list of shell commands to run after cloning the repo\n# but before building. Useful for models that generate cog.yaml from\n# templates or need other preparation steps.\n# Example:\n# setup:\n# - \"script/select.sh dev\"\n#\n# Required tools:\n# Optional `requires_tools` list of CLI tools that must be on PATH for\n# setup commands to work. The harness checks before running and prints\n# install instructions for known tools (yq, envsubst).\n# Example:\n# requires_tools: [\"yq\"]\n#\n# Model resolution (base_dir takes priority over repo):\n# - Set `base_dir` for local models -- `path` is resolved under it. A\n# relative `base_dir` is resolved against the repo root; an absolute path\n# is used as-is. Local only; never clones.\n# - Set `repo: owner/name` (with no `base_dir`) to clone from GitHub.\n# - Set neither to default to fixtures/models/ (local fixtures).\n# `repo: local` is no longer supported -- use `base_dir` instead.\n\ndefaults:\n sdk_version: \"latest\" # \"latest\" = newest stable from PyPI; or pin e.g. \"0.16.12\"\n cog_version: \"latest\" # \"latest\" = newest stable release; or pin e.g. \"v0.16.12\"\n\nmodels:\n # ── cog-examples (CPU) ──────────────────────────────────────────────\n\n - name: hello-world\n base_dir: examples\n path: hello-world\n gpu: false\n tests:\n - description: \"basic predict\"\n inputs:\n text: \"world\"\n expect:\n type: exact\n value: \"hello world\"\n\n - name: canary\n base_dir: examples\n path: canary\n gpu: false\n tests:\n - description: \"streaming concatenate iterator\"\n inputs:\n text: \"friend\"\n expect:\n type: contains\n value: \"friend\"\n\n - name: blur\n base_dir: examples\n path: blur\n gpu: false\n tests:\n - description: \"blur an image\"\n inputs:\n image: \"@test_image.png\"\n blur: 5\n expect:\n type: file_exists\n mime: \"image/png\"\n\n - name: hello-image\n base_dir: examples\n path: hello-image\n gpu: false\n tests:\n - description: \"return a static image\"\n inputs: {}\n expect:\n type: file_exists\n\n - name: hello-concurrency\n base_dir: examples\n path: hello-concurrency\n gpu: false\n tests:\n - description: \"async streaming output\"\n inputs:\n total: 3\n interval: 0\n expect:\n type: contains\n value: \"Apple\"\n\n - name: hello-context\n base_dir: examples\n path: hello-context\n gpu: false\n # NOTE: This model uses current_scope().context which may not be\n # available in coglet yet. A failure here is a real compatibility signal.\n tests:\n - description: \"returns input and context\"\n inputs:\n text: \"testing\"\n expect:\n type: json_match\n match:\n inputs:\n text: \"testing\"\n\n - name: hello-train\n base_dir: examples\n path: hello-train\n gpu: false\n train_tests:\n - description: \"train produces weights file\"\n inputs:\n prefix: \"custom\"\n expect:\n type: not_empty\n tests:\n - description: \"predict with default weights\"\n inputs:\n text: \"world\"\n expect:\n type: contains\n value: \"world\"\n\n - name: notebook\n base_dir: examples\n path: notebook\n gpu: false\n tests:\n - description: \"basic predict\"\n inputs:\n name: \"world\"\n expect:\n type: contains\n value: \"hello\"\n\n - name: managed-weights\n base_dir: examples\n path: managed-weights\n gpu: false\n requires_env:\n - COG_MANAGED_WEIGHTS_READY\n tests:\n - description: \"weight validation smoke test\"\n inputs: {}\n expect:\n type: not_empty\n\n - name: streaming-text\n base_dir: examples\n path: streaming-text\n gpu: false\n tests:\n - description: \"streaming text generation\"\n inputs:\n prompt: \"Hello\"\n expect:\n type: not_empty\n\n # ── cog-examples (GPU) ──────────────────────────────────────────────\n\n - name: resnet\n base_dir: examples\n path: resnet\n gpu: true\n tests:\n - description: \"classify hotdog image\"\n inputs:\n image: \"@hotdog.png\"\n expect:\n type: json_keys\n\n - name: z-image-turbo\n base_dir: examples\n path: z-image-turbo\n gpu: true\n timeout: 600\n tests:\n - description: \"generate image from prompt\"\n inputs:\n prompt: \"a cat sitting on a windowsill\"\n expect:\n type: file_exists\n mime: \"image/png\"\n\n # ── Local fixture models ─────────────────────────────────────────────\n # These set base_dir to the fixtures dir (relative to the repo root), so\n # path is relative to it. They cover the full input type matrix for build\n # and prediction testing.\n\n - name: fixture-scalar-types\n base_dir: tools/test-harness/fixtures/models\n path: scalar-types\n gpu: false\n tests:\n - description: \"basic scalar inputs\"\n inputs:\n text: \"hello\"\n expect:\n type: contains\n value: \"hello\"\n\n - name: fixture-optional-types\n base_dir: tools/test-harness/fixtures/models\n path: optional-types\n gpu: false\n tests:\n - description: \"optional inputs default to None\"\n inputs:\n text: \"hello\"\n expect:\n type: exact\n value: \"hello\"\n\n - name: fixture-list-types\n base_dir: tools/test-harness/fixtures/models\n path: list-types\n gpu: false\n\n - name: fixture-optional-list-types\n base_dir: tools/test-harness/fixtures/models\n path: optional-list-types\n gpu: false\n tests:\n - description: \"optional lists default to None\"\n inputs:\n text: \"hello\"\n expect:\n type: exact\n value: \"hello\"\n\n - name: fixture-constraints-and-choices\n base_dir: tools/test-harness/fixtures/models\n path: constraints-and-choices\n gpu: false\n tests:\n - description: \"defaults with constraints and choices\"\n inputs: {}\n expect:\n type: exact\n value: \"hello-0.7-50-balanced-1\"\n\n - name: fixture-file-path-types\n base_dir: tools/test-harness/fixtures/models\n path: file-path-types\n gpu: false\n\n - name: fixture-complex-output\n base_dir: tools/test-harness/fixtures/models\n path: complex-output\n gpu: false\n tests:\n - description: \"structured BaseModel output\"\n inputs:\n prompt: \"test\"\n expect:\n type: json_match\n match:\n text: \"generated: test\"\n score: 0.95\n\n # ── External models (add your own below) ────────────────────────────\n # - name: my-custom-model\n # repo: myorg/my-model-repo\n # path: \".\"\n # gpu: true\n # # sdk_version: \"0.16.12\" # optional per-model override\n # env:\n # HF_TOKEN: \"${HF_TOKEN}\"\n # timeout: 600\n # tests:\n # - description: \"smoke test\"\n # inputs:\n # prompt: \"hello\"\n # expect:\n # type: contains\n # value: \"result\"\n #\n # Models that need setup commands before building (e.g. generating cog.yaml):\n # - name: flux-dev\n # repo: replicate/cog-flux\n # path: \".\"\n # gpu: true\n # setup:\n # - \"script/select.sh dev\"\n # timeout: 900\n # tests:\n # - description: \"generate image\"\n # inputs:\n # prompt: \"a cat\"\n # expect:\n # type: file_exists\n\n\n# Cog Model Test Harness\n\nAutomated test harness for validating Cog models against new CLI/SDK versions.\n\nThis is the Go implementation and replaces the previous Python harness.\n\n## Quick Start\n\n```bash\ncd tools/test-harness\n\n# List all models in the manifest\ngo run . list\n\n# Run all non-GPU models\ngo run . run --no-gpu\n\n# Run a specific model\ngo run . run --model hello-world\n\n# Run GPU models only (requires NVIDIA GPU + nvidia-docker)\ngo run . run --gpu-only\n\n# Output JSON report\ngo run . run --no-gpu --output json --output-file results/report.json\n\n# Build images only (no predictions)\ngo run . build --no-gpu\n\n# Use a locally-built cog binary\ngo run . run --no-gpu --cog-binary ../../dist/go/darwin-arm64/cog\n\n# Test from a git branch (builds CLI + SDK wheel automatically)\ngo run . run --no-gpu --cog-ref main\n```\n\nYou can also run from repo root:\n\n```bash\n(cd tools/test-harness && go run . list)\n(cd tools/test-harness && go run . run --no-gpu)\n```\n\n## Prerequisites\n\n- Go (see repo toolchain)\n- Docker\n- For `--cog-ref`: Go + uv on PATH (`mise install` sets this up)\n- For GPU models: NVIDIA GPU + nvidia-docker runtime\n\n## Manifest and Fixtures\n\n- Manifest: `tools/test-harness/manifest.yaml`\n- Fixtures: `tools/test-harness/fixtures/`\n\nInput values prefixed with `@` resolve to fixture paths. Example:\n\n```yaml\ninputs:\n image: \"@test_image.png\"\n```\n\n## CLI Reference\n\n```text\nusage: test-harness {run,build,list} [options]\n\nCommands:\n run Build and test models (full pipeline)\n build Build Docker images only (no predictions)\n list List models defined in the manifest\n\nCommon options:\n --manifest PATH Path to manifest.yaml\n --model NAME Run only this model (repeatable)\n --no-gpu Skip GPU models\n --gpu-only Only run GPU models\n --sdk-version VER SDK version override\n --cog-version TAG CLI version to download\n --cog-binary PATH Path to local cog binary\n --cog-ref REF Git ref to build cog from source\n --sdk-wheel PATH Local SDK wheel path\n --clean-images Remove Docker images after run (default: keep them)\n --keep-outputs Preserve prediction outputs and print the work directory path\n\nRun options:\n --output {console,json} Output format\n --output-file PATH Write JSON report to file\n```\n\n## Project Layout\n\n```text\ntools/test-harness/\n├── go.mod\n├── main.go\n├── README.md\n├── manifest.yaml\n├── fixtures/\n├── cmd/\n└── internal/\n```\n\n\n# test-registry-util Documentation\n\nPath: `tools/test-registry-util/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/replicate/cog/HEAD/tools/test-registry-util/README.md)\n\n\n---\ndescription: Read-only code reviewer for pull requests\nmode: primary\nmodel: cf-gateway/deepseek-v4-flash-0731\ntemperature: 0.1\npermission:\n edit: deny\n bash:\n \"*\": deny\n \"git diff*\": allow\n \"git log*\": allow\n \"git show*\": allow\n webfetch: deny\n---\n\nYou are a code reviewer for the Cog project. Cog packages ML models into\nproduction-ready containers. The codebase has three languages: Go (CLI and\ntooling), Python (SDK), and Rust (coglet prediction server).\n\n## How to review\n\n1. Read the full diff. Understand what the PR is trying to do before commenting.\n2. Load the appropriate review skill(s) for the languages touched in the PR:\n - Go files: load `go-review`\n - Python files: load `python-review`\n - Rust files: load `rust-review`\n - Architecture/cross-cutting: load `cog-review`\n3. Focus on substance, not style. Linters handle formatting.\n4. Scope feedback to the PR's purpose. Don't demand unrelated refactors.\n5. Be direct. Say what the problem is, why it matters, and how to fix it.\n\n## Comment format\n\n- Reference specific files and lines: `path/to/file.go:42`\n- Use ```suggestion blocks when proposing concrete code changes\n- Categorize findings:\n - **Blocker**: Must fix before merge (bugs, security, data loss)\n - **Should fix**: Important but not blocking (error handling gaps, edge cases)\n - **Nit**: Minor improvement, take it or leave it\n- If the PR looks good, say so briefly. Don't invent problems.\n"}