exo Documentation

Python repository by @exo-explore. Desktop release binaries & installers with 48k stars.

48,005 stars Python Python 2 files · ~28,948 tokens
RAW Doc

File: README.md

exo connects all your devices into an AI cluster. Not only does exo enable running models larger than would fit on a single device, but with day-0 support for RDMA over Thunderbolt, makes models run faster as you add more devices.

Features

  • Automatic Device Discovery: Devices running exo automatically discover each other - no manual configuration.
  • RDMA over Thunderbolt: exo ships with day-0 support for RDMA over Thunderbolt 5, enabling 99% reduction in latency between devices.
  • Topology-Aware Auto Parallel: exo figures out the best way to split your model across all available devices based on a realtime view of your device topology. It takes into account device resources and network latency/bandwidth between each link.
  • Tensor Parallelism: exo supports sharding models, for up to 1.8x speedup on 2 devices and 3.2x speedup on 4 devices.
  • MLX Support: exo uses MLX as an inference backend and MLX distributed for distributed communication.
  • Multiple API Compatibility: Compatible with OpenAI Chat Completions API, Claude Messages API, OpenAI Responses API, and Ollama API - use your existing tools and clients.
  • Custom Model Support: Load custom models from HuggingFace hub to expand the range of available models.

Dashboard

exo includes a built-in dashboard for managing your cluster and chatting with models.

Benchmarks

Qwen3-235B (8-bit) on 4 × M3 Ultra Mac Studio with Tensor Parallel RDMA

text
**Source:** [Jeff Geerling: 15 TB VRAM on Mac Studio – RDMA over Thunderbolt 5](https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-studio-rdma-over-thunderbolt-5)

DeepSeek v3.1 671B (8-bit) on 4 × M3 Ultra Mac Studio with Tensor Parallel RDMA

text
**Source:** [Jeff Geerling: 15 TB VRAM on Mac Studio – RDMA over Thunderbolt 5](https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-studio-rdma-over-thunderbolt-5)

Kimi K2 Thinking (native 4-bit) on 4 × M3 Ultra Mac Studio with Tensor Parallel RDMA

text
**Source:** [Jeff Geerling: 15 TB VRAM on Mac Studio – RDMA over Thunderbolt 5](https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-studio-rdma-over-thunderbolt-5)

Quick Start

Devices running exo automatically discover each other, without needing any manual configuration. Each device provides an API and a dashboard for interacting with your cluster (runs at http://localhost:52415).

You can run exo with the macOS app, the easiest way on a Mac, or from source on macOS or Linux.

macOS App

exo ships a macOS app that runs in the background on your Mac.

The macOS app requires macOS Tahoe 26.2 or later.

Download the latest build here: EXO-latest.dmg.

You can also install the latest build with Homebrew:

bash
brew install --cask exo

The app will ask for permission to modify system settings and install a new Network profile. Improvements to this are being worked on.

Custom Namespace for Cluster Isolation:

The macOS app includes a custom namespace feature that allows you to isolate your exo cluster from others on the same network. This is configured through the EXO_ZENOH_NAMESPACE setting:

  • Use cases:

    • Running multiple separate exo clusters on the same network
    • Isolating development/testing clusters from production clusters
    • Preventing accidental cluster joining

  • Configuration: Access this setting in the app's Advanced settings (or set the EXO_ZENOH_NAMESPACE environment variable when running from source)

The namespace is logged on startup for debugging purposes.

Uninstalling the macOS App

The recommended way to uninstall is through the app itself: click the menu bar icon → Advanced → Uninstall. This cleanly removes all system components.

If you've already deleted the app, you can run the standalone uninstaller script:

bash
sudo ./app/EXO/uninstall-exo.sh

This removes:

  • Network setup LaunchDaemon
  • Network configuration script
  • Log files
  • The "exo" network location

Note: You'll need to manually remove EXO from Login Items in System Settings → General → Login Items.

Run from Source (macOS)

If you have Nix installed, you can skip most of the steps below and run exo directly:

bash
nix run .#exo

Note: To accept the Cachix binary cache (and avoid the Xcode Metal ToolChain), add these lines to /etc/nix/nix.conf, replacing your-username with your macOS username (whoami prints it):

text
trusted-users = root your-username
experimental-features = nix-command flakes

Then restart the Nix daemon: sudo launchctl kickstart -k system/org.nixos.nix-daemon

Prerequisites:

  • Xcode (provides the Metal ToolChain required for MLX compilation)

  • brew (for simple package management on macOS)

    bash
    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

  • uv (for Python dependency management)

  • node (for building the dashboard)

    bash
    brew install uv node

  • rust (stable, to build the Rust bindings)

    bash
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

  • macmon (optional, for GPU usage, power and temperature monitoring on Apple Silicon; without it exo falls back to psutil and monitors memory only)

    Install the pinned fork revision used by this repo instead of Homebrew macmon.
    Homebrew macmon 0.6.1 still crashes on Apple M5.

    bash
    cargo install --git https://github.com/vladkens/macmon \
  --rev a1cd06b6cc0d5e61db24fd8832e74cd992097a7d \
  macmon \
  --force

Clone the repo, build the dashboard, install the dependencies, and run exo:

bash
# Clone exo
git clone https://github.com/exo-explore/exo

# Build dashboard
cd exo/dashboard && npm install && npm run build && cd ..

# Install Python dependencies, including the MLX backend
uv sync --extra mlx

# Run exo
uv run exo

This starts the exo dashboard and API at http://localhost:52415/

Please view the section on RDMA to enable this feature on MacOS >=26.2!

Run from Source (Linux)

Prerequisites:

  • uv (for Python dependency management)
  • node (for building the dashboard) - version 18 or higher
  • rust (stable, to build the Rust bindings)

Installation methods:

Option 1: Using system package manager (Ubuntu/Debian example):

bash
# Install Node.js and npm
sudo apt update
sudo apt install nodejs npm

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install Rust (using rustup)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Option 2: Using Homebrew on Linux (if preferred):

bash
# Install Homebrew on Linux
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Install dependencies
brew install uv node

# Install Rust (using rustup)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Note: The macmon package is macOS-only and not required for Linux.

Clone the repo, build the dashboard, install the dependencies, and run exo:

bash
# Clone exo
git clone https://github.com/exo-explore/exo

# Build dashboard
cd exo/dashboard && npm install && npm run build && cd ..

# Install Python dependencies with the MLX backend for your hardware
# (NVIDIA: --extra mlx-cuda13 or --extra mlx-cuda12)
uv sync --extra mlx-cpu

# Run exo
uv run exo

This starts the exo dashboard and API at http://localhost:52415/

Important note for Linux users: Currently, exo runs on CPU on Linux. GPU support for Linux platforms is under development. If you'd like to see support for your specific Linux hardware, please search for existing feature requests or create a new one.

Configuration Options:

  • --no-worker: Run exo without the worker component. Useful for coordinator-only nodes that handle networking and orchestration but don't execute inference tasks. This is helpful for machines without sufficient GPU resources but with good network connectivity.

    bash
    uv run exo --no-worker

  • --legacy-daemon: Run exo as a legacy SysV-style background daemon using double-fork daemonization. This is intended for legacy init scripts; systemd and launchd should run exo in the foreground without this flag.

    bash
    uv run exo --legacy-daemon

File Locations (Linux):

exo follows the XDG Base Directory Specification on Linux:

  • Configuration files: ~/.config/exo/ (or $XDG_CONFIG_HOME/exo/)
  • Data files: ~/.local/share/exo/ (or $XDG_DATA_HOME/exo/)
  • Cache files: ~/.cache/exo/ (or $XDG_CACHE_HOME/exo/)
  • Log files: ~/.cache/exo/exo_log/ (with automatic log rotation)
  • Custom model cards: ~/.local/share/exo/custom_model_cards/

You can override these locations by setting the corresponding XDG environment variables.

Enabling RDMA on macOS

RDMA is a new capability added to macOS 26.2. It works on any Mac with Thunderbolt 5 (M4 Pro Mac Mini, M4 Max Mac Studio, M4 Max MacBook Pro, M3 Ultra Mac Studio).

Please refer to the caveats for immediate troubleshooting.

To enable RDMA on macOS, follow these steps:

  1. Shut down your Mac.
  2. Hold down the power button for 10 seconds until the boot menu appears.
  3. Select "Options" to enter Recovery mode.
  4. When the Recovery UI appears, open the Terminal from the Utilities menu.
  5. In the Terminal, type:
    text
    rdma_ctl enable
    and press Enter.
  6. Reboot your Mac.

After that, RDMA will be enabled in macOS and exo will take care of the rest.

Important Caveats

  1. Devices that wish to be part of an RDMA cluster must be connected to all other devices in the cluster.
  2. The cables must support TB5.
  3. On a Mac Studio, you cannot use the Thunderbolt 5 port next to the Ethernet port.
  4. If running from source, please use the script found at tmp/set_rdma_network_config.sh, which will disable Thunderbolt Bridge and set dhcp on each RDMA port.
  5. RDMA ports may be unable to discover each other on different versions of MacOS. Please ensure that OS versions match exactly (even beta version numbers) on all devices.

Environment Variables

exo supports several environment variables for configuration:

Variable Description Default
EXO_DEFAULT_MODELS_DIR Default directory for model downloads and caches. Always first in the writable dirs list. ~/.local/share/exo/models (Linux) or ~/.exo/models (macOS)
EXO_MODELS_DIRS Colon-separated additional writable directories for model downloads. Checked in order after the default; first with enough free space is used. None
EXO_MODELS_READ_ONLY_DIRS Colon-separated read-only directories to search for pre-downloaded models (e.g., NFS mounts, shared storage). Models here cannot be deleted. None
EXO_OFFLINE Run without internet connection (uses only local models) false
EXO_ENABLE_IMAGE_MODELS Enable image model support false
EXO_ZENOH_NAMESPACE Custom namespace for cluster isolation (same as --namespace) exo version
EXO_FAST_SYNCH Control MLX_METAL_FAST_SYNCH behavior (for JACCL backend) Auto
EXO_TRACING_ENABLED Enable distributed tracing for performance analysis false

Example usage:

bash
# Use pre-downloaded models from NFS mount (read-only)
EXO_MODELS_READ_ONLY_DIRS=/mnt/nfs/models:/opt/ai-models uv run exo

# Download models to an external SSD (falls back to default dir if full)
EXO_MODELS_DIRS=/Volumes/ExternalSSD/exo-models uv run exo

# Run in offline mode
EXO_OFFLINE=true uv run exo

# Enable image models
EXO_ENABLE_IMAGE_MODELS=true uv run exo

# Use custom namespace for cluster isolation
EXO_ZENOH_NAMESPACE=my-dev-cluster uv run exo

Using the API

exo provides multiple API-compatible interfaces for maximum compatibility with existing tools:

  • OpenAI Chat Completions API - Compatible with OpenAI clients
  • Claude Messages API - Compatible with Anthropic's Claude format
  • OpenAI Responses API - Compatible with OpenAI's Responses format
  • Ollama API - Compatible with Ollama and tools like OpenWebUI

If you prefer to interact with exo via the API, here is an example creating an instance of a small model (mlx-community/Llama-3.2-1B-Instruct-4bit), sending a chat completions request and deleting the instance.

1. Preview instance placements

The /instance/previews endpoint will preview all valid placements for your model.

bash
curl "http://localhost:52415/instance/previews?model_id=mlx-community/Llama-3.2-1B-Instruct-4bit"

Sample response:

json
{
  "previews": [
    {
      "model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit",
      "sharding": "Pipeline",
      "instance_meta": "MlxRing",
      "instance": {...},
      "memory_delta_by_node": {"local": 729808896},
      "error": null
    }
    // ...possibly more placements...
  ]
}

This will return all valid placements for this model. Pick a placement that you like.
To pick the first one, pipe into jq:

bash
curl "http://localhost:52415/instance/previews?model_id=mlx-community/Llama-3.2-1B-Instruct-4bit" | jq -c '.previews[] | select(.error == null) | .instance' | head -n1

2. Create a model instance

Send a POST to /instance with your desired placement in the instance field (the full payload must match types as in CreateInstanceParams), which you can copy from step 1:

bash
curl -X POST http://localhost:52415/instance \
  -H 'Content-Type: application/json' \
  -d '{
    "instance": {...}
  }'

Sample response:

json
{
  "message": "Command received.",
  "command_id": "e9d1a8ab-...."
}

This command is asynchronous. Before sending inference requests, wait until the
API sees the new instance for this model:

bash
curl -N "http://localhost:52415/instance/await?model_id=mlx-community/Llama-3.2-1B-Instruct-4bit"

The endpoint returns an SSE stream. A successful wait emits a message with
"type": "ready" and the matching instance; a timeout emits "type": "timeout".
By default it waits indefinitely. Set timeout_seconds to a positive value to
bound the wait.

3. Send a chat completion

Now, make a POST to /v1/chat/completions (the same format as OpenAI's API):

bash
curl -N -X POST http://localhost:52415/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "user", "content": "What is Llama 3.2 1B?"}
    ],
    "stream": true
  }'

4. Delete the instance

When you're done, delete the instance by its ID (find it via /state or /instance endpoints):

bash
curl -X DELETE http://localhost:52415/instance/YOUR_INSTANCE_ID

Claude Messages API Compatibility

Use the Claude Messages API format with the /v1/messages endpoint:

bash
curl -N -X POST http://localhost:52415/v1/messages \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "user", "content": "Hello"}
    ],
    "max_tokens": 1024,
    "stream": true
  }'

OpenAI Responses API Compatibility

Use the OpenAI Responses API format with the /v1/responses endpoint:

bash
curl -N -X POST http://localhost:52415/v1/responses \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "input": "Hello",
    "stream": true
  }'

Ollama API Compatibility

exo supports Ollama API endpoints for compatibility with tools like OpenWebUI:

bash
# Ollama chat
curl -X POST http://localhost:52415/ollama/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "user", "content": "Hello"}
    ],
    "stream": false
  }'

# List models (Ollama format)
curl http://localhost:52415/ollama/api/tags

Custom Model Loading from HuggingFace

You can add custom models from the HuggingFace hub:

bash
curl -X POST http://localhost:52415/models/add \
  -H 'Content-Type: application/json' \
  -d '{
    "model_id": "mlx-community/my-custom-model"
  }'

Security Note:

Custom models requiring trust_remote_code in their configuration must be explicitly enabled (default is false) for security. Only enable this if you trust the model's remote code execution. Models are fetched from HuggingFace and stored locally as custom model cards.

Other useful API endpoints:*

  • List all models: curl http://localhost:52415/models
  • List downloaded models only: curl http://localhost:52415/models?status=downloaded
  • Search HuggingFace: curl "http://localhost:52415/models/search?query=llama&limit=10"
  • Inspect instance IDs and deployment state: curl http://localhost:52415/state

For further details, see:

Benchmarking

The exo-bench tool measures model prefill and token generation speed across different placement configurations. This helps you optimize model performance and validate improvements.

Prerequisites:

  • Nodes should be running with uv run exo before benchmarking
  • The tool uses the /bench/chat/completions endpoint

Basic usage:

bash
uv run bench/exo_bench.py \
  --model Llama-3.2-1B-Instruct-4bit \
  --pp 128,256,512 \
  --tg 128,256

Key parameters:

  • --model: Model to benchmark (short ID or HuggingFace ID)
  • --pp: Prompt size hints (comma-separated integers)
  • --tg: Generation lengths (comma-separated integers)
  • --max-nodes: Limit placements to N nodes (default: 4)
  • --instance-meta: Filter by ring, jaccl, or both (default: both)
  • --sharding: Filter by pipeline, tensor, or both (default: both)
  • --repeat: Number of repetitions per configuration (default: 1)
  • --warmup: Warmup runs per placement (default: 0)
  • --json-out: Output file for results (default: bench/results.json)

Example with filters:

bash
uv run bench/exo_bench.py \
  --model Llama-3.2-1B-Instruct-4bit \
  --pp 128,512 \
  --tg 128 \
  --max-nodes 2 \
  --sharding tensor \
  --repeat 3 \
  --json-out my-results.json

The tool outputs performance metrics including prompt tokens per second (prompt_tps), generation tokens per second (generation_tps), and peak memory usage for each configuration.

Hardware Accelerator Support

On macOS, exo uses the GPU. On Linux, exo currently runs on CPU. We are working on extending hardware accelerator support. If you'd like support for a new hardware platform, please search for an existing feature request and add a thumbs up so we know what hardware is important to the community.

Contributing

See CONTRIBUTING.md for guidelines on how to contribute to exo.

File: AGENTS.md

AGENTS.md

This file provides guidance to AI coding agents when working with code in this repository.

Project Overview

exo is a distributed AI inference system that connects multiple devices into a cluster. It enables running large language models across multiple machines using MLX as the inference backend and zenoh for peer-to-peer networking.

Build & Run Commands

bash
# Build the dashboard (required before running exo)
cd dashboard && npm install && npm run build && cd ..

# Run exo (starts both master and worker with API at http://localhost:52415)
uv run exo

# Run with verbose logging
uv run exo -v   # or -vv for more verbose

# Run tests (excludes slow tests by default)
uv run pytest

# Run all tests including slow tests
uv run pytest -m ""

# Run a specific test file
uv run pytest src/exo/shared/tests/test_election.py

# Run a specific test function
uv run pytest src/exo/shared/tests/test_election.py::test_function_name

# Type checking (strict mode)
uv run basedpyright

# Linting
uv run ruff check

# Format code (using nix)
nix fmt

Pre-Commit Checks (REQUIRED)

IMPORTANT: Always run these checks before committing code. CI will fail if these don't pass.

bash
# 1. Type checking - MUST pass with 0 errors
uv run basedpyright

# 2. Linting - MUST pass
uv run ruff check

# 3. Formatting - MUST be applied
nix fmt

# 4. Tests - MUST pass
uv run pytest

Run all checks in sequence:

bash
uv run basedpyright && uv run ruff check && nix fmt && uv run pytest

If nix fmt changes any files, stage them before committing. The CI runs nix flake check which verifies formatting, linting, and runs Rust tests.

Architecture

Node Composition

A single exo Node (src/exo/main.py) runs multiple components:

  • Router: zenoh-based pub/sub messaging via Rust bindings (exo_rs)
  • Worker: Handles inference tasks, downloads models, manages runner processes
  • Master: Coordinates cluster state, places model instances across nodes
  • Election: Bully algorithm for master election
  • API: FastAPI server for OpenAI-compatible chat completions

Message Flow

Components communicate via typed pub/sub topics (src/exo/routing/topics.py):

  • GLOBAL_EVENTS: Master broadcasts indexed events to all workers
  • LOCAL_EVENTS: Workers send events to master for indexing
  • COMMANDS: Workers/API send commands to master
  • ELECTION_MESSAGES: Election protocol messages
  • CONNECTION_MESSAGES: zenoh connection updates

Event Sourcing

The system uses event sourcing for state management:

  • State (src/exo/shared/types/state.py): Immutable state object
  • apply() (src/exo/shared/apply.py): Pure function that applies events to state
  • Master indexes events and broadcasts; workers apply indexed events
  • The master only keeps recent events. A node that falls further behind (e.g. a late joiner) gets a StateSnapshot instead of a replay, so never assume a node has seen every event since the session started

Key Type Hierarchy

  • src/exo/shared/types/: Pydantic models for all shared types
    • events.py: Event types (discriminated union)
    • commands.py: Command types
    • tasks.py: Task types for worker execution
    • state.py: Cluster state model

Rust Components

Rust code in rust/ provides:

  • networking: zenoh networking (gossipsub, peer discovery)
  • exo_rs: PyO3 bindings exposing Rust to Python
  • system_custodian: System-level operations

Dashboard

Svelte 5 + TypeScript frontend in dashboard/. Build output goes to dashboard/build/ and is served by the API.

Code Style Requirements

From .cursorrules:

  • Strict, exhaustive typing - never bypass the type-checker
  • Use Literal[...] for enum-like sets, typing.NewType for primitives
  • Pydantic models with frozen=True and strict=True
  • Pure functions with injectable effect handlers for side-effects
  • Descriptive names - no abbreviations or 3-letter acronyms
  • Catch exceptions only where you can handle them meaningfully
  • Use @final and immutability wherever applicable

Testing

Tests use pytest-asyncio with asyncio_mode = "auto". Tests are in tests/ subdirectories alongside the code they test. The EXO_TESTS=1 env var is set during tests.

Dashboard UI Testing & Screenshots

Building and Running the Dashboard

bash
# Build the dashboard (must be done before running exo)
cd dashboard && npm install && npm run build && cd ..

# Start exo (serves the dashboard at http://localhost:52415)
uv run exo &
sleep 8  # Wait for server to start

Taking Headless Screenshots with Playwright

Use Playwright with headless Chromium for programmatic screenshots — no manual browser interaction needed.

Setup (one-time):

bash
npx --yes playwright install chromium
cd /tmp && npm init -y && npm install playwright

Taking screenshots:

javascript
// Run from /tmp where playwright is installed: cd /tmp && node -e "..."
const { chromium } = require('playwright');
(async () => {
  const browser = await chromium.launch({ headless: true });
  const page = await browser.newPage({ viewport: { width: 1280, height: 800 } });
  await page.goto('http://localhost:52415', { waitUntil: 'networkidle' });
  await page.waitForTimeout(2000);

  // Inject test data into localStorage if needed (e.g., recent models)
  await page.evaluate(() => {
    localStorage.setItem('exo-recent-models', JSON.stringify([
      { modelId: 'mlx-community/Qwen3-30B-A3B-4bit', launchedAt: Date.now() },
    ]));
  });
  await page.reload({ waitUntil: 'networkidle' });
  await page.waitForTimeout(2000);

  // Interact with UI elements
  await page.locator('text=SELECT MODEL').click();
  await page.waitForTimeout(1000);

  // Take screenshot
  await page.screenshot({ path: '/tmp/screenshot.png', fullPage: false });
  await browser.close();
})();

Uploading Images to GitHub PRs

GitHub's API doesn't support direct image upload for PR comments. Workaround:

  1. Commit images to the branch (temporarily):

    bash
    cp /tmp/screenshot.png .
git add screenshot.png
git commit -m "temp: add screenshots for PR"
git push origin <branch>
COMMIT_SHA=$(git rev-parse HEAD)

  2. Post PR comment referencing the raw image URL (uses permanent commit SHA so images survive deletion):

    bash
    gh pr comment <PR_NUMBER> --body "![Screenshot](https://raw.githubusercontent.com/exo-explore/exo/${COMMIT_SHA}/screenshot.png)"

  3. Remove the images from the branch:

    bash
    git rm screenshot.png
git commit -m "chore: remove temporary screenshot files"
git push origin <branch>

    The images still render in the PR comment because they reference the permanent commit SHA.

File: CONTRIBUTING.md

Contributing to EXO

Thank you for your interest in contributing to EXO!

Getting Started

To run EXO from source:

Prerequisites:

  • uv (for Python dependency management)
    bash
    brew install uv
  • rust (to build Rust bindings, nightly for now)
    bash
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup toolchain install nightly
  • macmon (for hardware monitoring on Apple Silicon)
    Use the pinned fork revision used by this repo instead of Homebrew macmon.
    bash
    cargo install --git https://github.com/vladkens/macmon \
  --rev a1cd06b6cc0d5e61db24fd8832e74cd992097a7d \
  macmon \
  --force
bash
git clone https://github.com/exo-explore/exo.git
cd exo/dashboard
npm install && npm run build && cd ..
uv sync --extra mlx
uv run exo

Development

EXO is built with a mix of Rust, Python, and TypeScript (Svelte for the dashboard), and the codebase is actively evolving. Before starting work:

  • Pull the latest source to ensure you're working with the most recent code
  • Keep your changes focused - implement one feature or fix per pull request
  • Avoid combining unrelated changes, even if they seem small

This makes reviews faster and helps us maintain code quality as the project evolves.

Code Style

Write pure functions where possible. When adding new code, prefer Rust unless there's a good reason otherwise. Leverage the type systems available to you - Rust's type system, Python type hints, and TypeScript types. Comments should explain why you're doing something, not what the code does - especially for non-obvious decisions.

Run nix fmt to auto-format your code before submitting.

Model Cards

EXO uses TOML-based model cards to define model metadata and capabilities. Model cards are stored in:

  • resources/inference_model_cards/ for text generation models
  • resources/image_model_cards/ for image generation models
  • ~/.exo/custom_model_cards/ for user-added custom models

Adding a Model Card

To add a new model, create a TOML file with the following structure:

toml
model_id = "mlx-community/Llama-3.2-1B-Instruct-4bit"
n_layers = 16
hidden_size = 2048
supports_tensor = true
tasks = ["TextGeneration"]
family = "llama"
quantization = "4bit"
base_model = "Llama 3.2 1B"
capabilities = ["text"]

[storage_size]
in_bytes = 729808896

Required Fields

  • model_id: Hugging Face model identifier
  • n_layers: Number of transformer layers
  • hidden_size: Hidden dimension size
  • supports_tensor: Whether the model supports tensor parallelism
  • tasks: List of supported tasks (TextGeneration, TextToImage, ImageToImage)
  • family: Model family (e.g., "llama", "deepseek", "qwen")
  • quantization: Quantization level (e.g., "4bit", "8bit", "bf16")
  • base_model: Human-readable base model name
  • capabilities: List of capabilities (e.g., ["text"], ["text", "thinking"])

Optional Fields

  • components: For multi-component models (like image models with separate text encoders and transformers)
  • uses_cfg: Whether the model uses classifier-free guidance (for image models)
  • trust_remote_code: Whether to allow remote code execution (defaults to false for security)

Capabilities

The capabilities field defines what the model can do:

  • text: Standard text generation
  • thinking: Model supports chain-of-thought reasoning
  • thinking_toggle: Thinking can be enabled/disabled via enable_thinking parameter
  • image_edit: Model supports image-to-image editing (FLUX.1-Kontext)

Security Note

By default, trust_remote_code is set to false for security. Only enable it if the model explicitly requires remote code execution from the Hugging Face hub.

API Adapters

EXO supports multiple API formats through an adapter pattern. Adapters convert API-specific request formats to the internal TextGenerationTaskParams format and convert internal token chunks back to API-specific responses.

Adapter Architecture

All adapters live in src/exo/master/adapters/ and follow the same pattern:

  1. Convert API-specific requests to TextGenerationTaskParams
  2. Handle both streaming and non-streaming response generation
  3. Convert internal TokenChunk objects to API-specific formats
  4. Manage error handling and edge cases

Existing Adapters

  • chat_completions.py: OpenAI Chat Completions API
  • claude.py: Anthropic Claude Messages API
  • responses.py: OpenAI Responses API
  • ollama.py: Ollama API (for OpenWebUI compatibility)

Adding a New API Adapter

To add support for a new API format:

  1. Create a new adapter file in src/exo/master/adapters/
  2. Implement a request conversion function:
    python
    def your_api_request_to_text_generation(
    request: YourAPIRequest,
) -> TextGenerationTaskParams:
    # Convert API request to internal format
    pass
  3. Implement streaming response generation:
    python
    async def generate_your_api_stream(
    command_id: CommandId,
    chunk_stream: AsyncGenerator[TokenChunk | ErrorChunk | ToolCallChunk, None],
) -> AsyncGenerator[str, None]:
    # Convert internal chunks to API-specific streaming format
    pass
  4. Implement non-streaming response collection:
    python
    async def collect_your_api_response(
    command_id: CommandId,
    chunk_stream: AsyncGenerator[TokenChunk | ErrorChunk | ToolCallChunk, None],
) -> AsyncGenerator[str]:
    # Collect all chunks and return single response
    pass
  5. Register the adapter endpoints in src/exo/master/api.py

The adapter pattern keeps API-specific logic isolated from core inference systems. Internal systems (worker, runner, event sourcing) only see TextGenerationTaskParams and TokenChunk objects - no API-specific types cross the adapter boundary.

For detailed API documentation, see docs/api.md.

Testing

EXO relies heavily on manual testing at this point in the project, but this is evolving. Before submitting a change, test both before and after to demonstrate how your change improves behavior. Do the best you can with the hardware you have available - if you need help testing, ask and we'll do our best to assist. Add automated tests where possible - we're actively working to substantially improve our automated testing story.

Submitting Changes

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/your-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin feature/your-feature)
  5. Open a Pull Request and follow the PR template

Reporting Issues

If you find a bug or have a feature request, please open an issue on GitHub with:

  • A clear description of the problem or feature
  • Steps to reproduce (for bugs)
  • Expected vs actual behavior
  • Your environment (macOS version, hardware, etc.)

Questions?

Join our community:

File: PLATFORMS.md

EXO Platform support (partial roadmap)

Tier 1 support - tested and maintained

Apple Silicon MacOS

  • Mac Studio: M3 Ultra
  • Mac Mini: M4 Pro
  • Macbook Pro: M5, M4 Max

Tier 2 support - checked occasionally, should run without crashing

Tier 3 support - minimal support and testing, but no theoretical reason it shouldnt work

Planned

Tier 1

Linux CUDA Support

  • Nvidia DGX Spark

Linux CPU Support

Tier 2

Linux Vulkan Support -- depends heavily on ecosystem

  • Framework Desktop

Linux CUDA Support -- depends heavily on ecosystem

  • Framework Desktop

Longer term!

Windows CUDA Support

Windows CPU Support

File: RULES.md

Repository Rules

  • if you see any code that violates these rules, raise it with me directly rather than trying to fix.
    • where applicable, file a GitHub Issue.
  • adhere to these rules strictly.

General Rules

  • if its possible to eliminate an extra try-catch or if-statement at runtime using type-level discipline, do it!
  • name your types, functions, and classes appropriately.
    • no three-letter acronyms.
    • no non-standard contractions.
    • each data type has a meaning, pick a name which is accurate and descriptive.
    • the average layman should be able to easily understand what your function does using the function signature alone!
      • sometimes, there will be exceptions. eg, when you're using specific technical terms that are well understood (saga, event, etc).
      • usually, you'll think that your code is an exception to the rules, but it won't be.

State, Functions and Classes

  • every function, given the same inputs, should produce the same outputs. ie, no hidden state.
  • use classes to prevent fixed state from being mutated arbitrarily (unsafely); methods provide a safe way of interfacing with state.
  • if your logic doesn't mutate fixed state, it probably belongs in a standalone function rather than a class.
  • functions shouldn't usually produce side-effects (they should be computationally pure).
    • if, for example, you're updating a state using an event (computationally pure), and you want to trigger a saga (computational side-effect), store the logic for triggering the saga into an effect handler (a function, capable of producing side-effects, that you pass into an otherwise computationally pure function, so that it may trigger side-effects safely).

Pydantic

  • read the Pydantic docs.
  • respect the Pydantic docs.
  • pydantic is all you need.
  • declare and re-use a central ConfigDict for your use-case, you'll usually want frozen and strict to be True.

Unique ID (UUID) Generation

  • inherit from Pydantic's UUID4 class to create your own UUID class.
  • use uuid.uuid4() to initialize your class with a fresh UUID where possible.
  • ensure that idempotency tags are generated by taking the salted hash of persisted state.
    • rationale: if a node crashes and resumes from an older state, it should not accidentally re-publish the same event twice under different idempotency tags.
    • every distinct function should feature a unique salt, so that there are no accidental collisions in idempotency tags.

Type Wrappers

  • reuse types that already exist in the Python standard library.
  • when two distinct data types are structurally identical (for example, different IDs which are both UUIDs but shouldn't never mixed up), make sure they can't be conflated by the type system.
    • if you're working with a primitive data type (str, int, etc), use NewType (it has zero runtime overhead).
    • if you're working with serializable data objects, consider adding a field (type str) that states its type.

Type Discipline

  • do not bypass the type-checker, preserve strict typing by any means necessary.
  • by default, use literal types (like Literal['one', 'two']) where an enum seems appropriate.

pro-tip: Python's type system is quite complex and feature-rich, so reading the documentation is often advisable; Matt discovered that Python typing library allows you to check that you've implemented a match exhaustively using Literal and get_args(type) after reading the docs.

Use of `@final`, Freezing

  • use wherever applicable.

Error Handling

  • don't try-catch for no reason.
  • make sure that you always know where and when the exceptions your code produces are meant to be handled, so that it's never a nasty surprise.
    • always write the rationale for your error-handling down in the docstring!
    • communicate the details to your colleagues when appropriate.

Dependencies

  • don't introduce any new dependencies without asking.
  • don't ask for any dependencies that aren't ubiquitous within production environments.

Commit Messages

  • use the imperative mood in the subject line.
  • prefix the subject line with a change type. our change types are:
    • documentation: documentation changes.
    • feature: a new feature.
    • refactor: a code change that neither fixes a bug nor adds a feature.
    • bugfix: a bug fix.
    • chore: routine tasks, maintenance, or tooling changes.
    • test: adding or correcting tests.
  • restrict the subject line to fifty characters or less.
  • capitalize the subject line.
  • do not end the subject line with a period.
  • separate subject from body with a blank line.

File: TODO.md

  1. EXO_BOOTSTRAP_PEERS is currently broken

  2. I'd like to see profiled network latency / bandwidth.

  3. I'd like to see how much bandwidth each link is using.

  4. Solve the problem of in continuous batching when a new prompt comes in, it will block decode of the current batch until the prefill is complete.

  5. We want people to be able to copy models over to a new device without ever connecting EXO to the internet. Right now EXO require internet connection once to cache some files to check if a download is complete. Instead, we should simply check if there is a non-empty model folder locally with no .partial files. This indicates it's a fully downloaded model that can be loaded.

  6. Memory pressure instead of memory used.

  7. Show the type of each connection (TB5, Ethernet, etc.) in the UI. Refer to old exo: https://github.com/exo-explore/exo/blob/56f783b38dc6b08ce606b07a5386dc40dae00330/exo/helpers.py#L251

  8. Dynamically switch to higher priority connection when it becomes available. Probably bring back InstanceReplacedAtomically.

  9. Faster model loads by streaming model from other devices in cluster.

  10. Add support for specifying the type of network connection to use in a test. Depends on 15/16.

  11. Log cleanup - per-module log filters and default to DEBUG log levels

  12. Validate RDMA connections with ibv_devinfo in the info gatherer

File: app/EXO/EXO/Assets.xcassets/Contents.json

{
"info" : {
"author" : "xcode",
"version" : 1
}
}

File: app/EXO/EXO/Assets.xcassets/AccentColor.colorset/Contents.json

{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}

File: app/EXO/EXO/Assets.xcassets/AppIcon.appiconset/Contents.json

{
"images" : [
{
"filename" : "16-mac.png",
"idiom" : "mac",
"scale" : "1x",
"size" : "16x16"
},
{
"filename" : "32-mac.png",
"idiom" : "mac",
"scale" : "2x",
"size" : "16x16"
},
{
"filename" : "32-mac 1.png",
"idiom" : "mac",
"scale" : "1x",
"size" : "32x32"
},
{
"filename" : "64-mac.png",
"idiom" : "mac",
"scale" : "2x",
"size" : "32x32"
},
{
"filename" : "128-mac.png",
"idiom" : "mac",
"scale" : "1x",
"size" : "128x128"
},
{
"filename" : "256-mac 1.png",
"idiom" : "mac",
"scale" : "2x",
"size" : "128x128"
},
{
"filename" : "256-mac.png",
"idiom" : "mac",
"scale" : "1x",
"size" : "256x256"
},
{
"filename" : "512-mac.png",
"idiom" : "mac",
"scale" : "2x",
"size" : "256x256"
},
{
"filename" : "512-mac 1.png",
"idiom" : "mac",
"scale" : "1x",
"size" : "512x512"
},
{
"filename" : "1024-mac.png",
"idiom" : "mac",
"scale" : "2x",
"size" : "512x512"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}

File: app/EXO/EXO/Assets.xcassets/menubar-icon.imageset/Contents.json

{
"images" : [
{
"filename" : "exo-logo-hq-square-transparent-bg.png",
"idiom" : "universal",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}

File: app/EXO/EXO/Preview Content/Preview Assets.xcassets/Contents.json

{
"info" : {
"author" : "xcode",
"version" : 1
}
}

File: bench/METHODOLOGY.md

Exo-Bench — Methodology

exo bench measures inference throughput and resource consumption of an exo cluster under controlled conditions. It sends prompts to the /bench/chat/completions endpoint, collects server-reported timing statistics, and records system-level metrics (power, GPU utilisation, temperature) throughout each run.

The goal is to have accurate, transparent and reproducible numbers to compare speed and scaling across different models and different setups, and to be able to track these results as optimizations and features are added to EXO.

Below is a technical summary of how Exo-Bench works. While the methodology and benchmark may change over time, this document will be kept up to date whenever this happens. If you find an issue with the methodology, or would like a feature to be added, please open a GitHub issue!

Prompt Construction

Benchmarks need prompts of an exact token length. Unfortunately, we do not have direct access to the model but just the chat completion endpoint. To get around this fact, we create a request that will tokenise to a certain prompt length.

This is achieved by:

  1. Tokenising a sample message through the model's apply_chat_template() to measure overhead (system tokens, special tokens, chat formatting).
  2. Binary-searching over a repeated atom string (default "a ") to find the content length that produces exactly the target number of tokens after template expansion.
  3. Returning both the content string and the verified token count.

The actual token count is recorded in every result row as pp_tokens, so downstream analysis can confirm the prompt hit its target.

Chat template formatting means that it may be impossible to attain very small pp benchmarks. e.g. pp=32 may not work. This tradeoff was made because the result of such a small prompt does not seem very interesting or useful for any real-world use cases.

Bench Endpoint

When a request reaches the server via the /bench/chat/completions endpoint, three things change compared to a normal chat completion:

  • KV prefix cache is disabled by default. Every request starts from a cold cache, ensuring prefill timing is not affected by prior requests. See Prefix Cache Mode for the --use-prefix-cache option.
  • EOS tokens are banned. A logits processor suppresses all end-of-sequence tokens, forcing the model to generate exactly max_tokens tokens. This guarantees consistent generation length for fair TPS comparison — the model cannot short-circuit a run by stopping early.
  • No model output parsing. The bench collection path concatenates raw token text without any model-specific post-processing (thinking tag extraction, structured output handling, etc.). This is to avoid model outputs such as tool parsing or any structural mistakes from breaking the benchmark - we are testing for speed; see Exo-Eval for performance metrics.

Timing

Prefill TPS

Measured server-side per task.

text
prefill_tps = num_prompt_tokens / prefill_wall_seconds

Generation TPS

Measured server-side per task. Each task records wall-clock timestamps as tokens arrive:

  • First generated token: timestamp recorded
  • Every subsequent token: timestamp updated

When generation completes:

text
gen_span = last_token_time - first_token_time
generation_tps = (completion_tokens - 1) / gen_span

The first token is excluded from the numerator because the rate measures inter-token throughput — the time between the first and last token divided by the number of intervals.

This does mean that tg=1 will not work.

Concurrency

Single Request

The client records wall-clock elapsed_s around the HTTP round-trip (network latency + server prefill + generation + response serialisation). This is a convenience metric for end-to-end latency. The authoritative TPS numbers come from the server-side per-task timing in the generation_stats response.

Concurrent Requests

When --concurrency N is set with N > 1, all N requests must hit the server at the same instant. The mechanism:

  1. The prompt is built once and shared across all threads.
  2. Each thread gets its own HTTP connection.
  3. A thread barrier blocks all threads until every thread is ready.
  4. The first thread past the barrier records the batch start time and signals the others.
  5. All threads use the same start time as their reference, then fire their HTTP request.
  6. Each thread's elapsed_s is measured from the shared start time to its own response completion.

Batch wall time is the maximum elapsed_s across all N requests — the time until the last request finishes.

Aggregate TPS

text
per_req_tps = max(generation_tps across N concurrent requests)
agg_gen_tps = per_req_tps * concurrency

max is used instead of mean because all requests run in parallel against the same model. The fastest request's generation rate represents the system's per-stream throughput capacity; multiplying by concurrency gives aggregate throughput.

Prefix Cache Mode

When --use-prefix-cache is passed, the KV prefix cache remains active during benchmarking. This speeds up repeated runs by skipping redundant prefill work, which is useful when prompt processing is not the focus of the benchmark (e.g. when measuring generation throughput or power consumption across many configurations).

Each response includes a prefix_cache_hit field ("none", "partial", or "exact"):

  • none: Cold prefill — no cached KV state was available. The reported prompt_tps is the real prefill throughput.
  • partial: A prefix of the prompt was found in cache. Only the remaining tokens were prefilled. The reported prompt_tps reflects the real throughput on the uncached portion. This occurs when multiple ascending --pp values share a common prefix (e.g. --pp 1000,5000 — the 5000-token prompt reuses the 1000-token cache entry and prefills the remaining 4000 tokens).
  • exact: The entire prompt was found in cache (e.g. same --pp value on a --repeat). No prefill work was done. The reported prompt_tps is the TPS from when the cache entry was originally created, not a new measurement.

Prompt TPS is approximate in this mode. Exact-hit runs report the stored TPS from the original cold/partial prefill rather than a freshly measured value. For accurate cold prefill numbers, run without --use-prefix-cache.

Ascending --pp order (e.g. --pp 1000,5000,10000) gives the most useful data: each size gets a meaningful partial hit except the first which is cold. Descending order produces exact hits with approximate TPS from a longer prompt's original run.

Warmup

Before measurement begins, --warmup N (default: 0) discarded requests are sent using the first pp/tg pair. Warmup results are not included in the output.

System Metrics

A background thread polls each node at 1 Hz, collecting:

  • GPU utilisation (%)
  • Temperature (C)
  • System power draw (W)
  • CPU cluster usage (performance and efficiency cores)

Energy is computed via trapezoidal integration of the power samples over each inference window (the wall-clock span of each benchmark request or concurrent batch). Average power is total_joules / total_inference_seconds. The server additionally returns a power_usage block in each non-stream /bench/chat/completions response that splits energy into prefill and generation phases, with the boundary anchored to the first non-PrefillProgressChunk from the runner.

Output Format

Results are written as JSON with three top-level keys:

  • runs: Array of per-request result objects, each containing:
    • elapsed_s, output_text_preview (first 200 chars)
    • stats: { prompt_tps, generation_tps, prompt_tokens, generation_tokens, peak_memory_usage }
    • power_usage: server-side total + prefill/generation split, per-node breakdown (non-stream requests only)
    • Placement metadata: model_id, placement_sharding, placement_instance_meta, placement_nodes
    • Run metadata: pp_tokens, tg, repeat_index, concurrency, concurrent_index
    • download_duration_s (if model was freshly downloaded)
  • cluster: Cluster state snapshot at time of benchmarking.
  • system_metrics: Per-node time-series samples (GPU, power, temperature).

Reproducing Results

bash
cd bench && uv run python exo_bench.py \
  --model "mlx-community/Qwen3.5-27B-4bit" \
  --instance-meta jaccl \
  --sharding tensor \
  --min-nodes 2 --max-nodes 2 \
  --pp 512 4096 --tg 128 \
  --repeat 3 \
  --warmup 1

Run --help for all the available flags.

File: dashboard/package.json

{
"name": "exo-dashboard",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite dev",
"build": "vite build",
"preview": "vite preview",
"prepare": "svelte-kit sync || echo ''",
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json"
},
"devDependencies": {
"@sveltejs/adapter-static": "^3.0.10",
"@sveltejs/kit": "^2.48.4",
"@sveltejs/vite-plugin-svelte": "^5.0.0",
"@tailwindcss/vite": "^4.0.0",
"@types/d3": "^7.4.3",
"@types/node": "^22",
"d3": "^7.9.0",
"prettier": "^3.4.2",
"prettier-plugin-svelte": "^3.3.3",
"svelte": "^5.0.0",
"svelte-check": "^4.0.0",
"tailwindcss": "^4.0.0",
"tw-animate-css": "^1.3.5",
"typescript": "^5.0.0",
"vite": "^6.0.0"
},
"dependencies": {
"devalue": "^5.6.2",
"highlight.js": "^11.11.1",
"katex": "^0.16.27",
"marked": "^17.0.1",
"mode-watcher": "^1.1.0",
"pdfjs-dist": "^5.6.205"
}
}

File: dashboard/tsconfig.json

{
"extends": "./.svelte-kit/tsconfig.json",
"compilerOptions": {
"allowJs": true,
"checkJs": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"skipLibCheck": true,
"sourceMap": true,
"strict": true,
"moduleResolution": "bundler"
}
}

File: docs/api.md

EXO API – Technical Reference

This document describes the REST API exposed by the EXO service, as implemented in:

src/exo/api/main.py (request and response types are in src/exo/api/types/)

The API is used to manage model instances in the cluster, inspect cluster state, and perform inference using multiple API-compatible interfaces.

Base URL example:

text
http://localhost:52415

1. General / Meta Endpoints

Get Master Node ID

GET /node_id

Returns the identifier of the current master node.

Response (example):

json
{
  "node_id": "node-1234"
}

Get Cluster State

GET /state

Returns the current state of the cluster, including nodes and active instances.

Response:
JSON object describing topology, nodes, and instances.

Get Events

GET /events

Returns the most recent internal events this node has applied, up to 10,000 of them (mainly for debugging and observability). Every generated token is an event, so under load this covers the last few seconds to minutes. A node that joined late, or fell far behind, catches up from a state snapshot, so it only has the events after that point.

Response:
Array of event objects.

2. Model Instance Management

Create Instance

POST /instance

Creates a new model instance in the cluster.

Request body (example):

json
{
  "instance": { ... }
}

The instance is a placement as returned by /instance/previews (each preview's
instance field), for example the first preview without an error.

Response:
Command acknowledgement. Instance creation is asynchronous; clients should wait
for the model to appear through /instance/await before sending inference
requests for that model.

Delete Instance

DELETE /instance/{instance_id}

Deletes an existing instance by ID.

Path parameters:

  • instance_id: string, ID of the instance to delete

Response:
Status / confirmation JSON.

Get Instance

GET /instance/{instance_id}

Returns details of a specific instance.

Path parameters:

  • instance_id: string

Response:
JSON description of the instance.

Await Instance

GET /instance/await?model_id=...&timeout_seconds=0

Waits until API state contains an instance for the requested model. The response
is an SSE stream so clients receive keep-alive comments while waiting.

Query parameters:

  • model_id: string, required
  • timeout_seconds: float, optional, default 0. 0 waits indefinitely;
    positive values time out after that many seconds. Maximum positive value:
    300.

Stream messages:

text
data: {"type": "ready", "instance": {...}}

data: {"type": "timeout", "message": "No instance found for model ..."}

The HTTP status is 200 for both messages because the stream starts before the
final result is known. The type field disambiguates the terminal message.

Preview Placements

GET /instance/previews?model_id=...

Returns possible placement previews for a given model.

Query parameters:

  • model_id: string, required

Response:
Array of placement preview objects.

Compute Placement

GET /instance/placement

Computes a placement for a potential instance without creating it.

Query parameters (typical):

  • model_id: string
  • sharding: string or config
  • instance_meta: JSON-encoded metadata
  • min_nodes: integer

Response:
JSON object describing the proposed placement / instance configuration.

Place Instance

POST /place_instance

Places an instance for a model using the server's placement logic.

Request body:
JSON describing the instance to be placed.

Response:
Command acknowledgement. The instance may not be ready immediately; wait for it
to appear through /instance/await before sending inference requests.

3. Models

List Models

GET /models
GET /v1/models (alias)

Returns the list of available models and their metadata.

Query parameters:

  • status: string (optional) - Filter by downloaded to show only downloaded models

Response:
Array of model descriptors including is_custom field for custom HuggingFace models.

Add Custom Model

POST /models/add

Add a custom model from HuggingFace hub.

Request body (example):

json
{
  "model_id": "mlx-community/my-custom-model"
}

Response:
Model descriptor for the added model.

Security note:
Models with trust_remote_code enabled in their configuration require explicit opt-in (default is false) for security.

Delete Custom Model

DELETE /models/custom/{model_id}

Delete a user-added custom model card.

Path parameters:

  • model_id: string, ID of the custom model to delete

Response:
Confirmation JSON with deleted model ID.

Search Models

GET /models/search

Search HuggingFace Hub for mlx-community models.

Query parameters:

  • query: string (optional) - Search query
  • limit: integer (default: 20) - Maximum number of results

Response:
Array of HuggingFace model search results.

4. Inference / Chat Completions

OpenAI-Compatible Chat Completions

POST /v1/chat/completions

Executes a chat completion request using an OpenAI-compatible schema. Supports streaming and non-streaming modes.

Request body (example):

json
{
  "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello" }
  ],
  "stream": false
}

Request parameters:

  • model: string, required - Model ID to use
  • messages: array, required - Conversation messages
  • stream: boolean (default: false) - Enable streaming responses
  • max_tokens: integer (optional) - Maximum tokens to generate
  • temperature: float (optional) - Sampling temperature
  • top_p: float (optional) - Nucleus sampling parameter
  • top_k: integer (optional) - Top-k sampling parameter
  • stop: string or array (optional) - Stop sequences
  • seed: integer (optional) - Random seed for reproducibility
  • enable_thinking: boolean (optional) - Enable thinking mode for capable models (DeepSeek V3.1, Qwen3, GLM-4.7)
  • tools: array (optional) - Tool definitions for function calling
  • logprobs: boolean (optional) - Return log probabilities
  • top_logprobs: integer (optional) - Number of top log probabilities to return

Response:
OpenAI-compatible chat completion response.

Streaming response format:
When stream=true, returns Server-Sent Events (SSE) with format:

text
data: {"id":"...","object":"chat.completion","created":...,"model":"...","choices":[...]}

data: [DONE]

Non-streaming response includes usage statistics:

json
{
  "id": "...",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello! How can I help you?"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 8,
    "total_tokens": 23
  }
}

Cancellation:
You can cancel an active generation by closing the HTTP connection. The server detects the disconnection and stops processing.

Claude Messages API

POST /v1/messages

Executes a chat completion request using the Claude Messages API format. Supports streaming and non-streaming modes.

Request body (example):

json
{
  "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
  "messages": [
    { "role": "user", "content": "Hello" }
  ],
  "max_tokens": 1024,
  "stream": false
}

Streaming response format:
When stream=true, returns Server-Sent Events with Claude-specific event types:

  • message_start - Message generation started
  • content_block_start - Content block started
  • content_block_delta - Incremental content chunk
  • content_block_stop - Content block completed
  • message_delta - Message metadata updates
  • message_stop - Message generation completed

Response:
Claude-compatible messages response.

OpenAI Responses API

POST /v1/responses

Executes a chat completion request using the OpenAI Responses API format. Supports streaming and non-streaming modes.

Request body (example):

json
{
  "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
  "input": [
    { "role": "user", "content": "Hello" }
  ],
  "stream": false
}

Request parameters:

  • model: string, required - Model ID to use
  • input: string or array, required - The prompt, or a list of input items such as messages
  • instructions: string (optional) - System instructions
  • max_output_tokens: integer (optional) - Maximum tokens to generate

Streaming response format:
When stream=true, returns Server-Sent Events with response-specific event types:

  • response.created - Response generation started
  • response.in_progress - Response is being generated
  • response.output_item.added - New output item added
  • response.output_text.delta - Incremental text chunk
  • response.output_item.done - Output item completed
  • response.completed - Response generation completed

Response:
OpenAI Responses API-compatible response.

Benchmarked Chat Completions

POST /bench/chat/completions

Same as /v1/chat/completions, but also returns performance and generation statistics.

Request body:
Same schema as /v1/chat/completions.

Response:
Chat completion plus benchmarking metrics including:

  • prompt_tps - Tokens per second during prompt processing
  • generation_tps - Tokens per second during generation
  • prompt_tokens - Number of prompt tokens
  • generation_tokens - Number of generated tokens
  • peak_memory_usage - Peak memory used during generation

Cancel Command

POST /v1/cancel/{command_id}

Cancels an active generation command (text or image). Notifies workers and closes the stream.

Path parameters:

  • command_id: string, ID of the command to cancel

Response (example):

json
{
  "message": "Command cancelled.",
  "command_id": "cmd-abc-123"
}

Returns 404 if the command is not found or already completed.

5. Ollama API Compatibility

EXO provides Ollama API compatibility for tools like OpenWebUI.

Ollama Chat

POST /ollama/api/chat
POST /ollama/api/api/chat (alias)
POST /ollama/api/v1/chat (alias)

Execute a chat request using Ollama API format.

Request body (example):

json
{
  "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
  "messages": [
    { "role": "user", "content": "Hello" }
  ],
  "stream": false
}

Response:
Ollama-compatible chat response.

Ollama Generate

POST /ollama/api/generate

Execute a text generation request using Ollama API format.

Request body (example):

json
{
  "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
  "prompt": "Hello",
  "stream": false
}

Response:
Ollama-compatible generation response.

Ollama Tags

GET /ollama/api/tags
GET /ollama/api/api/tags (alias)
GET /ollama/api/v1/tags (alias)

Returns list of downloaded models in Ollama tags format.

Response:
Array of model tags with metadata.

Ollama Show

POST /ollama/api/show

Returns model information in Ollama show format.

Request body:

json
{
  "name": "mlx-community/Llama-3.2-1B-Instruct-4bit"
}

Response:
Model details including modelfile and family.

Ollama PS

GET /ollama/api/ps

Returns list of running models (active instances).

Response:
Array of active model instances.

Ollama Version

GET /ollama/api/version
HEAD /ollama/ (alias)
HEAD /ollama/api/version (alias)

Returns version information for Ollama API compatibility.

Response:

json
{
  "version": "exo v1.0"
}

6. Image Generation & Editing

Image Generation

POST /v1/images/generations

Executes an image generation request using an OpenAI-compatible schema with additional advanced_params. Supports both streaming and non-streaming modes.

Request body (example):

json
{
  "prompt": "a robot playing chess",
  "model": "exolabs/FLUX.1-dev",
  "n": 1,
  "size": "1024x1024",
  "stream": false,
  "response_format": "b64_json"
}

Request parameters:

  • prompt: string, required - Text description of the image
  • model: string, required - Image model ID
  • n: integer (default: 1) - Number of images to generate
  • size: string (default: "auto") - Image dimensions. Supported sizes:
    • 512x512
    • 768x768
    • 1024x768
    • 768x1024
    • 1024x1024
    • 1024x1536
    • 1536x1024
    • 1024x1365
    • 1365x1024
  • stream: boolean (default: false) - Enable streaming for partial images
  • partial_images: integer (default: 0) - Number of partial images to stream during generation
  • response_format: string (default: "b64_json") - Either url or b64_json
  • quality: string (default: "medium") - Either high, medium, or low
  • output_format: string (default: "png") - Either png, jpeg, or webp
  • advanced_params: object (optional) - Advanced generation parameters

Advanced Parameters (advanced_params):

Parameter Type Constraints Description
seed int >= 0 Random seed for reproducible generation
num_inference_steps int 1-100 Number of denoising steps
guidance float 1.0-20.0 Classifier-free guidance scale
negative_prompt string - Text describing what to avoid in the image

Non-streaming response:

json
{
  "created": 1234567890,
  "data": [
    {
      "b64_json": "iVBORw0KGgoAAAANSUhEUgAA...",
      "url": null
    }
  ]
}

Streaming response format:
When stream=true and partial_images > 0, returns Server-Sent Events:

text
data: {"type":"partial","image_index":0,"partial_index":1,"total_partials":5,"format":"png","data":{"b64_json":"..."}}

data: {"type":"final","image_index":0,"format":"png","data":{"b64_json":"..."}}

data: [DONE]

Image Editing

POST /v1/images/edits

Executes an image editing request (img2img) using FLUX.1-Kontext-dev or similar models.

Request (multipart/form-data):

  • image: file, required - Input image to edit
  • prompt: string, required - Text description of desired changes
  • model: string, required - Image editing model ID (e.g., exolabs/FLUX.1-Kontext-dev)
  • n: integer (default: 1) - Number of edited images to generate
  • size: string (optional) - Output image dimensions
  • response_format: string (default: "b64_json") - Either url or b64_json
  • input_fidelity: string (default: "low") - Either low or high - Controls how closely the output follows the input image
  • stream: string (default: "false") - Enable streaming
  • partial_images: string (default: "0") - Number of partial images to stream
  • quality: string (default: "medium") - Either high, medium, or low
  • output_format: string (default: "png") - Either png, jpeg, or webp
  • advanced_params: string (optional) - JSON-encoded advanced parameters

Response:
Same format as /v1/images/generations.

Benchmarked Image Generation

POST /bench/images/generations

Same as /v1/images/generations, but also returns generation statistics.

Request body:
Same schema as /v1/images/generations.

Response:
Image generation plus benchmarking metrics including:

  • seconds_per_step - Average time per denoising step
  • total_generation_time - Total generation time
  • num_inference_steps - Number of inference steps used
  • num_images - Number of images generated
  • image_width - Output image width
  • image_height - Output image height
  • peak_memory_usage - Peak memory used during generation

Benchmarked Image Editing

POST /bench/images/edits

Same as /v1/images/edits, but also returns generation statistics.

Request:
Same schema as /v1/images/edits.

Response:
Same format as /bench/images/generations, including generation_stats.

List Images

GET /images

List all stored images.

Response:
Array of image metadata including URLs and expiration times.

Get Image

GET /images/{image_id}

Retrieve a stored image by ID.

Path parameters:

  • image_id: string, ID of the image

Response:
Image file with appropriate content type.

7. Complete Endpoint Summary

text
# General
GET     /node_id
GET     /state
GET     /events

# Instance Management
POST    /instance
GET     /instance/await
GET     /instance/previews
GET     /instance/placement
GET     /instance/{instance_id}
DELETE  /instance/{instance_id}
POST    /place_instance

# Models
GET     /models
GET     /v1/models
POST    /models/add
DELETE  /models/custom/{model_id}
GET     /models/search

# Text Generation (OpenAI Chat Completions)
POST    /v1/chat/completions
POST    /bench/chat/completions

# Text Generation (Claude Messages API)
POST    /v1/messages

# Text Generation (OpenAI Responses API)
POST    /v1/responses

# Text Generation (Ollama API)
POST    /ollama/api/chat
POST    /ollama/api/api/chat
POST    /ollama/api/v1/chat
POST    /ollama/api/generate
GET     /ollama/api/tags
GET     /ollama/api/api/tags
GET     /ollama/api/v1/tags
POST    /ollama/api/show
GET     /ollama/api/ps
GET     /ollama/api/version
HEAD    /ollama/
HEAD    /ollama/api/version

# Command Control
POST    /v1/cancel/{command_id}

# Image Generation
POST    /v1/images/generations
POST    /bench/images/generations
POST    /v1/images/edits
POST    /bench/images/edits
GET     /images
GET     /images/{image_id}

8. Notes

API Compatibility

EXO provides multiple API-compatible interfaces:

  • OpenAI Chat Completions API - Compatible with OpenAI clients and tools
  • Claude Messages API - Compatible with Anthropic's Claude API format
  • OpenAI Responses API - Compatible with OpenAI's Responses API format
  • Ollama API - Compatible with Ollama and tools like OpenWebUI

Existing OpenAI, Claude, or Ollama clients can be pointed to EXO by changing the base URL.

Custom Models

You can add custom models from HuggingFace using the /models/add endpoint. Custom models are identified by the is_custom field in model list responses.

Security: Models requiring trust_remote_code must be explicitly enabled (default is false) for security. Only enable this if you trust the model's remote code.

Usage Statistics

Chat completion responses include usage statistics with:

  • prompt_tokens - Number of tokens in the prompt
  • completion_tokens - Number of tokens generated
  • total_tokens - Sum of prompt and completion tokens

Request Cancellation

You can cancel active requests by:

  1. Closing the HTTP connection (for streaming requests)
  2. Calling /v1/cancel/{command_id} (for any request)

The server detects cancellation and stops processing immediately.

Instance Placement

The instance placement endpoints allow you to plan and preview cluster allocations before creating instances. This helps optimize resource usage across nodes.

Observability

The /events and /state endpoints are primarily intended for operational visibility and debugging.

File: nix/apple-sdk/metadata/versions.json

{
"14": {
"urls": [
"https://swcdn.apple.com/content/downloads/14/48/052-59890-A_I0F5YGAY0Y/p9n40hio7892gou31o1v031ng6fnm9sb3c/CLTools_macOSNMOS_SDK.pkg",
"https://web.archive.org/web/20250211001355/https://swcdn.apple.com/content/downloads/14/48/052-59890-A_I0F5YGAY0Y/p9n40hio7892gou31o1v031ng6fnm9sb3c/CLTools_macOSNMOS_SDK.pkg"
],
"version": "14.4",
"hash": "sha256-QozDiwY0Czc0g45vPD7G4v4Ra+3DujCJbSads3fJjjM="
},
"15": {
"urls": [
"https://swcdn.apple.com/content/downloads/52/01/082-41241-A_0747ZN8FHV/dectd075r63pppkkzsb75qk61s0lfee22j/CLTools_macOSNMOS_SDK.pkg",
"https://web.archive.org/web/20250530132510/https://swcdn.apple.com/content/downloads/52/01/082-41241-A_0747ZN8FHV/dectd075r63pppkkzsb75qk61s0lfee22j/CLTools_macOSNMOS_SDK.pkg"
],
"version": "15.5",
"hash": "sha256-HBiSJuw1XBUK5R/8Sj65c3rftSEvQl/O9ZZVp/g1Amo="
},
"26": {
"urls": [
"https://swcdn.apple.com/content/downloads/60/22/089-71960-A_W8BL1RUJJ6/5zkyplomhk1cm7z6xja2ktgapnhhti6wwd/CLTools_macOSNMOS_SDK.pkg",
"https://web.archive.org/web/20250915230423/https://swcdn.apple.com/content/downloads/60/22/089-71960-A_W8BL1RUJJ6/5zkyplomhk1cm7z6xja2ktgapnhhti6wwd/CLTools_macOSNMOS_SDK.pkg"
],
"version": "26.2",
"hash": "sha256-hXRlMieVv0smna5uiWRwq87IWOaPWtAjAldbi+wQXcw="
}
}

File: rust/exo_rs/README.md

TODO: do something here....

File: tmp/prompt.txt

Summarise this Wikipedia article for me:

Transition from Republic to Empire

Augustus of Prima Porta
Rome had begun expanding shortly after the founding of the Roman Republic in the 6th century BC, though not outside the Italian Peninsula until the 3rd century BC. The Republic was not a nation-state in the modern sense, but a network of self-ruled towns (with varying degrees of independence from the Senate) and provinces administered by military commanders. It was governed by annually elected magistrates (Roman consuls above all) in conjunction with the Senate.[22] The 1st century BC was a time of political and military upheaval, which ultimately led to rule by emperors.[23][24][25] The consuls' military power rested in the Roman legal concept of imperium, meaning "command" (typically in a military sense).[26] Occasionally, successful consuls or generals were given the honorary title imperator (commander); this is the origin of the word emperor, since this title was always bestowed to the early emperors.[27][g]

Rome suffered a long series of internal conflicts, conspiracies, and civil wars from the late second century BC (see Crisis of the Roman Republic) while greatly extending its power beyond Italy. In 44 BC Julius Caesar was briefly perpetual dictator before being assassinated by a faction that opposed his concentration of power. This faction was driven from Rome and defeated at the Battle of Philippi in 42 BC by Mark Antony and Caesar's adopted son Octavian. Antony and Octavian divided the Roman world between them, but this did not last long. Octavian's forces defeated those of Mark Antony and Cleopatra at the Battle of Actium in 31 BC. In 27 BC the Senate gave him the title Augustus ("venerated") and made him princeps ("foremost") with proconsular imperium, thus beginning the Principate, the first epoch of Roman imperial history. Although the republic stood in name, Augustus had all meaningful authority.[29] During his 40-year rule, a new constitutional order emerged so that, upon his death, Tiberius would succeed him as the new de facto monarch.[30]

Pax Romana
Main article: Pax Romana
The so-called "Five Good Emperors" of 96–180 AD

Nerva (r. 96–98)

Trajan (r. 98–117)

Hadrian (r. 117–138)

Antoninus Pius (r. 138–161)

Marcus Aurelius (r. 161–180)
The 200 years that began with Augustus's rule are traditionally regarded as the Pax Romana ("Roman Peace"). The cohesion of the empire was furthered by a degree of social stability and economic prosperity that Rome had never before experienced. Uprisings in the provinces were infrequent and put down "mercilessly and swiftly".[31] The success of Augustus in establishing principles of dynastic succession was limited by his outliving a number of talented potential heirs. The Julio-Claudian dynasty lasted for four more emperors—Tiberius, Caligula, Claudius, and Nero—before it yielded in 69 AD to the strife-torn Year of the Four Emperors, from which Vespasian emerged as the victor. Vespasian became the founder of the brief Flavian dynasty, followed by the Nerva–Antonine dynasty which produced the "Five Good Emperors": Nerva, Trajan, Hadrian, Antoninus Pius, and Marcus Aurelius.[32]

Among the so-called “Five Good Emperors,” Hadrian (r. 117–138) is particularly noted for consolidating the empire’s frontiers and embarking on ambitious building projects throughout the provinces.[33] In Judaea, which had long been the center of Jewish national and religious life, his reign marked a decisive turning point. After earlier Jewish resistance to Roman rule, Hadrian visited the region in 129/130 CE and refounded Jerusalem as the Roman colony Aelia Capitolina, naming it after his family (Aelius) and the Capitoline Triad.[34] The refoundation overlaid the destroyed Jewish city with a new Roman urban plan, and included the construction of a Temple to Jupiter on the site of the former Jewish Temple.[35] Later tradition and archaeological evidence also indicate a Temple of Venus near the site of the Holy Sepulchre.[36]

Hadrian’s measures, combined with restrictions on Jewish practices, helped spark the Bar Kokhba Revolt (132–135 CE). After crushing the uprising, Roman forces expelled most Jews from Jerusalem, barring their entry except on certain days, and rebuilt the city as a statement of imperial power and domination.[33] Most scholars consider Hadrianic Aelia to have been unwalled, with free-standing gate complexes (such as the northern gate beneath today’s Damascus Gate) rather than a continuous defensive circuit.[37]

Transition from classical to late antiquity
Main articles: Later Roman Empire and Fall of the Western Roman Empire
See also: Barbarian kingdoms and Byzantine Empire

The Barbarian invasions consisted of the movement of (mainly) ancient Germanic peoples into Roman territory. Historically, this event marked the transition between classical antiquity and the Middle Ages.
In the view of contemporary Greek historian Cassius Dio, the accession of Commodus in 180 marked the descent "from a kingdom of gold to one of rust and iron",[38] a comment which has led some historians, notably Edward Gibbon, to take Commodus' reign as the beginning of the Empire's decline.[39][40]

In 212, during the reign of Caracalla, Roman citizenship was granted to all freeborn inhabitants of the empire. The Severan dynasty was tumultuous; an emperor's reign was ended routinely by his murder or execution and, following its collapse, the Empire was engulfed by the Crisis of the Third Century, a period of invasions, civil strife, economic disorder, and plague.[41] In defining historical epochs, this crisis sometimes marks the transition from Classical to Late Antiquity. Aurelian (r. 270–275) stabilised the empire militarily and Diocletian reorganised and restored much of it in 285.[42] Diocletian's reign brought the empire's most concerted effort against the perceived threat of Christianity, the "Great Persecution".[43]

Diocletian divided the empire into four regions, each ruled by a separate tetrarch.[44] Confident that he fixed the disorder plaguing Rome, he abdicated along with his co-emperor, but the Tetrarchy collapsed shortly after. Order was eventually restored by Constantine the Great, who became the first emperor to convert to Christianity, and who established Constantinople as the new capital of the Eastern Empire. During the decades of the Constantinian and Valentinian dynasties, the empire was divided along an east–west axis, with dual power centres in Constantinople and Rome. Julian, who under the influence of his adviser Mardonius attempted to restore Classical Roman and Hellenistic religion, only briefly interrupted the succession of Christian emperors. Theodosius I, the last emperor to rule over both East and West, died in 395 after making Christianity the state religion.[45]

The Roman Empire by 476, noting western and eastern divisions

The administrative divisions of the Roman Empire in 395 AD
Fall in the West and survival in the East
The Western Roman Empire began to disintegrate in the early 5th century. The Romans fought off all invaders, most famously Attila,[46] but the empire had assimilated so many Germanic peoples of dubious loyalty to Rome that the empire started to dismember itself.[47] Most chronologies place the end of the Western Roman Empire in 476, when Romulus Augustulus was forced to abdicate to the Germanic warlord Odoacer.[48][49][50]

Odoacer ended the Western Empire by declaring Zeno sole emperor and placing himself as Zeno's nominal subordinate. In reality, Italy was ruled by Odoacer alone.[48][49][51] The Eastern Roman Empire, called the Byzantine Empire by later historians, continued until the reign of Constantine XI Palaiologos, the last Roman emperor. He died in battle in 1453 against Mehmed II and his Ottoman forces during the siege of Constantinople. Mehmed II adopted the title of caesar in an attempt to claim a connection to the former Empire.[52][53] His claim was soon recognized by the Patriarchate of Constantinople, but not by European monarchs.

File: tmp/config_examples/opencode.json

{
"$schema": "https://opencode.ai/config.json",
"model": "exo/mlx-community/gpt-oss-120b-MXFP4-Q8",
"provider": {
"exo": {
"api": "http://localhost:52415/v1",
"models": {
"mlx-community/gpt-oss-120b-MXFP4-Q8": {
"name": "GPT OSS 120B",
"limit": {
"context": 32768,
"output": 8192
}
}
}
}
}
}

File: .github/pull_request_template.md

Motivation

Changes

Why It Works

Test Plan

Manual Testing

Automated Testing

File: .github/actions/conditional-commit/action.yml

name: Commit if changed
inputs:
message:
required: true

runs:
using: composite
steps:
- name: Commit changed files
shell: bash
run: |
git diff --quiet && exit 0
git commit -am "${{ inputs.message }}"

File: .github/actions/format/action.yml

name: Format Code
runs:
using: "composite"
steps:
- name: Format code
run: nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just fmt
shell: bash

File: .github/actions/lint/action.yml

name: Lint Code
runs:
using: "composite"
steps:
- name: Lint code
run: nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just lint
shell: bash

File: .github/actions/lint-check/action.yml

name: Lint Check
runs:
using: "composite"
steps:
- name: Lint check
run: nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just lint-check
shell: bash

File: .github/actions/regenerate-protobufs/action.yml

name: Regenerate Protobufs
runs:
using: "composite"
steps:
- name: Regenerate protobufs
run: nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just regenerate-protobufs
shell: bash

File: .github/actions/setup-python-uv/action.yml

name: Setup Python & uv
runs:
using: "composite"
steps:
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-dependency-glob: uv.lock

text
- name: Install Python
  run: uv python install
  shell: bash

- name: Sync
  run: uv sync --locked --all-extras --dev
  shell: bash

File: .github/actions/unit-test/action.yml

name: Unit Test
runs:
using: "composite"
steps:
- name: Run unit tests
run: |
nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just sync-clean
nix --extra-experimental-features nix-command --extra-experimental-features flakes develop -c just test-fast
shell: bash

File: .github/actions/verify-clean/action.yml

name: Verify Clean Working Tree
inputs:
step:
required: true

runs:
using: composite
steps:
- name: Check git diff
shell: bash
run: |
if ! git diff --quiet; then
echo "Error: ${{ inputs.step }} left working tree dirty." >&2
git --no-pager diff >&2
exit 1
fi

File: .github/ISSUE_TEMPLATE/bug_report.md

name: Bug Report about: Create a report to help us improve title: '[BUG] ' labels: bug assignees: ''

Describe the bug

A clear and concise description of what the bug is.

To Reproduce

Steps to reproduce the behavior:
1.
2.
3.

Expected behavior

A clear and concise description of what you expected to happen.

Actual behavior

A clear and concise description of what actually happened.

Environment

  • macOS Version:
  • EXO Version:
  • Hardware:
    • Device 1: (e.g., MacBook Pro M1 Max, 32GB RAM)
    • Device 2: (e.g., Mac Mini M2, 16GB RAM)
    • Additional devices:
  • Interconnection:
    • (e.g., Thunderbolt 4 cable between Device 1 and 2)
    • (e.g., WiFi 6 for Device 3)
    • (e.g., 10GbE Ethernet between all devices)

Additional context

Add any other context about the problem here.

File: .github/ISSUE_TEMPLATE/feature_request.md

name: Feature Request about: Suggest an idea for this project title: '' labels: enhancement assignees: ''

Describe what you'd like to see added to EXO.

File: .github/workflows/build-app.yml

name: Build EXO macOS DMG

Release workflow:

1. Create a draft GitHub Release with the tag name (e.g. v1.0.0) and write release notes in markdown

2. Push the tag: git tag v1.0.0 && git push origin v1.0.0

3. This workflow builds, signs, and notarizes the DMG

4. Release notes are embedded in appcast.xml for Sparkle (rendered as markdown)

5. DMG and appcast.xml are uploaded to S3

6. The draft GitHub Release is published with the DMG attached

For alpha releases (e.g. v1.0.0-alpha.1): draft release and notes are optional.

If no draft exists, a release is auto-created with generated notes.

on:
workflow_dispatch:
push:
tags:
- "v*"
branches:
- "test-app"

jobs:
build-macos-app:
runs-on: "macos-26"
permissions:
contents: write
env:
SPARKLE_VERSION: 2.9.0-beta.1
SPARKLE_DOWNLOAD_PREFIX: ${{ secrets.SPARKLE_DOWNLOAD_PREFIX }}
SPARKLE_FEED_URL: ${{ secrets.SPARKLE_FEED_URL }}
SPARKLE_ED25519_PUBLIC: ${{ secrets.SPARKLE_ED25519_PUBLIC }}
SPARKLE_ED25519_PRIVATE: ${{ secrets.SPARKLE_ED25519_PRIVATE }}
SPARKLE_S3_BUCKET: ${{ secrets.SPARKLE_S3_BUCKET }}
SPARKLE_S3_PREFIX: ${{ secrets.SPARKLE_S3_PREFIX }}
AWS_REGION: ${{ secrets.AWS_REGION }}
EXO_BUILD_NUMBER: ${{ github.run_number }}
EXO_LIBP2P_NAMESPACE: ${{ github.ref_name }}

text
steps:
  # ============================================================
  # Checkout and tag validation
  # ============================================================

  - name: Checkout
    uses: actions/checkout@v4
    with:
      fetch-depth: 0

  - name: Derive release version from tag
    run: |
      if [[ "$GITHUB_REF_NAME" == "test-app" || "${{ github.event_name }}" == "workflow_dispatch" ]]; then
        VERSION="0.0.0-alpha.0"
        echo "IS_ALPHA=true" >> $GITHUB_ENV
      else
        VERSION="${GITHUB_REF_NAME#v}"
        if [[ "$VERSION" == *-alpha* ]]; then
          echo "IS_ALPHA=true" >> $GITHUB_ENV
        else
          echo "IS_ALPHA=false" >> $GITHUB_ENV
        fi
      fi
      echo "RELEASE_VERSION=$VERSION" >> $GITHUB_ENV

  - name: Compute build version from semver
    run: |
      VERSION="$RELEASE_VERSION"
      # Extract major.minor.patch (strip prerelease suffix)
      BASE_VERSION="${VERSION%%-*}"
      MAJOR=$(echo "$BASE_VERSION" | cut -d. -f1)
      MINOR=$(echo "$BASE_VERSION" | cut -d. -f2)
      PATCH=$(echo "$BASE_VERSION" | cut -d. -f3)

      # Extract prerelease number (e.g., "alpha.2" -> 2, or 999 for releases)
      if [[ "$VERSION" == *-* ]]; then
        PRERELEASE_PART="${VERSION#*-}"
        PRERELEASE_NUM="${PRERELEASE_PART##*.}"
        # Default to 0 if not a number
        if ! [[ "$PRERELEASE_NUM" =~ ^[0-9]+$ ]]; then
          PRERELEASE_NUM=0
        fi
      else
        PRERELEASE_NUM=999
      fi

      # Compute: PRERELEASE + (1000 * PATCH) + (1_000_000 * MINOR) + (1_000_000_000 * MAJOR)
      BUILD_VERSION=$((PRERELEASE_NUM + 1000 * PATCH + 1000000 * MINOR + 1000000000 * MAJOR))
      echo "EXO_BUILD_VERSION=$BUILD_VERSION" >> $GITHUB_ENV
      echo "Computed build version: $BUILD_VERSION from $VERSION"

  - name: Ensure tag commit is on main
    if: github.ref_type == 'tag'
    run: |
      git fetch origin main
      # Alpha tags can be on any branch, production tags must be on main
      if [[ "$IS_ALPHA" == "true" ]]; then
        echo "Alpha tag detected, skipping main branch check"
      elif ! git merge-base --is-ancestor origin/main HEAD; then
        echo "Production tag must point to a commit on main"
        exit 1
      fi

  - name: Fetch and validate release notes
    if: github.ref_type == 'tag'
    env:
      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    run: |
      # Find draft release by name using gh release list (more reliable with default token)
      echo "Looking for draft release named '$GITHUB_REF_NAME'..."
      DRAFT_EXISTS=$(gh release list --json name,isDraft --jq ".[] | select(.isDraft == true) | select(.name == \"$GITHUB_REF_NAME\") | .name" 2>/dev/null || echo "")

      if [[ -z "$DRAFT_EXISTS" ]]; then
        if [[ "$IS_ALPHA" == "true" ]]; then
          echo "No draft release found for alpha tag $GITHUB_REF_NAME (optional for alphas)"
          echo "HAS_RELEASE_NOTES=false" >> $GITHUB_ENV
          exit 0
        fi
        echo "ERROR: No draft release found for tag $GITHUB_REF_NAME"
        echo "Please create a draft release with release notes before pushing the tag."
        exit 1
      fi

      # Fetch full release details via API to get body and ID
      echo "Found draft release, fetching details..."
      RELEASE_JSON=$(gh api repos/${{ github.repository }}/releases --jq ".[] | select(.draft == true) | select(.name == \"$GITHUB_REF_NAME\")" 2>/dev/null || echo "")

      # Extract release notes
      NOTES=$(echo "$RELEASE_JSON" | jq -r '.body // ""')
      if [[ -z "$NOTES" || "$NOTES" == "null" ]]; then
        if [[ "$IS_ALPHA" == "true" ]]; then
          echo "Draft release has no notes (optional for alphas)"
          echo "HAS_RELEASE_NOTES=false" >> $GITHUB_ENV
          exit 0
        fi
        echo "ERROR: Draft release exists but has no release notes"
        echo "Please add release notes to the draft release before pushing the tag."
        exit 1
      fi

      # Save release ID for later publishing
      RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
      echo "DRAFT_RELEASE_ID=$RELEASE_ID" >> $GITHUB_ENV
      echo "HAS_RELEASE_NOTES=true" >> $GITHUB_ENV

      echo "Found draft release (ID: $RELEASE_ID), saving release notes..."
      echo "$NOTES" > /tmp/release_notes.md
      echo "RELEASE_NOTES_FILE=/tmp/release_notes.md" >> $GITHUB_ENV

  # ============================================================
  # Install dependencies
  # ============================================================

  - name: Select Xcode 26.2
    run: |
      sudo xcode-select -s /Applications/Xcode_26.2.app
      if ! xcrun -f metal >/dev/null 2>&1; then
        echo "Metal toolchain is not installed."
        exit 1
      fi

  - name: Install Homebrew packages
    run: brew install just awscli

  - name: Install UV
    uses: astral-sh/setup-uv@v6
    with:
      enable-cache: true
      cache-dependency-glob: uv.lock

  - name: Setup Python
    run: |
      uv python install
      uv sync --locked --extra mlx

  - name: Install Nix
    uses: cachix/install-nix-action@v31
    with:
      nix_path: nixpkgs=channel:nixos-unstable

  - name: Configure Cachix
    uses: cachix/cachix-action@v14
    with:
      name: exo
      authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"

  - name: Build dashboard
    run: |
      DASHBOARD_OUT=$(nix build .#dashboard --print-build-logs --no-link --print-out-paths)
      mkdir -p dashboard/build
      cp -r "$DASHBOARD_OUT"/* dashboard/build/

  - name: Install Sparkle CLI
    run: |
      CLI_URL="${SPARKLE_CLI_URL:-https://github.com/sparkle-project/Sparkle/releases/download/${SPARKLE_VERSION}/Sparkle-${SPARKLE_VERSION}.tar.xz}"
      echo "Downloading Sparkle CLI from: $CLI_URL"
      mkdir -p /tmp/sparkle
      curl --fail --location --output /tmp/sparkle.tar.xz "$CLI_URL"
      tar -xJf /tmp/sparkle.tar.xz -C /tmp/sparkle --strip-components=1
      echo "SPARKLE_BIN=/tmp/sparkle/bin" >> $GITHUB_ENV

  - name: Prepare code-signing keychain
    env:
      MACOS_CERTIFICATE: ${{ secrets.MACOS_CERTIFICATE }}
      MACOS_CERTIFICATE_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PASSWORD }}
      PROVISIONING_PROFILE: ${{ secrets.PROVISIONING_PROFILE }}
    run: |
      KEYCHAIN_PATH="$HOME/Library/Keychains/build.keychain-db"

      # Create fresh keychain
      security create-keychain -p "$MACOS_CERTIFICATE_PASSWORD" "$KEYCHAIN_PATH"

      # Disable auto-lock (no timeout, no lock-on-sleep)
      security set-keychain-settings "$KEYCHAIN_PATH"

      # Add to search list while preserving existing keychains
      security list-keychains -d user -s "$KEYCHAIN_PATH" $(security list-keychains -d user | tr -d '"')

      # Set as default and unlock
      security default-keychain -s "$KEYCHAIN_PATH"
      security unlock-keychain -p "$MACOS_CERTIFICATE_PASSWORD" "$KEYCHAIN_PATH"

      # Import certificate with full access for codesign
      echo "$MACOS_CERTIFICATE" | base64 --decode > /tmp/cert.p12
      security import /tmp/cert.p12 -k "$KEYCHAIN_PATH" -P "$MACOS_CERTIFICATE_PASSWORD" \
        -T /usr/bin/codesign -T /usr/bin/security -T /usr/bin/productbuild
      rm /tmp/cert.p12

      # Allow codesign to access the key without prompting
      security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$MACOS_CERTIFICATE_PASSWORD" "$KEYCHAIN_PATH"

      # Verify keychain is unlocked and identity is available
      echo "Verifying signing identity..."
      security find-identity -v -p codesigning "$KEYCHAIN_PATH"

      # Setup provisioning profile
      mkdir -p "$HOME/Library/Developer/Xcode/UserData/Provisioning Profiles"
      echo "$PROVISIONING_PROFILE" | base64 --decode > "$HOME/Library/Developer/Xcode/UserData/Provisioning Profiles/EXO.provisionprofile"

      # Export keychain path for other steps
      echo "BUILD_KEYCHAIN_PATH=$KEYCHAIN_PATH" >> $GITHUB_ENV

  # ============================================================
  # Pre-flight credential / profile validation
  # Runs BEFORE the ~16 min build so auth/expiry failures surface in <1 min.
  # ============================================================

  - name: Validate Apple notarization credentials
    env:
      APPLE_NOTARIZATION_USERNAME: ${{ secrets.APPLE_NOTARIZATION_USERNAME }}
      APPLE_NOTARIZATION_PASSWORD: ${{ secrets.APPLE_NOTARIZATION_PASSWORD }}
      APPLE_NOTARIZATION_TEAM: ${{ secrets.APPLE_NOTARIZATION_TEAM }}
    run: |
      # All-or-nothing: either all three creds are set, or none are.
      CRED_COUNT=0
      for v in "$APPLE_NOTARIZATION_USERNAME" "$APPLE_NOTARIZATION_PASSWORD" "$APPLE_NOTARIZATION_TEAM"; do
        [[ -n "$v" ]] && CRED_COUNT=$((CRED_COUNT + 1))
      done
      if [[ "$CRED_COUNT" -eq 0 ]]; then
        echo "No notarization credentials configured — skipping notarization for this build."
        exit 0
      fi
      if [[ "$CRED_COUNT" -ne 3 ]]; then
        echo "ERROR: partial notarization credentials set ($CRED_COUNT/3). Aborting before build."
        exit 1
      fi
      # Cheap, ~5s, auth-only call. Fails instantly with a clear message if
      # the app-specific password is stale, wrong team-id, etc.
      echo "Verifying Apple notarization credentials via notarytool history..."
      if ! xcrun notarytool history \
        --apple-id "$APPLE_NOTARIZATION_USERNAME" \
        --password "$APPLE_NOTARIZATION_PASSWORD" \
        --team-id "$APPLE_NOTARIZATION_TEAM" >/dev/null; then
        echo "ERROR: notarytool rejected the provided credentials. Fix before rerunning."
        echo "Common causes: app-specific password expired/revoked, wrong team-id,"
        echo "Apple ID not on the team, or 2FA not configured for this Apple ID."
        exit 1
      fi
      echo "Apple notarization credentials OK."

  - name: Validate provisioning profile expiry
    run: |
      PROFILE="$HOME/Library/Developer/Xcode/UserData/Provisioning Profiles/EXO.provisionprofile"
      if [[ ! -f "$PROFILE" ]]; then
        echo "ERROR: provisioning profile not found at $PROFILE"
        exit 1
      fi
      EXPIRY=$(security cms -D -i "$PROFILE" | plutil -extract ExpirationDate raw -o - - 2>/dev/null || true)
      if [[ -z "$EXPIRY" ]]; then
        echo "WARNING: could not read ExpirationDate from provisioning profile; skipping expiry check."
        exit 0
      fi
      # Try a couple of known plutil date formats. If none parse, skip the check rather
      # than risk a false-positive "expired" block on a format we didn't anticipate.
      EXPIRY_EPOCH=""
      for fmt in "%Y-%m-%dT%H:%M:%SZ" "%Y-%m-%d %H:%M:%S %z" "%Y-%m-%d %H:%M:%S +0000"; do
        if parsed=$(date -j -f "$fmt" "$EXPIRY" +%s 2>/dev/null); then
          EXPIRY_EPOCH="$parsed"
          break
        fi
      done
      if [[ -z "$EXPIRY_EPOCH" ]]; then
        echo "WARNING: could not parse ExpirationDate '$EXPIRY'; skipping expiry check."
        exit 0
      fi
      NOW_EPOCH=$(date +%s)
      if [[ "$EXPIRY_EPOCH" -le "$NOW_EPOCH" ]]; then
        echo "ERROR: provisioning profile expired on $EXPIRY. Regenerate it before rerunning."
        exit 1
      fi
      DAYS_LEFT=$(( (EXPIRY_EPOCH - NOW_EPOCH) / 86400 ))
      echo "Provisioning profile valid until $EXPIRY ($DAYS_LEFT days remaining)."
      if [[ "$DAYS_LEFT" -lt 14 ]]; then
        echo "WARNING: profile expires in under 14 days — regenerate soon."
      fi

  # ============================================================
  # Build the bundle
  # ============================================================

  - name: Add pinned macmon to PATH
    run: |
      MACMON_DIR=$(nix develop --command sh -c 'dirname $(which macmon)')
      echo "Using macmon from: $MACMON_DIR"
      echo "$MACMON_DIR" >> $GITHUB_PATH
      # Remove any Homebrew macmon so PyInstaller can't accidentally pick it up
      brew uninstall macmon 2>/dev/null || true

  - name: Build PyInstaller bundle
    run: uv run pyinstaller packaging/pyinstaller/exo.spec

  - name: Build Swift app
    env:
      MACOS_CERTIFICATE_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PASSWORD }}
      SPARKLE_FEED_URL: ${{ secrets.SPARKLE_FEED_URL }}
      SPARKLE_ED25519_PUBLIC: ${{ secrets.SPARKLE_ED25519_PUBLIC }}
    run: |
      cd app/EXO
      security unlock-keychain -p "$MACOS_CERTIFICATE_PASSWORD" "$BUILD_KEYCHAIN_PATH"
      SIGNING_IDENTITY=$(security find-identity -v -p codesigning "$BUILD_KEYCHAIN_PATH" | awk -F '"' '{print $2}')
      xcodebuild clean build \
        -scheme EXO \
        -configuration Release \
        -derivedDataPath build \
        MARKETING_VERSION="$RELEASE_VERSION" \
        CURRENT_PROJECT_VERSION="$EXO_BUILD_VERSION" \
        EXO_BUILD_TAG="$RELEASE_VERSION" \
        EXO_BUILD_COMMIT="$GITHUB_SHA" \
        SPARKLE_FEED_URL="$SPARKLE_FEED_URL" \
        SPARKLE_ED25519_PUBLIC="$SPARKLE_ED25519_PUBLIC" \
        CODE_SIGNING_IDENTITY="$SIGNING_IDENTITY" \
        CODE_SIGN_INJECT_BASE_ENTITLEMENTS=YES
      mkdir -p ../../output
      cp -R build/Build/Products/Release/EXO.app ../../output/EXO.app

  - name: Inject PyInstaller runtime
    run: |
      rm -rf output/EXO.app/Contents/Resources/exo
      mkdir -p output/EXO.app/Contents/Resources
      cp -R dist/exo output/EXO.app/Contents/Resources/exo

  - name: Codesign PyInstaller runtime
    env:
      MACOS_CERTIFICATE_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PASSWORD }}
    run: |
      cd output
      security unlock-keychain -p "$MACOS_CERTIFICATE_PASSWORD" "$BUILD_KEYCHAIN_PATH"
      SIGNING_IDENTITY=$(security find-identity -v -p codesigning "$BUILD_KEYCHAIN_PATH" | awk -F '"' '{print $2}')
      RUNTIME_DIR="EXO.app/Contents/Resources/exo"
      find "$RUNTIME_DIR" -type f \( -perm -111 -o -name "*.dylib" -o -name "*.so" \) -print0 |
        while IFS= read -r -d '' file; do
          /usr/bin/codesign --force --timestamp --options runtime \
            --sign "$SIGNING_IDENTITY" "$file"
        done

  - name: Sign, notarize, and create DMG
    env:
      MACOS_CERTIFICATE_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PASSWORD }}
      APPLE_NOTARIZATION_USERNAME: ${{ secrets.APPLE_NOTARIZATION_USERNAME }}
      APPLE_NOTARIZATION_PASSWORD: ${{ secrets.APPLE_NOTARIZATION_PASSWORD }}
      APPLE_NOTARIZATION_TEAM: ${{ secrets.APPLE_NOTARIZATION_TEAM }}
    run: |
      set -o pipefail
      cd output
      security unlock-keychain -p "$MACOS_CERTIFICATE_PASSWORD" "$BUILD_KEYCHAIN_PATH"
      SIGNING_IDENTITY=$(security find-identity -v -p codesigning "$BUILD_KEYCHAIN_PATH" | awk -F '"' '{print $2}')

      # Fail fast if notarization creds are partial. All-or-nothing.
      CRED_COUNT=0
      for v in "$APPLE_NOTARIZATION_USERNAME" "$APPLE_NOTARIZATION_PASSWORD" "$APPLE_NOTARIZATION_TEAM"; do
        [[ -n "$v" ]] && CRED_COUNT=$((CRED_COUNT + 1))
      done
      if [[ "$CRED_COUNT" -ne 0 && "$CRED_COUNT" -ne 3 ]]; then
        echo "ERROR: partial Apple notarization credentials set ($CRED_COUNT/3). Aborting."
        exit 1
      fi

      /usr/bin/codesign --deep --force --timestamp --options runtime \
        --sign "$SIGNING_IDENTITY" EXO.app

      # Pre-flight: verify the signed app BEFORE building DMG and submitting to Apple.
      # If this fails, notarization will fail too — cheap way to fail in seconds, not 15 minutes.
      echo "===== codesign --verify EXO.app ====="
      if ! /usr/bin/codesign --verify --deep --strict --verbose=2 EXO.app; then
        echo "ERROR: EXO.app failed codesign verification. Dumping signing status of every executable:"
        find EXO.app -type f \( -perm -111 -o -name "*.dylib" -o -name "*.so" -o -name "*.framework" \) -print0 |
          while IFS= read -r -d '' f; do
            printf -- '--- %s\n' "$f"
            /usr/bin/codesign -dv --verbose=2 "$f" 2>&1 | sed 's/^/    /' || true
          done
        exit 1
      fi

      # Gatekeeper assessment. A failure here strongly predicts notarization rejection.
      echo "===== spctl assessment (predicts notarization outcome) ====="
      /usr/bin/spctl -a -vvv -t install EXO.app || echo "WARNING: spctl assessment failed — notarization is likely to fail too."

      mkdir -p dmg-root
      cp -R EXO.app dmg-root/
      ln -s /Applications dmg-root/Applications
      DMG_NAME="EXO-${RELEASE_VERSION}.dmg"
      hdiutil create -volname "EXO" -srcfolder dmg-root -ov -format UDZO "$DMG_NAME"
      /usr/bin/codesign --force --timestamp --options runtime \
        --sign "$SIGNING_IDENTITY" "$DMG_NAME"

      echo "===== codesign --verify DMG ====="
      if ! /usr/bin/codesign --verify --verbose=2 "$DMG_NAME"; then
        echo "ERROR: DMG failed codesign verification."
        exit 1
      fi

      if [[ -n "$APPLE_NOTARIZATION_USERNAME" ]]; then
        echo "===== notarytool submit ====="
        # `|| true` so set -e doesn't abort before we can echo output / fetch the log.
        # We rely on the parsed STATUS below to decide pass/fail.
        SUBMISSION_OUTPUT=$(xcrun notarytool submit "$DMG_NAME" \
          --apple-id "$APPLE_NOTARIZATION_USERNAME" \
          --password "$APPLE_NOTARIZATION_PASSWORD" \
          --team-id "$APPLE_NOTARIZATION_TEAM" \
          --wait --timeout 15m 2>&1) || true
        echo "$SUBMISSION_OUTPUT"

        SUBMISSION_ID=$(echo "$SUBMISSION_OUTPUT" | awk 'tolower($1)=="id:" && $2 ~ /^[0-9a-fA-F-]+$/ {print $2; exit}')
        STATUS=$(echo "$SUBMISSION_OUTPUT" | awk 'tolower($1)=="status:" {print $2; exit}')

        if [[ -n "$SUBMISSION_ID" ]]; then
          xcrun notarytool log "$SUBMISSION_ID" \
            --apple-id "$APPLE_NOTARIZATION_USERNAME" \
            --password "$APPLE_NOTARIZATION_PASSWORD" \
            --team-id "$APPLE_NOTARIZATION_TEAM" > notarization-log.txt || true
          echo "===== Notarization Log ====="
          cat notarization-log.txt
          echo "============================"
        fi

        if [[ "$STATUS" != "Accepted" ]]; then
          echo "Notarization failed with status: ${STATUS:-Unknown}"
          exit 1
        fi

        xcrun stapler staple "$DMG_NAME"
      fi

  - name: Generate Sparkle appcast
    env:
      SPARKLE_DOWNLOAD_PREFIX: ${{ env.SPARKLE_DOWNLOAD_PREFIX }}
      SPARKLE_ED25519_PRIVATE: ${{ secrets.SPARKLE_ED25519_PRIVATE }}
      IS_ALPHA: ${{ env.IS_ALPHA }}
    run: |
      set -euo pipefail
      cd output
      DOWNLOAD_PREFIX="${SPARKLE_DOWNLOAD_PREFIX:-https://assets.exolabs.net}"
      echo "$SPARKLE_ED25519_PRIVATE" > sparkle_ed25519.key
      chmod 600 sparkle_ed25519.key

      CHANNEL_FLAG=""
      if [[ "$IS_ALPHA" == "true" ]]; then
        CHANNEL_FLAG="--channel alpha"
        echo "Generating appcast for alpha channel"
      fi

      $SPARKLE_BIN/generate_appcast \
        --ed-key-file sparkle_ed25519.key \
        --download-url-prefix "$DOWNLOAD_PREFIX" \
        $CHANNEL_FLAG \
        .

  - name: Inject release notes into appcast
    if: github.ref_type == 'tag' && env.HAS_RELEASE_NOTES == 'true'
    env:
      RELEASE_VERSION: ${{ env.RELEASE_VERSION }}
    run: |
      # Inject markdown release notes with sparkle:format="markdown" (Sparkle 2.9+)
      export NOTES=$(cat "$RELEASE_NOTES_FILE")

      # Insert description after the enclosure tag for this version
      awk '
        /<enclosure[^>]*>/ && index($0, ENVIRON["RELEASE_VERSION"]) {
          print
          print "            <description sparkle:format=\"markdown\"><![CDATA["
          print ENVIRON["NOTES"]
          print "            ]]></description>"
          next
        }
        { print }
      ' output/appcast.xml > output/appcast.xml.tmp && mv output/appcast.xml.tmp output/appcast.xml

      echo "Injected markdown release notes for version $RELEASE_VERSION"

  # ============================================================
  # Upload artifacts
  # ============================================================

  - name: Upload DMG
    uses: actions/upload-artifact@v4
    with:
      name: EXO-dmg-${{ env.RELEASE_VERSION }}
      path: output/EXO-${{ env.RELEASE_VERSION }}.dmg

  - name: Upload to S3
    if: env.SPARKLE_S3_BUCKET != ''
    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_REGION: ${{ env.AWS_REGION }}
      SPARKLE_S3_BUCKET: ${{ env.SPARKLE_S3_BUCKET }}
      SPARKLE_S3_PREFIX: ${{ env.SPARKLE_S3_PREFIX }}
      IS_ALPHA: ${{ env.IS_ALPHA }}
    run: |
      set -euo pipefail
      cd output
      PREFIX="${SPARKLE_S3_PREFIX:-}"
      if [[ -n "$PREFIX" && "${PREFIX: -1}" != "/" ]]; then
        PREFIX="${PREFIX}/"
      fi
      DMG_NAME="EXO-${RELEASE_VERSION}.dmg"

      if [[ "${{ github.ref_type }}" != "tag" ]]; then
        aws s3 cp "$DMG_NAME" "s3://${SPARKLE_S3_BUCKET}/${PREFIX}EXO-${GITHUB_SHA}.dmg"
        exit 0
      fi

      aws s3 cp "$DMG_NAME" "s3://${SPARKLE_S3_BUCKET}/${PREFIX}${DMG_NAME}"
      if [[ "$IS_ALPHA" != "true" ]]; then
        aws s3 cp "$DMG_NAME" "s3://${SPARKLE_S3_BUCKET}/${PREFIX}EXO-latest.dmg"
        aws s3 cp appcast.xml "s3://${SPARKLE_S3_BUCKET}/${PREFIX}appcast.xml" --content-type application/xml --cache-control no-cache
      fi

  - name: Publish GitHub Release
    if: github.ref_type == 'tag'
    env:
      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    run: |
      DMG_PATH="output/EXO-${RELEASE_VERSION}.dmg"

      if [[ "$HAS_RELEASE_NOTES" == "true" ]]; then
        # Update the draft release with the tag and upload DMG
        gh api --method PATCH "repos/${{ github.repository }}/releases/$DRAFT_RELEASE_ID" \
          -f tag_name="$GITHUB_REF_NAME" \
          -F draft=false
        gh release upload "$GITHUB_REF_NAME" "$DMG_PATH" --clobber
        echo "Published release $GITHUB_REF_NAME with DMG attached"
      else
        # Alpha without draft release - create one with auto-generated notes
        gh release create "$GITHUB_REF_NAME" "$DMG_PATH" \
          --title "$GITHUB_REF_NAME" \
          --generate-notes \
          --prerelease
        echo "Created alpha release $GITHUB_REF_NAME with auto-generated notes"
      fi

File: .github/workflows/pipeline.yml

name: ci-pipeline

on:
push:
pull_request:
branches:
- staging
- main

jobs:
nix:
name: Build and check (${{ matrix.system }})
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
- runner: macos-26
system: aarch64-darwin
- runner: ubuntu-latest
system: x86_64-linux
- runner: ubuntu-24.04-arm
system: aarch64-linux
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
lfs: false

text
- uses: cachix/install-nix-action@v31
    with:
      nix_path: nixpkgs=channel:nixos-unstable

  - uses: cachix/cachix-action@v14
    name: Configure Cachix
    with:
      name: exo
      authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"

  - name: Build Metal packages (macOS only)
    if: runner.os == 'macOS'
    run: |
      # Try to build metal-toolchain first (may succeed via cachix cache hit)
      if nix build .#metal-toolchain 2>/dev/null; then
        echo "metal-toolchain built successfully (likely cache hit)"
      else
        echo "metal-toolchain build failed, extracting from Xcode..."

        NAR_HASH="sha256-ayR5mXN4sZAddwKEG2OszGRF93k9ZFc7H0yi2xbylQw="
        NAR_NAME="metal-toolchain-17C48.nar"

        # Use RUNNER_TEMP to avoid /tmp symlink issues on macOS
        WORK_DIR="${RUNNER_TEMP}/metal-work"
        mkdir -p "$WORK_DIR"

        # Download the Metal toolchain component
        xcodebuild -downloadComponent MetalToolchain

        # Find and mount the DMG
        DMG_PATH=$(find /System/Library/AssetsV2/com_apple_MobileAsset_MetalToolchain -name '*.dmg' 2>/dev/null | head -1)
        if [ -z "$DMG_PATH" ]; then
          echo "Error: Could not find Metal toolchain DMG"
          exit 1
        fi

        echo "Found DMG at: $DMG_PATH"
        hdiutil attach "$DMG_PATH" -mountpoint "${WORK_DIR}/metal-dmg"

        # Copy the toolchain
        cp -R "${WORK_DIR}/metal-dmg/Metal.xctoolchain" "${WORK_DIR}/metal-export"
        hdiutil detach "${WORK_DIR}/metal-dmg"

        # Create NAR and add to store
        nix nar pack "${WORK_DIR}/metal-export" > "${WORK_DIR}/${NAR_NAME}"
        STORE_PATH=$(nix store add --mode flat "${WORK_DIR}/${NAR_NAME}")
        echo "Added NAR to store: $STORE_PATH"

        # Verify the hash matches
        ACTUAL_HASH=$(nix hash file "${WORK_DIR}/${NAR_NAME}")
        if [ "$ACTUAL_HASH" != "$NAR_HASH" ]; then
          echo "Warning: NAR hash mismatch!"
          echo "Expected: $NAR_HASH"
          echo "Actual:   $ACTUAL_HASH"
          echo "The metal-toolchain.nix may need updating"
        fi

        # Clean up
        rm -rf "$WORK_DIR"

        # Retry the build now that NAR is in store
        nix build .#metal-toolchain
      fi

  - name: Build all Nix outputs
    run: |
      nix flake show --json | jq -r '
        [
          (.packages."${{ matrix.system }}" // {} | keys[] | ".#packages.${{ matrix.system }}.\(.)"),
          (.devShells."${{ matrix.system }}" // {} | keys[] | ".#devShells.${{ matrix.system }}.\(.)")
        ] | .[]
      ' | xargs nix build

  - name: Run nix flake check
    run: nix flake check

  - name: Run pytest (macOS only)
    if: runner.os == 'macOS'
    run: |
      # Build the test environment (requires relaxed sandbox for uv2nix on macOS)
      TEST_ENV=$(nix build '.#exo-test-env' --option sandbox relaxed --print-out-paths)

      # Run pytest outside sandbox (needs GPU access for MLX)
      export HOME="$RUNNER_TEMP"
      export EXO_TESTS=1
      export EXO_DASHBOARD_DIR="$PWD/dashboard/" 
      export EXO_RESOURCES_DIR="$PWD/resources" 
      $TEST_ENV/bin/python -m pytest src -m "not slow" --import-mode=importlib

File: .zed/settings.json

// Folder-specific settings
//
// For a full list of overridable settings, and general information on folder-specific settings,
// see the documentation: https://zed.dev/docs/configuring-zed#settings-files
{
"lsp": {
"nix_python": {
"binary": {
"path": "nix",
"arguments": [
"run",
"--quiet",
"--no-warn-dirty",
"--no-allow-import-from-derivation",
"--print-build-logs",
"never",
"${projectRoot}#python-lsp",
"--",
"--stdio"
]
}
}
},
"languages": {
"Python": {
"language_servers": ["nix_python"]
}
}
}

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →