{"owner":"lemonade-sdk","repo":"lemonade","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md"],"skills":{"AGENTS.md":"# AGENTS.md\n\nThis file provides guidance to agent driven code reviews when working with this repository.\n\n## Project Overview\n\nLemonade is a local LLM server providing GPU and NPU acceleration for running large language models on consumer hardware. It exposes OpenAI-compatible, Ollama-compatible, and Anthropic-compatible REST APIs, plus a WebSocket Realtime API. It supports multiple backends: llama.cpp, FastFlowLM, RyzenAI, whisper.cpp, stable-diffusion.cpp, Kokoro TTS, and Moonshine.\n\n## Architecture\n\n### Executables\n\n- **lemond** — Pure HTTP server. Handles REST API, routes requests to backends, manages model loading/unloading. Configured via `config.json` in the lemonade cache directory. CLI args: `[cache_dir] [--port PORT] [--host HOST]`.\n- **lemonade** — CLI client (`src/cpp/cli/`). Commands: `list`, `pull`, `delete`, `run`, `status`, `logs`, `launch`, `backends`, `scan`, etc. Communicates with router via HTTP. Discovers running server via UDP beacon.\n- **LemonadeServer.exe** (Windows) — SUBSYSTEM:WINDOWS GUI app that embeds `lemond` and shows a system tray icon. Auto-starts via Windows startup folder.\n- **lemonade-tray** (macOS/Linux) — Lightweight tray client that connects to a running `lemond`. Platform code in `src/cpp/tray/platform/`.\n\n### Backend Abstraction\n\n`WrappedServer` (`src/cpp/include/lemon/wrapped_server.h`) is the abstract base class. Each backend inherits it and implements `load()`, `unload()`, `chat_completion()`, `completion()`, `responses()`, and optionally `install()` / `download_model()`. Backends run as **subprocesses** — Lemonade forwards HTTP requests to them.\n\n| Backend | Class | Capabilities | Device | Purpose |\n|---------|-------|-------------|--------|---------|\n| llama.cpp | `LlamaCppServer` | Completion, Embeddings, Reranking | GPU | LLM inference — CPU/GPU (Vulkan, ROCm, Metal) |\n| FastFlowLM | `FastFlowLMServer` | Completion, Embeddings, Audio | NPU | NPU inference (multi-modal: LLM, ASR, embeddings) |\n| RyzenAI | `RyzenAIServer` | Completion | NPU | Hybrid NPU inference |\n| vLLM | `VLLMServer` | Completion | GPU | LLM inference — ROCm on AMD iGPU/dGPU (Linux). **Experimental**, validated only on gfx1151 (Strix Halo). |\n| whisper.cpp | `WhisperServer` | Audio | CPU | Audio transcription |\n| stable-diffusion.cpp | `SdServer` | Image | CPU | Image generation, editing, variations |\n| Kokoro | `KokoroServer` | TTS | CPU | Text-to-speech |\n| Moonshine | `MoonshineServer` | Audio | CPU | Streaming speech-to-text (ONNX-based) |\n\nCapability interfaces: `ICompletionServer`, `IEmbeddingsServer`, `IRerankingServer`, `ITranscriptionServer`, `IImageServer`, `ITextToSpeechServer` (defined in `server_capabilities.h`). Use `supports_capability<T>(server)` template for runtime checks.\n\n### Router & Multi-Model Support\n\n`Router` (`src/cpp/server/router.cpp`) manages a vector of `WrappedServer` instances. Routes requests based on model recipe, maintains LRU caches per model type (LLM, embedding, reranking, audio, image, TTS — see `model_types.h`), and enforces NPU exclusivity. Configurable via `--max-loaded-models`. On non-file-not-found errors, the router uses a \"nuclear option\" — evicts all models and retries the load.\n\n### Model Manager & Recipe System\n\n`ModelManager` (`src/cpp/server/model_manager.cpp`) loads the registry from `src/cpp/resources/server_models.json`. Each model has \"recipes\" defining which backend and config to use. Backend versions are pinned in `src/cpp/resources/backend_versions.json`. Models download from Hugging Face.\n\n### API Routes\n\nAll core endpoints are registered under **4 path prefixes**:\n- `/api/v0/` — Legacy\n- `/api/v1/` — Current\n- `/v0/` — Legacy short\n- `/v1/` — OpenAI SDK / LiteLLM compatibility\n\n**Core endpoints:** `chat/completions`, `completions`, `embeddings`, `reranking`, `models`, `models/{id}`, `health`, `pull`, `pull/variants`, `registry/search`, `load`, `unload`, `delete`, `params`, `install`, `uninstall`, `audio/transcriptions`, `audio/speech`, `images/generations`, `images/edits`, `images/variations`, `responses`, `stats`, `system-info`, `system-stats`, `log-level`, `logs/stream`, `jobs`, `jobs/{id}`, `jobs/{id}/pause`, `jobs/{id}/interrupt`, `jobs/{id}/resume`\n\n**Job engine** (`POST jobs`, `GET jobs`, `GET/DELETE jobs/{id}`, `POST jobs/{id}/{pause,interrupt,resume}`): server-side sequences of ops (`system_info`, `system_stats`, `models`, `sleep`, `load`, `unload`, `chat`) with data passing, forward-only branching, and a pause/interrupt/resume lifecycle persisted across restart. Exclusive ops hold a Router slot so normal traffic queues. See `docs/dev/job-system.md` and `docs/dev/job-expression-language.md`.\n\n**Ollama-compatible endpoints** (under `/api/` without version prefix): `chat`, `generate`, `tags`, `show`, `delete`, `pull`, `embed`, `embeddings`, `ps`, `version`\n\n**Anthropic-compatible endpoint:** `POST /api/messages` — supports message completion, tool use, and SSE streaming.\n\n**MCP gateway endpoint:** `POST /mcp` — Model Context Protocol (Streamable HTTP transport, spec `2025-06-18`). Single JSON-RPC 2.0 endpoint exposing 5 tools (`lemonade_list_models`, `lemonade_chat`, `lemonade_transcribe_audio`, `lemonade_generate_image`, `lemonade_omni`). GET returns 405.\n\n**WebSocket Realtime API**: OpenAI-compatible Realtime protocol for real-time audio transcription. `/realtime` and `/logs/stream` accept WebSocket upgrades directly on the main HTTP port; a dedicated listener on an OS-assigned port (9000+, exposed via the `websocket_port` field in the `/health` response) also remains for backward compatibility.\n\n**Internal endpoints:** `POST /internal/shutdown`, `GET /internal/aliases`, `POST /internal/aliases`, `DELETE /internal/aliases/{alias}`\n\nOptional API key auth via `LEMONADE_API_KEY` env var (regular API endpoints) or `LEMONADE_ADMIN_API_KEY` env var (full access including internal endpoints). Clients prefer `LEMONADE_ADMIN_API_KEY` if set. CORS enabled on all routes.\n\n### Desktop & Web App\n\n- **Tauri app** — React 19 + TypeScript in `src/app/`, Rust host in `src/app/src-tauri/`. Uses native OS webview (WebView2 on Windows, WKWebView on macOS, webkit2gtk on Linux). Pure CSS (dark theme), context-based state. Key components: `ChatWindow.tsx`, `ModelManager.tsx`, `DownloadManager.tsx`, `BackendManager.tsx`. Feature panels: LLMChat, ImageGeneration, Transcription, TTS, Embedding, Reranking. The renderer keeps its `window.api` contract via `src/app/src/renderer/tauriShim.ts`, which maps each call to a Tauri `invoke()` or event `listen()`.\n- **Web app** — Browser-only version in `src/web-app/`. Reuses the shared renderer from `src/app/src/` via webpack's `entry`/`template` paths (no OS symlinks); the `BuildWebApp.cmake` script stages both trees side-by-side under `build/web-app-staging/` for the actual webpack build. Built via CMake `BUILD_WEB_APP=ON`. Served at `/app`. A mock `window.api` is injected by the C++ server (`src/cpp/server/server.cpp`) so the shared renderer works unchanged in the browser.\n\n### Key Dependencies\n\n**C++ (FetchContent):** cpp-httplib, nlohmann/json, CLI11, libcurl, zstd, libwebsockets, brotli (macOS). Platform SSL: Schannel (Windows), SecureTransport (macOS), OpenSSL (Linux).\n\n**Desktop app:** Tauri v2 (Rust), React 19, TypeScript 5.3, Webpack 5, markdown-it, highlight.js, katex. Rust crates: `tauri`, `tauri-plugin-{opener,clipboard-manager,single-instance,deep-link}`, `tokio`, `reqwest`, `serde`.\n\n## Build Commands\n\nCMakeLists.txt is at the repository root. Build uses CMake presets — run the setup script first, then build with `--preset`.\n\n```bash\n# 1. Setup (configures build directory and installs deps)\n./setup.sh          # Linux / macOS\n./setup.ps1         # Windows (PowerShell)\n\n# 2. Build C++ server\ncmake --build --preset default          # Linux / macOS (Ninja)\ncmake --build --preset windows          # Windows (Visual Studio 2022)\ncmake --build --preset vs18             # Windows (Visual Studio 2026)\n\n# 3. Tauri desktop app (optional, requires Node.js 20+ and Rust via rustup)\ncmake --build --preset default --target tauri-app    # Linux / macOS\ncmake --build --preset windows --target tauri-app    # Windows (VS 2022)\ncmake --build --preset vs18 --target tauri-app       # Windows (VS 2026)\n\n# 4. Web app (auto-built on all platforms)\ncmake --build --preset default --target web-app         # Linux / macOS\ncmake --build --preset windows --target web-app         # Windows\n\n# 5. Windows MSI installer (WiX 5.0+ required)\ncmake --build --preset windows --target wix_installer_minimal  # server + web-app\ncmake --build --preset windows --target wix_installer_full     # server + Tauri app + web-app\n\n# 6. macOS signed installer\ncmake --build --preset default --target package-macos\n\n# 7. Linux .deb / .rpm\ncd build && cpack            # .deb\ncd build && cpack -G RPM     # .rpm\n```\n\nCMake presets: `default` (Ninja, Release), `windows` (VS 2022), `vs18` (VS 2026), `debug` (Ninja, Debug).\n\nCMake options: `BUILD_WEB_APP` (ON by default on all platforms), `BUILD_TAURI_APP` (Linux only, include Tauri desktop app in deb), `LEMONADE_SYSTEMD_UNIT_NAME` (default: `lemond.service`).\n\n## Testing\n\nIntegration tests in Python against a live server. Tests auto-discover the `lemonade` CLI binary from the build directory; use `--cli-binary` to override.\n\n```bash\npip install -r test/requirements.txt\n\n# CLI tests (no inference backend needed)\npython test/server_cli2.py\n\n# Endpoint tests (no inference backend needed)\npython test/server_endpoints.py\n\n# LLM tests (specify wrapped server and backend)\npython test/server_llm.py --wrapped-server llamacpp --backend vulkan\n\n# Audio transcription tests\npython test/server_whisper.py\n\n# Image generation tests (slow)\npython test/server_sd.py\n```\n\nTest utilities in `test/utils/` with `server_base.py` as the base class. Test dependencies include `requests`, `httpx`, `openai`, `huggingface_hub`, `psutil`, `numpy`, `websockets`, and `ollama`.\n\n### C++ unit tests\n\nC++ unit tests live in `test/cpp/` and are wired up in the root `CMakeLists.txt`. The packaging workflow builds the `cpp-ci-tests` aggregate target and runs `ctest -L cpp-ci`, so a test only runs in CI if it is both labeled `cpp-ci` **and** a dependency of that aggregate target.\n\n**Direct `add_test()` is disabled** (the built-in is overridden to fail with a fatal error just before the test section). Every test MUST be declared with the `add_cpp_ci_test()` helper, which forces an explicit `CI <ON|OFF>` decision at the call site so a test is never silently omitted from — or accidentally added to — CI.\n\n**The enclosing `if()` MUST test `BUILD_TESTING`.** Distro packaging (`contrib/debian/rules`, the RPM job) configures with `BUILD_TESTING=OFF` so it does not build ~45 test binaries it then discards; calling `add_cpp_ci_test()` in that configuration is a fatal error rather than a silent return to the slow build.\n\n```cmake\nif(BUILD_TESTING AND EXISTS \"${CMAKE_CURRENT_SOURCE_DIR}/test/cpp/test_my_feature.cpp\")\n    add_executable(test_my_feature test/cpp/test_my_feature.cpp ...)\n    # ...target_include_directories / target_link_libraries...\n\n    include(CTest)\n    add_cpp_ci_test(MyFeatureTest CI ON COMMAND test_my_feature)\nendif()\n```\n\n```cmake\nadd_cpp_ci_test(<TestName>\n                CI <ON|OFF>                 # required — run under `ctest -L cpp-ci`?\n                COMMAND <command> [args...] # required — what CTest runs\n                [DEPENDS <target>...])      # CI build deps; defaults to the\n                                            # first COMMAND token (the test target)\n```\n\n- `CI ON` labels the test `cpp-ci` and makes its build target(s) a dependency of `cpp-ci-tests`.\n- `CI OFF` still creates the CTest test (for local/other runs) but keeps it out of packaging CI. Use this only for tests that are intentionally excluded (e.g. tests that need a backend, are platform-gated, or are slow CMake-configuration tests).\n\nPass `DEPENDS` only when the CI build needs targets beyond the `COMMAND` executable. `add_cpp_ci_test` calls `register_cpp_ci_test()` internally; do not call `add_test()` or `register_cpp_ci_test()` directly.\n\n## Code Style\n\n### Comments & Documentation\n\n**Default to writing no comments.** Only add a comment when the WHY is non-obvious: a hidden constraint, a subtle invariant, a workaround for a specific bug, or behavior that would surprise a reader. If removing the comment wouldn't confuse a future reader, don't write it.\n\n**Never write comments that explain WHAT the code does** — well-named identifiers already do that. Don't reference the current task, fix, or callers (\"used by X\", \"added for the Y flow\", \"handles the case from issue #123\") — those belong in the PR description and rot as the codebase evolves.\n\n**PR descriptions should be concise.** 1-3 sentences for the summary. No essays. The diff shows what changed; the description explains why and any non-obvious context. Bullet points over paragraphs.\n\n### C++\n- C++17, `lemon::` namespace\n- `snake_case` for functions/variables, `CamelCase` for classes/types\n- 4-space indent, `#pragma once` for headers\n- Keep `#include` directives in alphabetical order within each include block\n- Platform guards: `#ifdef _WIN32`, `#ifdef __APPLE__`, `#ifdef __linux__`\n\n### Python\n- **Black** formatting (v26.1.0, enforced in CI)\n- Pylint with `.pylintrc`\n- Pre-commit hooks: trailing-whitespace, end-of-file-fixer, check-yaml, check-added-large-files\n\n### TypeScript/React\n- React 19, pure CSS (dark theme), context-based state\n- UI/frontend changes are handled by core maintainers only\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `CMakeLists.txt` | Root build config (version, deps, targets) |\n| `src/cpp/server/server.cpp` | HTTP route registration and all handlers |\n| `src/cpp/server/router.cpp` | Request routing and multi-model orchestration |\n| `src/cpp/server/model_manager.cpp` | Model registry, downloads, recipe resolution |\n| `src/cpp/include/lemon/wrapped_server.h` | Backend abstract base class |\n| `src/cpp/include/lemon/server_capabilities.h` | Backend capability interfaces |\n| `src/cpp/resources/server_models.json` | Model registry |\n| `src/cpp/resources/backend_versions.json` | Backend version pins |\n| `docs/tools/gen_backend_boilerplate.py` | Regenerates committed artifacts from the C++ backend descriptors. Outputs: the whole of `src/cpp/resources/defaults.json` (per-recipe sections only; global keys stay hand-maintained in that file), and `<!-- BEGIN/END GENERATED -->` regions in `docs/dev/backends-reference.md`, root `README.md`, `docs/guide/cli.md`, `docs/guide/configuration/{README,multi-model,custom-models}.md`, and `docs/assets/models.js`. Don't hand-edit those regions/sections; CI runs `--check` and fails on drift. |\n| `src/cpp/server/anthropic_api.cpp` | Anthropic API compatibility |\n| `src/cpp/server/ollama_api.cpp` | Ollama API compatibility |\n| `src/cpp/server/mcp_server.cpp` | MCP gateway (POST /mcp) |\n| `src/cpp/include/lemon/websocket_server.h` | WebSocket Realtime API server |\n| `src/cpp/include/lemon/model_types.h` | Model type and device type enums |\n| `src/cpp/include/lemon/config_file.h` | config.json load/save/migrate |\n| `src/cpp/include/lemon/recipe_options.h` | Per-recipe JSON configuration |\n| `src/cpp/tray/tray_app.cpp` | Tray application UI and logic |\n| `src/app/src/renderer/ModelManager.tsx` | Model management UI |\n| `src/app/src/renderer/ChatWindow.tsx` | Chat interface |\n\n## Critical Invariants\n\nThese MUST be maintained in all changes:\n\n1. **Quad-prefix registration** — Every new endpoint MUST be registered under `/api/v0/`, `/api/v1/`, `/v0/`, AND `/v1/`. Documented exceptions: Ollama (`/api/*` without version prefix), Anthropic (`POST /v1/messages` only), and MCP (`POST /mcp`) — each of those protocols mandates a fixed URL shape that conflicts with the quad-prefix scheme.\n2. **NPU exclusivity** — Exclusive-NPU recipes (`ryzenai-llm`, `whispercpp` on NPU) evict ALL other NPU models before loading. FastFlowLM (`flm`) can coexist with other FLM types (max 1 per FLM type) but not with exclusive-NPU recipes.\n3. **WrappedServer contract** — New backends MUST implement all core virtual methods: `load()`, `unload()`, `chat_completion()`, `completion()`, `responses()`.\n4. **Subprocess model** — Backends run as subprocesses (llama-server, whisper-server, sd-server, koko, flm, ryzenai-server, moonshine-server). They must NOT run in-process.\n5. **Recipe integrity** — Changes to `server_models.json` must have valid recipes referencing backends in `backend_versions.json`. When adding or updating `vllm` models, also update `src/cpp/resources/vllm_model_config.json` if the model family needs vLLM-specific args such as tool-call parser settings.\n6. **Cross-platform** — Code must compile on Windows (MSVC), Linux (GCC/Clang), macOS (AppleClang). Platform-specific code must use `#ifdef` guards.\n7. **No hardcoded paths** — Use path utilities. Windows/Linux/macOS paths differ.\n8. **Thread safety** — Router serves concurrent HTTP requests. Shared state must be properly guarded.\n9. **Ollama compatibility** — Changes to model listing or management must not break `/api/*` Ollama endpoints.\n10. **API key passthrough** — When `LEMONADE_API_KEY` is set, all API routes must enforce authentication.\n11. **Many-clients-one-server topology** — A single `lemond` can be driven by multiple desktop/tray/CLI clients, potentially on different machines. Per-client UI state (layout, zoom, view selection, the client's own base URL and API key) MUST live locally in the client, never in `lemond`. Do not move `app_settings.json` behind an HTTP endpoint. **Shared infrastructure config** (cloud provider URLs, backend version pins) lives in `lemond`'s `config.json` so it's visible to every client and to the CLI. **Cloud API keys** specifically MUST NOT be written to disk: they live in `LEMONADE_<PROVIDER>_API_KEY` env vars (persistent) or in `lemond`'s process memory via `POST /v1/cloud/auth` (ephemeral, dies on restart).\n12. **Web-app dependencies constrained by Debian native packaging** — `src/web-app/package.json` is kept separate from `src/app/package.json` because the native Debian package (`lemonade-server` .deb) must build using only npm modules available in Debian's `/usr/share/nodejs` (see `USE_SYSTEM_NODEJS_MODULES` in `src/web-app/webpack.config.js`). The old Electron app depended on packages Debian does not ship. Do NOT consolidate the two `package.json` files — the split is required for reproducible distro packaging.\n13. **Desktop app is on-demand; `lemond` runs independently** — On Windows, `LemonadeServer.exe` (which embeds `lemond` + tray icon) is the always-on process, auto-started via the Windows startup folder. The Tauri desktop app (`lemonade-app.exe`) is opened on demand when the user wants the UI and must not be added to startup. The desktop app must not embed or manage `lemond`'s lifecycle — it discovers the already-running server (UDP beacon for local, explicit base URL for remote) and speaks to it over HTTP.\n\n## Contributing\n\n- Open an Issue before submitting major PRs\n- UI/frontend changes are handled by core maintainers only\n- Python formatting with Black is required\n- PRs trigger CI for linting, formatting, and integration tests\n"},"files":{"AGENTS.md":"# AGENTS.md\n\nThis file provides guidance to agent driven code reviews when working with this repository.\n\n## Project Overview\n\nLemonade is a local LLM server providing GPU and NPU acceleration for running large language models on consumer hardware. It exposes OpenAI-compatible, Ollama-compatible, and Anthropic-compatible REST APIs, plus a WebSocket Realtime API. It supports multiple backends: llama.cpp, FastFlowLM, RyzenAI, whisper.cpp, stable-diffusion.cpp, Kokoro TTS, and Moonshine.\n\n## Architecture\n\n### Executables\n\n- **lemond** — Pure HTTP server. Handles REST API, routes requests to backends, manages model loading/unloading. Configured via `config.json` in the lemonade cache directory. CLI args: `[cache_dir] [--port PORT] [--host HOST]`.\n- **lemonade** — CLI client (`src/cpp/cli/`). Commands: `list`, `pull`, `delete`, `run`, `status`, `logs`, `launch`, `backends`, `scan`, etc. Communicates with router via HTTP. Discovers running server via UDP beacon.\n- **LemonadeServer.exe** (Windows) — SUBSYSTEM:WINDOWS GUI app that embeds `lemond` and shows a system tray icon. Auto-starts via Windows startup folder.\n- **lemonade-tray** (macOS/Linux) — Lightweight tray client that connects to a running `lemond`. Platform code in `src/cpp/tray/platform/`.\n\n### Backend Abstraction\n\n`WrappedServer` (`src/cpp/include/lemon/wrapped_server.h`) is the abstract base class. Each backend inherits it and implements `load()`, `unload()`, `chat_completion()`, `completion()`, `responses()`, and optionally `install()` / `download_model()`. Backends run as **subprocesses** — Lemonade forwards HTTP requests to them.\n\n| Backend | Class | Capabilities | Device | Purpose |\n|---------|-------|-------------|--------|---------|\n| llama.cpp | `LlamaCppServer` | Completion, Embeddings, Reranking | GPU | LLM inference — CPU/GPU (Vulkan, ROCm, Metal) |\n| FastFlowLM | `FastFlowLMServer` | Completion, Embeddings, Audio | NPU | NPU inference (multi-modal: LLM, ASR, embeddings) |\n| RyzenAI | `RyzenAIServer` | Completion | NPU | Hybrid NPU inference |\n| vLLM | `VLLMServer` | Completion | GPU | LLM inference — ROCm on AMD iGPU/dGPU (Linux). **Experimental**, validated only on gfx1151 (Strix Halo). |\n| whisper.cpp | `WhisperServer` | Audio | CPU | Audio transcription |\n| stable-diffusion.cpp | `SdServer` | Image | CPU | Image generation, editing, variations |\n| Kokoro | `KokoroServer` | TTS | CPU | Text-to-speech |\n| Moonshine | `MoonshineServer` | Audio | CPU | Streaming speech-to-text (ONNX-based) |\n\nCapability interfaces: `ICompletionServer`, `IEmbeddingsServer`, `IRerankingServer`, `ITranscriptionServer`, `IImageServer`, `ITextToSpeechServer` (defined in `server_capabilities.h`). Use `supports_capability<T>(server)` template for runtime checks.\n\n### Router & Multi-Model Support\n\n`Router` (`src/cpp/server/router.cpp`) manages a vector of `WrappedServer` instances. Routes requests based on model recipe, maintains LRU caches per model type (LLM, embedding, reranking, audio, image, TTS — see `model_types.h`), and enforces NPU exclusivity. Configurable via `--max-loaded-models`. On non-file-not-found errors, the router uses a \"nuclear option\" — evicts all models and retries the load.\n\n### Model Manager & Recipe System\n\n`ModelManager` (`src/cpp/server/model_manager.cpp`) loads the registry from `src/cpp/resources/server_models.json`. Each model has \"recipes\" defining which backend and config to use. Backend versions are pinned in `src/cpp/resources/backend_versions.json`. Models download from Hugging Face.\n\n### API Routes\n\nAll core endpoints are registered under **4 path prefixes**:\n- `/api/v0/` — Legacy\n- `/api/v1/` — Current\n- `/v0/` — Legacy short\n- `/v1/` — OpenAI SDK / LiteLLM compatibility\n\n**Core endpoints:** `chat/completions`, `completions`, `embeddings`, `reranking`, `models`, `models/{id}`, `health`, `pull`, `pull/variants`, `registry/search`, `load`, `unload`, `delete`, `params`, `install`, `uninstall`, `audio/transcriptions`, `audio/speech`, `images/generations`, `images/edits`, `images/variations`, `responses`, `stats`, `system-info`, `system-stats`, `log-level`, `logs/stream`, `jobs`, `jobs/{id}`, `jobs/{id}/pause`, `jobs/{id}/interrupt`, `jobs/{id}/resume`\n\n**Job engine** (`POST jobs`, `GET jobs`, `GET/DELETE jobs/{id}`, `POST jobs/{id}/{pause,interrupt,resume}`): server-side sequences of ops (`system_info`, `system_stats`, `models`, `sleep`, `load`, `unload`, `chat`) with data passing, forward-only branching, and a pause/interrupt/resume lifecycle persisted across restart. Exclusive ops hold a Router slot so normal traffic queues. See `docs/dev/job-system.md` and `docs/dev/job-expression-language.md`.\n\n**Ollama-compatible endpoints** (under `/api/` without version prefix): `chat`, `generate`, `tags`, `show`, `delete`, `pull`, `embed`, `embeddings`, `ps`, `version`\n\n**Anthropic-compatible endpoint:** `POST /api/messages` — supports message completion, tool use, and SSE streaming.\n\n**MCP gateway endpoint:** `POST /mcp` — Model Context Protocol (Streamable HTTP transport, spec `2025-06-18`). Single JSON-RPC 2.0 endpoint exposing 5 tools (`lemonade_list_models`, `lemonade_chat`, `lemonade_transcribe_audio`, `lemonade_generate_image`, `lemonade_omni`). GET returns 405.\n\n**WebSocket Realtime API**: OpenAI-compatible Realtime protocol for real-time audio transcription. `/realtime` and `/logs/stream` accept WebSocket upgrades directly on the main HTTP port; a dedicated listener on an OS-assigned port (9000+, exposed via the `websocket_port` field in the `/health` response) also remains for backward compatibility.\n\n**Internal endpoints:** `POST /internal/shutdown`, `GET /internal/aliases`, `POST /internal/aliases`, `DELETE /internal/aliases/{alias}`\n\nOptional API key auth via `LEMONADE_API_KEY` env var (regular API endpoints) or `LEMONADE_ADMIN_API_KEY` env var (full access including internal endpoints). Clients prefer `LEMONADE_ADMIN_API_KEY` if set. CORS enabled on all routes.\n\n### Desktop & Web App\n\n- **Tauri app** — React 19 + TypeScript in `src/app/`, Rust host in `src/app/src-tauri/`. Uses native OS webview (WebView2 on Windows, WKWebView on macOS, webkit2gtk on Linux). Pure CSS (dark theme), context-based state. Key components: `ChatWindow.tsx`, `ModelManager.tsx`, `DownloadManager.tsx`, `BackendManager.tsx`. Feature panels: LLMChat, ImageGeneration, Transcription, TTS, Embedding, Reranking. The renderer keeps its `window.api` contract via `src/app/src/renderer/tauriShim.ts`, which maps each call to a Tauri `invoke()` or event `listen()`.\n- **Web app** — Browser-only version in `src/web-app/`. Reuses the shared renderer from `src/app/src/` via webpack's `entry`/`template` paths (no OS symlinks); the `BuildWebApp.cmake` script stages both trees side-by-side under `build/web-app-staging/` for the actual webpack build. Built via CMake `BUILD_WEB_APP=ON`. Served at `/app`. A mock `window.api` is injected by the C++ server (`src/cpp/server/server.cpp`) so the shared renderer works unchanged in the browser.\n\n### Key Dependencies\n\n**C++ (FetchContent):** cpp-httplib, nlohmann/json, CLI11, libcurl, zstd, libwebsockets, brotli (macOS). Platform SSL: Schannel (Windows), SecureTransport (macOS), OpenSSL (Linux).\n\n**Desktop app:** Tauri v2 (Rust), React 19, TypeScript 5.3, Webpack 5, markdown-it, highlight.js, katex. Rust crates: `tauri`, `tauri-plugin-{opener,clipboard-manager,single-instance,deep-link}`, `tokio`, `reqwest`, `serde`.\n\n## Build Commands\n\nCMakeLists.txt is at the repository root. Build uses CMake presets — run the setup script first, then build with `--preset`.\n\n```bash\n# 1. Setup (configures build directory and installs deps)\n./setup.sh          # Linux / macOS\n./setup.ps1         # Windows (PowerShell)\n\n# 2. Build C++ server\ncmake --build --preset default          # Linux / macOS (Ninja)\ncmake --build --preset windows          # Windows (Visual Studio 2022)\ncmake --build --preset vs18             # Windows (Visual Studio 2026)\n\n# 3. Tauri desktop app (optional, requires Node.js 20+ and Rust via rustup)\ncmake --build --preset default --target tauri-app    # Linux / macOS\ncmake --build --preset windows --target tauri-app    # Windows (VS 2022)\ncmake --build --preset vs18 --target tauri-app       # Windows (VS 2026)\n\n# 4. Web app (auto-built on all platforms)\ncmake --build --preset default --target web-app         # Linux / macOS\ncmake --build --preset windows --target web-app         # Windows\n\n# 5. Windows MSI installer (WiX 5.0+ required)\ncmake --build --preset windows --target wix_installer_minimal  # server + web-app\ncmake --build --preset windows --target wix_installer_full     # server + Tauri app + web-app\n\n# 6. macOS signed installer\ncmake --build --preset default --target package-macos\n\n# 7. Linux .deb / .rpm\ncd build && cpack            # .deb\ncd build && cpack -G RPM     # .rpm\n```\n\nCMake presets: `default` (Ninja, Release), `windows` (VS 2022), `vs18` (VS 2026), `debug` (Ninja, Debug).\n\nCMake options: `BUILD_WEB_APP` (ON by default on all platforms), `BUILD_TAURI_APP` (Linux only, include Tauri desktop app in deb), `LEMONADE_SYSTEMD_UNIT_NAME` (default: `lemond.service`).\n\n## Testing\n\nIntegration tests in Python against a live server. Tests auto-discover the `lemonade` CLI binary from the build directory; use `--cli-binary` to override.\n\n```bash\npip install -r test/requirements.txt\n\n# CLI tests (no inference backend needed)\npython test/server_cli2.py\n\n# Endpoint tests (no inference backend needed)\npython test/server_endpoints.py\n\n# LLM tests (specify wrapped server and backend)\npython test/server_llm.py --wrapped-server llamacpp --backend vulkan\n\n# Audio transcription tests\npython test/server_whisper.py\n\n# Image generation tests (slow)\npython test/server_sd.py\n```\n\nTest utilities in `test/utils/` with `server_base.py` as the base class. Test dependencies include `requests`, `httpx`, `openai`, `huggingface_hub`, `psutil`, `numpy`, `websockets`, and `ollama`.\n\n### C++ unit tests\n\nC++ unit tests live in `test/cpp/` and are wired up in the root `CMakeLists.txt`. The packaging workflow builds the `cpp-ci-tests` aggregate target and runs `ctest -L cpp-ci`, so a test only runs in CI if it is both labeled `cpp-ci` **and** a dependency of that aggregate target.\n\n**Direct `add_test()` is disabled** (the built-in is overridden to fail with a fatal error just before the test section). Every test MUST be declared with the `add_cpp_ci_test()` helper, which forces an explicit `CI <ON|OFF>` decision at the call site so a test is never silently omitted from — or accidentally added to — CI.\n\n**The enclosing `if()` MUST test `BUILD_TESTING`.** Distro packaging (`contrib/debian/rules`, the RPM job) configures with `BUILD_TESTING=OFF` so it does not build ~45 test binaries it then discards; calling `add_cpp_ci_test()` in that configuration is a fatal error rather than a silent return to the slow build.\n\n```cmake\nif(BUILD_TESTING AND EXISTS \"${CMAKE_CURRENT_SOURCE_DIR}/test/cpp/test_my_feature.cpp\")\n    add_executable(test_my_feature test/cpp/test_my_feature.cpp ...)\n    # ...target_include_directories / target_link_libraries...\n\n    include(CTest)\n    add_cpp_ci_test(MyFeatureTest CI ON COMMAND test_my_feature)\nendif()\n```\n\n```cmake\nadd_cpp_ci_test(<TestName>\n                CI <ON|OFF>                 # required — run under `ctest -L cpp-ci`?\n                COMMAND <command> [args...] # required — what CTest runs\n                [DEPENDS <target>...])      # CI build deps; defaults to the\n                                            # first COMMAND token (the test target)\n```\n\n- `CI ON` labels the test `cpp-ci` and makes its build target(s) a dependency of `cpp-ci-tests`.\n- `CI OFF` still creates the CTest test (for local/other runs) but keeps it out of packaging CI. Use this only for tests that are intentionally excluded (e.g. tests that need a backend, are platform-gated, or are slow CMake-configuration tests).\n\nPass `DEPENDS` only when the CI build needs targets beyond the `COMMAND` executable. `add_cpp_ci_test` calls `register_cpp_ci_test()` internally; do not call `add_test()` or `register_cpp_ci_test()` directly.\n\n## Code Style\n\n### Comments & Documentation\n\n**Default to writing no comments.** Only add a comment when the WHY is non-obvious: a hidden constraint, a subtle invariant, a workaround for a specific bug, or behavior that would surprise a reader. If removing the comment wouldn't confuse a future reader, don't write it.\n\n**Never write comments that explain WHAT the code does** — well-named identifiers already do that. Don't reference the current task, fix, or callers (\"used by X\", \"added for the Y flow\", \"handles the case from issue #123\") — those belong in the PR description and rot as the codebase evolves.\n\n**PR descriptions should be concise.** 1-3 sentences for the summary. No essays. The diff shows what changed; the description explains why and any non-obvious context. Bullet points over paragraphs.\n\n### C++\n- C++17, `lemon::` namespace\n- `snake_case` for functions/variables, `CamelCase` for classes/types\n- 4-space indent, `#pragma once` for headers\n- Keep `#include` directives in alphabetical order within each include block\n- Platform guards: `#ifdef _WIN32`, `#ifdef __APPLE__`, `#ifdef __linux__`\n\n### Python\n- **Black** formatting (v26.1.0, enforced in CI)\n- Pylint with `.pylintrc`\n- Pre-commit hooks: trailing-whitespace, end-of-file-fixer, check-yaml, check-added-large-files\n\n### TypeScript/React\n- React 19, pure CSS (dark theme), context-based state\n- UI/frontend changes are handled by core maintainers only\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `CMakeLists.txt` | Root build config (version, deps, targets) |\n| `src/cpp/server/server.cpp` | HTTP route registration and all handlers |\n| `src/cpp/server/router.cpp` | Request routing and multi-model orchestration |\n| `src/cpp/server/model_manager.cpp` | Model registry, downloads, recipe resolution |\n| `src/cpp/include/lemon/wrapped_server.h` | Backend abstract base class |\n| `src/cpp/include/lemon/server_capabilities.h` | Backend capability interfaces |\n| `src/cpp/resources/server_models.json` | Model registry |\n| `src/cpp/resources/backend_versions.json` | Backend version pins |\n| `docs/tools/gen_backend_boilerplate.py` | Regenerates committed artifacts from the C++ backend descriptors. Outputs: the whole of `src/cpp/resources/defaults.json` (per-recipe sections only; global keys stay hand-maintained in that file), and `<!-- BEGIN/END GENERATED -->` regions in `docs/dev/backends-reference.md`, root `README.md`, `docs/guide/cli.md`, `docs/guide/configuration/{README,multi-model,custom-models}.md`, and `docs/assets/models.js`. Don't hand-edit those regions/sections; CI runs `--check` and fails on drift. |\n| `src/cpp/server/anthropic_api.cpp` | Anthropic API compatibility |\n| `src/cpp/server/ollama_api.cpp` | Ollama API compatibility |\n| `src/cpp/server/mcp_server.cpp` | MCP gateway (POST /mcp) |\n| `src/cpp/include/lemon/websocket_server.h` | WebSocket Realtime API server |\n| `src/cpp/include/lemon/model_types.h` | Model type and device type enums |\n| `src/cpp/include/lemon/config_file.h` | config.json load/save/migrate |\n| `src/cpp/include/lemon/recipe_options.h` | Per-recipe JSON configuration |\n| `src/cpp/tray/tray_app.cpp` | Tray application UI and logic |\n| `src/app/src/renderer/ModelManager.tsx` | Model management UI |\n| `src/app/src/renderer/ChatWindow.tsx` | Chat interface |\n\n## Critical Invariants\n\nThese MUST be maintained in all changes:\n\n1. **Quad-prefix registration** — Every new endpoint MUST be registered under `/api/v0/`, `/api/v1/`, `/v0/`, AND `/v1/`. Documented exceptions: Ollama (`/api/*` without version prefix), Anthropic (`POST /v1/messages` only), and MCP (`POST /mcp`) — each of those protocols mandates a fixed URL shape that conflicts with the quad-prefix scheme.\n2. **NPU exclusivity** — Exclusive-NPU recipes (`ryzenai-llm`, `whispercpp` on NPU) evict ALL other NPU models before loading. FastFlowLM (`flm`) can coexist with other FLM types (max 1 per FLM type) but not with exclusive-NPU recipes.\n3. **WrappedServer contract** — New backends MUST implement all core virtual methods: `load()`, `unload()`, `chat_completion()`, `completion()`, `responses()`.\n4. **Subprocess model** — Backends run as subprocesses (llama-server, whisper-server, sd-server, koko, flm, ryzenai-server, moonshine-server). They must NOT run in-process.\n5. **Recipe integrity** — Changes to `server_models.json` must have valid recipes referencing backends in `backend_versions.json`. When adding or updating `vllm` models, also update `src/cpp/resources/vllm_model_config.json` if the model family needs vLLM-specific args such as tool-call parser settings.\n6. **Cross-platform** — Code must compile on Windows (MSVC), Linux (GCC/Clang), macOS (AppleClang). Platform-specific code must use `#ifdef` guards.\n7. **No hardcoded paths** — Use path utilities. Windows/Linux/macOS paths differ.\n8. **Thread safety** — Router serves concurrent HTTP requests. Shared state must be properly guarded.\n9. **Ollama compatibility** — Changes to model listing or management must not break `/api/*` Ollama endpoints.\n10. **API key passthrough** — When `LEMONADE_API_KEY` is set, all API routes must enforce authentication.\n11. **Many-clients-one-server topology** — A single `lemond` can be driven by multiple desktop/tray/CLI clients, potentially on different machines. Per-client UI state (layout, zoom, view selection, the client's own base URL and API key) MUST live locally in the client, never in `lemond`. Do not move `app_settings.json` behind an HTTP endpoint. **Shared infrastructure config** (cloud provider URLs, backend version pins) lives in `lemond`'s `config.json` so it's visible to every client and to the CLI. **Cloud API keys** specifically MUST NOT be written to disk: they live in `LEMONADE_<PROVIDER>_API_KEY` env vars (persistent) or in `lemond`'s process memory via `POST /v1/cloud/auth` (ephemeral, dies on restart).\n12. **Web-app dependencies constrained by Debian native packaging** — `src/web-app/package.json` is kept separate from `src/app/package.json` because the native Debian package (`lemonade-server` .deb) must build using only npm modules available in Debian's `/usr/share/nodejs` (see `USE_SYSTEM_NODEJS_MODULES` in `src/web-app/webpack.config.js`). The old Electron app depended on packages Debian does not ship. Do NOT consolidate the two `package.json` files — the split is required for reproducible distro packaging.\n13. **Desktop app is on-demand; `lemond` runs independently** — On Windows, `LemonadeServer.exe` (which embeds `lemond` + tray icon) is the always-on process, auto-started via the Windows startup folder. The Tauri desktop app (`lemonade-app.exe`) is opened on demand when the user wants the UI and must not be added to startup. The desktop app must not embed or manage `lemond`'s lifecycle — it discovers the already-running server (UDP beacon for local, explicit base URL for remote) and speaks to it over HTTP.\n\n## Contributing\n\n- Open an Issue before submitting major PRs\n- UI/frontend changes are handled by core maintainers only\n- Python formatting with Black is required\n- PRs trigger CI for linting, formatting, and integration tests\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"# AGENTS.md\n\nThis file provides guidance to agent driven code reviews when working with this repository.\n\n## Project Overview\n\nLemonade is a local LLM server providing GPU and NPU acceleration for running large language models on consumer hardware. It exposes OpenAI-compatible, Ollama-compatible, and Anthropic-compatible REST APIs, plus a WebSocket Realtime API. It supports multiple backends: llama.cpp, FastFlowLM, RyzenAI, whisper.cpp, stable-diffusion.cpp, Kokoro TTS, and Moonshine.\n\n## Architecture\n\n### Executables\n\n- **lemond** — Pure HTTP server. Handles REST API, routes requests to backends, manages model loading/unloading. Configured via `config.json` in the lemonade cache directory. CLI args: `[cache_dir] [--port PORT] [--host HOST]`.\n- **lemonade** — CLI client (`src/cpp/cli/`). Commands: `list`, `pull`, `delete`, `run`, `status`, `logs`, `launch`, `backends`, `scan`, etc. Communicates with router via HTTP. Discovers running server via UDP beacon.\n- **LemonadeServer.exe** (Windows) — SUBSYSTEM:WINDOWS GUI app that embeds `lemond` and shows a system tray icon. Auto-starts via Windows startup folder.\n- **lemonade-tray** (macOS/Linux) — Lightweight tray client that connects to a running `lemond`. Platform code in `src/cpp/tray/platform/`.\n\n### Backend Abstraction\n\n`WrappedServer` (`src/cpp/include/lemon/wrapped_server.h`) is the abstract base class. Each backend inherits it and implements `load()`, `unload()`, `chat_completion()`, `completion()`, `responses()`, and optionally `install()` / `download_model()`. Backends run as **subprocesses** — Lemonade forwards HTTP requests to them.\n\n| Backend | Class | Capabilities | Device | Purpose |\n|---------|-------|-------------|--------|---------|\n| llama.cpp | `LlamaCppServer` | Completion, Embeddings, Reranking | GPU | LLM inference — CPU/GPU (Vulkan, ROCm, Metal) |\n| FastFlowLM | `FastFlowLMServer` | Completion, Embeddings, Audio | NPU | NPU inference (multi-modal: LLM, ASR, embeddings) |\n| RyzenAI | `RyzenAIServer` | Completion | NPU | Hybrid NPU inference |\n| vLLM | `VLLMServer` | Completion | GPU | LLM inference — ROCm on AMD iGPU/dGPU (Linux). **Experimental**, validated only on gfx1151 (Strix Halo). |\n| whisper.cpp | `WhisperServer` | Audio | CPU | Audio transcription |\n| stable-diffusion.cpp | `SdServer` | Image | CPU | Image generation, editing, variations |\n| Kokoro | `KokoroServer` | TTS | CPU | Text-to-speech |\n| Moonshine | `MoonshineServer` | Audio | CPU | Streaming speech-to-text (ONNX-based) |\n\nCapability interfaces: `ICompletionServer`, `IEmbeddingsServer`, `IRerankingServer`, `ITranscriptionServer`, `IImageServer`, `ITextToSpeechServer` (defined in `server_capabilities.h`). Use `supports_capability<T>(server)` template for runtime checks.\n\n### Router & Multi-Model Support\n\n`Router` (`src/cpp/server/router.cpp`) manages a vector of `WrappedServer` instances. Routes requests based on model recipe, maintains LRU caches per model type (LLM, embedding, reranking, audio, image, TTS — see `model_types.h`), and enforces NPU exclusivity. Configurable via `--max-loaded-models`. On non-file-not-found errors, the router uses a \"nuclear option\" — evicts all models and retries the load.\n\n### Model Manager & Recipe System\n\n`ModelManager` (`src/cpp/server/model_manager.cpp`) loads the registry from `src/cpp/resources/server_models.json`. Each model has \"recipes\" defining which backend and config to use. Backend versions are pinned in `src/cpp/resources/backend_versions.json`. Models download from Hugging Face.\n\n### API Routes\n\nAll core endpoints are registered under **4 path prefixes**:\n- `/api/v0/` — Legacy\n- `/api/v1/` — Current\n- `/v0/` — Legacy short\n- `/v1/` — OpenAI SDK / LiteLLM compatibility\n\n**Core endpoints:** `chat/completions`, `completions`, `embeddings`, `reranking`, `models`, `models/{id}`, `health`, `pull`, `pull/variants`, `registry/search`, `load`, `unload`, `delete`, `params`, `install`, `uninstall`, `audio/transcriptions`, `audio/speech`, `images/generations`, `images/edits`, `images/variations`, `responses`, `stats`, `system-info`, `system-stats`, `log-level`, `logs/stream`, `jobs`, `jobs/{id}`, `jobs/{id}/pause`, `jobs/{id}/interrupt`, `jobs/{id}/resume`\n\n**Job engine** (`POST jobs`, `GET jobs`, `GET/DELETE jobs/{id}`, `POST jobs/{id}/{pause,interrupt,resume}`): server-side sequences of ops (`system_info`, `system_stats`, `models`, `sleep`, `load`, `unload`, `chat`) with data passing, forward-only branching, and a pause/interrupt/resume lifecycle persisted across restart. Exclusive ops hold a Router slot so normal traffic queues. See `docs/dev/job-system.md` and `docs/dev/job-expression-language.md`.\n\n**Ollama-compatible endpoints** (under `/api/` without version prefix): `chat`, `generate`, `tags`, `show`, `delete`, `pull`, `embed`, `embeddings`, `ps`, `version`\n\n**Anthropic-compatible endpoint:** `POST /api/messages` — supports message completion, tool use, and SSE streaming.\n\n**MCP gateway endpoint:** `POST /mcp` — Model Context Protocol (Streamable HTTP transport, spec `2025-06-18`). Single JSON-RPC 2.0 endpoint exposing 5 tools (`lemonade_list_models`, `lemonade_chat`, `lemonade_transcribe_audio`, `lemonade_generate_image`, `lemonade_omni`). GET returns 405.\n\n**WebSocket Realtime API**: OpenAI-compatible Realtime protocol for real-time audio transcription. `/realtime` and `/logs/stream` accept WebSocket upgrades directly on the main HTTP port; a dedicated listener on an OS-assigned port (9000+, exposed via the `websocket_port` field in the `/health` response) also remains for backward compatibility.\n\n**Internal endpoints:** `POST /internal/shutdown`, `GET /internal/aliases`, `POST /internal/aliases`, `DELETE /internal/aliases/{alias}`\n\nOptional API key auth via `LEMONADE_API_KEY` env var (regular API endpoints) or `LEMONADE_ADMIN_API_KEY` env var (full access including internal endpoints). Clients prefer `LEMONADE_ADMIN_API_KEY` if set. CORS enabled on all routes.\n\n### Desktop & Web App\n\n- **Tauri app** — React 19 + TypeScript in `src/app/`, Rust host in `src/app/src-tauri/`. Uses native OS webview (WebView2 on Windows, WKWebView on macOS, webkit2gtk on Linux). Pure CSS (dark theme), context-based state. Key components: `ChatWindow.tsx`, `ModelManager.tsx`, `DownloadManager.tsx`, `BackendManager.tsx`. Feature panels: LLMChat, ImageGeneration, Transcription, TTS, Embedding, Reranking. The renderer keeps its `window.api` contract via `src/app/src/renderer/tauriShim.ts`, which maps each call to a Tauri `invoke()` or event `listen()`.\n- **Web app** — Browser-only version in `src/web-app/`. Reuses the shared renderer from `src/app/src/` via webpack's `entry`/`template` paths (no OS symlinks); the `BuildWebApp.cmake` script stages both trees side-by-side under `build/web-app-staging/` for the actual webpack build. Built via CMake `BUILD_WEB_APP=ON`. Served at `/app`. A mock `window.api` is injected by the C++ server (`src/cpp/server/server.cpp`) so the shared renderer works unchanged in the browser.\n\n### Key Dependencies\n\n**C++ (FetchContent):** cpp-httplib, nlohmann/json, CLI11, libcurl, zstd, libwebsockets, brotli (macOS). Platform SSL: Schannel (Windows), SecureTransport (macOS), OpenSSL (Linux).\n\n**Desktop app:** Tauri v2 (Rust), React 19, TypeScript 5.3, Webpack 5, markdown-it, highlight.js, katex. Rust crates: `tauri`, `tauri-plugin-{opener,clipboard-manager,single-instance,deep-link}`, `tokio`, `reqwest`, `serde`.\n\n## Build Commands\n\nCMakeLists.txt is at the repository root. Build uses CMake presets — run the setup script first, then build with `--preset`.\n\n```bash\n# 1. Setup (configures build directory and installs deps)\n./setup.sh          # Linux / macOS\n./setup.ps1         # Windows (PowerShell)\n\n# 2. Build C++ server\ncmake --build --preset default          # Linux / macOS (Ninja)\ncmake --build --preset windows          # Windows (Visual Studio 2022)\ncmake --build --preset vs18             # Windows (Visual Studio 2026)\n\n# 3. Tauri desktop app (optional, requires Node.js 20+ and Rust via rustup)\ncmake --build --preset default --target tauri-app    # Linux / macOS\ncmake --build --preset windows --target tauri-app    # Windows (VS 2022)\ncmake --build --preset vs18 --target tauri-app       # Windows (VS 2026)\n\n# 4. Web app (auto-built on all platforms)\ncmake --build --preset default --target web-app         # Linux / macOS\ncmake --build --preset windows --target web-app         # Windows\n\n# 5. Windows MSI installer (WiX 5.0+ required)\ncmake --build --preset windows --target wix_installer_minimal  # server + web-app\ncmake --build --preset windows --target wix_installer_full     # server + Tauri app + web-app\n\n# 6. macOS signed installer\ncmake --build --preset default --target package-macos\n\n# 7. Linux .deb / .rpm\ncd build && cpack            # .deb\ncd build && cpack -G RPM     # .rpm\n```\n\nCMake presets: `default` (Ninja, Release), `windows` (VS 2022), `vs18` (VS 2026), `debug` (Ninja, Debug).\n\nCMake options: `BUILD_WEB_APP` (ON by default on all platforms), `BUILD_TAURI_APP` (Linux only, include Tauri desktop app in deb), `LEMONADE_SYSTEMD_UNIT_NAME` (default: `lemond.service`).\n\n## Testing\n\nIntegration tests in Python against a live server. Tests auto-discover the `lemonade` CLI binary from the build directory; use `--cli-binary` to override.\n\n```bash\npip install -r test/requirements.txt\n\n# CLI tests (no inference backend needed)\npython test/server_cli2.py\n\n# Endpoint tests (no inference backend needed)\npython test/server_endpoints.py\n\n# LLM tests (specify wrapped server and backend)\npython test/server_llm.py --wrapped-server llamacpp --backend vulkan\n\n# Audio transcription tests\npython test/server_whisper.py\n\n# Image generation tests (slow)\npython test/server_sd.py\n```\n\nTest utilities in `test/utils/` with `server_base.py` as the base class. Test dependencies include `requests`, `httpx`, `openai`, `huggingface_hub`, `psutil`, `numpy`, `websockets`, and `ollama`.\n\n### C++ unit tests\n\nC++ unit tests live in `test/cpp/` and are wired up in the root `CMakeLists.txt`. The packaging workflow builds the `cpp-ci-tests` aggregate target and runs `ctest -L cpp-ci`, so a test only runs in CI if it is both labeled `cpp-ci` **and** a dependency of that aggregate target.\n\n**Direct `add_test()` is disabled** (the built-in is overridden to fail with a fatal error just before the test section). Every test MUST be declared with the `add_cpp_ci_test()` helper, which forces an explicit `CI <ON|OFF>` decision at the call site so a test is never silently omitted from — or accidentally added to — CI.\n\n**The enclosing `if()` MUST test `BUILD_TESTING`.** Distro packaging (`contrib/debian/rules`, the RPM job) configures with `BUILD_TESTING=OFF` so it does not build ~45 test binaries it then discards; calling `add_cpp_ci_test()` in that configuration is a fatal error rather than a silent return to the slow build.\n\n```cmake\nif(BUILD_TESTING AND EXISTS \"${CMAKE_CURRENT_SOURCE_DIR}/test/cpp/test_my_feature.cpp\")\n    add_executable(test_my_feature test/cpp/test_my_feature.cpp ...)\n    # ...target_include_directories / target_link_libraries...\n\n    include(CTest)\n    add_cpp_ci_test(MyFeatureTest CI ON COMMAND test_my_feature)\nendif()\n```\n\n```cmake\nadd_cpp_ci_test(<TestName>\n                CI <ON|OFF>                 # required — run under `ctest -L cpp-ci`?\n                COMMAND <command> [args...] # required — what CTest runs\n                [DEPENDS <target>...])      # CI build deps; defaults to the\n                                            # first COMMAND token (the test target)\n```\n\n- `CI ON` labels the test `cpp-ci` and makes its build target(s) a dependency of `cpp-ci-tests`.\n- `CI OFF` still creates the CTest test (for local/other runs) but keeps it out of packaging CI. Use this only for tests that are intentionally excluded (e.g. tests that need a backend, are platform-gated, or are slow CMake-configuration tests).\n\nPass `DEPENDS` only when the CI build needs targets beyond the `COMMAND` executable. `add_cpp_ci_test` calls `register_cpp_ci_test()` internally; do not call `add_test()` or `register_cpp_ci_test()` directly.\n\n## Code Style\n\n### Comments & Documentation\n\n**Default to writing no comments.** Only add a comment when the WHY is non-obvious: a hidden constraint, a subtle invariant, a workaround for a specific bug, or behavior that would surprise a reader. If removing the comment wouldn't confuse a future reader, don't write it.\n\n**Never write comments that explain WHAT the code does** — well-named identifiers already do that. Don't reference the current task, fix, or callers (\"used by X\", \"added for the Y flow\", \"handles the case from issue #123\") — those belong in the PR description and rot as the codebase evolves.\n\n**PR descriptions should be concise.** 1-3 sentences for the summary. No essays. The diff shows what changed; the description explains why and any non-obvious context. Bullet points over paragraphs.\n\n### C++\n- C++17, `lemon::` namespace\n- `snake_case` for functions/variables, `CamelCase` for classes/types\n- 4-space indent, `#pragma once` for headers\n- Keep `#include` directives in alphabetical order within each include block\n- Platform guards: `#ifdef _WIN32`, `#ifdef __APPLE__`, `#ifdef __linux__`\n\n### Python\n- **Black** formatting (v26.1.0, enforced in CI)\n- Pylint with `.pylintrc`\n- Pre-commit hooks: trailing-whitespace, end-of-file-fixer, check-yaml, check-added-large-files\n\n### TypeScript/React\n- React 19, pure CSS (dark theme), context-based state\n- UI/frontend changes are handled by core maintainers only\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `CMakeLists.txt` | Root build config (version, deps, targets) |\n| `src/cpp/server/server.cpp` | HTTP route registration and all handlers |\n| `src/cpp/server/router.cpp` | Request routing and multi-model orchestration |\n| `src/cpp/server/model_manager.cpp` | Model registry, downloads, recipe resolution |\n| `src/cpp/include/lemon/wrapped_server.h` | Backend abstract base class |\n| `src/cpp/include/lemon/server_capabilities.h` | Backend capability interfaces |\n| `src/cpp/resources/server_models.json` | Model registry |\n| `src/cpp/resources/backend_versions.json` | Backend version pins |\n| `docs/tools/gen_backend_boilerplate.py` | Regenerates committed artifacts from the C++ backend descriptors. Outputs: the whole of `src/cpp/resources/defaults.json` (per-recipe sections only; global keys stay hand-maintained in that file), and `<!-- BEGIN/END GENERATED -->` regions in `docs/dev/backends-reference.md`, root `README.md`, `docs/guide/cli.md`, `docs/guide/configuration/{README,multi-model,custom-models}.md`, and `docs/assets/models.js`. Don't hand-edit those regions/sections; CI runs `--check` and fails on drift. |\n| `src/cpp/server/anthropic_api.cpp` | Anthropic API compatibility |\n| `src/cpp/server/ollama_api.cpp` | Ollama API compatibility |\n| `src/cpp/server/mcp_server.cpp` | MCP gateway (POST /mcp) |\n| `src/cpp/include/lemon/websocket_server.h` | WebSocket Realtime API server |\n| `src/cpp/include/lemon/model_types.h` | Model type and device type enums |\n| `src/cpp/include/lemon/config_file.h` | config.json load/save/migrate |\n| `src/cpp/include/lemon/recipe_options.h` | Per-recipe JSON configuration |\n| `src/cpp/tray/tray_app.cpp` | Tray application UI and logic |\n| `src/app/src/renderer/ModelManager.tsx` | Model management UI |\n| `src/app/src/renderer/ChatWindow.tsx` | Chat interface |\n\n## Critical Invariants\n\nThese MUST be maintained in all changes:\n\n1. **Quad-prefix registration** — Every new endpoint MUST be registered under `/api/v0/`, `/api/v1/`, `/v0/`, AND `/v1/`. Documented exceptions: Ollama (`/api/*` without version prefix), Anthropic (`POST /v1/messages` only), and MCP (`POST /mcp`) — each of those protocols mandates a fixed URL shape that conflicts with the quad-prefix scheme.\n2. **NPU exclusivity** — Exclusive-NPU recipes (`ryzenai-llm`, `whispercpp` on NPU) evict ALL other NPU models before loading. FastFlowLM (`flm`) can coexist with other FLM types (max 1 per FLM type) but not with exclusive-NPU recipes.\n3. **WrappedServer contract** — New backends MUST implement all core virtual methods: `load()`, `unload()`, `chat_completion()`, `completion()`, `responses()`.\n4. **Subprocess model** — Backends run as subprocesses (llama-server, whisper-server, sd-server, koko, flm, ryzenai-server, moonshine-server). They must NOT run in-process.\n5. **Recipe integrity** — Changes to `server_models.json` must have valid recipes referencing backends in `backend_versions.json`. When adding or updating `vllm` models, also update `src/cpp/resources/vllm_model_config.json` if the model family needs vLLM-specific args such as tool-call parser settings.\n6. **Cross-platform** — Code must compile on Windows (MSVC), Linux (GCC/Clang), macOS (AppleClang). Platform-specific code must use `#ifdef` guards.\n7. **No hardcoded paths** — Use path utilities. Windows/Linux/macOS paths differ.\n8. **Thread safety** — Router serves concurrent HTTP requests. Shared state must be properly guarded.\n9. **Ollama compatibility** — Changes to model listing or management must not break `/api/*` Ollama endpoints.\n10. **API key passthrough** — When `LEMONADE_API_KEY` is set, all API routes must enforce authentication.\n11. **Many-clients-one-server topology** — A single `lemond` can be driven by multiple desktop/tray/CLI clients, potentially on different machines. Per-client UI state (layout, zoom, view selection, the client's own base URL and API key) MUST live locally in the client, never in `lemond`. Do not move `app_settings.json` behind an HTTP endpoint. **Shared infrastructure config** (cloud provider URLs, backend version pins) lives in `lemond`'s `config.json` so it's visible to every client and to the CLI. **Cloud API keys** specifically MUST NOT be written to disk: they live in `LEMONADE_<PROVIDER>_API_KEY` env vars (persistent) or in `lemond`'s process memory via `POST /v1/cloud/auth` (ephemeral, dies on restart).\n12. **Web-app dependencies constrained by Debian native packaging** — `src/web-app/package.json` is kept separate from `src/app/package.json` because the native Debian package (`lemonade-server` .deb) must build using only npm modules available in Debian's `/usr/share/nodejs` (see `USE_SYSTEM_NODEJS_MODULES` in `src/web-app/webpack.config.js`). The old Electron app depended on packages Debian does not ship. Do NOT consolidate the two `package.json` files — the split is required for reproducible distro packaging.\n13. **Desktop app is on-demand; `lemond` runs independently** — On Windows, `LemonadeServer.exe` (which embeds `lemond` + tray icon) is the always-on process, auto-started via the Windows startup folder. The Tauri desktop app (`lemonade-app.exe`) is opened on demand when the user wants the UI and must not be added to startup. The desktop app must not embed or manage `lemond`'s lifecycle — it discovers the already-running server (UDP beacon for local, explicit base URL for remote) and speaks to it over HTTP.\n\n## Contributing\n\n- Open an Issue before submitting major PRs\n- UI/frontend changes are handled by core maintainers only\n- Python formatting with Black is required\n- PRs trigger CI for linting, formatting, and integration tests\n","category":"root","tokens":4811}]}