lemonade Documentation

File: README.md Copy Chapter

🍋 Lemonade: Refreshingly fast local AI Copy Chapter

[Download](https://lemonade-server.ai/docs/guide/install/) | [Documentation](https://lemonade-server.ai/docs/) | [Discord](https://discord.gg/5xXzkMu8Zk)

Lemonade is the local AI server that gives you the same capabilities as cloud APIs, except 100% free and private. Use the latest models for chat, coding, speech, and image generation on your own NPU and GPU.

Lemonade comes in two flavors:

• Lemonade Server installs a service you can connect to hundreds of great apps using standard OpenAI, Anthropic, and Ollama APIs.
• Embeddable Lemonade is a portable binary you can package into your own application to give it multi-modal local AI that auto-optimizes for your user’s PC.

This project is built by the community for every PC, with optimizations by AMD engineers to get the most from Ryzen AI, Radeon, and Strix Halo PCs.

Getting Started Copy Chapter

1. Install: Windows · Linux · macOS · Docker · Source
2. Get Models: Browse and download with the Model Manager
3. Generate: Try models with the built-in interfaces for chat, image gen, speech gen, and more
4. Mobile: Take your lemonade to go: iOS · Android · Source
5. Connect: Use Lemonade with your favorite apps:

Supported Platforms Copy Chapter

Using the CLI Copy Chapter

To run and chat with Gemma:

lemonade run Gemma-4-E2B-it-GGUF

To code with Lemonade models:

lemonade launch claude

Multi-modality:

# image gen
lemonade run SDXL-Turbo

# speech gen
lemonade run kokoro-v1

# transcription
lemonade run Whisper-Large-v3-Turbo

To see available models and download them:

lemonade list

lemonade pull Gemma-4-E2B-it-GGUF

To manage model aliases for environment-independent naming and active-standby failover:

lemonade alias add production-llm Gemma-4-E2B-it-GGUF
lemonade alias list

# Instant active-standby failover to a different model target
lemonade alias add production-llm Qwen3-0.6B-GGUF
lemonade alias remove production-llm

To see the backends available on your PC:

lemonade backends

For hybrid setups, Lemonade can also route to any OpenAI-compatible cloud provider (Fireworks, OpenAI, OpenRouter, Together, …) alongside local models — see Cloud Offload. (Experimental.)

Model Library Copy Chapter

Lemonade supports a wide variety of LLMs (GGUF, FLM, and ONNX), whisper, stable diffusion, etc. models across CPU, GPU, and NPU.

Use lemonade pull or the built-in Model Manager to download models. Custom GGUF/ONNX models can be pulled from Hugging Face or ModelScope, with their source retained for future updates.

Browse all built-in models →

Supported Configurations Copy Chapter

Lemonade supports multiple inference engines for LLM, speech, TTS, and image generation, and each has its own backend and hardware requirements.

To check exactly which recipes/backends are supported on your own machine, run:

lemonade backends

** See supported AMD ROCm platforms*

*** See supported NVIDIA CUDA platforms*

Project Roadmap Copy Chapter

Lemonade's roadmap is defined by a set of working groups. Visit the landing page here to learn each group's goal and roadmap.

Integrate Embeddable Lemonade in Your Application Copy Chapter

Embeddable Lemonade is a binary version of Lemonade that you can bundle into your own app to give it a portable, auto-optimizing, multi-modal local AI stack. This lets users focus on your app, with zero Lemonade installers, branding, or telemetry.

Check out the Embeddable Lemonade guide.

Connect Lemonade Server to Your Application Copy Chapter

You can use any OpenAI-compatible client library by configuring it to use http://localhost:13305/v1 as the base URL. A table containing official and popular OpenAI clients on different languages is shown below.

Feel free to pick and choose your preferred language.

Python Client Example Copy Section

from openai import OpenAI

# Initialize the client to use Lemonade Server
client = OpenAI(
    base_url="http://localhost:13305/api/v1",
    api_key="lemonade"  # required but unused
)

# Create a chat completion
completion = client.chat.completions.create(
    model="Gemma-4-E2B-it-GGUF",  # or any other available model
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

# Print the response
print(completion.choices[0].message.content)

Click to learn more about the available APIs and how to embed Lemonade in your own application.

FAQ Copy Chapter

To read our frequently asked questions, see our FAQ Guide

Contributing Copy Chapter

Lemonade is built by the local AI community! If you would like to contribute to this project, please check out our contribution guide.

Maintainers Copy Chapter

This is a community project maintained by @amd-pworfolk @bitgamma @danielholanda @jeremyfowers @kenvandine @Geramy @ramkrishna2910 @sawansri @siavashhub @sofiageo @superm1 @vgodsoe, and sponsored by AMD. You can reach us by filing an issue, emailing [email protected], or joining our Discord.

Code Signing Policy Copy Chapter

Free code signing provided by SignPath.io, certificate by SignPath Foundation.

• Committers and reviewers: Maintainers of this repo
• Approvers: Owners

Privacy policy: This program will not transfer any information to other networked systems unless specifically requested by the user or the person installing or operating it. When the user requests a model download or registry lookup, Lemonade may contact Hugging Face Hub (see their privacy policy) or ModelScope, according to the model source selected by the user or packager.

License and Attribution Copy Chapter

This project is:

• Built with C++ (server) and React (app) with ❤️ for the open source community,
• Standing on the shoulders of great tools from:
  • ggml/llama.cpp
  • ggml/whisper.cpp
  • ggml/stable-diffusion.cpp
  • kokoros
  • OnnxRuntime GenAI
  • Hugging Face Hub
  • ModelScope
  • OpenAI API
  • IRON/MLIR-AIE
  • and more...
• Licensed under the Apache 2.0 License.
  • Portions of the project are licensed as described in LICENSE.

File: contrib/launchpad-downloads/README.md Copy Chapter

Launchpad PPA Download Statistics Tool

A Python tool to retrieve download statistics for packages in Launchpad PPAs (Personal Package Archives).

Installation Copy Chapter

1. Install dependencies:

pip install -r requirements.txt

Or install directly:

pip install launchpadlib

Usage Copy Chapter

Basic usage:

python ppa_stats.py username/ppa-name

Examples Copy Section

Get stats for all packages in a PPA:

python ppa_stats.py developmentseed/mapbox

Get stats for a specific package:

python ppa_stats.py developmentseed/mapbox -p tilemill

Show all versions including those with 0 downloads:

python ppa_stats.py username/ppa-name --all

Show summary grouped by package name:

python ppa_stats.py username/ppa-name --summary

Options Copy Section

• -p, --package PACKAGE - Query a specific package name
• --all - Show all versions including those with 0 downloads
• --summary - Show summary by package name instead of individual versions

How It Works Copy Chapter

The tool uses the Launchpad API via the launchpadlib Python library to:

1. Connect anonymously to Launchpad's production API
2. Retrieve the specified PPA for the given user/team
3. Fetch published binary packages and their download counts
4. Display formatted statistics

API Documentation Copy Chapter

The tool uses Launchpad's public API:

• API endpoint: https://api.launchpad.net/
• No authentication required for read-only access
• Uses the getDownloadCount() method on published binaries

Output Format Copy Chapter

The tool displays:

• Total downloads across all packages/versions
• Individual package versions with download counts
• Architecture information for each binary package
• Sorted by download count (most popular first)

Notes Copy Chapter

• Download statistics are provided by Launchpad and may have some delay
• The tool caches API responses in ~/.launchpadlib/cache/ for performance
• Anonymous access is sufficient for read-only statistics

Sources Copy Chapter

Based on information from:

• Launchpad API Documentation
• PPA Download Statistics Examples
• ppa-stats GitHub Project

File: docs/api/anthropic.md Copy Chapter

Anthropic-Compatible API

Lemonade supports an initial Anthropic Messages compatibility endpoint for applications that call Claude-style APIs.

Current scope focuses on message generation parity for common fields (model, messages, system, max_tokens, temperature, stream, and basic tools). Unsupported or unimplemented Anthropic-specific fields are ignored and surfaced via warning logs/headers.

File: docs/api/lemonade.md Copy Chapter

Lemonade API

We have designed a set of Lemonade-specific endpoints to enable client applications by extending the existing cloud-focused APIs (e.g., OpenAI). These extensions allow for a greater degree of UI/UX responsiveness in native applications by allowing applications to:

• Download models at setup time.
• Pre-load models at UI-loading-time, as opposed to completion-request time.
• Unload models to save memory space.
• Understand system resources and state to make dynamic choices.

`POST /v1/classify` Copy Chapter

Run an encoder text-classifier (PII, prompt-safety, domain, etc.) on an input string and return per-label scores in [0, 1]. The target model must use the onnxruntime recipe. Both sequence-classification (one label set) and token-classification (aggregated span labels) models are supported.

Supported architectures: single-sequence encoder families — BERT, DistilBERT, RoBERTa, XLM-RoBERTa, DeBERTa (v1/v2), ELECTRA, ALBERT, CamemBERT. A stock optimum-cli export onnx directory of one of these works as-is.

A servable model directory is model.onnx + tokenizer.json + config.json. The config.json is always required: it declares the architecture, which is checked against the list above so an unsupported family (e.g. XLNet, which uses different segment/special-token conventions) is rejected at load time rather than served with wrong scores. The output contract (labels, normalization, token budget) is read from that same config; an optional manifest.json overrides it but does not replace the config. Without a manifest, inference assumes single-label softmax; a multi-label (sigmoid) model must declare problem_type: multi_label_classification in its config or ship a manifest.json.

This endpoint provides the classification capability that the router's classifier condition type will consume; the live routing-policy wiring is tracked in #2384.

The endpoint is available at:

• /v1/classify
• /api/v1/classify
• /v0/classify
• /api/v0/classify

Parameters Copy Section

Example request Copy Section

curl -X POST http://localhost:13305/v1/classify   -H "Content-Type: application/json"   -d '{"model": "Phishing-Email-Detection-ONNX", "input": "Please verify your account at http://secure-login.example now."}'

Response format Copy Section

{
  "object": "classification",
  "model": "Phishing-Email-Detection-ONNX",
  "labels": {
    "LABEL_1": 0.982,
    "LABEL_0": 0.011,
    "LABEL_2": 0.005,
    "LABEL_3": 0.002
  }
}

Label names come from the model's id2label — from config.json, or from manifest.json when one is present to override it; some upstream models only declare generic LABEL_<n> names — see the model card for their meaning.

Malformed requests (invalid JSON, missing input/text, non-string fields, non-positive top_k) return 400 with an error object before any model is loaded.

Routing (`collection.router`) Copy Chapter

Naming a registered collection.router model in the model field of a
chat/completions or completions request triggers the routing engine: the
server picks a candidate by the policy's first-matching rule (fail-open to
default_model) and forwards the request to it. No dedicated endpoint or "auto"
model is involved.

The decision is reported on the response:

• Header x-lemonade-route — the matched rule id, or default.
• Request field route_trace: true adds an x_lemonade_route object to the
  response body: { route_to, matched_rule, default_used, outputs, trace[] }
  (route_to is the candidate that answered). For streaming responses it is
  attached to the first SSE event.

See Router Policies for authoring the policy.

`POST /v1/models/check-updates` Copy Chapter

Explicitly checks downloaded Hugging Face-backed models for newer upstream
commits. This is the manual counterpart to the startup update check and works
even when auto_check_model_updates=false.

Full offline mode remains authoritative: when offline=true, this endpoint
returns HTTP 409 and does not make network requests.

Example request Copy Section

curl -X POST http://localhost:13305/v1/models/check-updates

The same action is available from the CLI:

lemonade check-updates

Response format Copy Section

{
  "status": "success",
  "updates_available": 2,
  "models": [
    "Qwen3-4B-GGUF",
    "Whisper-Tiny"
  ]
}

The endpoint is available at:

• /v1/models/check-updates
• /api/v1/models/check-updates
• /v0/models/check-updates
• /api/v0/models/check-updates

`GET /v1/models/{id}/files` Copy Chapter

List resolved local file metadata for a single model. This endpoint is intended for model-detail UIs such as the Files tab. It is per-model inventory, not system or drive storage accounting.

The endpoint is available at:

• /v1/models/{id}/files
• /api/v1/models/{id}/files
• /v0/models/{id}/files
• /api/v0/models/{id}/files

By default, the response does not include absolute filesystem paths. Trusted local clients that need paths for native UI actions can request them explicitly with ?include_paths=true. Absolute paths may reveal local usernames and cache layout, so clients should only request them when that disclosure is acceptable.

Example request Copy Section

curl http://localhost:13305/v1/models/Qwen3-4B/files

Response format Copy Section

{
  "model_id": "Qwen3-4B",
  "files": [
    {
      "name": "model.gguf",
      "role": "main",
      "size_bytes": 123456789,
      "exists": true
    },
    {
      "name": "mmproj.gguf",
      "role": "mmproj",
      "size_bytes": 12345678,
      "exists": true
    }
  ]
}

Optional path disclosure Copy Section

curl 'http://localhost:13305/v1/models/Qwen3-4B/files?include_paths=true'

When include_paths=true is supplied, each file entry also includes path:

{
  "name": "model.gguf",
  "path": "/abs/path/model.gguf",
  "role": "main",
  "size_bytes": 123456789,
  "exists": true
}

Fields Copy Section

`GET /v1/models/{id}/options` Copy Chapter

Read a model's recipe options, separated by layer, without loading it. With POST and DELETE on the same path, this manages per-model options independently of /v1/load.

Example request Copy Section

curl http://localhost:13305/v1/models/Qwen3-0.6B-GGUF/options

Response format Copy Section

effective is the exact request body a POST /v1/load for this model uses right now, with every option the recipe accepts resolved through the full priority chain. defaults is what a reset model would get. For llamacpp, with --no-mmap saved and the context size left automatic:

{
  "model_name": "Qwen3-0.6B-GGUF",
  "recipe": "llamacpp",
  "saved": {
    "llamacpp_args": "--no-mmap"
  },
  "effective": {
    "auto_evict": null,
    "ctx_size": -1,
    "downsize_idle_timeout": 60,
    "evict_idle_timeout": 300,
    "evict_weight_factor": 1.0,
    "llamacpp_args": "--no-mmap",
    "llamacpp_backend": "vulkan",
    "llamacpp_device": "",
    "merge_args": true,
    "model_name": "Qwen3-0.6B-GGUF"
  },
  "defaults": {
    "auto_evict": null,
    "ctx_size": -1,
    "downsize_idle_timeout": 60,
    "evict_idle_timeout": 300,
    "evict_weight_factor": 1.0,
    "llamacpp_args": "",
    "llamacpp_backend": "vulkan",
    "llamacpp_device": "",
    "merge_args": true,
    "model_name": "Qwen3-0.6B-GGUF"
  },
  "resolved_ctx_size": 32768,
  "load_command": "curl -X POST $LEMONADE_BASE_URL/v1/load -H \"Content-Type: application/json\" -d '{\"ctx_size\":-1, ...}'"
}

load_command is effective posted to /v1/load: the exact load the model would get right now, as a command.

The base URL is left as $LEMONADE_BASE_URL, and the API key, when the server requires one, as $LEMONADE_API_KEY. The server cannot fill either in: it always listens over plain HTTP, so a client that reached it through a TLS-terminating proxy is on https without the server ever knowing, and the Host header is only the caller's own claim about where it sent the request. Export the variables to run the command as-is:

export LEMONADE_BASE_URL=http://localhost:13305

A client that already knows its base URL — every GUI and CLI does — can substitute the two tokens before displaying the command.

The quoting is for POSIX shells: bash, zsh, and PowerShell 7.3 and newer, which is where PowerShell began passing arguments to native commands unchanged. cmd.exe does not treat ' as a quote character at all. In either of those, build the request from effective rather than pasting the command.

Note: per-architecture defaults come from the model's GGUF metadata. For a model that has not been downloaded yet, every key is still present but carries the value it has before those defaults apply.

`POST /v1/models/{id}/options` Copy Chapter

Save recipe options for a model without loading it. The request body is a flat object of the same recipe options /v1/load accepts. The URL identifies the model; a model_name in the body is ignored.

The request merges into the model's saved entry, so keys you don't mention are left alone. null removes an option, and the model falls back to the next layer of the priority chain. DELETE removes every saved option at once.

dry_run: true validates and resolves the request identically but persists nothing: effective, resolved_ctx_size, and load_command describe the state the save would produce, while saved keeps reporting the entry on disk. Use it to preview a change before committing it.

ctx_size takes a positive whole number, or -1 to pin the model to automatic sizing even when the server-wide ctx_size is a specific number.

A 400 reports an unrecognized option name, an option from a different recipe, a value of the wrong type, or an invalid ctx_size, and nothing from that request is saved.

Saving never loads or reloads the model, so a model that is already running keeps its current options until it is next loaded.

Note: pinned is not settable here and is omitted from effective and defaults. It belongs to /v1/load and /internal/pin.

Example requests Copy Section

Save a context size without loading the model:

curl -X POST http://localhost:13305/v1/models/Qwen3-0.6B-GGUF/options \
  -H "Content-Type: application/json" \
  -d '{"ctx_size": 8192, "llamacpp_backend": "vulkan"}'

Set the context size back to automatic, leaving the backend choice saved:

curl -X POST http://localhost:13305/v1/models/Qwen3-0.6B-GGUF/options \
  -H "Content-Type: application/json" \
  -d '{"ctx_size": -1}'

Response format Copy Section

Same as GET /v1/models/{id}/options, reflecting the state after the write.

`DELETE /v1/models/{id}/options` Copy Chapter

Reset a model to its defaults by erasing its recipe_options.json entry entirely. The model keeps the defaults that come from its registry entry and from the server's global configuration; only the user's saved overrides are removed.

Example request Copy Section

curl -X DELETE http://localhost:13305/v1/models/Qwen3-0.6B-GGUF/options

Response format Copy Section

Same as GET /v1/models/{id}/options, with saved now {}.

`POST /v1/models/register` Copy Chapter

Register or update a user.* model definition without downloading model files.
Use this endpoint when registration and installation are separate actions.
POST /v1/pull remains the install/download path and performs the same internal
registration step before downloading.

The endpoint is available at:

• /v1/models/register
• /api/v1/models/register
• /v0/models/register
• /api/v0/models/register

Parameters Copy Section

A checkpoint is intentionally not universally required: registration is a model
metadata operation and some present or future model types may not have local
weights. /pull remains the operation that attempts installation/download.

The endpoint accepts one model definition. An embedded models array represents
multiple definitions and remains a collection-import concern; register those
component definitions first when using this endpoint.

Example request:

curl -X POST http://localhost:13305/v1/models/register \\
  -H "Content-Type: application/json" \\
  -d '{
    "model_name": "user.Phi-4-Mini-GGUF",
    "checkpoint": "unsloth/Phi-4-mini-instruct-GGUF:Q4_K_M",
    "recipe": "llamacpp"
  }'

Example response:

{
  "status": "success",
  "model_name": "Phi-4-Mini-GGUF",
  "canonical_model_name": "user.Phi-4-Mini-GGUF",
  "model": {
    "id": "Phi-4-Mini-GGUF",
    "recipe": "llamacpp",
    "downloaded": false
  }
}

model_name is the public ID exposed by /v1/models; canonical_model_name is
the stable user.* registration ID. Registration updates user_models.json and
invalidates the model cache, but does not start a model download.

`POST /v1/pull` Copy Chapter

Register and install models for use with Lemonade Server.

Parameters Copy Section

The Lemonade Server built-in model registry has a collection of model names that can be pulled and loaded. The pull endpoint can install any registered model, and it can also register-then-install any model available on Hugging Face.

Common Parameters

Install a Model that is Already Registered

Example request:

curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF"
  }'

Response format:

{
  "status":"success",
  "message":"Installed model: Qwen3-0.6B-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

Register and Install a Model

Registration will place an entry for that model in the user_models.json file, which is located in the user's Lemonade cache (default: ~/.cache/lemonade). Then, the model will be installed. Once the model is registered and installed, it will show up in the models endpoint alongside the built-in models and can be loaded.

The recipe field defines which software framework and device will be used to load and run the model.

Note: the model_name for registering a new model must use the user namespace, to prevent collisions with built-in models. For example, user.Phi-4-Mini-GGUF.

A model definition requires at least a main checkpoint. This can be either
be specified with the checkpoint parameter, or a main key in the
checkpoints dict.

Each backend serves a fixed set of deployment modes,
and a model deploys in exactly one of them. Naming a mode the recipe cannot
serve, or naming two — whether through labels or through the embedding /
reranking parameters — is rejected with 400 and nothing is registered:

curl -X POST http://localhost:8000/api/v1/pull \
  -H "Content-Type: application/json" \
  -d '{"model_name": "user.Clf", "recipe": "llamacpp",
       "checkpoint": "example/model:Q4_K_M", "labels": ["classification"]}'

{"error": "Model 'user.Clf': recipe 'llamacpp' cannot serve 'classification'. It serves 'chat', 'embeddings', 'reranking'. Omit the label to deploy as 'chat'."}

curl -X POST http://localhost:8000/api/v1/pull \
  -H "Content-Type: application/json" \
  -d '{"model_name": "user.Both", "recipe": "llamacpp",
       "checkpoint": "example/model:Q4_K_M", "labels": ["chat", "embeddings"]}'

{"error": "Model 'user.Both': a model deploys in exactly one mode, but these labels name two: 'chat' and 'embeddings'. Register one model per mode."}

Omitting the deployment label entirely is always valid — the recipe's default is
applied.

Other checkpoint types may also be specified depending on the model type.
This list is not exhaustive, and may change or grow over time as models
and backends evolve:

• mmproj - used by vision models, if not already embedded in main
• draft - used by dflash, eagle, and multitoken-prediction, if not already embedded in main
• text_encoder - text-to-token encoder used by image generation
• vae - variational autoencoder used by image generation

Example request:

# Single checkpoint
curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "user.Phi-4-Mini-GGUF",
    "checkpoint": "unsloth/Phi-4-mini-instruct-GGUF:Q4_K_M",
    "recipe": "llamacpp"
  }'

Instead of defining a model by checkpoint and mmproj, a model can also
be defined with a dict of checkpoint types and paths. These requests do the
same thing, but the syntax for pulling the mmproj differs.

# Multi-checkpoint
curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "user.My-Gemma3",
    "checkpoint": "ggml-org/gemma-3-4b-it-GGUF:Q4_K_M",
    "mmproj": "mmproj-model-f16.gguf",
	"vision": true,
    "recipe": "llamacpp"
  }'

# Multi-checkpoint
curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "user.My-Gemma3",
    "checkpoints": {
        "main": "ggml-org/gemma-3-4b-it-GGUF:Q4_K_M",
        "mmproj": "ggml-org/gemma-3-4b-it-GGUF:mmproj-model-f16.gguf"
    },
	"vision": true,
    "recipe": "llamacpp"
  }'

Response format:

{
  "status":"success",
  "message":"Installed model: user.Phi-4-Mini-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

Register an Omni-Model

An omni collection is a collection type that bundles several models into a single entry that can be loaded, pulled, or deleted as a unit. Use recipe: "collection.omni" with a components array instead of checkpoint.

Components do not need to be downloaded already — any not-yet-downloaded components are pulled by the same call. Deleting the collection removes only the collection entry; components stay on disk.

Example request:

curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "user.MyKit",
    "recipe": "collection.omni",
    "components": ["Qwen3-0.6B-GGUF", "Whisper-Tiny", "SD-Turbo"]
  }'

Import an Exported Model File Copy Section

Files written by lemonade export (and the desktop app's Export button) are import-ready
/v1/pull request bodies — POST the file contents verbatim to register and install the model.
This works for regular models and collections alike; exported collection files additionally
carry components plus a models array embedding each component's definition (see the
models parameter above). For the file format and the export/import/Hugging Face workflows,
see Share a collection.

Streaming Response (stream=true) Copy Section

When stream=true, the endpoint returns Server-Sent Events with real-time download progress:

event: progress
data: {"file":"model.gguf","file_index":1,"total_files":2,"bytes_downloaded":1073741824,"bytes_total":2684354560,"percent":40}

event: progress
data: {"file":"config.json","file_index":2,"total_files":2,"bytes_downloaded":1024,"bytes_total":1024,"percent":100}

event: complete
data: {"file_index":2,"total_files":2,"percent":100}

Event Types:

Server-owned download mode (`stream=true`, `subscribe=false`) Copy Section

By default, stream=true keeps the /v1/pull HTTP response subscribed to Server-Sent Events until the download finishes. Clients that need download state to survive a renderer reload, tab close, or reconnect can also send subscribe=false.

When stream=true and subscribe=false, /v1/pull starts a server-owned model download job and returns a JSON snapshot immediately. The job continues on the server. Clients can poll GET /v1/downloads to restore progress and can use POST /v1/downloads/control to pause, cancel, or remove the job.

Example request:

curl -X POST http://localhost:13305/v1/pull \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF",
    "stream": true,
    "subscribe": false
  }'

Example response:

{
  "id": "model:Qwen3-0.6B-GGUF",
  "type": "model",
  "model_name": "Qwen3-0.6B-GGUF",
  "status": "downloading",
  "running": true,
  "file": "",
  "file_index": 0,
  "total_files": 0,
  "bytes_downloaded": 0,
  "bytes_total": 0,
  "total_download_size": 0,
  "bytes_previously_downloaded": 0,
  "completed_files_bytes": 0,
  "cumulative_bytes_downloaded": 0,
  "overall_bytes_downloaded": 0,
  "percent": 0,
  "complete": false
}

`GET /v1/downloads` Copy Chapter

List server-owned model download jobs that were started with POST /v1/pull using stream=true and subscribe=false.

This endpoint is intended for clients that need to restore download-manager state after a reload or reconnect. Active, paused, cancelled, and errored jobs remain visible until the client removes them. Completed jobs remain visible briefly so clients can observe completion and refresh model state.

Example request Copy Section

curl http://localhost:13305/v1/downloads

Response format Copy Section

[
  {
    "id": "model:Qwen3-0.6B-GGUF",
    "type": "model",
    "model_name": "Qwen3-0.6B-GGUF",
    "status": "downloading",
    "running": true,
    "file": "model.gguf",
    "file_index": 1,
    "total_files": 2,
    "bytes_downloaded": 1073741824,
    "bytes_total": 2684354560,
    "total_download_size": 2684355584,
    "bytes_previously_downloaded": 0,
    "completed_files_bytes": 0,
    "cumulative_bytes_downloaded": 1073741824,
    "overall_bytes_downloaded": 1073741824,
    "percent": 40,
    "complete": false
  }
]

Download job fields Copy Section

`POST /v1/downloads/control` Copy Chapter

Control a server-owned model download job.

Parameters Copy Section

Actions Copy Section

Example request Copy Section

curl -X POST http://localhost:13305/v1/downloads/control \
  -H "Content-Type: application/json" \
  -d '{
    "id": "model:Qwen3-0.6B-GGUF",
    "action": "pause"
  }'

Response format Copy Section

For pause and cancel, the endpoint returns the latest job snapshot:

{
  "id": "model:Qwen3-0.6B-GGUF",
  "type": "model",
  "model_name": "Qwen3-0.6B-GGUF",
  "status": "paused",
  "running": false,
  "file": "model.gguf",
  "file_index": 1,
  "total_files": 2,
  "bytes_downloaded": 1073741824,
  "bytes_total": 2684354560,
  "percent": 40,
  "complete": false
}

For remove, the endpoint returns:

{"status":"ok"}

If the job is already missing and action is remove, the endpoint returns:

{"status":"ok","missing":true}

`GET /v1/registry/search` Copy Chapter

Search a remote model registry (Hugging Face or ModelScope) for repositories matching a text query. This endpoint returns candidate repositories based on registry metadata; it does not verify that a repository contains servable files. The desktop app's Model Manager follows up with /v1/pull/variants on each candidate and only offers a download for repositories whose file listing passes that validation.

Requires network access: returns 400 with code lemond_offline when the server is in offline mode.

Parameters Copy Section

Example request:

curl 'http://localhost:13305/v1/registry/search?source=modelscope&query=qwen&format=gguf'

Response Copy Section

{
  "source": "modelscope",
  "query": "qwen",
  "format": "gguf",
  "total": 128,
  "results": [
    {
      "repository_id": "Qwen/Qwen2.5-3B-Instruct-GGUF",
      "display_name": "Qwen2.5-3B-Instruct-GGUF",
      "source": "modelscope",
      "repository_type": "model",
      "description": "GGUF quantizations of Qwen2.5-3B-Instruct",
      "tags": ["gguf", "chat"],
      "task": "text-generation",
      "downloads": 222500,
      "likes": 12,
      "has_gguf": true
    }
  ]
}

Error responses Copy Section

`GET /v1/pull/variants` Copy Chapter

Inspect a Hugging Face GGUF repository and enumerate the variants (quantizations and sharded folder groups) available for installation. Used by the lemonade pull <owner/repo> CLI flow and by the desktop app's model search to auto-populate the install form. The endpoint reads only public Hugging Face metadata; if the HF_TOKEN environment variable is set on the server, it is forwarded as a bearer token to access gated repositories.

Parameters Copy Section

Example request:

curl 'http://localhost:13305/v1/pull/variants?checkpoint=unsloth/Qwen3-8B-GGUF'

Response Copy Section

{
  "checkpoint": "unsloth/Qwen3-8B-GGUF",
  "recipe": "llamacpp",
  "suggested_name": "Qwen3-8B-GGUF",
  "suggested_labels": ["vision"],
  "mmproj_files": ["mmproj-model-f16.gguf"],
  "variants": [
    {
      "name": "Q4_K_M",
      "primary_file": "Qwen3-8B-Q4_K_M.gguf",
      "files": ["Qwen3-8B-Q4_K_M.gguf"],
      "sharded": false,
      "size_bytes": 4920000000
    },
    {
      "name": "Q8_0",
      "primary_file": "Q8_0/Qwen3-8B-Q8_0-00001-of-00002.gguf",
      "files": ["Q8_0/Qwen3-8B-Q8_0-00001-of-00002.gguf", "Q8_0/Qwen3-8B-Q8_0-00002-of-00002.gguf"],
      "sharded": true,
      "size_bytes": 8500000000
    }
  ]
}

Error responses Copy Section

`POST /v1/delete` Copy Chapter

Delete a model by removing it from local storage. If the model is currently loaded, it will be unloaded first.

Note: deleting a collection (recipe: "collection.omni") removes only the collection entry from user_models.json; its components stay on disk. Delete the components individually if you want to free their disk space.

Parameters Copy Section

Example request:

curl -X POST http://localhost:13305/v1/delete \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF"
  }'

Response format:

{
  "status":"success",
  "message":"Deleted model: Qwen3-0.6B-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

`POST /v1/load` Copy Chapter

Explicitly load a registered model into memory. This is useful to ensure that the model is loaded before you make a request. Installs the model if necessary.

Note: loading a collection (recipe: "collection.omni") loads each of its components in turn. Per-model options like ctx_size or llamacpp_backend are not forwarded to components — set them on each component's own recipe_options.json entry instead.

Recipe option fields on /v1/load have three-state semantics. Omitting a field keeps using its saved per-model value. Passing explicit null ignores only that saved key for this load and falls through to the lower default layers without changing recipe_options.json. Passing a concrete value overrides the saved value. ctx_size: -1 is a concrete value meaning automatic context sizing, not a tombstone. With save_options: true, concrete values are persisted as usual while a null tombstone preserves the existing saved value for that key.

Parameters Copy Section

Setting Priority:

When loading a model, settings are applied in this priority order:

1. Values explicitly passed in the load request (highest priority)
2. Per-model values configurable in recipe_options.json (see below for details)
3. Values from environment variables or server startup arguments (see Server Configuration)
4. Default hardcoded values in lemond (lowest priority)

Per-model options Copy Section

You can configure recipe-specific options on a per-model basis. Lemonade manages a file called recipe_options.json in the user's Lemonade cache (default: ~/.cache/lemonade). The available options depend on the model's recipe:

{
  "user.Qwen2.5-Coder-1.5B-Instruct": {
    "ctx_size": 16384,
    "llamacpp_backend": "vulkan",
    "llamacpp_args": "-np 2 -kvu"
  },
  "Qwen3-Coder-30B-A3B-Instruct-GGUF" : {
    "llamacpp_backend": "rocm"
  },
  "whisper-large-v3-turbo-q8_0.bin": {
    "whispercpp_backend": "npu",
    "whispercpp_args": "--convert"
  }
}

Note that model names include any applicable prefix, such as user. and extra..

Example requests Copy Section

Basic load:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF"
  }'

Load with custom settings:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF",
    "ctx_size": 8192,
    "llamacpp_backend": "rocm",
    "llamacpp_args": "--flash-attn on --no-mmap"
  }'

Load and save settings:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "Qwen3-0.6B-GGUF",
    "ctx_size": 8192,
    "llamacpp_backend": "vulkan",
    "llamacpp_args": "--no-context-shift --no-mmap",
    "save_options": true
  }'

Load a Whisper model with NPU backend and conversion enabled:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "whisper-large-v3-turbo-q8_0.bin",
    "whispercpp_backend": "npu",
    "whispercpp_args": "--convert"
  }'

Load an image generation model with custom settings:

curl -X POST http://localhost:13305/v1/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "sd-turbo",
    "steps": 4,
    "cfg_scale": 1.0,
    "width": 512,
    "height": 512
  }'

Response format Copy Section

{
  "status":"success",
  "message":"Loaded model: Qwen3-0.6B-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

`POST /v1/unload` Copy Chapter

Explicitly unload a model from memory. This is useful to free up memory while still leaving the server process running (which takes minimal resources but a few seconds to start).

Parameters Copy Section

Example requests Copy Section

Unload a specific model:

curl -X POST http://localhost:13305/v1/unload \
  -H "Content-Type: application/json" \
  -d '{"model_name": "Qwen3-0.6B-GGUF"}'

Unload all models:

curl -X POST http://localhost:13305/v1/unload

Response format Copy Section

Success response:

{
  "status": "success",
  "message": "Model unloaded successfully"
}

Error response (model not found):

{
  "status": "error",
  "message": "Model not found: Qwen3-0.6B-GGUF"
}

In case of an error, the status will be error and the message will contain the error message.

`POST /v1/audio/generations` Copy Chapter

Audio Generation API. You provide a text prompt and receive a generated audio clip. The loaded model decides the kind of audio: music with ACE-Step models (e.g. ACE-Step-Music), sound effects with ThinkSound models (e.g. ThinkSound-SFX).

This endpoint is not part of the OpenAI API (OpenAI's audio endpoints cover speech and transcription only), so it is a Lemonade-specific extension.

Performance: generation runs on the GPU (Vulkan, ROCm, or CUDA) and takes from seconds (short sound effects) to minutes (full-length music) depending on duration and hardware.

Parameters Copy Section

Lyrics Copy Section

ACE-Step vocals are a two-stage pipeline inside the backend: a language model first turns the style description and lyrics into audio codes, then the diffusion synthesizer renders those codes into audio. The instrumental path skips the language-model stage entirely, which also means lyrics embedded in the prompt field are treated as style text — they are never sung. Vocal generations take noticeably longer than instrumental ones of the same duration because of the extra language-model pass.

Format the lyrics value the way the ACE-Step authors recommend:

• Mark each song section with a structure tag on its own line: [verse], [chorus], [bridge], [intro], [outro].
• Write one sung phrase per line and separate sections with a blank line.
• Describe the voice ("gentle female vocals", "raspy male baritone") in prompt, not in the lyrics.
• Lyrics may be in any supported language; set vocal_language to match.

Response Copy Section

On success the raw audio bytes are returned with the matching content type (audio/wav). On failure the response is JSON with an error object: 400 for invalid requests, 404 for unknown models, 500 when the backend reports an error, and 502 when the backend produces no output.

Example request Copy Section

curl -X POST http://localhost:13305/v1/audio/generations \
  -H "Content-Type: application/json" \
  -d '{
        "model": "ThinkSound-SFX",
        "prompt": "glass shattering on a stone floor",
        "duration": 5,
        "seed": 42
      }' \
  --output clip.wav

Example request (music with vocals) Copy Section

curl -X POST http://localhost:13305/v1/audio/generations \
  -H "Content-Type: application/json" \
  -d '{
        "model": "ACE-Step-Music",
        "prompt": "warm acoustic folk ballad, fingerpicked guitar, gentle female vocals",
        "lyrics": "[verse]\nMoonlight spills across the floor\nShadows dancing by the door\n\n[chorus]\nWe sing until the morning light\nCarried on the wind tonight",
        "duration": 60
      }' \
  --output song.wav

`POST /v1/3d/generations` Copy Chapter

3D Generation API. You provide an input image and receive a textured 3D mesh as a glTF-binary (.glb) file. Serves TRELLIS models (e.g. TRELLIS-3D). The input image must be PNG, JPEG, BMP, or GIF.

This endpoint is not part of the OpenAI API, so it is a Lemonade-specific extension.

Performance: 3D reconstruction runs on the GPU (Vulkan, ROCm, or CUDA) and takes on the order of minutes; higher cascade resolutions take longer.

Parameters Copy Section

Response Copy Section

On success the raw mesh bytes are returned as model/gltf-binary. On failure the response is JSON with an error object: 400 for invalid requests, 404 for unknown models, 500 when the backend reports an error, and 502 when the backend produces no output.

Example request Copy Section

curl -X POST http://localhost:13305/v1/3d/generations \
  -H "Content-Type: application/json" \
  -d "{
        \"model\": \"TRELLIS-3D\",
        \"image\": \"$(base64 -w0 input.png)\",
        \"resolution\": 512,
        \"seed\": 42
      }" \
  --output model.glb

`GET /v1/health` Copy Chapter

Check the health of the server. This endpoint returns information about loaded models.

Parameters Copy Section

This endpoint does not take any parameters.

Example request Copy Section

curl http://localhost:13305/v1/health

Response format Copy Section

{
  "status": "ok",
  "version":"9.3.3",
  "websocket_port":9000,
  "model_loaded": "Llama-3.2-1B-Instruct-Hybrid",
  "all_models_loaded": [
    {
      "model_name": "Llama-3.2-1B-Instruct-Hybrid",
      "checkpoint": "amd/Llama-3.2-1B-Instruct-awq-g128-int4-asym-fp16-onnx-hybrid",
      "last_use": 1732123456.789,
      "type": "llm",
      "device": "gpu npu",
      "pinned": true,
      "recipe": "ryzenai-llm",
      "pid": 12345,
      "recipe_options": {
        "ctx_size": 4096
      },
      "backend_url": "http://127.0.0.1:8001/v1"
    },
    {
      "model_name": "nomic-embed-text-v1-GGUF",
      "checkpoint": "nomic-ai/nomic-embed-text-v1-GGUF:Q4_K_S",
      "last_use": 1732123450.123,
      "type": "embedding",
      "device": "gpu",
      "pinned": false,
      "recipe": "llamacpp",
      "pid": 12346,
      "recipe_options": {
        "ctx_size": 8192,
        "llamacpp_args": "--no-mmap",
        "llamacpp_backend": "rocm"
      },
      "backend_url": "http://127.0.0.1:8002/v1"
    }
  ],
  "pinned_models": {
    "transcription":0,
    "embedding":0,
    "image":0,
    "llm":1,
    "reranking":0,
    "tts":0
  },
  "max_models": {
    "transcription":1,
    "embedding":1,
    "image":1,
    "llm":1,
    "reranking":1,
    "tts":1
  },
  "telemetry": {
    "enabled": false
  },
  "update_check_done": true
}

Field Descriptions:

• status - Server health status, always "ok"
• version - Version number of Lemonade Server
• model_loaded - Model name of the most recently accessed model
• update_check_done - Whether the background HuggingFace model update check has completed at startup. Poll this field after server start to know when update_available fields are ready.
• all_models_loaded - Array of all currently loaded models with details:
  • model_name - Name of the loaded model
  • checkpoint - Full checkpoint identifier
  • last_use - Unix timestamp of last access (load or inference)
  • type - Model type: "llm", "embedding", "reranking", "transcription", "image", or "tts"
  • device - Space-separated device list: "cpu", "gpu", "npu", or combinations like "gpu npu"
  • pinned - Boolean indicating if the model is currently pinned to prevent auto-eviction
  • is_busy - Boolean indicating if the model has active requests or maintenance in progress
  • is_streaming - Boolean indicating if the model is actively generating output tokens (true after first chunk arrives, false when all streaming requests complete)
  • backend_url - URL of the backend server process handling this model (useful for debugging)
  • pid - The Process ID (PID) of the backend engine handling this model
  • recipe - Backend/device recipe used to load the model (e.g., "ryzenai-llm", "llamacpp", "flm")
  • recipe_options - Options used to load the model (e.g., "ctx_size", "llamacpp_backend", "llamacpp_args", "whispercpp_args")
• pinned_models - Counts of pinned models currently loaded in memory per model type (e.g., llm, embedding, etc.)
• max_models - Maximum number of models that can be loaded simultaneously per type (set via max_loaded_models in Server Configuration):
  • llm - Maximum LLM/chat models
  • embedding - Maximum embedding models
  • reranking - Maximum reranking models
  • transcription - Maximum speech-to-text models
  • image - Maximum image models
  • tts - Maximum text-to-speech models
• websocket_port - (optional) Port of the WebSocket server for the Realtime Audio Transcription API and Log Streaming API. Only present when the WebSocket server is running. The port is OS-assigned or set via --websocket-port.
• telemetry - Structured telemetry state object:
  • enabled - Boolean indicating if telemetry collection is active
  • captures - (optional) Array of captured telemetry components (e.g., ["inputs", "outputs", "thinking"]), only present when enabled is true.

`GET /v1/stats` Copy Chapter

Performance statistics from the last request.

Parameters Copy Section

This endpoint does not take any parameters.

Example request Copy Section

curl http://localhost:13305/v1/stats

Response format Copy Section

{
  "time_to_first_token": 2.14,
  "tokens_per_second": 33.33,
  "input_tokens": 128,
  "output_tokens": 5,
  "prompt_tokens": 9,
  "cache_tokens": 96,
  "request_count_total": 12,
  "input_tokens_total": 1536,
  "output_tokens_total": 60,
  "prompt_tokens_total": 108,
  "cache_tokens_total": 1152,
  "routing_decisions_total": 4,
  "routing_switches_total": 1
}

Field Descriptions:

• time_to_first_token - Time in seconds until the first token was generated
• tokens_per_second - Generation speed in tokens per second
• input_tokens - Number of tokens processed
• output_tokens - Number of tokens generated
• prompt_tokens - Total prompt tokens including cached tokens
• cache_tokens - Prompt tokens served from the backend's prefix cache on the last request (llama.cpp timings.cache_n, or usage.prompt_tokens_details.cached_tokens / Responses-API input_tokens_details.cached_tokens from OpenAI-compatible cloud providers). null when the last request did not report cache usage
• *_total - Cumulative counters since server start
• routing_decisions_total - Routing decisions made by collection.router dispatch
• routing_switches_total - Routing decisions that changed a conversation's routed model (a proxy for route ping-pong; conversations are identified by a hash of the system prompt and first user message)

`GET /v1/system-stats` Copy Chapter

Current host resource usage as measured by the Lemonade Server process. This endpoint is useful for first-party clients and dashboards that need lightweight runtime telemetry without scraping Prometheus.

Parameters Copy Section

This endpoint does not take any parameters.

Example request Copy Section

curl http://localhost:13305/v1/system-stats

Response format Copy Section

{
  "cpu_percent": 12.3,
  "memory_gb": 8.4,
  "gpu_percent": 45.0,
  "vram_gb": 2.1,
  "npu_percent": null
}

Field Descriptions:

• cpu_percent - System CPU utilization percentage, or null when unavailable
• memory_gb - System RAM currently in use, in GiB
• gpu_percent - GPU utilization percentage, or null when unavailable
• vram_gb - GPU memory currently in use, in GiB, or null when unavailable
• npu_percent - NPU utilization percentage, or null when unavailable

GPU, VRAM, and NPU telemetry availability depends on the operating system and installed drivers. Unsupported values are returned as null.

`GET /metrics` Copy Chapter

Prometheus scrape endpoint for Lemonade Server. The endpoint returns Prometheus text exposition format and is intended to be scraped by Prometheus, not by Grafana directly.

Unlike most Lemonade API endpoints, /metrics is root-level only. It is not mounted under /api/v0/, /api/v1/, /v0/, or /v1/.

HEAD /metrics is also supported and returns 200 OK with an empty body.

Authentication Copy Section

If LEMONADE_API_KEY is set, /metrics requires bearer authentication. Either the regular API key or LEMONADE_ADMIN_API_KEY is accepted.

If only LEMONADE_ADMIN_API_KEY is set and LEMONADE_API_KEY is unset, /metrics is accessible without authentication, matching regular API endpoint behavior.

Polling and Refresh Rate Copy Section

The /metrics endpoint has no internal refresh timer. It renders the latest server state at the moment it is scraped.

Polling frequency is configured in Prometheus via scrape_interval, for example:

global:
  scrape_interval: 10s

Grafana queries Prometheus. Grafana's dashboard refresh controls how often panels query Prometheus, but it does not control how often Prometheus scrapes Lemonade.

Example request Copy Section

curl http://localhost:13305/metrics

With API-key auth:

curl http://localhost:13305/metrics \
  -H "Authorization: Bearer $LEMONADE_API_KEY"

Response format Copy Section

The response uses Prometheus text exposition format:

# HELP lemonade_server_up Whether the Lemonade server is running.
# TYPE lemonade_server_up gauge
lemonade_server_up 1
# HELP lemonade_server_info Lemonade server build information.
# TYPE lemonade_server_info gauge
lemonade_server_info{version="10.4.0"} 1

Content type:

text/plain; version=0.0.4; charset=utf-8

Lemonade Metric Families Copy Section

The authoritative metric-family list is generated by the /metrics implementation in src/cpp/server/server.cpp. Search for handle_metrics and metrics.describe(...) to see the current names, types, labels, and descriptions.

Unsupported, unavailable, null, NaN, and infinity values are omitted rather than emitted as samples.

llama.cpp Backend Metrics Copy Section

When a loaded model uses the llamacpp recipe, Lemonade makes a best-effort scrape of the loaded backend process's private /metrics endpoint. Backend scrape failures do not fail the Lemonade /metrics response.

Scraped llama.cpp metrics are normalized under the lemonade_llamacpp_* prefix and labeled with the same Lemonade model metadata used by lemonade_model_info.

Lemonade starts llama.cpp backends with metrics enabled so these backend metrics are available whenever the backend supports them.

`GET /v1/system-info` Copy Chapter

System information endpoint that provides complete hardware details and device enumeration.

Example request Copy Section

curl "http://localhost:13305/v1/system-info"

Response format Copy Section

/* Detailed source-code truncated for AI context efficiency. */

Field Descriptions:

• System fields:

  • OS Version - Operating system name and version
  • Processor - CPU model name
  • Physical Memory - Total RAM
  • OEM System - System/laptop model name (Windows only)
  • BIOS Version - BIOS information (Windows only)
  • CPU Max Clock - Maximum CPU clock speed (Windows only)
  • Windows Power Setting - Current power plan (Windows only)

• model_storage - Drive-level storage information for the active configured model storage path. Values are reported in bytes for storage meters; this is not a recursive sum of Lemonade model files.

  • path - Active model storage path from server configuration
  • used_bytes - Used bytes on the model-storage drive
  • total_bytes - Total capacity of the model-storage drive
  • free_bytes - Free bytes available to the Lemonade Server process on the model-storage drive

• devices - Hardware devices detected on the system (no software/support information)

  • cpu - CPU information (name, cores, threads)
  • amd_gpu - Array of AMD GPUs, both integrated and discrete (if present)
  • nvidia_gpu - Array of NVIDIA GPUs (if present)
  • amd_npu - AMD NPU device (if present)

• recipes - Software recipes and their backend support status

  • Each recipe (e.g., llamacpp, whispercpp, flm) contains:
    • default_backend - Preferred backend selected by server policy for this system (present when at least one backend is not unsupported)
    • backends - Available backends for this recipe
      • Each backend contains:
        • devices - List of devices on this system that support this backend (empty if not supported)
        • state - Backend lifecycle state: unsupported, installable, update_required, or installed
        • message - Human-readable status text for GUI and CLI users. Required for unsupported, installable, and update_required; empty for installed.
        • action - Actionable user instruction string. For install/update cases this is typically an exact CLI command; for other states it may be empty or another actionable value (for example, a URL).
        • version - Installed or configured backend version (when available)

• cloud - Cloud OpenAI-compatible providers configured on this server (omitted when no providers are installed). Contains:

  • providers - Array, one entry per installed provider:
    • name - Provider name used as the model-name prefix (e.g. fireworks).
    • base_url - Persisted base URL from config.json.
    • env_var - Canonical environment variable name for this provider's API key (e.g. LEMONADE_FIREWORKS_API_KEY). The variable's name is reported, never its value.
    • env_var_set - true if the env var is set in lemond's environment.
    • runtime_key_set - true if an in-memory key has been supplied via POST /v1/cloud/auth this session.
    • models_discovered - Number of chat-capable models currently in the catalog for this provider.

`POST /v1/install` Copy Chapter

Install or update a backend for a specific recipe/backend pair, or register a cloud OpenAI-compatible provider. The request body is dispatched by the backend field: any value other than "cloud" is treated as a local backend install.

Install a local backend Copy Section

If the backend is already installed but outdated, this endpoint updates it to the configured version.

Example request:

curl -X POST http://localhost:13305/v1/install \
  -H "Content-Type: application/json" \
  -d '{
    "recipe": "llamacpp",
    "backend": "vulkan",
    "stream": false
  }'

Response format:

{
  "status":"success",
  "recipe":"llamacpp",
  "backend":"vulkan"
}

In case of an error, returns an error field with details.

Install a cloud provider Copy Section

Registers an OpenAI-compatible chat provider. The base URL is persisted to config.json; the optional api_key lives in lemond process memory only (cleared on restart). See the Cloud Offload guide for the full workflow.

Example request:

curl -X POST http://localhost:13305/v1/install \
  -H "Content-Type: application/json" \
  -d '{
    "backend": "cloud",
    "provider": "fireworks",
    "base_url": "https://api.fireworks.ai/inference/v1"
  }'

Response format:

{
  "status": "success",
  "backend": "cloud",
  "provider": "fireworks",
  "base_url": "https://api.fireworks.ai/inference/v1",
  "models_discovered": 12,
  "auth_state": {
    "env_var_set": true,
    "runtime_key_set": false
  }
}

models_discovered is 0 when no API key is resolvable. If api_key is supplied but the provider's env var is also set, the response includes a warning string explaining the env var took precedence.

`POST /v1/uninstall` Copy Chapter

Uninstall a backend for a specific recipe/backend pair, or remove a cloud provider. Dispatched by the backend field, mirroring /v1/install.

Uninstall a local backend Copy Section

If loaded models are using that backend, they are unloaded first.

Example request:

curl -X POST http://localhost:13305/v1/uninstall \
  -H "Content-Type: application/json" \
  -d '{
    "recipe": "llamacpp",
    "backend": "vulkan"
  }'

Response format:

{
  "status":"success",
  "recipe":"llamacpp",
  "backend":"vulkan"
}

In case of an error, returns an error field with details.

Uninstall a cloud provider Copy Section

Removes the provider record from config.json, drops its in-memory API key (if any), and evicts every discovered model for that provider from the cache. Returns 404 if the provider was never installed.

Example request:

curl -X POST http://localhost:13305/v1/uninstall \
  -H "Content-Type: application/json" \
  -d '{
    "backend": "cloud",
    "provider": "fireworks"
  }'

Response format:

{
  "status": "success",
  "backend": "cloud",
  "provider": "fireworks",
  "models_evicted": 12
}

`POST /v1/cloud/auth` Copy Chapter

Set an in-memory API key for a previously-installed cloud provider, and trigger a refresh of that provider's discovered model list. The key lives in lemond process memory only — it is never written to disk and is cleared on lemond restart. For persistence across restarts, set LEMONADE__API_KEY in lemond's environment instead.

Authentication precedence Copy Section

If LEMONADE__API_KEY is set in lemond's environment, the env var takes precedence and this endpoint returns 409 Conflict without storing the supplied key. This is the safety guarantee that lets an operator provision a "house" key via env without worrying about a client silently overriding it.

Parameters Copy Section

Example request Copy Section

curl -X POST http://localhost:13305/v1/cloud/auth \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "fireworks",
    "api_key": "fw-XXXXX"
  }'

Response format (success — 200) Copy Section

{
  "provider": "fireworks",
  "auth_state": {
    "env_var_set": false,
    "runtime_key_set": true
  },
  "models_discovered": 12
}

Response format (env-var conflict — 409) Copy Section

{
  "error": {
    "type": "auth_conflict",
    "env_var": "LEMONADE_FIREWORKS_API_KEY",
    "message": "LEMONADE_FIREWORKS_API_KEY is set in the lemond process; the env var takes precedence and the supplied API key was not stored."
  }
}

Other error responses Copy Section

`DELETE /v1/cloud/auth/{provider}` Copy Chapter

Clear the in-memory API key for a provider. Any env-var-based key (LEMONADE__API_KEY) remains in effect. If no env-var key is set, the provider's discovered models are evicted from the catalog since they are no longer authenticatable.

Example request Copy Section

curl -X DELETE http://localhost:13305/v1/cloud/auth/fireworks

Response format Copy Section

{
  "provider": "fireworks",
  "cleared_runtime_key": true,
  "auth_state": {
    "env_var_set": false,
    "runtime_key_set": false
  }
}

cleared_runtime_key is false when no in-memory key was present (e.g., the only key was from the env var).

Log Streaming API (WebSocket) Copy Chapter

Stream server logs over WebSocket. Clients connect, send a subscribe message, and receive a snapshot of recent log history followed by live log entries as they occur.

Connection Copy Section

The WebSocket server shares the same port as the Realtime Audio Transcription API. Discover the port via the /v1/health endpoint (websocket_port field), then connect:

ws://localhost:<websocket_port>/logs/stream

After connecting, send a logs.subscribe message to start receiving logs.

Client → Server Messages Copy Section

Server → Client Messages Copy Section

Example: Subscribe to Logs Copy Section

Subscribe from the beginning (full backlog):

{
  "type": "logs.subscribe",
  "after_seq": null
}

Resume after a known sequence number (e.g., on reconnect):

{
  "type": "logs.subscribe",
  "after_seq": 1042
}

Example: Snapshot Response Copy Section

{
  "type": "logs.snapshot",
  "entries": [
    {
      "seq": 1,
      "timestamp": "2025-03-30 14:22:01.123",
      "severity": "Info",
      "tag": "Server",
      "line": "2025-03-30 14:22:01.123 [Info] (Server) Starting Lemonade Server..."
    }
  ]
}

Example: Live Entry Copy Section

{
  "type": "logs.entry",
  "entry": {
    "seq": 1043,
    "timestamp": "2025-03-30 14:22:05.456",
    "severity": "Info",
    "tag": "Router",
    "line": "2025-03-30 14:22:05.456 [Info] (Router) Model loaded successfully"
  }
}

Log Entry Fields Copy Section

Integration Notes Copy Section

• Reconnection: Track the last seq received and pass it as after_seq on reconnect to avoid duplicate entries.
• Backlog: The server retains up to 5000 recent log entries. The snapshot may be smaller if fewer entries exist.
• Platform availability: WebSocket log streaming is available on all platforms (Windows, Linux, and macOS).

`GET /live` Copy Chapter

Lightweight liveness probe for load balancers and orchestrators. Unlike /v1/health, this endpoint does no work beyond confirming the process is up — it does not inspect loaded models or backends — so it is safe to poll at high frequency. HEAD /live is also supported and returns 200 OK with an empty body.

Unlike the other endpoints on this page, /live is not versioned and is not mounted under the /api/v0/, /api/v1/, /v0/, /v1/ prefixes.

Example request Copy Section

curl http://localhost:13305/live

Response format Copy Section

{"status":"ok"}

Job Engine API Copy Chapter

Run client-posted sequences of server operations as durable, background jobs — steps that pass data forward, branch on results, and have a pause / interrupt / resume / delete / query lifecycle that survives client disconnect and server restart. Exclusive ops (load/unload/chat) hold a Router slot so normal traffic queues behind a running job.

See docs/dev/job-system.md for the step schema, op set, and lifecycle, and docs/dev/job-expression-language.md for the when/branch expression grammar.

Internal Endpoints Copy Chapter

Internal endpoints are used for server control and configuration. By default, they are secured by LEMONADE_ADMIN_API_KEY (if set) to separate control privileges from standard inference operations.

`POST /internal/telemetry/flush` Copy Chapter

Forces the in-memory telemetry queue to flush all buffered trace spans immediately to the configured OTLP collector. This call blocks until all currently queued spans are serialized and sent.

Parameters

None.

Example request:

curl -X POST http://localhost:13305/internal/telemetry/flush

Response Format

Returns a JSON object indicating successful completion of the flush operation:

{
  "status": "flushed"
}

`GET /internal/aliases` Copy Chapter

Retrieves a list of all active model alias mappings.

Parameters

None.

Example request:

curl http://localhost:13305/internal/aliases

Response Format

Returns a JSON object containing an array of active alias objects:

{
  "aliases": [
    {
      "alias": "my-alias-1",
      "target": "user.custom-llama",
      "downloaded": true,
      "recipe": "llamacpp"
    }
  ]
}

`POST /internal/aliases` Copy Chapter

Binds a model alias to a target model name.

Parameters

Example request:

curl -X POST http://localhost:13305/internal/aliases \
  -H "Content-Type: application/json" \
  -d '{
    "alias": "my-alias-1",
    "target": "user.custom-llama"
  }'

Response Format

Returns a JSON object confirming the alias binding:

{
  "status": "ok",
  "alias": "my-alias-1",
  "target": "user.custom-llama"
}

Returns HTTP 400 Bad Request if required fields are missing or invalid.

`DELETE /internal/aliases/{alias}` Copy Chapter

Removes an existing model alias binding by name.

Parameters

Example request:

curl -X DELETE http://localhost:13305/internal/aliases/my-alias-1

Response Format

Returns a JSON object confirming deletion:

{
  "status": "deleted",
  "alias": "my-alias-1"
}

Returns HTTP 404 Not Found if the alias does not exist.

File: docs/api/llamacpp.md Copy Chapter

llama.cpp-Specific API

This page documents Lemonade's llama.cpp-specific compatibility surface.

Summary Copy Chapter

`POST /v1/rerank` Copy Chapter

Reranking API for llama.cpp-compatible reranker models. You provide a query and a list of documents, and receive relevance scores for each document. Lemonade will load the requested model automatically if it is not already loaded.

Note: This endpoint is part of Lemonade's llama.cpp compatibility layer. Internally, Lemonade forwards the request to llama.cpp's /v1/rerank endpoint.

Note: Lemonade also accepts /reranking and /reranker as aliases — all three route to the same handler and behave identically.

Note: The endpoint is available under all four path prefixes: /api/v0/, /api/v1/, /v0/, and /v1/.

Note: This endpoint is only available for reranker-specific models using the llamacpp recipe, such as bge-reranker-v2-m3-GGUF.

Parameters Copy Section

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/rerank" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{
    "model": "bge-reranker-v2-m3-GGUF",
    "query": "What is the capital of France?",
    "documents": [
      "Paris is the capital of France.",
      "Berlin is the capital of Germany.",
      "Madrid is the capital of Spain."
    ]
  }' -UseBasicParsing
```

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
        "model": "bge-reranker-v2-m3-GGUF",
        "query": "What is the capital of France?",
        "documents": [
          "Paris is the capital of France.",
          "Berlin is the capital of Germany.",
          "Madrid is the capital of Spain."
        ]
      }'
```

Response format Copy Section

{
  "model": "bge-reranker-v2-m3-GGUF",
  "object": "list",
  "results": [
    {
      "index": 0,
      "relevance_score": 8.60673713684082
    },
    {
      "index": 1,
      "relevance_score": -5.3886260986328125
    },
    {
      "index": 2,
      "relevance_score": -3.555561065673828
    }
  ],
  "usage": {
    "prompt_tokens": 51,
    "total_tokens": 51
  }
}

Field Descriptions:

• model - Model identifier used for reranking
• object - Type of response object, always "list"
• results - Array of all input documents with relevance scores
  • index - Original index of the document in the input array
  • relevance_score - Relevance score assigned by the model; higher means more relevant
• usage - Token usage statistics
  • prompt_tokens - Number of tokens in the input
  • total_tokens - Total tokens processed

Note: Results are returned in input order. To rank documents by relevance, sort results by relevance_score in descending order on the client side.

`GET /v1/slots` Copy Chapter

Returns the current state of all processing slots in the llama.cpp server. Slots are parallel processing contexts that can handle multiple requests concurrently.

Note: This endpoint is part of Lemonade's llama.cpp compatibility layer. Internally, Lemonade forwards the request to llama.cpp's /slots endpoint.

Note: This endpoint is only available when a llama.cpp model is loaded.

Note: This endpoint supports all four path prefixes: /api/v0/slots, /api/v1/slots, /v0/slots, and /v1/slots.

Parameters Copy Section

This endpoint accepts no parameters.

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/slots" `
  -Method GET -UseBasicParsing
```

=== "Bash"

```bash
curl http://localhost:13305/v1/slots
```

Response format Copy Section

[
  {
    "id": 0,
    "state": "idle",
    "next_token": {
      "has_next_token": false,
      "n_remain": 0,
      "n_decoded": 0
    },
    "task_id": -1,
    "cache_tokens": 1024
  },
  {
    "id": 1,
    "state": "processing",
    "next_token": {
      "has_next_token": true,
      "n_remain": 42,
      "n_decoded": 15
    },
    "task_id": 123,
    "cache_tokens": 512
  }
]

Field Descriptions:

• id - Unique identifier for the slot
• state - Current processing state ("idle", "processing", etc.)
• next_token - Information about token generation state
  • has_next_token - Whether more tokens are expected
  • n_remain - Number of tokens remaining to generate
  • n_decoded - Number of tokens already decoded
• task_id - Identifier of the current task being processed (-1 if idle)
• cache_tokens - Number of cached tokens in the slot's prompt cache

`POST /v1/slots/{id}?action=save` Copy Chapter

Save the prompt cache of a specific slot to a file. This allows you to persist the current context state for later restoration.

Note: This endpoint is part of Lemonade's llama.cpp compatibility layer. Internally, Lemonade forwards the request to llama.cpp's /slots/{id}?action=save endpoint.

Note: The llama.cpp server must be started with the --slot-save-path argument for save operations to work. See Server Configuration for details on configuring backend arguments.

Example configuration:

bash Copy

lemonade config set llamacpp.args="--slot-save-path /path/to/slot/saves"

Note: This endpoint supports all four path prefixes: /api/v0/slots/{id}, /api/v1/slots/{id}, /v0/slots/{id}, and /v1/slots/{id}.

Parameters Copy Section

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/slots/0?action=save" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{"filename": "my_conversation_cache.bin"}' -UseBasicParsing
```

=== "PowerShell (/api/v1)"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/api/v1/slots/0?action=save" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{"filename": "my_conversation_cache.bin"}' -UseBasicParsing
```

=== "Bash"

```bash
curl -X POST "http://localhost:13305/v1/slots/0?action=save" \
  -H "Content-Type: application/json" \
  -d '{"filename": "my_conversation_cache.bin"}'
```

Response format Copy Section

{
  "id_slot": 0,
  "filename": "my_conversation_cache.bin",
  "n_saved": 1024
}

Field Descriptions:

• id_slot - The slot ID that was saved
• filename - The filename where the cache was saved
• n_saved - Number of tokens saved to the cache file

`POST /v1/slots/{id}?action=restore` Copy Chapter

Restore the prompt cache of a specific slot from a previously saved file. This allows you to resume a conversation or context from where you left off.

Note: This endpoint is part of Lemonade's llama.cpp compatibility layer. Internally, Lemonade forwards the request to llama.cpp's /slots/{id}?action=restore endpoint.

Note: The llama.cpp server must be started with the --slot-save-path argument for restore operations to work.

Note: This endpoint supports all four path prefixes: /api/v0/slots/{id}, /api/v1/slots/{id}, /v0/slots/{id}, and /v1/slots/{id}.

Parameters Copy Section

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/slots/0?action=restore" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{"filename": "my_conversation_cache.bin"}' -UseBasicParsing
```

=== "PowerShell (/api/v1)"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/api/v1/slots/0?action=restore" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{"filename": "my_conversation_cache.bin"}' -UseBasicParsing
```

=== "Bash"

```bash
curl -X POST "http://localhost:13305/v1/slots/0?action=restore" \
  -H "Content-Type: application/json" \
  -d '{"filename": "my_conversation_cache.bin"}'
```

Response format Copy Section

{
  "id_slot": 0,
  "filename": "my_conversation_cache.bin",
  "n_restored": 1024
}

Field Descriptions:

• id_slot - The slot ID that was restored
• filename - The filename from which the cache was restored
• n_restored - Number of tokens restored from the cache file

`POST /v1/slots/{id}?action=erase` Copy Chapter

Erase (clear) the prompt cache of a specific slot. This removes all cached context from the slot, resetting it to an empty state.

Note: This endpoint is part of Lemonade's llama.cpp compatibility layer. Internally, Lemonade forwards the request to llama.cpp's /slots/{id}?action=erase endpoint.

Note: This endpoint supports all four path prefixes: /api/v0/slots/{id}, /api/v1/slots/{id}, /v0/slots/{id}, and /v1/slots/{id}.

Parameters Copy Section

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/slots/0?action=erase" `
  -Method POST -UseBasicParsing
```

=== "PowerShell (/api/v1)"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/api/v1/slots/0?action=erase" `
  -Method POST -UseBasicParsing
```

=== "Bash"

```bash
curl -X POST "http://localhost:13305/v1/slots/0?action=erase"
```

Response format Copy Section

{
  "id_slot": 0
}

Field Descriptions:

• id_slot - The slot ID that was erased

Note: If the server returns an error, it may indicate that the slot was not found or that the operation failed.

`POST /v1/tokenize` Copy Chapter

Tokenize a given text. Does not count towards the current model's context window.

Note: This endpoint is part of Lemonade's llama.cpp compatibility layer. Internally, Lemonade forwards the request to llama.cpp's /tokenize endpoint.

Note: This endpoint supports all four path prefixes: /api/v0/tokenize, /api/v1/tokenize, /v0/tokenize, and /v1/tokenize.

Note: Actual response values may vary for the same string across different models if the models do not share the same tokenizer.

Parameters Copy Section

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/tokenize" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{"content": "This is a string to tokenize"}' -UseBasicParsing
```

=== "PowerShell (/api/v1)"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/api/v1/tokenize" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{"content": "This is a string to tokenize"}' -UseBasicParsing
```

=== "Bash"

```bash
curl -X POST "http://localhost:13305/v1/tokenize" \
  -H "Content-Type: application/json" \
  -d '{"content": "This is a string to tokenize"}'
```

Response format Copy Section

{
  "tokens": [1919,369,264,886,310,74995]
}

If with_pieces is true:

{
  "tokens": [
    {"id": 123, "piece": "Hello"},
    {"id": 456, "piece": " world"},
    {"id": 789, "piece": "!"}
  ]
}

Field Descriptions:

• tokens - Array of token IDs

File: docs/api/mcp.md Copy Chapter

MCP Gateway

Lemonade exposes its inference capabilities as a Model Context Protocol (MCP) server, so any MCP-compatible client (GitHub Copilot, Claude Desktop, MCP Inspector, Cursor, the mcp Python client, etc.) can call your locally running models as tools.

The gateway implements the MCP "Streamable HTTP" transport (spec version 2025-06-18) with the tools capability only. All traffic flows through a single endpoint:

Why a single path? The MCP specification mandates one endpoint URL per server, so /mcp is an intentional exception to Lemonade's quad-prefix convention.

Authentication Copy Chapter

/mcp is treated as a regular API route, so it honors LEMONADE_API_KEY exactly like /api/v1/chat/completions:

curl -s http://localhost:13305/mcp \
    -H "Authorization: Bearer $LEMONADE_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"jsonrpc":"2.0","id":1,"method":"ping"}'

Supported methods Copy Chapter

Tools Copy Chapter

All tools auto-load (and download, if missing) the requested model on first call, exactly like POST /v1/chat/completions. Errors are returned as MCP results with "isError": true rather than JSON-RPC errors, matching the spec's guidance for tool failures.

`lemonade_list_models` Copy Section

Discover what's loaded, what's downloaded, and what's recommended. Call this first if you don't already know the exact model name to pass to the other tools — passing a wrong name may trigger a multi-GB download.

{
  "name": "lemonade_list_models",
  "arguments": {
    "include_available": true,
    "include_suggested": true
  }
}

Returns a summary text block plus a JSON-stringified text block with {loaded, available, suggested_to_pull, recommended_chat_model}.

`lemonade_chat` Copy Section

Chat completion against any LLM in the registry.

{
  "name": "lemonade_chat",
  "arguments": {
    "model": "Qwen3-1.7B-GGUF",
    "messages": [
      {"role": "system", "content": "You are concise."},
      {"role": "user", "content": "Summarize MCP in one line."}
    ],
    "max_tokens": 64,
    "temperature": 0.2
  }
}

Returns one text block with the assistant content. If the model emits tool calls, a second text block containing tool_calls: <json> is appended.

Reasoning models (Qwen3, DeepSeek-R1, ...) have the `` block disabled by default to keep small max_tokens budgets from being consumed by reasoning. Pass "chat_template_kwargs": {"enable_thinking": true} to opt back in.

Picking a portable model. The example above uses Qwen3-1.7B-GGUF because GGUF (llama.cpp) runs everywhere lemonade does — Windows, Linux/Docker, macOS, CPU and Vulkan/ROCm/Metal GPUs. Hybrid/NPU variants such as *-Hybrid (recipe ryzenai-llm, Windows + AMD RyzenAI only) or *-FLM (recipe flm, AMD Ryzen AI NPU only) are faster on supported hardware but unavailable on others. If your client picks one that isn't supported, the tool returns a structured error suggesting a portable alternative — prefer lemonade_list_models to discover what's actually available on the running server.

`lemonade_transcribe_audio` Copy Section

Transcribe a local audio file with a Whisper-class model. Prefer audio_path (the server runs on the same machine as the caller); use audio_base64 only when you genuinely have bytes in memory.

{
  "name": "lemonade_transcribe_audio",
  "arguments": {
    "model": "Whisper-Large-v3-Turbo",
    "audio_path": "C:/clips/meeting.wav",
    "response_format": "verbose_json"
  }
}

Returns two text blocks: the bare transcript, followed by the full OpenAI-shaped response (for callers that need timestamps or segments).

`lemonade_generate_image` Copy Section

Generate one or more PNGs from a prompt. Prefer writing to disk via output_path (single image) or output_dir (one or more) — base64 image content blocks cost tens of thousands of tokens per image and some clients surface them as opaque resource URIs.

{
  "name": "lemonade_generate_image",
  "arguments": {
    "model": "SDXL-Turbo",
    "prompt": "a lemon-shaped car driving across the moon",
    "size": "512x512",
    "output_path": "lemon-car.png"
  }
}

When disk paths are provided, returns text block(s) with the absolute path(s). Otherwise, returns one inline image content block per image ({"type":"image", "data":"", "mimeType":"image/png"}).

Sandboxed disk writes. To prevent a cross-origin or unauthenticated caller from overwriting arbitrary files, output_path and output_dir are confined to a sandbox directory:

• Default: <cache_dir>/mcp-images.
• Override with the LEMONADE_MCP_IMAGE_DIR environment variable (absolute path).
• Relative paths resolve against the sandbox root; absolute paths must stay within it. Paths that escape the sandbox (via .. or symlinks) are rejected.
• output_dir writes use auto-generated, unique filenames (image_<token>_.png), so concurrent callers never clobber one another's images — the returned paths tell you the exact names. Use output_path when you need an exact, caller-chosen filename (it is written as named, replacing any existing file at that path).

`lemonade_omni` Copy Section

One-shot multimodal turn against a Lemonade Omni collection (a model bundle that pairs a planner LLM with an image model, an image-edit model, and a TTS voice under a single collection.omni recipe — see the Omni docs). The server runs the orchestrator's internal tool-calling loop, executes the collection's generate_image / edit_image / text_to_speech tools by routing to the bundled components, and returns the result as a text block plus native MCP image / audio content blocks — one per artifact, in the order they were produced.

model is optional and defaults to LMX-Omni-5.5B-Lite (smaller and faster). Pass model explicitly to opt into a larger collection (e.g. LMX-Omni-52B-Halo on capable hardware) or any other collection.omni model surfaced by lemonade_list_models. The collection is downloaded on first use and may be multi-GB.

Use lemonade_chat instead when you only need plain-text LLM output and don't want the planner-loop overhead.

{
  "name": "lemonade_omni",
  "arguments": {
    "messages": [
      {"role": "user", "content": "Generate an image of a lemon car, then read out a one-line description."}
    ],
    "output_dir": "omni"
  }
}

Disk vs. inline output. A single Omni turn can produce both images and audio in arbitrary order. Pass an output_dir to write each artifact to disk under a unique auto-generated name (omni_<token>_.<ext>) — the tool returns one text block per artifact with its absolute path, plus a JSON-stringified paths array. This is strongly preferred over inline base64 for the same reasons documented under lemonade_generate_image — and is the only way to get audio out on clients that don't render audio content blocks. Like lemonade_generate_image, output_dir is confined to the MCP image sandbox (see Sandboxed disk writes above): relative paths resolve against the sandbox root, paths escaping it are rejected, and unique filenames mean concurrent callers never clobber each other.

When output_dir is omitted, artifacts are inlined as MCP content blocks: {"type":"image", "data":"", "mimeType":"image/png"} and {"type":"audio", "data":"", "mimeType":"audio/mpeg"}.

If the planner emits app-defined tool calls (those you passed in via tools/tool_choice), an extra text block tool_calls: <json> is appended, matching lemonade_chat's passthrough semantics.

Passing a non-collection model (e.g. a plain LLM) returns isError: true with a hint to use lemonade_chat.

Error model Copy Chapter

Tool-level failures (bad arguments, model load errors, backend exceptions) are returned as successful JSON-RPC results with "isError": true and a text content block describing the failure, so MCP-aware models can self-correct.

Limitations (MVP) Copy Chapter

• No server-initiated SSE (GET /mcp returns 405). Tools return their full result in the POST response.
• No session resumption (Mcp-Session-Id header is not issued).
• resources/* and prompts/* capabilities are not implemented.
• Streaming chat output is not exposed via MCP — stream=true is ignored. Use POST /v1/chat/completions directly for streamed tokens.
• Embeddings and text-to-speech are not currently exposed as MCP tools; use the OpenAI-compatible endpoints (/v1/embeddings, /v1/audio/speech) for those.

Quick test with curl Copy Chapter

# 1. Initialize
curl -s http://localhost:13305/mcp -H "Content-Type: application/json" \
    -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{}}}'

# 2. List tools
curl -s http://localhost:13305/mcp -H "Content-Type: application/json" \
    -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}'

# 3. Call lemonade_chat
curl -s http://localhost:13305/mcp -H "Content-Type: application/json" \
    -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"lemonade_chat","arguments":{"model":"Qwen3-1.7B-GGUF","messages":[{"role":"user","content":"hi"}],"max_tokens":16}}}'

File: docs/api/ollama.md Copy Chapter

Ollama-Compatible API

Lemonade supports the Ollama API, allowing applications built for Ollama to work with Lemonade without modification.

To enable auto-detection by Ollama-integrated apps, configure the server to use the Ollama default port 11434. See Server Configuration for how to change the port.

File: docs/api/openai.md Copy Chapter

OpenAI-Compatible API

This spec defines Lemonade's implementation of the OpenAI API.

`POST /v1/chat/completions` Copy Chapter

Chat Completions API. You provide a list of messages and receive a completion. This API will also load the model if it is not already loaded.

Parameters Copy Section

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/chat/completions" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{
    "model": "Qwen3-0.6B-GGUF",
    "messages": [
      {
        "role": "user",
        "content": "What is the population of Paris?"
      }
    ],
    "stream": false
  }'
```

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "Qwen3-0.6B-GGUF",
        "messages": [
          {"role": "user", "content": "What is the population of Paris?"}
        ],
        "stream": false
      }'
```

Response format Copy Section

=== "Non-streaming responses"

```json
{
  "id": "0",
  "object": "chat.completion",
  "created": 1742927481,
  "model": "Qwen3-0.6B-GGUF",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Paris has a population of approximately 2.2 million people in the city proper."
    },
    "finish_reason": "stop"
  }]
}
```

=== "Streaming responses"
For streaming responses, the API returns a stream of server-sent events (however, Open AI recommends using their streaming libraries for parsing streaming responses):

```json
{
  "id": "0",
  "object": "chat.completion.chunk",
  "created": 1742927481,
  "model": "Qwen3-0.6B-GGUF",
  "choices": [{
    "index": 0,
    "delta": {
      "role": "assistant",
      "content": "Paris"
    }
  }]
}
```

Image understanding input format (OpenAI-compatible) Copy Section

To send images to chat/completions, pass a messages[*].content array that mixes text and image_url items. The image can be provided as a base64 data URL (for example, from FileReader.readAsDataURL(...) in web apps).

Example request

curl -X POST http://localhost:13305/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "Qwen2.5-VL-7B-Instruct",
        "messages": [
          {
            "role": "user",
            "content": [
              {"type": "text", "text": "What is in this image?"},
              {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD..."}}
            ]
          }
        ],
        "stream": false
      }'

Example response

{
  "id": "0",
  "object": "chat.completion",
  "created": 1742927481,
  "model": "Qwen2.5-VL-7B-Instruct",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "The image shows a red apple resting on a wooden table."
    },
    "finish_reason": "stop"
  }]
}

Server-side tools Copy Section

Note: Omni collection orchestration is a Lemonade-specific extension to the Chat Completions API; it is not part of the OpenAI specification. Requests that target ordinary (non-collection) models are unaffected and are fully OpenAI-compatible.

When model names an Omni collection model (recipe: "collection.omni", e.g., LMX-Omni-52B-Halo), this endpoint runs an internal tool-calling loop instead of a plain completion. The server injects the reference system prompt and tools, routes to the collection's chat component, executes the omni tools (image generation/editing, text-to-speech) against the matching components, and returns one OpenAI-compatible response. Generated media is embedded in the assistant content:

• images → markdown ``
• speech → <audio>data:audio/mpeg;base64,…</audio>

Both non-streaming and stream: true are supported. In streaming mode the media arrives as a content delta on a chat.completion.chunk frame the moment its tool finishes.

Merge semantics. A client-provided system prompt is prepended by the built-in omni system prompt. Client-provided tools are merged with the built-in omni tools. The server resolves omni tool calls internally; calls to client-provided tools are returned in a finish_reason: "tool_calls" response for the client to execute and resume. Targeting a collection name invokes the server-side loop; targeting a component LLM name bypasses it and returns a plain completion. See Lemonade Omni Models for details.

Example request

curl -X POST http://localhost:13305/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "LMX-Omni-52B-Halo",
        "messages": [{"role": "user", "content": "Draw a red apple on a table."}],
        "stream": false
      }'

Example response

The image tool runs during the loop and its output is embedded as a markdown image in the assistant content:

{
  "id": "0",
  "object": "chat.completion",
  "created": 1742927481,
  "model": "LMX-Omni-52B-Halo",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Here is a red apple on a table.\n\n"
    },
    "finish_reason": "stop"
  }]
}

`POST /v1/completions` Copy Chapter

Text Completions API. You provide a prompt and receive a completion. This API will also load the model if it is not already loaded.

Parameters Copy Section

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest -Uri "http://localhost:13305/v1/completions" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{
    "model": "Qwen3-0.6B-GGUF",
    "prompt": "What is the population of Paris?",
    "stream": false
  }'
```

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "Qwen3-0.6B-GGUF",
        "prompt": "What is the population of Paris?",
        "stream": false
      }'
```

Response format Copy Section

The following format is used for both streaming and non-streaming responses:

{
  "id": "0",
  "object": "text_completion",
  "created": 1742927481,
  "model": "Qwen3-0.6B-GGUF",
  "choices": [{
    "index": 0,
    "text": "Paris has a population of approximately 2.2 million people in the city proper.",
    "finish_reason": "stop"
  }],
}

`POST /v1/embeddings` Copy Chapter

Embeddings API. You provide input text and receive vector representations (embeddings) that can be used for semantic search, clustering, and similarity comparisons. This API will also load the model if it is not already loaded.

Note: This endpoint is only available for models using the llamacpp or flm recipes. ONNX models (OGA recipes) do not support embeddings.

Parameters Copy Section

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/embeddings" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{
    "model": "nomic-embed-text-v1-GGUF",
    "input": ["Hello, world!", "How are you?"],
    "encoding_format": "float"
  }'
```

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
        "model": "nomic-embed-text-v1-GGUF",
        "input": ["Hello, world!", "How are you?"],
        "encoding_format": "float"
      }'
```

Response format Copy Section

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.0234, -0.0567, 0.0891, ...]
    },
    {
      "object": "embedding",
      "index": 1,
      "embedding": [0.0456, -0.0678, 0.1234, ...]
    }
  ],
  "model": "nomic-embed-text-v1-GGUF",
  "usage": {
    "prompt_tokens": 12,
    "total_tokens": 12
  }
}

Field Descriptions:

• object - Type of response object, always "list"
• data - Array of embedding objects
  • object - Type of embedding object, always "embedding"
  • index - Index position of the input text in the request
  • embedding - Vector representation as an array of floats
• model - Model identifier used to generate the embeddings
• usage - Token usage statistics
  • prompt_tokens - Number of tokens in the input
  • total_tokens - Total tokens processed

`POST /v1/responses` Copy Chapter

Responses API. You provide an input and receive a response. This API will also load the model if it is not already loaded.

Parameters Copy Section

Streaming Events Copy Section

The Responses API uses semantic events for streaming. Each event is typed with a predefined schema, so you can listen for events you care about. Our initial implementation only offers support to:

• response.created
• response.output_text.delta
• response.completed

For a full list of event types, see the API reference for streaming.

Example request Copy Section

=== "PowerShell"

```powershell
Invoke-WebRequest -Uri "http://localhost:13305/v1/responses" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{
    "model": "Llama-3.2-1B-Instruct-Hybrid",
    "input": "What is the population of Paris?",
    "stream": false
  }'
```

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
        "model": "Llama-3.2-1B-Instruct-Hybrid",
        "input": "What is the population of Paris?",
        "stream": false
      }'
```

Response format Copy Section

=== "Non-streaming responses"

```json
{
  "id": "0",
  "created_at": 1746225832.0,
  "model": "Llama-3.2-1B-Instruct-Hybrid",
  "object": "response",
  "output": [{
    "id": "0",
    "content": [{
      "annotations": [],
      "text": "Paris has a population of approximately 2.2 million people in the city proper."
    }]
  }]
}
```

=== "Streaming Responses"
For streaming responses, the API returns a series of events. Refer to OpenAI streaming guide for details.

`POST /v1/audio/transcriptions` Copy Chapter

Audio Transcription API. You provide an audio file and receive a text transcription. This API will also load the model if it is not already loaded.

Note: This endpoint uses whisper.cpp as the backend. Whisper models are automatically downloaded when first used.

Limitations: Only wav audio input is currently supported. The response_format field supports json, verbose_json, text, srt, and vtt. On the FastFlowLM (FLM) backend, srt and vtt are rejected with a 400 because FLM returns no segment timestamps, and verbose_json returns the compact shape without a segments field.

Parameters Copy Section

Example request Copy Section

=== "Windows"

```bash
curl -X POST http://localhost:13305/v1/audio/transcriptions ^
  -F "file=@C:\path\to\audio.wav" ^
  -F "model=Whisper-Tiny"
```

=== "Linux"

```bash
curl -X POST http://localhost:13305/v1/audio/transcriptions \
  -F "file=@/path/to/audio.wav" \
  -F "model=Whisper-Tiny"
```

Response format Copy Section

{
  "text": "Hello, this is a sample transcription of the audio file."
}

Field Descriptions:

• text - The transcribed text from the audio file

`WS /realtime` Copy Chapter

Realtime Audio Transcription API via WebSocket (OpenAI SDK compatible). Stream audio from a microphone and receive transcriptions in real-time with Voice Activity Detection (VAD).

Limitations: Only 16kHz mono PCM16 audio format is supported. Uses the same Whisper models as the HTTP transcription endpoint.

Connection Copy Section

WebSocket upgrades are accepted directly on the main HTTP port (default 13305) — the same port as all REST endpoints. Connect with the model name:

ws://localhost:13305/v1/realtime?model=Whisper-Tiny

Accepted paths are /realtime and /logs/stream, bare or under any of the standard prefixes (/v1, /v0, /api/v1, /api/v0), so OpenAI Realtime SDK clients (/v1/realtime) connect as-is. When LEMONADE_API_KEY is set, pass ?api_key=KEY as a query parameter.

A dedicated WebSocket port also remains for backward compatibility; it is OS-assigned and reported by the /v1/health endpoint (websocket_port field). New clients should prefer the main port:

ws://localhost:<websocket_port>/realtime?model=Whisper-Tiny

Upon connection, the server sends a session.created message with a session ID.

Client → Server Messages Copy Section

Server → Client Messages Copy Section

Example: Configure Session Copy Section

{
  "type": "session.update",
  "session": {
    "model": "Whisper-Tiny"
  }
}

Example: Send Audio Copy Section

{
  "type": "input_audio_buffer.append",
  "audio": ""
}

Audio should be:

• 16kHz sample rate
• Mono (single channel)
• 16-bit signed integer (PCM16)
• Base64 encoded
• Sent in chunks (~85ms recommended)

Example: Transcription Result Copy Section

{
  "type": "conversation.item.input_audio_transcription.completed",
  "transcript": "Hello, this is a test transcription."
}

VAD Configuration Copy Section

VAD settings can be configured via session.update:

{
  "type": "session.update",
  "session": {
    "model": "Whisper-Tiny",
    "turn_detection": {
      "threshold": 0.01,
      "silence_duration_ms": 800,
      "prefix_padding_ms": 250
    }
  }
}

Set turn_detection to null to disable server-side VAD and use explicit commits instead:

{
  "type": "session.update",
  "session": {
    "model": "Whisper-Tiny",
    "turn_detection": null
  }
}

Code Examples Copy Section

A complete, runnable example:

• realtime_transcription.py - Python CLI for microphone streaming

# Stream from microphone
python examples/realtime_transcription.py --model Whisper-Tiny

Integration Notes Copy Section

• Audio Format: Server expects 16kHz mono PCM16. Higher sample rates must be downsampled client-side.
• Chunk Size: Send audio in ~85-256ms chunks for optimal latency/efficiency.
• VAD Behavior: Server automatically detects speech boundaries and triggers transcription on speech end.
• Manual Commit: Set turn_detection to null, then use input_audio_buffer.commit to force transcription. In this mode the server buffers audio but does not emit VAD or interim transcription events.
• Clear Buffer: Use input_audio_buffer.clear to discard audio without transcribing.
• Chunking: We are still tuning the chunking to balance latency vs. accuracy.
• Migrating off the dedicated port: Clients that discover websocket_port via /v1/health and connect there can switch to ws://HOST:13305/v1/realtime?model=... — the protocol (events, audio format, auth) is identical on both ports, so it is a URL change only. This also simplifies remote setups (one port to expose) and works through reverse proxies that pass Upgrade: websocket. Keep the websocket_port fallback only if you must support servers older than this release.

`POST /v1/images/generations` Copy Chapter

Image Generation API. You provide a text prompt and receive a generated image. This API uses stable-diffusion.cpp as the backend.

Note: Image generation uses Stable Diffusion models. Available models include SD-Turbo (fast, ~4 steps), SDXL-Turbo, SD-1.5, and SDXL-Base-1.0.

Performance: CPU inference takes ~4-5 minutes per image. GPU (Vulkan) is faster but may have compatibility issues with some hardware.

Parameters Copy Section

Example request Copy Section

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
        "model": "SD-Turbo",
        "prompt": "A serene mountain landscape at sunset",
        "size": "512x512",
        "steps": 4,
        "response_format": "b64_json"
      }'
```

`POST /v1/images/edits` Copy Chapter

Image Editing API. You provide a source image and a text prompt describing the desired change, and receive an edited image. This API uses stable-diffusion.cpp as the backend.

Note: This endpoint accepts multipart/form-data requests (not JSON). Use editing-capable models such as Flux-2-Klein-4B or SD-Turbo.

Performance: CPU inference takes several minutes per image. GPU (ROCm) is significantly faster.

Parameters Copy Section

Example request Copy Section

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/images/edits \
  -F "model=Flux-2-Klein-4B" \
  -F "prompt=Add a red barn and mountains in the background, photorealistic" \
  -F "size=512x512" \
  -F "n=1" \
  -F "response_format=b64_json" \
  -F "image=@/path/to/source_image.png"
```

=== "Python (OpenAI client)"

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:13305/api/v1", api_key="not-needed")
with open("source_image.png", "rb") as image_file:
    response = client.images.edit(
        model="Flux-2-Klein-4B",
        image=image_file,
        prompt="Add a red barn and mountains in the background, photorealistic",
        size="512x512",
    )
import base64
image_data = base64.b64decode(response.data[0].b64_json)
open("edited_image.png", "wb").write(image_data)
```

`POST /v1/images/variations` Copy Chapter

Image Variations API. You provide a source image and receive a variation of it. This API uses stable-diffusion.cpp as the backend.

Note: This endpoint accepts multipart/form-data requests (not JSON). Unlike /images/edits, a prompt parameter is not supported and will be ignored — the model generates a variation based solely on the input image.

Performance: CPU inference takes several minutes per image. GPU (ROCm) is significantly faster.

Parameters Copy Section

Example request Copy Section

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/images/variations \
  -F "model=Flux-2-Klein-4B" \
  -F "size=512x512" \
  -F "n=1" \
  -F "response_format=b64_json" \
  -F "image=@/path/to/source_image.png"
```

=== "Python (OpenAI client)"

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:13305/api/v1", api_key="not-needed")
with open("source_image.png", "rb") as image_file:
    response = client.images.create_variation(
        model="Flux-2-Klein-4B",
        image=image_file,
        size="512x512",
        n=1,
    )
import base64
image_data = base64.b64decode(response.data[0].b64_json)
open("variation.png", "wb").write(image_data)
```

`POST /v1/images/upscale` Copy Chapter

Image Upscaling API. You provide a base64-encoded image and a Real-ESRGAN model name, and receive a 4x upscaled image. This API uses the sd-cli binary from stable-diffusion.cpp to perform super-resolution.

Note: Available upscale models are RealESRGAN-x4plus (general-purpose, 64 MB) and RealESRGAN-x4plus-anime (optimized for anime-style art, 17 MB). Both produce a 4x resolution increase (e.g., 256x256 → 1024x1024).

Note: Unlike /images/edits and /images/variations, this endpoint accepts a JSON body (not multipart/form-data). The image must be provided as a base64-encoded string.

Parameters Copy Section

Example request Copy Section

A typical workflow is to generate an image first, then upscale it:

=== "Bash"

```bash
# Step 1: Generate an image and save the base64 response
RESPONSE=$(curl -s -X POST http://localhost:13305/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
        "model": "SD-Turbo",
        "prompt": "A serene mountain landscape at sunset",
        "size": "512x512",
        "steps": 4,
        "response_format": "b64_json"
      }')

# Step 2: Build the upscale JSON payload and pipe it to curl via stdin
# (base64 images are too large for command-line interpolation)
echo "$RESPONSE" | python3 -c "
import sys, json
b64 = json.load(sys.stdin)['data'][0]['b64_json']
print(json.dumps({'image': b64, 'model': 'RealESRGAN-x4plus'}))
" | curl -X POST http://localhost:13305/v1/images/upscale \
  -H "Content-Type: application/json" \
  -d @-
```

=== "PowerShell"

```powershell
# Step 1: Generate an image
$genResponse = Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/images/generations" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body '{
    "model": "SD-Turbo",
    "prompt": "A serene mountain landscape at sunset",
    "size": "512x512",
    "steps": 4,
    "response_format": "b64_json"
  }'

# Step 2: Extract the base64 image
$imageB64 = ($genResponse.Content | ConvertFrom-Json).data[0].b64_json

# Step 3: Upscale the image with Real-ESRGAN
$body = @{ image = $imageB64; model = "RealESRGAN-x4plus" } | ConvertTo-Json
Invoke-WebRequest `
  -Uri "http://localhost:13305/v1/images/upscale" `
  -Method POST `
  -Headers @{ "Content-Type" = "application/json" } `
  -Body $body
```

=== "Python (requests)"

```python
import requests
import base64

BASE_URL = "http://localhost:13305/api/v1"

# Step 1: Generate an image
gen_response = requests.post(f"{BASE_URL}/images/generations", json={
    "model": "SD-Turbo",
    "prompt": "A serene mountain landscape at sunset",
    "size": "512x512",
    "steps": 4,
    "response_format": "b64_json",
})
image_b64 = gen_response.json()["data"][0]["b64_json"]

# Step 2: Upscale the image with Real-ESRGAN (512x512 -> 2048x2048)
upscale_response = requests.post(f"{BASE_URL}/images/upscale", json={
    "image": image_b64,
    "model": "RealESRGAN-x4plus",
})

# Step 3: Save the upscaled image to a file
upscaled_b64 = upscale_response.json()["data"][0]["b64_json"]
with open("upscaled.png", "wb") as f:
    f.write(base64.b64decode(upscaled_b64))
```

Response format Copy Section

{
  "created": 1742927481,
  "data": [
    {
      "b64_json": ""
    }
  ]
}

Field Descriptions:

• created - Unix timestamp of when the upscaled image was generated
• data - Array containing the upscaled image
  • b64_json - Base64-encoded PNG of the upscaled image

Error responses Copy Section

`POST /v1/audio/speech` Copy Chapter

Speech Generation API. You provide a text input and receive an audio file. Which engine serves the request depends on the model.

Note: Supported models are kokoro-v1 (fixed voices, Kokoros backend) and the OpenMOSS family — OpenMOSS-TTS (voice cloning from a reference WAV) and MOSS-VoiceGen (voice design from a text description).

Limitations: Which response_format values are accepted depends on the model's backend: kokoro-v1 encodes mp3, wav, opus, and pcm, while OpenMOSS models natively produce wav only. A format the backend cannot encode is rejected with 400 Bad Request listing the ones it does support. Streaming works for any TTS backend, but a backend's streamable set can be narrower than its buffered set — kokoro-v1 streams pcm only, so an explicit response_format of mp3 alongside stream_format is rejected rather than silently answered with PCM.

Parameters Copy Section

Example request Copy Section

=== "Bash"

```bash
curl -X POST http://localhost:13305/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
        "model": "kokoro-v1",
        "input": "Lemonade can speak!",
        "speed": 1.0,
        "steps": 4,
        "response_format": "mp3"
      }'
```

Response format Copy Section

The generated audio file is returned as-is.

`GET /v1/models` Copy Chapter

Returns a list of models available on the server in an OpenAI-compatible format. Each model object includes extended fields like checkpoint, recipe, size, downloaded, labels, and, when known, max_context_window.

By default, only models available locally (downloaded) are shown, matching OpenAI API behavior.

When lemond is configured with cloud providers, cloud-routed models appear here alongside local ones with recipe: "cloud" and a cloud_provider field. They are dot-namespaced by provider (e.g. fireworks.kimi-k2p5) and accept the standard chat-completions / completions requests below — see Cloud Offload.

Parameters Copy Section

Example request Copy Section

# Show only downloaded models (OpenAI-compatible)
curl http://localhost:13305/v1/models

# Show all models including not-yet-downloaded (extended usage)
curl http://localhost:13305/v1/models?show_all=true

Response format Copy Section

{
  "object": "list",
  "data": [
    {
      "id": "Qwen3-0.6B-GGUF",
      "created": 1744173590,
      "object": "model",
      "owned_by": "lemonade",
      "checkpoint": "unsloth/Qwen3-0.6B-GGUF:Q4_0",
      "recipe": "llamacpp",
      "size": 0.38,
      "max_context_window": 40960,
      "downloaded": true,
      "suggested": true,
      "update_available": false,
      "labels": ["reasoning"]
    },
    {
      "id": "Gemma-3-4b-it-GGUF",
      "created": 1744173590,
      "object": "model",
      "owned_by": "lemonade",
      "checkpoint": "ggml-org/gemma-3-4b-it-GGUF:Q4_K_M",
      "recipe": "llamacpp",
      "size": 3.61,
      "downloaded": true,
      "suggested": true,
      "labels": ["hot", "vision"]
    },
    {
      "id": "SD-Turbo",
      "created": 1744173590,
      "object": "model",
      "owned_by": "lemonade",
      "checkpoint": "stabilityai/sd-turbo:sd_turbo.safetensors",
      "recipe": "sd-cpp",
      "size": 5.2,
      "downloaded": true,
      "suggested": true,
      "labels": ["image"],
      "image_defaults": {
        "steps": 4,
        "cfg_scale": 1.0,
        "width": 512,
        "height": 512
      }
    }
  ]
}

Field Descriptions:

• object - Type of response object, always "list"
• data - Array of model objects with the following fields:
  • id - Model identifier (used for loading and inference requests)
  • created - Unix timestamp of when the model entry was created
  • object - Type of object, always "model"
  • owned_by - Owner of the model, always "lemonade"
  • checkpoint - Full checkpoint identifier on Hugging Face
  • recipe - Backend/device recipe used to load the model (e.g., "ryzenai-llm", "llamacpp", "flm")
  • size - Model size in GB (omitted for models without size information)
  • max_context_window - Optional integer indicating the maximum model-supported text context discovered from local static metadata. Currently populated for downloaded GGUF/llama.cpp models and installed FLM text-context models.
  • downloaded - Boolean indicating if the model is downloaded and available locally
  • update_available - Boolean indicating a newer commit exists on HuggingFace for this model. Only set for downloaded HF-backed models. false otherwise.
  • suggested - Boolean indicating if the model is recommended for general use
  • labels - Array of tags describing the model's capabilities and characteristics. See Model Labels for the full list.
  • image_defaults - (Image models only) Default generation parameters for the model:
    • steps - Number of inference steps (e.g., 4 for turbo models, 20 for standard models)
    • cfg_scale - Classifier-free guidance scale (e.g., 1.0 for turbo models, 7.5 for standard models)
    • width - Default image width in pixels
    • height - Default image height in pixels
  • components - (Omni collections only, recipe: "collection.omni") Ordered array of the component model names that make up the collection
  • models - (Omni collections only) Ordered array embedding each component's full model object (same shape as the entries in this list), parallel to components. This makes a collection's /v1/models/{model_id} response self-contained — exporting it produces a file that can be imported elsewhere via /v1/pull

Model Labels Copy Section

Labels describe what a model can do. A model may carry multiple labels.

Deployment labels — determine which backend endpoint the model is routed to.
Every model names exactly one deployment mode, and a model is never given two
labels that name different modes:

When a model declares no deployment label at all, it inherits its recipe's
default — chat for llamacpp, flm, ryzenai-llm, vllm and cloud,
transcription for whispercpp, image for sd-cpp, tts for kokoro, and
so on.

Two label sets describe a model that cannot exist, and are refused rather than
repaired:

• A mode the recipe's backend does not serve. /classify is served only by
  onnxruntime, so labels: ["classification"] on a llamacpp model is an
  error — register it as the chat model it is.
• Two different modes. labels: ["chat", "embeddings"] on a llamacpp model
  is an error even though llama.cpp serves both: the subprocess is launched for
  one mode, so the second would name an endpoint it was never configured to
  answer. Register one model per mode. The legacy embedding and reranking
  booleans count as mode claims here, exactly as the labels do.

POST /v1/pull answers 400 and registers nothing.
An entry already stored in user_models.json — written before these rules — is
skipped at startup with an error naming it, and the file is left untouched so it
can be corrected by hand.

Input-modality labels — the model accepts additional input types in /chat/completions:

Streaming labels — capability flags for real-time features:

Runtime labels — affect backend launch defaults:

Image capability labels — carried alongside image; they refine what the model is offered for without changing its deployment mode:

Characteristic labels — informational, do not affect routing:

`GET /v1/models/{model_id}` Copy Chapter

Retrieve a specific model by its ID. Returns the same model object format as the list endpoint above.

Parameters Copy Section

Example request Copy Section

curl http://localhost:13305/v1/models/Qwen3-0.6B-GGUF

Response format Copy Section

Returns a single model object with the same fields as described in the models list endpoint above. For Omni collections (recipe: "collection.omni"), the object additionally carries components (ordered component names) and models (each component's full model object) — see the collection file documentation.

{
  "id": "Qwen3-0.6B-GGUF",
  "created": 1744173590,
  "object": "model",
  "owned_by": "lemonade",
  "checkpoint": "unsloth/Qwen3-0.6B-GGUF:Q4_0",
  "recipe": "llamacpp",
  "size": 0.38,
  "max_context_window": 40960,
  "downloaded": true,
  "suggested": true,
  "labels": ["reasoning"],
  "recipe_options": {
    "ctx_size": 8192,
    "llamacpp_args": "--no-mmap",
    "llamacpp_backend": "rocm"
  }
}

Error responses Copy Section

If the model is not found, the endpoint returns a 404 error:

{
  "error": {
    "message": "Model Qwen3-0.6B-GGUF has not been found",
    "type": "not_found"
  }
}

File: docs/api/README.md Copy Chapter

Lemonade Endpoints Spec

The Lemonade HTTP service provides a wide array of standards-compliant and custom endpoints.

Our design philosophy is:

1. Ensure that Lemonade works out-of-box with all popular local AI apps.
2. Prioritize using a pre-existing standard for functionality whenever possible.
3. Add sufficient custom functionality to enable developers to build highly polished experiences.

This spec details all supported endpoints. It is organized into pages that correspond to which organization (OpenAI, Ollama, Lemonade, etc.) defined the endpoints.

File: docs/dev/working-groups/auto-tune.md Copy Chapter

Working Group: Auto-Tune

Overview Copy Chapter

Lead: This working group is led by Michele Balistreri, whose handle is @bitgamma on GitHub and @mikkoph on Discord.

Background: Running a model well requires choosing the right backend and tuning performance parameters such as batch size, GPU layer offload, thread count, and context size. Today, users either guess, look up Reddit threads, or ask an LLM — approaches that are error-prone and often outdated due to knowledge cutoff. Lemonade already has a bench command that measures TTFT, TPS, and VRAM usage across backends and parameter combinations, and a recipe system that defines per-model configuration. What is missing is a mechanism to turn benchmark data into actionable, hardware-aware defaults that apply automatically.

Why: A user should install Lemonade and get good performance without needing to understand backend flags or run manual benchmarks. Hardware-aware defaults lower the barrier to entry and make Lemonade competitive with cloud-based alternatives where performance is abstracted away.

Goal: Enable Lemonade instances to self-optimize models and backends by detecting the machine's hardware profile and applying community-validated performance parameters. The end state is that a user pulls a model, loads it, and Lemonade selects the best backend and tuning flags for their hardware — with the option to override or fine-tune manually.

Contributing Copy Chapter

Please see the general contribution guidelines, then contact @mikkoph on Discord before getting started to discuss the roadmap.

Scope Copy Chapter

This working group focuses on performance parameters (batch size, GPU layers, threads, context size, backend selection) that are determined by hardware characteristics. Quality parameters such as temperature, top_p, and chat template are model- or use-case-dependent and are explicitly out of scope.

Roadmap Copy Chapter

Roadmap items are high-level objectives that may span multiple issues and PRs. Details can also be re-defined.

Phase 1: Archetype Detection & Profile Schema Copy Section

• Define the hardware archetype classification logic. Archetypes are identified by GPU family/architecture, VRAM capacity bucket, memory bandwidth tier, and whether memory is unified or discrete. This keeps the profile space bounded — two machines with the same archetype receive the same recommendations.
• Design the profile JSON schema. A profile maps an archetype ID to recommended performance parameters per backend (e.g., strix-halo-128gb → { "llamacpp_backend": "vulkan", "llamacpp_vulkan_args": { "-b 2048 -ub 1024" }, "vllm_args": { ... } }). The schema is backend-agnostic so it works for llama.cpp, FastFlowLM, vLLM, RyzenAI, and future backends.
• Implement archetype detection in lemond, using data already collected by the system-info endpoint (GPU name, VRAM, bandwidth, unified vs. discrete memory).
• Ship an initial set of hand-curated profiles for common hardware configurations (e.g., Strix Halo, Radeon 780M, Apple M3).

Phase 2: Auto-Apply at Load Time Copy Section

• On startup, lemond detects the machine archetype and caches it.
• When loading a model, the router checks for archetype-specific profile overrides and merges them into the model's recipe options. A fallback chain ensures graceful degradation: exact archetype + model overlay → archetype defaults for the backend → recipe defaults (current behavior).
• Optional per-model overlays allow model-specific tuning on top of global archetype defaults (for cases where certain models benefit from different flags on the same hardware).
• lemonade status displays the detected archetype and any active auto-tune overrides.
• CLI or config option to enable/disable auto-tune (enabled by default).

Phase 3: Profile Distribution Copy Section

• Profiles are fetched at runtime from a remote source, allowing updates without reinstalling Lemonade. A set of default profiles ships bundled with Lemonade as a fallback when no network is available.
• Profile cache is stored locally with versioning; stale profiles are re-fetched periodically.
• lemonade bench --submit generates a structured benchmark contribution from local benchmark runs. The submission workflow (upload endpoint, PR-based contribution, or other) will be designed based on infrastructure study.

Phase 4: Community Data Collection & Curation Copy Section

• Tooling to merge community-submitted benchmarks into curated profiles. This includes selecting the best-performing parameter sets per scenario, validating consistency across submissions, and detecting outliers.
• Documentation for the contribution workflow so community members can benchmark their hardware and contribute results.
• Profiles cover a broad range of hardware archetypes across all supported backends.

Phase 5: Adaptive Tuning Copy Section

• Runtime monitoring: if observed performance (TPS, TTFT) deviates significantly from profile expectations, lemond can suggest re-benchmarking or flag the profile for review.
• "Learn from this run" — after a session, offer to save locally-observed good parameters into the user's config for persistence.
• To be continued, based on learnings from the above.