## File: README.md --- ## Contents - [Why this exists](#why-this-exists) - [Supported providers](#supported-providers) - [Compatible CLIs & coding agents](#compatible-clis--coding-agents) - [How it compares](#how-it-compares) - [Features](#features) - [Quick start](#quick-start) - [Desktop app](#desktop-app) - [Works with OpenAI-compatible clients](#works-with-openai-compatible-clients) - [Languages](#languages) - [Premium (live catalog)](#premium-live-catalog) - [Using the API](#using-the-api) - [Screenshots](#screenshots) - [How it works](#how-it-works) - [Limitations](#limitations) - [Contributing](#contributing) - [Disclaimer](#disclaimer) **Guides:** [Install & deploy](docs/install.md) · [API reference](docs/api.md) · [Clients & coding agents](docs/clients.md) · [Prompt compression](docs/compression.md) · [Architecture & internals](docs/architecture.md) · [Documentation index](docs/README.md) · [Contributor guide](CONTRIBUTING.md) ## Why this exists Every serious AI lab now offers a free tier, a few million tokens a month, a few thousand requests a day. On its own each tier is a toy. Stacked together, they add up to roughly **4 billion tokens per month** of working inference capacity, across **251 model families / 358 provider endpoints** from small-and-fast to reasonably capable. The problem is that stacking them by hand is painful: twenty-nine different SDKs, twenty-nine different rate limits, twenty-nine places a request can fail. FreeLLMAPI collapses that into one OpenAI-compatible endpoint. Point any OpenAI client library at your local server, and it routes transparently across whichever providers you've added keys for. And the free-tier landscape shifts weekly: providers launch models, retire them, and change quotas without notice. FreeLLMAPI tracks all of that for you. The router pulls a signed model catalog from [freellmapi.co](https://freellmapi.co) on its own, so your install keeps up without a `git pull`. See [Premium (live catalog)](#premium-live-catalog) for how fast it keeps up. ## Supported providers Plus a **custom** provider — point chat, embedding, image, or audio models at any OpenAI-compatible endpoint (llama.cpp, LM Studio, vLLM, a local Ollama, or a remote gateway) from the Keys page. The full, always-current list lives at **[freellmapi.co/models](https://freellmapi.co/models.html)** with per-model rate limits, context windows, and free-token budgets. ## Compatible CLIs & coding agents Most of these configure themselves with one command — `npx freellmapi setup-claude`, `setup-codex`, `setup-aider`, and ten more generators that fetch your live catalog, back up existing config, and never clobber what's already there. Claude Code and Codex also get zero-persistence launchers (`freellmapi launch`, `freellmapi launch-codex`) that inject credentials into the child process only. Zed and JetBrains AI connect through the opt-in [Ollama emulation](docs/clients.md#ollama-clients); Gemini CLI speaks its native wire on `/v1beta`. Per-tool recipes, the setup CLI reference, revocable URL tokens for headerless clients, and the MCP server all live in **[Clients & coding agents →](docs/clients.md)** ## How it compares Based on public documentation, July 2026 — corrections welcome. ## Features - **Every OpenAI surface** — `/v1/chat/completions`, `/v1/responses` (what Codex CLI needs), `/v1/completions` (editor ghost-text autocomplete), `/v1/images/generations`, `/v1/audio/speech`, `/v1/embeddings`, and `/v1/models` — streaming and non-streaming, from the official SDKs or any OpenAI-compatible client. [API reference →](docs/api.md) - **Anthropic Messages API** — `/v1/messages` speaks Anthropic's wire format over the same router, so **Claude Code** and the official Anthropic SDKs run against your free pool. [Details →](docs/api.md#anthropic--claude-clients) - **Native Gemini + Ollama surfaces** — Gemini CLI can use `/v1beta` (`generateContent`, streaming, token counting, models), while opt-in Ollama emulation serves NDJSON chat/generate, tags, metadata, and embeddings for Zed, JetBrains, and other local-model clients. - **Fusion (multi-model synthesis)** — request the virtual `fusion` model and the router fans your prompt out to a panel of diverse free models in parallel, then a judge model synthesizes one answer from the drafts. [Details →](docs/api.md#fusion-multi-model-synthesis) - **Image generation & text-to-speech** — `/v1/images/generations` and `/v1/audio/speech` route across the providers that serve media models, including custom OpenAI-compatible media endpoints. - **Tool calling & structured outputs** — OpenAI-style `tools` round-trip across providers (plain-text tool calls are rescued into real `tool_calls`), plus `response_format`, `seed`, `logprobs`, penalties, and the rest of the sampling params passed through per provider. - **Smart routing, six strategies** — live per-model speed/capability/reliability scores rank your chain; automatic fallover retries the next model on 429/5xx with cooldowns and key rotation. [Routing in detail →](docs/architecture.md#routing-in-detail) - **Unified models & profiles** — the same model on several providers collapses into one entry with strict in-group failover; named fallback-chain profiles (a coding chain, a vision chain) switch from the dashboard or per request via `auto:`. - **Per-key rate tracking** — RPM/RPD/TPM/TPD counters per `(platform, model, key)` that learn providers' reported ceilings, so routing always stays under every cap. - **Self-updating model catalog** — the router syncs a signed catalog from freellmapi.co twice a day: new models, quota changes, and provider quirk fixes land automatically. [Premium →](#premium-live-catalog) - **Sticky sessions & context handoff** — conversations stay on one model for 30 minutes; an optional compact handoff note keeps the thread coherent when a mid-chat switch does happen. [Details →](docs/clients.md#context-handoff) - **Prompt compression (opt-in)** — a shared, fail-open request pipeline can deduplicate prompts, filter tool output, compact repeated JSON, and trim stale context before cache lookup and routing. [Details →](docs/compression.md) - **Encrypted keys, one token out** — provider keys are AES-256-GCM encrypted in SQLite and decrypted in-memory per request; your apps only ever see a single unified `freellmapi-…` bearer token. - **Admin dashboard & analytics** — React UI to manage keys, reorder the chain, run a playground, and read p50/p95/TTFT analytics over 24h–90d windows; login-gated, dark/light themes, [60 languages](#languages). - **MCP server & interactive docs** — agents can introspect usable models, provider health, and routing strategy over `/mcp`; a dependency-free OpenAPI viewer lives at `/v1/docs`. [Coding agents →](docs/clients.md) - **Ops niceties** — opt-in response cache, encrypted DB backups, periodic key health checks, bulk key import/export, declarative startup config. [Install & deploy →](docs/install.md) - **Runs anywhere Node 20+ runs** — Windows, macOS, Linux servers, or a small ARM SBC (Raspberry Pi included). ~40 MB RSS at idle behind PM2 / systemd / whatever supervisor you prefer. The scope is deliberately narrow — see [what's not supported yet](docs/architecture.md#not-yet-supported). ## Quick start **One-liner** (Docker required — sets up `~/freellmapi`, generates an encryption key, pulls the image, and starts the container): ```bash curl -fsSL https://freellmapi.co/install.sh | bash ``` Prefer to read before you pipe to bash? [The script is here](https://freellmapi.co/install.sh). Re-running it is safe: your `.env` (and encryption key) is preserved and the container updates to `:latest`. Open http://localhost:3001, add your provider keys on the **Keys** page, reorder the **Fallback Chain** to taste, and grab your unified API key from the **Keys** page header. That unified key is what you point your OpenAI SDK at. On Windows, the easiest path is the desktop **[`.exe` installer from Releases](https://github.com/tashfeenahmed/freellmapi/releases/latest)** (below). On Android, see the experimental [Termux guide](docs/install/android-termux.md). Everything else — Docker Compose, local development, declarative startup config, production builds, LAN access, and backups — is in **[docs/install.md](docs/install.md)**. ## Desktop app A native menu-bar app lives in [`desktop/`](./desktop): the entire router + dashboard running locally from your tray, with a glass popover showing live request stats. **[Download from Releases](https://github.com/tashfeenahmed/freellmapi/releases/latest)** — the macOS `.dmg` and the Windows `.exe` installer are attached to every release. No account or password to set up: the only credential you need is the unified API key from the tray popover. Build-from-source steps and where your data lives: [docs/install.md](docs/install.md#desktop-app). ## Works with OpenAI-compatible clients Anything that can target an OpenAI-compatible base URL works: set it to `http://localhost:3001/v1` with the unified key from the dashboard. **Claude Code**, **Codex CLI**, **Cline / Roo Code**, **Continue** (including inline autocomplete), **Aider**, **opencode**, and **Cursor** each have a short recipe in **[docs/clients.md](docs/clients.md)** — and the router doubles as an MCP server your agents can introspect mid-session. The fastest setup is generated from the models available on your live server: ```bash npx freellmapi setup-claude --url http://localhost:3001 --api-key ``` Every generator supports `--dry-run`, creates a timestamped backup before changing an existing file, and merges into the user's configuration. Launchers keep credentials out of config files entirely: `npx freellmapi launch` for Claude Code and `npx freellmapi launch-codex` for Codex. | Agent | Automated setup | Base URL | | --- | --- | --- | | Claude Code | `setup-claude` | root | | Codex CLI | `setup-codex` | `/v1` | | Cline | `setup-cline` | `/v1` | | Continue | `setup-continue` | `/v1` | | Aider | `setup-aider` | `/v1` | | OpenCode | `setup-opencode` | `/v1` | | Goose | `setup-goose` | `/v1` | | Qwen Code | `setup-qwen` | `/v1` (or native `/v1beta`) | | Roo / Kilo / Crush | `setup-roo` / `setup-kilo` / `setup-crush` | `/v1` | | Cursor | `setup-cursor` guide | public `/v1` URL | FreeLLMAPI is local-first and single-user by design. Your provider keys stay in your SQLite database, encrypted at rest, and requests go from your machine to the upstream providers you enabled. ## Languages The dashboard ships in **60 languages** (the desktop tray menu in 6). The UI auto-detects your browser/system language on first load and you can switch any time from **⋯ → Settings**; the choice is remembered. Right-to-left languages (العربية, עברית, فارسی, اردو) flip the whole layout automatically, and only the active language's dictionary is loaded — the rest never touch your bandwidth. The full list of locales lives in [`client/src/i18n/locale-config.ts`](./client/src/i18n/locale-config.ts). The original six locales are human-reviewed; the newer ones are machine- translated and improve as native speakers send corrections — a one-string PR is a great first contribution. Translations live in [`client/src/i18n/locales/`](./client/src/i18n/locales) as flat JSON files. To fix a string, edit the value in the locale's JSON file. To add a language, copy `en.json`, translate the values, and register the locale in `client/src/i18n/locale-config.ts` (and `desktop/src/i18n.ts` for the tray strings); `npm test` checks every locale for key/placeholder parity — PRs welcome. ## Premium (live catalog) The router keeps its model catalog fresh on its own: it pulls a signed catalog from [freellmapi.co](https://freellmapi.co) twice a day and applies new models, quota changes, and provider quirk fixes to your local DB. Your own enable/disable choices and custom providers are never touched, and every download is verified against a pinned Ed25519 key before it is applied. The catalog currently tracks **29 providers**, **251 model families**, **358 provider/model endpoints**, and roughly **4 billion tokens per month** of listed free-tier capacity. Browse the full set at **[freellmapi.co/models](https://freellmapi.co/models.html)**. Premium keeps that signed catalog live on every router you run. When a provider launches a strong free model, quietly tightens a quota, or breaks a wire format, live-feed routers receive the update as soon as we ship it. **[Go live at freellmapi.co →](https://freellmapi.co/?utm_source=github&utm_medium=readme&utm_campaign=premium&utm_content=readme_bottom#pricing)** - $19/year or $49 once, lifetime. Stripe checkout; cancel anytime, self-serve. - One `fla_` key covers every router you run: desktop, homelab, Raspberry Pi. - Activate in the dashboard under **Premium**; cancel or manage billing self-serve at [freellmapi.co/manage](https://freellmapi.co/manage). - The router itself stays MIT-licensed and fully free, forever. Premium is only the live feed, and it's what funds the daily model testing and catalog maintenance that keeps the catalog working. The catalog server never sees your prompts, completions, or provider keys — the router stays fully self-hosted either way. ## Using the API ```python from openai import OpenAI client = OpenAI( base_url="http://localhost:3001/v1", api_key="freellmapi-your-unified-key", ) resp = client.chat.completions.create( model="auto", # let the router pick; or "auto:fast", "auto:smart", a profile, or a model id messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}], ) print(resp.choices[0].message.content) print("Routed via:", resp.headers.get("x-routed-via")) ``` Streaming, the `auto:*` routing strategies, tool calling, vision input, Gemini Google Search grounding, embeddings, and the Anthropic Messages surface — with curl and Python examples for each — are all in **[docs/api.md](docs/api.md)**. Every response carries an `X-Routed-Via: /` header so you can see which provider actually served it. ## Screenshots ### Models Pick a routing strategy and watch the monthly token budget fill across the whole provider fleet. Every model shows live reliability, speed, and intelligence scores — the order below is how requests route right now. ### Keys Manage provider credentials and grab the unified API key your apps connect with. Each key shows a status dot and when it was last health-checked. ### Playground Send a chat completion through the router and see which provider served it, with the model ID and latency printed right on the message. Attach files by button, drag-and-drop, or paste: images (PNG/JPEG/WebP/GIF) are downscaled in the browser and sent as image content parts to a vision-capable model, and text files (TXT/MD/CSV/JSON/LOG) are inlined into the prompt as fenced blocks. ### Analytics Request volume, success rate, tokens in and out, average latency, and per-provider breakdowns over 24h / 7d / 30d / 90d windows. ## How it works One request in, the best free model out: the router picks the highest-priority model with a healthy key that's under all its rate limits, decrypts the key in memory, and calls the provider — on a 429/5xx it cools that key down and retries the next model in your chain. The component walkthrough, routing internals, and operational details live in **[docs/architecture.md](docs/architecture.md)**. ## Limitations Stacking free tiers has real trade-offs: no frontier models, variable latency, no SLA — and the effective intelligence of the endpoint dips late in the day as top models hit their daily caps, then resets at UTC midnight. Read the honest list in **[docs/architecture.md#limitations](docs/architecture.md#limitations)** before building anything real on this. ## Contributing Contributors very welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for the dev loop, PR expectations, and the policy on AI/LLM-assisted contributions (short version: welcome, same quality bar as any other PR). Good first PRs: - **Add a provider** — copy `server/src/providers/openai-compat.ts` as a template, wire it into `server/src/providers/index.ts`, seed its models in `server/src/db/index.ts`, add a test in `server/src/__tests__/providers/`. - **Add an endpoint** — moderations and other OpenAI-compatible surfaces. The provider base class can grow new methods; adapters declare which they support. - **Improve the router** — cost-aware routing (cheapest-healthy-fastest tradeoffs), better latency-weighted priority, regional pinning. - **Dashboard polish** — charts on the Analytics page, key rotation UX, batch import of keys from `.env`. - **Docs** — more examples, client library snippets for Go/Rust/etc., a deployment recipe for Docker or Fly. `npm install && npm run dev` gets you the server on :3001 and the dashboard on :5173, both with HMR. For a repeatable setup, use `./scripts/dev-bootstrap.sh` on Bash or `.\scripts\dev-bootstrap.ps1` on PowerShell; each preserves an existing `.env`. PRs should include a test, keep the existing suite green (`npm test`), and match the `.editorconfig` / tsconfig defaults already in the repo. Database migration workflow and the full contributor loop are in [CONTRIBUTING.md](./CONTRIBUTING.md). ### Contributors [](https://github.com/moaaz12-web) [](https://github.com/lukasulc) [](https://github.com/VinhPhamAI) [](https://github.com/deadc) [](https://github.com/zhangyu1324) [](https://github.com/chongjiazhen) [](https://github.com/vjsai) [](https://github.com/long2ice) [](https://github.com/sadesguy) [](https://github.com/hodlmybeer69-bit) [](https://github.com/phoenixikkifullstack) [](https://github.com/jtbrennan-git) [](https://github.com/praveenkumarpranjal) [](https://github.com/nordbyte) [](https://github.com/mybropro) [](https://github.com/danscMax) [](https://github.com/jhash) [](https://github.com/JammyJames1234) [](https://github.com/coffcoe) [](https://github.com/Sumit4codes) [](https://github.com/meliani) [](https://github.com/thedavidweng) [](https://github.com/bharvey42) [](https://github.com/yuvrxj-afk) [](https://github.com/Tushar49) [](https://github.com/nicyoong) [](https://github.com/Aldo-f) [](https://github.com/Tazrif-Raim) [](https://github.com/m1nuzz) [](https://github.com/suantea) [](https://github.com/OhOkThisIsFine) [](https://github.com/LoneRifle) [](https://github.com/ita333) [](https://github.com/barbotkonv) [](https://github.com/Naster17) [](https://github.com/StealthTensor) [](https://github.com/EmranAhmed) [](https://github.com/itsfuad) [](https://github.com/RobinHoodO) [](https://github.com/hmm183) [](https://github.com/duemilionidieuro-bot) [](https://github.com/cagedbird043) [](https://github.com/jasnoorgill) [](https://github.com/Joey9024) [](https://github.com/AskingConical) [](https://github.com/ProAlit) [](https://github.com/hjhhoni) [](https://github.com/immanuelsavio) [](https://github.com/Slyker) [](https://github.com/wells1013) [](https://github.com/evgkrsk) [](https://github.com/aaronjmars) [](https://github.com/Robs87) [](https://github.com/dashitongzhi) [](https://github.com/QingJ01) [](https://github.com/3215) [](https://github.com/saifulaiub123) [](https://github.com/PietFourie) [](https://github.com/mhmdkrmabd) [](https://github.com/DemeulemeesterxMaxime) [](https://github.com/HoodBlah) [](https://github.com/SeanPedersen) [](https://github.com/andersmmg) [](https://github.com/chirag127) [](https://github.com/allababbot) [](https://github.com/johan-droid) [](https://github.com/redenfire) [](https://github.com/itzpingcat) [](https://github.com/kairwang01) [](https://github.com/gongjurenzhangwei) [](https://github.com/jsonring) [](https://github.com/1029734570) [](https://github.com/86TheCactus) [](https://github.com/AmiroKD) [](https://github.com/ecryptomillionaire-dev) [](https://github.com/4riful) [](https://github.com/fix2015) [](https://github.com/iisyw) [](https://github.com/xsfhacg) [](https://github.com/noobix) [](https://github.com/nandukmelath) [](https://github.com/NirvanaCh7) [](https://github.com/Mohamed3nan) [](https://github.com/Arman-Espiar) [](https://github.com/MetaMysteries8) [](https://github.com/lujun880726) [](https://github.com/qq97693453) [](https://github.com/emv33) [](https://github.com/ousamabenyounes) [](https://github.com/yfdyh000) [](https://github.com/s-uryansh) [](https://github.com/arsalanyavari) [](https://github.com/RoboMWM) [](https://github.com/gaurang-py) [](https://github.com/ddy4633) [](https://github.com/UrbsKali) [](https://github.com/hb-0) [](https://github.com/xyblue135) [](https://github.com/Icesenator) [](https://github.com/ZER0-auto) ## Disclaimer **This project is for personal experimentation and learning, not production.** Free tiers exist so developers can prototype against them; they aren't a stable, supported inference substrate and shouldn't be treated as one. If you build something real on top of FreeLLMAPI, swap in a paid API before you ship. Your relationship with each upstream provider is governed by the terms you accepted when you created your account — those terms still apply when the traffic is proxied through this project, and you're responsible for complying with them. How each provider's ToS views a personal, single-user proxy — reviewed provider by provider in May 2026 — is in [docs/architecture.md#terms-of-service-review](docs/architecture.md#terms-of-service-review). ## License [MIT](./LICENSE) --- ## File: client/README.md # React + TypeScript + Vite This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules. Currently, two official plugins are available: - [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Oxc](https://oxc.rs) - [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) ## React Compiler The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation). ## Expanding the ESLint configuration If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules: ```js export default defineConfig([ globalIgnores(['dist']), { files: ['**/*.{ts,tsx}'], extends: [ // Other configs... // Remove tseslint.configs.recommended and replace with this tseslint.configs.recommendedTypeChecked, // Alternatively, use this for stricter rules tseslint.configs.strictTypeChecked, // Optionally, add this for stylistic rules tseslint.configs.stylisticTypeChecked, // Other configs... ], languageOptions: { parserOptions: { project: ['./tsconfig.node.json', './tsconfig.app.json'], tsconfigRootDir: import.meta.dirname, }, // other options... }, }, ]) ``` You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules: ```js // eslint.config.js import reactX from 'eslint-plugin-react-x' import reactDom from 'eslint-plugin-react-dom' export default defineConfig([ globalIgnores(['dist']), { files: ['**/*.{ts,tsx}'], extends: [ // Other configs... // Enable lint rules for React reactX.configs['recommended-typescript'], // Enable lint rules for React DOM reactDom.configs.recommended, ], languageOptions: { parserOptions: { project: ['./tsconfig.node.json', './tsconfig.app.json'], tsconfigRootDir: import.meta.dirname, }, // other options... }, }, ]) ``` --- ## File: desktop/README.md # FreeLLMAPI Desktop Lean Electron menu-bar app for [freellmapi](https://github.com/tashfeenahmed/freellmapi): runs the whole router locally on `127.0.0.1:31415`, with a dark-glass tray popover (live 24h request chart, quick stats, copy URL/key) and the full dashboard in a native-feeling window. The app lives in the main repo (`desktop/`) and builds against the server and client sources in the same checkout — no extra setup, no published binaries: **you build it yourself** (a few minutes, commands below). ## Prerequisites - Node.js 20+ - A native build toolchain for `better-sqlite3`: - **macOS** — Xcode Command Line Tools (`xcode-select --install`) - **Windows** — Visual Studio Build Tools ("Desktop development with C++") ## Build it yourself From the **repo root** (one-time client build + package): ```bash npm install npm install --prefix desktop # install desktop dependencies npm run desktop:dist # macOS → desktop/dist-electron/FreeLLMAPI-…-arm64.dmg npm run desktop:dist:win # Windows → desktop/dist-electron/FreeLLMAPI-…-x64 installer ``` > Locally built apps don't carry the macOS quarantine attribute or Windows > Mark-of-the-Web, so they launch without Gatekeeper/SmartScreen warnings — > no code signing needed. > > The Windows build is config-complete but largely untested — issues and PRs > welcome. ## Develop ```bash cd desktop npm install npm run rebuild:native # better-sqlite3 against Electron's ABI (dev loop only) npm run dev # bundle + launch ``` The dev run serves the repo's own `client/dist` — build it first (`npm run build -w client` from the root) or use `npm run desktop:dev` from the root, which does both. `FREEAPI_REPO=/path/to/checkout` overrides the client/server source location if you ever need a different one. UI iteration helpers (dev only): `FREEAPI_SHOT=1 npx electron .` captures the popover + dashboard to /tmp and quits; `FREEAPI_SHOT=hold` pins the popover open for real-screen captures. ## Notes - DB + config live in `~/Library/Application Support/FreeLLMAPI/` (macOS) / `%APPDATA%/FreeLLMAPI/` (Windows). - The server binds loopback only; default port 31415, scan-up on conflict, resolved port persisted. - Do not run plain `npm rebuild` here — it rebuilds better-sqlite3 for the system Node ABI and breaks the Electron dev loop (`npm run rebuild:native` fixes it). Packaging rebuilds natives itself. --- ## File: docker/README.md # Docker Guide Docker Compose is the recommended way to run FreeLLMAPI for personal use. The container serves the Express API and the built React dashboard from one process on port 3001, with SQLite persisted in a named volume. ## Prerequisites - Docker - Docker Compose - OpenSSL for generating `ENCRYPTION_KEY` ## Quick Start Create a `.env` file with a 32-byte encryption key: ```bash ENCRYPTION_KEY="$(openssl rand -hex 32)" printf "ENCRYPTION_KEY=%s\nPORT=3001\n" "$ENCRYPTION_KEY" > .env ``` Start the app: ```bash docker compose up -d ``` Open http://localhost:3001, add provider keys on the **Keys** page, then use the generated `freellmapi-...` key with any OpenAI-compatible client. ## Example API Call ```bash curl http://localhost:3001/v1/chat/completions \ -H "Authorization: Bearer freellmapi-your-unified-key" \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "messages": [{"role": "user", "content": "Say hello from FreeLLMAPI."}] }' ``` ## Operations Check status: ```bash docker compose ps ``` Tail logs: ```bash docker compose logs -f freellmapi ``` Stop the app: ```bash docker compose down ``` Update to the latest GHCR image after a release: ```bash docker compose pull docker compose up -d ``` Rebuild locally from source: ```bash docker compose up -d --build ``` ## Configuration | Variable | Required | Default | Description | | --- | --- | --- | --- | | `ENCRYPTION_KEY` | Yes | None | 64-character hex key used to encrypt provider API keys at rest. Generate it once and keep it stable. | | `PORT` | No | `3001` | Host port exposed by Docker Compose. The container listens on port 3001. | | `FREEAPI_DB_PATH` | No | `/app/server/data/freellmapi.db` | SQLite file path. Set this when your host only persists one mounted directory. | | `FREEAPI_DB_BACKUP_PATH` | No | None | Local encrypted backup file. Restored on startup if the DB file is missing, then refreshed while the app runs. | | `FREEAPI_DB_BACKUP_URL` | No | None | HTTP(S) encrypted backup target. Startup uses `GET`; periodic backups use `PUT`. | | `FREEAPI_DB_BACKUP_TOKEN` | No | None | Optional bearer token for `FREEAPI_DB_BACKUP_URL`. | | `FREEAPI_DB_BACKUP_KEY` | No | `ENCRYPTION_KEY` | 64-character hex key for backup encryption. Use a separate stable key if possible. | | `FREEAPI_CONFIG_PATH` | No | None | JSON config file applied idempotently after migrations on every boot. | | `FREEAPI_CONFIG_JSON` | No | None | Inline JSON config. Takes precedence over `FREEAPI_CONFIG_PATH`. | The `freellmapi-data` volume stores SQLite data at `/app/server/data`. Keep the same volume and `ENCRYPTION_KEY` when upgrading, otherwise existing encrypted provider keys cannot be decrypted. Example `freellmapi.config.json`: ```json { "keys": [ { "platform": "groq", "key": "gsk_...", "label": "main" } ], "customProviders": [ { "baseUrl": "http://host.docker.internal:11434/v1", "label": "Ollama", "models": [ { "model": "llama3.1:8b", "displayName": "Local Llama", "supportsTools": true } ] } ], "routing": { "strategy": "balanced" } } ``` ## Published Image Images are published to GitHub Container Registry: ```bash docker pull ghcr.io/tashfeenahmed/freellmapi:latest ``` The Docker workflow builds pull requests without pushing. After this repository receives the workflow on `main`, pushes to `main` and version tags publish images to GHCR automatically. --- ## File: docs/README.md **English** · [简体中文](i18n/zh-CN/docs/README.md) # FreeLLMAPI documentation The root [README](../README.md) is the product overview; the detailed guides live here. ## Guides - **[Install & deploy](install.md)** — quick start, Docker Compose, local development, declarative startup config, the Docker image, backups, the desktop app, and where your data lives. - **[API reference](api.md)** — chat completions, `auto:*` routing strategies, streaming, tool calling, vision, Gemini Google Search grounding, embeddings, response headers, and the Anthropic Messages surface. - **[Clients & coding agents](clients.md)** — OpenAI-compatible clients, recipes for Claude Code / Codex CLI / Cline / Continue / Aider / opencode / Cursor, the MCP server, editor autocomplete, and Context Handoff. - **[Prompt compression](compression.md)** — request-side modes, safeguards, per-request controls, custom tool-output filters, statistics, and preview APIs. - **[Architecture & internals](architecture.md)** — how the router works, routing and operational details, what's not supported, honest limitations, and the provider Terms-of-Service review. ## More - [Android with Termux](install/android-termux.md) — experimental local installation using Node's built-in SQLite driver. - [Docker deployment](../docker/README.md) — container configuration and persistent storage. - [Desktop app](../desktop/README.md) — build and package the Electron application. - [Contributor guide](../CONTRIBUTING.md) — development loop, testing expectations, and contribution policy. - [Database migrations](../server/src/db/README.md) — create, apply, inspect, and roll back schema migrations. ## Website assets in this directory - [`index.html`](index.html) — project landing page. - [`install.sh`](install.sh) — Unix Docker bootstrap script. - [`install.ps1`](install.ps1) — PowerShell bootstrap script. - [`success.html`](success.html) — post-install success page.