### Providers/Subscription Based Providers
# Subscription-based providers
Reuse a paid plan you already have: ChatGPT Plus, Claude Max, GitHub Copilot, ModelArk Coding Plan, Command Code, Qwen Token Plan, Xiaomi MiMo Token Plan, Kimi Coding Plan, GLM Coding Plan, Google sign-in, MiniMax Coding Plan, Kiro, Grok, Ollama Cloud, or OpenCode Go.
If you already pay for one of the plans below, Manifest can route through that subscription instead of an API key. Auth is browser OAuth, device code, or a subscription token depending on the provider.
## Supported subscriptions
| Provider | Plan | Auth flow | Model catalog |
| ----------------------------------------------------------- | --------------------------- | -------------------- | ---------------------------------------------------- |
| [OpenAI](https://openai.com) | ChatGPT Plus/Pro/Team | OAuth (browser) | Fixed ChatGPT and Codex model list |
| [Anthropic](https://www.anthropic.com) | Claude Max / Pro | Setup token | `claude-opus-4`, `claude-sonnet-4`, `claude-haiku-4` |
| [GitHub Copilot](https://github.com/features/copilot) | GitHub Copilot subscription | Device code | Fixed Copilot model list |
| [BytePlus](https://www.byteplus.com/en/activity/codingplan) | ModelArk Coding Plan | Subscription token | Fixed ModelArk Coding Plan model list |
| [Command Code](https://commandcode.ai/studio) | Command Code subscription | Subscription token | Dynamic Provider API catalog |
| [Qwen](https://home.qwencloud.com/api-keys) | Qwen Token Plan | Subscription token | Dynamic Token Plan catalog |
| [Xiaomi MiMo](https://platform.xiaomimimo.com) | MiMo Token Plan | Subscription token | Fixed MiMo Token Plan model list |
| [Moonshot](https://www.kimi.com/code/console) | Kimi Coding Plan | Subscription token | Fixed Kimi Coding Plan model list |
| [Z.ai](https://z.ai) | GLM Coding Plan | Subscription token | Fixed GLM Coding Plan model list |
| [Google](https://ai.google.dev) | Sign in with Google | OAuth (browser) | Fixed Gemini CodeAssist model list |
| [MiniMax](https://www.minimax.io) | MiniMax Coding Plan | Device code or token | Fixed MiniMax Coding Plan model list |
| [Kiro](https://app.kiro.dev) | Kiro subscription | Device code | Fixed Kiro model list |
| [xAI](https://x.ai) | Grok subscription | OAuth (browser) | Dynamic xAI model catalog |
| [Ollama Cloud](https://ollama.com) | Ollama Cloud subscription | Subscription token | Dynamic Ollama Cloud tag catalog |
| [OpenCode Go](https://opencode.ai) | OpenCode Go (beta) | Subscription token | Dynamic OpenCode Go catalog |
## Connect a subscription
1. Open the Routing page in the dashboard.
2. Click the provider tile.
3. Pick the Subscription tab.
4. Complete the provider's auth flow.
Providers that support both API keys and subscriptions show both tabs. Subscription-only providers show only the subscription flow. In routing pickers, subscription rows appear only after a usable subscription credential is saved (`has_api_key=true`).
## How auth works per provider
### OpenAI
Manifest uses browser OAuth for ChatGPT Plus, Pro, and Team accounts. Once approved, requests route through the ChatGPT Codex backend instead of the public OpenAI API. Tokens refresh automatically.
### Anthropic
Run `claude setup-token` and paste the `sk-ant-oat` token into Manifest. Requests carry Anthropic subscription auth headers. Subscription mode does not support prompt caching or batching.
### GitHub Copilot
Manifest uses GitHub's device-code flow. Manifest gives you a short user code, you open `https://github.com/login/device`, paste the code, and approve access. Requests then route to the Copilot chat or responses endpoint with a short-lived token that Manifest refreshes.
### BytePlus
Paste a ModelArk Coding Plan API key from the BytePlus console. Manifest routes OpenAI-compatible and Anthropic-compatible coding-plan requests through BytePlus ModelArk endpoints.
### Command Code
Paste a Command Code API key from Command Code Studio. Command Code is subscription-only in Manifest and requires Command Code Pro or higher. Manifest discovers models dynamically from Command Code's public Provider API catalog.
### Qwen
Paste a Qwen Token Plan API key. Token Plan keys use the `sk-sp-` prefix. Manifest discovers available models from Qwen's Token Plan endpoint and can route both chat-completions and responses-style requests where supported.
### Xiaomi MiMo
Paste a MiMo Token Plan API key. Token Plan keys use the `tp-` prefix. Manifest routes through MiMo's OpenAI-compatible Token Plan endpoint and lets self-hosted setups choose the China, Singapore, or Europe token-plan host.
### Moonshot
Paste a Kimi Code API key from the Kimi Code console. Requests route through Kimi Code's Anthropic-compatible endpoint for the fixed `k3`, `k3-256k`, `kimi-for-coding`, and `kimi-for-coding-highspeed` model catalog.
### Z.ai
Paste a GLM Coding Plan token from Z.ai. Manifest routes to the Z.ai coding-plan endpoint. Self-hosted setups can choose the global or China Mainland endpoint when both are available.
### Google
Manifest uses browser OAuth for Google's CodeAssist flow. The subscription route uses a fixed Gemini CodeAssist model list rather than the public Gemini API-key catalog.
### MiniMax
Manifest supports the MiniMax Coding Plan device-code flow. Users with a Coding Plan token can also paste the token directly when the UI offers the token alternative.
### Kiro
Manifest uses Kiro's device-code flow. The Kiro subscription exposes `kiro/auto` plus curated Claude, DeepSeek, MiniMax, GLM, and Qwen routes behind one subscription.
### xAI
Manifest uses browser OAuth for Grok subscriptions and fetches the model list dynamically from xAI's OpenAI-compatible models endpoint.
### Ollama Cloud
Paste an Ollama Cloud API key. Ollama Cloud accepts generic API-key formats; Manifest does not require a fixed prefix. Models are discovered dynamically from the cloud tag catalog.
### OpenCode Go
Sign in to OpenCode Go, copy your API key, and paste it into Manifest. Model discovery is dynamic. Depending on the model, Manifest can route through OpenAI-compatible or Anthropic-compatible wire formats.
## Why mix subscriptions and API keys
A common setup is a subscription as the primary route for predictable monthly cost, plus API-key providers as fallbacks for plan limits, provider outages, or models the subscription does not include. Pin subscription models to routing tiers and add API-key models to the fallback list. Manifest handles the switch.
---
### Autofix Self Healing Poc
# Autofix (self-healing requests) — POC specification
**Status:** Backend + frontend implemented, green, verified live · **Last updated:** 2026-07-01
> **⚠️ This spec describes the original aspirational design and has diverged from
> what shipped. Where they disagree, the code is the source of truth.** Known
> divergences:
>
> - **No multi-attempt loop, no budget.** Healing is a **single attempt**
> (`runHealOnce` — one heal, one reforward). There is no retry budget, so
> `autofix_max_attempts`, the `maxAttempts` endpoint field, and the
> `AUTOFIX_DEFAULT_MAX_ATTEMPTS` env var referenced below **do not exist**.
> - **`agents.autofix_enabled` is nullable** (`boolean | null`, not
> `default: false`). `NULL` inherits the deployment-mode default (ON in cloud,
> OFF in self-hosted); an explicit `true`/`false` wins.
> - **Production default client is `NoopHealingClient`** (inert), not the mock —
> the mock runs only in dev/test. See §4.
> - **`agent_messages` also has `autofix_phoenix`** (jsonb — the Phoenix
> `{issueId, patchId, healAttemptId}`), in addition to the columns listed in §5.2.
> - Autofix is now generally available. The early-access waitlist and tenant
> entitlement gate have been retired; only the per-agent toggle remains.
> **Implementation status.** Full stack built against the real **Phoenix**
> contract (§4) and passing: backend unit suite green (6900 tests), frontend suite
> green (3877 tests), every new `routing/autofix/*` file + the new UI at 100%
> coverage on changed lines. The healing client defaults to an in-process mock
> (implements the MVP `max_tokens` rename) until `AUTOFIX_HEALING_URL` points at a
> real Phoenix.
>
> **Verified live** (`/serve`, cloud mode): migrations applied, the Routing-page
> Autofix toggle persists to the DB, and the real
> `AutofixService` + mock Phoenix heal the MVP `max_tokens` case end-to-end
> (400 → `rename_param` → resend `max_output_tokens` → 200 healed, chain recorded).
>
> **Not yet:** backend e2e (needs Postgres); a live _HTTP_ heal needs a provider
> that actually rejects `max_tokens` (real OpenAI Responses key, or a localhost
> mock provider in self-hosted mode). Nothing committed.
> Manifest intercepts a request that failed with a _repairable_ error, ships the
> failed request + full provider response to an external **healing service**, gets a
> patched request back, and re-sends it **once** — **before**
> the normal fallback chain runs. The attempt is recorded on the
> message so you can see the first error, every request Autofix sent, what changed,
> and the final result.
This is a **proof of concept**. The goal is to stand the loop up end-to-end against
a mock (then the real healing service) and see what comes out of it — not to ship a
hardened feature. Decisions favour the smallest contained change over generality.
---
## 1. What already exists (important)
"Autofix" is a named, marketed feature in this repo. The original POC started
from a waitlist shell; today the engine and settings surface are generally available.
| Surface | File | What it is |
| ----------- | -------------------------------------------------------- | ------------------------------------------------ |
| Settings | `packages/frontend/src/pages/SettingsAutofixSection.tsx` | Per-agent Autofix toggle, shown to every tenant |
| Routing API | `packages/backend/src/routing/tier.controller.ts` | `GET/PATCH /api/v1/routing/:agentName/autofix` |
The marketing copy is the product promise we're now building the engine for:
> **"Autofix repairs failing requests before they reach the model"**
> Real-time fix · Zero downtime · Observability · Notifications
> —
**Naming:** display name is **"Autofix"**; backend prefix is `autofix_`.
---
## 2. Scope (locked for the POC)
| Decision | Choice | Rationale |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Streaming** | Non-streaming **+** streaming requests that fail _before the first byte_ | A repairable 4xx returns a non-200 status inside `proxyRequest()`, before the controller streams anything — healing is transparent. Mid-stream is out of scope. |
| **Which errors** | **Request-side 4xx only** (`400`, `404`, `422` by default) | Malformed params, wrong format, unknown model. Explicitly **not** `401/403` (auth), `429` (rate limit), `408` (timeout), or `5xx` (provider availability — owned by fallback). |
| **Order vs. fallback** | **Autofix runs FIRST, fallback is the safety net** | Healing a fixable 400 beats spraying the same broken body across every fallback provider. Only if healing is exhausted/unfixable does the existing fallback chain run. |
| **Budget** | **Per-agent**, default **3**, next to the toggle on the Routing page | One entity (`agents`), one UI location. |
| **Availability** | **Open to all** (cloud + self-hosted), no entitlement gate | Simplest for the POC; revisit gating later. |
| **Healing contract** | **Simple, self-authored** (§4): full request + full response in, patched request out | Real repo's OpenAPI isn't available yet; we build to a stable internal port + mock and reconcile the HTTP adapter later. |
| **Recording** | **One row per client request + full `autofix_chain` JSONB** capturing every attempt, each tagged `original` vs `autofix` | "Track all requests, see the full chain, know which Autofix sent" — without distorting message-count KPIs. |
**Why 4xx-only is safe to retry:** a 4xx means the provider _rejected_ the request —
no tokens generated, no tool calls executed, no side effects. Re-sending a patched
version is idempotent-safe. (This is why we do **not** heal 200-but-bad responses.)
---
## 3. Flow
```
agent ──► POST /v1/chat/completions (or /messages, /responses)
│
▼
proxyService.proxyRequest(body) ← Autofix lives HERE, not the controller
│
route = resolve(body)
forward = forwardToPrimary(route, body) ──► provider
│
┌───────────────── AUTOFIX LOOP (before fallback) ──────────────────┐
│ while !forward.ok AND autofix_enabled │
│ AND status ∈ repairable(4xx) AND attempt < autofix_max_attempts:
│ errorBody = await forward.response.text() // full response │
│ chain.push({ origin, attempt, request: body, status, errorBody }) │
│ heal = healingClient.heal({ request: body, response, context, chain })
│ if heal.outcome == 'unfixable': break │
│ body = heal.patched_request │
│ route = resolve(body) // model may have changed │
│ forward = forwardToPrimary(route, body) ──► provider │
│ attempt++ │
└────────────────────────────────────────────────────────────────────┘
│
still !forward.ok AND shouldTriggerFallback(status)?
│ yes → existing FALLBACK CHAIN (unchanged safety net)
▼
return { forward, meta, autofix_chain }
│
▼
controller: ok? → stream / json to client, recordSuccess(+chain)
!ok? → handleProviderError(+chain), return last error
```
**Integration point:** inside `packages/backend/src/routing/proxy/proxy.service.ts`,
right after the **primary** forward and **before** the `shouldTriggerFallback(...)`
check (~line 315). The controller (`proxy.controller.ts:187`) barely changes — it
just receives an `autofix_chain` alongside `forward`/`meta` and threads it into
recording. Because `proxyRequest()` returns before the controller streams, we're
always in the "pre-first-byte" zone — the streaming-safety constraint is automatic.
**Re-send mechanism:** each heal attempt re-resolves the route for the patched body
(so an unknown-model → known-model fix actually changes provider/model) and forwards
to the **primary** only. The fallback chain is deliberately _not_ run per-attempt —
it's the single safety net after Autofix gives up.
---
## 4. The healing service (external, mocked first)
Manifest talks to the healing service through a **stable internal port** so proxy
code never depends on the wire format:
```ts
interface HealingClient {
heal(input: HealRequest): Promise;
}
```
Three implementations, selected by config at boot:
- **`HttpHealingClient`** — used whenever `AUTOFIX_HEALING_URL` is set. POSTs to that
URL, maps response → `HealResult`.
- **`NoopHealingClient`** — the **production default when `AUTOFIX_HEALING_URL` is
unset**. Inert: never heals, never mutates traffic. Keeps the dev mock off real
traffic when no healer is wired.
- **`MockHealingClient`** — the **dev/test default when `AUTOFIX_HEALING_URL` is
unset**. In-process, deterministic (implements the MVP `max_tokens` rename) so the
heal → resend → confirm flow can be exercised without an external Phoenix.
### Phoenix contract (implemented)
The real service is **Phoenix** (`mnfst/phoenix`). Two endpoints, mapped 1:1 in
`packages/backend/src/routing/autofix/phoenix.types.ts` (that + the OpenAPI draft
are the source of truth):
- **`POST /api/heal`** — send the full failed request + the normalised provider
error; get back a decision discriminated on `status`:
`patched` / `unverified` (both carry a `healedBody` to resend, plus a
`healAttemptId`; `patched` = the issue is already verified, `unverified` = a
fresh patch) · `resolving` (Phoenix is authoring a patch; nothing to resend
this round) · `no_patch`.
- **`PATCH /api/heal-attempts/{healAttemptId}`** — after resending the `healedBody`,
report the retry outcome `{ retryStatusCode, error? }` (`error` required on ≥400).
Phoenix decides succeeded/failed itself (cleared target vs. same error recurring) —
we don't send a verdict. Manifest fires this **fire-and-forget** so it never
delays the client.
`provider` + `api` (`chat_completions|responses|messages` — exactly Manifest's
`apiMode`) are the fingerprint dimensions. `traceId` (**required**) is the stable
per-logical-request id — Manifest reuses the message-link group id and sends it as
Phoenix's `traceId` so Phoenix can group the heal-attempt timeline across retries.
The provider error is normalised to `{ message, type, param, code }` by
`provider-error-normalizer.ts`.
`POST /api/heal` request (Manifest → Phoenix):
```jsonc
{
"traceId": "…", // REQUIRED — stable per logical request (group id)
"provider": "openai",
"api": "responses", // | "chat_completions" | "messages"
"url": "https://api.openai.com/v1/responses", // optional, must be an absolute URL
"request": {
/* FULL failed request body */
},
"response": {
"statusCode": 400,
"error": { "message": "…", "type": "…", "param": "…", "code": "…" },
},
}
```
`POST /api/heal` response (Phoenix → Manifest), discriminated on `status`:
```jsonc
{
"status": "patched", // | unverified | resolving | no_patch
"issueId": "…",
"patchId": "…", // nullable
"healAttemptId": "…", // present on patched / unverified
"operations": [{ "type": "rename_param", "from": "max_tokens", "to": "max_output_tokens" }],
"healedBody": {
/* FULL body to resend */
},
"retryAfterMs": 2000, // present only on resolving
}
```
`PATCH /api/heal-attempts/{healAttemptId}` — report the retry outcome:
```jsonc
{ "retryStatusCode": 200 } // cleared → Phoenix marks succeeded
{ "retryStatusCode": 400, "error": { "code": "…" } } // error present & required on ≥400
```
`no_patch` / `resolving` let the service stop the loop early instead of burning the
whole budget.
---
## 5. Data model changes
### 5.1 `agents` (toggle) — mirrors `complexity_routing_enabled`
```ts
// Nullable: NULL inherits the mode default (ON in cloud, OFF in self-hosted);
// an explicit true/false wins. There is no max-attempts column — healing is a
// single attempt.
@Column('boolean', { nullable: true }) autofix_enabled!: boolean | null;
```
### 5.2 `agent_messages` — two linked rows per healed request
> **Revised per user feedback (2026-07-01).** A healed request is recorded as **two
> rows in the log**, not one: the **failed original** and the **successful retry**,
> so each real upstream request is its own line. They're linked by a shared
> `autofix_group_id` (in the DB) and by a clickable link in the UI.
```ts
@Column('boolean', { default: false }) autofix_applied!: boolean; // part of an Autofix flow?
@Column('varchar', { nullable: true }) autofix_group_id!: string | null; // links original ↔ retry (indexed)
@Column('varchar', { nullable: true }) autofix_role!: string | null; // 'original' | 'retry'
@Column('jsonb', { nullable: true }) autofix_operations!: object | null; // the Phoenix edits that fixed it
@Column('jsonb', { nullable: true }) autofix_phoenix!: object | null; // Phoenix ids {issueId, patchId, healAttemptId}
```
| Row | `status` | `autofix_role` | Notes |
| ---------------- | ------------------------------ | -------------- | ------------------------------------------------------------------------------------------------------------ |
| Failed original | **`auto_fixed`** (new, orange) | `original` | carries the error + `autofix_operations`; 0 tokens; timestamped ~1s before the retry so they sort adjacently |
| Successful retry | `ok` | `retry` | the real completion (tokens/cost) |
- **KPI consistency:** `auto_fixed` is added to `ERROR_MESSAGE_STATUSES`, so the
failed original is **excluded** from message-count KPIs (the retry `ok` row is the
single counted success — no double-counting) and is included in the Messages-log
"errors"/"failed" filter. The Messages-log _total_ (unfiltered) shows both rows —
the two lines the user asked for.
- **The link.** `autofix_group_id` is the DB link. The message-detail endpoint
resolves the paired row (`autofix_sibling { id, role, status }`) so the UI renders
"→ View the successful autofix retry" / "← View the original failed request",
which scrolls+highlights the sibling (mirrors `scrollToFallbackSuccess`).
Recorder: `recordSuccessMessage` tags the retry row (`autofixColumns(autofix,'retry')`);
a new `recordAutofixOriginals()` inserts the failed original row(s), called from
`recordSuccess` when `outcome==='healed'`.
Migrations: two `ALTER TABLE` migrations (agents, agent_messages, + a
`(tenant_id, autofix_group_id)` index) with fresh unique timestamps, registered in
`data-source-definitions.ts`. No new entities.
---
## 6. Backend components
| Component | Location | Responsibility |
| ------------------------------------------ | ------------------------------------------------------------------------------ | ------------------------------------------------------------ |
| `AutofixModule` | `packages/backend/src/routing/autofix/` | Wires client + config |
| `HealingClient` port + `Mock`/`Http` impls | same | Talk to the healing service |
| `AutofixService` | same | The loop: repairable check, budget, re-route, chain building |
| Repairable classifier | reuse `proxy-error-sanitizer.ts` (`classifyProviderError`) + status allow-list | Decide if an error is healable |
| Loop hook | `proxy.service.ts` (~L315, before `shouldTriggerFallback`) | Run `AutofixService` on the primary forward |
| Recorder changes | `proxy-message-recorder.ts` | Accept optional `autofix_chain` on success/error recording |
| Toggle+budget endpoints | `routing/tier.controller.ts` (or new `autofix.controller.ts`) | `GET`/`PATCH /api/v1/routing/:agentName/autofix` |
Endpoint shape (mirrors the complexity toggle; PATCH since there are two fields):
```
GET /api/v1/routing/:agentName/autofix → { enabled, maxAttempts }
PATCH /api/v1/routing/:agentName/autofix { enabled?, maxAttempts? } → { enabled, maxAttempts }
```
Scope via `@TenantCtx()` + `resolveAgentService.resolve(tenantId, agentName)`, and
invalidate the agent cache on write (same as `toggleComplexity`).
---
## 7. Frontend
- **Routing page** — add an "Autofix" control next to "Route by complexity" in
`RoutingDefaultTierSection.tsx` (same `routing-switch` styling): toggle +, when on, a
small number input for max attempts. New api fns `getAutofix()` / `updateAutofix()`
in `services/api/routing.ts`.
- **Message detail** (`MessageDetails.tsx`) — when `autofix_applied`, render an
**"Autofix"** section: outcome badge, then the `autofix_chain` as a visible timeline
— each attempt showing origin (agent vs Autofix), the request that was sent, the
status/response, and what changed (`patch_summary` / `changed_fields`).
- **Message list** — small "Autofixed" badge on healed rows. If shown in the table,
add the scalar columns to `selectMessageRowColumns()` + `MESSAGE_ROW_SELECT_ALIASES`
- `MessageRow` (keep the shared projection contract intact — CLAUDE.md).
---
## 8. Configuration (env)
| Var | Default | Purpose |
| ------------------------------ | ---------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `AUTOFIX_HEALING_URL` | _(unset → mock)_ | External healing service endpoint |
| `AUTOFIX_HEALING_API_KEY` | _(unset)_ | Sent as `x-api-key` to Phoenix (Phoenix fails closed in production without a key; omit for a keyless dev/test Phoenix) |
| `AUTOFIX_DEFAULT_MAX_ATTEMPTS` | `3` | Fallback budget when an agent has none set |
| `AUTOFIX_TIMEOUT_MS` | `10000` | Per heal round-trip timeout |
| `AUTOFIX_REPAIRABLE_STATUSES` | `400,404,422` | Allow-list of healable statuses |
| `AUTOFIX_GLOBAL_ENABLED` | `true` | Kill switch independent of per-agent toggles |
---
## 9. Explicitly out of scope (POC)
- Mid-stream healing (after bytes are sent).
- Healing `401/403/429/408/5xx` / transport errors (owned by auth refresh, rate-limit
cooldown, model fallback).
- Healing 200-but-semantically-bad responses.
- Hardened scrubbing/PII controls on request/response bodies (basic scrub only — §11).
- Pre-emptive fixing _before_ the first attempt (the marketing "before they reach the
model" — POC is reactive).
- Entitlement or rollout gating (Autofix is open to all).
---
## 10. Open questions / decisions still needed
1. **Chain modeling — RESOLVED (2026-07-01):** the user chose **one row per attempt**
(failed original + successful retry as sibling log rows, linked by `autofix_group_id`
- a UI link), matching the `recordFailedFallbacks` pattern. Implemented in §5.2.
2. **Fallback body after Autofix fails.** When healing is exhausted and the safety-net
fallback chain runs, forward the **original** body or the **last patched** body?
_My lean:_ original (fallbacks are a separate axis; patched guesses were for the
primary). Minor.
3. **Real healing schema** — no OpenAPI yet; building to §4. When it lands, reconcile
the `HttpHealingClient` adapter. Not a blocker for the POC.
_(Resolved from discussion: streaming = non-stream + pre-first-byte · errors = 4xx only
· Autofix before fallback · per-agent budget default 3 · open to all · full req+resp to
the healer · track the full chain and label Autofix requests.)_
---
## 11. Risks & considerations
- **Data footprint / trust boundary.** We now (a) send full request + response bodies
to an external service and (b) store them on Autofix messages — both new for
Manifest. Fine against an in-process/same-infra mock; before the real external
service, this needs the care of the error-cluster CMS boundary (scrub + consent +
retention). Scope storage to Autofix messages only; scrub + truncate. Flagging, not
hardening, for the POC.
- **Latency.** Worst case ≈ `budget × (heal round-trip + provider re-send)` before
success/failure. Transparent for non-streaming but slow; bounded by
`AUTOFIX_TIMEOUT_MS` and the budget.
- **Loop safety.** Budget is a hard cap; `unfixable` short-circuits; only runs while
no bytes have been sent. No unbounded retry.
- **Observability parity.** The message + its `autofix_chain` is the single source of
truth for "how it went" — matches the "Observability" pillar the Autofix modal
already advertises.
---
## 12. Rough build order
1. Entities + 2 migrations (`agents`, `agent_messages`) + `helpers.ts` sync.
2. `AutofixModule` + `HealingClient` port + `MockHealingClient` + `AutofixService`
(loop, repairable classifier, re-route, chain).
3. Hook into `proxy.service.ts` before the fallback trigger; thread `autofix_chain`
through `proxyRequest` → controller → recorder.
4. Toggle + budget endpoints + Routing UI control.
5. Message-detail "Autofix" timeline (+ optional list badge).
6. `HttpHealingClient` once the real schema lands.
7. Tests (100% line coverage — CLAUDE.md): loop paths (healed / exhausted / unfixable /
disabled / non-repairable / re-route-on-model-change), before-fallback ordering,
classifier, endpoints, UI, recorder.
```
```
---
### Glossary
# Analytics glossary and metric definitions
This document is the canonical contract for Manifest analytics terminology and metrics. Schema, API, analytics, and UI code must use these definitions consistently. It defines the concepts and counting rules only; it does not specify page layout or frontend behavior.
## The two worlds
Manifest analytics live in two distinct worlds. Mixing them produces totals that answer different questions.
**The Request world (agent side).** An agent makes one logical Request to Manifest. Manifest may try several providers to serve it, but the agent sees one outcome. Requests belong to agents and to the global Overview. Recovery is a Request-level concept.
**The Attempt world (provider side).** Every provider call is a Provider Attempt. A Request has zero or more Attempts. Attempts belong to providers, Provider Connections, and models. An Attempt may be triggered by fallback or Autofix, but that context does not change its status.
## Glossary
### Manifest Request
One logical request from an agent to Manifest.
- Direction: agent → Manifest
- Database table: `requests`
- Primary key: `requests.id`
- Status: `requests.status`
- A Request ultimately has one caller-visible outcome and may have one or many Provider Attempts.
### Provider Attempt
One route evaluation while serving a Manifest Request. Most Attempts call an AI provider; Manifest-local failures are also retained as failed Attempts so the full routing chain is visible.
- Direction: Manifest → AI provider, or internal when Manifest rejects the route locally
- Database table: `agent_messages` (the physical legacy name is retained for safe rolling deploys)
- Parent Request: `agent_messages.request_id → requests.id`
- Order within the Request: `agent_messages.attempt_number`
- Status: `agent_messages.status`
Every provider call counts as an Attempt, including failed calls, fallback attempts, and Autofix retries. Manifest-local failures such as a route skipped during provider cooldown also count as failed Attempts, but their Manifest error origin keeps them out of provider-reliability metrics.
### Status
Requests and newly written Provider Attempts use the same canonical status values:
| Value | Meaning | UI label |
| ----------- | -------------------------------------------------- | --------- |
| `pending` | The operation has no terminal outcome yet. | Pending |
| `cancelled` | The caller disconnected before a provider outcome. | Cancelled |
| `success` | The operation completed successfully. | Success |
| `failed` | The operation completed without succeeding. | Failed |
`cancelled`, `success`, and `failed` are terminal. Outcome metrics use only `success` and `failed`; caller cancellations do not count as provider or Manifest errors. In this document, **completed** means a terminal outcome included in metrics (`success` or `failed`); it is not a separate status value.
Historical `agent_messages` rows may retain the legacy physical values `ok`, `error`, `fallback_error`, or `rate_limited`. Analytics readers normalize `ok` to `success` and the legacy failure values to `failed`. Writers must use the canonical values above.
Manifest creates a Request with `pending` status when it accepts the Request and creates an Attempt with `pending` status when it starts the provider call. Each transitions to one terminal status. If the caller disconnects first, active rows transition to `cancelled` without error fields.
Every newly written `failed` Request and Attempt has a non-empty `error_message`. Machine classification remains in the `error_origin`, `error_class`, and `error_http_status` fields; the message is the human-readable explanation.
A successful Request has `requests.status = 'success'`. A successful newly written Attempt has `agent_messages.status = 'success'`; a historical successful Attempt may retain `ok`. `requests.status` is authoritative for the caller-visible outcome.
When a Request with at least one Attempt succeeds, its Last Attempt must also be successful. A failed Request may have no Attempt at all, and `requests.status` remains authoritative if a Request-level failure happens outside the provider call.
### Last Attempt
The Last Attempt is the final Provider Attempt within a Request: the Attempt with the highest `attempt_number`. For a completed Request, it is also the Attempt that concluded the Request: the successful Attempt when the Request succeeded, otherwise the terminal non-superseded failure. A zero-attempt Request has no Last Attempt.
Attempt numbers are positive, unique within their Request, and increase in the order Manifest evaluates routes. Do not derive Attempt order from timestamps. `agent_messages.timestamp` records the real route-evaluation or provider-call start time, and `agent_messages.duration_ms` records measured elapsed time once the Attempt is terminal. Neither may be fabricated to create an ordering. Superseded Attempts are never the Last Attempt of a completed Request.
Historical Attempts linked during the migration may have a null `attempt_number` when their order could not be reconstructed safely. Readers may use the legacy compatibility ranking (successful outcome, then non-superseded failure, then timestamp and id) to select a representative terminal Attempt, but must not present that inferred position as an Attempt number. All newly recorded Attempts require a positive `attempt_number`.
### Superseded Attempt
A Superseded Attempt is a failed Provider Attempt after which Manifest continued the same Request with another Attempt.
Newly written Superseded Attempts keep `agent_messages.status = 'failed'` and have `agent_messages.superseded = true`; historical rows may retain a legacy failure status such as `fallback_error`. They count in Attempt metrics, but do not determine the Request outcome.
### Recovered Request
A Recovered Request is a successful Request after Manifest continued beyond a failed Attempt by applying Autofix or fallback.
Recovery belongs to Requests only. Providers, Provider Connections, models, and Attempts are never “recovered.” Applying a recovery method is not enough: the Request must ultimately succeed.
Each Request belongs to exactly one outcome category, evaluated in this order:
1. **Pending:** `requests.status = 'pending'`.
2. **Cancelled:** `requests.status = 'cancelled'`.
3. **Failed:** `requests.status = 'failed'`.
4. **Recovered by Autofix:** `requests.status = 'success'` and `requests.autofix_status = 'retry_succeeded'`.
5. **Recovered by fallback:** `requests.status = 'success'`, the Last Attempt has a non-null `agent_messages.fallback_from_model`, and the Request was not recovered by Autofix.
6. **Success:** any other Request with `requests.status = 'success'`.
The ordering makes Autofix the tie-breaker if inconsistent or historical data satisfies both recovery criteria. If an Autofix retry fails and a fallback succeeds, the Request is recovered by fallback because `requests.autofix_status` is not `retry_succeeded`.
### Recovery attempt
A recovery attempt is a recovery method Manifest tried during a Request, whether or not it succeeded. The Requests table lists them in its "Recovery attempts" column and filter. Autofix and fallback fields on Attempts describe what happened in the chain; they do not replace the Request status or recovery category.
### AI Provider and Provider Connection
An **AI Provider** is an upstream platform such as OpenAI or Anthropic. Each Attempt records its Provider in `agent_messages.provider`.
A **Provider Connection** is a tenant's configured connection to one AI Provider. It lives in `tenant_providers`; `tenant_providers.provider` identifies its Provider and `tenant_providers.auth_type` identifies its authentication method. A Provider may have several Connections, while each Connection belongs to one Provider.
An Attempt identifies the Connection it used through `agent_messages.tenant_provider_id → tenant_providers.id`. `agent_messages.auth_type` and `agent_messages.provider_key_label` are historical display snapshots, not Connection identity. Authentication credentials such as API keys or access tokens remain encrypted Connection-level data and must never be copied onto Attempts.
“Connection Attempts” are Provider Attempts filtered by `tenant_provider_id`, not a separate event type or table. The reference may be null for legacy data, local providers, or paths where Manifest could not identify a Connection, so Provider totals may exceed the sum of their Connection totals.
## Request and Attempt data
| Data | Canonical source |
| ------------------------------- | -------------------------------------------------------------------------------------- |
| Request identity and scope | `requests.id`, `requests.tenant_id`, `requests.agent_id` |
| Caller-visible status | `requests.status` and the `requests.error_*` columns |
| End-to-end duration | `requests.duration_ms` |
| Model requested by the agent | `requests.requested_model` |
| Autofix outcome | `requests.autofix_status` |
| Attempt identity and parent | `agent_messages.id`, `agent_messages.request_id` |
| Attempt order | `agent_messages.attempt_number` |
| Attempt status | `agent_messages.status` and the `agent_messages.error_*` columns |
| Provider, model, and Connection | `agent_messages.provider`, `agent_messages.model`, `agent_messages.tenant_provider_id` |
| Attempt usage | Token and cost columns on `agent_messages` |
| Routing and recovery context | `agent_messages.routing_*`, `fallback_*`, `superseded`, and `autofix_*` |
## Metric definitions and perimeter
Dashboard metrics use completed Requests and Attempts within the selected tenant, agent, and time filters:
- Pending and cancelled Requests and Attempts are excluded from totals and success rates.
- Playground traffic is excluded.
- Completed zero-attempt Requests are included in Request metrics.
- During the historical transition, each unlinked legacy Attempt may be represented as one synthetic Request. It counts once in Request metrics and still counts normally in Attempt metrics.
| Metric | Definition |
| -------------------- | ------------------------------------------------------------------------------------------------------ |
| Total Requests | Count each completed stored Request once, plus each eligible synthetic Request once during transition. |
| Request success rate | Requests with `status = 'success'` divided by completed Requests. No traffic means no rate, not 100%. |
| Total Attempts | Count each completed `agent_messages.id`; do not deduplicate by Request. |
| Attempt success rate | Attempts with `status = 'success'` divided by completed Attempts. |
Request-level surfaces count Requests; Provider-, Connection-, and model-level surfaces count Attempts. The two totals answer different questions and are not expected to match.
## Autofix outcomes
`requests.autofix_status` is the single Request-level Autofix verdict. `NULL` means Autofix did not record an outcome.
| Value | Meaning |
| ----------------- | -------------------------------------------------- |
| `no_patch` | Phoenix was consulted but returned no known patch. |
| `resolving` | Phoenix is still investigating; no retry was sent. |
| `retry_succeeded` | Manifest applied a patch and the retry succeeded. |
| `retry_failed` | Manifest applied a patch but the retry failed. |
| `service_error` | The Phoenix service call failed. |
Only `retry_succeeded` means the Request was recovered by Autofix.
## Examples
| Scenario | Counted Requests | Counted Attempts | Request status | Attempt statuses | Recovery |
| ---------------------------------------------- | ---------------: | ---------------: | -------------- | ----------------------------- | -------- |
| Request or Attempt is still in progress | 0 | 0 | `pending` | `pending` where applicable | Excluded |
| Caller disconnects before an outcome | 0 | 0 | `cancelled` | `cancelled` where applicable | Excluded |
| Primary Provider succeeds | 1 | 1 | `success` | `success` | None |
| Manifest rejects before contacting a Provider | 1 | 0 | `failed` | None | None |
| Primary Attempt fails, fallback succeeds | 1 | 2 | `success` | `failed`, `success` | Fallback |
| Primary Attempt fails, Autofix retry succeeds | 1 | 2 | `success` | `failed`, `success` | Autofix |
| Autofix retry fails, fallback succeeds | 1 | 3 | `success` | `failed`, `failed`, `success` | Fallback |
| Every Provider Attempt fails | 1 | N | `failed` | `failed` × N | None |
The Counted Requests and Counted Attempts columns show how many rows the completed dashboard metrics count for each scenario; pending rows are excluded.
## Reading rules
1. Grouping is a lens, not a filter. Switching the grouping within one chart never changes its total.
2. A Request counts once in Request-world surfaces; an Attempt counts once in Attempt-world surfaces. The totals are not expected to match.
3. “Recovered” only qualifies Requests. Providers, Connections, models, and Attempts have no recovery metric.
4. Autofix and fallback fields on an Attempt describe its trigger or context, never its result. The Attempt's status records its result.
5. A Request-level Provider lens attributes each Request once to its Last Attempt's Provider. Zero-attempt Requests use the Manifest bucket. It must never count every Attempt as a separate Request.
6. Connection and model surfaces remain Attempt-level because one Request may use several Connections or models.
## Legacy naming and statuses
- `agent_messages` remains the physical table for Provider Attempts. The legacy name is retained so old and new application versions can write safely during rolling deploys.
- `AgentMessage`, `/api/v1/messages`, and frontend `Message*` names are legacy code and API names; they do not define the analytics unit.
- The physical Autofix column remains `agent_messages.autofix_phoenix`; the entity and API expose it as `autofix_decision`.
- Legacy `ok` maps to `success`. Legacy `error`, `rate_limited`, `fallback_error`, and `auto_fixed` map to `failed`; their error, fallback, supersession, and Autofix fields preserve the additional context.
---
### Model Parameters Schema
# MPS Schema
MPS stands for Model Parameters Schema.
This document defines the JSON language used by the model parameter schema
catalog. The catalog is metadata: it describes which request parameters Manifest
can configure for a provider/auth/model tuple. User-selected values still live in
`agent_model_params`.
The runtime source is the versioned `modelparams` npm package, generated from
the same modelparams.dev catalog. Manifest loads that package catalog at runtime
and keeps provider/auth/model route matching inside
`ProviderParamSpecService`.
The executable validator is `isParamApplicability` in
`packages/shared/src/provider-params-spec.ts`. Any schema change must update:
- this document
- the shared TypeScript types
- `isParamApplicability`
- the shared tests that prove invalid shapes are rejected
## Schema Entry
Each entry describes one provider/auth/model tuple and its available parameters.
```json
{
"provider": "anthropic",
"authType": "api_key",
"model": "claude-haiku-4-5-20251001",
"params": [
{
"path": "top_p",
"type": "number",
"label": "Top P",
"description": "Controls nucleus sampling by limiting generation to tokens whose cumulative probability reaches this value.",
"default": 1,
"range": { "min": 0, "max": 1, "step": 0.01 },
"group": "sampling",
"applicability": {
"except": [{ "thinking.type": ["adaptive", "enabled"] }, { "temperature": { "not": 1 } }]
}
}
]
}
```
Rules:
- `provider`, `authType`, and `model` identify exactly one model route.
- `params` is the non-empty list of parameters for that exact route.
- `path` is a dot path into stored params and outbound request params.
- `type` is semantic data type, not a UI control kind.
- `label` is user-facing copy.
- `description` is developer-facing explanatory copy for the raw parameter.
- `default` is the provider default Manifest should display when known.
- `values` is allowed only for finite choices.
- `range` describes numeric bounds and optional step.
- `group` is a semantic grouping for ordering and display.
- `applicability` is optional; omitted means always available.
- Do not add ad hoc rule fields such as `conflictsWith`, `disabledWhen`,
`ui`, or provider-specific metadata. Express availability through
`applicability`.
## Applicability
`applicability` controls whether a parameter is available for the current draft
or request params.
Only two top-level keys are allowed:
- `only`: the parameter is available only when the rule matches.
- `except`: the parameter is unavailable when the rule matches.
At least one of `only` or `except` must be present. Unknown keys are invalid.
### Rule Shape
A rule is either:
- one non-empty match object
- a non-empty array of match objects
Array rules are OR semantics. A single match object is AND semantics.
```json
{
"except": [{ "thinking.type": ["adaptive", "enabled"] }, { "temperature": { "not": 1 } }]
}
```
This means: disable the param when `thinking.type` is `adaptive` or `enabled`,
or when `temperature` exists and is not `1`.
### Match Values
Each match key is a dot path. Each match value must be one of:
- JSON primitive: string, number, boolean, or null
- non-empty array of JSON primitives
- `{ "not": }`
Objects other than `{ "not": ... }` are invalid.
Empty arrays are invalid.
Empty match objects are invalid.
Examples:
```json
{ "thinking.type": "enabled" }
```
```json
{ "thinking.type": ["adaptive", "enabled"] }
```
```json
{ "temperature": { "not": 1 } }
```
## Evaluation Semantics
Evaluation uses the current params object after dot-path expansion.
For a normal match:
- primitive value matches by JSON equality
- array value matches if any primitive item equals the actual value
- missing paths do not match
For `{ "not": value }`:
- missing paths do not match
- present paths match when the actual value is not equal to `value`
For a param spec:
1. If `only` is present and does not match, the param is unavailable.
2. If `except` is present and matches, the param is unavailable.
3. Otherwise the param is available.
## Storage And Request Merge
Saved model params store UI values using the same dot-path shape as `path`.
They do not store derived UI state or provider-specific rule state.
The outbound proxy merge:
1. expands needed nested defaults for configured nested roots
2. merges configured Manifest values last, so Manifest values win for the same provider path
3. omits params that are not applicable under the final effective values
Client request body values that do not overlap configured Manifest model
params stay in the outbound provider request.
## Adding A New Rule
Prefer expressing new provider behavior with existing `applicability` syntax.
Add new schema syntax only when a provider rule cannot be represented with:
- exact match
- one-of match
- negated match
- OR of match objects
- AND within one match object
When extending the language, update the executable validator and add tests for
both accepted and rejected shapes in
`packages/shared/__tests__/provider-params-spec.spec.ts`.
---