{"owner":"maximhq","repo":"bifrost","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md"],"skills":{"AGENTS.md":"# AGENTS.md — Bifrost AI Gateway\n\n> Context for AI agents (Claude Code, Copilot, Cursor, etc.) working on this codebase. Read this fully before making changes.\n\n## What is Bifrost?\n\nBifrost is a high-performance AI gateway that unifies 20+ LLM providers behind a single OpenAI-compatible API with ~11µs overhead at 5,000 RPS. It also serves as an MCP (Model Context Protocol) gateway, turning static chat models into tool-calling agents.\n\nGitHub: `maximhq/bifrost`\n\n---\n\n## Repository Layout\n\n```\nbifrost/\n├── core/ # Go core library — the engine\n│ ├── bifrost.go # Main struct, request queuing, provider lifecycle (~3.4K lines)\n│ ├── inference.go # Inference routing, fallbacks, streaming dispatch (~1.9K lines)\n│ ├── mcp.go # MCP integration entry point\n│ ├── schemas/ # ALL shared Go types — 41 files\n│ │ ├── bifrost.go # BifrostConfig, ModelProvider enum, RequestType enum, context keys\n│ │ ├── provider.go # Provider interface (30+ methods), NetworkConfig, ProviderConfig\n│ │ ├── plugin.go # LLMPlugin, MCPPlugin, HTTPTransportPlugin, ObservabilityPlugin\n│ │ ├── context.go # BifrostContext (custom context.Context with mutable values)\n│ │ ├── chatcompletions.go # Chat completion request/response types\n│ │ ├── responses.go # OpenAI Responses API types\n│ │ ├── embedding.go # Embedding types\n│ │ ├── images.go # Image generation types\n│ │ ├── batch.go # Batch operation types\n│ │ ├── files.go # File management types\n│ │ ├── mcp.go # MCP types\n│ │ ├── trace.go # Tracer interface\n│ │ └── logger.go # Logger interface\n│ ├── providers/ # 20+ provider implementations\n│ │ ├── openai/ # Reference implementation (largest, most complete)\n│ │ ├── anthropic/ # Non-OpenAI-compatible example\n│ │ ├── bedrock/ # AWS event-stream protocol\n│ │ ├── gemini/ # Google-specific API shape\n│ │ ├── groq/ # OpenAI-compatible (minimal, delegates to openai/)\n│ │ └── utils/ # Shared: HTTP client, SSE parsing, error handling, scanner pool\n│ ├── pool/ # Generic Pool[T] — dual-mode (prod: sync.Pool, debug: full tracking)\n│ │ ├── pool_prod.go # Zero-overhead sync.Pool wrapper (default build)\n│ │ └── pool_debug.go # Double-release/use-after-release/leak detection (-tags pooldebug)\n│ ├── mcp/ # MCP protocol implementation\n│ │ ├── agent.go # Agent orchestration loop (multi-turn tool calling)\n│ │ ├── clientmanager.go # MCP client lifecycle management\n│ │ ├── toolmanager.go # Tool registration, discovery, filtering\n│ │ ├── healthmonitor.go # Client health monitoring\n│ │ └── codemode/starlark/ # Starlark sandbox for code-mode execution\n│ └── internal/\n│ ├── llmtests/ # LLM integration test infra (48 files, scenario-based)\n│ └── mcptests/ # MCP/Agent test infra (40+ files, mock-based)\n│\n├── framework/ # Data persistence, streaming, ecosystem utilities\n│ ├── configstore/ # Config storage backends (file, postgres)\n│ ├── logstore/ # Log storage backends (file, postgres)\n│ ├── vectorstore/ # Vector storage (Weaviate, Qdrant, Redis, Pinecone)\n│ ├── streaming/ # Streaming accumulator, delta copying, response marshaling\n│ │ ├── accumulator.go # Chunk accumulation into full response (~24KB)\n│ │ ├── chat.go # Chat stream handling (~17KB)\n│ │ └── responses.go # Response stream marshaling (~35KB)\n│ ├── modelcatalog/ # Model metadata registry\n│ ├── tracing/ # Distributed tracing helpers\n│ └── encrypt/ # Encryption utilities\n│\n├── transports/\n│ ├── config.schema.json # JSON Schema — THE source of truth for config.json (~2700 lines)\n│ └── bifrost-http/ # HTTP gateway transport\n│ ├── server/ # Server lifecycle, route registration\n│ ├── handlers/ # 27 HTTP endpoint handlers\n│ │ ├── inference.go # Chat/text completions, responses API (~109KB)\n│ │ ├── mcpinference.go # MCP tool execution\n│ │ ├── governance.go # Virtual keys, teams, customers, budgets (~100KB)\n│ │ ├── providers.go # Provider CRUD, key management\n│ │ ├── mcp.go # MCP client registry management\n│ │ ├── logging.go # Log queries, stats, histograms\n│ │ ├── config.go # System configuration\n│ │ ├── plugins.go # Plugin CRUD\n│ │ ├── cache.go # Cache management\n│ │ ├── session.go # Auth/session management\n│ │ ├── health.go # Health checks\n│ │ ├── mcpserver.go # MCP server (SSE/streamable HTTP)\n│ │ ├── websocket.go # WebSocket handler\n│ │ ├── devpprof.go # Pool debug profiler endpoint (~23KB)\n│ │ └── middlewares.go # Middleware definitions\n│ ├── lib/ # ChainMiddlewares, config, context conversion\n│ └── integrations/ # SDK compatibility layers\n│ ├── openai.go # OpenAI SDK drop-in compatibility\n│ ├── anthropic.go # Anthropic SDK compatibility\n│ ├── bedrock.go # AWS Bedrock SDK compatibility\n│ ├── genai.go # Google GenAI SDK compatibility\n│ ├── langchain.go # LangChain compatibility\n│ ├── litellm.go # LiteLLM compatibility\n│ └── pydanticai.go # PydanticAI compatibility\n│\n├── plugins/ # Go plugins — each has own go.mod\n│ ├── governance/ # Budget, rate limiting, virtual keys, routing, RBAC\n│ ├── telemetry/ # Prometheus metrics, push gateway\n│ ├── logging/ # Request/response audit logging\n│ ├── semanticcache/ # Semantic response caching via vector store\n│ ├── otel/ # OpenTelemetry tracing\n│ ├── mocker/ # Mock responses for testing\n│ ├── jsonparser/ # JSON extraction utilities\n│ ├── maxim/ # Maxim observability\n│ └── compat/ # LiteLLM SDK compatibility (HTTP transport)\n│\n├── ui/ # React + vite web interface\n│ ├── app/workspace/ # Feature pages (20+ workspace sections)\n│ ├── components/ # Shared React components\n│ └── lib/ # Constants, utilities, types\n│\n├── tests/e2e/ # Playwright E2E tests\n│ ├── core/ # Fixtures, page objects, helpers, API actions\n│ └── features/ # Per-feature test suites\n│\n├── docs/ # Mintlify MDX documentation\n│ ├── docs.json # Navigation config\n│ ├── media/ # Screenshots (ui-*.png naming convention)\n│ └── (architecture|features|providers|mcp|plugins|enterprise|...)\n│\n├── .claude/skills/ # Claude Code skill definitions (4 skills)\n├── go.work # Go workspace — requires Go 1.26.1\n├── Makefile # Build, test, dev commands (1300+ lines)\n└── terraform/ # Infrastructure as Code\n```\n\n---\n\n## Go Workspace\n\nBifrost is a **multi-module Go workspace**. Each module has its own `go.mod`:\n\n```\ngo.work\n├── core/go.mod # github.com/maximhq/bifrost/core\n├── framework/go.mod # github.com/maximhq/bifrost/framework\n├── transports/go.mod # github.com/maximhq/bifrost/transports\n└── plugins/*/go.mod # 9 plugin modules (governance, telemetry, logging, etc.)\n```\n\n**Rules:**\n- Run `go mod tidy` in the **specific module directory**, not the root\n- Cross-module imports resolve via workspace locally, but need explicit `require` in `go.mod` for releases\n- The workspace requires **Go 1.26.1** (`go.work` directive)\n\n---\n\n## Build, Test & Dev Commands\n\n```bash\n# Development\nmake dev # Full local dev (UI + API with hot reload via air)\nmake build # Build bifrost-http binary\n\n# Core tests (provider integration tests — hit live APIs)\nmake test-core # All providers\nmake test-core PROVIDER=openai # Specific provider\nmake test-core PROVIDER=openai TESTCASE=TestSimpleChat # Specific test\nmake test-core PATTERN=TestStreaming # Tests matching pattern\nmake test-core DEBUG=1 # With Delve debugger on :2345\n\n# MCP/Agent tests (mock-based, no live APIs)\nmake test-mcp # All MCP tests\nmake test-mcp TESTCASE=TestAgentLoop # Specific test\nmake test-mcp TYPE=agent # By category (agent|tool|connection|codemode)\n\n# Framework tests (require local backing services — bring them up FIRST)\ndocker compose -f tests/docker-compose.yml up -d # postgres, weaviate, qdrant, pinecone, and the 4 redis variants\nmake test-framework # All framework packages\n\n# Plugin tests\nmake test-plugins # All plugins\nmake test-governance # Governance plugin specifically\n\n# Integration tests (SDK compatibility)\nmake test-integrations-py # Python SDK tests\nmake test-integrations-ts # TypeScript SDK tests\n\n# E2E tests (Playwright, requires running dev server)\nmake run-e2e # All E2E tests\nmake run-e2e FLOW=providers # Specific feature\n\n# Code quality\nmake lint # Linting\nmake fmt # Format code\n```\n\n---\n\n## Architecture\n\n### Request Flow\n\n```\nClient HTTP Request\n → FastHTTP Transport (parsing, validation ~2µs)\n → SDK Integration Layer (OpenAI/Anthropic/Bedrock format → Bifrost format)\n → Middleware Chain (lib.ChainMiddlewares, applied per-route)\n → HTTPTransportPreHook (HTTP-level plugins, can short-circuit)\n → PreLLMHook Pipeline (auth, rate-limit, cache check — registration order)\n → MCP Tool Discovery & Injection (if tool_choice present)\n → Provider Queue (channel-based, per-provider isolation)\n → Worker picks up request\n → Key Selection (~10ns weighted random)\n → Provider API Call (fasthttp client, connection pooling)\n → Response / SSE Stream\n → PostLLMHook Pipeline (reverse order of PreLLMHooks)\n → Tool Execution Loop (if tool_calls in response, MCP agent loop)\n → HTTPTransportPostHook (reverse order)\n → Response Serialization\n → HTTP Response to Client\n```\n\n### Design Principles\n\n- **Provider isolation**: Each provider has its own worker pool and queue. One provider going down doesn't cascade to others.\n- **Channel-based async**: Request routing uses Go channels (`chan *ChannelMessage`), not mutexes. The `ProviderQueue` struct manages channel lifecycle with atomic flags.\n- **Object pooling everywhere**: `sync.Pool` wrappers reduce GC pressure. Pools exist for: channel messages, response channels, error channels, stream channels, plugin pipelines, MCP requests, HTTP request/response objects, scanner buffers.\n- **Plugin pipeline symmetry**: Pre-hooks execute in registration order, post-hooks in **reverse** order (LIFO). For every pre-hook executed, the corresponding post-hook is guaranteed to run.\n- **Streaming**: SSE chunks flow through `chan chan *schemas.BifrostStreamChunk`. Accumulated into full response for post-hooks via `framework/streaming/accumulator.go`.\n\n### BifrostContext — Custom Context\n\n`BifrostContext` (`core/schemas/context.go`) is a custom `context.Context` with **thread-safe mutable values**. Unlike standard Go contexts, values can be set after creation:\n\n```go\nctx := schemas.NewBifrostContext(parent, deadline)\nctx.SetValue(key, value) // Thread-safe, uses RWMutex\nctx.WithValue(key, value) // Chainable variant\n```\n\n**Reserved context keys** (set by Bifrost internals — DO NOT set manually):\n- `BifrostContextKeySelectedKeyID/Name` — Set by governance plugin\n- `BifrostContextKeyGovernance*` — Set by governance plugin\n- `BifrostContextKeyNumberOfRetries`, `BifrostContextKeyFallbackIndex` — Set by retry/fallback logic\n- `BifrostContextKeyStreamEndIndicator` — Set by streaming infrastructure\n- `BifrostContextKeyTrace*`, `BifrostContextKeySpan*` — Set by tracing middleware\n\n**User-settable keys** (plugins and handlers can set these):\n- `BifrostContextKeyVirtualKey` (`x-bf-vk`) — Virtual key for governance\n- `BifrostContextKeyAPIKeyName` (`x-bf-api-key`) — Explicit key selection by name\n- `BifrostContextKeyAPIKeyID` (`x-bf-api-key-id`) — Explicit key selection by ID (takes priority over name)\n- `BifrostContextKeyRequestID` — Request ID\n- `BifrostContextKeyExtraHeaders` — Extra headers to forward to provider\n- `BifrostContextKeyURLPath` — Custom URL path for provider\n- `BifrostContextKeySkipKeySelection` — Skip key selection (pass empty key)\n- `BifrostContextKeyUseRawRequestBody` — Send raw body directly to provider\n\n**Gotcha**: `BlockRestrictedWrites()` silently drops writes to reserved keys. This prevents plugins from accidentally overwriting internal state.\n\n**Hard rule — never store stream-sized data in `BifrostContext`.** Context holds small handles only: IDs, durations, booleans, interface pointers. Any per-request state that scales with stream content (chunk buffers, accumulated payloads, replay queues, large per-request slices/maps) must live in a top-level manager keyed by `RequestID`, not in `ctx`. Reference implementations:\n\n- `framework/streaming.Accumulator` — owns a `sync.Map` of per-stream `StreamAccumulator` entries keyed by `RequestID`. Only `BifrostContextKeyAccumulatorID` (the ID string) is stored on the context; the chunk buffers live in the manager. The pause/resume gate (`gate.go`) extends the same per-stream entry with a state machine — again, **no buffer in ctx**.\n- The `Tracer` interface (in ctx as a small pointer) is the access path for plugins/providers to reach managers without putting bulky data on the context itself.\n\nWhen in doubt: if your new ctx key would hold a slice/map that grows with request content, route the storage through a manager and keep only the ID in ctx.\n\n---\n\n## Core Patterns\n\n### Provider Implementation\n\nThere are **two categories** of providers:\n\n**Category 1: Non-OpenAI-compatible** (Anthropic, Bedrock, Gemini, Cohere, HuggingFace, Replicate, ElevenLabs):\n```\ncore/providers//\n├── .go # Controller: constructor, interface methods, HTTP orchestration\n├── _test.go # Tests\n├── types.go # ALL provider-specific structs (PascalCase prefixed with provider name)\n├── utils.go # Constants, base URLs, helpers (camelCase for unexported)\n├── errors.go # Error parsing: provider HTTP error → *schemas.BifrostError\n├── chat.go # Chat request/response converters\n├── embedding.go # Embedding converters (if supported)\n├── images.go # Image generation (if supported)\n├── speech.go # TTS/STT (if supported)\n└── responses.go # Responses API + streaming converters\n```\n\n**Category 2: OpenAI-compatible** (Groq, Cerebras, Ollama, Perplexity, OpenRouter, Parasail, Nebius, xAI, SGL):\n```\ncore/providers//\n├── .go # Minimal — constructor + delegates to openai.HandleOpenAI* functions\n└── _test.go # Tests\n```\n\n**Converter function naming convention:**\n- `ToRequest()` — Bifrost schema → Provider API format\n- `ToBifrostResponse()` — Provider API format → Bifrost schema\n- These must be **pure transformation functions** — no HTTP calls, no logging, no side effects\n\n**Provider constructor pattern:**\n```go\nfunc NewProvider(config schemas.ProviderConfig) (*Provider, error) {\n // Validate config, set up fasthttp.Client with connection pooling\n client := &fasthttp.Client{\n MaxConnsPerHost: config.NetworkConfig.MaxConnsPerHost, // configurable, default 5000\n MaxIdleConnDuration: 30 * time.Second,\n }\n // After ConfigureProxy/ConfigureDialer/ConfigureTLS, build a sibling client\n // for streaming. BuildStreamingClient zeros ReadTimeout/WriteTimeout/MaxConnDuration\n // so streams aren't killed by fasthttp's whole-response deadline; per-chunk idle\n // is enforced at the app layer via NewIdleTimeoutReader.\n streamingClient := providerUtils.BuildStreamingClient(client)\n return &Provider{client: client, streamingClient: streamingClient, ...}, nil\n}\n```\n\n**Streaming vs unary client:** Every provider holds two clients — `client` for unary requests (`ReadTimeout=30s` bounds the whole response) and `streamingClient` for SSE / EventStream / chunked paths (`ReadTimeout=0`; the per-chunk `NewIdleTimeoutReader` is the only governor). Pass `provider.streamingClient` to every `Handle*Streaming` / `Handle*StreamRequest` helper and to direct `Do` calls inside `*Stream` methods. For new providers, apply the same pattern — missing the switch means streams get killed at 30s.\n\n**Note:** Bedrock uses `net/http` (not fasthttp) with HTTP/2 support. Its `http.Transport` is configured with `ForceAttemptHTTP2: true` and `MaxConnsPerHost` from `NetworkConfig` to allow multiple HTTP/2 connections when the server's per-connection stream limit (100 for AWS Bedrock) is reached. Use `providerUtils.BuildStreamingHTTPClient(client)` to derive the streaming variant — it shares the base `Transport` (safe for concurrent reuse) but clears `Client.Timeout`.\n\n### The Provider Interface\n\n`core/schemas/provider.go` defines the `Provider` interface with **30+ methods**. Every provider must implement all of them (returning \"not supported\" for unsupported operations). The interface covers:\n\n- `ListModels`, `ChatCompletion`, `ChatCompletionStream`\n- `Responses`, `ResponsesStream` (OpenAI Responses API)\n- `TextCompletion`, `TextCompletionStream`\n- `Embedding`, `Speech`, `SpeechStream`, `Transcription`, `TranscriptionStream`\n- `ImageGeneration`, `ImageGenerationStream`, `ImageEdit`, `ImageEditStream`, `ImageVariation`\n- `CountTokens`\n- `Batch*` (Create, List, Retrieve, Cancel, Results)\n- `File*` (Upload, List, Retrieve, Delete, Content)\n- `Container*` and `ContainerFile*` (Create, List, Retrieve, Delete, Content)\n\n**Streaming methods** receive a `PostHookRunner` callback and return `chan *BifrostStreamChunk`:\n```go\nChatCompletionStream(ctx *BifrostContext, postHookRunner PostHookRunner, key Key, request *BifrostChatRequest) (chan *BifrostStreamChunk, *BifrostError)\n```\n\n### Error Handling\n\nEach provider has `errors.go` with an `ErrorConverter` function:\n```go\ntype ErrorConverter func(resp *fasthttp.Response, requestType schemas.RequestType, providerName schemas.ModelProvider, model string) *schemas.BifrostError\n```\n\nThe shared utility `providerUtils.HandleProviderAPIError()` handles common HTTP error parsing. Provider-specific parsers add extra field mapping. Errors always carry metadata:\n```go\nbifrostErr.ExtraFields.Provider = providerName\nbifrostErr.ExtraFields.ModelRequested = model\nbifrostErr.ExtraFields.RequestType = requestType\n```\n\n### Plugin System\n\nFour plugin interfaces exist:\n\n| Interface | Hook Methods | When Called |\n|-----------|-------------|------------|\n| `LLMPlugin` | `PreLLMHook`, `PostLLMHook` | Every LLM request (SDK + HTTP) |\n| `MCPPlugin` | `PreMCPHook`, `PostMCPHook` | Every MCP tool execution |\n| `HTTPTransportPlugin` | `HTTPTransportPreHook`, `HTTPTransportPostHook`, `HTTPTransportStreamChunkHook` | HTTP gateway only (not Go SDK) |\n| `ObservabilityPlugin` | `Inject(ctx, trace)` | Async, after response written to wire |\n\n**Key plugin behaviors:**\n- Plugin errors are **logged as warnings**, never returned to the caller\n- Pre-hooks can **short-circuit** by returning `*LLMPluginShortCircuit` (cache hit, auth failure, rate limit)\n- Post-hooks receive both response and error — either can be nil. Plugins can **recover from errors** (set error to nil, provide response) or **invalidate responses** (set response to nil, provide error)\n- `BifrostError.AllowFallbacks` controls whether fallback providers are tried: `nil` or `&true` = allow, `&false` = block\n- `HTTPTransportStreamChunkHook` is called **per-chunk** during streaming — can modify, skip, or abort the stream\n\n### Pool System\n\n`core/pool/` provides `Pool[T]` with two build modes:\n\n```go\n// Production (default): zero-overhead sync.Pool wrapper\n// Debug (-tags pooldebug): tracks double-release, use-after-release, leaks with stack traces\np := pool.New[MyType](\"descriptive-name\", func() *MyType { return &MyType{} })\nobj := p.Get()\n// ... use obj ...\n// MUST reset ALL fields before Put — pool does not auto-reset\np.Put(obj)\n```\n\n**Acquire/Release pattern** for types with complex reset logic (used in `schemas/plugin.go`):\n```go\nreq := schemas.AcquireHTTPRequest() // Get from pool, pre-allocated maps\ndefer schemas.ReleaseHTTPRequest(req) // Clears all maps and fields, returns to pool\n```\n\n### HTTP Transport Layer\n\n**Handler pattern:** Handlers are structs with injected dependencies:\n```go\ntype CompletionHandler struct {\n client *bifrost.Bifrost\n handlerStore lib.HandlerStore\n config *lib.Config\n}\n```\n\n**Route registration:** Each handler implements `RegisterRoutes(router, middlewares...)` — routes get middleware chains applied per-route via `lib.ChainMiddlewares()`.\n\n**SDK integration layers** (`transports/bifrost-http/integrations/`) provide request/response converters between provider-native SDK formats and Bifrost's internal format. This enables drop-in replacement of OpenAI SDK, Anthropic SDK, AWS Bedrock SDK, Google GenAI SDK, LangChain, and LiteLLM.\n\n---\n\n## Gotchas\n\n### 1. Always Reset Pooled Objects Before Put\n\nEvery pooled object must have **all** fields zeroed before `pool.Put()`. Stale data leaks between requests. The debug build catches double-release and use-after-release but **not** missing resets.\n\n```go\n// WRONG — stale data from previous request leaks to next user\npool.Put(msg)\n\n// RIGHT\nmsg.Response = nil\nmsg.Error = nil\nmsg.Context = nil\nmsg.ResponseStream = nil\npool.Put(msg)\n```\n\n### 2. Channel Lifecycle — ProviderQueue Pattern\n\n`ProviderQueue` uses atomic flags and `sync.Once` to prevent \"send on closed channel\" panics:\n```go\ntype ProviderQueue struct {\n queue chan *ChannelMessage\n done chan struct{}\n closing uint32 // atomic: 0=open, 1=closing\n signalOnce sync.Once // ensure signal fires only once\n closeOnce sync.Once // ensure close fires only once\n}\n```\nAlways check the atomic closing flag before sending. Never close a channel without this pattern.\n\n### 3. NetworkConfig Duration Serialization\n\n`RetryBackoffInitial` and `RetryBackoffMax` are `time.Duration` (nanoseconds) in Go but **milliseconds** (integers) in JSON. Custom `MarshalJSON`/`UnmarshalJSON` handles conversion. If adding new duration fields to any config struct, follow this pattern exactly.\n\n### 4. ExtraHeaders — Defensive Map Copy\n\n`NetworkConfig.ExtraHeaders` is deep-copied in `CheckAndSetDefaults()` to prevent data races between concurrent requests. Apply the same `maps.Copy()` pattern to any new map fields in config structs.\n\n### 5. Provider Interface Has 30+ Methods\n\nAdding a new operation type requires changes across the entire codebase:\n1. Add method to `Provider` interface in `core/schemas/provider.go`\n2. Implement in **all** 20+ providers (most return \"not supported\")\n3. Add `RequestType` constant in `core/schemas/bifrost.go`\n4. Add to `AllowedRequests` struct and `IsOperationAllowed()` switch\n5. Add handler endpoint in `transports/bifrost-http/handlers/`\n6. Wire up in `core/bifrost.go` and `core/inference.go`\n\n### 6. OpenAI Provider Changes Cascade to 9+ Providers\n\nGroq, Cerebras, Ollama, Perplexity, OpenRouter, Parasail, Nebius, xAI, and SGL all delegate to `openai.HandleOpenAI*` functions. **Any change to OpenAI converter logic affects all of them.** Always test broadly: `make test-core` (all providers).\n\n### 7. Scanner Buffer Pool Has a Capacity Cap\n\nThe SSE scanner buffer pool in `core/providers/utils/utils.go` starts at 4KB. Buffers grow dynamically but those exceeding **64KB are discarded** (not returned to pool) to prevent memory bloat. Be aware when working with providers that send very large SSE events.\n\n### 8. Plugin Execution Order is Meaningful\n\nPre-hooks: registration order (first registered → first to run). Post-hooks: **reverse** order. This creates \"wrapping\" semantics — the first plugin registered is the outermost wrapper (its pre-hook runs first, post-hook runs last). Changing registration order changes behavior.\n\n### 9. Fallbacks Re-execute the Full Plugin Pipeline\n\nWhen a provider fails and the request falls to a fallback, the **entire plugin pipeline** re-executes from scratch. Governance checks, caching, and logging all run again for each attempt. Intentional, but surprising when debugging request counts or cost tracking.\n\n### 10. `AllowedRequests` Nil Semantics\n\nA **nil** `*AllowedRequests` means \"all operations allowed.\" A **non-nil** value only allows fields explicitly set to `true`. This applies to both `ProviderConfig.AllowedRequests` and `CustomProviderConfig.AllowedRequests`.\n\n### 11. BifrostContext Reserved Keys Are Silently Dropped\n\nWhen `BlockRestrictedWrites()` is active, writes to reserved keys (governance IDs, retry counts, fallback index, etc.) are **silently ignored** — no error. If your plugin needs to pass data through context, use your own custom key type.\n\n### 12. `fasthttp`, Not `net/http`\n\nBifrost uses `github.com/valyala/fasthttp` for provider HTTP calls. The API is different from `net/http`:\n- Use `fasthttp.AcquireRequest()`/`fasthttp.ReleaseRequest()` for lifecycle\n- `fasthttp.Client` pools connections per-host (`NetworkConfig.MaxConnsPerHost`, default 5000, 30s idle)\n- Request/response bodies accessed via `resp.Body()` (returns `[]byte`, not `io.Reader`)\n- **Exception:** Bedrock uses `net/http` (for AWS SigV4 signing) with `http.Transport` configured for HTTP/2 multi-connection support\n\n### 13. `sonic`, Not `encoding/json`\n\nJSON marshaling in hot paths uses `github.com/bytedance/sonic` for performance. `core/schemas/` uses standard `encoding/json` for custom marshaling (e.g., `NetworkConfig`). Don't mix them accidentally.\n\nFor reading or writing a **single field** (or a handful) inside a larger raw JSON payload, prefer `github.com/tidwall/gjson`/`github.com/tidwall/sjson` over decoding into `map[string]interface{}` and re-encoding — the shared helpers `providerUtils.GetJSONField`/`SetRawJSONField`/`DeleteJSONField`/`JSONFieldExists`/`GetJSONSubtree` (`core/providers/utils/utils.go`) wrap these and should be reused where the path is a lookup on an already-in-scope `[]byte`/`json.RawMessage`. Full-document decode into a map/struct is still correct when you need the whole shape (e.g. re-marshaling an entire object to normalize it) — the point is not to round-trip an entire object through a `map[string]interface{}` just to inspect one key. When marshaling back out, always use `providerUtils.MarshalSorted` (never a raw `sonic.Marshal`/`json.Marshal`), since unsorted map keys reorder nondeterministically and break prompt-cache-relevant byte stability.\n\n### 14. Atomic Pointer for Hot Config Reload\n\n`Bifrost` uses `atomic.Pointer` for providers and plugins lists. On updates: create new slice → atomically swap pointer. **Never mutate the slice in place** — concurrent readers would see partial state.\n\n### 15. MCP Tool Filtering is 4 Levels Deep\n\nTool access follows: Global filter → Client-level filter → Tool-level filter → Per-request filter (HTTP headers). All four levels must agree for a tool to be available. Changes to filtering logic must respect this hierarchy.\n\n### 16. `config.schema.json` is the Source of Truth\n\n`transports/config.schema.json` (~2700 lines) is the authoritative definition for all `config.json` fields. Documentation examples must match. When adding config fields: update schema first → handlers → docs.\n\n### 17. UI `data-testid` Attributes Are Load-Bearing\n\nE2E tests depend on `data-testid` attributes. Convention: `data-testid=\"--\"`. If you rename or remove one, search `tests/e2e/` for references. If you add new interactive elements, add `data-testid`.\n\n### 18. E2E Tests — Never Marshal Payloads to Maps\n\nIn `tests/e2e/core/`, **never marshal API payloads to a `Record`/`Map`/plain-object and then re-serialize**. Field ordering matters for backend validation and snapshot comparisons. Construct payloads as object literals with fields in the intended order and pass directly to Playwright's `request.post({ data })`. Avoid `Object.fromEntries()`, `JSON.parse(JSON.stringify(...))` round-trips, or destructuring into an intermediate `Record` — these can silently reorder fields.\n\n### 19. Framework Tests Need `tests/docker-compose.yml`, Not `framework/docker-compose.yml`\n\n`make test-framework` fails ~30 tests in `framework/vectorstore` with no services running. Bring the stack up first:\n\n```bash\ndocker compose -f tests/docker-compose.yml up -d\n```\n\nTwo compose files define overlapping services on the **same host ports** (9000, 6379, 6334, 5081), so only one can run at a time. Use the `tests/` one:\n\n| | `tests/docker-compose.yml` | `framework/docker-compose.yml` |\n|---|---|---|\n| Redis | plain 6379, **TLS 6380, cluster 7000, cluster-TLS 7100** | plain 6379 only |\n| TLS certs | `redis-certs-init` writes `tests/redis-certs/` | none |\n| Weaviate | 1.32.4, pins `CLUSTER_ADVERTISE_ADDR` | 1.25.0, no advertise addr |\n\nThe differences are load-bearing, not cosmetic:\n\n- `redis_test.go` dials **6380** and **7100** for the TLS and TLS-cluster client tests, and `readTestCACert` reads `tests/redis-certs/ca.crt`. The `framework/` file provides neither, so 5 tests fail against it.\n- Weaviate's memberlist aborts startup with `Failed to get final advertise address: No private IP address found` unless `CLUSTER_ADVERTISE_ADDR` is set ([weaviate#7474](https://github.com/weaviate/weaviate/issues/7474)). The `tests/` file pins a static IP; the `framework/` file does not, so its Weaviate crash-loops and 4 more tests fail.\n\nNote that `qdrant` and `pinecone` report `(unhealthy)` in `docker compose ps` under the `framework/` file because those images have no `wget` for the healthcheck. The services themselves are fine, so ignore that specific signal and probe the port instead.\n\nOnly `framework/vectorstore` needs any of this. Every other framework package passes with nothing running.\n\n---\n\n## Adding a New Provider — Full Checklist\n\n1. Create `core/providers//` with files per the pattern (see \"Provider Implementation\" above)\n2. Add `ModelProvider` constant in `core/schemas/bifrost.go`\n3. Add to `StandardProviders` list in `core/schemas/bifrost.go`\n4. Register in `core/bifrost.go` — add import + case in provider init switch\n5. **UI integration** (all required):\n - `ui/lib/constants/config.ts` — model placeholder + key requirement\n - `ui/lib/constants/icons.tsx` — provider icon\n - `ui/lib/constants/logs.ts` — provider display name (2 places)\n - `docs/openapi/openapi.json` — OpenAPI spec update\n - `transports/config.schema.json` — config schema (2 locations)\n6. **CI/CD**: Add env vars to `.github/workflows/pr-tests.yml` and `release-pipeline.yml` (4 jobs)\n7. **Docs**: Create `docs/providers/supported-providers/.mdx`\n8. **Test**: `make test-core PROVIDER=`\n\n---\n\n## Testing\n\n### Bug fixes: red before green\n\nBefore writing a fix, add (or extend) a test that reproduces the bug and confirm it fails for the expected reason — a wrong assertion, not a compile error or an unrelated panic. Only then implement the fix, and confirm the same test now passes. For bugs reachable through `make run-provider-harness-test`, add the harness regression case (see `.claude/skills/harness-test-writer/SKILL.md`) alongside Go-level tests: Go tests give a fast, free red/green loop while coding; the harness case is the live end-to-end pin, expected red pre-fix and green post-fix, validated structurally (`augment-provider-harness.mjs` / `filter-collection.mjs`) without needing a live paid run during development.\n\n### Every `core/` change ships with a provider-harness case\n\nAny change under `core/` that a client can observe on the wire must land together with a case in `tests/e2e/api/collections/provider-harness.json` (see `.claude/skills/harness-test-writer/SKILL.md`). This covers new features and refactors, not only bug fixes — the rule in the previous section is the narrower instance of this one.\n\n`core/` is the only layer every transport, integration and provider funnels through, so its behaviour is what the harness exists to pin. A Go unit test proves the function does what you meant; only the harness proves the bytes a real client sends still come back correct through the whole stack. The gap between those two is where regressions live: a fail-soft that fires on one request shape and silently skips a sibling shape passes every unit test it has.\n\nWrite the case so it is **red before the change and green after**, and validate it structurally while developing — no live paid run needed:\n\n```bash\nnode tests/e2e/api/runners/augment-provider-harness.mjs --source tests/e2e/api/collections/provider-harness.json --out tmp/harness-augmented.json\nnode tests/e2e/api/runners/filter-collection.mjs --source tmp/harness-augmented.json --out tmp/filtered.json --feature \"\"\n```\n\nInsert into the collection surgically (a script that splices the new object in, never a whole-file reserialize) — the file is ~50k lines and a reformat buries the actual change.\n\nThe narrow exemptions: changes with no wire-visible effect (comments, internal renames, log lines) and behaviour no HTTP request can reach. If a change is exempt, say so explicitly in the PR rather than leaving the omission unexplained.\n\n### Always prefer `make test-core` over raw `go test` for provider-level tests\n\nThe `make test-core` target is the canonical harness for provider tests — it wires up env vars from `.env` (provider API keys), invokes the per-provider `{provider}_test.go` entrypoint in `core/providers//`, and routes through the shared `core/internal/llmtests/` scenario suite that validates end-to-end behavior (including streaming).\n\nRunning bare `go test ./core/providers//...` only executes unit tests and skips the llmtests scenarios — so it won't catch regressions in streaming, tool-calling, or provider-specific response shapes.\n\n```bash\nmake test-core PROVIDER=anthropic TESTCASE=TestChatCompletionStream # exact test\nmake test-core PROVIDER=openai PATTERN=Stream # substring match\nmake test-core PROVIDER=bedrock # all scenarios for one provider\nmake test-core DEBUG=1 PROVIDER=gemini TESTCASE=TestResponsesStream # attach Delve on :2345\n```\n\n`PATTERN` and `TESTCASE` are mutually exclusive. Provider name must match a directory under `core/providers/` (e.g. `anthropic`, `openai`, `bedrock`, `vertex`, `azure`, `gemini`, `cohere`, `mistral`, `groq`, etc.).\n\n### LLM Tests (`core/internal/llmtests/`)\n\nScenario-based tests that run against **live provider APIs** with dual-API testing (Chat Completions + Responses API):\n\n```go\nfunc RunMyScenarioTest(t *testing.T, client *bifrost.Bifrost, ctx context.Context, cfg ComprehensiveTestConfig) {\n // Use validation presets: BasicChatExpectations(), ToolCallExpectations(), etc.\n // Use retry framework for flaky assertions\n}\n```\n\n- Register in `tests.go` `testScenarios` slice\n- Add `Scenarios.MyScenario` flag to `ComprehensiveTestConfig`\n- Run: `make test-core PROVIDER= TESTCASE=`\n\n### MCP Tests (`core/internal/mcptests/`)\n\nMock-based tests with `DynamicLLMMocker` and declarative setup:\n\n```go\nmanager, mocker, ctx := SetupAgentTest(t, AgentTestConfig{\n InProcessTools: []string{\"echo\", \"calculator\"},\n AutoExecuteTools: []string{\"*\"},\n MaxDepth: 5,\n})\n// Queue mock LLM responses, assert tool execution order\n```\n\nCategories: `agent_*_test.go`, `tool_*_test.go`, `connection_*_test.go`, `codemode_*_test.go`\n\nRun: `make test-mcp TESTCASE=`\n\n### E2E Tests (`tests/e2e/`)\n\nPlaywright tests with page objects, data factories, fixtures:\n\n- Page objects extend `BasePage`, use `getByTestId()` as primary selector strategy\n- Data factories use `Date.now()` for unique names (prevents collision in parallel runs)\n- Track created resources in arrays, clean up in `afterEach`\n- Import `test`/`expect` from `../../core/fixtures/base.fixture` (never from `@playwright/test`)\n- **Never marshal API payloads to a `Record`/`Map`/plain-object and then re-serialize.** Field ordering matters for snapshot comparisons and some backend validations. Construct payloads as object literals with fields in the intended order and pass directly to Playwright's `request.post({ data })`. Do NOT destructure into an intermediate `Record` or use `Object.fromEntries()` / `JSON.parse(JSON.stringify(...))` round-trips, as these can reorder fields.\n\nRun: `make run-e2e FLOW=`\n\n---\n\n## Claude Code Skills\n\nFour skills are available via `/skill-name`:\n\n### `/docs-writer `\nWrite, update, or review Mintlify MDX documentation. Researches UI code, Go handlers, and config schema. Validates `config.json` examples against `transports/config.schema.json`. Outputs docs with Web UI / API / config.json tabs.\n\nVariants: `/docs-writer update `, `/docs-writer review `\n\n### `/e2e-test `\nCreate, run, debug, audit, or auto-update Playwright E2E tests.\n\nVariants:\n- `/e2e-test fix ` — Debug and fix a failing test\n- `/e2e-test sync` — Detect UI changes, update affected tests automatically\n- `/e2e-test audit` — Scan specs for incorrect/weak assertions (P0-P6 severity scale)\n\n### `/investigate-issue `\nInvestigate a GitHub issue from `maximhq/bifrost`. Fetches issue details, classifies by type/area, searches codebase, traces dependencies, analyzes side effects, suggests tests (LLM/MCP/E2E), and presents an implementation plan with per-change approval gates.\n\n### `/resolve-pr-comments `\nSystematically address unresolved PR review comments. Uses GraphQL to get unresolved threads, presents each with FIX/REPLY/SKIP options, collects fixes locally, and only posts replies **after code is pushed** to remote.\n\n---\n\n## Common Workflows\n\n### Modify chat completions across all providers\n1. Change types in `core/schemas/chatcompletions.go`\n2. Update converter functions in each provider's `chat.go`\n3. If streaming affected, update `framework/streaming/` (accumulator, delta copy)\n4. Run `make test-core` (all providers)\n\n### Add a new field to API responses\n1. Add to schema type in `core/schemas/`\n2. Map in provider response converter (`ToBifrost*Response`)\n3. Handle in streaming accumulator if applicable\n4. Update HTTP handler if field needs special serialization\n5. Update `transports/config.schema.json` if configurable\n\n### Add a new plugin\n1. Create `plugins//` with its own `go.mod`\n2. Implement `LLMPlugin`, `MCPPlugin`, or `HTTPTransportPlugin` interface\n3. Add to `go.work`\n4. Register in transport layer or Bifrost config\n5. Add test targets to `Makefile`\n\n### Modify a UI feature\n1. Find workspace page: `ui/app/workspace//`\n2. Check existing `data-testid` attributes — E2E tests depend on them\n3. Add `data-testid` to new interactive elements\n4. Run `make run-e2e FLOW=` to verify\n5. If E2E tests break, use `/e2e-test sync` to update them\n\n---\n\n## Key Files Quick Reference\n\n| What | Where |\n|------|-------|\n| Main Bifrost struct & queuing | `core/bifrost.go` |\n| Inference routing & fallbacks | `core/inference.go` |\n| Provider interface (30+ methods) | `core/schemas/provider.go` |\n| ModelProvider enum & context keys | `core/schemas/bifrost.go` |\n| Plugin interfaces & pooled HTTP types | `core/schemas/plugin.go` |\n| BifrostContext (mutable context) | `core/schemas/context.go` |\n| Chat completion types | `core/schemas/chatcompletions.go` |\n| Responses API types | `core/schemas/responses.go` |\n| Object pool (prod + debug) | `core/pool/pool_prod.go`, `pool_debug.go` |\n| Shared provider utils & SSE parsing | `core/providers/utils/utils.go` |\n| Streaming accumulator | `framework/streaming/accumulator.go` |\n| HTTP inference handler | `transports/bifrost-http/handlers/inference.go` |\n| Governance handler | `transports/bifrost-http/handlers/governance.go` |\n| Config schema (source of truth) | `transports/config.schema.json` |\n| Pool debug profiler | `transports/bifrost-http/handlers/devpprof.go` |\n| LLM test infrastructure | `core/internal/llmtests/` |\n| MCP test infrastructure | `core/internal/mcptests/` |\n| E2E test infrastructure | `tests/e2e/core/` |\n| Docs navigation config | `docs/docs.json` |\n| CI/CD workflows | `.github/workflows/` |\n\n---\n\n## Code Style\n\n- **Go**: `gofmt`/`goimports`. No custom linter config.\n- **TypeScript/React**: Oxfmt. TanStack Router.\n- **JSON tags**: `snake_case` matching provider API conventions.\n- **Error strings**: Lowercase, no trailing punctuation (Go convention).\n- **Provider types**: Prefixed with provider name in PascalCase (`AnthropicChatRequest`, `GeminiEmbeddingResponse`).\n- **Converter functions**: Pure — no side effects, no logging, no HTTP.\n- **Pool names**: Descriptive string passed to `pool.New()` (e.g., `\"channel-message\"`, `\"response-stream\"`).\n- **Context keys**: Use `BifrostContextKey` type. Custom plugins should define their own key types to avoid collisions.\n- **Go filenames**: No underscores. The only permitted underscore is the `_test.go` suffix. Examples: `pluginpipeline.go`, `pluginpipeline_test.go` — never `plugin_pipeline.go` or `plugin_pipeline_race_test.go`. Concatenate words (lowercase, no separators) for multi-word filenames.\n\n# Frontend Code Guidelines & Patterns\n\nThis document defines the standards, structure, and best practices for writing frontend code in this project.\n\n---\n\n## Tech Stack\n\n- **React** (with Vite)\n- **TypeScript**\n- **@tanstack/react-router** (type-safe routing)\n- **Tailwind CSS v4**\n- **Radix UI** (primitives)\n- **Local UI component library** (`ui/components/ui/`) built on Radix primitives\n\n---\n\n## Folder Structure\n\n```text\n\n/ui\n├── app # Routes & pages\n├── components # Shared components\n│ └── ui # Core design system components\n├── hooks # Custom React hooks\n├── lib # Utilities, helpers, shared logic\n└── app/enterprise # Enterprise-specific code (via symlink)\n\n```\n\n### Rules\n\n- All frontend code must live inside `/ui`\n- Routes and pages → `ui/app`\n- Shared/reusable components → `ui/components`\n- Core UI primitives → `ui/components/ui`\n- Utilities and libraries → `ui/lib`\n- Custom hooks → `ui/hooks`\n\n---\n\n## Libraries & Usage\n\n### Core Libraries\n\n- `react` → UI library\n- `typescript` → Type safety\n- `tailwindcss` → Styling\n- `@tanstack/react-router` → Routing\n\n### UI & Visualization\n\n- `@radix-ui/react-*` → UI primitives\n- `ui/components/ui/*` → Project's Radix-based component system\n- `recharts` → Charts\n- `monaco-editor` → Code editor\n\n### Utilities\n\n- `date-fns` → Date/time formatting\n- `nuqs` → Query param state management\n\n### Tooling\n\n- `Oxfmt` → Code formatting\n- `vitest` → Testing\n\n---\n\n## Routing Convention\n\nFor every new route:\n\n```text\n\nui/app//\n├── layout.tsx # Route definition using createFileRoute\n├── page.tsx # Page content\n└── views/ # Optional: route-specific components\n\n```\n\n### Rules\n\n- Folder name must match route name\n- Always use `createFileRoute` in `layout.tsx`\n- `page.tsx` should only handle composition (not heavy logic)\n- Route-specific components go inside `views/`\n\n---\n\n## Component Guidelines\n\n### Reusability First\n\n- Always check if similar components/functions already exist\n- Prefer extending or refactoring existing code over duplication\n- Only create new components if reuse is not feasible\n\n---\n\n### Component Placement\n\n- Shared → `ui/components`\n- Route-specific → `views/` inside route folder\n\n---\n\n### Entity Selectors — never hand-roll an entity picker\n\nAny UI that lets a user pick an existing entity (virtual key, team, customer, user, business unit, …) **must** go through `ui/components/entitySelectors/`. Do not build a new `Select`/`Combobox` + `useState` + debounce + fetch stack for this — that pattern was already duplicated across surfaces and consolidated here.\n\n**Use an existing selector** — import it and pass one of the three modes:\n\n```tsx\nimport { VirtualKeySelector } from \"@/components/entitySelectors/virtualKeySelector\";\n\n // single\n // multi (chips inside the control)\n appendRow(o)} /> // fire-and-forget add\n```\n\nAvailable today: `virtualKeySelector`, `teamSelector`, `customerSelector` (OSS); `userSelector`, `businessUnitSelector` (enterprise — reached via registry, see below).\n\n**Always pass `fallbackOption` / `fallbackOptions`** when editing an existing row. Selectors fetch nothing until the popover opens, so a preselected id renders as a raw UUID otherwise.\n\n**Adding a selector for a new entity** — write a thin wrapper, never a new picker. Copy `customerSelector.tsx` (the simplest one) and change only what genuinely differs: the list query, the by-id label resolver, and the label/description fields. The wrapper must:\n\n1. Call `useEntitySelectorSearch()` for open/search/debounce state, and pass `skip` to the RTK Query hook — nothing is fetched until the picker opens.\n2. `useMemo` the `options` array. Multi mode feeds it to react-select as `defaultOptions`, which re-syncs on identity change and will loop if the identity churns.\n3. Ship a `LabelResolver` component (`EntityLabelResolverProps`) that fetches one entity by id and calls `onResolved` — this is what keeps selected-but-unfetched ids from rendering as UUIDs.\n4. Type its props as `OwnProps & EntitySelectorModeProps` and extend `EntitySelectorCommonProps`, so all three modes and the shared prop surface come for free.\n5. Default `limit` to `ENTITY_SELECTOR_PAGE_SIZE`; expose a `filters` prop only if the endpoint supports server-side scoping.\n6. Search is **server-side** — never fetch a page and filter it client-side.\n\nDo not edit `entitySelector.tsx` to accommodate one surface. It only carries behaviour identical across every entity; per-entity differences belong in the wrapper, per-surface differences in props (`trigger`, `triggerClassName`, `excludeIds`, `noPortal`, `className`).\n\n**OSS ↔ enterprise placement.** `entitySelector.tsx` and any selector whose API is OSS live in `ui/components/entitySelectors/`. A selector for an enterprise-only API lives in `bifrost-enterprise/enterprise-ui/app/components/entitySelectors/` and OSS must never import it directly — OSS reaches it through a runtime registry (`ui/lib/registries/userPicker.tsx`, `ui/lib/registries/modelLimitScopes.tsx`), with an empty fallback under `ui/app/_fallbacks/enterprise/` so OSS-only builds simply hide the option. Keep single mode prop-compatible with the registry contract (`{ value, onChange, disabled, fallbackOption }`) so the selector can be registered as-is.\n\n---\n\n### JSX & Rendering\n\n- Avoid deeply nested conditional rendering\n- Break complex UI into smaller components\n- Keep components readable and maintainable\n\n---\n\n### Lists & Keys\n\n- Always use **stable, unique keys**\n- Never use array index as key (unless unavoidable)\n\n---\n\n## React Best Practices\n\n- Avoid unnecessary or unstable dependencies in hooks\n- Prevent infinite loops in `useEffect`\n- Keep dependency arrays accurate and minimal\n- Prefer derived state over duplicated state\n\n---\n\n## State Management\n\n### Priority Order\n\n1. Query Params (`nuqs`) → for persistent/shareable state\n2. Local State → for UI-only state\n3. Redux → only when truly necessary\n\n---\n\n### Query Params (`nuqs`)\n\n- Use for state that should persist across refresh/navigation\n- Use proper parsers like `parseAsString` or `parseAsInteger`\n- Do NOT mix query param state with local/redux state\n- Follow a single consistent pattern across the codebase\n\n---\n\n### Redux\n\n- Use only when global/shared state is required\n- Avoid unnecessary slices\n- Prefer simpler alternatives when possible\n\n---\n\n### RTK Query (`@reduxjs/toolkit/query`)\n\n- Use for API calls and caching\n- Use **granular tags** for cache invalidation\n- Avoid invalidating entire datasets unnecessarily\n- Implement **optimistic updates** where applicable\n\n---\n\n## Forms\n\nWe use:\n\n- `react-hook-form`\n- `zod v4` (for schema validation)\n\n### Rules\n\n- Always define a Zod schema\n- Include meaningful validation messages\n- Prefer **inline field errors** (not toast notifications)\n- Use `refine` / `superRefine` for complex validation\n- Store schemas in: `ui/lib/types/schemas.ts`\n\n---\n\n## Tables\n\n- Use `@tanstack/react-table` **only for large/complex datasets**\n- For simple tables → build custom lightweight components\n- Prioritize performance over abstraction\n\n---\n\n## ⚡ Performance Guidelines\n\n- Lazy load heavy or rarely-used libraries\n- Avoid unnecessary re-renders\n- Split large components into smaller ones\n- Keep bundle size minimal\n\n---\n\n## Dependency Rules\n\n- Do NOT add new dependencies unless absolutely necessary\n- Always pin exact versions (no `^` or `~`)\n- Prefer existing libraries in the codebase\n\n---\n\n## TypeScript Guidelines\n\n- Avoid using `any` unless absolutely unavoidable\n- Prefer strict typing and inference\n- Define reusable types in shared locations\n\n---\n\n## Code Quality & Formatting\n\nAfter writing code:\n\n```bash\ncd ui && npm run format\n````\n\nThen verify build:\n\n```bash\ncd ui && npm run build\n```\n\n* Code must pass formatting and build checks\n* Follow consistent naming and structure conventions\n\n---\n\n## Anti-Patterns to Avoid\n\n* Duplicate components without considering reuse\n* Mixing multiple state management approaches unnecessarily\n* Overusing Redux\n* Using unstable hook dependencies\n* Adding heavy libraries for simple use cases\n* Poorly structured or deeply nested JSX\n\n---\n\n## Summary\n\n* Prioritize **reusability, performance, and consistency**\n* Follow **strict folder structure and routing conventions**\n* Use **the right tool for the right problem**\n* Keep code **simple, predictable, and maintainable**\n"},"files":{"AGENTS.md":"# AGENTS.md — Bifrost AI Gateway\n\n> Context for AI agents (Claude Code, Copilot, Cursor, etc.) working on this codebase. Read this fully before making changes.\n\n## What is Bifrost?\n\nBifrost is a high-performance AI gateway that unifies 20+ LLM providers behind a single OpenAI-compatible API with ~11µs overhead at 5,000 RPS. It also serves as an MCP (Model Context Protocol) gateway, turning static chat models into tool-calling agents.\n\nGitHub: `maximhq/bifrost`\n\n---\n\n## Repository Layout\n\n```\nbifrost/\n├── core/ # Go core library — the engine\n│ ├── bifrost.go # Main struct, request queuing, provider lifecycle (~3.4K lines)\n│ ├── inference.go # Inference routing, fallbacks, streaming dispatch (~1.9K lines)\n│ ├── mcp.go # MCP integration entry point\n│ ├── schemas/ # ALL shared Go types — 41 files\n│ │ ├── bifrost.go # BifrostConfig, ModelProvider enum, RequestType enum, context keys\n│ │ ├── provider.go # Provider interface (30+ methods), NetworkConfig, ProviderConfig\n│ │ ├── plugin.go # LLMPlugin, MCPPlugin, HTTPTransportPlugin, ObservabilityPlugin\n│ │ ├── context.go # BifrostContext (custom context.Context with mutable values)\n│ │ ├── chatcompletions.go # Chat completion request/response types\n│ │ ├── responses.go # OpenAI Responses API types\n│ │ ├── embedding.go # Embedding types\n│ │ ├── images.go # Image generation types\n│ │ ├── batch.go # Batch operation types\n│ │ ├── files.go # File management types\n│ │ ├── mcp.go # MCP types\n│ │ ├── trace.go # Tracer interface\n│ │ └── logger.go # Logger interface\n│ ├── providers/ # 20+ provider implementations\n│ │ ├── openai/ # Reference implementation (largest, most complete)\n│ │ ├── anthropic/ # Non-OpenAI-compatible example\n│ │ ├── bedrock/ # AWS event-stream protocol\n│ │ ├── gemini/ # Google-specific API shape\n│ │ ├── groq/ # OpenAI-compatible (minimal, delegates to openai/)\n│ │ └── utils/ # Shared: HTTP client, SSE parsing, error handling, scanner pool\n│ ├── pool/ # Generic Pool[T] — dual-mode (prod: sync.Pool, debug: full tracking)\n│ │ ├── pool_prod.go # Zero-overhead sync.Pool wrapper (default build)\n│ │ └── pool_debug.go # Double-release/use-after-release/leak detection (-tags pooldebug)\n│ ├── mcp/ # MCP protocol implementation\n│ │ ├── agent.go # Agent orchestration loop (multi-turn tool calling)\n│ │ ├── clientmanager.go # MCP client lifecycle management\n│ │ ├── toolmanager.go # Tool registration, discovery, filtering\n│ │ ├── healthmonitor.go # Client health monitoring\n│ │ └── codemode/starlark/ # Starlark sandbox for code-mode execution\n│ └── internal/\n│ ├── llmtests/ # LLM integration test infra (48 files, scenario-based)\n│ └── mcptests/ # MCP/Agent test infra (40+ files, mock-based)\n│\n├── framework/ # Data persistence, streaming, ecosystem utilities\n│ ├── configstore/ # Config storage backends (file, postgres)\n│ ├── logstore/ # Log storage backends (file, postgres)\n│ ├── vectorstore/ # Vector storage (Weaviate, Qdrant, Redis, Pinecone)\n│ ├── streaming/ # Streaming accumulator, delta copying, response marshaling\n│ │ ├── accumulator.go # Chunk accumulation into full response (~24KB)\n│ │ ├── chat.go # Chat stream handling (~17KB)\n│ │ └── responses.go # Response stream marshaling (~35KB)\n│ ├── modelcatalog/ # Model metadata registry\n│ ├── tracing/ # Distributed tracing helpers\n│ └── encrypt/ # Encryption utilities\n│\n├── transports/\n│ ├── config.schema.json # JSON Schema — THE source of truth for config.json (~2700 lines)\n│ └── bifrost-http/ # HTTP gateway transport\n│ ├── server/ # Server lifecycle, route registration\n│ ├── handlers/ # 27 HTTP endpoint handlers\n│ │ ├── inference.go # Chat/text completions, responses API (~109KB)\n│ │ ├── mcpinference.go # MCP tool execution\n│ │ ├── governance.go # Virtual keys, teams, customers, budgets (~100KB)\n│ │ ├── providers.go # Provider CRUD, key management\n│ │ ├── mcp.go # MCP client registry management\n│ │ ├── logging.go # Log queries, stats, histograms\n│ │ ├── config.go # System configuration\n│ │ ├── plugins.go # Plugin CRUD\n│ │ ├── cache.go # Cache management\n│ │ ├── session.go # Auth/session management\n│ │ ├── health.go # Health checks\n│ │ ├── mcpserver.go # MCP server (SSE/streamable HTTP)\n│ │ ├── websocket.go # WebSocket handler\n│ │ ├── devpprof.go # Pool debug profiler endpoint (~23KB)\n│ │ └── middlewares.go # Middleware definitions\n│ ├── lib/ # ChainMiddlewares, config, context conversion\n│ └── integrations/ # SDK compatibility layers\n│ ├── openai.go # OpenAI SDK drop-in compatibility\n│ ├── anthropic.go # Anthropic SDK compatibility\n│ ├── bedrock.go # AWS Bedrock SDK compatibility\n│ ├── genai.go # Google GenAI SDK compatibility\n│ ├── langchain.go # LangChain compatibility\n│ ├── litellm.go # LiteLLM compatibility\n│ └── pydanticai.go # PydanticAI compatibility\n│\n├── plugins/ # Go plugins — each has own go.mod\n│ ├── governance/ # Budget, rate limiting, virtual keys, routing, RBAC\n│ ├── telemetry/ # Prometheus metrics, push gateway\n│ ├── logging/ # Request/response audit logging\n│ ├── semanticcache/ # Semantic response caching via vector store\n│ ├── otel/ # OpenTelemetry tracing\n│ ├── mocker/ # Mock responses for testing\n│ ├── jsonparser/ # JSON extraction utilities\n│ ├── maxim/ # Maxim observability\n│ └── compat/ # LiteLLM SDK compatibility (HTTP transport)\n│\n├── ui/ # React + vite web interface\n│ ├── app/workspace/ # Feature pages (20+ workspace sections)\n│ ├── components/ # Shared React components\n│ └── lib/ # Constants, utilities, types\n│\n├── tests/e2e/ # Playwright E2E tests\n│ ├── core/ # Fixtures, page objects, helpers, API actions\n│ └── features/ # Per-feature test suites\n│\n├── docs/ # Mintlify MDX documentation\n│ ├── docs.json # Navigation config\n│ ├── media/ # Screenshots (ui-*.png naming convention)\n│ └── (architecture|features|providers|mcp|plugins|enterprise|...)\n│\n├── .claude/skills/ # Claude Code skill definitions (4 skills)\n├── go.work # Go workspace — requires Go 1.26.1\n├── Makefile # Build, test, dev commands (1300+ lines)\n└── terraform/ # Infrastructure as Code\n```\n\n---\n\n## Go Workspace\n\nBifrost is a **multi-module Go workspace**. Each module has its own `go.mod`:\n\n```\ngo.work\n├── core/go.mod # github.com/maximhq/bifrost/core\n├── framework/go.mod # github.com/maximhq/bifrost/framework\n├── transports/go.mod # github.com/maximhq/bifrost/transports\n└── plugins/*/go.mod # 9 plugin modules (governance, telemetry, logging, etc.)\n```\n\n**Rules:**\n- Run `go mod tidy` in the **specific module directory**, not the root\n- Cross-module imports resolve via workspace locally, but need explicit `require` in `go.mod` for releases\n- The workspace requires **Go 1.26.1** (`go.work` directive)\n\n---\n\n## Build, Test & Dev Commands\n\n```bash\n# Development\nmake dev # Full local dev (UI + API with hot reload via air)\nmake build # Build bifrost-http binary\n\n# Core tests (provider integration tests — hit live APIs)\nmake test-core # All providers\nmake test-core PROVIDER=openai # Specific provider\nmake test-core PROVIDER=openai TESTCASE=TestSimpleChat # Specific test\nmake test-core PATTERN=TestStreaming # Tests matching pattern\nmake test-core DEBUG=1 # With Delve debugger on :2345\n\n# MCP/Agent tests (mock-based, no live APIs)\nmake test-mcp # All MCP tests\nmake test-mcp TESTCASE=TestAgentLoop # Specific test\nmake test-mcp TYPE=agent # By category (agent|tool|connection|codemode)\n\n# Framework tests (require local backing services — bring them up FIRST)\ndocker compose -f tests/docker-compose.yml up -d # postgres, weaviate, qdrant, pinecone, and the 4 redis variants\nmake test-framework # All framework packages\n\n# Plugin tests\nmake test-plugins # All plugins\nmake test-governance # Governance plugin specifically\n\n# Integration tests (SDK compatibility)\nmake test-integrations-py # Python SDK tests\nmake test-integrations-ts # TypeScript SDK tests\n\n# E2E tests (Playwright, requires running dev server)\nmake run-e2e # All E2E tests\nmake run-e2e FLOW=providers # Specific feature\n\n# Code quality\nmake lint # Linting\nmake fmt # Format code\n```\n\n---\n\n## Architecture\n\n### Request Flow\n\n```\nClient HTTP Request\n → FastHTTP Transport (parsing, validation ~2µs)\n → SDK Integration Layer (OpenAI/Anthropic/Bedrock format → Bifrost format)\n → Middleware Chain (lib.ChainMiddlewares, applied per-route)\n → HTTPTransportPreHook (HTTP-level plugins, can short-circuit)\n → PreLLMHook Pipeline (auth, rate-limit, cache check — registration order)\n → MCP Tool Discovery & Injection (if tool_choice present)\n → Provider Queue (channel-based, per-provider isolation)\n → Worker picks up request\n → Key Selection (~10ns weighted random)\n → Provider API Call (fasthttp client, connection pooling)\n → Response / SSE Stream\n → PostLLMHook Pipeline (reverse order of PreLLMHooks)\n → Tool Execution Loop (if tool_calls in response, MCP agent loop)\n → HTTPTransportPostHook (reverse order)\n → Response Serialization\n → HTTP Response to Client\n```\n\n### Design Principles\n\n- **Provider isolation**: Each provider has its own worker pool and queue. One provider going down doesn't cascade to others.\n- **Channel-based async**: Request routing uses Go channels (`chan *ChannelMessage`), not mutexes. The `ProviderQueue` struct manages channel lifecycle with atomic flags.\n- **Object pooling everywhere**: `sync.Pool` wrappers reduce GC pressure. Pools exist for: channel messages, response channels, error channels, stream channels, plugin pipelines, MCP requests, HTTP request/response objects, scanner buffers.\n- **Plugin pipeline symmetry**: Pre-hooks execute in registration order, post-hooks in **reverse** order (LIFO). For every pre-hook executed, the corresponding post-hook is guaranteed to run.\n- **Streaming**: SSE chunks flow through `chan chan *schemas.BifrostStreamChunk`. Accumulated into full response for post-hooks via `framework/streaming/accumulator.go`.\n\n### BifrostContext — Custom Context\n\n`BifrostContext` (`core/schemas/context.go`) is a custom `context.Context` with **thread-safe mutable values**. Unlike standard Go contexts, values can be set after creation:\n\n```go\nctx := schemas.NewBifrostContext(parent, deadline)\nctx.SetValue(key, value) // Thread-safe, uses RWMutex\nctx.WithValue(key, value) // Chainable variant\n```\n\n**Reserved context keys** (set by Bifrost internals — DO NOT set manually):\n- `BifrostContextKeySelectedKeyID/Name` — Set by governance plugin\n- `BifrostContextKeyGovernance*` — Set by governance plugin\n- `BifrostContextKeyNumberOfRetries`, `BifrostContextKeyFallbackIndex` — Set by retry/fallback logic\n- `BifrostContextKeyStreamEndIndicator` — Set by streaming infrastructure\n- `BifrostContextKeyTrace*`, `BifrostContextKeySpan*` — Set by tracing middleware\n\n**User-settable keys** (plugins and handlers can set these):\n- `BifrostContextKeyVirtualKey` (`x-bf-vk`) — Virtual key for governance\n- `BifrostContextKeyAPIKeyName` (`x-bf-api-key`) — Explicit key selection by name\n- `BifrostContextKeyAPIKeyID` (`x-bf-api-key-id`) — Explicit key selection by ID (takes priority over name)\n- `BifrostContextKeyRequestID` — Request ID\n- `BifrostContextKeyExtraHeaders` — Extra headers to forward to provider\n- `BifrostContextKeyURLPath` — Custom URL path for provider\n- `BifrostContextKeySkipKeySelection` — Skip key selection (pass empty key)\n- `BifrostContextKeyUseRawRequestBody` — Send raw body directly to provider\n\n**Gotcha**: `BlockRestrictedWrites()` silently drops writes to reserved keys. This prevents plugins from accidentally overwriting internal state.\n\n**Hard rule — never store stream-sized data in `BifrostContext`.** Context holds small handles only: IDs, durations, booleans, interface pointers. Any per-request state that scales with stream content (chunk buffers, accumulated payloads, replay queues, large per-request slices/maps) must live in a top-level manager keyed by `RequestID`, not in `ctx`. Reference implementations:\n\n- `framework/streaming.Accumulator` — owns a `sync.Map` of per-stream `StreamAccumulator` entries keyed by `RequestID`. Only `BifrostContextKeyAccumulatorID` (the ID string) is stored on the context; the chunk buffers live in the manager. The pause/resume gate (`gate.go`) extends the same per-stream entry with a state machine — again, **no buffer in ctx**.\n- The `Tracer` interface (in ctx as a small pointer) is the access path for plugins/providers to reach managers without putting bulky data on the context itself.\n\nWhen in doubt: if your new ctx key would hold a slice/map that grows with request content, route the storage through a manager and keep only the ID in ctx.\n\n---\n\n## Core Patterns\n\n### Provider Implementation\n\nThere are **two categories** of providers:\n\n**Category 1: Non-OpenAI-compatible** (Anthropic, Bedrock, Gemini, Cohere, HuggingFace, Replicate, ElevenLabs):\n```\ncore/providers//\n├── .go # Controller: constructor, interface methods, HTTP orchestration\n├── _test.go # Tests\n├── types.go # ALL provider-specific structs (PascalCase prefixed with provider name)\n├── utils.go # Constants, base URLs, helpers (camelCase for unexported)\n├── errors.go # Error parsing: provider HTTP error → *schemas.BifrostError\n├── chat.go # Chat request/response converters\n├── embedding.go # Embedding converters (if supported)\n├── images.go # Image generation (if supported)\n├── speech.go # TTS/STT (if supported)\n└── responses.go # Responses API + streaming converters\n```\n\n**Category 2: OpenAI-compatible** (Groq, Cerebras, Ollama, Perplexity, OpenRouter, Parasail, Nebius, xAI, SGL):\n```\ncore/providers//\n├── .go # Minimal — constructor + delegates to openai.HandleOpenAI* functions\n└── _test.go # Tests\n```\n\n**Converter function naming convention:**\n- `ToRequest()` — Bifrost schema → Provider API format\n- `ToBifrostResponse()` — Provider API format → Bifrost schema\n- These must be **pure transformation functions** — no HTTP calls, no logging, no side effects\n\n**Provider constructor pattern:**\n```go\nfunc NewProvider(config schemas.ProviderConfig) (*Provider, error) {\n // Validate config, set up fasthttp.Client with connection pooling\n client := &fasthttp.Client{\n MaxConnsPerHost: config.NetworkConfig.MaxConnsPerHost, // configurable, default 5000\n MaxIdleConnDuration: 30 * time.Second,\n }\n // After ConfigureProxy/ConfigureDialer/ConfigureTLS, build a sibling client\n // for streaming. BuildStreamingClient zeros ReadTimeout/WriteTimeout/MaxConnDuration\n // so streams aren't killed by fasthttp's whole-response deadline; per-chunk idle\n // is enforced at the app layer via NewIdleTimeoutReader.\n streamingClient := providerUtils.BuildStreamingClient(client)\n return &Provider{client: client, streamingClient: streamingClient, ...}, nil\n}\n```\n\n**Streaming vs unary client:** Every provider holds two clients — `client` for unary requests (`ReadTimeout=30s` bounds the whole response) and `streamingClient` for SSE / EventStream / chunked paths (`ReadTimeout=0`; the per-chunk `NewIdleTimeoutReader` is the only governor). Pass `provider.streamingClient` to every `Handle*Streaming` / `Handle*StreamRequest` helper and to direct `Do` calls inside `*Stream` methods. For new providers, apply the same pattern — missing the switch means streams get killed at 30s.\n\n**Note:** Bedrock uses `net/http` (not fasthttp) with HTTP/2 support. Its `http.Transport` is configured with `ForceAttemptHTTP2: true` and `MaxConnsPerHost` from `NetworkConfig` to allow multiple HTTP/2 connections when the server's per-connection stream limit (100 for AWS Bedrock) is reached. Use `providerUtils.BuildStreamingHTTPClient(client)` to derive the streaming variant — it shares the base `Transport` (safe for concurrent reuse) but clears `Client.Timeout`.\n\n### The Provider Interface\n\n`core/schemas/provider.go` defines the `Provider` interface with **30+ methods**. Every provider must implement all of them (returning \"not supported\" for unsupported operations). The interface covers:\n\n- `ListModels`, `ChatCompletion`, `ChatCompletionStream`\n- `Responses`, `ResponsesStream` (OpenAI Responses API)\n- `TextCompletion`, `TextCompletionStream`\n- `Embedding`, `Speech`, `SpeechStream`, `Transcription`, `TranscriptionStream`\n- `ImageGeneration`, `ImageGenerationStream`, `ImageEdit`, `ImageEditStream`, `ImageVariation`\n- `CountTokens`\n- `Batch*` (Create, List, Retrieve, Cancel, Results)\n- `File*` (Upload, List, Retrieve, Delete, Content)\n- `Container*` and `ContainerFile*` (Create, List, Retrieve, Delete, Content)\n\n**Streaming methods** receive a `PostHookRunner` callback and return `chan *BifrostStreamChunk`:\n```go\nChatCompletionStream(ctx *BifrostContext, postHookRunner PostHookRunner, key Key, request *BifrostChatRequest) (chan *BifrostStreamChunk, *BifrostError)\n```\n\n### Error Handling\n\nEach provider has `errors.go` with an `ErrorConverter` function:\n```go\ntype ErrorConverter func(resp *fasthttp.Response, requestType schemas.RequestType, providerName schemas.ModelProvider, model string) *schemas.BifrostError\n```\n\nThe shared utility `providerUtils.HandleProviderAPIError()` handles common HTTP error parsing. Provider-specific parsers add extra field mapping. Errors always carry metadata:\n```go\nbifrostErr.ExtraFields.Provider = providerName\nbifrostErr.ExtraFields.ModelRequested = model\nbifrostErr.ExtraFields.RequestType = requestType\n```\n\n### Plugin System\n\nFour plugin interfaces exist:\n\n| Interface | Hook Methods | When Called |\n|-----------|-------------|------------|\n| `LLMPlugin` | `PreLLMHook`, `PostLLMHook` | Every LLM request (SDK + HTTP) |\n| `MCPPlugin` | `PreMCPHook`, `PostMCPHook` | Every MCP tool execution |\n| `HTTPTransportPlugin` | `HTTPTransportPreHook`, `HTTPTransportPostHook`, `HTTPTransportStreamChunkHook` | HTTP gateway only (not Go SDK) |\n| `ObservabilityPlugin` | `Inject(ctx, trace)` | Async, after response written to wire |\n\n**Key plugin behaviors:**\n- Plugin errors are **logged as warnings**, never returned to the caller\n- Pre-hooks can **short-circuit** by returning `*LLMPluginShortCircuit` (cache hit, auth failure, rate limit)\n- Post-hooks receive both response and error — either can be nil. Plugins can **recover from errors** (set error to nil, provide response) or **invalidate responses** (set response to nil, provide error)\n- `BifrostError.AllowFallbacks` controls whether fallback providers are tried: `nil` or `&true` = allow, `&false` = block\n- `HTTPTransportStreamChunkHook` is called **per-chunk** during streaming — can modify, skip, or abort the stream\n\n### Pool System\n\n`core/pool/` provides `Pool[T]` with two build modes:\n\n```go\n// Production (default): zero-overhead sync.Pool wrapper\n// Debug (-tags pooldebug): tracks double-release, use-after-release, leaks with stack traces\np := pool.New[MyType](\"descriptive-name\", func() *MyType { return &MyType{} })\nobj := p.Get()\n// ... use obj ...\n// MUST reset ALL fields before Put — pool does not auto-reset\np.Put(obj)\n```\n\n**Acquire/Release pattern** for types with complex reset logic (used in `schemas/plugin.go`):\n```go\nreq := schemas.AcquireHTTPRequest() // Get from pool, pre-allocated maps\ndefer schemas.ReleaseHTTPRequest(req) // Clears all maps and fields, returns to pool\n```\n\n### HTTP Transport Layer\n\n**Handler pattern:** Handlers are structs with injected dependencies:\n```go\ntype CompletionHandler struct {\n client *bifrost.Bifrost\n handlerStore lib.HandlerStore\n config *lib.Config\n}\n```\n\n**Route registration:** Each handler implements `RegisterRoutes(router, middlewares...)` — routes get middleware chains applied per-route via `lib.ChainMiddlewares()`.\n\n**SDK integration layers** (`transports/bifrost-http/integrations/`) provide request/response converters between provider-native SDK formats and Bifrost's internal format. This enables drop-in replacement of OpenAI SDK, Anthropic SDK, AWS Bedrock SDK, Google GenAI SDK, LangChain, and LiteLLM.\n\n---\n\n## Gotchas\n\n### 1. Always Reset Pooled Objects Before Put\n\nEvery pooled object must have **all** fields zeroed before `pool.Put()`. Stale data leaks between requests. The debug build catches double-release and use-after-release but **not** missing resets.\n\n```go\n// WRONG — stale data from previous request leaks to next user\npool.Put(msg)\n\n// RIGHT\nmsg.Response = nil\nmsg.Error = nil\nmsg.Context = nil\nmsg.ResponseStream = nil\npool.Put(msg)\n```\n\n### 2. Channel Lifecycle — ProviderQueue Pattern\n\n`ProviderQueue` uses atomic flags and `sync.Once` to prevent \"send on closed channel\" panics:\n```go\ntype ProviderQueue struct {\n queue chan *ChannelMessage\n done chan struct{}\n closing uint32 // atomic: 0=open, 1=closing\n signalOnce sync.Once // ensure signal fires only once\n closeOnce sync.Once // ensure close fires only once\n}\n```\nAlways check the atomic closing flag before sending. Never close a channel without this pattern.\n\n### 3. NetworkConfig Duration Serialization\n\n`RetryBackoffInitial` and `RetryBackoffMax` are `time.Duration` (nanoseconds) in Go but **milliseconds** (integers) in JSON. Custom `MarshalJSON`/`UnmarshalJSON` handles conversion. If adding new duration fields to any config struct, follow this pattern exactly.\n\n### 4. ExtraHeaders — Defensive Map Copy\n\n`NetworkConfig.ExtraHeaders` is deep-copied in `CheckAndSetDefaults()` to prevent data races between concurrent requests. Apply the same `maps.Copy()` pattern to any new map fields in config structs.\n\n### 5. Provider Interface Has 30+ Methods\n\nAdding a new operation type requires changes across the entire codebase:\n1. Add method to `Provider` interface in `core/schemas/provider.go`\n2. Implement in **all** 20+ providers (most return \"not supported\")\n3. Add `RequestType` constant in `core/schemas/bifrost.go`\n4. Add to `AllowedRequests` struct and `IsOperationAllowed()` switch\n5. Add handler endpoint in `transports/bifrost-http/handlers/`\n6. Wire up in `core/bifrost.go` and `core/inference.go`\n\n### 6. OpenAI Provider Changes Cascade to 9+ Providers\n\nGroq, Cerebras, Ollama, Perplexity, OpenRouter, Parasail, Nebius, xAI, and SGL all delegate to `openai.HandleOpenAI*` functions. **Any change to OpenAI converter logic affects all of them.** Always test broadly: `make test-core` (all providers).\n\n### 7. Scanner Buffer Pool Has a Capacity Cap\n\nThe SSE scanner buffer pool in `core/providers/utils/utils.go` starts at 4KB. Buffers grow dynamically but those exceeding **64KB are discarded** (not returned to pool) to prevent memory bloat. Be aware when working with providers that send very large SSE events.\n\n### 8. Plugin Execution Order is Meaningful\n\nPre-hooks: registration order (first registered → first to run). Post-hooks: **reverse** order. This creates \"wrapping\" semantics — the first plugin registered is the outermost wrapper (its pre-hook runs first, post-hook runs last). Changing registration order changes behavior.\n\n### 9. Fallbacks Re-execute the Full Plugin Pipeline\n\nWhen a provider fails and the request falls to a fallback, the **entire plugin pipeline** re-executes from scratch. Governance checks, caching, and logging all run again for each attempt. Intentional, but surprising when debugging request counts or cost tracking.\n\n### 10. `AllowedRequests` Nil Semantics\n\nA **nil** `*AllowedRequests` means \"all operations allowed.\" A **non-nil** value only allows fields explicitly set to `true`. This applies to both `ProviderConfig.AllowedRequests` and `CustomProviderConfig.AllowedRequests`.\n\n### 11. BifrostContext Reserved Keys Are Silently Dropped\n\nWhen `BlockRestrictedWrites()` is active, writes to reserved keys (governance IDs, retry counts, fallback index, etc.) are **silently ignored** — no error. If your plugin needs to pass data through context, use your own custom key type.\n\n### 12. `fasthttp`, Not `net/http`\n\nBifrost uses `github.com/valyala/fasthttp` for provider HTTP calls. The API is different from `net/http`:\n- Use `fasthttp.AcquireRequest()`/`fasthttp.ReleaseRequest()` for lifecycle\n- `fasthttp.Client` pools connections per-host (`NetworkConfig.MaxConnsPerHost`, default 5000, 30s idle)\n- Request/response bodies accessed via `resp.Body()` (returns `[]byte`, not `io.Reader`)\n- **Exception:** Bedrock uses `net/http` (for AWS SigV4 signing) with `http.Transport` configured for HTTP/2 multi-connection support\n\n### 13. `sonic`, Not `encoding/json`\n\nJSON marshaling in hot paths uses `github.com/bytedance/sonic` for performance. `core/schemas/` uses standard `encoding/json` for custom marshaling (e.g., `NetworkConfig`). Don't mix them accidentally.\n\nFor reading or writing a **single field** (or a handful) inside a larger raw JSON payload, prefer `github.com/tidwall/gjson`/`github.com/tidwall/sjson` over decoding into `map[string]interface{}` and re-encoding — the shared helpers `providerUtils.GetJSONField`/`SetRawJSONField`/`DeleteJSONField`/`JSONFieldExists`/`GetJSONSubtree` (`core/providers/utils/utils.go`) wrap these and should be reused where the path is a lookup on an already-in-scope `[]byte`/`json.RawMessage`. Full-document decode into a map/struct is still correct when you need the whole shape (e.g. re-marshaling an entire object to normalize it) — the point is not to round-trip an entire object through a `map[string]interface{}` just to inspect one key. When marshaling back out, always use `providerUtils.MarshalSorted` (never a raw `sonic.Marshal`/`json.Marshal`), since unsorted map keys reorder nondeterministically and break prompt-cache-relevant byte stability.\n\n### 14. Atomic Pointer for Hot Config Reload\n\n`Bifrost` uses `atomic.Pointer` for providers and plugins lists. On updates: create new slice → atomically swap pointer. **Never mutate the slice in place** — concurrent readers would see partial state.\n\n### 15. MCP Tool Filtering is 4 Levels Deep\n\nTool access follows: Global filter → Client-level filter → Tool-level filter → Per-request filter (HTTP headers). All four levels must agree for a tool to be available. Changes to filtering logic must respect this hierarchy.\n\n### 16. `config.schema.json` is the Source of Truth\n\n`transports/config.schema.json` (~2700 lines) is the authoritative definition for all `config.json` fields. Documentation examples must match. When adding config fields: update schema first → handlers → docs.\n\n### 17. UI `data-testid` Attributes Are Load-Bearing\n\nE2E tests depend on `data-testid` attributes. Convention: `data-testid=\"--\"`. If you rename or remove one, search `tests/e2e/` for references. If you add new interactive elements, add `data-testid`.\n\n### 18. E2E Tests — Never Marshal Payloads to Maps\n\nIn `tests/e2e/core/`, **never marshal API payloads to a `Record`/`Map`/plain-object and then re-serialize**. Field ordering matters for backend validation and snapshot comparisons. Construct payloads as object literals with fields in the intended order and pass directly to Playwright's `request.post({ data })`. Avoid `Object.fromEntries()`, `JSON.parse(JSON.stringify(...))` round-trips, or destructuring into an intermediate `Record` — these can silently reorder fields.\n\n### 19. Framework Tests Need `tests/docker-compose.yml`, Not `framework/docker-compose.yml`\n\n`make test-framework` fails ~30 tests in `framework/vectorstore` with no services running. Bring the stack up first:\n\n```bash\ndocker compose -f tests/docker-compose.yml up -d\n```\n\nTwo compose files define overlapping services on the **same host ports** (9000, 6379, 6334, 5081), so only one can run at a time. Use the `tests/` one:\n\n| | `tests/docker-compose.yml` | `framework/docker-compose.yml` |\n|---|---|---|\n| Redis | plain 6379, **TLS 6380, cluster 7000, cluster-TLS 7100** | plain 6379 only |\n| TLS certs | `redis-certs-init` writes `tests/redis-certs/` | none |\n| Weaviate | 1.32.4, pins `CLUSTER_ADVERTISE_ADDR` | 1.25.0, no advertise addr |\n\nThe differences are load-bearing, not cosmetic:\n\n- `redis_test.go` dials **6380** and **7100** for the TLS and TLS-cluster client tests, and `readTestCACert` reads `tests/redis-certs/ca.crt`. The `framework/` file provides neither, so 5 tests fail against it.\n- Weaviate's memberlist aborts startup with `Failed to get final advertise address: No private IP address found` unless `CLUSTER_ADVERTISE_ADDR` is set ([weaviate#7474](https://github.com/weaviate/weaviate/issues/7474)). The `tests/` file pins a static IP; the `framework/` file does not, so its Weaviate crash-loops and 4 more tests fail.\n\nNote that `qdrant` and `pinecone` report `(unhealthy)` in `docker compose ps` under the `framework/` file because those images have no `wget` for the healthcheck. The services themselves are fine, so ignore that specific signal and probe the port instead.\n\nOnly `framework/vectorstore` needs any of this. Every other framework package passes with nothing running.\n\n---\n\n## Adding a New Provider — Full Checklist\n\n1. Create `core/providers//` with files per the pattern (see \"Provider Implementation\" above)\n2. Add `ModelProvider` constant in `core/schemas/bifrost.go`\n3. Add to `StandardProviders` list in `core/schemas/bifrost.go`\n4. Register in `core/bifrost.go` — add import + case in provider init switch\n5. **UI integration** (all required):\n - `ui/lib/constants/config.ts` — model placeholder + key requirement\n - `ui/lib/constants/icons.tsx` — provider icon\n - `ui/lib/constants/logs.ts` — provider display name (2 places)\n - `docs/openapi/openapi.json` — OpenAPI spec update\n - `transports/config.schema.json` — config schema (2 locations)\n6. **CI/CD**: Add env vars to `.github/workflows/pr-tests.yml` and `release-pipeline.yml` (4 jobs)\n7. **Docs**: Create `docs/providers/supported-providers/.mdx`\n8. **Test**: `make test-core PROVIDER=`\n\n---\n\n## Testing\n\n### Bug fixes: red before green\n\nBefore writing a fix, add (or extend) a test that reproduces the bug and confirm it fails for the expected reason — a wrong assertion, not a compile error or an unrelated panic. Only then implement the fix, and confirm the same test now passes. For bugs reachable through `make run-provider-harness-test`, add the harness regression case (see `.claude/skills/harness-test-writer/SKILL.md`) alongside Go-level tests: Go tests give a fast, free red/green loop while coding; the harness case is the live end-to-end pin, expected red pre-fix and green post-fix, validated structurally (`augment-provider-harness.mjs` / `filter-collection.mjs`) without needing a live paid run during development.\n\n### Every `core/` change ships with a provider-harness case\n\nAny change under `core/` that a client can observe on the wire must land together with a case in `tests/e2e/api/collections/provider-harness.json` (see `.claude/skills/harness-test-writer/SKILL.md`). This covers new features and refactors, not only bug fixes — the rule in the previous section is the narrower instance of this one.\n\n`core/` is the only layer every transport, integration and provider funnels through, so its behaviour is what the harness exists to pin. A Go unit test proves the function does what you meant; only the harness proves the bytes a real client sends still come back correct through the whole stack. The gap between those two is where regressions live: a fail-soft that fires on one request shape and silently skips a sibling shape passes every unit test it has.\n\nWrite the case so it is **red before the change and green after**, and validate it structurally while developing — no live paid run needed:\n\n```bash\nnode tests/e2e/api/runners/augment-provider-harness.mjs --source tests/e2e/api/collections/provider-harness.json --out tmp/harness-augmented.json\nnode tests/e2e/api/runners/filter-collection.mjs --source tmp/harness-augmented.json --out tmp/filtered.json --feature \"\"\n```\n\nInsert into the collection surgically (a script that splices the new object in, never a whole-file reserialize) — the file is ~50k lines and a reformat buries the actual change.\n\nThe narrow exemptions: changes with no wire-visible effect (comments, internal renames, log lines) and behaviour no HTTP request can reach. If a change is exempt, say so explicitly in the PR rather than leaving the omission unexplained.\n\n### Always prefer `make test-core` over raw `go test` for provider-level tests\n\nThe `make test-core` target is the canonical harness for provider tests — it wires up env vars from `.env` (provider API keys), invokes the per-provider `{provider}_test.go` entrypoint in `core/providers//`, and routes through the shared `core/internal/llmtests/` scenario suite that validates end-to-end behavior (including streaming).\n\nRunning bare `go test ./core/providers//...` only executes unit tests and skips the llmtests scenarios — so it won't catch regressions in streaming, tool-calling, or provider-specific response shapes.\n\n```bash\nmake test-core PROVIDER=anthropic TESTCASE=TestChatCompletionStream # exact test\nmake test-core PROVIDER=openai PATTERN=Stream # substring match\nmake test-core PROVIDER=bedrock # all scenarios for one provider\nmake test-core DEBUG=1 PROVIDER=gemini TESTCASE=TestResponsesStream # attach Delve on :2345\n```\n\n`PATTERN` and `TESTCASE` are mutually exclusive. Provider name must match a directory under `core/providers/` (e.g. `anthropic`, `openai`, `bedrock`, `vertex`, `azure`, `gemini`, `cohere`, `mistral`, `groq`, etc.).\n\n### LLM Tests (`core/internal/llmtests/`)\n\nScenario-based tests that run against **live provider APIs** with dual-API testing (Chat Completions + Responses API):\n\n```go\nfunc RunMyScenarioTest(t *testing.T, client *bifrost.Bifrost, ctx context.Context, cfg ComprehensiveTestConfig) {\n // Use validation presets: BasicChatExpectations(), ToolCallExpectations(), etc.\n // Use retry framework for flaky assertions\n}\n```\n\n- Register in `tests.go` `testScenarios` slice\n- Add `Scenarios.MyScenario` flag to `ComprehensiveTestConfig`\n- Run: `make test-core PROVIDER= TESTCASE=`\n\n### MCP Tests (`core/internal/mcptests/`)\n\nMock-based tests with `DynamicLLMMocker` and declarative setup:\n\n```go\nmanager, mocker, ctx := SetupAgentTest(t, AgentTestConfig{\n InProcessTools: []string{\"echo\", \"calculator\"},\n AutoExecuteTools: []string{\"*\"},\n MaxDepth: 5,\n})\n// Queue mock LLM responses, assert tool execution order\n```\n\nCategories: `agent_*_test.go`, `tool_*_test.go`, `connection_*_test.go`, `codemode_*_test.go`\n\nRun: `make test-mcp TESTCASE=`\n\n### E2E Tests (`tests/e2e/`)\n\nPlaywright tests with page objects, data factories, fixtures:\n\n- Page objects extend `BasePage`, use `getByTestId()` as primary selector strategy\n- Data factories use `Date.now()` for unique names (prevents collision in parallel runs)\n- Track created resources in arrays, clean up in `afterEach`\n- Import `test`/`expect` from `../../core/fixtures/base.fixture` (never from `@playwright/test`)\n- **Never marshal API payloads to a `Record`/`Map`/plain-object and then re-serialize.** Field ordering matters for snapshot comparisons and some backend validations. Construct payloads as object literals with fields in the intended order and pass directly to Playwright's `request.post({ data })`. Do NOT destructure into an intermediate `Record` or use `Object.fromEntries()` / `JSON.parse(JSON.stringify(...))` round-trips, as these can reorder fields.\n\nRun: `make run-e2e FLOW=`\n\n---\n\n## Claude Code Skills\n\nFour skills are available via `/skill-name`:\n\n### `/docs-writer `\nWrite, update, or review Mintlify MDX documentation. Researches UI code, Go handlers, and config schema. Validates `config.json` examples against `transports/config.schema.json`. Outputs docs with Web UI / API / config.json tabs.\n\nVariants: `/docs-writer update `, `/docs-writer review `\n\n### `/e2e-test `\nCreate, run, debug, audit, or auto-update Playwright E2E tests.\n\nVariants:\n- `/e2e-test fix ` — Debug and fix a failing test\n- `/e2e-test sync` — Detect UI changes, update affected tests automatically\n- `/e2e-test audit` — Scan specs for incorrect/weak assertions (P0-P6 severity scale)\n\n### `/investigate-issue `\nInvestigate a GitHub issue from `maximhq/bifrost`. Fetches issue details, classifies by type/area, searches codebase, traces dependencies, analyzes side effects, suggests tests (LLM/MCP/E2E), and presents an implementation plan with per-change approval gates.\n\n### `/resolve-pr-comments `\nSystematically address unresolved PR review comments. Uses GraphQL to get unresolved threads, presents each with FIX/REPLY/SKIP options, collects fixes locally, and only posts replies **after code is pushed** to remote.\n\n---\n\n## Common Workflows\n\n### Modify chat completions across all providers\n1. Change types in `core/schemas/chatcompletions.go`\n2. Update converter functions in each provider's `chat.go`\n3. If streaming affected, update `framework/streaming/` (accumulator, delta copy)\n4. Run `make test-core` (all providers)\n\n### Add a new field to API responses\n1. Add to schema type in `core/schemas/`\n2. Map in provider response converter (`ToBifrost*Response`)\n3. Handle in streaming accumulator if applicable\n4. Update HTTP handler if field needs special serialization\n5. Update `transports/config.schema.json` if configurable\n\n### Add a new plugin\n1. Create `plugins//` with its own `go.mod`\n2. Implement `LLMPlugin`, `MCPPlugin`, or `HTTPTransportPlugin` interface\n3. Add to `go.work`\n4. Register in transport layer or Bifrost config\n5. Add test targets to `Makefile`\n\n### Modify a UI feature\n1. Find workspace page: `ui/app/workspace//`\n2. Check existing `data-testid` attributes — E2E tests depend on them\n3. Add `data-testid` to new interactive elements\n4. Run `make run-e2e FLOW=` to verify\n5. If E2E tests break, use `/e2e-test sync` to update them\n\n---\n\n## Key Files Quick Reference\n\n| What | Where |\n|------|-------|\n| Main Bifrost struct & queuing | `core/bifrost.go` |\n| Inference routing & fallbacks | `core/inference.go` |\n| Provider interface (30+ methods) | `core/schemas/provider.go` |\n| ModelProvider enum & context keys | `core/schemas/bifrost.go` |\n| Plugin interfaces & pooled HTTP types | `core/schemas/plugin.go` |\n| BifrostContext (mutable context) | `core/schemas/context.go` |\n| Chat completion types | `core/schemas/chatcompletions.go` |\n| Responses API types | `core/schemas/responses.go` |\n| Object pool (prod + debug) | `core/pool/pool_prod.go`, `pool_debug.go` |\n| Shared provider utils & SSE parsing | `core/providers/utils/utils.go` |\n| Streaming accumulator | `framework/streaming/accumulator.go` |\n| HTTP inference handler | `transports/bifrost-http/handlers/inference.go` |\n| Governance handler | `transports/bifrost-http/handlers/governance.go` |\n| Config schema (source of truth) | `transports/config.schema.json` |\n| Pool debug profiler | `transports/bifrost-http/handlers/devpprof.go` |\n| LLM test infrastructure | `core/internal/llmtests/` |\n| MCP test infrastructure | `core/internal/mcptests/` |\n| E2E test infrastructure | `tests/e2e/core/` |\n| Docs navigation config | `docs/docs.json` |\n| CI/CD workflows | `.github/workflows/` |\n\n---\n\n## Code Style\n\n- **Go**: `gofmt`/`goimports`. No custom linter config.\n- **TypeScript/React**: Oxfmt. TanStack Router.\n- **JSON tags**: `snake_case` matching provider API conventions.\n- **Error strings**: Lowercase, no trailing punctuation (Go convention).\n- **Provider types**: Prefixed with provider name in PascalCase (`AnthropicChatRequest`, `GeminiEmbeddingResponse`).\n- **Converter functions**: Pure — no side effects, no logging, no HTTP.\n- **Pool names**: Descriptive string passed to `pool.New()` (e.g., `\"channel-message\"`, `\"response-stream\"`).\n- **Context keys**: Use `BifrostContextKey` type. Custom plugins should define their own key types to avoid collisions.\n- **Go filenames**: No underscores. The only permitted underscore is the `_test.go` suffix. Examples: `pluginpipeline.go`, `pluginpipeline_test.go` — never `plugin_pipeline.go` or `plugin_pipeline_race_test.go`. Concatenate words (lowercase, no separators) for multi-word filenames.\n\n# Frontend Code Guidelines & Patterns\n\nThis document defines the standards, structure, and best practices for writing frontend code in this project.\n\n---\n\n## Tech Stack\n\n- **React** (with Vite)\n- **TypeScript**\n- **@tanstack/react-router** (type-safe routing)\n- **Tailwind CSS v4**\n- **Radix UI** (primitives)\n- **Local UI component library** (`ui/components/ui/`) built on Radix primitives\n\n---\n\n## Folder Structure\n\n```text\n\n/ui\n├── app # Routes & pages\n├── components # Shared components\n│ └── ui # Core design system components\n├── hooks # Custom React hooks\n├── lib # Utilities, helpers, shared logic\n└── app/enterprise # Enterprise-specific code (via symlink)\n\n```\n\n### Rules\n\n- All frontend code must live inside `/ui`\n- Routes and pages → `ui/app`\n- Shared/reusable components → `ui/components`\n- Core UI primitives → `ui/components/ui`\n- Utilities and libraries → `ui/lib`\n- Custom hooks → `ui/hooks`\n\n---\n\n## Libraries & Usage\n\n### Core Libraries\n\n- `react` → UI library\n- `typescript` → Type safety\n- `tailwindcss` → Styling\n- `@tanstack/react-router` → Routing\n\n### UI & Visualization\n\n- `@radix-ui/react-*` → UI primitives\n- `ui/components/ui/*` → Project's Radix-based component system\n- `recharts` → Charts\n- `monaco-editor` → Code editor\n\n### Utilities\n\n- `date-fns` → Date/time formatting\n- `nuqs` → Query param state management\n\n### Tooling\n\n- `Oxfmt` → Code formatting\n- `vitest` → Testing\n\n---\n\n## Routing Convention\n\nFor every new route:\n\n```text\n\nui/app//\n├── layout.tsx # Route definition using createFileRoute\n├── page.tsx # Page content\n└── views/ # Optional: route-specific components\n\n```\n\n### Rules\n\n- Folder name must match route name\n- Always use `createFileRoute` in `layout.tsx`\n- `page.tsx` should only handle composition (not heavy logic)\n- Route-specific components go inside `views/`\n\n---\n\n## Component Guidelines\n\n### Reusability First\n\n- Always check if similar components/functions already exist\n- Prefer extending or refactoring existing code over duplication\n- Only create new components if reuse is not feasible\n\n---\n\n### Component Placement\n\n- Shared → `ui/components`\n- Route-specific → `views/` inside route folder\n\n---\n\n### Entity Selectors — never hand-roll an entity picker\n\nAny UI that lets a user pick an existing entity (virtual key, team, customer, user, business unit, …) **must** go through `ui/components/entitySelectors/`. Do not build a new `Select`/`Combobox` + `useState` + debounce + fetch stack for this — that pattern was already duplicated across surfaces and consolidated here.\n\n**Use an existing selector** — import it and pass one of the three modes:\n\n```tsx\nimport { VirtualKeySelector } from \"@/components/entitySelectors/virtualKeySelector\";\n\n // single\n // multi (chips inside the control)\n appendRow(o)} /> // fire-and-forget add\n```\n\nAvailable today: `virtualKeySelector`, `teamSelector`, `customerSelector` (OSS); `userSelector`, `businessUnitSelector` (enterprise — reached via registry, see below).\n\n**Always pass `fallbackOption` / `fallbackOptions`** when editing an existing row. Selectors fetch nothing until the popover opens, so a preselected id renders as a raw UUID otherwise.\n\n**Adding a selector for a new entity** — write a thin wrapper, never a new picker. Copy `customerSelector.tsx` (the simplest one) and change only what genuinely differs: the list query, the by-id label resolver, and the label/description fields. The wrapper must:\n\n1. Call `useEntitySelectorSearch()` for open/search/debounce state, and pass `skip` to the RTK Query hook — nothing is fetched until the picker opens.\n2. `useMemo` the `options` array. Multi mode feeds it to react-select as `defaultOptions`, which re-syncs on identity change and will loop if the identity churns.\n3. Ship a `LabelResolver` component (`EntityLabelResolverProps`) that fetches one entity by id and calls `onResolved` — this is what keeps selected-but-unfetched ids from rendering as UUIDs.\n4. Type its props as `OwnProps & EntitySelectorModeProps` and extend `EntitySelectorCommonProps`, so all three modes and the shared prop surface come for free.\n5. Default `limit` to `ENTITY_SELECTOR_PAGE_SIZE`; expose a `filters` prop only if the endpoint supports server-side scoping.\n6. Search is **server-side** — never fetch a page and filter it client-side.\n\nDo not edit `entitySelector.tsx` to accommodate one surface. It only carries behaviour identical across every entity; per-entity differences belong in the wrapper, per-surface differences in props (`trigger`, `triggerClassName`, `excludeIds`, `noPortal`, `className`).\n\n**OSS ↔ enterprise placement.** `entitySelector.tsx` and any selector whose API is OSS live in `ui/components/entitySelectors/`. A selector for an enterprise-only API lives in `bifrost-enterprise/enterprise-ui/app/components/entitySelectors/` and OSS must never import it directly — OSS reaches it through a runtime registry (`ui/lib/registries/userPicker.tsx`, `ui/lib/registries/modelLimitScopes.tsx`), with an empty fallback under `ui/app/_fallbacks/enterprise/` so OSS-only builds simply hide the option. Keep single mode prop-compatible with the registry contract (`{ value, onChange, disabled, fallbackOption }`) so the selector can be registered as-is.\n\n---\n\n### JSX & Rendering\n\n- Avoid deeply nested conditional rendering\n- Break complex UI into smaller components\n- Keep components readable and maintainable\n\n---\n\n### Lists & Keys\n\n- Always use **stable, unique keys**\n- Never use array index as key (unless unavoidable)\n\n---\n\n## React Best Practices\n\n- Avoid unnecessary or unstable dependencies in hooks\n- Prevent infinite loops in `useEffect`\n- Keep dependency arrays accurate and minimal\n- Prefer derived state over duplicated state\n\n---\n\n## State Management\n\n### Priority Order\n\n1. Query Params (`nuqs`) → for persistent/shareable state\n2. Local State → for UI-only state\n3. Redux → only when truly necessary\n\n---\n\n### Query Params (`nuqs`)\n\n- Use for state that should persist across refresh/navigation\n- Use proper parsers like `parseAsString` or `parseAsInteger`\n- Do NOT mix query param state with local/redux state\n- Follow a single consistent pattern across the codebase\n\n---\n\n### Redux\n\n- Use only when global/shared state is required\n- Avoid unnecessary slices\n- Prefer simpler alternatives when possible\n\n---\n\n### RTK Query (`@reduxjs/toolkit/query`)\n\n- Use for API calls and caching\n- Use **granular tags** for cache invalidation\n- Avoid invalidating entire datasets unnecessarily\n- Implement **optimistic updates** where applicable\n\n---\n\n## Forms\n\nWe use:\n\n- `react-hook-form`\n- `zod v4` (for schema validation)\n\n### Rules\n\n- Always define a Zod schema\n- Include meaningful validation messages\n- Prefer **inline field errors** (not toast notifications)\n- Use `refine` / `superRefine` for complex validation\n- Store schemas in: `ui/lib/types/schemas.ts`\n\n---\n\n## Tables\n\n- Use `@tanstack/react-table` **only for large/complex datasets**\n- For simple tables → build custom lightweight components\n- Prioritize performance over abstraction\n\n---\n\n## ⚡ Performance Guidelines\n\n- Lazy load heavy or rarely-used libraries\n- Avoid unnecessary re-renders\n- Split large components into smaller ones\n- Keep bundle size minimal\n\n---\n\n## Dependency Rules\n\n- Do NOT add new dependencies unless absolutely necessary\n- Always pin exact versions (no `^` or `~`)\n- Prefer existing libraries in the codebase\n\n---\n\n## TypeScript Guidelines\n\n- Avoid using `any` unless absolutely unavoidable\n- Prefer strict typing and inference\n- Define reusable types in shared locations\n\n---\n\n## Code Quality & Formatting\n\nAfter writing code:\n\n```bash\ncd ui && npm run format\n````\n\nThen verify build:\n\n```bash\ncd ui && npm run build\n```\n\n* Code must pass formatting and build checks\n* Follow consistent naming and structure conventions\n\n---\n\n## Anti-Patterns to Avoid\n\n* Duplicate components without considering reuse\n* Mixing multiple state management approaches unnecessarily\n* Overusing Redux\n* Using unstable hook dependencies\n* Adding heavy libraries for simple use cases\n* Poorly structured or deeply nested JSX\n\n---\n\n## Summary\n\n* Prioritize **reusability, performance, and consistency**\n* Follow **strict folder structure and routing conventions**\n* Use **the right tool for the right problem**\n* Keep code **simple, predictable, and maintainable**\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"# AGENTS.md — Bifrost AI Gateway\n\n> Context for AI agents (Claude Code, Copilot, Cursor, etc.) working on this codebase. Read this fully before making changes.\n\n## What is Bifrost?\n\nBifrost is a high-performance AI gateway that unifies 20+ LLM providers behind a single OpenAI-compatible API with ~11µs overhead at 5,000 RPS. It also serves as an MCP (Model Context Protocol) gateway, turning static chat models into tool-calling agents.\n\nGitHub: `maximhq/bifrost`\n\n---\n\n## Repository Layout\n\n```\nbifrost/\n├── core/ # Go core library — the engine\n│ ├── bifrost.go # Main struct, request queuing, provider lifecycle (~3.4K lines)\n│ ├── inference.go # Inference routing, fallbacks, streaming dispatch (~1.9K lines)\n│ ├── mcp.go # MCP integration entry point\n│ ├── schemas/ # ALL shared Go types — 41 files\n│ │ ├── bifrost.go # BifrostConfig, ModelProvider enum, RequestType enum, context keys\n│ │ ├── provider.go # Provider interface (30+ methods), NetworkConfig, ProviderConfig\n│ │ ├── plugin.go # LLMPlugin, MCPPlugin, HTTPTransportPlugin, ObservabilityPlugin\n│ │ ├── context.go # BifrostContext (custom context.Context with mutable values)\n│ │ ├── chatcompletions.go # Chat completion request/response types\n│ │ ├── responses.go # OpenAI Responses API types\n│ │ ├── embedding.go # Embedding types\n│ │ ├── images.go # Image generation types\n│ │ ├── batch.go # Batch operation types\n│ │ ├── files.go # File management types\n│ │ ├── mcp.go # MCP types\n│ │ ├── trace.go # Tracer interface\n│ │ └── logger.go # Logger interface\n│ ├── providers/ # 20+ provider implementations\n│ │ ├── openai/ # Reference implementation (largest, most complete)\n│ │ ├── anthropic/ # Non-OpenAI-compatible example\n│ │ ├── bedrock/ # AWS event-stream protocol\n│ │ ├── gemini/ # Google-specific API shape\n│ │ ├── groq/ # OpenAI-compatible (minimal, delegates to openai/)\n│ │ └── utils/ # Shared: HTTP client, SSE parsing, error handling, scanner pool\n│ ├── pool/ # Generic Pool[T] — dual-mode (prod: sync.Pool, debug: full tracking)\n│ │ ├── pool_prod.go # Zero-overhead sync.Pool wrapper (default build)\n│ │ └── pool_debug.go # Double-release/use-after-release/leak detection (-tags pooldebug)\n│ ├── mcp/ # MCP protocol implementation\n│ │ ├── agent.go # Agent orchestration loop (multi-turn tool calling)\n│ │ ├── clientmanager.go # MCP client lifecycle management\n│ │ ├── toolmanager.go # Tool registration, discovery, filtering\n│ │ ├── healthmonitor.go # Client health monitoring\n│ │ └── codemode/starlark/ # Starlark sandbox for code-mode execution\n│ └── internal/\n│ ├── llmtests/ # LLM integration test infra (48 files, scenario-based)\n│ └── mcptests/ # MCP/Agent test infra (40+ files, mock-based)\n│\n├── framework/ # Data persistence, streaming, ecosystem utilities\n│ ├── configstore/ # Config storage backends (file, postgres)\n│ ├── logstore/ # Log storage backends (file, postgres)\n│ ├── vectorstore/ # Vector storage (Weaviate, Qdrant, Redis, Pinecone)\n│ ├── streaming/ # Streaming accumulator, delta copying, response marshaling\n│ │ ├── accumulator.go # Chunk accumulation into full response (~24KB)\n│ │ ├── chat.go # Chat stream handling (~17KB)\n│ │ └── responses.go # Response stream marshaling (~35KB)\n│ ├── modelcatalog/ # Model metadata registry\n│ ├── tracing/ # Distributed tracing helpers\n│ └── encrypt/ # Encryption utilities\n│\n├── transports/\n│ ├── config.schema.json # JSON Schema — THE source of truth for config.json (~2700 lines)\n│ └── bifrost-http/ # HTTP gateway transport\n│ ├── server/ # Server lifecycle, route registration\n│ ├── handlers/ # 27 HTTP endpoint handlers\n│ │ ├── inference.go # Chat/text completions, responses API (~109KB)\n│ │ ├── mcpinference.go # MCP tool execution\n│ │ ├── governance.go # Virtual keys, teams, customers, budgets (~100KB)\n│ │ ├── providers.go # Provider CRUD, key management\n│ │ ├── mcp.go # MCP client registry management\n│ │ ├── logging.go # Log queries, stats, histograms\n│ │ ├── config.go # System configuration\n│ │ ├── plugins.go # Plugin CRUD\n│ │ ├── cache.go # Cache management\n│ │ ├── session.go # Auth/session management\n│ │ ├── health.go # Health checks\n│ │ ├── mcpserver.go # MCP server (SSE/streamable HTTP)\n│ │ ├── websocket.go # WebSocket handler\n│ │ ├── devpprof.go # Pool debug profiler endpoint (~23KB)\n│ │ └── middlewares.go # Middleware definitions\n│ ├── lib/ # ChainMiddlewares, config, context conversion\n│ └── integrations/ # SDK compatibility layers\n│ ├── openai.go # OpenAI SDK drop-in compatibility\n│ ├── anthropic.go # Anthropic SDK compatibility\n│ ├── bedrock.go # AWS Bedrock SDK compatibility\n│ ├── genai.go # Google GenAI SDK compatibility\n│ ├── langchain.go # LangChain compatibility\n│ ├── litellm.go # LiteLLM compatibility\n│ └── pydanticai.go # PydanticAI compatibility\n│\n├── plugins/ # Go plugins — each has own go.mod\n│ ├── governance/ # Budget, rate limiting, virtual keys, routing, RBAC\n│ ├── telemetry/ # Prometheus metrics, push gateway\n│ ├── logging/ # Request/response audit logging\n│ ├── semanticcache/ # Semantic response caching via vector store\n│ ├── otel/ # OpenTelemetry tracing\n│ ├── mocker/ # Mock responses for testing\n│ ├── jsonparser/ # JSON extraction utilities\n│ ├── maxim/ # Maxim observability\n│ └── compat/ # LiteLLM SDK compatibility (HTTP transport)\n│\n├── ui/ # React + vite web interface\n│ ├── app/workspace/ # Feature pages (20+ workspace sections)\n│ ├── components/ # Shared React components\n│ └── lib/ # Constants, utilities, types\n│\n├── tests/e2e/ # Playwright E2E tests\n│ ├── core/ # Fixtures, page objects, helpers, API actions\n│ └── features/ # Per-feature test suites\n│\n├── docs/ # Mintlify MDX documentation\n│ ├── docs.json # Navigation config\n│ ├── media/ # Screenshots (ui-*.png naming convention)\n│ └── (architecture|features|providers|mcp|plugins|enterprise|...)\n│\n├── .claude/skills/ # Claude Code skill definitions (4 skills)\n├── go.work # Go workspace — requires Go 1.26.1\n├── Makefile # Build, test, dev commands (1300+ lines)\n└── terraform/ # Infrastructure as Code\n```\n\n---\n\n## Go Workspace\n\nBifrost is a **multi-module Go workspace**. Each module has its own `go.mod`:\n\n```\ngo.work\n├── core/go.mod # github.com/maximhq/bifrost/core\n├── framework/go.mod # github.com/maximhq/bifrost/framework\n├── transports/go.mod # github.com/maximhq/bifrost/transports\n└── plugins/*/go.mod # 9 plugin modules (governance, telemetry, logging, etc.)\n```\n\n**Rules:**\n- Run `go mod tidy` in the **specific module directory**, not the root\n- Cross-module imports resolve via workspace locally, but need explicit `require` in `go.mod` for releases\n- The workspace requires **Go 1.26.1** (`go.work` directive)\n\n---\n\n## Build, Test & Dev Commands\n\n```bash\n# Development\nmake dev # Full local dev (UI + API with hot reload via air)\nmake build # Build bifrost-http binary\n\n# Core tests (provider integration tests — hit live APIs)\nmake test-core # All providers\nmake test-core PROVIDER=openai # Specific provider\nmake test-core PROVIDER=openai TESTCASE=TestSimpleChat # Specific test\nmake test-core PATTERN=TestStreaming # Tests matching pattern\nmake test-core DEBUG=1 # With Delve debugger on :2345\n\n# MCP/Agent tests (mock-based, no live APIs)\nmake test-mcp # All MCP tests\nmake test-mcp TESTCASE=TestAgentLoop # Specific test\nmake test-mcp TYPE=agent # By category (agent|tool|connection|codemode)\n\n# Framework tests (require local backing services — bring them up FIRST)\ndocker compose -f tests/docker-compose.yml up -d # postgres, weaviate, qdrant, pinecone, and the 4 redis variants\nmake test-framework # All framework packages\n\n# Plugin tests\nmake test-plugins # All plugins\nmake test-governance # Governance plugin specifically\n\n# Integration tests (SDK compatibility)\nmake test-integrations-py # Python SDK tests\nmake test-integrations-ts # TypeScript SDK tests\n\n# E2E tests (Playwright, requires running dev server)\nmake run-e2e # All E2E tests\nmake run-e2e FLOW=providers # Specific feature\n\n# Code quality\nmake lint # Linting\nmake fmt # Format code\n```\n\n---\n\n## Architecture\n\n### Request Flow\n\n```\nClient HTTP Request\n → FastHTTP Transport (parsing, validation ~2µs)\n → SDK Integration Layer (OpenAI/Anthropic/Bedrock format → Bifrost format)\n → Middleware Chain (lib.ChainMiddlewares, applied per-route)\n → HTTPTransportPreHook (HTTP-level plugins, can short-circuit)\n → PreLLMHook Pipeline (auth, rate-limit, cache check — registration order)\n → MCP Tool Discovery & Injection (if tool_choice present)\n → Provider Queue (channel-based, per-provider isolation)\n → Worker picks up request\n → Key Selection (~10ns weighted random)\n → Provider API Call (fasthttp client, connection pooling)\n → Response / SSE Stream\n → PostLLMHook Pipeline (reverse order of PreLLMHooks)\n → Tool Execution Loop (if tool_calls in response, MCP agent loop)\n → HTTPTransportPostHook (reverse order)\n → Response Serialization\n → HTTP Response to Client\n```\n\n### Design Principles\n\n- **Provider isolation**: Each provider has its own worker pool and queue. One provider going down doesn't cascade to others.\n- **Channel-based async**: Request routing uses Go channels (`chan *ChannelMessage`), not mutexes. The `ProviderQueue` struct manages channel lifecycle with atomic flags.\n- **Object pooling everywhere**: `sync.Pool` wrappers reduce GC pressure. Pools exist for: channel messages, response channels, error channels, stream channels, plugin pipelines, MCP requests, HTTP request/response objects, scanner buffers.\n- **Plugin pipeline symmetry**: Pre-hooks execute in registration order, post-hooks in **reverse** order (LIFO). For every pre-hook executed, the corresponding post-hook is guaranteed to run.\n- **Streaming**: SSE chunks flow through `chan chan *schemas.BifrostStreamChunk`. Accumulated into full response for post-hooks via `framework/streaming/accumulator.go`.\n\n### BifrostContext — Custom Context\n\n`BifrostContext` (`core/schemas/context.go`) is a custom `context.Context` with **thread-safe mutable values**. Unlike standard Go contexts, values can be set after creation:\n\n```go\nctx := schemas.NewBifrostContext(parent, deadline)\nctx.SetValue(key, value) // Thread-safe, uses RWMutex\nctx.WithValue(key, value) // Chainable variant\n```\n\n**Reserved context keys** (set by Bifrost internals — DO NOT set manually):\n- `BifrostContextKeySelectedKeyID/Name` — Set by governance plugin\n- `BifrostContextKeyGovernance*` — Set by governance plugin\n- `BifrostContextKeyNumberOfRetries`, `BifrostContextKeyFallbackIndex` — Set by retry/fallback logic\n- `BifrostContextKeyStreamEndIndicator` — Set by streaming infrastructure\n- `BifrostContextKeyTrace*`, `BifrostContextKeySpan*` — Set by tracing middleware\n\n**User-settable keys** (plugins and handlers can set these):\n- `BifrostContextKeyVirtualKey` (`x-bf-vk`) — Virtual key for governance\n- `BifrostContextKeyAPIKeyName` (`x-bf-api-key`) — Explicit key selection by name\n- `BifrostContextKeyAPIKeyID` (`x-bf-api-key-id`) — Explicit key selection by ID (takes priority over name)\n- `BifrostContextKeyRequestID` — Request ID\n- `BifrostContextKeyExtraHeaders` — Extra headers to forward to provider\n- `BifrostContextKeyURLPath` — Custom URL path for provider\n- `BifrostContextKeySkipKeySelection` — Skip key selection (pass empty key)\n- `BifrostContextKeyUseRawRequestBody` — Send raw body directly to provider\n\n**Gotcha**: `BlockRestrictedWrites()` silently drops writes to reserved keys. This prevents plugins from accidentally overwriting internal state.\n\n**Hard rule — never store stream-sized data in `BifrostContext`.** Context holds small handles only: IDs, durations, booleans, interface pointers. Any per-request state that scales with stream content (chunk buffers, accumulated payloads, replay queues, large per-request slices/maps) must live in a top-level manager keyed by `RequestID`, not in `ctx`. Reference implementations:\n\n- `framework/streaming.Accumulator` — owns a `sync.Map` of per-stream `StreamAccumulator` entries keyed by `RequestID`. Only `BifrostContextKeyAccumulatorID` (the ID string) is stored on the context; the chunk buffers live in the manager. The pause/resume gate (`gate.go`) extends the same per-stream entry with a state machine — again, **no buffer in ctx**.\n- The `Tracer` interface (in ctx as a small pointer) is the access path for plugins/providers to reach managers without putting bulky data on the context itself.\n\nWhen in doubt: if your new ctx key would hold a slice/map that grows with request content, route the storage through a manager and keep only the ID in ctx.\n\n---\n\n## Core Patterns\n\n### Provider Implementation\n\nThere are **two categories** of providers:\n\n**Category 1: Non-OpenAI-compatible** (Anthropic, Bedrock, Gemini, Cohere, HuggingFace, Replicate, ElevenLabs):\n```\ncore/providers//\n├── .go # Controller: constructor, interface methods, HTTP orchestration\n├── _test.go # Tests\n├── types.go # ALL provider-specific structs (PascalCase prefixed with provider name)\n├── utils.go # Constants, base URLs, helpers (camelCase for unexported)\n├── errors.go # Error parsing: provider HTTP error → *schemas.BifrostError\n├── chat.go # Chat request/response converters\n├── embedding.go # Embedding converters (if supported)\n├── images.go # Image generation (if supported)\n├── speech.go # TTS/STT (if supported)\n└── responses.go # Responses API + streaming converters\n```\n\n**Category 2: OpenAI-compatible** (Groq, Cerebras, Ollama, Perplexity, OpenRouter, Parasail, Nebius, xAI, SGL):\n```\ncore/providers//\n├── .go # Minimal — constructor + delegates to openai.HandleOpenAI* functions\n└── _test.go # Tests\n```\n\n**Converter function naming convention:**\n- `ToRequest()` — Bifrost schema → Provider API format\n- `ToBifrostResponse()` — Provider API format → Bifrost schema\n- These must be **pure transformation functions** — no HTTP calls, no logging, no side effects\n\n**Provider constructor pattern:**\n```go\nfunc NewProvider(config schemas.ProviderConfig) (*Provider, error) {\n // Validate config, set up fasthttp.Client with connection pooling\n client := &fasthttp.Client{\n MaxConnsPerHost: config.NetworkConfig.MaxConnsPerHost, // configurable, default 5000\n MaxIdleConnDuration: 30 * time.Second,\n }\n // After ConfigureProxy/ConfigureDialer/ConfigureTLS, build a sibling client\n // for streaming. BuildStreamingClient zeros ReadTimeout/WriteTimeout/MaxConnDuration\n // so streams aren't killed by fasthttp's whole-response deadline; per-chunk idle\n // is enforced at the app layer via NewIdleTimeoutReader.\n streamingClient := providerUtils.BuildStreamingClient(client)\n return &Provider{client: client, streamingClient: streamingClient, ...}, nil\n}\n```\n\n**Streaming vs unary client:** Every provider holds two clients — `client` for unary requests (`ReadTimeout=30s` bounds the whole response) and `streamingClient` for SSE / EventStream / chunked paths (`ReadTimeout=0`; the per-chunk `NewIdleTimeoutReader` is the only governor). Pass `provider.streamingClient` to every `Handle*Streaming` / `Handle*StreamRequest` helper and to direct `Do` calls inside `*Stream` methods. For new providers, apply the same pattern — missing the switch means streams get killed at 30s.\n\n**Note:** Bedrock uses `net/http` (not fasthttp) with HTTP/2 support. Its `http.Transport` is configured with `ForceAttemptHTTP2: true` and `MaxConnsPerHost` from `NetworkConfig` to allow multiple HTTP/2 connections when the server's per-connection stream limit (100 for AWS Bedrock) is reached. Use `providerUtils.BuildStreamingHTTPClient(client)` to derive the streaming variant — it shares the base `Transport` (safe for concurrent reuse) but clears `Client.Timeout`.\n\n### The Provider Interface\n\n`core/schemas/provider.go` defines the `Provider` interface with **30+ methods**. Every provider must implement all of them (returning \"not supported\" for unsupported operations). The interface covers:\n\n- `ListModels`, `ChatCompletion`, `ChatCompletionStream`\n- `Responses`, `ResponsesStream` (OpenAI Responses API)\n- `TextCompletion`, `TextCompletionStream`\n- `Embedding`, `Speech`, `SpeechStream`, `Transcription`, `TranscriptionStream`\n- `ImageGeneration`, `ImageGenerationStream`, `ImageEdit`, `ImageEditStream`, `ImageVariation`\n- `CountTokens`\n- `Batch*` (Create, List, Retrieve, Cancel, Results)\n- `File*` (Upload, List, Retrieve, Delete, Content)\n- `Container*` and `ContainerFile*` (Create, List, Retrieve, Delete, Content)\n\n**Streaming methods** receive a `PostHookRunner` callback and return `chan *BifrostStreamChunk`:\n```go\nChatCompletionStream(ctx *BifrostContext, postHookRunner PostHookRunner, key Key, request *BifrostChatRequest) (chan *BifrostStreamChunk, *BifrostError)\n```\n\n### Error Handling\n\nEach provider has `errors.go` with an `ErrorConverter` function:\n```go\ntype ErrorConverter func(resp *fasthttp.Response, requestType schemas.RequestType, providerName schemas.ModelProvider, model string) *schemas.BifrostError\n```\n\nThe shared utility `providerUtils.HandleProviderAPIError()` handles common HTTP error parsing. Provider-specific parsers add extra field mapping. Errors always carry metadata:\n```go\nbifrostErr.ExtraFields.Provider = providerName\nbifrostErr.ExtraFields.ModelRequested = model\nbifrostErr.ExtraFields.RequestType = requestType\n```\n\n### Plugin System\n\nFour plugin interfaces exist:\n\n| Interface | Hook Methods | When Called |\n|-----------|-------------|------------|\n| `LLMPlugin` | `PreLLMHook`, `PostLLMHook` | Every LLM request (SDK + HTTP) |\n| `MCPPlugin` | `PreMCPHook`, `PostMCPHook` | Every MCP tool execution |\n| `HTTPTransportPlugin` | `HTTPTransportPreHook`, `HTTPTransportPostHook`, `HTTPTransportStreamChunkHook` | HTTP gateway only (not Go SDK) |\n| `ObservabilityPlugin` | `Inject(ctx, trace)` | Async, after response written to wire |\n\n**Key plugin behaviors:**\n- Plugin errors are **logged as warnings**, never returned to the caller\n- Pre-hooks can **short-circuit** by returning `*LLMPluginShortCircuit` (cache hit, auth failure, rate limit)\n- Post-hooks receive both response and error — either can be nil. Plugins can **recover from errors** (set error to nil, provide response) or **invalidate responses** (set response to nil, provide error)\n- `BifrostError.AllowFallbacks` controls whether fallback providers are tried: `nil` or `&true` = allow, `&false` = block\n- `HTTPTransportStreamChunkHook` is called **per-chunk** during streaming — can modify, skip, or abort the stream\n\n### Pool System\n\n`core/pool/` provides `Pool[T]` with two build modes:\n\n```go\n// Production (default): zero-overhead sync.Pool wrapper\n// Debug (-tags pooldebug): tracks double-release, use-after-release, leaks with stack traces\np := pool.New[MyType](\"descriptive-name\", func() *MyType { return &MyType{} })\nobj := p.Get()\n// ... use obj ...\n// MUST reset ALL fields before Put — pool does not auto-reset\np.Put(obj)\n```\n\n**Acquire/Release pattern** for types with complex reset logic (used in `schemas/plugin.go`):\n```go\nreq := schemas.AcquireHTTPRequest() // Get from pool, pre-allocated maps\ndefer schemas.ReleaseHTTPRequest(req) // Clears all maps and fields, returns to pool\n```\n\n### HTTP Transport Layer\n\n**Handler pattern:** Handlers are structs with injected dependencies:\n```go\ntype CompletionHandler struct {\n client *bifrost.Bifrost\n handlerStore lib.HandlerStore\n config *lib.Config\n}\n```\n\n**Route registration:** Each handler implements `RegisterRoutes(router, middlewares...)` — routes get middleware chains applied per-route via `lib.ChainMiddlewares()`.\n\n**SDK integration layers** (`transports/bifrost-http/integrations/`) provide request/response converters between provider-native SDK formats and Bifrost's internal format. This enables drop-in replacement of OpenAI SDK, Anthropic SDK, AWS Bedrock SDK, Google GenAI SDK, LangChain, and LiteLLM.\n\n---\n\n## Gotchas\n\n### 1. Always Reset Pooled Objects Before Put\n\nEvery pooled object must have **all** fields zeroed before `pool.Put()`. Stale data leaks between requests. The debug build catches double-release and use-after-release but **not** missing resets.\n\n```go\n// WRONG — stale data from previous request leaks to next user\npool.Put(msg)\n\n// RIGHT\nmsg.Response = nil\nmsg.Error = nil\nmsg.Context = nil\nmsg.ResponseStream = nil\npool.Put(msg)\n```\n\n### 2. Channel Lifecycle — ProviderQueue Pattern\n\n`ProviderQueue` uses atomic flags and `sync.Once` to prevent \"send on closed channel\" panics:\n```go\ntype ProviderQueue struct {\n queue chan *ChannelMessage\n done chan struct{}\n closing uint32 // atomic: 0=open, 1=closing\n signalOnce sync.Once // ensure signal fires only once\n closeOnce sync.Once // ensure close fires only once\n}\n```\nAlways check the atomic closing flag before sending. Never close a channel without this pattern.\n\n### 3. NetworkConfig Duration Serialization\n\n`RetryBackoffInitial` and `RetryBackoffMax` are `time.Duration` (nanoseconds) in Go but **milliseconds** (integers) in JSON. Custom `MarshalJSON`/`UnmarshalJSON` handles conversion. If adding new duration fields to any config struct, follow this pattern exactly.\n\n### 4. ExtraHeaders — Defensive Map Copy\n\n`NetworkConfig.ExtraHeaders` is deep-copied in `CheckAndSetDefaults()` to prevent data races between concurrent requests. Apply the same `maps.Copy()` pattern to any new map fields in config structs.\n\n### 5. Provider Interface Has 30+ Methods\n\nAdding a new operation type requires changes across the entire codebase:\n1. Add method to `Provider` interface in `core/schemas/provider.go`\n2. Implement in **all** 20+ providers (most return \"not supported\")\n3. Add `RequestType` constant in `core/schemas/bifrost.go`\n4. Add to `AllowedRequests` struct and `IsOperationAllowed()` switch\n5. Add handler endpoint in `transports/bifrost-http/handlers/`\n6. Wire up in `core/bifrost.go` and `core/inference.go`\n\n### 6. OpenAI Provider Changes Cascade to 9+ Providers\n\nGroq, Cerebras, Ollama, Perplexity, OpenRouter, Parasail, Nebius, xAI, and SGL all delegate to `openai.HandleOpenAI*` functions. **Any change to OpenAI converter logic affects all of them.** Always test broadly: `make test-core` (all providers).\n\n### 7. Scanner Buffer Pool Has a Capacity Cap\n\nThe SSE scanner buffer pool in `core/providers/utils/utils.go` starts at 4KB. Buffers grow dynamically but those exceeding **64KB are discarded** (not returned to pool) to prevent memory bloat. Be aware when working with providers that send very large SSE events.\n\n### 8. Plugin Execution Order is Meaningful\n\nPre-hooks: registration order (first registered → first to run). Post-hooks: **reverse** order. This creates \"wrapping\" semantics — the first plugin registered is the outermost wrapper (its pre-hook runs first, post-hook runs last). Changing registration order changes behavior.\n\n### 9. Fallbacks Re-execute the Full Plugin Pipeline\n\nWhen a provider fails and the request falls to a fallback, the **entire plugin pipeline** re-executes from scratch. Governance checks, caching, and logging all run again for each attempt. Intentional, but surprising when debugging request counts or cost tracking.\n\n### 10. `AllowedRequests` Nil Semantics\n\nA **nil** `*AllowedRequests` means \"all operations allowed.\" A **non-nil** value only allows fields explicitly set to `true`. This applies to both `ProviderConfig.AllowedRequests` and `CustomProviderConfig.AllowedRequests`.\n\n### 11. BifrostContext Reserved Keys Are Silently Dropped\n\nWhen `BlockRestrictedWrites()` is active, writes to reserved keys (governance IDs, retry counts, fallback index, etc.) are **silently ignored** — no error. If your plugin needs to pass data through context, use your own custom key type.\n\n### 12. `fasthttp`, Not `net/http`\n\nBifrost uses `github.com/valyala/fasthttp` for provider HTTP calls. The API is different from `net/http`:\n- Use `fasthttp.AcquireRequest()`/`fasthttp.ReleaseRequest()` for lifecycle\n- `fasthttp.Client` pools connections per-host (`NetworkConfig.MaxConnsPerHost`, default 5000, 30s idle)\n- Request/response bodies accessed via `resp.Body()` (returns `[]byte`, not `io.Reader`)\n- **Exception:** Bedrock uses `net/http` (for AWS SigV4 signing) with `http.Transport` configured for HTTP/2 multi-connection support\n\n### 13. `sonic`, Not `encoding/json`\n\nJSON marshaling in hot paths uses `github.com/bytedance/sonic` for performance. `core/schemas/` uses standard `encoding/json` for custom marshaling (e.g., `NetworkConfig`). Don't mix them accidentally.\n\nFor reading or writing a **single field** (or a handful) inside a larger raw JSON payload, prefer `github.com/tidwall/gjson`/`github.com/tidwall/sjson` over decoding into `map[string]interface{}` and re-encoding — the shared helpers `providerUtils.GetJSONField`/`SetRawJSONField`/`DeleteJSONField`/`JSONFieldExists`/`GetJSONSubtree` (`core/providers/utils/utils.go`) wrap these and should be reused where the path is a lookup on an already-in-scope `[]byte`/`json.RawMessage`. Full-document decode into a map/struct is still correct when you need the whole shape (e.g. re-marshaling an entire object to normalize it) — the point is not to round-trip an entire object through a `map[string]interface{}` just to inspect one key. When marshaling back out, always use `providerUtils.MarshalSorted` (never a raw `sonic.Marshal`/`json.Marshal`), since unsorted map keys reorder nondeterministically and break prompt-cache-relevant byte stability.\n\n### 14. Atomic Pointer for Hot Config Reload\n\n`Bifrost` uses `atomic.Pointer` for providers and plugins lists. On updates: create new slice → atomically swap pointer. **Never mutate the slice in place** — concurrent readers would see partial state.\n\n### 15. MCP Tool Filtering is 4 Levels Deep\n\nTool access follows: Global filter → Client-level filter → Tool-level filter → Per-request filter (HTTP headers). All four levels must agree for a tool to be available. Changes to filtering logic must respect this hierarchy.\n\n### 16. `config.schema.json` is the Source of Truth\n\n`transports/config.schema.json` (~2700 lines) is the authoritative definition for all `config.json` fields. Documentation examples must match. When adding config fields: update schema first → handlers → docs.\n\n### 17. UI `data-testid` Attributes Are Load-Bearing\n\nE2E tests depend on `data-testid` attributes. Convention: `data-testid=\"--\"`. If you rename or remove one, search `tests/e2e/` for references. If you add new interactive elements, add `data-testid`.\n\n### 18. E2E Tests — Never Marshal Payloads to Maps\n\nIn `tests/e2e/core/`, **never marshal API payloads to a `Record`/`Map`/plain-object and then re-serialize**. Field ordering matters for backend validation and snapshot comparisons. Construct payloads as object literals with fields in the intended order and pass directly to Playwright's `request.post({ data })`. Avoid `Object.fromEntries()`, `JSON.parse(JSON.stringify(...))` round-trips, or destructuring into an intermediate `Record` — these can silently reorder fields.\n\n### 19. Framework Tests Need `tests/docker-compose.yml`, Not `framework/docker-compose.yml`\n\n`make test-framework` fails ~30 tests in `framework/vectorstore` with no services running. Bring the stack up first:\n\n```bash\ndocker compose -f tests/docker-compose.yml up -d\n```\n\nTwo compose files define overlapping services on the **same host ports** (9000, 6379, 6334, 5081), so only one can run at a time. Use the `tests/` one:\n\n| | `tests/docker-compose.yml` | `framework/docker-compose.yml` |\n|---|---|---|\n| Redis | plain 6379, **TLS 6380, cluster 7000, cluster-TLS 7100** | plain 6379 only |\n| TLS certs | `redis-certs-init` writes `tests/redis-certs/` | none |\n| Weaviate | 1.32.4, pins `CLUSTER_ADVERTISE_ADDR` | 1.25.0, no advertise addr |\n\nThe differences are load-bearing, not cosmetic:\n\n- `redis_test.go` dials **6380** and **7100** for the TLS and TLS-cluster client tests, and `readTestCACert` reads `tests/redis-certs/ca.crt`. The `framework/` file provides neither, so 5 tests fail against it.\n- Weaviate's memberlist aborts startup with `Failed to get final advertise address: No private IP address found` unless `CLUSTER_ADVERTISE_ADDR` is set ([weaviate#7474](https://github.com/weaviate/weaviate/issues/7474)). The `tests/` file pins a static IP; the `framework/` file does not, so its Weaviate crash-loops and 4 more tests fail.\n\nNote that `qdrant` and `pinecone` report `(unhealthy)` in `docker compose ps` under the `framework/` file because those images have no `wget` for the healthcheck. The services themselves are fine, so ignore that specific signal and probe the port instead.\n\nOnly `framework/vectorstore` needs any of this. Every other framework package passes with nothing running.\n\n---\n\n## Adding a New Provider — Full Checklist\n\n1. Create `core/providers//` with files per the pattern (see \"Provider Implementation\" above)\n2. Add `ModelProvider` constant in `core/schemas/bifrost.go`\n3. Add to `StandardProviders` list in `core/schemas/bifrost.go`\n4. Register in `core/bifrost.go` — add import + case in provider init switch\n5. **UI integration** (all required):\n - `ui/lib/constants/config.ts` — model placeholder + key requirement\n - `ui/lib/constants/icons.tsx` — provider icon\n - `ui/lib/constants/logs.ts` — provider display name (2 places)\n - `docs/openapi/openapi.json` — OpenAPI spec update\n - `transports/config.schema.json` — config schema (2 locations)\n6. **CI/CD**: Add env vars to `.github/workflows/pr-tests.yml` and `release-pipeline.yml` (4 jobs)\n7. **Docs**: Create `docs/providers/supported-providers/.mdx`\n8. **Test**: `make test-core PROVIDER=`\n\n---\n\n## Testing\n\n### Bug fixes: red before green\n\nBefore writing a fix, add (or extend) a test that reproduces the bug and confirm it fails for the expected reason — a wrong assertion, not a compile error or an unrelated panic. Only then implement the fix, and confirm the same test now passes. For bugs reachable through `make run-provider-harness-test`, add the harness regression case (see `.claude/skills/harness-test-writer/SKILL.md`) alongside Go-level tests: Go tests give a fast, free red/green loop while coding; the harness case is the live end-to-end pin, expected red pre-fix and green post-fix, validated structurally (`augment-provider-harness.mjs` / `filter-collection.mjs`) without needing a live paid run during development.\n\n### Every `core/` change ships with a provider-harness case\n\nAny change under `core/` that a client can observe on the wire must land together with a case in `tests/e2e/api/collections/provider-harness.json` (see `.claude/skills/harness-test-writer/SKILL.md`). This covers new features and refactors, not only bug fixes — the rule in the previous section is the narrower instance of this one.\n\n`core/` is the only layer every transport, integration and provider funnels through, so its behaviour is what the harness exists to pin. A Go unit test proves the function does what you meant; only the harness proves the bytes a real client sends still come back correct through the whole stack. The gap between those two is where regressions live: a fail-soft that fires on one request shape and silently skips a sibling shape passes every unit test it has.\n\nWrite the case so it is **red before the change and green after**, and validate it structurally while developing — no live paid run needed:\n\n```bash\nnode tests/e2e/api/runners/augment-provider-harness.mjs --source tests/e2e/api/collections/provider-harness.json --out tmp/harness-augmented.json\nnode tests/e2e/api/runners/filter-collection.mjs --source tmp/harness-augmented.json --out tmp/filtered.json --feature \"\"\n```\n\nInsert into the collection surgically (a script that splices the new object in, never a whole-file reserialize) — the file is ~50k lines and a reformat buries the actual change.\n\nThe narrow exemptions: changes with no wire-visible effect (comments, internal renames, log lines) and behaviour no HTTP request can reach. If a change is exempt, say so explicitly in the PR rather than leaving the omission unexplained.\n\n### Always prefer `make test-core` over raw `go test` for provider-level tests\n\nThe `make test-core` target is the canonical harness for provider tests — it wires up env vars from `.env` (provider API keys), invokes the per-provider `{provider}_test.go` entrypoint in `core/providers//`, and routes through the shared `core/internal/llmtests/` scenario suite that validates end-to-end behavior (including streaming).\n\nRunning bare `go test ./core/providers//...` only executes unit tests and skips the llmtests scenarios — so it won't catch regressions in streaming, tool-calling, or provider-specific response shapes.\n\n```bash\nmake test-core PROVIDER=anthropic TESTCASE=TestChatCompletionStream # exact test\nmake test-core PROVIDER=openai PATTERN=Stream # substring match\nmake test-core PROVIDER=bedrock # all scenarios for one provider\nmake test-core DEBUG=1 PROVIDER=gemini TESTCASE=TestResponsesStream # attach Delve on :2345\n```\n\n`PATTERN` and `TESTCASE` are mutually exclusive. Provider name must match a directory under `core/providers/` (e.g. `anthropic`, `openai`, `bedrock`, `vertex`, `azure`, `gemini`, `cohere`, `mistral`, `groq`, etc.).\n\n### LLM Tests (`core/internal/llmtests/`)\n\nScenario-based tests that run against **live provider APIs** with dual-API testing (Chat Completions + Responses API):\n\n```go\nfunc RunMyScenarioTest(t *testing.T, client *bifrost.Bifrost, ctx context.Context, cfg ComprehensiveTestConfig) {\n // Use validation presets: BasicChatExpectations(), ToolCallExpectations(), etc.\n // Use retry framework for flaky assertions\n}\n```\n\n- Register in `tests.go` `testScenarios` slice\n- Add `Scenarios.MyScenario` flag to `ComprehensiveTestConfig`\n- Run: `make test-core PROVIDER= TESTCASE=`\n\n### MCP Tests (`core/internal/mcptests/`)\n\nMock-based tests with `DynamicLLMMocker` and declarative setup:\n\n```go\nmanager, mocker, ctx := SetupAgentTest(t, AgentTestConfig{\n InProcessTools: []string{\"echo\", \"calculator\"},\n AutoExecuteTools: []string{\"*\"},\n MaxDepth: 5,\n})\n// Queue mock LLM responses, assert tool execution order\n```\n\nCategories: `agent_*_test.go`, `tool_*_test.go`, `connection_*_test.go`, `codemode_*_test.go`\n\nRun: `make test-mcp TESTCASE=`\n\n### E2E Tests (`tests/e2e/`)\n\nPlaywright tests with page objects, data factories, fixtures:\n\n- Page objects extend `BasePage`, use `getByTestId()` as primary selector strategy\n- Data factories use `Date.now()` for unique names (prevents collision in parallel runs)\n- Track created resources in arrays, clean up in `afterEach`\n- Import `test`/`expect` from `../../core/fixtures/base.fixture` (never from `@playwright/test`)\n- **Never marshal API payloads to a `Record`/`Map`/plain-object and then re-serialize.** Field ordering matters for snapshot comparisons and some backend validations. Construct payloads as object literals with fields in the intended order and pass directly to Playwright's `request.post({ data })`. Do NOT destructure into an intermediate `Record` or use `Object.fromEntries()` / `JSON.parse(JSON.stringify(...))` round-trips, as these can reorder fields.\n\nRun: `make run-e2e FLOW=`\n\n---\n\n## Claude Code Skills\n\nFour skills are available via `/skill-name`:\n\n### `/docs-writer `\nWrite, update, or review Mintlify MDX documentation. Researches UI code, Go handlers, and config schema. Validates `config.json` examples against `transports/config.schema.json`. Outputs docs with Web UI / API / config.json tabs.\n\nVariants: `/docs-writer update `, `/docs-writer review `\n\n### `/e2e-test `\nCreate, run, debug, audit, or auto-update Playwright E2E tests.\n\nVariants:\n- `/e2e-test fix ` — Debug and fix a failing test\n- `/e2e-test sync` — Detect UI changes, update affected tests automatically\n- `/e2e-test audit` — Scan specs for incorrect/weak assertions (P0-P6 severity scale)\n\n### `/investigate-issue `\nInvestigate a GitHub issue from `maximhq/bifrost`. Fetches issue details, classifies by type/area, searches codebase, traces dependencies, analyzes side effects, suggests tests (LLM/MCP/E2E), and presents an implementation plan with per-change approval gates.\n\n### `/resolve-pr-comments `\nSystematically address unresolved PR review comments. Uses GraphQL to get unresolved threads, presents each with FIX/REPLY/SKIP options, collects fixes locally, and only posts replies **after code is pushed** to remote.\n\n---\n\n## Common Workflows\n\n### Modify chat completions across all providers\n1. Change types in `core/schemas/chatcompletions.go`\n2. Update converter functions in each provider's `chat.go`\n3. If streaming affected, update `framework/streaming/` (accumulator, delta copy)\n4. Run `make test-core` (all providers)\n\n### Add a new field to API responses\n1. Add to schema type in `core/schemas/`\n2. Map in provider response converter (`ToBifrost*Response`)\n3. Handle in streaming accumulator if applicable\n4. Update HTTP handler if field needs special serialization\n5. Update `transports/config.schema.json` if configurable\n\n### Add a new plugin\n1. Create `plugins//` with its own `go.mod`\n2. Implement `LLMPlugin`, `MCPPlugin`, or `HTTPTransportPlugin` interface\n3. Add to `go.work`\n4. Register in transport layer or Bifrost config\n5. Add test targets to `Makefile`\n\n### Modify a UI feature\n1. Find workspace page: `ui/app/workspace//`\n2. Check existing `data-testid` attributes — E2E tests depend on them\n3. Add `data-testid` to new interactive elements\n4. Run `make run-e2e FLOW=` to verify\n5. If E2E tests break, use `/e2e-test sync` to update them\n\n---\n\n## Key Files Quick Reference\n\n| What | Where |\n|------|-------|\n| Main Bifrost struct & queuing | `core/bifrost.go` |\n| Inference routing & fallbacks | `core/inference.go` |\n| Provider interface (30+ methods) | `core/schemas/provider.go` |\n| ModelProvider enum & context keys | `core/schemas/bifrost.go` |\n| Plugin interfaces & pooled HTTP types | `core/schemas/plugin.go` |\n| BifrostContext (mutable context) | `core/schemas/context.go` |\n| Chat completion types | `core/schemas/chatcompletions.go` |\n| Responses API types | `core/schemas/responses.go` |\n| Object pool (prod + debug) | `core/pool/pool_prod.go`, `pool_debug.go` |\n| Shared provider utils & SSE parsing | `core/providers/utils/utils.go` |\n| Streaming accumulator | `framework/streaming/accumulator.go` |\n| HTTP inference handler | `transports/bifrost-http/handlers/inference.go` |\n| Governance handler | `transports/bifrost-http/handlers/governance.go` |\n| Config schema (source of truth) | `transports/config.schema.json` |\n| Pool debug profiler | `transports/bifrost-http/handlers/devpprof.go` |\n| LLM test infrastructure | `core/internal/llmtests/` |\n| MCP test infrastructure | `core/internal/mcptests/` |\n| E2E test infrastructure | `tests/e2e/core/` |\n| Docs navigation config | `docs/docs.json` |\n| CI/CD workflows | `.github/workflows/` |\n\n---\n\n## Code Style\n\n- **Go**: `gofmt`/`goimports`. No custom linter config.\n- **TypeScript/React**: Oxfmt. TanStack Router.\n- **JSON tags**: `snake_case` matching provider API conventions.\n- **Error strings**: Lowercase, no trailing punctuation (Go convention).\n- **Provider types**: Prefixed with provider name in PascalCase (`AnthropicChatRequest`, `GeminiEmbeddingResponse`).\n- **Converter functions**: Pure — no side effects, no logging, no HTTP.\n- **Pool names**: Descriptive string passed to `pool.New()` (e.g., `\"channel-message\"`, `\"response-stream\"`).\n- **Context keys**: Use `BifrostContextKey` type. Custom plugins should define their own key types to avoid collisions.\n- **Go filenames**: No underscores. The only permitted underscore is the `_test.go` suffix. Examples: `pluginpipeline.go`, `pluginpipeline_test.go` — never `plugin_pipeline.go` or `plugin_pipeline_race_test.go`. Concatenate words (lowercase, no separators) for multi-word filenames.\n\n# Frontend Code Guidelines & Patterns\n\nThis document defines the standards, structure, and best practices for writing frontend code in this project.\n\n---\n\n## Tech Stack\n\n- **React** (with Vite)\n- **TypeScript**\n- **@tanstack/react-router** (type-safe routing)\n- **Tailwind CSS v4**\n- **Radix UI** (primitives)\n- **Local UI component library** (`ui/components/ui/`) built on Radix primitives\n\n---\n\n## Folder Structure\n\n```text\n\n/ui\n├── app # Routes & pages\n├── components # Shared components\n│ └── ui # Core design system components\n├── hooks # Custom React hooks\n├── lib # Utilities, helpers, shared logic\n└── app/enterprise # Enterprise-specific code (via symlink)\n\n```\n\n### Rules\n\n- All frontend code must live inside `/ui`\n- Routes and pages → `ui/app`\n- Shared/reusable components → `ui/components`\n- Core UI primitives → `ui/components/ui`\n- Utilities and libraries → `ui/lib`\n- Custom hooks → `ui/hooks`\n\n---\n\n## Libraries & Usage\n\n### Core Libraries\n\n- `react` → UI library\n- `typescript` → Type safety\n- `tailwindcss` → Styling\n- `@tanstack/react-router` → Routing\n\n### UI & Visualization\n\n- `@radix-ui/react-*` → UI primitives\n- `ui/components/ui/*` → Project's Radix-based component system\n- `recharts` → Charts\n- `monaco-editor` → Code editor\n\n### Utilities\n\n- `date-fns` → Date/time formatting\n- `nuqs` → Query param state management\n\n### Tooling\n\n- `Oxfmt` → Code formatting\n- `vitest` → Testing\n\n---\n\n## Routing Convention\n\nFor every new route:\n\n```text\n\nui/app//\n├── layout.tsx # Route definition using createFileRoute\n├── page.tsx # Page content\n└── views/ # Optional: route-specific components\n\n```\n\n### Rules\n\n- Folder name must match route name\n- Always use `createFileRoute` in `layout.tsx`\n- `page.tsx` should only handle composition (not heavy logic)\n- Route-specific components go inside `views/`\n\n---\n\n## Component Guidelines\n\n### Reusability First\n\n- Always check if similar components/functions already exist\n- Prefer extending or refactoring existing code over duplication\n- Only create new components if reuse is not feasible\n\n---\n\n### Component Placement\n\n- Shared → `ui/components`\n- Route-specific → `views/` inside route folder\n\n---\n\n### Entity Selectors — never hand-roll an entity picker\n\nAny UI that lets a user pick an existing entity (virtual key, team, customer, user, business unit, …) **must** go through `ui/components/entitySelectors/`. Do not build a new `Select`/`Combobox` + `useState` + debounce + fetch stack for this — that pattern was already duplicated across surfaces and consolidated here.\n\n**Use an existing selector** — import it and pass one of the three modes:\n\n```tsx\nimport { VirtualKeySelector } from \"@/components/entitySelectors/virtualKeySelector\";\n\n // single\n // multi (chips inside the control)\n appendRow(o)} /> // fire-and-forget add\n```\n\nAvailable today: `virtualKeySelector`, `teamSelector`, `customerSelector` (OSS); `userSelector`, `businessUnitSelector` (enterprise — reached via registry, see below).\n\n**Always pass `fallbackOption` / `fallbackOptions`** when editing an existing row. Selectors fetch nothing until the popover opens, so a preselected id renders as a raw UUID otherwise.\n\n**Adding a selector for a new entity** — write a thin wrapper, never a new picker. Copy `customerSelector.tsx` (the simplest one) and change only what genuinely differs: the list query, the by-id label resolver, and the label/description fields. The wrapper must:\n\n1. Call `useEntitySelectorSearch()` for open/search/debounce state, and pass `skip` to the RTK Query hook — nothing is fetched until the picker opens.\n2. `useMemo` the `options` array. Multi mode feeds it to react-select as `defaultOptions`, which re-syncs on identity change and will loop if the identity churns.\n3. Ship a `LabelResolver` component (`EntityLabelResolverProps`) that fetches one entity by id and calls `onResolved` — this is what keeps selected-but-unfetched ids from rendering as UUIDs.\n4. Type its props as `OwnProps & EntitySelectorModeProps` and extend `EntitySelectorCommonProps`, so all three modes and the shared prop surface come for free.\n5. Default `limit` to `ENTITY_SELECTOR_PAGE_SIZE`; expose a `filters` prop only if the endpoint supports server-side scoping.\n6. Search is **server-side** — never fetch a page and filter it client-side.\n\nDo not edit `entitySelector.tsx` to accommodate one surface. It only carries behaviour identical across every entity; per-entity differences belong in the wrapper, per-surface differences in props (`trigger`, `triggerClassName`, `excludeIds`, `noPortal`, `className`).\n\n**OSS ↔ enterprise placement.** `entitySelector.tsx` and any selector whose API is OSS live in `ui/components/entitySelectors/`. A selector for an enterprise-only API lives in `bifrost-enterprise/enterprise-ui/app/components/entitySelectors/` and OSS must never import it directly — OSS reaches it through a runtime registry (`ui/lib/registries/userPicker.tsx`, `ui/lib/registries/modelLimitScopes.tsx`), with an empty fallback under `ui/app/_fallbacks/enterprise/` so OSS-only builds simply hide the option. Keep single mode prop-compatible with the registry contract (`{ value, onChange, disabled, fallbackOption }`) so the selector can be registered as-is.\n\n---\n\n### JSX & Rendering\n\n- Avoid deeply nested conditional rendering\n- Break complex UI into smaller components\n- Keep components readable and maintainable\n\n---\n\n### Lists & Keys\n\n- Always use **stable, unique keys**\n- Never use array index as key (unless unavoidable)\n\n---\n\n## React Best Practices\n\n- Avoid unnecessary or unstable dependencies in hooks\n- Prevent infinite loops in `useEffect`\n- Keep dependency arrays accurate and minimal\n- Prefer derived state over duplicated state\n\n---\n\n## State Management\n\n### Priority Order\n\n1. Query Params (`nuqs`) → for persistent/shareable state\n2. Local State → for UI-only state\n3. Redux → only when truly necessary\n\n---\n\n### Query Params (`nuqs`)\n\n- Use for state that should persist across refresh/navigation\n- Use proper parsers like `parseAsString` or `parseAsInteger`\n- Do NOT mix query param state with local/redux state\n- Follow a single consistent pattern across the codebase\n\n---\n\n### Redux\n\n- Use only when global/shared state is required\n- Avoid unnecessary slices\n- Prefer simpler alternatives when possible\n\n---\n\n### RTK Query (`@reduxjs/toolkit/query`)\n\n- Use for API calls and caching\n- Use **granular tags** for cache invalidation\n- Avoid invalidating entire datasets unnecessarily\n- Implement **optimistic updates** where applicable\n\n---\n\n## Forms\n\nWe use:\n\n- `react-hook-form`\n- `zod v4` (for schema validation)\n\n### Rules\n\n- Always define a Zod schema\n- Include meaningful validation messages\n- Prefer **inline field errors** (not toast notifications)\n- Use `refine` / `superRefine` for complex validation\n- Store schemas in: `ui/lib/types/schemas.ts`\n\n---\n\n## Tables\n\n- Use `@tanstack/react-table` **only for large/complex datasets**\n- For simple tables → build custom lightweight components\n- Prioritize performance over abstraction\n\n---\n\n## ⚡ Performance Guidelines\n\n- Lazy load heavy or rarely-used libraries\n- Avoid unnecessary re-renders\n- Split large components into smaller ones\n- Keep bundle size minimal\n\n---\n\n## Dependency Rules\n\n- Do NOT add new dependencies unless absolutely necessary\n- Always pin exact versions (no `^` or `~`)\n- Prefer existing libraries in the codebase\n\n---\n\n## TypeScript Guidelines\n\n- Avoid using `any` unless absolutely unavoidable\n- Prefer strict typing and inference\n- Define reusable types in shared locations\n\n---\n\n## Code Quality & Formatting\n\nAfter writing code:\n\n```bash\ncd ui && npm run format\n````\n\nThen verify build:\n\n```bash\ncd ui && npm run build\n```\n\n* Code must pass formatting and build checks\n* Follow consistent naming and structure conventions\n\n---\n\n## Anti-Patterns to Avoid\n\n* Duplicate components without considering reuse\n* Mixing multiple state management approaches unnecessarily\n* Overusing Redux\n* Using unstable hook dependencies\n* Adding heavy libraries for simple use cases\n* Poorly structured or deeply nested JSX\n\n---\n\n## Summary\n\n* Prioritize **reusability, performance, and consistency**\n* Follow **strict folder structure and routing conventions**\n* Use **the right tool for the right problem**\n* Keep code **simple, predictable, and maintainable**\n","category":"root","tokens":12817}]}