# Technical Documentation: vectorize-io/hindsight > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/vectorize-io/hindsight) ### Overview (README.md)
[Documentation](https://hindsight.vectorize.io) • [Integrations](https://hindsight.vectorize.io/integrations) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Benchmarks](https://benchmarks.hindsight.vectorize.io/) • [Paper](https://arxiv.org/abs/2512.12818) • [Hindsight Cloud](https://ui.hindsight.vectorize.io) [](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml) [](https://pypi.org/project/hindsight-api/) [](https://pypi.org/project/hindsight-client/) [](https://www.npmjs.com/package/@vectorize-io/hindsight-client) [](https://vectorize.io/slack) [](https://opensource.org/licenses/MIT)

Star History Rank GitHub Trending Repository of the Day

--- ## What is Hindsight? Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember. It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks. **Contents** - [Memory Performance & Accuracy](#memory-performance--accuracy) - [Quick Start](#quick-start) — [server](#1-start-a-server) · [clients](#2-connect-a-client) · [platforms](#supported-platforms) · [embedded](#python-embedded-no-server-required) - [Adding Hindsight to Your Agent](#adding-hindsight-to-your-agent) — [LLM Wrapper](#llm-wrapper-2-lines-of-code) · [integrations](#integrations) · [coding agents](#coding-agents) · [MCP](#mcp-server) - [Core Concepts](#core-concepts) — [memory types](#memory-types) · [retain / recall / reflect](#the-three-operations) · [observations](#observations) · [mental models & knowledge pages](#mental-models--knowledge-pages) · [banks](#memory-banks) - [Use Cases](#use-cases) - [Running in Production](#running-in-production) - [Resources](#resources) --- ## Memory Performance & Accuracy Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here: > Live, continuously updated results — including per-model accuracy, latency and cost — are published at [benchmarks.hindsight.vectorize.io](https://benchmarks.hindsight.vectorize.io/). The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors. Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups. --- > 🤖 **Using a coding agent?** Install the Hindsight documentation skill for instant access to docs while you code: > ```bash > npx skills add https://github.com/vectorize-io/hindsight --skill hindsight-docs > ``` > Works with Claude Code, Cursor, and other AI coding assistants. --- ## Quick Start ### 1. Start a server #### Docker (recommended) ```bash export OPENAI_API_KEY=sk-xxx docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \ -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \ -v hindsight-data:/home/hindsight/.pg0 \ ghcr.io/vectorize-io/hindsight:latest ``` >API: http://localhost:8888 >UI: http://localhost:9999 Hindsight works with **25+ LLM providers** via `HINDSIGHT_API_LLM_PROVIDER` — hosted (`openai`, `anthropic`, `gemini`, `groq`, `bedrock`, `vertexai`, `minimax`, `deepseek`, `atlas`, `meta`, …), fully local (`ollama`, `lmstudio`, `llamacpp`), any OpenAI-compatible endpoint, and gateways (`litellm`, `litellmrouter`) that reach the rest. Existing subscriptions work too: `openai-codex` (ChatGPT Plus/Pro), `claude-code` (Claude Pro/Max), `cursor` (Cursor) and `github-copilot` (GitHub Copilot) need no API key. See [supported models](https://hindsight.vectorize.io/developer/models). #### Docker (external PostgreSQL) ```bash export OPENAI_API_KEY=sk-xxx export HINDSIGHT_DB_PASSWORD=choose-a-password cd docker/docker-compose docker compose up ``` > Oracle AI Database is also supported for enterprise deployments with full feature parity. See the [storage documentation](https://hindsight.vectorize.io/developer/storage) for details. #### Bare metal (pip) ```bash pip install hindsight-api export HINDSIGHT_API_LLM_API_KEY=sk-xxx hindsight-api ``` #### Kubernetes (Helm) ```bash helm install hindsight oci://ghcr.io/vectorize-io/charts/hindsight \ --set api.llm.provider=openai \ --set api.llm.apiKey=sk-xxx \ --set postgresql.enabled=true ``` #### Managed (no server) [Hindsight Cloud](https://vectorize.io/pricing) is the hosted option: managed infrastructure that scales automatically, plus a dashboard, backups, team collaboration and a 99.9% uptime SLA. Billing is usage-based with free credits to start — no fixed monthly or per-seat fee. Point any client at `https://api.hindsight.vectorize.io` with your API key and skip the deployment entirely. [Compare self-hosted, Cloud and Enterprise →](https://vectorize.io/pricing) · [Sign up →](https://ui.hindsight.vectorize.io) All options, including Windows and air-gapped setups, are covered in the [installation guide](https://hindsight.vectorize.io/developer/installation). ### 2. Connect a client ```bash pip install hindsight-client -U # Python npm install @vectorize-io/hindsight-client # Node.js / TypeScript go get github.com/vectorize-io/hindsight/hindsight-clients/go # Go curl -fsSL https://hindsight.vectorize.io/get-cli | bash # CLI ``` #### Python ```python from hindsight_client import Hindsight client = Hindsight(base_url="http://localhost:8888") # Retain: Store information client.retain(bank_id="my-bank", content="Alice works at Google as a software engineer") # Recall: Search memories client.recall(bank_id="my-bank", query="What does Alice do?") # Reflect: Generate disposition-aware response client.reflect(bank_id="my-bank", query="Tell me about Alice") ``` #### Node.js / TypeScript ```javascript const { HindsightClient } = require('@vectorize-io/hindsight-client'); const main = async () => { const client = new HindsightClient({ baseUrl: 'http://localhost:8888' }); await client.retain('my-bank', 'Alice loves hiking in Yosemite'); const results = await client.recall('my-bank', 'What does Alice like?'); console.log(results); } main(); ``` Full reference: [Python](https://hindsight.vectorize.io/sdks/python) · [Node.js](https://hindsight.vectorize.io/sdks/nodejs) · [Go](https://hindsight.vectorize.io/sdks/go) · [CLI](https://hindsight.vectorize.io/sdks/cli) · [REST API](https://hindsight.vectorize.io/api-reference) ### Supported Platforms | Platform | Docker | Bare Metal (pip) | Embedded DB (pg0) | |----------|--------|------------------|--------------------| | **Linux** (x86_64, ARM64) | ✅ | ✅ | ✅ | | **macOS** (Apple Silicon / arm64) | ✅ | ✅ | ✅ | | **macOS** (Intel / x86_64) | ✅ | ⚠️ | ✅ | | **Windows** (x86_64) | ✅ | ✅ | ✅ | ⚠️ Intel Macs: use `hindsight-all-slim` — see the [installation guide](https://hindsight.vectorize.io/developer/installation#supported-platforms) for details. ### Python Embedded (no server required) ```bash pip install hindsight-all -U ``` On Intel (x86_64) Macs, install `hindsight-all-slim` instead — see [Supported Platforms](#supported-platforms). ```python import os from hindsight import HindsightServer, HindsightClient with HindsightServer( llm_provider="openai", llm_model="gpt-5-mini", llm_api_key=os.environ["OPENAI_API_KEY"] ) as server: client = HindsightClient(base_url=server.url) client.retain(bank_id="my-bank", content="Alice works at Google") results = client.recall(bank_id="my-bank", query="Where does Alice work?") ``` A [Node.js equivalent](https://hindsight.vectorize.io/sdks/hindsight-all-npm) and a [daemon CLI](https://hindsight.vectorize.io/sdks/embed) are also available. --- ## Adding Hindsight to Your Agent ### LLM Wrapper (2 lines of code) The easiest way to add memory to an existing agent is the LLM Wrapper. Swap your LLM client for a wrapped one — memories are then stored and retrieved automatically on every call, with no other changes to your code. ```bash pip install hindsight-litellm ``` ```python from openai import OpenAI from hindsight_litellm import wrap_openai # Wrap your existing LLM client and you're done. # Defaults to Hindsight Cloud; pass hindsight_api_url for a self-hosted server. client = wrap_openai( OpenAI(), bank_id="user-123", hindsight_api_url="http://localhost:8888", ) # Hindsight recalls relevant memories before the call # and retains the conversation after it. response = client.chat.completions.create( model="gpt-5-mini", messages=[{"role": "user", "content": "What do you know about me?"}], ) ``` `wrap_anthropic()` does the same for the Anthropic SDK, and every setting — bank, recall budget, fact types, reflect instead of recall — can be overridden per call with `hindsight_*` kwargs. LiteLLM sits underneath, so the same integration covers **100+ models**. See the [LiteLLM integration](https://hindsight.vectorize.io/sdks/integrations/litellm). If you need explicit control over *when* memories are stored and recalled, use the [SDKs or REST API](#2-connect-a-client) directly instead. ### Integrations **60+ integrations** — most need no code changes. | | | |---|---| | **Coding agents** | [Claude Code](https://hindsight.vectorize.io/sdks/integrations/claude-code) · [Codex](https://hindsight.vectorize.io/sdks/integrations/codex) · [Cursor](https://hindsight.vectorize.io/sdks/integrations/cursor) · [GitHub Copilot](https://hindsight.vectorize.io/sdks/integrations/github-copilot) · [opencode](https://hindsight.vectorize.io/sdks/integrations/opencode) · [Cline](https://hindsight.vectorize.io/sdks/integrations/cline) · [Aider](https://hindsight.vectorize.io/sdks/integrations/aider) · [Zed](https://hindsight.vectorize.io/sdks/integrations/zed) · [Continue](https://hindsight.vectorize.io/sdks/integrations/continue) · [Roo Code](https://hindsight.vectorize.io/sdks/integrations/roo-code) · [OpenHands](https://hindsight.vectorize.io/sdks/integrations/openhands) | | **Agent frameworks** | [LangGraph / LangChain](https://hindsight.vectorize.io/sdks/integrations/langgraph) · [LlamaIndex](https://hindsight.vectorize.io/sdks/integrations/llamaindex) · [CrewAI](https://hindsight.vectorize.io/sdks/integrations/crewai) · [Pydantic AI](https://hindsight.vectorize.io/sdks/integrations/pydantic-ai) · [OpenAI Agents SDK](https://hindsight.vectorize.io/sdks/integrations/openai-agents) · [Google ADK](https://hindsight.vectorize.io/sdks/integrations/google-adk) · [Agno](https://hindsight.vectorize.io/sdks/integrations/agno) · [Strands](https://hindsight.vectorize.io/sdks/integrations/strands) · [AutoGen](https://hindsight.vectorize.io/sdks/integrations/autogen) · [Microsoft Agent Framework](https://hindsight.vectorize.io/sdks/integrations/agent-framework) · [Vercel AI SDK](https://hindsight.vectorize.io/sdks/integrations/ai-sdk) · [Haystack](https://hindsight.vectorize.io/sdks/integrations/haystack) | | **No-code / low-code** | [n8n](https://hindsight.vectorize.io/sdks/integrations/n8n) · [Zapier](https://hindsight.vectorize.io/sdks/integrations/zapier) · [Dify](https://hindsight.vectorize.io/sdks/integrations/dify) · [Flowise](https://hindsight.vectorize.io/sdks/integrations/flowise) | | **Apps & tools** | [ChatGPT](https://hindsight.vectorize.io/sdks/integrations/chatgpt) · [Perplexity](https://hindsight.vectorize.io/sdks/integrations/perplexity) · [Obsidian](https://hindsight.vectorize.io/sdks/integrations/obsidian) · [Pipecat](https://hindsight.vectorize.io/sdks/integrations/pipecat) · [Vapi](https://hindsight.vectorize.io/sdks/integrations/vapi) | 👉 [**Browse all integrations**](https://hindsight.vectorize.io/integrations) ### Coding Agents One package gives CLI coding agents long-term project memory: a per-repo bank built automatically from git history and past sessions, injected into the agent as it starts working, plus curated knowledge pages covering architecture, conventions and in-flight work. ```bash npx @vectorize-io/hindsight-coding-agents install all # every detected agent, wired natively npx @vectorize-io/hindsight-coding-agents install claude-code # or just one ``` Supports Claude Code, Codex CLI, Cursor CLI, GitHub Copilot CLI, opencode, Kilo CLI, Cline CLI, Antigravity CLI, Devin CLI, pi, Prime Agent, Grok Build and DeepSeek Harness. Ingestion is automatic — there is no setup command. See the [coding agents integration](https://hindsight.vectorize.io/sdks/integrations/coding-agents). ### MCP Server Every server ships a built-in [Model Context Protocol](https://modelcontextprotocol.io/) endpoint, one per bank, enabled by default: ``` http://localhost:8888/mcp/{bank_id}/ ``` Point any MCP client at it to expose retain, recall and reflect as tools. See the [MCP server docs](https://hindsight.vectorize.io/developer/mcp-server). --- ## Core Concepts ### Memory Types Most agent memory implementations rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works: - **World facts:** facts about the world ("The stove gets hot") - **Experiences:** the agent's own experiences ("I touched the stove and it really hurt") - **Observations:** consolidated, evidence-backed beliefs formed from many memories - **Mental models:** learned understanding of the agent's world, synthesized from observations and facts Memories live in **banks**. When memories are added, they are pushed into either the world facts or the experiences pathway, then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall. ### The Three Operations #### Retain The `retain` operation is used to push new memories into Hindsight. It tells Hindsight to _retain_ the information you pass in as an input. ```python client.retain( bank_id="my-bank", content="Alice got promoted to senior engineer", context="career update", timestamp="2025-06-15T10:00:00Z", ) ``` Behind the scenes, retain uses an LLM to extract key facts, temporal data, entities, and relationships. It passes these through a normalization process to transform extracted data into canonical entities, time series, and search indexes along with metadata. These representations create the pathways for accurate memory retrieval in the recall and reflect operations. [Retain docs →](https://hindsight.vectorize.io/developer/retain) #### Recall The recall operation is used to retrieve memories. These memories can come from any of the memory types (world, experiences, etc.) ```python client.recall(bank_id="my-bank", query="What does Alice do?") client.recall(bank_id="my-bank", query="What happened in June?") # temporal ``` Recall performs 4 retrieval strategies in parallel: - Semantic: Vector similarity - Keyword: BM25 exact matching - Graph: Entity/temporal/causal links - Temporal: Time range filtering The individual results are merged, ordered by relevance using reciprocal rank fusion and a cross-encoder reranking model, then trimmed as needed to fit within the token limit. [Recall docs →](https://hindsight.vectorize.io/developer/retrieval) #### Reflect The reflect operation performs a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world — or to answer a question that needs deep thinking rather than lookup. ```python client.reflect(bank_id="my-bank", query="What should I know about Alice?") ``` For example, reflect supports use cases such as: - An **AI Project Manager** reflecting on what risks need to be mitigated on a project. - A **Sales Agent** reflecting on why certain outreach messages have gotten responses while others haven't. - A **Support Agent** reflecting on opportunities where customers have questions not answered by current product documentation. [Reflect docs →](https://hindsight.vectorize.io/developer/reflect) ### Observations Retained facts don't stay a flat pile. In the background, Hindsight consolidates related facts into **observations** — deduplicated beliefs the bank has built up over time. Each observation keeps its supporting evidence with exact quotes and a proof count, and is *refined* rather than overwritten when new evidence arrives, so new information strengthens, weakens or extends an existing belief instead of silently replacing it. [Observations docs →](https://hindsight.vectorize.io/developer/observations) ### Mental Models & Knowledge Pages A **mental model** is a standing answer to a question about a bank ("What are this user's preferences?"). You define the question once; Hindsight writes the answer, stores it, and rewrites it in the background as the bank learns more. Reading one is a database read — no retrieval, no LLM call — so an agent can boot with a page of settled knowledge instead of rediscovering it every session. **Knowledge pages** are mental models with the mechanics hidden: living documents a bank writes about itself, organized in folders like a wiki, searchable, and projectable onto disk as ordinary markdown. Supply a name and a question; every other decision is a default you can override. [Mental models →](https://hindsight.vectorize.io/developer/mental-models) · [Knowledge pages →](https://hindsight.vectorize.io/developer/knowledge-pages) ### Memory Banks A **bank** is an isolated memory store — one "brain" for one user, agent, or project. Isolation is strict: no cross-bank leakage. Banks carry background context and **disposition traits** (skepticism, literalism, empathy) that shape how reflect reasons over their memories, and can be created from declarative [bank templates](https://hindsight.vectorize.io/developer/api/bank-templates). Two more things worth knowing: - **Multilingual by default.** Input language is detected and preserved end to end — facts stay in their original language and entities keep their native script (张伟 stays 张伟, not "Zhang Wei"). [Docs →](https://hindsight.vectorize.io/developer/multilingual) - **Memory Defense.** An opt-in, per-bank policy that scans every retain for secrets and PII against 45 patterns and either redacts the match (`[REDACTED:github_token]`) or blocks the item before it reaches storage. [Docs →](https://hindsight.vectorize.io/developer/memory-defense) --- ## Use Cases Hindsight is built to support conversational AI agents as well as agents that are intended to perform tasks autonomously. The ideal use case for Hindsight are agents that require a blend of these features such as AI employees that need to handle open-ended tasks, change behavior based on user feedback, and learn to perform complex tasks to automate work at a level that approximates a human work. Hindsight can be used with simple AI workflows like those built with n8n and other similar tools, but may be overkill for such applications. ### Per-User Memories and Chat History One of the simpler use cases you can use Hindsight for is to personalize AI chatbots and other conversational agents by storing and recalling memories associated with individual users. The requirements for this use case usually look something like this: Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories. More patterns in the [Cookbook](https://hindsight.vectorize.io/cookbook) and [Best Practices](https://hindsight.vectorize.io/best-practices). --- ## Running in Production | | | |---|---| | **Storage** | PostgreSQL + pgvector, or Oracle AI Database 23ai with full feature parity — [storage](https://hindsight.vectorize.io/developer/storage) | | **Configuration** | Hierarchical: global env vars → per-tenant → per-bank — [configuration](https://hindsight.vectorize.io/developer/configuration) | | **Monitoring** | Prometheus metrics and dashboards for LLM calls, tokens and latency — [monitoring](https://hindsight.vectorize.io/developer/monitoring) | | **Operations** | Admin CLI for migrations, bank repair and stuck operations — [admin CLI](https://hindsight.vectorize.io/developer/admin-cli) | | **Events** | Webhooks for retain, consolidation and refresh lifecycle events — [webhooks](https://hindsight.vectorize.io/developer/api/webhooks) | | **Extensibility** | Tenant, auth and storage extension points — [extensions](https://hindsight.vectorize.io/developer/extensions) | | **Managed** | Skip all of it with [Hindsight Cloud](https://vectorize.io/pricing) — managed, usage-based, 99.9% uptime SLA | --- ## Resources **Documentation:** - [Docs](https://hindsight.vectorize.io) · [FAQ](https://hindsight.vectorize.io/faq) · [Best Practices](https://hindsight.vectorize.io/best-practices) · [Cookbook](https://hindsight.vectorize.io/cookbook) · [Blog](https://hindsight.vectorize.io/blog) - [Paper](https://arxiv.org/abs/2512.12818) · [Benchmarks](https://benchmarks.hindsight.vectorize.io/) · [RAG vs Memory](https://hindsight.vectorize.io/developer/rag-vs-hindsight) **Clients:** - [Python](https://hindsight.vectorize.io/sdks/python) · [Node.js](https://hindsight.vectorize.io/sdks/nodejs) · [Go](https://hindsight.vectorize.io/sdks/go) · [CLI](https://hindsight.vectorize.io/sdks/cli) · [REST API](https://hindsight.vectorize.io/api-reference) **Community:** - [Slack](https://vectorize.io/slack) - [GitHub Issues](https://github.com/vectorize-io/hindsight/issues) --- ## Star History [](https://www.star-history.com/?repos=vectorize-io%2Fhindsight&type=date&legend=top-left) --- ## Contributing See [CONTRIBUTING.md](./CONTRIBUTING.md). ## License MIT — see [LICENSE](./LICENSE) --- Built by [Vectorize.io](https://vectorize.io) --- ### Hindsight Docs Docs Cloud Enterprise Overview (`hindsight-docs/docs/cloud-enterprise/overview.md`) --- title: Ways to run Hindsight sidebar_label: Overview description: Run open-source Hindsight yourself, use the managed Hindsight Cloud, or get a dedicated Hindsight Enterprise deployment. --- # Ways to run Hindsight Hindsight runs three ways. All of them use the same memory engine: the same retain, recall and reflect, the same mental models and knowledge pages, and the same API and SDKs. You can start on one and move to another later. | | Self-hosted (open source) | Hindsight Cloud | Hindsight Enterprise | |---|---|---|---| | **What it is** | The open-source engine, run by you | Managed, self-serve, multi-tenant service | A single-tenant deployment, installed and run with Vectorize | | **Where it runs** | Anywhere: your laptop, Docker, Kubernetes | Hindsight Cloud | Our cloud, your AWS, Google Cloud or Azure account, or your data center | | **Who runs it** | You | Vectorize | Vectorize, or you with Vectorize's help on-premises | | **Memory store** | PostgreSQL (or Oracle) | Managed by Vectorize | PostgreSQL, or [Hindsight Memlake](./enterprise.md#hindsight-memlake) on object storage | | **Models** | Bring your own LLM and embeddings | Included | Included, or your own (Bedrock, Vertex AI, Azure OpenAI) | | **Sign-in and access** | API keys you configure | Console, teams and API keys | Plus SSO, enforced MFA and private networking | | **Pricing** | Free (MIT license) | Pay as you go, from prepaid credits | Contract | | **Get started** | [Installation](/developer/installation) | [Sign up](https://ui.hindsight.vectorize.io/signup) | [Book a call](https://calendly.com/d/dvy5-v3f-jhg?utm_source=oss_docs&utm_campaign=enterprise_docs) | ## How to choose - **Self-hosted** if you want full control, want to try Hindsight locally, or already run Postgres and your own models. These docs cover it: start with [Installation](/developer/installation). - **[Hindsight Cloud](./hindsight-cloud.md)** if you want memory for your agents in minutes, with nothing to deploy and the models included. - **[Hindsight Enterprise](./enterprise.md)** if you need your own deployment: single-tenant isolation, a region or cloud account of your choice, private networking, SSO and compliance features, contracted support, or memory at a scale where [Hindsight Memlake](./enterprise.md#hindsight-memlake) pays off. Hindsight Cloud and Enterprise have their own documentation at [docs.hindsight.vectorize.io](https://docs.hindsight.vectorize.io). --- ### Hindsight Docs Docs Developer Index (`hindsight-docs/docs/developer/index.mdx`) --- sidebar_position: 1 slug: / title: Overview hide_title: true description: State of the art long-term memory for your agents. Hindsight gives AI agents memory that learns across sessions. hide_table_of_contents: true --- import HomeHero from '@site/src/components/HomeHero'; import HomeFlow from '@site/src/components/HomeFlow'; import HomeBenchmarks from '@site/src/components/HomeBenchmarks'; import SkillBanner from '@site/src/components/SkillBanner'; import Figure from '@site/src/components/Figure'; import whatHindsightDoes from '@site/figures/what-hindsight-does.json'; {/* The page's own `# Overview` H1 is suppressed via hide_title: this is the site root, and the hero's headline is the H1 a visitor and a crawler should see. The sidebar and page metadata still read "Overview" from the frontmatter. */} ## Benchmarks {/* Retrieval accuracy only. The coding-agent chart opens inside the coding-agents card above, where it is evidence for a path the reader has just been shown, and is explained in full on /sdks/integrations/coding-agents. */} ## Why Hindsight? AI agents forget everything between sessions. Every conversation starts from zero—no context about who you are, what you've discussed, or what the assistant has learned. This isn't just an implementation detail; it fundamentally limits what AI Agents can do. **The problem is harder than it looks:** - **Simple vector search isn't enough** — "What did Alice do last spring?" requires temporal reasoning, not just semantic similarity - **Facts get disconnected** — Knowing "Alice works at Google" and "Google is in Mountain View" should let you answer "Where does Alice work?" even if you never stored that directly - **AI Agents need to consolidate knowledge** — A coding assistant that remembers "the user prefers functional programming" should consolidate this into an observation and weigh it when making recommendations - **Context matters** — The same information means different things to different memory banks with different personalities Hindsight solves these problems with a memory system designed specifically for AI agents. {/* Rendered here rather than by the DocBreadcrumbs wrapper (which serves every other page): at the top it sat between the navbar and the hero band as a bordered strip and broke the full-bleed, and it lands better as the answer to the problem the section above just described. */} ## How It Works {/* The router: three kinds of visitor land on this page — one who has never heard of agent memory, one looking for the coding-agent plugin, one looking for the Hermes/OpenClaw plugin — and this is where each of them can see their own path. It opens "How It Works" because the cards answer "what does this look like for me" before the sections below answer "how". */} ### Memory Types Hindsight does not store conversations. It extracts what was said into typed facts and then builds on them: - **World fact** — an objective claim it was told. *"Alice works at Google."* - **Experience fact** — something the bank itself did. *"I recommended Python to Bob."* - **Observation** — a belief consolidated from many facts, with its evidence and its history. *"User was a React enthusiast, has now switched to Vue."* - **Mental model** — a curated summary you write for a question you ask often. - **Knowledge page** — a living document the bank writes about itself. Facts are not a list. Each is linked to the entities it mentions and to the other facts that share them, which is what makes "where does Alice work?" answerable from two facts that were never stored together — the graph at the top of this page is a sample bank's; hover a memory to see what it links to. ### Multi-Strategy Retrieval `recall()` runs four searches **in parallel**, because no single one handles every question: - **Semantic** — meaning rather than wording, so "Alice's job" finds "Alice works as a software engineer" - **Keyword (BM25)** — the names and technical terms an embedding blurs together; five pluggable Postgres backends, including one that works on a Citus cluster - **Graph** — entity links, so a fact reachable in two hops comes back even when it shares no words with the query - **Temporal** — time expressions parsed into a window, then filled by relevance *and spread across the range*, so "what happened in 2023?" is not all from December Then the part that matters more than any single arm: - **Fused by rank, not score** — a memory several strategies agree on wins, and no arm's scoring scale can dominate the others - **Re-ranked by a cross-encoder** that reads the query and the memory together - **Cut to a token budget, not a top-k** — you say how much context you can afford and Hindsight fills it, because agents budget in tokens and not in result counts See [Recall](/developer/retrieval) for the strategies in full. ### Observation Consolidation A background worker keeps turning raw facts into durable beliefs: - **Deduplicated** — overlapping facts merge into one observation instead of piling up as repeats - **Evidence-grounded** — each observation points at the memories that support it, with exact quotes and a proof count - **Refined, not overwritten** — new evidence updates an observation and its history is kept, so you can see a belief change - **Freshness-aware** — when memories have landed but not yet been consolidated, `reflect` treats the affected observations as stale and checks them against raw facts before trusting them ### Knowledge Pages Observations answer one question at a time. A **knowledge page** is a living document the bank writes about itself — "What are the components here?", "What's our error-handling convention?" — rewritten incrementally as consolidation produces new knowledge in its scope: - **A wiki, not a blob** — pages live in a tree of folders, browsable and searchable, each answering one question - **Built from observations** — synthesized from consolidated beliefs rather than raw conversation, so a page is not a transcript summary - **Never self-citing** — a page never reads another page, so they cannot cite each other into a feedback loop - **Real files when you want them** — `hindsight fs mount` projects the tree onto disk as ordinary markdown, so `grep`, an editor or an agent's file tools all work with no SDK See [Knowledge Pages](/developer/knowledge-pages) for the full model. ### Reflect `recall()` returns memories. `reflect()` returns an **answer**, by running an agentic loop over the bank rather than a single query: - **It gathers its own evidence** — the agent decides what it needs and calls its own tools, up to ten rounds, and cannot answer before it has retrieved something - **It checks sources in priority order** — mental models and knowledge pages first, then observations, and only then raw facts, so it reads the distilled answer before the transcript - **It cites what it used** — and only IDs it actually retrieved can be cited What makes two banks answer the same question differently is their configuration: - **Mission** — the bank's identity in plain language, which tells it what to prioritise. *"I am a research assistant specializing in ML. I prefer simplicity over cutting-edge."* - **Directives** — hard rules it must never break. *"Never recommend specific stocks."* - **Disposition** — skepticism, literalism and empathy on a 1–5 scale, shaping how it interprets what it finds These shape `reflect` only. `recall` returns the same memories whoever is asking. See [Reflect](/developer/reflect) for the loop in detail. ## Architecture Deep Dive Everything above, end to end and in motion: what a document turns into on the way in, what each of the three operations touches, and what the worker changes behind them. Play it, or step through it at your own pace.
## Integrations **Coding agents** — [one install](/sdks/integrations/coding-agents) wires 20+ harnesses (Claude Code, Codex CLI, Cursor CLI, opencode, Copilot CLI and more) to a per-repo memory bank. **Personal agents** — [Hermes](/sdks/integrations/hermes) and [OpenClaw](/sdks/integrations/openclaw) install Hindsight as their memory provider: every turn recalls what matters before answering and retains what was said afterwards. The Hermes desktop app configures it [in Settings](/sdks/integrations/hermes-desktop), no terminal. Everything else — frameworks, MCP servers, tools — is in the [Integrations Hub](/integrations). ## Next Steps ### Getting Started - [**Quick Start**](/developer/api/quickstart) — Install and get up and running in 60 seconds - [**RAG vs Hindsight**](/developer/rag-vs-hindsight) — See how Hindsight differs from traditional RAG with real examples ### Core Concepts - [**Retain**](/developer/retain) — How memories are stored with multi-dimensional facts - [**Recall**](/developer/retrieval) — How the 4-way parallel search retrieves memories - [**Reflect**](/developer/reflect) — How mission, directives, and disposition shape reasoning ### API Methods - [**Retain**](/developer/api/retain) — Store information in memory banks - [**Recall**](/developer/api/recall) — Search and retrieve memories - [**Reflect**](/developer/api/reflect) — Agentic reasoning with memory - [**Mental Models**](/developer/api/mental-models) — User-curated summaries for common queries - [**Memory Banks**](/developer/api/memory-banks) — Configure mission, directives, and disposition - [**Documents**](/developer/api/documents) — Manage document sources - [**Operations**](/developer/api/operations) — Monitor async tasks ### Deployment - [**Server Setup**](/developer/installation) — Deploy with Docker Compose, Helm, or pip --- ### Hindsight Docs Docs Developer Api Quickstart (`hindsight-docs/docs/developer/api/quickstart.mdx`) --- sidebar_position: 0 --- # Quick Start Get up and running with Hindsight in 60 seconds. import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeSnippet from '@site/src/components/CodeSnippet'; import {ClientsGrid} from '@site/src/components/SupportedGrids'; {/* Import raw source files */} import quickstartPy from '!!raw-loader!@site/examples/api/quickstart.py'; import quickstartMjs from '!!raw-loader!@site/examples/api/quickstart.mjs'; import quickstartSh from '!!raw-loader!@site/examples/api/quickstart.sh'; import quickstartGo from '!!raw-loader!@site/examples/api/quickstart.go'; ## Clients ## Start the API Server ```bash pip install hindsight-api export OPENAI_API_KEY=sk-xxx export HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY hindsight-api ``` API available at [http://localhost:8888](http://localhost:8888/docs) ```bash export OPENAI_API_KEY=sk-xxx docker run -it --pull always --name hindsight --restart unless-stopped --shm-size=1g -p 8888:8888 -p 9999:9999 \ -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \ -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \ ghcr.io/vectorize-io/hindsight:latest ``` - **API**: http://localhost:8888 - **Control Plane** (Web UI): http://localhost:9999 :::tip Set a stable `HINDSIGHT_API_WORKER_ID` in production The worker uses the container hostname as its identity, which Docker sets to the container ID by default. That value changes on every restart, so any task that was being processed when the container went down stays parked under the old ID with no way for the new container to recognize it as its own. Set `HINDSIGHT_API_WORKER_ID` to a stable value (e.g., `-e HINDSIGHT_API_WORKER_ID=hindsight-prod`) so the worker keeps the same identity across restarts. This is recommended even for single-container deployments. For diagnosis and recovery commands, see [Admin CLI - Recovering stuck operations](/developer/admin-cli#recovering-stuck-or-zombie-operations). ::: :::tip LLM Provider Hindsight requires an LLM with structured output support. Recommended: **Groq** with `gpt-oss-20b` for fast, cost-effective inference. See [LLM Providers](/developer/models#llm) for more details. ::: --- ## Use the Client ```bash pip install hindsight-client ``` ```bash npm install @vectorize-io/hindsight-client ``` ```bash curl -fsSL https://hindsight.vectorize.io/get-cli | bash ``` ```bash go get github.com/vectorize-io/hindsight/hindsight-clients/go ``` --- ## What's Happening | Operation | What it does | |-----------|--------------| | **Retain** | Content is processed, facts are extracted, entities are identified and linked in a knowledge graph | | **Recall** | Four search strategies (semantic, keyword, graph, temporal) run in parallel to find relevant memories | | **Reflect** | Retrieved memories are used to generate a disposition-aware response | --- ## Integrations Browse all supported integrations in the [Integrations Hub](/integrations). ## Next Steps - [**Retain**](./retain) — Advanced options for storing memories - [**Recall**](./recall) — Search and retrieval strategies - [**Reflect**](./reflect) — Disposition-aware reasoning - [**Memory Banks**](./memory-banks) — Configure disposition and mission - [**Server Deployment**](/developer/installation) — Docker Compose, Helm, and production setup --- ### Hindsight Docs Docs Developer Memory Defense Index (`hindsight-docs/docs/developer/memory-defense/index.md`) --- sidebar_position: 95 --- # Memory Defense Hindsight scrubs secrets and PII from retain content using a 45-pattern regex set. Each match is replaced with a `[REDACTED:type]` marker before content reaches memory units or the document body. The feature is configured per bank and disabled by default. ## How it works Memory Defense is opt-in per bank. The extension is always present, but it sits dormant until you give a bank a policy that turns it on. When a policy is set, every memory the agent writes to that bank is scanned before it lands in storage. When the scanner recognizes a credential, an API key, a database connection string, or a known PII format, the matched substring is replaced with a redaction marker like `[REDACTED:github_token]`. The scrubbed version is what actually gets stored. Memory units and document bodies persist the redacted text, so future recall responses, exports, and reflect operations never see the original secret. A policy only affects future retain calls on the bank where it is set. Existing memories are not retroactively scanned when you add or change a policy. ## Configuring Memory Defense Memory Defense is configured per bank via the bank's `memory_defense` config field. You can set the policy at bank creation time or update it later via `PATCH /v1/{tenant}/banks/{bank_id}/config`. The open-source version implements the `sensitive_data` rule with two possible actions: - **`redact`** — replace each matched secret with a `[REDACTED:type]` marker and store the scrubbed memory. - **`block`** — drop any item that contains a match. If every item in a retain request is blocked, the call returns `422`. A minimal policy: ```json { "memory_defense": { "enabled": true, "rules": [ { "on": "sensitive_data", "action": "redact" } ] } } ``` Once that policy is on a bank, every retain to that bank is screened with the 45 redaction patterns documented below. :::note Existing memories are not retroactively scanned Enabling Memory Defense on a bank only affects future retain calls. Memories already in the bank are not re-scanned or modified when you add or change a policy. If you need to scrub a bank that already contains unredacted content, you have to re-ingest the affected memories or remove them manually. ::: ### Disabled by default Memory Defense is off on every bank until you set a policy. A bank with no `memory_defense` field, with `enabled: false`, or with no `sensitive_data` rule is treated identically: the extension returns ALLOW and content passes through unchanged. To stop redacting on a bank that has it on, set `enabled: false` or remove the policy. ## Notifications When an item is redacted or blocked, Hindsight fires a [`memory_defense.triggered` webhook](../api/webhooks.mdx#memory_defensetriggered) if a webhook on the bank is subscribed to that event type. The payload reports the action taken, the document ID, and which redaction patterns matched — useful for routing security alerts to a SIEM or Slack. Clean items fire no event. The same redact/block decisions are also recorded as `memory_defense` entries in the [audit log](../configuration.mdx#audit-logging) (when audit logging is enabled), with the action and matched pattern labels in the entry metadata. ## Patterns covered The 45 bundled patterns cover the categories below. ### AI and LLM providers | Label | Catches | |---|---| | `anthropic_key` | `sk-ant-...` | | `openai_key`, `openai_project_key`, `openai_admin_key` | `sk-...`, `sk-proj-...`, `sk-admin-...` | | `google_api_key` | `AIza...` (39 chars) | | `google_oauth_token` | `ya29.` | | `xai_key` | `xai-...` | | `groq_key` | `gsk_...` | | `huggingface_token` | `hf_...` | | `replicate_token` | `r8_...` | | `perplexity_key` | `pplx-...` | | `databricks_token` | `dapi` | ### Cloud providers | Label | Catches | |---|---| | `aws_access_key` | `AKIA<16>` | | `aws_session_token` | `ASIA<16>` | | `digitalocean_token` | `dop_v1_` | ### Source control and CI | Label | Catches | |---|---| | `github_fg_pat` | `github_pat_...` | | `github_token` | `ghp_<36>` | | `github_app_token` | `ghs_<36>` | | `github_user_token` | `ghu_<36>` | | `github_refresh` | `ghr_<36>` | | `github_oauth` | `gho_<36>` | | `gitlab_pat` | `glpat-...` | | `npm_token` | `npm_...` | | `pypi_token` | `pypi-AgEIcHlwaS5vcmc...` | ### Payment processors | Label | Catches | |---|---| | `stripe_secret` | `sk_live_...`, `sk_test_...` | | `stripe_restricted` | `rk_live_...`, `rk_test_...` | | `square_token` | `sq0...` | | `braintree_token` | `access_token$production$...` | ### Communications and email | Label | Catches | |---|---| | `slack_token` | `xoxb-`, `xoxp-`, `xoxa-`, `xoxr-` | | `slack_webhook` | `https://hooks.slack.com/services/...` | | `twilio_api_key` | `SK` | | `twilio_account_sid` | `AC` | | `sendgrid_key` | `SG.<22>.<43>` | | `mailgun_key` | `key-<32>` | | `discord_bot` | `<23>.<6>.<27>` | | `telegram_bot` | `<8-10 digits>:<35>` | ### Commerce | Label | Catches | |---|---| | `shopify_token` | `shpat_` | ### Database connection strings | Label | Catches | |---|---| | `db_url_postgres` | `postgres://user:pass@host` or `postgresql://...` | | `db_url_mysql` | `mysql://user:pass@host` | | `db_url_mongodb` | `mongodb://user:pass@host` or `mongodb+srv://...` | ### Private keys, JWTs, and generic credentials | Label | Catches | |---|---| | `private_key_pem` | `-----BEGIN ... PRIVATE KEY-----` PEM blocks | | `jwt` | `eyJ
.eyJ.` | ### PII (US defaults) | Label | Catches | |---|---| | `credit_card` | 13 to 19 digits with regular separators | | `ssn_us` | `123-45-6789` shape | --- ### Hindsight Docs Docs Cloud Enterprise Enterprise (`hindsight-docs/docs/cloud-enterprise/enterprise.md`) --- title: Hindsight Enterprise description: Hindsight Enterprise is a single-tenant Hindsight deployment, in our cloud, your cloud or your data center, with Hindsight Memlake for memory at scale. --- # Hindsight Enterprise Hindsight Enterprise is Hindsight for organizations that need their own deployment. Instead of sharing the multi-tenant Hindsight Cloud, you get a **single-tenant** installation that Vectorize sets up and runs with you. It includes everything in Hindsight Cloud, plus the security, compliance, networking and scale features below. **[Book a call](https://calendly.com/d/dvy5-v3f-jhg?utm_source=oss_docs&utm_campaign=enterprise_docs)** · **[Hindsight Enterprise docs](https://docs.hindsight.vectorize.io/enterprise)** ## Deployment options | Option | Where it runs | Who runs it | |---|---|---| | **Hindsight Cloud with Enterprise features** | Hindsight Cloud (multi-tenant) | Vectorize | | **Private cloud** | A dedicated deployment in Vectorize's cloud account, in the region you choose | Vectorize | | **Bring your own cloud (BYOC)** | Your AWS, Google Cloud or Azure account | Vectorize, through access you grant | | **On-premises** | Your Kubernetes infrastructure, including air-gapped | You, with Vectorize's help | Private cloud and BYOC can be reached over private networking (AWS PrivateLink, Google Cloud Private Service Connect, Azure Private Link), so traffic never crosses the public internet. See [Choosing a deployment option](https://docs.hindsight.vectorize.io/enterprise/deployment/overview) for a full comparison, including who is responsible for what. ## What Enterprise adds - **Single-tenant deployment:** its own database, storage, compute and identity service. - **[Hindsight Memlake](#hindsight-memlake):** memory on object storage, for large and growing banks. - **Single sign-on and MFA:** Okta, Microsoft Entra ID, Google Workspace or any OIDC or SAML provider. - **Memory Defense:** detect and redact secrets and sensitive data, and block prompt injection before it is stored. - **Data privacy (GDPR):** erase, retain and export a person's data across banks. - **Audit logs, webhooks and SIEM:** a record of who did what, streamed to your own systems. - **Enterprise support:** a ticket portal, a shared Slack channel and contracted response times. - **Invoiced billing:** annual or contract billing instead of card payments. ## Hindsight Memlake Hindsight Memlake is the Enterprise storage engine for agent memory. Open-source Hindsight stores memories in PostgreSQL, which works well for most banks. But agent memory only grows, and as banks reach hundreds of thousands of memories, a relational database gets slower and more expensive to keep fast. Memlake keeps memories in **object storage** (S3, Google Cloud Storage, Azure Blob Storage, or an S3-compatible store on-premises), with one bucket per organization: - **Built for scale.** In our tests, 100 million memories were served at under 100 ms p99 recall, on four servers. - **Fast and steady.** Recall time barely moves as a bank grows, because each bank has its own index, kept warm in a cache on the servers. - **Lower cost as you grow.** Memories live in object storage, the cheapest durable storage there is, and servers hold only a cache, so you add servers for traffic, not for data. - **Durable.** A write is in object storage before it is acknowledged, and continuously backed up to a separate bucket. - **Nothing changes for your agents.** Same API, SDKs and features; banks, settings and background jobs stay in PostgreSQL. If your banks stay under roughly 50,000 memories each, PostgreSQL is a good fit and Memlake is not needed. Memlake is available on private cloud, BYOC and on-premises deployments. Read more: [Hindsight Memlake overview](https://docs.hindsight.vectorize.io/enterprise/memlake), [architecture](https://docs.hindsight.vectorize.io/enterprise/memlake/architecture), [performance and cost](https://docs.hindsight.vectorize.io/enterprise/memlake/performance) and [tradeoffs](https://docs.hindsight.vectorize.io/enterprise/memlake/tradeoffs). ## Learn more - [Hindsight Enterprise documentation](https://docs.hindsight.vectorize.io/enterprise) - [Ways to run Hindsight](./overview.md): self-hosted, Cloud and Enterprise compared - [Book a call](https://calendly.com/d/dvy5-v3f-jhg?utm_source=oss_docs&utm_campaign=enterprise_docs) with the Hindsight team --- ### Hindsight Docs Docs Cloud Enterprise Hindsight Cloud (`hindsight-docs/docs/cloud-enterprise/hindsight-cloud.md`) --- title: Hindsight Cloud description: Hindsight Cloud is the managed, self-serve Hindsight service. Nothing to deploy, models included. --- # Hindsight Cloud Hindsight Cloud is Hindsight run for you. Sign up, create a memory bank and connect your agents, with nothing to deploy or operate. It runs the same engine as open-source Hindsight, so everything in these docs about how memory works applies. **[Start on Hindsight Cloud](https://ui.hindsight.vectorize.io/signup)** · **[Hindsight Cloud docs](https://docs.hindsight.vectorize.io/cloud)** ## What you get - **Nothing to run.** No servers, databases or upgrades. The LLM, embedding and reranking models are included. - **The same API.** Retain, recall, reflect, mental models and knowledge pages, through the same [Python](/sdks/python), [TypeScript](/sdks/nodejs) and [Go](/sdks/go) clients, the [CLI](/sdks/cli) and the MCP server. Point them at your Cloud API URL and key. - **[Integrations](/integrations).** Pre-built integrations with agent frameworks, coding agents and AI tools. - **A console.** Browse banks, memories, mental models and knowledge pages, and see how a bank is learning. - **Teams.** Invite your team with role-based access, and manage API keys. - **Usage analytics.** Dashboards for what each bank uses. ## Pricing You pay for what you use, from a prepaid credit balance: per token for retain and recall, per call for reflect and for creating or refreshing mental models and knowledge pages, and for storage of memories kept longer than 30 days. See [Billing](https://docs.hindsight.vectorize.io/cloud/billing) for current rates. ## Enterprise features on Cloud Some features are part of Hindsight Enterprise and can be turned on for a Cloud organization: SSO, enforced MFA, Memory Defense, data privacy (GDPR), audit logs, webhooks and SIEM, and Enterprise support. See [Hindsight Cloud with Enterprise features](https://docs.hindsight.vectorize.io/enterprise/deployment/cloud-enterprise). Hindsight Cloud is shared and multi-tenant. If you need a dedicated deployment, see [Hindsight Enterprise](./enterprise.md). ## Learn more - [Getting started](https://docs.hindsight.vectorize.io/cloud/getting-started): your first memory bank, step by step - [Hindsight Cloud documentation](https://docs.hindsight.vectorize.io/cloud) - [Ways to run Hindsight](./overview.md): self-hosted, Cloud and Enterprise compared --- ### Hindsight Docs Docs Developer Admin Cli (`hindsight-docs/docs/developer/admin-cli.md`) # Admin CLI The `hindsight-admin` CLI provides administrative commands for managing your Hindsight deployment, including database migrations, backup, and restore operations. ## Installation The admin CLI is included with the `hindsight-api` package — installing it puts the `hindsight-admin` executable on your `PATH`: ```bash pip install hindsight-api # or uv add hindsight-api ``` ## Running the CLI `hindsight-admin` connects **directly to PostgreSQL** — it does not call the HTTP API. It reads the **same configuration as the API service** (environment variables, and a `.env` file in the current working directory), so it operates on whatever database `HINDSIGHT_API_DATABASE_URL` points to: - **Default**: `pg0`, the embedded development database (must be run on the host that owns the pg0 data). - **Production**: set `HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/hindsight`. Because it talks to the database directly (binary `COPY`, `TRUNCATE`, etc.), the admin CLI is **PostgreSQL-only** (not supported on Oracle). Run it on the same host/container as your API deployment so it inherits the right configuration and has network access to the database: ```bash # Bare metal / virtualenv (with the API's env or a .env in the working dir) hindsight-admin worker-status # Docker — exec into the API container docker exec -it hindsight-api hindsight-admin backup /data/backup.zip # Kubernetes — exec into an API pod kubectl exec deploy/hindsight-api -- hindsight-admin run-db-migration ``` Use `--schema` to target a specific tenant schema (commands default to the configured base schema). See [Environment Variables](#environment-variables) below. ## Commands ### run-db-migration Run database migrations to the latest version. By default this migrates the base schema plus all tenant schemas discovered by the tenant extension. Use `--schema` for targeted migration of one schema. This is useful when you want to run migrations separately from API startup (e.g., in CI/CD pipelines or before deploying a new version). ```bash hindsight-admin run-db-migration [OPTIONS] ``` **Options:** | Option | Description | Default | |--------|-------------|---------| | `--schema`, `-s` | Database schema to run migrations on. If omitted, migrate the base schema plus all discovered tenant schemas. | All schemas | | `--embedding-dimension` | Expected embedding dimension to enforce after migrations. Omit to skip the post-migration dimension sync. | Skipped | | `--skip-extension-reconcile` | Skip the post-migration vector / text-search index reconcile (it only does work when `HINDSIGHT_API_VECTOR_EXTENSION` / `HINDSIGHT_API_TEXT_SEARCH_EXTENSION` differs from a schema's existing indexes). Makes a no-change re-migration across many tenant schemas much faster; only use when the backend is unchanged. | Reconcile runs | **Examples:** ```bash # Run migrations on the base schema plus all discovered tenant schemas hindsight-admin run-db-migration # Run migrations on a specific tenant schema hindsight-admin run-db-migration --schema tenant_acme ``` :::tip Disabling Auto-Migrations To disable automatic migrations on API startup, set `HINDSIGHT_API_RUN_MIGRATIONS_ON_STARTUP=false`. This is useful when you want to run migrations as a separate step in your deployment pipeline. ::: --- ### repair-bank Verify and repair a bank's derived state: its per-`(bank, fact_type)` vector index coverage, and observations whose source memories no longer exist. These partial indexes are normally created when a bank is first created (instant on an empty bank), and PostgreSQL maintains them incrementally as the bank grows. A bank that becomes **populated outside that create-time path** — via a logical restore, a cross-version upgrade, or a vector-extension switch — never gets them, so its bank-scoped recall silently falls back to a global index + post-filter. That fallback is both **slower** and can **under-return** results (the approximate nearest-neighbour search draws its candidates from every bank, then filters to yours afterward). Run this after any of those events to restore full coverage. It detects **missing or invalid** coverage (an INVALID leftover from an interrupted build, or an index whose type drifted after a backend switch, both count as missing) and rebuilds it with `CREATE INDEX CONCURRENTLY`, so it never blocks live retain/recall/consolidation. It is idempotent and safe to re-run. ```bash hindsight-admin repair-bank (--bank BANK_ID | --all) [OPTIONS] ``` **Options:** | Option | Description | Default | |--------|-------------|---------| | `--bank`, `-b` | Bank id to repair. Mutually exclusive with `--all`. | — | | `--all` | Repair every bank in the base schema and all discovered tenant schemas. | — | | `--schema`, `-s` | Limit to a single schema. | All schemas | | `--dry-run` | Report what would be repaired without creating, dropping or deleting anything. | Off | Exactly one of `--bank` or `--all` is required. Index repair is skipped for backends that use a single global vector index (AlloyDB ScaNN); the observation cleanup below still runs. PostgreSQL only. It is idempotent — safe to re-run and safe to run while the API is serving traffic. It is also a no-op for a bank whose memories a custom store owns, this time decided per bank rather than per deployment: such a bank has no rows in `memory_units`, so it is owed no per-bank index and nothing is built for it. Nothing is dropped either — a bank that carries indexes from before it moved to the store keeps them, because shedding them is an operator decision with its own timing, not something a repair does on your behalf. A store-owned bank therefore reports `0 present, 0 created` whether or not it still carries indexes; to see what one actually carries, query the catalog for that bank's own partial indexes: ```sql SELECT indexname FROM pg_indexes WHERE schemaname = '' AND tablename = 'memory_units' AND indexname LIKE 'idx_mu_emb_%' AND strpos(indexdef, $$bank_id = ''$$) > 0; ``` Drop the last condition to list every per-bank index in the schema instead. Those indexes are empty, and Postgres plans against every index on a relation, so on a deployment with many store-owned banks they are charged to every other query that touches `memory_units`. Drop them with `DROP INDEX CONCURRENTLY` when you are ready. **Orphan observations.** An observation summarises the facts it was built from, so when one of those facts is deleted the observation is deleted too, and its other source facts are queued for consolidation again. Older releases missed this on some delete paths, leaving observations behind that cite facts that no longer exist. They keep showing up in recall and reflect, and `export-bank` skips them. `repair-bank` removes them using the same rule a normal delete uses. The surviving facts are consolidated again on the bank's next consolidation run. Use `--dry-run` to only count them. **Observation keyword index (VectorChord only).** With `HINDSIGHT_API_TEXT_SEARCH_EXTENSION=vchord`, older releases did not update an observation's keyword index when consolidation rewrote or merged its text, so keyword recall kept matching the old wording. `repair-bank` rebuilds the keyword index of every observation in the bank from its current text. A stale one cannot be told apart from an up-to-date one, so all of them are rebuilt. Other text-search backends have nothing to rebuild here. Use `--dry-run` to only count them. **Exit code.** `0` when every schema was reconciled. `1` when an index failed to build (the names are printed, and a re-run retries them), or when a schema was not fully reconciled — for example because the memories store could not say which banks it owns. The command does not guess there, because guessing would rebuild the very indexes described above. A schema is named as skipped whether it failed before its first bank or partway through, so it may already have had some banks reconciled; when it got that far the per-schema line says how many, and those builds are not undone. Both the failed-index names and the skipped schemas are reported before the command exits, so one never hides the other. If the database connection itself is lost mid-sweep, the run stops there rather than repeating the same error for every remaining schema, and still reports what it did. **Examples:** ```bash # See which banks are missing coverage without changing anything hindsight-admin repair-bank --all --dry-run # Repair every bank across all schemas (run once after a restore/upgrade) hindsight-admin repair-bank --all # Repair a single bank hindsight-admin repair-bank --bank acme-prod ``` --- ### rename-bank :::tip Consider an alias instead A rename requires stopping the bank's clients, because the old id stops working the moment it commits. If what you want is to move callers onto a new id, **[bank aliases](./api/memory-banks.mdx#aliases)** do that with no downtime and no row rewrite: the bank answers to both ids until you are done. Reach for `rename-bank` when the old id must genuinely stop existing — an alias never removes one. ::: Change a bank's id in place. Every memory, document, entity, mental model and knowledge page is kept exactly as it is. Nothing is re-extracted or re-embedded, and no LLM calls are made. ```bash hindsight-admin rename-bank --from OLD_ID --to NEW_ID [OPTIONS] ``` **Options:** | Option | Description | Default | |--------|-------------|---------| | `--from` | Current bank id. | — | | `--to` | New bank id. Must not already exist. | — | | `--schema`, `-s` | Schema the bank lives in. | Configured base schema | | `--dry-run` | Run the whole rename, print the rows that would move per table, then roll back. | Off | **Before you run it:** - **Stop the bank's clients.** Anything still calling the old id gets a 404, or, if it creates banks on demand, silently starts a new, empty bank under the old id. Point every client, and any API key scoped to the bank, at the new id before restarting them. - **Let the bank's operations finish.** The command refuses while the bank has pending or processing operations. A file's storage key spells the bank id, so after the rows move the bank's stored files — document attachments and uploaded originals — are copied to keys under the new id, their rows repointed, and the old id's prefix cleared. That last step also removes any export or import archive left under the old id: their download links stop working at the rename, and the operation can produce them again. Deleting the bank afterwards removes its files with it. The rename runs in a single transaction and only locks that bank's rows. Other banks keep serving. Its duration grows with the bank's size. Afterwards, the bank's vector indexes are rebuilt with `CREATE INDEX CONCURRENTLY`, and until that finishes, recall on the renamed bank runs without them. Running API servers may keep answering for the old id until their bank-info cache expires. PostgreSQL only. **Examples:** ```bash # See what would move, change nothing hindsight-admin rename-bank --from acme --to acme-prod --dry-run hindsight-admin rename-bank --from acme --to acme-prod ``` --- ### backup Create a backup of all Hindsight data to a zip file. ```bash hindsight-admin backup OUTPUT [OPTIONS] ``` **Arguments:** | Argument | Description | |----------|-------------| | `OUTPUT` | Output file path (will add `.zip` extension if not present) | **Options:** | Option | Description | Default | |--------|-------------|---------| | `--schema`, `-s` | Database schema to backup | `public` | **Examples:** ```bash # Backup to a file hindsight-admin backup /backups/hindsight-2024-01-15.zip # Backup a specific tenant schema hindsight-admin backup /backups/tenant-acme.zip --schema tenant_acme ``` The backup includes: - Memory banks and their configuration - Documents and chunks - Entities and their relationships - Memory units (facts, experiences, observations) - Entity cooccurrences and memory links - Mental models and directives - Webhooks and file storage - Internal operational tables (async operations, audit log, graph-maintenance queue, and similar bookkeeping) so a restore reproduces a faithful full-database snapshot :::note Consistency Backups are created within a database transaction with `REPEATABLE READ` isolation, ensuring a consistent snapshot across all tables. ::: --- ### restore Restore data from a backup file. **Warning: This deletes all existing data in the target schema.** ```bash hindsight-admin restore INPUT [OPTIONS] ``` **Arguments:** | Argument | Description | |----------|-------------| | `INPUT` | Input backup file (.zip) | **Options:** | Option | Description | Default | |--------|-------------|---------| | `--schema`, `-s` | Database schema to restore to | `public` | | `--yes`, `-y` | Skip confirmation prompt | `false` | **Examples:** ```bash # Restore with confirmation prompt hindsight-admin restore /backups/hindsight-2024-01-15.zip # Restore without confirmation (for scripts) hindsight-admin restore /backups/hindsight-2024-01-15.zip --yes # Restore to a specific tenant schema hindsight-admin restore /backups/tenant-acme.zip --schema tenant_acme --yes ``` :::warning Data Loss Restore will **delete all existing data** in the target schema before importing the backup. Always verify you have a recent backup before performing a restore. ::: --- ### decommission-worker Release all tasks owned by a worker, resetting them from "processing" back to "pending" status so they can be picked up by other workers. ```bash hindsight-admin decommission-worker WORKER_ID [OPTIONS] ``` **Arguments:** | Argument | Description | |----------|-------------| | `WORKER_ID` | ID of the worker to decommission | **Options:** | Option | Description | Default | |--------|-------------|---------| | `--schema`, `-s` | Database schema | `public` | | `--yes`, `-y` | Skip confirmation prompt | `false` | **Examples:** ```bash # Before scaling down - release tasks from workers being removed hindsight-admin decommission-worker hindsight-worker-4 hindsight-admin decommission-worker hindsight-worker-3 # Release tasks from a crashed worker hindsight-admin decommission-worker worker-2 # For a specific tenant schema hindsight-admin decommission-worker worker-1 --schema tenant_acme ``` **When to Use:** - **Scaling down**: Before removing worker replicas in Kubernetes - **Graceful removal**: When taking a worker offline for maintenance - **Crash recovery**: If a worker crashed while processing tasks - **Stuck worker**: When a worker is unresponsive :::tip Finding Worker IDs Worker IDs default to the hostname. In Kubernetes StatefulSets, this is the pod name (e.g., `hindsight-worker-0`). You can also set a custom ID with `HINDSIGHT_API_WORKER_ID` or `--worker-id`. ::: ### decommission-workers Release all currently-processing tasks from every worker, resetting them from "processing" back to "pending" status. Use this when one or more workers have crashed or been removed without graceful shutdown and you don't know which worker IDs to target. ```bash hindsight-admin decommission-workers [OPTIONS] ``` **Options:** | Option | Description | Default | |--------|-------------|---------| | `--schema`, `-s` | Database schema | `public` | | `--yes`, `-y` | Skip confirmation prompt | `false` | **Examples:** ```bash # Release all processing tasks across all workers (with confirmation) hindsight-admin decommission-workers # Skip the confirmation prompt (useful in scripts) hindsight-admin decommission-workers --yes # Release tasks in a specific tenant schema hindsight-admin decommission-workers --schema tenant_acme ``` **When to Use:** - **Unknown dead workers**: Multiple workers crashed and you do not know their IDs - **Fleet-wide recovery**: After an infrastructure event where many workers went down - **"Just fix everything"**: A quick full-queue drain when per-worker cleanup is overkill :::warning Disruptive This releases **every** processing task regardless of worker, including tasks owned by healthy workers. Prefer `decommission-worker ` when you know which workers need cleanup. ::: --- ### worker-status Show all currently-processing tasks grouped by worker, including operation type, bank, how long each task has been running, and when it was last updated. Useful for identifying orphaned tasks before decommissioning. ```bash hindsight-admin worker-status [OPTIONS] ``` **Options:** | Option | Description | Default | |--------|-------------|---------| | `--schema`, `-s` | Database schema | `public` | **Examples:** ```bash # Show all processing tasks across all workers hindsight-admin worker-status # Show processing tasks for a specific tenant schema hindsight-admin worker-status --schema tenant_acme ``` **When to Use:** - **Before decommissioning**: Inspect which workers have stale tasks and how long they have been stuck - **Debugging throughput**: Diagnose why the queue is not draining (are tasks stuck in processing?) - **Worker health check**: Spot workers whose `last_update_ago` keeps growing, indicating a dead or unresponsive worker --- ### export-bank Export an entire bank to a portable ZIP archive — documents, facts, observations, bank configuration, mental models, directives, and webhooks. Embeddings are **never** included; they are regenerated on import. This is the source half of a cross-instance migration (e.g. moving to a different embedding model, vector extension, or text-search backend). PostgreSQL only. ```bash hindsight-admin export-bank --bank --output [OPTIONS] ``` **Options:** | Option | Description | Default | |--------|-------------|---------| | `--bank`, `-b` | Bank id to export. | (required) | | `--output`, `-o` | Path to write the `.zip` archive. | (required) | | `--schema`, `-s` | Schema the bank lives in. | base schema | | `--include-history` | Also export operational history (`audit_log`, `llm_requests`). | `false` | **Examples:** ```bash hindsight-admin export-bank --bank my-bank --output my-bank.zip # include operational history hindsight-admin export-bank --bank my-bank --output my-bank.zip --include-history ``` Read-only — safe to run against a live instance. --- ### import-bank Restore a whole-bank archive (produced by `export-bank`) into **this** instance. Facts are re-embedded with this instance's configured embedding model and links/indexes are rebuilt; bank configuration, mental models, directives, and webhooks are restored exactly. No LLM fact-extraction runs, and because a migration restores state, it does **not** fire webhooks or re-run consolidation. PostgreSQL only. ```bash hindsight-admin import-bank --archive [OPTIONS] ``` **Options:** | Option | Description | Default | |--------|-------------|---------| | `--archive`, `-a` | Path to the `.zip` produced by `export-bank`. | (required) | | `--schema`, `-s` | Target schema. | base schema | | `--target-bank` | Override the bank id (defaults to the archive's source bank). | source bank | | `--include-history` | Also restore history if present in the archive. | `false` | **Examples:** ```bash hindsight-admin import-bank --archive my-bank.zip ``` Run this against an instance configured with the **target** embedding model / vector extension / text-search backend — that's what re-embedding uses. :::warning Target bank must not exist Import restores a **whole bank** (config, facts, mental models, …) — it is **not a merge**. If a bank with the target id already exists, the command fails. Delete that bank first, or use `--target-bank` to restore under a fresh id. ::: --- ## Migrating a bank to a new instance Changing a bank's **embedding model** (e.g. a 384-dim encoder → a 1024-dim one), **vector extension** (pgvector / vchord / pgvectorscale), or **text-search backend** can't be done in place on a populated bank — the stored vectors and indexes are tied to those settings. Because every embedding and index is a deterministic function of text already on disk, the supported path is to **move the bank to a fresh instance configured with the new settings and re-derive everything there — with no LLM re-extraction**. `export-bank` / `import-bank` carry documents, facts, observations, bank config, mental models, directives, and webhooks — but never embeddings, which the target instance regenerates with its own model. **Blue-green runbook:** 1. Stand up a **new instance** on a fresh database, configured with the new embedding model / vector extension / text-search backend. 2. Quiesce writes to the source bank (maintenance window) and run `hindsight-admin backup` for safety. 3. Export from the source, then import into the target: ```bash # on the source instance: hindsight-admin export-bank --bank my-bank --output my-bank.zip # on the target instance (configured with the new settings): hindsight-admin import-bank --archive my-bank.zip ``` 4. Verify on the target: run representative recall queries and compare results. 5. Cut traffic over to the new instance. The old instance stays as an instant rollback until you're confident. :::note Why a new instance, not in-place The embedding model is server-level, and a bank's `memory_units.embedding` column has a single dimension shared across the schema, so a different-dimension or different-backend bank needs its own instance/database. The old vectors are never mutated, which makes rollback trivial. ::: --- ## Recovering stuck or zombie operations A "zombie" operation is one stuck in `processing` indefinitely because the worker that claimed it is gone. The most common cause is an unstable `HINDSIGHT_API_WORKER_ID`: when it defaults to the container hostname, a Docker restart produces a new container ID, the new worker doesn't recognize the old worker's claims as its own, and those tasks are stranded. **How to spot them:** ```bash # List processing tasks grouped by worker — workers with a growing last_update_ago are dead hindsight-admin worker-status # Bank-level counters; pending_consolidation that never decreases is the usual symptom curl -s http://localhost:8888/v1/default/banks//stats ``` **How to recover:** ```bash # You know which worker is dead (e.g. from worker-status): hindsight-admin decommission-worker # You don't know — release every processing task across the fleet: hindsight-admin decommission-workers ``` Both commands reset `processing` rows back to `pending` so a live worker can claim them on the next poll. **How to prevent it:** Set `HINDSIGHT_API_WORKER_ID` to a stable value so worker identity survives restarts: - **Docker**: pass `-e HINDSIGHT_API_WORKER_ID=hindsight-prod` (or per-replica names if running multiple containers) - **Kubernetes (Helm)**: the chart's StatefulSet uses the pod name automatically — no extra config needed - **Bare metal / pip**: pass `--worker-id ` or set the env var per process See [Installation - Docker](./installation#docker) and [Configuration - Distributed Workers](./configuration#distributed-workers). --- ## Environment Variables The admin CLI uses the same environment variables as the API service. The most important one is: | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) | **Example:** ```bash # Use a specific database export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@localhost:5432/hindsight hindsight-admin backup /backups/mybackup.zip ``` --- ### Hindsight Docs Docs Developer Configuration (`hindsight-docs/docs/developer/configuration.mdx`) import CodeSnippet from '@site/src/components/CodeSnippet'; import configurationPy from '!!raw-loader!@site/examples/api/configuration.py'; import configurationSh from '!!raw-loader!@site/examples/api/configuration.sh'; # Configuration Complete reference for configuring Hindsight services through environment variables. Hindsight has two services, each with its own configuration prefix: | Service | Prefix | Description | |---------|--------|-------------| | **API Service** | `HINDSIGHT_API_*` | Core memory engine | | **Control Plane** | `HINDSIGHT_CP_*` | Web UI | --- ## API Service The API service handles all memory operations (retain, recall, reflect). ### Database | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) | | `HINDSIGHT_API_READ_DATABASE_URL` | Optional read-replica PostgreSQL URL. When set, recall queries (semantic, BM25, graph, temporal) are routed through a separate connection pool against this URL, offloading the primary. Typically points to a read-only endpoint (e.g., CNPG's `-ro` service or Aurora reader endpoint). | Unset (uses primary) | | `HINDSIGHT_API_MIGRATION_DATABASE_URL` | Direct PostgreSQL URL for running migrations, bypassing connection poolers (e.g. PgBouncer). When set, advisory locks and Alembic migrations use this URL instead of `DATABASE_URL`. | Falls back to `DATABASE_URL` | | `HINDSIGHT_API_DATABASE_SCHEMA` | PostgreSQL schema name for tables | `public` | | `HINDSIGHT_API_RUN_MIGRATIONS_ON_STARTUP` | Run database migrations on API startup | `true` | | `HINDSIGHT_API_MIGRATION_ISOLATION` | Run migrations in a subprocess instead of in the calling process: `true` (keeps Alembic's import graph and its psycopg2 sync engine out of a long-lived server process) or `false` | `false` | | `HINDSIGHT_API_MIGRATION_CONCURRENCY` | Number of tenant schemas to migrate concurrently (PostgreSQL only). Each schema runs in its own process; within a schema migrations are always sequential. Each worker has a fixed startup cost (~1–2s to boot a fresh interpreter), so this only pays off with **many** schemas (roughly tens or more) or slow/high-latency migrations — for a handful of schemas it is slower than sequential. Each worker uses ~3 database connections, so keep `concurrency × 3` within your database's spare `max_connections` (and any PgBouncer pool limit). `1` = fully sequential. Measured at 20k schemas: the per-restart no-op resweep dropped from ~60min to ~11min (≈5×) at `concurrency=12`. | `1` | | `HINDSIGHT_API_EXTERNALLY_OWNED_ROUTINES` | Comma-separated list of maintenance discovery routines this deployment installs itself (see [Owning a maintenance routine](#owning-a-maintenance-routine)). Migrations skip anything named here. | Empty (every routine installed) | | `HINDSIGHT_API_DATABASE_BACKEND` | Database engine backend: `postgresql` or `oracle` (Oracle 23ai) | `postgresql` | If not provided, the server uses embedded `pg0` — convenient for development but not recommended for production. To tune the embedded database, add PostgreSQL settings as a query string: `pg0://hindsight?max_connections=300&shared_buffers=256MB`. They apply when the embedded database starts, so restart it after changing them. To run against Oracle Database 23ai instead, set `HINDSIGHT_API_DATABASE_BACKEND=oracle` and use an `oracle+oracledb://…` URL. See the [Oracle Database guide](./oracle) for full setup instructions. The `DATABASE_SCHEMA` setting allows you to use a custom PostgreSQL schema instead of the default `public` schema. This is useful for: - Multi-database setups where you want Hindsight tables in a dedicated schema - Hosting platforms (e.g., Supabase) where `public` schema is reserved or shared - Organizational preferences for schema naming conventions ```bash # Example: Using a custom schema export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/dbname export HINDSIGHT_API_DATABASE_SCHEMA=hindsight ``` Migrations will automatically create the schema if it doesn't exist and create all tables in the configured schema. #### Owning a maintenance routine The background maintenance loop finds work by calling four PostgreSQL routines — `mental_models_with_cron`, `banks_needing_consolidation`, `schemas_with_expired_rows` and `schemas_with_expired_operations`. The stock implementations scan every schema in the database on each tick. On a large multi-tenant install that is a serial loop over thousands of schemas, repeated once per tick in every process running the loop. Because they are called by name, you can replace one with an implementation suited to your installation — typically a registry table maintained by a row trigger, so discovery costs one indexed read instead of a scan. List the ones you have replaced and migrations will leave them alone: ```bash export HINDSIGHT_API_EXTERNALLY_OWNED_ROUTINES=mental_models_with_cron,banks_needing_consolidation ``` Read from the environment of whatever process runs the migration, so it applies equally to `hindsight-admin run-db-migration`, a migration Job, and migrate-on-startup. Without this, your replacement survives only until the next migration that reinstalls the routine: they are all `CREATE OR REPLACE` against the same name, so the migration wins and nothing records that a custom implementation was discarded. A few things to know: - **Your replacement must match the stock signature.** The loop calls these with fixed arguments and reads fixed columns back. - **Nothing verifies that a replacement exists.** Naming a routine you have not installed leaves whatever was there before — nothing, on a fresh database — and the maintenance loop then fails on the missing function rather than quietly running the scan you were trying to avoid. - **Ownership is not stamped into the database.** Remove a name from the list and the next migration reinstalls the stock routine, with no data fix-up. - **Empty by default.** An installation that has not replaced anything is unaffected. ### Database Connection Pool | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_DB_POOL_MIN_SIZE` | Minimum connections in the primary pool | `5` | | `HINDSIGHT_API_DB_POOL_MAX_SIZE` | Maximum connections in the primary pool | `100` | | `HINDSIGHT_API_READ_DB_POOL_MIN_SIZE` | Minimum connections in the read-replica pool (only used when `READ_DATABASE_URL` is set) | Falls back to `DB_POOL_MIN_SIZE` | | `HINDSIGHT_API_READ_DB_POOL_MAX_SIZE` | Maximum connections in the read-replica pool (only used when `READ_DATABASE_URL` is set) | Falls back to `DB_POOL_MAX_SIZE` | | `HINDSIGHT_API_DB_COMMAND_TIMEOUT` | PostgreSQL command timeout in seconds (asyncpg client-side) | `60` | | `HINDSIGHT_API_DB_ACQUIRE_TIMEOUT` | Connection acquisition timeout in seconds. Bounds how long a caller waits for a free pool connection before failing (retried by the caller); `0` waits indefinitely. | `30` | | `HINDSIGHT_API_DB_POOL_SLOW_ACQUIRE_THRESHOLD_SECONDS` | Log a `[DB POOL] Slow acquire` warning when getting a pool connection takes longer than this many seconds. Raise it when new connections are slow by nature (e.g. TLS to Aurora Serverless); `0` disables the warning. | `0.05` | | `HINDSIGHT_API_DB_STATEMENT_TIMEOUT` | Postgres `statement_timeout` applied to every pool connection, in seconds. Server-side safety net for runaway queries. Does **not** apply to Alembic migrations (which run on a separate psycopg2 engine). Set to `0` to disable. | `600` | | `HINDSIGHT_API_DB_MAX_PARALLEL_WORKERS_PER_GATHER` | Optional Postgres `max_parallel_workers_per_gather` applied to every pool connection of this process. Unset leaves the server default. Set to `0` on background-worker processes so bulk maintenance queries (consolidation, graph upkeep) run serially instead of fanning out across CPU cores shared with latency-sensitive traffic. | unset | | `HINDSIGHT_API_DB_SESSION_SETUP_ON_ACQUIRE` | Whether the per-connection session settings above (`statement_timeout`, `max_parallel_workers_per_gather`, the trigram threshold, the vector-search tuning, and — on the `vchord` text-search backend — the search path) are re-applied every time a connection is taken from the pool, not only when it is first opened. Keep this on unless the same settings are already pinned on the database role or the database itself (`ALTER ROLE … SET`), because behind a transaction-mode connection pooler an acquire can be linked to a server connection that never received them (on a direct connection they now survive, so the re-apply is redundant there) — with the re-apply off and nothing pinned server-side, reused connections quietly run without them, and on `vchord` recall fails outright rather than merely degrading. When they *are* pinned server-side the re-apply changes nothing and only costs a round trip per acquire, which is worth reclaiming on busy deployments behind a transaction-mode connection pooler. `application_name` is always re-applied and is unaffected by this setting. | `true` | | `HINDSIGHT_API_ENTITY_TRGM_SIMILARITY_THRESHOLD` | Postgres `pg_trgm.similarity_threshold` applied to every pool connection, governing how close a name must be for entity resolution's `%` trigram match to treat it as a candidate. Must be between `0` (exclusive) and `1`. Never applied below `HINDSIGHT_API_ENTITY_MERGE_MIN_SIMILARITY`, since a candidate under the merge floor can never be merged. Raise it above that floor to look at fewer, closer names. | `0.3` | | `HINDSIGHT_API_ENTITY_INTRABATCH_MERGE_SIMILARITY` | Trigram similarity (pg_trgm-equivalent, computed in-memory) at/above which two brand-new names created by the **same** retain are merged into a single entity (in-batch dedup of surface-form variants — e.g. the same name with different emoji/case/suffix). Must be between `0` (exclusive) and `1`. This is a *merge* cutoff, deliberately stricter than the recall-only threshold above; raise it toward `1.0` to merge only near-identical forms. | `0.5` | | `HINDSIGHT_API_ENTITY_MERGE_MIN_SIMILARITY` | Minimum trigram similarity a name must have with an **existing** entity before that entity can be reused for it, whatever the other resolution signals say. Sits between the recall threshold above (`0.15`, which only decides what is *considered*) and the same-batch fold-in cutoff below (`0.5`). Must be between `0` (exclusive) and `1`. Lower it for corpora of very short names, where trigram similarity is unavoidably low (`Jon`/`John` is `0.29`); raise it to merge only clear surface variants. | `0.3` | For high-concurrency workloads, increase `DB_POOL_MAX_SIZE`. Each concurrent recall/think operation can use 2-4 connections. To run migrations manually (e.g., before starting the API), use the admin CLI: ```bash # Migrate the base schema plus all discovered tenant schemas hindsight-admin run-db-migration # Or migrate a specific schema only: hindsight-admin run-db-migration --schema tenant_acme ``` ### Vector Extension | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_VECTOR_EXTENSION` | Vector index algorithm: `pgvector`, `vchord`, `pgvectorscale`, or `scann` | `pgvector` | | `HINDSIGHT_API_ANN_ITERATIVE_SCAN` | Let a vector index scan resume until the query's `LIMIT` is satisfied, instead of stopping when its first candidate list drains. With it off, a recall can never retrieve more rows than that list holds — on pgvector, `hnsw.ef_search` (200) — so a larger recall budget widens the SQL and retrieves nothing extra. Requires pgvector 0.8.0+; older servers reject the setting and it is dropped automatically after the first attempt. This is the operational kill switch: setting it to `false` and restarting restores the previous retrieval depth exactly, with no code change. | `true` | | `HINDSIGHT_API_ANN_MAX_SCAN_TUPLES` | Ceiling on how many tuples a single resumed scan may visit (`hnsw.max_scan_tuples`). This is the knob that governs what iterative scans cost: the filters that thin a result — the similarity floor, tags, date ranges — are applied *after* the index scan, so a selective query resumes repeatedly, and this bounds both the CPU it can spend and the memory it can hold (pgvector otherwise caps the latter at `work_mem × hnsw.scan_mem_multiplier`, which at this default is never approached). Lower it to trade retrieval depth back for latency; the initial scan is not counted, so even `1` leaves the pre-existing depth intact. pgvector's own default is `20000`. Ignored when iterative scans are off. | `4000` | Hindsight supports four PostgreSQL vector extensions: #### **pgvector** (HNSW - default) - In-memory index using Hierarchical Navigable Small World algorithm - Works well for most embeddings and dataset sizes - Fast for small-medium datasets (<10M vectors) - Higher memory usage for large datasets - Most widely deployed and supported #### **pgvectorscale** (DiskANN - recommended for scale) ⭐ - Disk-based index using StreamingDiskANN algorithm - **28x lower p95 latency** and **16x higher throughput** vs dedicated vector DBs - **60-75% cost reduction** at scale (SSDs cheaper than RAM) - Superior filtering performance with streaming retrieval model - Optimized for large datasets (10M+ vectors) - Supports both **pgvectorscale** (open source) and **pg_diskann** (Azure) - **Installation:** - Open source/self-hosted: `CREATE EXTENSION vector; CREATE EXTENSION vectorscale CASCADE;` - Azure PostgreSQL: `CREATE EXTENSION vector; CREATE EXTENSION pg_diskann CASCADE;` #### **vchord** (vchordrq) - Alternative high-performance vector index - Optimized for high-dimensional embeddings (3000+ dimensions) - Includes integrated BM25 search capabilities - Requires `vchord` extension #### **scann** (AlloyDB ScaNN) - Google's ScaNN index, available on **AlloyDB** and **AlloyDB Omni** - Uses a single global vector index in `AUTO` mode (per-bank partial indexes are not used) - **Installation:** `CREATE EXTENSION vector; CREATE EXTENSION alloydb_scann CASCADE;` - **Index build is deferred** until a table reaches **10,000 populated embedding rows** — AlloyDB cannot build a ScaNN AUTO index on a near-empty table. Until that threshold is crossed, recall falls back to a sequential scan; the global index is built on the next API startup once enough rows exist. - A ready-to-use Docker Compose stack is provided at [`docker/docker-compose/alloydb/docker-compose.yaml`](https://github.com/vectorize-io/hindsight/blob/main/docker/docker-compose/alloydb/docker-compose.yaml) for running Hindsight against AlloyDB Omni locally. **When to use pgvectorscale (DiskANN):** - Large datasets (10M+ vectors) ⭐ - Complex filtering requirements - Cost-sensitive deployments - Production workloads requiring high throughput - When disk I/O is not a bottleneck **When to use pgvector (HNSW):** - Small-medium datasets (<10M vectors) - Maximum query speed when all data fits in memory - Simple nearest-neighbor queries without filters - Standard PostgreSQL deployment preference **When to use vchord:** - High-dimensional embeddings (3000+ dimensions) - Want integrated BM25 search - Already using vchord for text search **When to use scann:** - Running on Google **AlloyDB** or **AlloyDB Omni** - Want managed ScaNN with `AUTO` mode tuning #### Limiting vector indexes on large deployments On `pgvector`, `pgvectorscale` and `vchord`, a bank's memories are indexed per `(bank, fact_type)`. By default every bank gets its own indexes when it is created, which is the right thing for most deployments and needs no configuration. It stops being the right thing when you have thousands of banks. These indexes all live on one shared table, and PostgreSQL inspects and locks **every** index on a table whenever it plans a query against it — so an index created for one bank is a cost paid by searches in every other bank. Past a few thousand banks, planning slows sharply and the server eventually runs out of lock-table space, failing reads *and* bank deletion alike. Setting a minimum size fixes that: a bank only gets its own indexes once it is large enough to benefit from them. Below the threshold PostgreSQL answers the same search from the `(bank_id, fact_type)` B-tree plus a top-N sort, which for a small bank is both faster and *exact* rather than approximate — so small banks lose nothing. The number of indexes becomes proportional to the number of **large** banks rather than to the number of banks, and bank count stops being a limit. | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_VECTOR_INDEX_MIN_ROWS` | Memories a bank needs, in one fact type, before that fact type gets its own vector index. `0` (the default) turns the threshold **off**: every bank is indexed from the moment it is created. `10000` is a good starting point for deployments with thousands of banks. A bank whose memories a custom store owns is outside this at any value — it has no rows here, so it is never given a per-bank index, and never has an existing one taken away (see `repair-bank` in the [admin CLI](./admin-cli.md)). | `0` | | `HINDSIGHT_API_VECTOR_INDEX_MAINTENANCE_MIN_INTERVAL_SECONDS` | Shortest gap between two index-maintenance runs for one bank. Stops a bank whose size hovers at the threshold from building and dropping the same index repeatedly. Unused while the threshold is off. | `900` | **With the threshold off (the default),** indexes are created inside the transaction that creates the bank — instantly, because the bank is empty — and dropped when the bank is deleted. Nothing inspects bank sizes, and no background operation runs. **With a threshold set,** bank creation, ingestion and import build no indexes at all, so no request ever waits on index DDL. Instead, a write that could move a bank across the threshold queues a background `vector_index_maintenance` operation, which builds an index when a bank grows past the threshold and removes it if the bank shrinks well below it (the gap between those two points prevents churn at the boundary). You will see these operations in the bank's operations list. To reconcile without waiting for a write — after a restore, an upgrade, or an extension switch — run: ```bash hindsight-admin repair-bank --all ``` This works in both modes: with the threshold off it rebuilds any index a bank is missing, and with one set it also drops what a bank no longer earns. **Switching extensions:** If you need to switch from one extension to another: 1. Set `HINDSIGHT_API_VECTOR_EXTENSION` to your desired extension (`pgvector`, `vchord`, `pgvectorscale`, or `scann`) 2. If your database has existing data, you'll get an error with migration instructions (note: switching **to** `scann` is allowed even with data — the existing index is dropped and rebuilt as ScaNN once the table has at least 10,000 embedding rows) 3. For empty databases, indexes will be automatically recreated on startup **Learn more:** - [HNSW vs. DiskANN comparison](https://www.tigerdata.com/learn/hnsw-vs-diskann) - [pgvectorscale GitHub](https://github.com/timescale/pgvectorscale) ### Text Search Extension | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_TEXT_SEARCH_EXTENSION` | Text search backend: `native`, `vchord`, `pg_textsearch`, `pgroonga`, or `pg_search` | `native` | | `HINDSIGHT_API_TEXT_SEARCH_EXTENSION_NATIVE_LANGUAGE` | PostgreSQL text search dictionary used by the `native` backend (e.g. `english`, `french`, `simple`, `zhparser`) | `english` | | `HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER` | ParadeDB `pg_search` tokenizer used when creating BM25 indexes. Empty uses ParadeDB's default tokenizer (`unicode_words`). | unset | | `HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_FUNCTION_SCHEMA` | Schema containing ParadeDB `pg_search` functions (`score`, `boolean`, `match`). Default is `paradedb`; set to `pgsearch` (or another valid identifier) on managed PostgreSQL distributions that install functions under a custom schema. | `paradedb` | | `HINDSIGHT_API_BM25_MAX_QUERY_TERMS` | Cap on the number of terms in the native PostgreSQL BM25 `tsquery`. Long queries OR-join every normalized token, and because native ranking has no IDF and re-ranks every match, an uncapped query over common terms can scan and rank a large fraction of the bank and time out. When the query exceeds the cap, the most **selective** terms are kept — those with the lowest tenant-wide document frequency, read for free from `pg_stats` (autovacuum-maintained, no reindex) — so a blunt truncation doesn't discard the high-signal terms. `0` restores the historical uncapped behavior; the cap bounds only the `native` backend (other BM25 backends receive the raw query and rank via their own index-native operators). | `16` | | `HINDSIGHT_API_BM25_SELECTIVE_TERMS` | When the cap above trims a query, whether to keep the most **selective** terms (lowest document frequency, from `pg_stats`) rather than the first N. `true` is strictly better for recall at no extra cost when stats exist; set `false` to opt out of the catalog read and cap by position. Ignored when the cap is `0`. | `true` | | `HINDSIGHT_API_LLM_OUTPUT_LANGUAGE` | When set, forces every LLM-generated artifact (retain facts, consolidation observations, reflect responses) into this language. Free-form (e.g. `Spanish`, `Japanese`). | unset | Hindsight supports five backends for BM25 keyword retrieval: - **native** — PostgreSQL's built-in full-text search (`tsvector` + GIN). Language configurable. - **vchord** — VectorChord BM25 (uses the `llmlingua2` multilingual tokenizer). - **pg_textsearch** — Timescale's pg_textsearch extension. English-only. - **pgroonga** — pgroonga full-text search. Multilingual / CJK out of the box. - **pg_search** — ParadeDB pg_search. True BM25; the only backend that is Citus-compatible. A bank can also use none of them: [`enable_text_search`](#recall-pipeline-stages) switches the keyword arm off per bank, leaving pure vector search. Every setting in this table is then unused by that bank. To switch backends: set `HINDSIGHT_API_TEXT_SEARCH_EXTENSION`. With existing data, you'll get an error and migration instructions; with an empty database the columns/indexes are recreated automatically on startup. `HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER` only applies when `HINDSIGHT_API_TEXT_SEARCH_EXTENSION=pg_search`, and only when BM25 indexes are created. Changing it for an existing database requires rebuilding the `pg_search` indexes or recreating the database. Supported values are empty/unset, `unicode_words`, `simple`, `whitespace`, `literal`, `literal_normalized`, `chinese_compatible`, `icu`, `jieba`, `source_code`, `chinese_lindera`/`lindera(chinese)`, `japanese_lindera`/`lindera(japanese)`, `korean_lindera`/`lindera(korean)`, `ngram(min,max)`, and `edge_ngram(min,max)`. For non-English banks (especially CJK) and the language/extraction-language tradeoffs, see the [Multilingual Support](./multilingual) page. ### LLM Provider | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_LLM_PROVIDER` | Provider: `openai`, `openai-responses`, `openai-codex`, `chatgpt`, `claude-code`, `cursor`, `github-copilot`, `anthropic`, `gemini`, `groq`, `minimax`, `deepseek`, `zai`, `opencode-go`, `meta`, `nous`, `xai-oauth`, `fireworks`, `ollama`, `ollama-cloud`, `lmstudio`, `llamacpp`, `vertexai`, `bedrock`, `litellm`, `litellmrouter`, `volcano`, `openrouter`, `requesty`, `none` | `openai` | | `HINDSIGHT_API_LLM_API_KEY` | API key for providers that require one; unused by `github-copilot`. Optional for `anthropic` when [Workload Identity Federation](https://platform.claude.com/docs/en/manage-claude/workload-identity-federation) is configured (see the Anthropic example below) | - | | `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-5-mini` | | `HINDSIGHT_API_LLM_BASE_URL` | Custom LLM endpoint | Provider default | | `HINDSIGHT_API_LLM_MAX_CONCURRENT` | Max concurrent LLM requests. When the cap is full, live API requests get the next free slot ahead of background jobs (async retain, consolidation) run by the in-process worker. | `32` | | `HINDSIGHT_API_LLM_MAX_RETRIES` | Max retry attempts for LLM API calls | `3` | | `HINDSIGHT_API_LLM_INITIAL_BACKOFF` | Initial retry backoff in seconds (exponential backoff) | `1.0` | | `HINDSIGHT_API_LLM_MAX_BACKOFF` | Max retry backoff cap in seconds | `60.0` | | `HINDSIGHT_API_LLM_TIMEOUT` | LLM request timeout in seconds. Honoured by every provider as a **total** deadline for one request, including the time spent reading a streamed response — a backend that keeps sending bytes is cut off at the deadline rather than read indefinitely. Raise it if your provider is legitimately slow; the per-operation variables below override it. | `120` | | `HINDSIGHT_API_LLM_CONNECT_TIMEOUT` | Ceiling on the **connect** phase (TCP + TLS handshake) of an LLM request, in seconds. The read, write and pool phases keep the full `HINDSIGHT_API_LLM_TIMEOUT` budget; capping connect separately means an unreachable or wedged endpoint fails in seconds instead of consuming the whole request timeout. Effective value is the smaller of this and the request timeout. Set to `0` to disable the cap and use one value across all four phases. Applies to every provider whose HTTP client Hindsight constructs (all OpenAI-compatible backends, `openai-responses`, `chatgpt`, `anthropic`, `openai-codex`, `xai-oauth`, `fireworks`, `nous`, `llamacpp`); `litellm`/`litellmrouter` and `gemini`/`vertexai` own their transports and take a single total deadline instead, and `claude-code`/`cursor`/`github-copilot` drive a CLI rather than a socket. | `10` | | `HINDSIGHT_API_LLM_HTTP_LOG_LEVEL` | Log level for the `httpx` and `httpcore` loggers. Set to `DEBUG` to trace each LLM request through its transport phases (`connect_tcp`, `send_request_headers`, `receive_response_headers`) — the way to tell a request that stalled before it was ever sent from one that was sent and never answered. Verbose; for diagnosis only. | `WARNING` | | `HINDSIGHT_API_LLM_REASONING_EFFORT` | Reasoning effort for providers/models that support it (for example `none`, `low`, `medium`, `high`, `xhigh`). Set it and the value is sent as given, whatever your model is called — which is how you control thinking-token volume on a self-hosted reasoning model (vLLM, Ollama, llama.cpp, TGI), where `none` is often the only value that removes the thinking block. Leave it unset and no reasoning parameter is sent at all, so each model runs at its own default effort. Honoured by `openai` and every OpenAI-compatible provider, `openai-responses`, `chatgpt`, `openai-codex`, `xai`, `llamacpp`, and `litellm`/`litellmrouter` (which translate it per target provider). The native `gemini`/`vertexai`, `anthropic`, `claude-code` and `cursor` providers have no reasoning-effort control and log a warning at startup if you set one — reach those models through `litellm` to apply it. | Unset (model's own default) | | `HINDSIGHT_API_LLM_TEMPERATURE` | Global override for the sampling temperature of internal LLM calls. Set a number in `[0.0, 2.0]`, or `none` (also `default`/`off`/empty) to **omit** the temperature parameter entirely — required for models that reject explicit temperatures, e.g. Azure `gpt-5.5`, which only accepts its default value. Per-operation variables below override this. | Per-operation defaults | | `HINDSIGHT_API_LLM_TEMPERATURE_VERIFICATION` | Temperature for the startup connection check. Number in `[0.0, 2.0]` or `none` to omit. Overrides `HINDSIGHT_API_LLM_TEMPERATURE`. | `0.0` | | `HINDSIGHT_API_LLM_TEMPERATURE_RETAIN` | Temperature for fact extraction during retain. Number in `[0.0, 2.0]` or `none` to omit. Overrides `HINDSIGHT_API_LLM_TEMPERATURE`. | `0.1` | | `HINDSIGHT_API_LLM_TEMPERATURE_REFLECT` | Temperature for the reflect "thinking" step. Number in `[0.0, 2.0]` or `none` to omit. Overrides `HINDSIGHT_API_LLM_TEMPERATURE`. | `0.9` | | `HINDSIGHT_API_LLM_TEMPERATURE_CONSOLIDATION` | Temperature for consolidation (dedup), and for the mental-model refresh's delta and retraction calls unless `HINDSIGHT_API_LLM_TEMPERATURE_MENTAL_MODEL_REFRESH` is set. Number in `[0.0, 2.0]` or `none` to omit. Overrides `HINDSIGHT_API_LLM_TEMPERATURE`. | `0.0` | | `HINDSIGHT_API_LLM_TEMPERATURE_MENTAL_MODEL_REFRESH` | Temperature for the mental-model refresh's structured edit calls (delta and retraction). Number in `[0.0, 2.0]` or `none` to omit. Set it when the refresh runs its own model (`HINDSIGHT_API_MENTAL_MODEL_REFRESH_LLM_*`) whose recommended temperature differs from consolidation's. | Falls back to `HINDSIGHT_API_LLM_TEMPERATURE_CONSOLIDATION` | | `HINDSIGHT_API_LLM_VISION` | Whether the configured LLM can read images, overriding what the provider reports about itself. Leave unset and each provider answers for itself: Anthropic and Gemini report yes, OpenAI reports yes, `none` reports no, and every gateway-style backend (LiteLLM, Ollama, LM Studio, OpenRouter, an OpenAI-compatible proxy) reports *unknown*, because its catalogue mixes vision-capable and text-only models. A retain carrying [inline attachments](#inline-attachments-in-retain) is refused with `422` on both "no" and "unknown" — dropping an attachment silently would leave a document that looks retained with the information the caller cared about gone. Set `true` when you are running a vision model behind such a gateway; set `false` to refuse images against an endpoint that rejects them despite its model name. | Provider decides | | `HINDSIGHT_API_VLM_PROVIDER` | Provider for the **vision slot** — the model used only for retain chunks that carry an [inline attachment](#inline-attachments-in-retain). Every text-only chunk keeps using the retain LLM, so a bank whose documents mostly *are* text does not pay a vision model's price for all of it. Unset, attachments go to the retain LLM. | Retain LLM | | `HINDSIGHT_API_VLM_API_KEY` | API key for the vision slot. | Retain LLM's | | `HINDSIGHT_API_VLM_MODEL` | Model for the vision slot. This is the model `HINDSIGHT_API_LLM_VISION` and the `422` vision check apply to, since it is the one that will actually be handed the attachment. | Retain LLM's | | `HINDSIGHT_API_VLM_BASE_URL` | Base URL for the vision slot. Follows `HINDSIGHT_API_VLM_PROVIDER`'s own default when that is set and this is not — it never inherits the retain provider's host, which would send the request to the wrong endpoint with the wrong key. | Provider default | | `HINDSIGHT_API_LLM_SEND_BANK_AS_USER` | Tag outbound LLM and embedding calls with `user=` so gateways (OpenRouter usage accounting, LiteLLM, Helicone) can attribute spend per bank. When enabled, the bank id is transmitted to the upstream provider as the end-user identifier. | `false` | | `HINDSIGHT_API_LLM_GROQ_SERVICE_TIER` | Groq service tier: `on_demand`, `flex`, `auto` | `auto` | | `HINDSIGHT_API_LLM_OPENAI_SERVICE_TIER` | OpenAI service tier: `flex` for 50% cost savings (OpenAI Flex Processing) | None (default) | | `HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER` | Bedrock service tier: `flex` for 50% cost savings (best-effort inference), `priority` (guaranteed throughput), or `reserved` (provisioned capacity) | Unset (default tier) | | `HINDSIGHT_API_LLM_GEMINI_SERVICE_TIER` | Gemini service tier: `flex` for 50% cost savings (best-effort inference) | Unset (default tier) | | `HINDSIGHT_API_LLM_EXTRA_BODY` | JSON dict of extra request-body params (e.g. `temperature`, `top_p`, `max_tokens`) merged into every LLM call. Applied across the OpenAI-compatible, Fireworks, Anthropic, Gemini/VertexAI and LiteLLM (incl. Bedrock/Router) providers. Each provider merges them in its own native parameter space, so use that provider's field names (e.g. `max_tokens` for OpenAI/Anthropic vs `max_output_tokens` for Gemini). The native Ollama structured-output path is a special case — see the note below. Also useful for custom model servers (e.g. vLLM `chat_template_kwargs`). | `null` | | `HINDSIGHT_API_LLM_DEFAULT_HEADERS` | JSON dict passed as `default_headers` to provider SDK clients. Used by operators routing through proxies / request-tracing middleware (e.g. Cloudflare AI Gateway, Helicone, corporate proxies). Wired into the Anthropic provider, the OpenAI-compatible providers, Fireworks, Nous, the OpenAI Responses API, and the LiteLLM-backed providers (`litellm`, `litellmrouter`, `bedrock`); other providers can opt in. | `null` | | `HINDSIGHT_API_LLM_CACHE_AFFINITY` | Pin a conversation to one backend prompt cache on the OpenAI-compatible providers, Fireworks and Nous. Server-side prompt caches are per backend server, so the same conversation has to reach the same one for a cache hit. `xai_conv_id` sends xAI's documented `x-grok-conv-id` header; `openai_prompt_cache_key` sends OpenAI's `prompt_cache_key` field; `auto` picks one from the configured base URL host (x.ai / grok.com → `xai_conv_id`; native OpenAI or an openai.com host → `openai_prompt_cache_key`; anything else, including Azure OpenAI, → `none`); `none` sends nothing. The id is the operation's trace id, so every LLM call of one retain/reflect/consolidation run shares it. `auto` is an allowlist rather than a best-effort probe, so an unknown OpenAI-compatible backend (vLLM, ollama, groq, openrouter, a custom proxy) receives byte-identical requests to before; set `none` to disable entirely. Anthropic and Gemini/Vertex do their own caching and ignore this. | `auto` | | `HINDSIGHT_API_LLM_STRICT_SCHEMA` | Grammar-enforce structured output via `json_schema` `strict: true` instead of the soft "schema-in-prompt + `json_object`" path. Typed Pydantic response models are serialized directly into the OpenAI strict subset: every object rejects additional properties, every declared property is required, and nullable fields remain nullable. Use it with weaker self-hosted models that return prose preambles, markdown ` ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Groq (recommended for fast inference) export HINDSIGHT_API_LLM_PROVIDER=groq export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx export HINDSIGHT_API_LLM_MODEL=openai/gpt-oss-20b # For free tier users: override to on_demand if you get service_tier errors # export HINDSIGHT_API_LLM_GROQ_SERVICE_TIER=on_demand # OpenAI export HINDSIGHT_API_LLM_PROVIDER=openai export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx export HINDSIGHT_API_LLM_MODEL=gpt-4o # Optional: Use Flex Processing for 50% cost savings (with variable latency) # export HINDSIGHT_API_LLM_OPENAI_SERVICE_TIER=flex # OpenAI Responses API (/v1/responses) export HINDSIGHT_API_LLM_PROVIDER=openai-responses export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx export HINDSIGHT_API_LLM_MODEL=gpt-5.6 # Uses the OpenAI Responses API instead of chat/completions. Unlike # chat/completions, it supports reasoning together with function tools, so # reflect's tool-calling search loop can run with a real reasoning effort. # Recommended for reasoning models (gpt-5.x, o-series) that reject # `reasoning_effort` alongside tools on chat/completions (e.g. gpt-5.6-terra). # export HINDSIGHT_API_LLM_REASONING_EFFORT=high # Gemini export HINDSIGHT_API_LLM_PROVIDER=gemini export HINDSIGHT_API_LLM_API_KEY=xxxxxxxxxxxx export HINDSIGHT_API_LLM_MODEL=gemini-2.0-flash # Optional: Use Gemini Flex for 50% cost savings (best-effort inference) # export HINDSIGHT_API_LLM_GEMINI_SERVICE_TIER=flex # Anthropic export HINDSIGHT_API_LLM_PROVIDER=anthropic export HINDSIGHT_API_LLM_API_KEY=sk-ant-xxxxxxxxxxxx export HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514 # Or, with no static key: Workload Identity Federation. Leave HINDSIGHT_API_LLM_API_KEY # unset and the Anthropic SDK trades the platform identity token for short-lived # access tokens, refreshing them on its own. # export ANTHROPIC_FEDERATION_RULE_ID=fdrl_xxxxxxxxxxxx # export ANTHROPIC_ORGANIZATION_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # export ANTHROPIC_SERVICE_ACCOUNT_ID=svac_xxxxxxxxxxxx # export ANTHROPIC_IDENTITY_TOKEN_FILE=/var/run/secrets/anthropic/token # e.g. a projected k8s service-account token # Vertex AI (Google Cloud - uses native genai SDK) export HINDSIGHT_API_LLM_PROVIDER=vertexai export HINDSIGHT_API_LLM_MODEL=gemini-2.0-flash-001 export HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=your-gcp-project-id export HINDSIGHT_API_LLM_VERTEXAI_REGION=us-central1 # Optional: use ADC (gcloud auth application-default login) or provide service account key: # export HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json # Ollama (local, no API key) export HINDSIGHT_API_LLM_PROVIDER=ollama export HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1 export HINDSIGHT_API_LLM_MODEL=llama3 # Ollama Cloud (hosted Ollama endpoint, requires API key) export HINDSIGHT_API_LLM_PROVIDER=ollama-cloud export HINDSIGHT_API_LLM_API_KEY=your-ollama-cloud-api-key export HINDSIGHT_API_LLM_MODEL=gemma3:12b # LM Studio (local, no API key) export HINDSIGHT_API_LLM_PROVIDER=lmstudio export HINDSIGHT_API_LLM_BASE_URL=http://localhost:1234/v1 export HINDSIGHT_API_LLM_MODEL=your-local-model # llama.cpp (built-in local inference, no external server needed) export HINDSIGHT_API_LLM_PROVIDER=llamacpp # No API key, base URL, or external server required. # Auto-downloads Gemma 4 E2B (~3.5 GB GGUF) on first run. # See "Built-in llama.cpp" section below for all configuration options. # OpenAI-compatible endpoint (Chat Completions API, /v1/chat/completions) export HINDSIGHT_API_LLM_PROVIDER=openai export HINDSIGHT_API_LLM_BASE_URL=https://your-endpoint.com/v1 export HINDSIGHT_API_LLM_API_KEY=your-api-key export HINDSIGHT_API_LLM_MODEL=your-model-name # OpenAI-compatible endpoint (Responses API, /v1/responses) # Same as above but targets /v1/responses instead of /v1/chat/completions. export HINDSIGHT_API_LLM_PROVIDER=openai-responses export HINDSIGHT_API_LLM_BASE_URL=https://your-endpoint.com/v1 export HINDSIGHT_API_LLM_API_KEY=your-api-key export HINDSIGHT_API_LLM_MODEL=your-model-name # OpenAI Codex (ChatGPT Plus/Pro subscription - uses OAuth, no API key needed) export HINDSIGHT_API_LLM_PROVIDER=openai-codex export HINDSIGHT_API_LLM_MODEL=gpt-5.6-luna # No API key needed - uses OAuth tokens from ~/.codex/auth.json # For long-running services, set CODEX_HOME to a dedicated auth directory so # Hindsight doesn't share (and lose) its refresh token with another Codex process. # See Models docs → "Isolating Codex auth for long-running services". # export CODEX_HOME=/var/lib/hindsight/codex # Per-provider override of CODEX_HOME. Set it (with the indexed # HINDSIGHT_API_LLM__CODEX_HOME below) to run two independently authorized # ChatGPT profiles in one process. # export HINDSIGHT_API_LLM_CODEX_HOME=/var/lib/hindsight/codex-a # Claude Code (Claude Pro/Max subscription - uses OAuth) export HINDSIGHT_API_LLM_PROVIDER=claude-code export HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-5-20250929 # No API key needed - uses `claude auth login` credentials. # Set HINDSIGHT_API_LLM_API_KEY to a token from `claude setup-token` to authenticate with a key instead. # Cursor (Cursor subscription - drives the cursor-agent CLI in headless mode) export HINDSIGHT_API_LLM_PROVIDER=cursor export HINDSIGHT_API_LLM_MODEL=auto # or any id from `cursor-agent --list-models` # No API key needed - uses `cursor-agent login` credentials. # Set HINDSIGHT_API_LLM_API_KEY (or CURSOR_API_KEY) to authenticate with a key instead. # An agent CLI turn takes 15-30s, well past the 30s reflect default, so raise the # timeouts or reflect burns its first iteration on a timeout retry. export HINDSIGHT_API_LLM_TIMEOUT=300 export HINDSIGHT_API_REFLECT_LLM_TIMEOUT=180 # The CLI exposes no response-format or tool-definition flags, so Hindsight emulates # structured output and tool calling through the prompt and parses the JSON back. That # is less reliable than a native schema: prefer a stronger named model over `auto` for # reflect, which is the operation that leans hardest on tool calls. # Volcano Engine (ByteDance - OpenAI-compatible) export HINDSIGHT_API_LLM_PROVIDER=volcano export HINDSIGHT_API_LLM_API_KEY=your-api-key export HINDSIGHT_API_LLM_BASE_URL=https://ark.cn-beijing.volces.com/api/v3 export HINDSIGHT_API_LLM_MODEL=doubao-pro-32k # OpenRouter (OpenAI-compatible, access 100+ models) export HINDSIGHT_API_LLM_PROVIDER=openrouter export HINDSIGHT_API_LLM_API_KEY=your-openrouter-api-key export HINDSIGHT_API_LLM_MODEL=qwen/qwen3.5-9b # Structured-output calls (fact extraction, consolidation) send # provider.require_parameters=true, so OpenRouter only routes them to upstreams # that support response_format. Any "provider" keys you set in # HINDSIGHT_API_LLM_EXTRA_BODY (e.g. "only") are kept and take precedence. # That also means every parameter sent must be supported, temperature included. # Models that reject temperature (e.g. anthropic/claude-haiku-5.5) then fail with # "No endpoints found that can handle the requested parameters". Fix: omit it. # export HINDSIGHT_API_LLM_TEMPERATURE=none # Reflect forces tool calls: tool_choice "required" after its first searches and a # named tool on the forced steps. Some upstreams accept only "auto" and fail those # turns with "No endpoints found that support the provided 'tool_choice' value" # (reported for several gpt-oss-120b hosts, #5832). Pin ones that support it with # HINDSIGHT_API_LLM_EXTRA_BODY='{"provider": {"only": ["deepinfra", "cerebras"]}}'. # Requesty (OpenAI-compatible gateway) export HINDSIGHT_API_LLM_PROVIDER=requesty export HINDSIGHT_API_LLM_API_KEY=your-requesty-api-key export HINDSIGHT_API_LLM_MODEL=openai/gpt-4o-mini # DeepSeek (OpenAI-compatible, https://api.deepseek.com) export HINDSIGHT_API_LLM_PROVIDER=deepseek export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx export HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash # Notes: # - `deepseek-v4-flash` defaults to thinking mode at the API level (treated as # `deepseek-reasoner`). Hindsight handles this transparently; the reflect # agent will not crash with "deepseek-reasoner does not support this # tool_choice". # - Use `deepseek-v4-pro` for the higher-quality reasoning route. # - Use `deepseek-chat` for the non-thinking alias (faster, cheaper). # z.ai (Zhipu GLM series, OpenAI-compatible, https://z.ai) export HINDSIGHT_API_LLM_PROVIDER=zai export HINDSIGHT_API_LLM_API_KEY=your-zai-api-key export HINDSIGHT_API_LLM_MODEL=glm-4.5-flash # or glm-4.5-air for the paid tier # Default base_url: https://api.z.ai/api/coding/paas/v4 (override with HINDSIGHT_API_LLM_BASE_URL if needed) # opencode-go (OpenAI-compatible) export HINDSIGHT_API_LLM_PROVIDER=opencode-go export HINDSIGHT_API_LLM_API_KEY=your-opencode-go-api-key export HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash # Default base_url: https://opencode.ai/zen/go/v1 (override with HINDSIGHT_API_LLM_BASE_URL if needed) # Meta Model API (Muse Spark, OpenAI-compatible, https://ai.developer.meta.com) export HINDSIGHT_API_LLM_PROVIDER=meta export HINDSIGHT_API_LLM_API_KEY=your-meta-model-api-key export HINDSIGHT_API_LLM_MODEL=muse-spark-1.3 # Default base_url: https://api.meta.ai/v1 (override with HINDSIGHT_API_LLM_BASE_URL if needed) # Muse Spark always reasons: leave HINDSIGHT_API_LLM_REASONING_EFFORT unset or set it to # minimal/low/medium/high/xhigh. "none" is rejected with HTTP 400. Reasoning tokens are # billed against the output budget, so keep the per-operation max-token limits generous. # Muse Spark reasons before every reply, so calls are slow: reflect's 30s default # deadline is not enough for its final synthesis and the call fails after retries. # Raise it (and the global deadline) when using this provider: export HINDSIGHT_API_REFLECT_LLM_TIMEOUT=300 export HINDSIGHT_API_LLM_TIMEOUT=300 # Nous Portal (OpenAI-compatible; no API key — uses your `hermes portal` login) export HINDSIGHT_API_LLM_PROVIDER=nous export HINDSIGHT_API_LLM_MODEL=deepseek/deepseek-v4-flash # No API key needed — reads a rotating JWT from ~/.hermes/auth.json (run `hermes portal` first). # Default base_url: https://inference-api.nousresearch.com/v1 (override with HINDSIGHT_API_LLM_BASE_URL if needed) # See the "Nous Portal Setup" section in the Models guide for the login flow. # SuperGrok subscription (device-code OAuth; no API key). This is the subscription # lane, NOT xAI API support — for API-key access to api.x.ai use # HINDSIGHT_API_LLM_PROVIDER=openai with HINDSIGHT_API_LLM_BASE_URL=https://api.x.ai/v1. export HINDSIGHT_API_LLM_PROVIDER=xai-oauth export HINDSIGHT_API_LLM_MODEL=grok-4.5 # No API key needed — reads an OAuth grant from ~/.hindsight/xai_oauth.json. # Log in once: python -m hindsight_api.engine.providers.xai_oauth_auth login # Default base_url: https://api.x.ai/v1 # See the "SuperGrok Subscription Setup" section in the Models guide for the login flow. # AWS Bedrock (native support - no API key needed, uses AWS credentials) export HINDSIGHT_API_LLM_PROVIDER=bedrock export HINDSIGHT_API_LLM_MODEL=us.amazon.nova-2-lite-v1:0 export AWS_ACCESS_KEY_ID=your-access-key export AWS_SECRET_ACCESS_KEY=your-secret-key export AWS_REGION_NAME=us-east-1 # Optional: Use Flex tier for 50% cost savings (with variable latency) # export HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER=flex # LiteLLM (100+ providers via LiteLLM SDK) # Azure OpenAI via LiteLLM export HINDSIGHT_API_LLM_PROVIDER=litellm export HINDSIGHT_API_LLM_API_KEY=your-azure-api-key export HINDSIGHT_API_LLM_MODEL=azure/gpt-4o # Together AI via LiteLLM export HINDSIGHT_API_LLM_PROVIDER=litellm export HINDSIGHT_API_LLM_API_KEY=your-together-api-key export HINDSIGHT_API_LLM_MODEL=together_ai/meta-llama/Llama-3-70b-chat-hf # No LLM (chunk storage + semantic search only, no API key needed) export HINDSIGHT_API_LLM_PROVIDER=none # Retain automatically uses chunks mode (no fact extraction) # Recall works normally (semantic search, BM25, graph retrieval) # Reflect returns HTTP 400 (requires an LLM) # Consolidation/observations are disabled ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash export HINDSIGHT_API_LLM_PROVIDER=litellmrouter export HINDSIGHT_API_LLM_LITELLMROUTER_CONFIG='{ "model_list": [ {"model_name": "default", "litellm_params": {"model": "openai/gpt-4o-mini", "api_key": "sk-..."}}, {"model_name": "fallback", "litellm_params": {"model": "anthropic/claude-sonnet-4-5", "api_key": "sk-ant-..."}} ], "fallbacks": [{"default": ["fallback"]}] }' ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Primary OpenAI, failover to Groq then Anthropic export HINDSIGHT_API_LLM_PROVIDER=openai export HINDSIGHT_API_LLM_API_KEY=sk-... export HINDSIGHT_API_LLM_1_PROVIDER=groq export HINDSIGHT_API_LLM_1_API_KEY=gsk-... export HINDSIGHT_API_LLM_2_PROVIDER=anthropic export HINDSIGHT_API_LLM_2_API_KEY=sk-ant-... export HINDSIGHT_API_LLM_STRATEGY='{"mode": "failover"}' # Weighted round-robin: serve the primary 3x as often as member 1 export HINDSIGHT_API_LLM_STRATEGY='{"mode": "round-robin", "weights": [3, 1]}' # Failover, re-probing a failed member after 5 minutes (then 10, 20, ... up to 1h) export HINDSIGHT_API_LLM_STRATEGY='{"mode": "failover", "cooldown_seconds": 300}' ``` #### Metadata routing `{"mode": "metadata"}` sends each retained item to the chain member its own metadata selects, so one deployment can extract different documents with different models: ```bash export HINDSIGHT_API_LLM_PROVIDER=openai export HINDSIGHT_API_LLM_API_KEY=sk-... export HINDSIGHT_API_LLM_1_PROVIDER=ollama export HINDSIGHT_API_LLM_1_MODEL=qwen3:8b export HINDSIGHT_API_LLM_STRATEGY='{ "mode": "metadata", "routes": [{"key": "classification", "value": "sensitive", "member": 1}] }' ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Minimal setup (auto-downloads model, uses GPU) export HINDSIGHT_API_LLM_PROVIDER=llamacpp # Custom model with tuning export HINDSIGHT_API_LLM_PROVIDER=llamacpp export HINDSIGHT_API_LLM_MAX_CONCURRENT=2 export HINDSIGHT_API_LLAMACPP_MODEL_PATH=~/.hindsight/models/my-model.gguf export HINDSIGHT_API_LLAMACPP_CONTEXT_SIZE=16384 export HINDSIGHT_API_LLAMACPP_NO_GRAMMAR=true # faster, less reliable JSON export HINDSIGHT_API_LLAMACPP_EXTRA_ARGS="--n_threads 8" ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Default LLM (used as fallback) export HINDSIGHT_API_LLM_PROVIDER=openai export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx export HINDSIGHT_API_LLM_MODEL=gpt-4o # Use GPT-4o for retain (strong structured output) export HINDSIGHT_API_RETAIN_LLM_MODEL=gpt-4o # Use faster/cheaper model for reflect export HINDSIGHT_API_REFLECT_LLM_PROVIDER=groq export HINDSIGHT_API_REFLECT_LLM_API_KEY=gsk_xxxxxxxxxxxx export HINDSIGHT_API_REFLECT_LLM_MODEL=llama-3.3-70b-versatile ``` **Example: Tuning Retry Behavior for Rate-Limited APIs** ```bash # For Anthropic with tight rate limits (10k output tokens/minute) export HINDSIGHT_API_LLM_PROVIDER=anthropic export HINDSIGHT_API_LLM_API_KEY=sk-ant-xxxxxxxxxxxx export HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514 # Reduce concurrent requests for retain to avoid rate limits export HINDSIGHT_API_RETAIN_LLM_MAX_CONCURRENT=3 # Fail faster with fewer retries export HINDSIGHT_API_RETAIN_LLM_MAX_RETRIES=3 # Or increase backoff times to wait out rate limit windows export HINDSIGHT_API_RETAIN_LLM_INITIAL_BACKOFF=2.0 # Start at 2s instead of 1s export HINDSIGHT_API_RETAIN_LLM_MAX_BACKOFF=120.0 # Cap at 2min instead of 1min ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash pip install 'hindsight-api-slim[local-onnx]' # or, in this repository: uv sync --project hindsight-api-slim --extra local-onnx ``` CUDA execution is opt-in. Set `HINDSIGHT_API_EMBEDDINGS_ONNX_DEVICE=cuda` only in an environment that has a compatible `onnxruntime-gpu` wheel and CUDA/cuDNN runtime. The official images intentionally keep the CPU runtime and do not grow when this feature is unused. For a Docker deployment, use the ready-to-build recipe at [`docker/docker-compose/cuda-onnx/`](https://github.com/vectorize-io/hindsight/tree/main/docker/docker-compose/cuda-onnx), which installs the GPU wheel into a private image based on the slim image: ```bash docker compose -f docker/docker-compose/cuda-onnx/docker-compose.yaml up --build ``` The recipe accepts `BASE_IMAGE`, `ONNXRUNTIME_GPU_VERSION`, and `HINDSIGHT_API_EMBEDDINGS_ONNX_CUDA_DEVICE_ID` for release, runtime, and device selection. Hindsight fails startup when CUDA is requested but the provider is unavailable or the initialized session does not activate it; it does not silently fall back to CPU in that mode. The recipe installs CUDA/cuDNN libraries in the private image; the host needs a compatible NVIDIA driver and GPU passthrough. Individual operators may still run on CPU through normal graph partitioning. See the [recipe README](https://github.com/vectorize-io/hindsight/tree/main/docker/docker-compose/cuda-onnx) for dependency isolation, unreleased-checkout builds, and real GPU tests. You can either let Hindsight download the model from Hugging Face at startup by setting `HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID`, or pre-download the ONNX graph and tokenizer files under the Hindsight repository root. ```bash cd /path/to/hindsight mkdir -p models MODEL_ID=intfloat/multilingual-e5-small MODEL_DIR=models/intfloat__multilingual-e5-small uv run --project hindsight-api-slim --extra local-onnx python - <<'PY' import os from huggingface_hub import snapshot_download snapshot_download( repo_id=os.environ["MODEL_ID"], local_dir=os.environ["MODEL_DIR"], allow_patterns=[ "onnx/model.onnx", "onnx/model.onnx_data", "*.json", "*.txt", "*.model", ], ) PY ``` Then start Hindsight with paths relative to the repository root: ```bash export HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH=./models/intfloat__multilingual-e5-small/onnx/model.onnx export HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH=./models/intfloat__multilingual-e5-small export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384 export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: " export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: " ``` For Docker deployments, mount the same model directory and use container paths: ```yaml services: hindsight: volumes: - ./models:/app/models:ro environment: HINDSIGHT_API_EMBEDDINGS_PROVIDER: onnx HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH: /app/models/intfloat__multilingual-e5-small/onnx/model.onnx HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH: /app/models/intfloat__multilingual-e5-small HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS: "384" HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX: "query: " HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX: "passage: " ``` Model-specific examples: ```bash # sentence-transformers/all-MiniLM-L6-v2: 384 dimensions, no E5 prefixes export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=sentence-transformers/all-MiniLM-L6-v2 export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384 export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="" export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="" # intfloat/multilingual-e5-small: 384 dimensions, keep E5 prefixes export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=intfloat/multilingual-e5-small export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384 export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: " export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: " # sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2: 384 dimensions, no E5 prefixes export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384 export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="" export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="" # BAAI/bge-m3: 1024 dimensions, no E5 prefixes; keep onnx/model.onnx_data next to model.onnx export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=BAAI/bge-m3 export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=1024 export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="" export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="" ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Local (default) - uses SentenceTransformers export HINDSIGHT_API_EMBEDDINGS_PROVIDER=local export HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5 # Local with custom model requiring trust_remote_code # WARNING: Only enable trust_remote_code for models you trust (security risk) # export HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=your-custom-model # export HINDSIGHT_API_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE=true # OpenAI - cloud-based embeddings export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=*** # or reuses HINDSIGHT_API_LLM_API_KEY export HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small # 1536 dimensions by default # export HINDSIGHT_API_EMBEDDINGS_OPENAI_DIMENSIONS=384 # optional reduced output size # OpenAI Codex OAuth - uses existing ChatGPT/Codex login, no API key needed export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai-codex export HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small # 1536 dimensions by default # export HINDSIGHT_API_EMBEDDINGS_OPENAI_DIMENSIONS=384 # optional reduced output size # Azure OpenAI - embeddings via Azure endpoint export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=your-azure-api-key export HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small export HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=https://your-resource.openai.azure.com/openai/deployments/your-deployment # TEI - HuggingFace Text Embeddings Inference (recommended for production) export HINDSIGHT_API_EMBEDDINGS_PROVIDER=tei export HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://localhost:8080 # OpenRouter - access 100+ embedding models export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openrouter export HINDSIGHT_API_EMBEDDINGS_OPENROUTER_API_KEY=your-openrouter-api-key # or reuses HINDSIGHT_API_LLM_API_KEY export HINDSIGHT_API_EMBEDDINGS_OPENROUTER_MODEL=perplexity/pplx-embed-v1-0.6b # ZeroEntropy - zembed-1 export HINDSIGHT_API_EMBEDDINGS_PROVIDER=zeroentropy export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_API_KEY=your-api-key export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_MODEL=zembed-1 export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_DIMENSIONS=1280 # export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_ENCODING_FORMAT=base64 # optional # export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_LATENCY=fast # optional # NVIDIA NIM - asymmetric retrieval embeddings (sends input_type=query/passage) export HINDSIGHT_API_EMBEDDINGS_PROVIDER=nvidia export HINDSIGHT_API_EMBEDDINGS_NVIDIA_API_KEY=nvapi-your-key # export HINDSIGHT_API_EMBEDDINGS_NVIDIA_MODEL=nvidia/llama-nemotron-embed-vl-1b-v2 # default # export HINDSIGHT_API_EMBEDDINGS_NVIDIA_DIMENSIONS=1024 # default # Cohere - cloud-based embeddings export HINDSIGHT_API_EMBEDDINGS_PROVIDER=cohere export HINDSIGHT_API_EMBEDDINGS_COHERE_API_KEY=your-api-key export HINDSIGHT_API_EMBEDDINGS_COHERE_MODEL=embed-english-v3.0 # 1024 dimensions # Optional: override output dimensions (for Matryoshka-capable models) # export HINDSIGHT_API_EMBEDDINGS_COHERE_OUTPUT_DIMENSIONS=512 # Azure-hosted Cohere - embeddings via custom endpoint export HINDSIGHT_API_EMBEDDINGS_PROVIDER=cohere export HINDSIGHT_API_EMBEDDINGS_COHERE_API_KEY=your-azure-api-key export HINDSIGHT_API_EMBEDDINGS_COHERE_MODEL=embed-english-v3.0 export HINDSIGHT_API_EMBEDDINGS_COHERE_BASE_URL=https://your-azure-cohere-endpoint.com # LiteLLM proxy - unified gateway for multiple providers export HINDSIGHT_API_EMBEDDINGS_PROVIDER=litellm export HINDSIGHT_API_EMBEDDINGS_LITELLM_API_BASE=http://localhost:4000 export HINDSIGHT_API_EMBEDDINGS_LITELLM_API_KEY=your-litellm-key # optional export HINDSIGHT_API_EMBEDDINGS_LITELLM_MODEL=text-embedding-3-small # or cohere/embed-english-v3.0 # export HINDSIGHT_API_EMBEDDINGS_LITELLM_DIMENSIONS=1536 # optional: declare the width, skip the startup probe # Google - Gemini API (API key auth) export HINDSIGHT_API_EMBEDDINGS_PROVIDER=google export HINDSIGHT_API_EMBEDDINGS_GEMINI_API_KEY=xxxxxxxxxxxx # or reuses HINDSIGHT_API_LLM_API_KEY export HINDSIGHT_API_EMBEDDINGS_GEMINI_MODEL=gemini-embedding-001 # 768 dimensions (default) # export HINDSIGHT_API_EMBEDDINGS_GEMINI_OUTPUT_DIMENSIONALITY=768 # configurable: 256, 512, 768, 1024, etc. # Google - Vertex AI auth (auto-detected when project ID is set) export HINDSIGHT_API_EMBEDDINGS_PROVIDER=google export HINDSIGHT_API_EMBEDDINGS_GEMINI_MODEL=gemini-embedding-001 export HINDSIGHT_API_EMBEDDINGS_VERTEXAI_PROJECT_ID=your-gcp-project-id # falls back to HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID # export HINDSIGHT_API_EMBEDDINGS_VERTEXAI_REGION=us-central1 # falls back to HINDSIGHT_API_LLM_VERTEXAI_REGION # export HINDSIGHT_API_EMBEDDINGS_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/key.json # falls back to LLM config, or uses ADC # LiteLLM SDK - direct API access without proxy server (recommended) export HINDSIGHT_API_EMBEDDINGS_PROVIDER=litellm-sdk export HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_API_KEY=your-provider-api-key export HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_MODEL=cohere/embed-english-v3.0 # Optional: request a specific output dimension when the provider supports it # export HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_OUTPUT_DIMENSIONS=768 # Optional (any provider): the input-token limit inputs are truncated to. # Defaults to 8192; set it to your model's real limit, or 0 to send text uncapped # export HINDSIGHT_API_EMBEDDINGS_MAX_INPUT_TOKENS=8192 # Supported LiteLLM SDK embedding providers: # - cohere/embed-english-v3.0 (1024 dimensions) # - openai/text-embedding-3-small (1536 dimensions) # - together_ai/togethercomputer/m2-bert-80M-8k-retrieval # - huggingface/sentence-transformers/all-MiniLM-L6-v2 # - voyage/voyage-2 ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Primary: a self-hosted reranker that is not always up. export HINDSIGHT_API_RERANKER_PROVIDER=tei export HINDSIGHT_API_RERANKER_TEI_URL=http://workstation:8081 # Member 1: a hosted reranker to fall back on. export HINDSIGHT_API_RERANKER_1_PROVIDER=cohere export HINDSIGHT_API_RERANKER_1_COHERE_API_KEY=... # Member 2: last resort — no reranking, keep the fusion order rather than fail. export HINDSIGHT_API_RERANKER_2_PROVIDER=rrf ``` Ending the chain with `rrf` makes recall **fail open**: results come back in the order the retrieval stages produced, exactly as if no reranker were configured, instead of the request failing. Without an `rrf` member (or with every member down) the error still surfaces, which is the default behaviour. Notes: - Members are tried in order on every request; there is no circuit breaker, so a member that is down costs its own timeout on each request before the next one is tried. Keep the timeouts of an unreliable primary short (`HINDSIGHT_API_RERANKER_TEI_HTTP_TIMEOUT` and the per-provider `*_TIMEOUT` settings). - A member that fails to initialize at startup is logged, not fatal — it is retried on the next request that reaches it. - Scores are not comparable across providers (some return calibrated `[0, 1]` relevance, others logits), so a request served by a fallback member can score differently from one served by the primary. Failovers are logged at `WARNING`. - The indexed members are credential fields — never returned by the bank-config API, and server-level only (not per-bank configurable). :::tip Sizing a TEI reranker TEI reserves one slot per text in a rerank request and rejects the request outright once its pool is full, so start it with at least `HINDSIGHT_API_RERANKER_TEI_MAX_CONCURRENT × HINDSIGHT_API_RERANKER_TEI_BATCH_SIZE` slots — 1024 for the defaults above, versus TEI's own default of 512: ```bash text-embeddings-router --max-concurrent-requests 2048 ... ``` Below that threshold, heavy recall traffic makes the reranker reject work it has the capacity to do. Rejections are retried with backoff, so an undersized pool shows up as slower recall rather than errors. ::: ```bash # Local (default) - uses SentenceTransformers CrossEncoder export HINDSIGHT_API_RERANKER_PROVIDER=local export HINDSIGHT_API_RERANKER_LOCAL_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2 # Local with custom model requiring trust_remote_code (e.g., jina-reranker-v2) # WARNING: Only enable trust_remote_code for models you trust (security risk) export HINDSIGHT_API_RERANKER_PROVIDER=local export HINDSIGHT_API_RERANKER_LOCAL_MODEL=jinaai/jina-reranker-v2-base-multilingual export HINDSIGHT_API_RERANKER_LOCAL_TRUST_REMOTE_CODE=true # TEI - for high-performance inference export HINDSIGHT_API_RERANKER_PROVIDER=tei export HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081 # OpenRouter - access reranking models via OpenRouter export HINDSIGHT_API_RERANKER_PROVIDER=openrouter export HINDSIGHT_API_RERANKER_OPENROUTER_API_KEY=your-openrouter-api-key # or reuses HINDSIGHT_API_LLM_API_KEY export HINDSIGHT_API_RERANKER_OPENROUTER_MODEL=cohere/rerank-v3.5 # Cohere - cloud-based reranking export HINDSIGHT_API_RERANKER_PROVIDER=cohere export HINDSIGHT_API_RERANKER_COHERE_API_KEY=your-api-key export HINDSIGHT_API_RERANKER_COHERE_MODEL=rerank-english-v3.0 # Any Cohere-compatible /rerank endpoint (Azure AI Foundry, Jina, Voyage, self-hosted BGE, etc.) # # Setting HINDSIGHT_API_RERANKER_COHERE_BASE_URL switches the `cohere` provider # off the Cohere SDK and onto a plain HTTP client that speaks the standard # Cohere rerank wire format: # Request: POST {base_url} (or {base_url}/rerank, depending on host) # Authorization: Bearer # {"model": "...", "query": "...", "documents": [...], "return_documents": false} # Response: {"results": [{"index": 0, "relevance_score": 0.9}, ...]} # # Any service implementing this contract works here. For Azure AI Foundry the # base_url is the full invoke URL; for SiliconFlow you can also use the # dedicated `siliconflow` provider below. export HINDSIGHT_API_RERANKER_PROVIDER=cohere export HINDSIGHT_API_RERANKER_COHERE_API_KEY=your-api-key export HINDSIGHT_API_RERANKER_COHERE_MODEL=rerank-english-v3.0 # whatever model the endpoint serves export HINDSIGHT_API_RERANKER_COHERE_BASE_URL=https://your-cohere-compatible-endpoint.com # ZeroEntropy - cloud-based reranking (state-of-the-art accuracy) export HINDSIGHT_API_RERANKER_PROVIDER=zeroentropy export HINDSIGHT_API_RERANKER_ZEROENTROPY_API_KEY=your-api-key export HINDSIGHT_API_RERANKER_ZEROENTROPY_MODEL=zerank-2 # or zerank-2-small # export HINDSIGHT_API_RERANKER_ZEROENTROPY_BASE_URL=https://your-custom-endpoint.com # optional # SiliconFlow - cloud reranking via SiliconFlow's Cohere-compatible /rerank endpoint export HINDSIGHT_API_RERANKER_PROVIDER=siliconflow export HINDSIGHT_API_RERANKER_SILICONFLOW_API_KEY=your-api-key export HINDSIGHT_API_RERANKER_SILICONFLOW_MODEL=BAAI/bge-reranker-v2-m3 # export HINDSIGHT_API_RERANKER_SILICONFLOW_BASE_URL=https://api.siliconflow.cn/v1 # default # NVIDIA NIM - native reranking API (POST {base}/retrieval/{model}/reranking) export HINDSIGHT_API_RERANKER_PROVIDER=nvidia export HINDSIGHT_API_RERANKER_NVIDIA_API_KEY=nvapi-your-key # export HINDSIGHT_API_RERANKER_NVIDIA_MODEL=nvidia/llama-nemotron-rerank-vl-1b-v2 # default # TypeSafe - relevance judged by a typed-decision model (Jev), not a /rerank endpoint export HINDSIGHT_API_RERANKER_PROVIDER=typesafe export HINDSIGHT_API_RERANKER_TYPESAFE_API_KEY=your-api-key # export HINDSIGHT_API_RERANKER_TYPESAFE_MODEL=jev-latest # default # export HINDSIGHT_API_RERANKER_TYPESAFE_PRUNE_CANDIDATES=true # also prune irrelevant candidates # Alibaba Cloud DashScope - qwen3-rerank via Cohere-compatible /reranks endpoint export HINDSIGHT_API_RERANKER_PROVIDER=alibaba export HINDSIGHT_API_RERANKER_ALIBABA_API_KEY=your-dashscope-api-key # or set DASHSCOPE_API_KEY export HINDSIGHT_API_RERANKER_ALIBABA_MODEL=qwen3-rerank # default, can omit # LiteLLM proxy - unified gateway for multiple reranking providers (requires running LiteLLM proxy server) export HINDSIGHT_API_RERANKER_PROVIDER=litellm export HINDSIGHT_API_RERANKER_LITELLM_API_BASE=http://localhost:4000 export HINDSIGHT_API_RERANKER_LITELLM_API_KEY=your-litellm-key # optional export HINDSIGHT_API_RERANKER_LITELLM_MODEL=cohere/rerank-english-v3.0 # or voyage/rerank-2, together_ai/... # LiteLLM SDK - direct API access without proxy (recommended for simplicity) export HINDSIGHT_API_RERANKER_PROVIDER=litellm-sdk export HINDSIGHT_API_RERANKER_LITELLM_SDK_API_KEY=your-deepinfra-api-key export HINDSIGHT_API_RERANKER_LITELLM_SDK_MODEL=deepinfra/Qwen3-reranker-8B # or cohere/rerank-english-v3.0, etc. # Google Discovery Engine - cloud-based semantic reranking export HINDSIGHT_API_RERANKER_PROVIDER=google export HINDSIGHT_API_RERANKER_GOOGLE_PROJECT_ID=your-gcp-project-id export HINDSIGHT_API_RERANKER_GOOGLE_SERVICE_ACCOUNT_KEY=/path/to/service-account.json # optional, uses ADC if unset export HINDSIGHT_API_RERANKER_GOOGLE_MODEL=semantic-ranker-default-004 # or semantic-ranker-fast-004 # Jina MLX - Apple Silicon native reranking (no GPU/cloud required) # Model (~1.2 GB) is downloaded automatically from HuggingFace Hub on first use. export HINDSIGHT_API_RERANKER_PROVIDER=jina-mlx ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash uvx --from "optimum-onnx[onnxruntime]" optimum-cli export onnx \ --model nlpai-lab/KURE-Reranker-nano --task text-classification ./models/kure-reranker-nano export HINDSIGHT_API_RERANKER_PROVIDER=onnx export HINDSIGHT_API_RERANKER_ONNX_MODEL_PATH=./models/kure-reranker-nano/model.onnx export HINDSIGHT_API_RERANKER_ONNX_TOKENIZER_NAME_OR_PATH=./models/kure-reranker-nano ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash export HINDSIGHT_API_DECISION_MODEL_PROVIDER=typesafe export HINDSIGHT_API_DECISION_MODEL_API_KEY=your-typesafe-key export HINDSIGHT_API_REFLECT_DECISION_MODEL_MODEL=jev-preview # reflect only export HINDSIGHT_API_CONSOLIDATION_DECISION_MODEL_PROVIDER=openai # consolidation on OpenAI export HINDSIGHT_API_CONSOLIDATION_DECISION_MODEL_API_KEY=your-openai-key ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Enable the built-in API key authentication export HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension export HINDSIGHT_API_TENANT_API_KEY=your-secret-api-key ``` When enabled, all requests must include the API key in the `Authorization` header: ```bash curl -H "Authorization: Bearer your-secret-api-key" \ http://localhost:8888/v1/default/banks ``` Requests without a valid API key receive a `401 Unauthorized` response. | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_TENANT_EXTENSION` | Dotted path to the loaded tenant extension. Set to `hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension` to require an API key on every request. | *(none; auth disabled)* | | `HINDSIGHT_API_TENANT_API_KEY` | Shared API key checked by the built-in API-key extension. Sent by clients as `Authorization: Bearer `. | *(none)* | | `HINDSIGHT_API_EXTENSION_PASSTHROUGH_HEADERS` | Comma-separated request headers copied into `RequestContext.extra_headers` for extensions to read, matched case-insensitively. Empty means extensions see only the `Authorization` header. | *(none)* | If you are enabling Memory Defense, see `docs/developer/memory-defense/` for the policy schema, detector catalog, and audit trail. #### Forwarding extra headers to extensions A custom `TenantExtension` normally only sees the `Authorization` header, as `RequestContext.api_key`. That is not enough when the bearer token identifies the *proxy* rather than the caller — for example behind an authenticating gateway that presents one shared service identity to Hindsight, and carries the per-caller identity in a second header. List those headers to have them forwarded: ```bash export HINDSIGHT_API_EXTENSION_PASSTHROUGH_HEADERS=x-user-assertion,x-request-origin ``` /* Detailed source-code truncated for AI context efficiency. */ ```json { "content": [ {"type": "text", "text": "To reset the VPN, click the button shown:"}, {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "..."}}, {"type": "text", "text": "...then reconnect."}, {"type": "file", "source": {"type": "base64", "media_type": "application/pdf", "data": "..."}, "filename": "escalation-policy.pdf"} ] } ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Fireworks batch retain (50% cost savings, async only) export HINDSIGHT_API_RETAIN_LLM_PROVIDER=fireworks export HINDSIGHT_API_RETAIN_LLM_API_KEY=fw_xxxxxxxxxxxx export HINDSIGHT_API_RETAIN_LLM_MODEL=accounts/fireworks/models/llama-v3p1-8b-instruct export HINDSIGHT_API_FIREWORKS_ACCOUNT_ID=your-account-id export HINDSIGHT_API_RETAIN_BATCH_ENABLED=true ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash export HINDSIGHT_API_STORE_DOCUMENT_TEXT=false ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash export HINDSIGHT_API_RETAIN_MISSION="Focus on technical decisions, architecture choices, and team member expertise. Deprioritize social or personal information." ``` **`HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbose` — more detail per fact** Use when you need richer facts with full context, relationships, and verbosity. Slower and uses more tokens than `concise`. **`HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbatim` — store chunks as-is** Each chunk is stored as a single memory unit with its original text preserved exactly — no summarization or rewriting. The LLM still runs to extract entities, temporal information, and location so the chunk is fully indexed and retrievable. Useful for RAG-style indexing, document ingestion pipelines, or benchmarks where you want the original text in memory rather than LLM-generated summaries. ```bash export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbatim ``` **`retain_strategies` / `retain_default_strategy` — per-call extraction strategy** Named strategies let you ingest different content types into the same bank using different extraction settings. A strategy is a set of hierarchical field overrides applied on top of the resolved bank config. Any field in the hierarchical config can be overridden per strategy, including `retain_extraction_mode`, `retain_chunk_size`, `retain_structured_chunk_size`, `entity_labels`, `entities_allow_free_form`, `retain_mission`, etc. Configure strategies via the bank config API: ```json { "retain_default_strategy": "conversations", "retain_strategies": { "conversations": { "retain_extraction_mode": "concise", "retain_chunk_size": 3000, "retain_structured_chunk_size": 12000 }, "documents": { "retain_extraction_mode": "chunks", "retain_chunk_size": 800, "entity_labels": null, "entities_allow_free_form": false } } } ``` Then specify the strategy at retain time: If no `strategy` is specified in a retain call, `retain_default_strategy` is used. If neither is set, the bank/global config applies directly. **`HINDSIGHT_API_RETAIN_EXTRACTION_MODE=chunks` — zero LLM cost** Each chunk is stored as-is with no LLM call whatsoever. No entity extraction, no temporal indexing — only embeddings are generated for semantic search. User-provided entities passed via `RetainContent.entities` are the sole source of entity data. Use when ingestion speed and cost matter more than structured metadata. ```bash export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=chunks ``` **`HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom` + `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` — full control** Replaces the built-in selectivity rules entirely. The structural parts of the prompt (output format, temporal handling, coreference resolution) remain intact — only the extraction guidelines are replaced. Use this when `retain_mission` isn't sufficient and you need strict inclusion/exclusion logic. ```bash export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom export HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS="ONLY extract facts that are: ✅ Technical decisions and their rationale ✅ Architecture patterns and design choices ✅ Performance metrics and benchmarks DO NOT extract: ❌ Greetings or social conversation ❌ Process chatter (\"let me check\", \"one moment\") ❌ Anything that would not be useful in 6 months" ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Server default: try iris first, fall back to markitdown if iris fails export HINDSIGHT_API_FILE_PARSER=iris,markitdown # Restrict what clients may request (optional — defaults to all registered parsers) export HINDSIGHT_API_FILE_PARSER_ALLOWLIST=markitdown,iris ``` ```json // Per-request override (in the JSON body of the file retain endpoint) { "parser": "iris", "files_metadata": [ { "document_id": "report" }, { "document_id": "fallback_doc", "parser": ["iris", "markitdown"] } ] } ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Configure a dedicated OpenAI-compatible OCR/vision endpoint for MarkItDown OCR export HINDSIGHT_API_FILE_PARSER=markitdown export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_ENABLED=true export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_API_KEY=your-vision-api-key export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_BASE_URL=https://vision.example/v1 export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_MODEL=ocr-or-vision-model # Optional custom headers for proxies / request tracing # export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_DEFAULT_HEADERS='{"X-Component-Id":"hindsight-ocr"}' ``` #### Parser: iris Cloud-based extraction via [Vectorize Iris](https://docs.vectorize.io/build-deploy/extract-information/understanding-iris/). Higher quality extraction for complex documents, powered by a remote AI service. | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_FILE_PARSER_IRIS_TOKEN` | Vectorize API token | — | | `HINDSIGHT_API_FILE_PARSER_IRIS_ORG_ID` | Vectorize organization ID | — | **Supported formats:** PDF, DOCX, DOC, PPTX, PPT, XLSX, XLS, images (JPG, JPEG, PNG, GIF, BMP, TIFF, WEBP), HTML, TXT, MD, CSV. ```bash # Use iris as the only parser export HINDSIGHT_API_FILE_PARSER=iris export HINDSIGHT_API_FILE_PARSER_IRIS_TOKEN=your-vectorize-token export HINDSIGHT_API_FILE_PARSER_IRIS_ORG_ID=your-org-id # Or: try iris first, fall back to markitdown if iris fails or rejects the file type export HINDSIGHT_API_FILE_PARSER=iris,markitdown ``` #### Parser: llama_parse Cloud-based extraction via [LlamaParse](https://docs.cloud.llamaindex.ai/llamaparse) (LlamaIndex). Strong extraction for complex layouts — tables, charts, multi-column PDFs. | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_FILE_PARSER_LLAMA_PARSE_API_KEY` | LlamaCloud API key (typically starts with `llx-`) | — | **Supported formats:** PDF, DOCX, PPTX, XLSX, HTML, EPUB, RTF, TXT, and many more — see the [LlamaParse docs](https://docs.cloud.llamaindex.ai/llamaparse/features/supported_document_types) for the full list. ```bash # Use llama_parse as the only parser export HINDSIGHT_API_FILE_PARSER=llama_parse export HINDSIGHT_API_FILE_PARSER_LLAMA_PARSE_API_KEY=llx-your-api-key # Or: try llama_parse first, fall back to markitdown export HINDSIGHT_API_FILE_PARSER=llama_parse,markitdown ``` ```bash # Increase batch limits for large file imports export HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE=20 export HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE_MB=500 # Keep files after processing (useful for debugging or re-processing) export HINDSIGHT_API_FILE_DELETE_AFTER_RETAIN=false ``` ### File Storage Files uploaded via the file retain API are stored in an object storage backend before conversion. Choose the backend that fits your infrastructure. | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_FILE_STORAGE_TYPE` | Storage backend: `native`, `s3`, `gcs`, or `azure` | `native` | | `HINDSIGHT_API_FILE_STORAGE_EXTENSION` | `module.path:ClassName` naming your own `FileStorage` implementation, used instead of the built-in backends. Every other `HINDSIGHT_API_FILE_STORAGE_*` variable is passed to it as a lowercased config dict. Implement `delete_prefix` too, or deleting a bank leaves that bank's stored files in your backend. | unset | #### Native (PostgreSQL) Files are stored as `BYTEA` in the `file_storage` table. No additional infrastructure required. Suitable for development and small deployments. ```bash # Native storage is the default — no additional configuration needed export HINDSIGHT_API_FILE_STORAGE_TYPE=native ``` #### S3 / S3-Compatible | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_FILE_STORAGE_S3_BUCKET` | S3 bucket name | - | | `HINDSIGHT_API_FILE_STORAGE_S3_REGION` | AWS region | - | | `HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT` | Custom endpoint URL (for S3-compatible stores like MinIO, Cloudflare R2, Tigris) | AWS default | | `HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID` | AWS access key ID | - | | `HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY` | AWS secret access key | - | For S3-compatible providers that don't expose AWS-style regions (MinIO, Cloudflare R2, Tigris), set `HINDSIGHT_API_FILE_STORAGE_S3_REGION=auto`. The value is required for SigV4 request signing but is ignored by the service. ```bash # AWS S3 export HINDSIGHT_API_FILE_STORAGE_TYPE=s3 export HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=my-hindsight-files export HINDSIGHT_API_FILE_STORAGE_S3_REGION=us-east-1 export HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY # S3-compatible (MinIO, Cloudflare R2, etc.) export HINDSIGHT_API_FILE_STORAGE_TYPE=s3 export HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=my-bucket export HINDSIGHT_API_FILE_STORAGE_S3_REGION=auto export HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT=https://your-minio.example.com export HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=minioadmin export HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=minioadmin # Tigris (S3-compatible, single global endpoint) export HINDSIGHT_API_FILE_STORAGE_TYPE=s3 export HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=my-hindsight-bucket export HINDSIGHT_API_FILE_STORAGE_S3_REGION=auto export HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT=https://t3.storage.dev export HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=tid_your_access_key export HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=tsec_your_secret_key ``` #### Google Cloud Storage | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_FILE_STORAGE_GCS_BUCKET` | GCS bucket name | - | | `HINDSIGHT_API_FILE_STORAGE_GCS_SERVICE_ACCOUNT_KEY` | Path to service account JSON key file | ADC if not set | ```bash export HINDSIGHT_API_FILE_STORAGE_TYPE=gcs export HINDSIGHT_API_FILE_STORAGE_GCS_BUCKET=my-hindsight-files # Optional: use service account key file (otherwise falls back to ADC) export HINDSIGHT_API_FILE_STORAGE_GCS_SERVICE_ACCOUNT_KEY=/path/to/key.json ``` #### Azure Blob Storage | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_FILE_STORAGE_AZURE_CONTAINER` | Azure container name | - | | `HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_NAME` | Azure storage account name | - | | `HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_KEY` | Azure storage account key | - | ```bash export HINDSIGHT_API_FILE_STORAGE_TYPE=azure export HINDSIGHT_API_FILE_STORAGE_AZURE_CONTAINER=hindsight-files export HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_NAME=mystorageaccount export HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_KEY=base64encodedkey== ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash HINDSIGHT_API_WORKER_ENABLED=false ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are broad summaries of project events grouped by week. Each observation should capture what happened, what was decided, and what was blocked — not individual facts. Merge related events into cohesive weekly narratives." ``` **Example: Person-centric knowledge** ```bash export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are durable facts about specific named people: their preferences, skills, relationships, and behavioral patterns. Only create observations for facts that are stable over time and tied to a named individual." ``` **Example: Support ticket patterns** ```bash export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are recurring patterns in customer support interactions: common failure modes, frequently requested features, and pain points that appear across multiple tickets." ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Expose only recall (read-only deployment) export HINDSIGHT_API_MCP_ENABLED_TOOLS=recall # Expose recall and reflect only export HINDSIGHT_API_MCP_ENABLED_TOOLS=recall,reflect ``` Available tool names: `retain`, `recall`, `reflect`, `list_banks`, `create_bank`, `list_mental_models`, `get_mental_model`, `create_mental_model`, `update_mental_model`, `delete_mental_model`, `refresh_mental_model`, `list_directives`, `create_directive`, `delete_directive`, `list_memories`, `get_memory`, `list_documents`, `get_document`, `delete_document`, `list_operations`, `get_operation`, `cancel_operation`, `list_tags`, `get_bank`, `get_bank_stats`, `update_bank`, `delete_bank`, `clear_memories`, `get_knowledge_base_tree`, `search_knowledge_base`, `get_knowledge_page`, `create_knowledge_folder`, `create_knowledge_page`, `update_knowledge_node`, `delete_knowledge_node`. This can also be overridden per bank via the [config API](#hierarchical-configuration): When a bank-level `mcp_enabled_tools` is set, tools not in the list return a clear error when invoked (they still appear in the tools list for MCP protocol compatibility). **MCP Authentication:** By default, the MCP endpoint is open. For production deployments, set `HINDSIGHT_API_MCP_AUTH_TOKEN` to require Bearer token authentication: ```bash export HINDSIGHT_API_MCP_AUTH_TOKEN=your-secret-token ``` Clients must then include the token in the `Authorization` header. See [MCP Server documentation](./mcp-server.md#authentication) for details. **Local MCP instructions:** ```bash # Example: instruct MCP to also store assistant actions export HINDSIGHT_API_MCP_INSTRUCTIONS="Also store every action you take, including tool calls and decisions made." ``` /* Detailed source-code truncated for AI context efficiency. */ ``` HINDSIGHT_API_WORKER_CONSOLIDATION_BANK_PRIORITY="shadow-*:10,staging-*:5,*:1" ``` /* Detailed source-code truncated for AI context efficiency. */ ```python from hindsight_api import MemoryEngine memory = MemoryEngine.from_env() await memory.initialize() ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Enable tracing export HINDSIGHT_API_OTEL_TRACES_ENABLED=true # Configure endpoint (example: OpenLIT Cloud) export HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.openlit.io export HINDSIGHT_API_OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer olit-xxx" # Optional: Custom service name and environment export HINDSIGHT_API_OTEL_SERVICE_NAME=hindsight-production export HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT=production ``` **Local Development:** For local development, we recommend the Grafana LGTM stack which provides traces, metrics, and logs in a single container: ```bash ./scripts/dev/start-grafana.sh ``` /* Detailed source-code truncated for AI context efficiency. */ ``` HINDSIGHT_API_PROFILE='{"every": 60, "top": 20}' ``` ``` [profile] thread hindsight-loop- 1.41 cores [profile] thread asyncio_ 0.37 cores [profile] 1598 functions active in the last 60s [profile] tottime=0.923 cumtime=1.833 ncalls=15842 [profile] tottime=0.691 cumtime=7.039 ncalls=1511590 ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash # Point Control Plane to a remote API service export HINDSIGHT_CP_DATAPLANE_API_URL=http://api.example.com:8888 # Authenticate to an auth-protected API service export HINDSIGHT_CP_DATAPLANE_API_KEY=my-dataplane-bearer-token # Protect the Control Plane with an access key export HINDSIGHT_CP_ACCESS_KEY=my-secret-key ``` ### Hierarchical Configuration Hindsight supports per-bank configuration overrides through a hierarchical system: **Global (env vars) → Tenant → Bank**. #### Type-Safe Config Access To prevent accidentally using global defaults when bank-specific overrides exist, Hindsight enforces type-safe config access: **In Application Code:** ```python from hindsight_api.config import get_config # ✅ Access static (infrastructure) fields config = get_config() host = config.host # OK - static field port = config.port # OK - static field # ❌ Attempting to access bank-configurable fields raises an error chunk_size = config.retain_chunk_size # ConfigFieldAccessError! ``` **Error Message:** ``` ConfigFieldAccessError: Field 'retain_chunk_size' is bank-configurable and cannot be accessed from global config. Use ConfigResolver.resolve_full_config(bank_id, context) to get bank-specific config. ``` **For Bank-Specific Config:** ```python # Internal code that needs bank-specific settings from hindsight_api.config_resolver import ConfigResolver # Resolve full config for a specific bank config = await config_resolver.resolve_full_config(bank_id, request_context) chunk_size = config.retain_chunk_size # ✅ Uses bank-specific value ``` /* Detailed source-code truncated for AI context efficiency. */ ```bash export HINDSIGHT_API_DEFAULT_BANK_TEMPLATE='{"version":"1","bank":{"reflect_mission":"Help support agents remember customer interactions.","retain_extraction_mode":"verbose","disposition_empathy":5},"directives":[{"name":"Be concise","content":"Always respond concisely.","priority":10}]}' ``` #### API Endpoints - `GET /v1/default/banks/{bank_id}/config` - View resolved config (filtered by permissions) - `PATCH /v1/default/banks/{bank_id}/config` - Update bank overrides (only allowed fields) - `DELETE /v1/default/banks/{bank_id}/config` - Reset to defaults #### Permission System Tenant extensions can control which fields banks are allowed to modify via `get_allowed_config_fields()`: ```python class CustomTenantExtension(TenantExtension): async def get_allowed_config_fields(self, context, bank_id): # Option 1: Allow all configurable fields return None # Option 2: Allow specific fields only return {"retain_chunk_size", "retain_custom_instructions"} # Option 3: Read-only (no modifications) return set() ``` #### Examples **Security Notes:** - Credentials (API keys, base URLs) are never returned in responses - Only configurable fields can be modified - Responses are filtered by tenant permissions - Attempting to set credentials returns 400 error ### Reverse Proxy / Subpath Deployment To deploy Hindsight under a subpath (e.g., `example.com/hindsight/`): 1. Set both environment variables to the same path: ```bash HINDSIGHT_API_BASE_PATH=/hindsight NEXT_PUBLIC_BASE_PATH=/hindsight ``` 2. Configure your reverse proxy to: - Forward `/hindsight/*` requests to Hindsight - Preserve the full path in forwarded requests - Set appropriate proxy headers (X-Forwarded-Proto, X-Forwarded-For) **Example: Nginx Configuration** ```nginx location /hindsight/ { proxy_pass http://localhost:8888/; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } ``` **Example: Traefik Configuration** ```yaml http: routers: hindsight: rule: "PathPrefix(`/hindsight`)" service: hindsight middlewares: - hindsight-stripprefix middlewares: hindsight-stripprefix: stripPrefix: prefixes: - "/hindsight" services: hindsight: loadBalancer: servers: - url: "http://localhost:8888" ``` **Important Notes:** - The base path must start with `/` and should NOT end with `/` - Both API and Control Plane should use the same base path - After setting environment variables, restart both services - OpenAPI docs will be available at `/docs` (e.g., `/hindsight/docs`) **Complete Examples:** See `docker/compose-examples/` directory for: - Nginx configuration files (`simple.conf`, `api-and-control-plane.conf`) - Docker Compose setups (`docker-compose.yml`, `reverse-proxy-only.yml`) - Traefik and other reverse proxy examples - Full deployment documentation --- ## Example .env File ```bash # API Service HINDSIGHT_API_DATABASE_URL=postgresql://hindsight:hindsight_dev@localhost:5432/hindsight # HINDSIGHT_API_DATABASE_SCHEMA=public # optional, defaults to 'public' HINDSIGHT_API_LLM_PROVIDER=groq HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx # Authentication (optional, recommended for production) # HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension # HINDSIGHT_API_TENANT_API_KEY=your-secret-api-key # File storage (optional, defaults to PostgreSQL native storage) # HINDSIGHT_API_FILE_STORAGE_TYPE=s3 # HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=my-hindsight-files # HINDSIGHT_API_FILE_STORAGE_S3_REGION=us-east-1 # HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE # HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY # Control Plane HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888 ``` --- For configuration issues not covered here, please [open an issue](https://github.com/vectorize-io/hindsight/issues) on GitHub. --- ### Hindsight Docs Docs Developer Development (`hindsight-docs/docs/developer/development.md`) --- sidebar_position: 7 --- # Development Guide Guide to setting up a local development environment for contributing to Hindsight. ## Prerequisites - Python 3.11+ - [uv](https://docs.astral.sh/uv/) - Fast Python package manager - Docker and Docker Compose - An LLM provider credential, or a local/subscription-backed provider such as Ollama or GitHub Copilot ## Local Development Setup ### 1. Clone the Repository ```bash git clone https://github.com/vectorize-io/hindsight.git cd hindsight ``` ### 2. Install Dependencies ```bash uv sync ``` ### 3. Start PostgreSQL Start only the database via Docker: ```bash cd docker && docker-compose up -d postgres ``` ### 4. Configure Environment ```bash cp .env.example .env ``` Edit `.env` with your LLM API key: ```bash # Database (connects to Docker postgres) HINDSIGHT_API_DATABASE_URL=postgresql://hindsight:hindsight_dev@localhost:5432/hindsight # LLM Provider (choose one) HINDSIGHT_API_LLM_PROVIDER=groq HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx HINDSIGHT_API_LLM_MODEL=llama-3.1-70b-versatile ``` ### 5. Start the API Server ```bash ./scripts/start-server.sh --env local ``` The server will be available at http://localhost:8888. ## Running Tests ```bash # Run all tests uv run pytest # Run specific test file uv run pytest tests/test_retrieval.py # Run with verbose output uv run pytest -v ``` ## Code Generation ### Regenerate API Clients When you modify the OpenAPI spec, regenerate the clients: ```bash ./scripts/generate-clients.sh ``` This generates: - Python client in `hindsight-clients/python/` - TypeScript client in `hindsight-clients/typescript/` ### Export OpenAPI Schema ```bash ./scripts/export-openapi.sh ``` ## Project Structure ``` hindsight/ ├── hindsight-api/ # Main API server │ ├── hindsight_api/ │ │ ├── api/ # HTTP endpoints │ │ ├── engine/ # Memory engine, retrieval, reasoning │ │ └── web/ # Server entry point │ └── tests/ ├── hindsight-clients/ # Generated SDK clients │ ├── python/ │ └── typescript/ ├── hindsight-control-plane/ # Admin UI (Next.js) ├── docker/ # Docker Compose setup └── scripts/ # Development scripts ``` ## Contributing Hindsight does not accept pull requests from outside the team. To report a bug or ask for a feature, [open an issue](https://github.com/vectorize-io/hindsight/issues/new/choose). See [CONTRIBUTING.md](https://github.com/vectorize-io/hindsight/blob/main/CONTRIBUTING.md) for details. ## Troubleshooting ### Database Connection Issues Ensure PostgreSQL is running: ```bash docker-compose ps ``` Check database connectivity: ```bash psql postgresql://hindsight:hindsight_dev@localhost:5432/hindsight ``` ### ML Model Download On first run, Hindsight downloads embedding and reranking models. This may take a few minutes. Models are cached in `~/.cache/huggingface/`. ### Port Conflicts If port 8888 is in use: ```bash HINDSIGHT_API_PORT=8889 ./scripts/start-server.sh --env local ``` --- ### Hindsight Docs Docs Developer Extensions (`hindsight-docs/docs/developer/extensions.md`) # Extensions Extensions allow you to customize and extend Hindsight behavior without modifying core code. They enable multi-tenancy, custom authentication, additional HTTP endpoints, and operation hooks. --- ## Available Extensions ### TenantExtension Handles multi-tenancy and API key authentication. Validates incoming requests and determines which PostgreSQL schema to use for database operations, enabling tenant isolation at the database level. **Built-in: ApiKeyTenantExtension** A simple implementation that validates API keys against an environment variable and uses the `public` schema for all authenticated requests. ```bash HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension HINDSIGHT_API_TENANT_API_KEY=your-secret-key ``` **No longer built in: SupabaseTenantExtension** Validates [Supabase](https://supabase.com) JWTs and gives each authenticated user their own PostgreSQL schema. It now lives in the [extensions registry](https://github.com/vectorize-io/hindsight/tree/main/hindsight-extensions/supabase-tenant), which documents its configuration and ships a Dockerfile that builds an image with it. :::warning Breaking change in 0.9.3 Up to 0.9.2 this extension was built in, at `hindsight_api.extensions.builtin.supabase_tenant`. That path no longer exists, so an install still pointing at it fails at startup with `ModuleNotFoundError`. Add the extension to your image and set `HINDSIGHT_API_TENANT_EXTENSION=hindsight_ext_supabase_tenant:SupabaseTenantExtension`. All `HINDSIGHT_API_TENANT_*` settings and the schema naming are unchanged. ::: **External: StaticKeysTenantExtension** A fully self-hosted multi-user mode: users and their API keys are declared in environment variables (no external identity provider, no users table). Each user maps to their own PostgreSQL schema (`{prefix}_{user_id}`), provisioned lazily on first access, giving database-level memory isolation between users. Multiple API keys may map to the same user and schema. User IDs are case-insensitive: they are lowercased (and dashes normalized to underscores) before building the schema name, so `Rafael`, `rafael` and `RAFAEL` all resolve to the same tenant schema. It lives in the [extensions registry](https://github.com/vectorize-io/hindsight/tree/main/hindsight-extensions/static-keys-tenant), which documents its configuration and ships a Dockerfile that builds an image with it. For other multi-tenant setups with separate schemas per tenant (e.g., custom JWT-based auth), implement a custom `TenantExtension`. --- ### HttpExtension Adds custom HTTP endpoints under the `/ext/` path prefix. Useful for adding domain-specific APIs that integrate with Hindsight's memory engine. Provides two router methods: - `get_router(memory)` — returns a FastAPI router mounted at `/ext/` - `get_root_router(memory)` — returns a FastAPI router mounted at the application root (for well-known endpoints or other paths that must be at specific locations). Returns `None` by default. **No built-in implementation** - implement your own to add custom endpoints. ```bash HINDSIGHT_API_HTTP_EXTENSION=mypackage.ext:MyHttpExtension ``` --- ### OperationValidatorExtension Hooks into retain/recall/reflect operations for validation and monitoring. Use cases include: - Rate limiting and quota enforcement - Permission checks and content filtering - Audit logging and usage tracking - Custom metrics collection **No built-in implementation** - implement your own based on your requirements. `self.context` is the process-wide extension context — use it for process-global handles such as `get_memory_engine()`. It holds no per-request state: take the tenant and bank from each hook's own argument (`ctx.bank_id` and `ctx.request_context`, the latter holding the identity resolved by the tenant extension). ```bash HINDSIGHT_API_OPERATION_VALIDATOR_EXTENSION=mypackage.validators:MyValidator ``` --- ### MCPExtension Registers additional MCP (Model Context Protocol) tools on the Hindsight MCP server. Enables external packages to add custom tools without modifying core code. **No built-in implementation** - implement your own to add custom MCP tools. ```bash HINDSIGHT_API_MCP_EXTENSION=mypackage.mcp:MyMCPExtension ``` --- ### LLMRequestExtension Decides, per LLM call, which LLM the call goes to and with which credentials. Use it to: - **Bill each customer on their own key.** Front your LLM spend with a gateway (LiteLLM proxy, Bifrost, OpenRouter, Portkey, ...) that gives each customer a virtual key, and send each bank's calls with its customer's key. - **Pick the model per bank or per operation.** A cheap model for one bank's retain and a strong one for its reflect, a different provider or cloud account per customer. It is asked once per LLM call with the tenant schema, the bank and the operation the call runs for — retain, reflect, consolidation, mental-model refresh — in the API process and in the worker alike. It answers with an `LLMRequestOverrides`, or `None` to use the server's own LLM. Raising refuses the call. ```bash HINDSIGHT_API_LLM_REQUEST_EXTENSION=mypackage.llms:MyLLMs ``` An answer can carry: | Field | Effect | | --- | --- | | `llm` | Send the call to this LLM instead of the server's: provider, model, base URL, key and the other per-LLM settings (`LLMMemberConfig`, the shape of a multi-LLM chain member). Fields left unset inherit the server's global LLM defaults, except its credentials (default headers, Vertex service account, Codex home, LiteLLM router config); timeouts and retries inherit the operation's. | | `api_key` | The key for the LLM the call goes to (the server's, or `llm`'s — it fills an `llm` that has no key of its own). | | `headers` | Extra request headers, e.g. a gateway's customer id. | Things to know: - **The choice is server-side.** Nothing comes from the API caller. If your extension builds an answer from something a caller can write (bank config, metadata), validate it — otherwise that is where an endpoint or a key would come from user input. - **Credentials without `llm` cost no new client.** The key and headers go on a per-call copy of the existing SDK client that shares its connection pool. Supported on OpenAI-compatible endpoints (`openai`, `groq`, `deepseek`, `openrouter`, `fireworks`, `lmstudio`, any gateway set as `base_url`, ...), `anthropic` and `litellm` routed to a keyed backend (OpenAI, Anthropic, Azure, Gemini AI Studio, OpenRouter, a LiteLLM proxy, ...). A call to any other provider (`gemini`, `vertexai`, `ollama`, `bedrock`, `litellmrouter`, the OAuth/CLI providers) that the answer gives a key or headers to is **refused**, never sent on the configured credentials. To change the key of one of those, answer with a whole `llm` instead. A key is also refused when the server's default headers carry a credential (`Authorization`, `x-api-key`, `x-portkey-api-key`, ...) the answer does not override, since the SDK would send that one instead. - **Asked once per LLM in a chain.** With a multi-LLM chain or a provider per operation, each LLM is asked separately; check `target.provider` before handing out a key meant for one of them. - **Reflect prompt caching is off** while the extension is installed: Gemini's step-by-step reflect cache is created on the server's LLM, outside the calls the extension decides. - **An `llm` answer is built once and reused**, up to 64 distinct LLMs per process (least recently used are dropped). An extension that answers with one LLM per user on a large user base should use a gateway and `api_key` instead. - **No batch retain.** `HINDSIGHT_API_RETAIN_BATCH_ENABLED=true` with this extension fails at startup: batch jobs are submitted on the server's LLM. - **LLM calls only.** Embeddings and reranking keep their own configuration. - **Lifecycle:** `on_startup()` runs when the memory engine initializes, before any LLM call, and `on_shutdown()` when it closes. - **Startup check:** the connection check at boot uses the server's own LLM. If it has no usable key (for example, the gateway has none for the server), set `HINDSIGHT_API_SKIP_LLM_VERIFICATION=true`. Load the same extension in the API and in every worker: a worker without it runs consolidation and refreshes on the server's LLM. The [`bank-llm-routing`](https://github.com/vectorize-io/hindsight/tree/main/hindsight-extensions/bank-llm-routing) community extension routes on bank-id and operation patterns from configuration, and is a working example of the interface. --- ### ModelCallExtension Receives every model call the server makes — LLM, embedding, rerank and decision-model calls, successes and failures — once it finishes, as a `ModelCall`: the tenant schema, bank, operation and trace it ran for, scope, provider and model, input / cached / output / thoughts tokens, duration, and the request and response themselves (plus the tool calls, when the model asked for tools). Use it to keep a per-call record of model spend and content outside the `llm_requests` table: object storage, a warehouse, an observability product. ```bash HINDSIGHT_API_MODEL_CALL_EXTENSION=mypackage.calls:MyRecorder ``` Things to know: - **`record()` runs on the call path, synchronously.** Only buffer there: no I/O, nothing that blocks. Send from a background task. An exception it raises never fails the call; it is swallowed and logged at DEBUG only, so catch and report your own errors. - **Lifecycle:** `on_startup()` runs when the memory engine initializes, before any model call; `on_shutdown()` when it closes, after the workers have stopped, so a buffer can be written out completely. - **It sees the same calls `llm_requests` tracing sees**, whether or not that table is written, and in the API and the worker alike. Load it in both. - **`input` and `output` are the call's content** (prompts and memories included). Treat what you store accordingly. - **Operation runs (optional):** override `record_operation(run)` to also receive an `OperationRun` when each retain, recall, reflect, consolidation or mental-model refresh ends, wherever it started (HTTP, MCP or a queued task). It carries the run's wall time, its success or error, the model calls it made (count, summed time and tokens), its calls into the memories store (count and summed time, measured at the store's interface), the engine's timed phases, and for a queued task its id and when it was queued. Every `ModelCall` made inside a run carries the run's id as `operation_id`, so the two join. A run opened inside another (a mental-model refresh's reflect pass) is its own run with `parent_id` set, and keeps only its own figures. Same rule as `record()`: buffer and return. - **`current_operation_id()`** (`hindsight_api.operation_runs`) returns the run in progress, for a memories store that wants to tag its own requests with it. --- ## Writing Custom Extensions ### Extension Basics Extensions are Python classes loaded via environment variables: ```bash HINDSIGHT_API__EXTENSION=mypackage.module:MyExtensionClass ``` Configuration is passed via prefixed environment variables: ```bash HINDSIGHT_API__SOME_CONFIG=value # Extension receives: {"some_config": "value"} ``` All extensions support lifecycle hooks: - `on_startup()` - Called when the application starts - `on_shutdown()` - Called when the application shuts down Extensions have access to an `ExtensionContext` that provides: - `run_migration(schema)` - Run database migrations for a schema - `get_memory_engine()` - Get the MemoryEngine interface ### Shipping your own database migrations An extension that keeps state of its own returns the directory holding its Alembic revision files, and they are applied in the same migration run as Hindsight's own — same ordering guarantees, same `alembic_version` table: ```python class MyExtension(TenantExtension): def alembic_version_locations(self) -> list[str]: return [str(Path(__file__).parent / "alembic" / "versions")] ``` The directory holds revision files only — there is no `env.py`; Hindsight's own configures the schema and the connection. The tree is independent of Hindsight's: give its first revision `down_revision = None` and a `branch_labels` naming your extension, so an operator can address it (`alembic upgrade