hindsight Documentation

Start the API Server Copy Chapter

pip install hindsight-api
export OPENAI_API_KEY=sk-xxx
export HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY

hindsight-api

export OPENAI_API_KEY=sk-xxx

docker run -it --pull always --name hindsight --restart unless-stopped --shm-size=1g -p 8888:8888 -p 9999:9999 \
  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
  -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \
  ghcr.io/vectorize-io/hindsight:latest

:::tip LLM Provider
Hindsight requires an LLM with structured output support. Recommended: Groq with gpt-oss-20b for fast, cost-effective inference.
See LLM Providers for more details.
:::

Use the Client Copy Chapter

pip install hindsight-client

npm install @vectorize-io/hindsight-client

curl -fsSL https://hindsight.vectorize.io/get-cli | bash

go get github.com/vectorize-io/hindsight/hindsight-clients/go

What's Happening Copy Chapter

Integrations Copy Chapter

Browse all supported integrations in the Integrations Hub.

Next Steps Copy Chapter

• Retain — Advanced options for storing memories
• Recall — Search and retrieval strategies
• Reflect — Disposition-aware reasoning
• Memory Banks — Configure disposition and mission
• Server Deployment — Docker Compose, Helm, and production setup

Hindsight Docs Docs Developer Memory Defense Index (`hindsight-docs/docs/developer/memory-defense/index.md`) Copy Section

sidebar_position: 95 Copy Chapter

Memory Defense

Hindsight scrubs secrets and PII from retain content using a 45-pattern regex set. Each match is replaced with a [REDACTED:type] marker before content reaches memory units or the document body. The feature is configured per bank and disabled by default.

How it works Copy Chapter

Memory Defense is opt-in per bank. The extension is always present, but it sits dormant until you give a bank a policy that turns it on. When a policy is set, every memory the agent writes to that bank is scanned before it lands in storage. When the scanner recognizes a credential, an API key, a database connection string, or a known PII format, the matched substring is replaced with a redaction marker like [REDACTED:github_token].

The scrubbed version is what actually gets stored. Memory units and document bodies persist the redacted text, so future recall responses, exports, and reflect operations never see the original secret.

A policy only affects future retain calls on the bank where it is set. Existing memories are not retroactively scanned when you add or change a policy.

Configuring Memory Defense Copy Chapter

Memory Defense is configured per bank via the bank's memory_defense config field. You can set the policy at bank creation time or update it later via PATCH /v1/{tenant}/banks/{bank_id}/config.

The open-source version implements the sensitive_data rule with two possible actions:

• redact — replace each matched secret with a [REDACTED:type] marker and store the scrubbed memory.
• block — drop any item that contains a match. If every item in a retain request is blocked, the call returns 422.

A minimal policy:

{
  "memory_defense": {
    "enabled": true,
    "rules": [
      { "on": "sensitive_data", "action": "redact" }
    ]
  }
}

Once that policy is on a bank, every retain to that bank is screened with the 45 redaction patterns documented below.

:::note Existing memories are not retroactively scanned
Enabling Memory Defense on a bank only affects future retain calls. Memories already in the bank are not re-scanned or modified when you add or change a policy. If you need to scrub a bank that already contains unredacted content, you have to re-ingest the affected memories or remove them manually.
:::

Disabled by default Copy Section

Memory Defense is off on every bank until you set a policy. A bank with no memory_defense field, with enabled: false, or with no sensitive_data rule is treated identically: the extension returns ALLOW and content passes through unchanged. To stop redacting on a bank that has it on, set enabled: false or remove the policy.

Notifications Copy Chapter

When an item is redacted or blocked, Hindsight fires a memory_defense.triggered webhook if a webhook on the bank is subscribed to that event type. The payload reports the action taken, the document ID, and which redaction patterns matched — useful for routing security alerts to a SIEM or Slack. Clean items fire no event.

The same redact/block decisions are also recorded as memory_defense entries in the audit log (when audit logging is enabled), with the action and matched pattern labels in the entry metadata.

Patterns covered Copy Chapter

The 45 bundled patterns cover the categories below.

AI and LLM providers Copy Section

Cloud providers Copy Section

Source control and CI Copy Section

Payment processors Copy Section

Communications and email Copy Section

Commerce Copy Section

Database connection strings Copy Section

Private keys, JWTs, and generic credentials Copy Section

PII (US defaults) Copy Section

Hindsight Docs Docs Cloud Enterprise Enterprise (`hindsight-docs/docs/cloud-enterprise/enterprise.md`) Copy Section

title: Hindsight Enterprise Copy Chapter

Hindsight Enterprise

Hindsight Enterprise is Hindsight for organizations that need their own deployment. Instead of sharing the multi-tenant Hindsight Cloud, you get a single-tenant installation that Vectorize sets up and runs with you. It includes everything in Hindsight Cloud, plus the security, compliance, networking and scale features below.

Book a call · Hindsight Enterprise docs

Deployment options Copy Chapter

Private cloud and BYOC can be reached over private networking (AWS PrivateLink, Google Cloud Private Service Connect, Azure Private Link), so traffic never crosses the public internet. See Choosing a deployment option for a full comparison, including who is responsible for what.

What Enterprise adds Copy Chapter

• Single-tenant deployment: its own database, storage, compute and identity service.
• Hindsight Memlake: memory on object storage, for large and growing banks.
• Single sign-on and MFA: Okta, Microsoft Entra ID, Google Workspace or any OIDC or SAML provider.
• Memory Defense: detect and redact secrets and sensitive data, and block prompt injection before it is stored.
• Data privacy (GDPR): erase, retain and export a person's data across banks.
• Audit logs, webhooks and SIEM: a record of who did what, streamed to your own systems.
• Enterprise support: a ticket portal, a shared Slack channel and contracted response times.
• Invoiced billing: annual or contract billing instead of card payments.

Hindsight Memlake Copy Chapter

Hindsight Memlake is the Enterprise storage engine for agent memory. Open-source Hindsight stores memories in PostgreSQL, which works well for most banks. But agent memory only grows, and as banks reach hundreds of thousands of memories, a relational database gets slower and more expensive to keep fast.

Memlake keeps memories in object storage (S3, Google Cloud Storage, Azure Blob Storage, or an S3-compatible store on-premises), with one bucket per organization:

• Built for scale. In our tests, 100 million memories were served at under 100 ms p99 recall, on four servers.
• Fast and steady. Recall time barely moves as a bank grows, because each bank has its own index, kept warm in a cache on the servers.
• Lower cost as you grow. Memories live in object storage, the cheapest durable storage there is, and servers hold only a cache, so you add servers for traffic, not for data.
• Durable. A write is in object storage before it is acknowledged, and continuously backed up to a separate bucket.
• Nothing changes for your agents. Same API, SDKs and features; banks, settings and background jobs stay in PostgreSQL.

If your banks stay under roughly 50,000 memories each, PostgreSQL is a good fit and Memlake is not needed. Memlake is available on private cloud, BYOC and on-premises deployments.

Read more: Hindsight Memlake overview, architecture, performance and cost and tradeoffs.

Learn more Copy Chapter

• Hindsight Enterprise documentation
• Ways to run Hindsight: self-hosted, Cloud and Enterprise compared
• Book a call with the Hindsight team

Hindsight Docs Docs Cloud Enterprise Hindsight Cloud (`hindsight-docs/docs/cloud-enterprise/hindsight-cloud.md`) Copy Section

title: Hindsight Cloud Copy Chapter

Hindsight Cloud

Hindsight Cloud is Hindsight run for you. Sign up, create a memory bank and connect your agents, with nothing to deploy or operate. It runs the same engine as open-source Hindsight, so everything in these docs about how memory works applies.

Start on Hindsight Cloud · Hindsight Cloud docs

What you get Copy Chapter

• Nothing to run. No servers, databases or upgrades. The LLM, embedding and reranking models are included.
• The same API. Retain, recall, reflect, mental models and knowledge pages, through the same Python, TypeScript and Go clients, the CLI and the MCP server. Point them at your Cloud API URL and key.
• Integrations. Pre-built integrations with agent frameworks, coding agents and AI tools.
• A console. Browse banks, memories, mental models and knowledge pages, and see how a bank is learning.
• Teams. Invite your team with role-based access, and manage API keys.
• Usage analytics. Dashboards for what each bank uses.

Pricing Copy Chapter

You pay for what you use, from a prepaid credit balance: per token for retain and recall, per call for reflect and for creating or refreshing mental models and knowledge pages, and for storage of memories kept longer than 30 days. See Billing for current rates.

Enterprise features on Cloud Copy Chapter

Some features are part of Hindsight Enterprise and can be turned on for a Cloud organization: SSO, enforced MFA, Memory Defense, data privacy (GDPR), audit logs, webhooks and SIEM, and Enterprise support. See Hindsight Cloud with Enterprise features.

Hindsight Cloud is shared and multi-tenant. If you need a dedicated deployment, see Hindsight Enterprise.

Learn more Copy Chapter

• Getting started: your first memory bank, step by step
• Hindsight Cloud documentation
• Ways to run Hindsight: self-hosted, Cloud and Enterprise compared

Hindsight Docs Docs Developer Admin Cli (`hindsight-docs/docs/developer/admin-cli.md`) Copy Section

Admin CLI

The hindsight-admin CLI provides administrative commands for managing your Hindsight deployment, including database migrations, backup, and restore operations.

Installation Copy Chapter

The admin CLI is included with the hindsight-api package — installing it puts the hindsight-admin executable on your PATH:

pip install hindsight-api
# or
uv add hindsight-api

Running the CLI Copy Chapter

hindsight-admin connects directly to PostgreSQL — it does not call the HTTP API. It reads the same configuration as the API service (environment variables, and a .env file in the current working directory), so it operates on whatever database HINDSIGHT_API_DATABASE_URL points to:

• Default: pg0, the embedded development database (must be run on the host that owns the pg0 data).
• Production: set HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/hindsight.

Because it talks to the database directly (binary COPY, TRUNCATE, etc.), the admin CLI is PostgreSQL-only (not supported on Oracle). Run it on the same host/container as your API deployment so it inherits the right configuration and has network access to the database:

# Bare metal / virtualenv (with the API's env or a .env in the working dir)
hindsight-admin worker-status

# Docker — exec into the API container
docker exec -it hindsight-api hindsight-admin backup /data/backup.zip

# Kubernetes — exec into an API pod
kubectl exec deploy/hindsight-api -- hindsight-admin run-db-migration

Use --schema to target a specific tenant schema (commands default to the configured base schema). See Environment Variables below.

Commands Copy Chapter

run-db-migration Copy Section

Run database migrations to the latest version. By default this migrates the base schema plus all tenant schemas discovered by the tenant extension. Use --schema for targeted migration of one schema. This is useful when you want to run migrations separately from API startup (e.g., in CI/CD pipelines or before deploying a new version).

hindsight-admin run-db-migration [OPTIONS]

Options:

Examples:

# Run migrations on the base schema plus all discovered tenant schemas
hindsight-admin run-db-migration

# Run migrations on a specific tenant schema
hindsight-admin run-db-migration --schema tenant_acme

:::tip Disabling Auto-Migrations
To disable automatic migrations on API startup, set HINDSIGHT_API_RUN_MIGRATIONS_ON_STARTUP=false. This is useful when you want to run migrations as a separate step in your deployment pipeline.
:::

repair-bank Copy Section

Verify and repair a bank's per-(bank, fact_type) vector index coverage.

These partial indexes are normally created when a bank is first created (instant on an empty bank), and PostgreSQL maintains them incrementally as the bank grows. A bank that becomes populated outside that create-time path — via a logical restore, a cross-version upgrade, or a vector-extension switch — never gets them, so its bank-scoped recall silently falls back to a global index + post-filter. That fallback is both slower and can under-return results (the approximate nearest-neighbour search draws its candidates from every bank, then filters to yours afterward).

Run this after any of those events to restore full coverage. It detects missing or invalid coverage (an INVALID leftover from an interrupted build, or an index whose type drifted after a backend switch, both count as missing) and rebuilds it with CREATE INDEX CONCURRENTLY, so it never blocks live retain/recall/consolidation. It is idempotent and safe to re-run.

hindsight-admin repair-bank (--bank BANK_ID | --all) [OPTIONS]

Options:

Exactly one of --bank or --all is required. No-op for backends that use a single global vector index (AlloyDB ScaNN, Oracle). It is idempotent — safe to re-run and safe to run while the API is serving traffic.

It is also a no-op for a bank whose memories a custom store owns, this time decided
per bank rather than per deployment: such a bank has no rows in memory_units, so it
is owed no per-bank index and nothing is built for it. Nothing is dropped either — a
bank that carries indexes from before it moved to the store keeps them, because
shedding them is an operator decision with its own timing, not something a repair
does on your behalf. A store-owned bank therefore reports 0 present, 0 created
whether or not it still carries indexes; to see what one actually carries, query
the catalog for that bank's own partial indexes:

SELECT indexname FROM pg_indexes
WHERE schemaname = '<tenant schema>'
  AND tablename = 'memory_units'
  AND indexname LIKE 'idx_mu_emb_%'
  AND strpos(indexdef, $$bank_id = '<bank id>'$$) > 0;

Drop the last condition to list every per-bank index in the schema instead.

Those indexes are empty, and Postgres plans against every index on a relation, so on
a deployment with many store-owned banks they are charged to every other query that
touches memory_units. Drop them with DROP INDEX CONCURRENTLY when you are ready.

Exit code. 0 when every schema was reconciled. 1 when an index failed to build
(the names are printed, and a re-run retries them), or when a schema was not fully
reconciled — for example because the memories store could not say which banks it
owns. The command does not guess there, because guessing would rebuild the very
indexes described above.

A schema is named as skipped whether it failed before its first bank or partway
through, so it may already have had some banks reconciled; when it got that far the
per-schema line says how many, and those builds are not undone. Both the failed-index names and the
skipped schemas are reported before the command exits, so one never hides the other.
If the database connection itself is lost mid-sweep, the run stops there rather than
repeating the same error for every remaining schema, and still reports what it did.

Examples:

# See which banks are missing coverage without changing anything
hindsight-admin repair-bank --all --dry-run

# Repair every bank across all schemas (run once after a restore/upgrade)
hindsight-admin repair-bank --all

# Repair a single bank
hindsight-admin repair-bank --bank acme-prod

rename-bank Copy Section

:::tip Consider an alias instead

A rename requires stopping the bank's clients, because the old id stops working the moment it commits. If what you want is to move callers onto a new id, bank aliases do that with no downtime and no row rewrite: the bank answers to both ids until you are done.

Reach for rename-bank when the old id must genuinely stop existing — an alias never removes one.

:::

Change a bank's id in place. Every memory, document, entity, mental model and knowledge page is kept exactly as it is. Nothing is re-extracted or re-embedded, and no LLM calls are made.

hindsight-admin rename-bank --from OLD_ID --to NEW_ID [OPTIONS]

Options:

Before you run it:

• Stop the bank's clients. Anything still calling the old id gets a 404, or, if it creates banks on demand, silently starts a new, empty bank under the old id. Point every client, and any API key scoped to the bank, at the new id before restarting them.
• Let the bank's operations finish. The command refuses while the bank has pending or processing operations.

A file's storage key spells the bank id, so after the rows move the bank's stored files — document attachments and uploaded originals — are copied to keys under the new id, their rows repointed, and the old id's prefix cleared. That last step also removes any export or import archive left under the old id: their download links stop working at the rename, and the operation can produce them again. Deleting the bank afterwards removes its files with it.

The rename runs in a single transaction and only locks that bank's rows. Other banks keep serving. Its duration grows with the bank's size. Afterwards, the bank's vector indexes are rebuilt with CREATE INDEX CONCURRENTLY, and until that finishes, recall on the renamed bank runs without them. Running API servers may keep answering for the old id until their bank-info cache expires. PostgreSQL only.

Examples:

# See what would move, change nothing
hindsight-admin rename-bank --from acme --to acme-prod --dry-run

hindsight-admin rename-bank --from acme --to acme-prod

backup Copy Section

Create a backup of all Hindsight data to a zip file.

hindsight-admin backup OUTPUT [OPTIONS]

Arguments:

Options:

Examples:

# Backup to a file
hindsight-admin backup /backups/hindsight-2024-01-15.zip

# Backup a specific tenant schema
hindsight-admin backup /backups/tenant-acme.zip --schema tenant_acme

The backup includes:

• Memory banks and their configuration
• Documents and chunks
• Entities and their relationships
• Memory units (facts, experiences, observations)
• Entity cooccurrences and memory links
• Mental models and directives
• Webhooks and file storage
• Internal operational tables (async operations, audit log, graph-maintenance queue, and similar bookkeeping) so a restore reproduces a faithful full-database snapshot

:::note Consistency
Backups are created within a database transaction with REPEATABLE READ isolation, ensuring a consistent snapshot across all tables.
:::

restore Copy Section

Restore data from a backup file. Warning: This deletes all existing data in the target schema.

hindsight-admin restore INPUT [OPTIONS]

Arguments:

Options:

Examples:

# Restore with confirmation prompt
hindsight-admin restore /backups/hindsight-2024-01-15.zip

# Restore without confirmation (for scripts)
hindsight-admin restore /backups/hindsight-2024-01-15.zip --yes

# Restore to a specific tenant schema
hindsight-admin restore /backups/tenant-acme.zip --schema tenant_acme --yes

:::warning Data Loss
Restore will delete all existing data in the target schema before importing the backup. Always verify you have a recent backup before performing a restore.
:::

decommission-worker Copy Section

Release all tasks owned by a worker, resetting them from "processing" back to "pending" status so they can be picked up by other workers.

hindsight-admin decommission-worker WORKER_ID [OPTIONS]

Arguments:

Options:

Examples:

# Before scaling down - release tasks from workers being removed
hindsight-admin decommission-worker hindsight-worker-4
hindsight-admin decommission-worker hindsight-worker-3

# Release tasks from a crashed worker
hindsight-admin decommission-worker worker-2

# For a specific tenant schema
hindsight-admin decommission-worker worker-1 --schema tenant_acme

When to Use:

• Scaling down: Before removing worker replicas in Kubernetes
• Graceful removal: When taking a worker offline for maintenance
• Crash recovery: If a worker crashed while processing tasks
• Stuck worker: When a worker is unresponsive

:::tip Finding Worker IDs
Worker IDs default to the hostname. In Kubernetes StatefulSets, this is the pod name (e.g., hindsight-worker-0). You can also set a custom ID with HINDSIGHT_API_WORKER_ID or --worker-id.
:::

decommission-workers Copy Section

Release all currently-processing tasks from every worker, resetting them from "processing" back to "pending" status. Use this when one or more workers have crashed or been removed without graceful shutdown and you don't know which worker IDs to target.

hindsight-admin decommission-workers [OPTIONS]

Options:

Examples:

# Release all processing tasks across all workers (with confirmation)
hindsight-admin decommission-workers

# Skip the confirmation prompt (useful in scripts)
hindsight-admin decommission-workers --yes

# Release tasks in a specific tenant schema
hindsight-admin decommission-workers --schema tenant_acme

When to Use:

• Unknown dead workers: Multiple workers crashed and you do not know their IDs
• Fleet-wide recovery: After an infrastructure event where many workers went down
• "Just fix everything": A quick full-queue drain when per-worker cleanup is overkill

:::warning Disruptive
This releases every processing task regardless of worker, including tasks owned by healthy workers. Prefer decommission-worker <WORKER_ID> when you know which workers need cleanup.
:::

worker-status Copy Section

Show all currently-processing tasks grouped by worker, including operation type, bank, how long each task has been running, and when it was last updated. Useful for identifying orphaned tasks before decommissioning.

hindsight-admin worker-status [OPTIONS]

Options:

Examples:

# Show all processing tasks across all workers
hindsight-admin worker-status

# Show processing tasks for a specific tenant schema
hindsight-admin worker-status --schema tenant_acme

When to Use:

• Before decommissioning: Inspect which workers have stale tasks and how long they have been stuck
• Debugging throughput: Diagnose why the queue is not draining (are tasks stuck in processing?)
• Worker health check: Spot workers whose last_update_ago keeps growing, indicating a dead or unresponsive worker

export-bank Copy Section

Export an entire bank to a portable ZIP archive — documents, facts, observations, bank configuration, mental models, directives, and webhooks. Embeddings are never included; they are regenerated on import. This is the source half of a cross-instance migration (e.g. moving to a different embedding model, vector extension, or text-search backend). PostgreSQL only.

hindsight-admin export-bank --bank <BANK_ID> --output <FILE.zip> [OPTIONS]

Options:

Examples:

hindsight-admin export-bank --bank my-bank --output my-bank.zip

# include operational history
hindsight-admin export-bank --bank my-bank --output my-bank.zip --include-history

Read-only — safe to run against a live instance.

import-bank Copy Section

Restore a whole-bank archive (produced by export-bank) into this instance. Facts are re-embedded with this instance's configured embedding model and links/indexes are rebuilt; bank configuration, mental models, directives, and webhooks are restored exactly. No LLM fact-extraction runs, and because a migration restores state, it does not fire webhooks or re-run consolidation. PostgreSQL only.

hindsight-admin import-bank --archive <FILE.zip> [OPTIONS]

Options:

Examples:

hindsight-admin import-bank --archive my-bank.zip

Run this against an instance configured with the target embedding model / vector extension / text-search backend — that's what re-embedding uses.

:::warning Target bank must not exist
Import restores a whole bank (config, facts, mental models, …) — it is not a merge. If a bank with the target id already exists, the command fails. Delete that bank first, or use --target-bank to restore under a fresh id.
:::

Migrating a bank to a new instance Copy Chapter

Changing a bank's embedding model (e.g. a 384-dim encoder → a 1024-dim one), vector extension (pgvector / vchord / pgvectorscale), or text-search backend can't be done in place on a populated bank — the stored vectors and indexes are tied to those settings. Because every embedding and index is a deterministic function of text already on disk, the supported path is to move the bank to a fresh instance configured with the new settings and re-derive everything there — with no LLM re-extraction.

export-bank / import-bank carry documents, facts, observations, bank config, mental models, directives, and webhooks — but never embeddings, which the target instance regenerates with its own model.

Blue-green runbook:

1. Stand up a new instance on a fresh database, configured with the new embedding model / vector extension / text-search backend.
2. Quiesce writes to the source bank (maintenance window) and run hindsight-admin backup for safety.
3. Export from the source, then import into the target:
   bash Copy

   # on the source instance:
   hindsight-admin export-bank --bank my-bank --output my-bank.zip
   # on the target instance (configured with the new settings):
   hindsight-admin import-bank --archive my-bank.zip

4. Verify on the target: run representative recall queries and compare results.
5. Cut traffic over to the new instance. The old instance stays as an instant rollback until you're confident.

:::note Why a new instance, not in-place
The embedding model is server-level, and a bank's memory_units.embedding column has a single dimension shared across the schema, so a different-dimension or different-backend bank needs its own instance/database. The old vectors are never mutated, which makes rollback trivial.
:::

Recovering stuck or zombie operations Copy Chapter

A "zombie" operation is one stuck in processing indefinitely because the worker that claimed it is gone. The most common cause is an unstable HINDSIGHT_API_WORKER_ID: when it defaults to the container hostname, a Docker restart produces a new container ID, the new worker doesn't recognize the old worker's claims as its own, and those tasks are stranded.

How to spot them:

# List processing tasks grouped by worker — workers with a growing last_update_ago are dead
hindsight-admin worker-status

# Bank-level counters; pending_consolidation that never decreases is the usual symptom
curl -s http://localhost:8888/v1/default/banks/<bank_id>/stats

How to recover:

# You know which worker is dead (e.g. from worker-status):
hindsight-admin decommission-worker <old-worker-id>

# You don't know — release every processing task across the fleet:
hindsight-admin decommission-workers

Both commands reset processing rows back to pending so a live worker can claim them on the next poll.

How to prevent it:

Set HINDSIGHT_API_WORKER_ID to a stable value so worker identity survives restarts:

• Docker: pass -e HINDSIGHT_API_WORKER_ID=hindsight-prod (or per-replica names if running multiple containers)
• Kubernetes (Helm): the chart's StatefulSet uses the pod name automatically — no extra config needed
• Bare metal / pip: pass --worker-id <name> or set the env var per process

See Installation - Docker and Configuration - Distributed Workers.

Environment Variables Copy Chapter

The admin CLI uses the same environment variables as the API service. The most important one is:

Example:

# Use a specific database
export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@localhost:5432/hindsight
hindsight-admin backup /backups/mybackup.zip

Hindsight Docs Docs Developer Configuration (`hindsight-docs/docs/developer/configuration.mdx`) Copy Section

import CodeSnippet from '@site/src/components/CodeSnippet';
import configurationPy from '!!raw-loader!@site/examples/api/configuration.py';
import configurationSh from '!!raw-loader!@site/examples/api/configuration.sh';

Configuration

Complete reference for configuring Hindsight services through environment variables.

Hindsight has two services, each with its own configuration prefix:

API Service Copy Chapter

The API service handles all memory operations (retain, recall, reflect).

Database Copy Section

If not provided, the server uses embedded pg0 — convenient for development but not recommended for production.

To tune the embedded database, add PostgreSQL settings as a query string: pg0://hindsight?max_connections=300&shared_buffers=256MB. They apply when the embedded database starts, so restart it after changing them.

To run against Oracle Database 23ai instead, set HINDSIGHT_API_DATABASE_BACKEND=oracle and use an oracle+oracledb://… URL. See the Oracle Database guide for full setup instructions.

The DATABASE_SCHEMA setting allows you to use a custom PostgreSQL schema instead of the default public schema. This is useful for:

• Multi-database setups where you want Hindsight tables in a dedicated schema
• Hosting platforms (e.g., Supabase) where public schema is reserved or shared
• Organizational preferences for schema naming conventions

# Example: Using a custom schema
export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/dbname
export HINDSIGHT_API_DATABASE_SCHEMA=hindsight

Migrations will automatically create the schema if it doesn't exist and create all tables in the configured schema.

Owning a maintenance routine

The background maintenance loop finds work by calling four PostgreSQL routines —
mental_models_with_cron, banks_needing_consolidation, schemas_with_expired_rows and
schemas_with_expired_operations. The stock implementations scan every schema in the
database on each tick. On a large multi-tenant install that is a serial loop over
thousands of schemas, repeated once per tick in every process running the loop.

Because they are called by name, you can replace one with an implementation suited to
your installation — typically a registry table maintained by a row trigger, so discovery
costs one indexed read instead of a scan. List the ones you have replaced and migrations
will leave them alone:

export HINDSIGHT_API_EXTERNALLY_OWNED_ROUTINES=mental_models_with_cron,banks_needing_consolidation

Read from the environment of whatever process runs the migration, so it applies equally to
hindsight-admin run-db-migration, a migration Job, and migrate-on-startup.

Without this, your replacement survives only until the next migration that reinstalls the
routine: they are all CREATE OR REPLACE against the same name, so the migration wins and
nothing records that a custom implementation was discarded.

A few things to know:

• Your replacement must match the stock signature. The loop calls these with fixed
  arguments and reads fixed columns back.
• Nothing verifies that a replacement exists. Naming a routine you have not installed
  leaves whatever was there before — nothing, on a fresh database — and the maintenance
  loop then fails on the missing function rather than quietly running the scan you were
  trying to avoid.
• Ownership is not stamped into the database. Remove a name from the list and the next
  migration reinstalls the stock routine, with no data fix-up.
• Empty by default. An installation that has not replaced anything is unaffected.

Database Connection Pool Copy Section

For high-concurrency workloads, increase DB_POOL_MAX_SIZE. Each concurrent recall/think operation can use 2-4 connections.

To run migrations manually (e.g., before starting the API), use the admin CLI:

# Migrate the base schema plus all discovered tenant schemas
hindsight-admin run-db-migration

# Or migrate a specific schema only:
hindsight-admin run-db-migration --schema tenant_acme

Vector Extension Copy Section

Hindsight supports four PostgreSQL vector extensions:

**pgvector** (HNSW - default)

• In-memory index using Hierarchical Navigable Small World algorithm
• Works well for most embeddings and dataset sizes
• Fast for small-medium datasets (<10M vectors)
• Higher memory usage for large datasets
• Most widely deployed and supported

**pgvectorscale** (DiskANN - recommended for scale) ⭐

• Disk-based index using StreamingDiskANN algorithm
• 28x lower p95 latency and 16x higher throughput vs dedicated vector DBs
• 60-75% cost reduction at scale (SSDs cheaper than RAM)
• Superior filtering performance with streaming retrieval model
• Optimized for large datasets (10M+ vectors)
• Supports both pgvectorscale (open source) and pg_diskann (Azure)
• Installation:
  • Open source/self-hosted: CREATE EXTENSION vector; CREATE EXTENSION vectorscale CASCADE;
  • Azure PostgreSQL: CREATE EXTENSION vector; CREATE EXTENSION pg_diskann CASCADE;

**vchord** (vchordrq)

• Alternative high-performance vector index
• Optimized for high-dimensional embeddings (3000+ dimensions)
• Includes integrated BM25 search capabilities
• Requires vchord extension

**scann** (AlloyDB ScaNN)

• Google's ScaNN index, available on AlloyDB and AlloyDB Omni
• Uses a single global vector index in AUTO mode (per-bank partial indexes are not used)
• Installation: CREATE EXTENSION vector; CREATE EXTENSION alloydb_scann CASCADE;
• Index build is deferred until a table reaches 10,000 populated embedding rows — AlloyDB cannot build a ScaNN AUTO index on a near-empty table. Until that threshold is crossed, recall falls back to a sequential scan; the global index is built on the next API startup once enough rows exist.
• A ready-to-use Docker Compose stack is provided at docker/docker-compose/alloydb/docker-compose.yaml for running Hindsight against AlloyDB Omni locally.

When to use pgvectorscale (DiskANN):

• Large datasets (10M+ vectors) ⭐
• Complex filtering requirements
• Cost-sensitive deployments
• Production workloads requiring high throughput
• When disk I/O is not a bottleneck

When to use pgvector (HNSW):

• Small-medium datasets (<10M vectors)
• Maximum query speed when all data fits in memory
• Simple nearest-neighbor queries without filters
• Standard PostgreSQL deployment preference

When to use vchord:

• High-dimensional embeddings (3000+ dimensions)
• Want integrated BM25 search
• Already using vchord for text search

When to use scann:

• Running on Google AlloyDB or AlloyDB Omni
• Want managed ScaNN with AUTO mode tuning

Limiting vector indexes on large deployments

On pgvector, pgvectorscale and vchord, a bank's memories are indexed per
(bank, fact_type). By default every bank gets its own indexes when it is
created, which is the right thing for most deployments and needs no
configuration.

It stops being the right thing when you have thousands of banks. These indexes
all live on one shared table, and PostgreSQL inspects and locks every index
on a table whenever it plans a query against it — so an index created for one
bank is a cost paid by searches in every other bank. Past a few thousand banks,
planning slows sharply and the server eventually runs out of lock-table space,
failing reads and bank deletion alike.

Setting a minimum size fixes that: a bank only gets its own indexes once it is
large enough to benefit from them. Below the threshold PostgreSQL answers the
same search from the (bank_id, fact_type) B-tree plus a top-N sort, which for
a small bank is both faster and exact rather than approximate — so small banks
lose nothing. The number of indexes becomes proportional to the number of
large banks rather than to the number of banks, and bank count stops being a
limit.

With the threshold off (the default), indexes are created inside the
transaction that creates the bank — instantly, because the bank is empty — and
dropped when the bank is deleted. Nothing inspects bank sizes, and no background
operation runs.

With a threshold set, bank creation, ingestion and import build no indexes at
all, so no request ever waits on index DDL. Instead, a write that could move a
bank across the threshold queues a background vector_index_maintenance
operation, which builds an index when a bank grows past the threshold and removes
it if the bank shrinks well below it (the gap between those two points prevents
churn at the boundary). You will see these operations in the bank's operations
list.

To reconcile without waiting for a write — after a restore, an upgrade, or an
extension switch — run:

hindsight-admin repair-bank --all

This works in both modes: with the threshold off it rebuilds any index a bank is
missing, and with one set it also drops what a bank no longer earns.

Switching extensions:

If you need to switch from one extension to another:

1. Set HINDSIGHT_API_VECTOR_EXTENSION to your desired extension (pgvector, vchord, pgvectorscale, or scann)
2. If your database has existing data, you'll get an error with migration instructions (note: switching to scann is allowed even with data — the existing index is dropped and rebuilt as ScaNN once the table has at least 10,000 embedding rows)
3. For empty databases, indexes will be automatically recreated on startup

Learn more:

• HNSW vs. DiskANN comparison
• pgvectorscale GitHub

Text Search Extension Copy Section

Hindsight supports five backends for BM25 keyword retrieval:

• native — PostgreSQL's built-in full-text search (tsvector + GIN). Language configurable.
• vchord — VectorChord BM25 (uses the llmlingua2 multilingual tokenizer).
• pg_textsearch — Timescale's pg_textsearch extension. English-only.
• pgroonga — pgroonga full-text search. Multilingual / CJK out of the box.
• pg_search — ParadeDB pg_search. True BM25; the only backend that is Citus-compatible.

A bank can also use none of them: enable_text_search
switches the keyword arm off per bank, leaving pure vector search. Every setting in
this table is then unused by that bank.

To switch backends: set HINDSIGHT_API_TEXT_SEARCH_EXTENSION. With existing data, you'll get an error and migration instructions; with an empty database the columns/indexes are recreated automatically on startup.

HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER only applies when HINDSIGHT_API_TEXT_SEARCH_EXTENSION=pg_search, and only when BM25 indexes are created. Changing it for an existing database requires rebuilding the pg_search indexes or recreating the database. Supported values are empty/unset, unicode_words, simple, whitespace, literal, literal_normalized, chinese_compatible, icu, jieba, source_code, chinese_lindera/lindera(chinese), japanese_lindera/lindera(japanese), korean_lindera/lindera(korean), ngram(min,max), and edge_ngram(min,max).

For non-English banks (especially CJK) and the language/extraction-language tradeoffs, see the Multilingual Support page.

LLM Provider Copy Section

# Groq (recommended for fast inference)
export HINDSIGHT_API_LLM_PROVIDER=groq
export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
export HINDSIGHT_API_LLM_MODEL=openai/gpt-oss-20b
# For free tier users: override to on_demand if you get service_tier errors
# export HINDSIGHT_API_LLM_GROQ_SERVICE_TIER=on_demand

# OpenAI
export HINDSIGHT_API_LLM_PROVIDER=openai
export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
export HINDSIGHT_API_LLM_MODEL=gpt-4o
# Optional: Use Flex Processing for 50% cost savings (with variable latency)
# export HINDSIGHT_API_LLM_OPENAI_SERVICE_TIER=flex

# OpenAI Responses API (/v1/responses)
export HINDSIGHT_API_LLM_PROVIDER=openai-responses
export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
export HINDSIGHT_API_LLM_MODEL=gpt-5.6
# Uses the OpenAI Responses API instead of chat/completions. Unlike
# chat/completions, it supports reasoning together with function tools, so
# reflect's tool-calling search loop can run with a real reasoning effort.
# Recommended for reasoning models (gpt-5.x, o-series) that reject
# `reasoning_effort` alongside tools on chat/completions (e.g. gpt-5.6-terra).
# export HINDSIGHT_API_LLM_REASONING_EFFORT=high

# Gemini
export HINDSIGHT_API_LLM_PROVIDER=gemini
export HINDSIGHT_API_LLM_API_KEY=xxxxxxxxxxxx
export HINDSIGHT_API_LLM_MODEL=gemini-2.0-flash
# Optional: Use Gemini Flex for 50% cost savings (best-effort inference)
# export HINDSIGHT_API_LLM_GEMINI_SERVICE_TIER=flex

# Anthropic
export HINDSIGHT_API_LLM_PROVIDER=anthropic
export HINDSIGHT_API_LLM_API_KEY=sk-ant-xxxxxxxxxxxx
export HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514
# Or, with no static key: Workload Identity Federation. Leave HINDSIGHT_API_LLM_API_KEY
# unset and the Anthropic SDK trades the platform identity token for short-lived
# access tokens, refreshing them on its own.
# export ANTHROPIC_FEDERATION_RULE_ID=fdrl_xxxxxxxxxxxx
# export ANTHROPIC_ORGANIZATION_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
# export ANTHROPIC_SERVICE_ACCOUNT_ID=svac_xxxxxxxxxxxx
# export ANTHROPIC_IDENTITY_TOKEN_FILE=/var/run/secrets/anthropic/token  # e.g. a projected k8s service-account token

# Vertex AI (Google Cloud - uses native genai SDK)
export HINDSIGHT_API_LLM_PROVIDER=vertexai
export HINDSIGHT_API_LLM_MODEL=gemini-2.0-flash-001
export HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=your-gcp-project-id
export HINDSIGHT_API_LLM_VERTEXAI_REGION=us-central1
# Optional: use ADC (gcloud auth application-default login) or provide service account key:
# export HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json

# Ollama (local, no API key)
export HINDSIGHT_API_LLM_PROVIDER=ollama
export HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
export HINDSIGHT_API_LLM_MODEL=llama3

# Ollama Cloud (hosted Ollama endpoint, requires API key)
export HINDSIGHT_API_LLM_PROVIDER=ollama-cloud
export HINDSIGHT_API_LLM_API_KEY=your-ollama-cloud-api-key
export HINDSIGHT_API_LLM_MODEL=gemma3:12b

# LM Studio (local, no API key)
export HINDSIGHT_API_LLM_PROVIDER=lmstudio
export HINDSIGHT_API_LLM_BASE_URL=http://localhost:1234/v1
export HINDSIGHT_API_LLM_MODEL=your-local-model

# llama.cpp (built-in local inference, no external server needed)
export HINDSIGHT_API_LLM_PROVIDER=llamacpp
# No API key, base URL, or external server required.
# Auto-downloads Gemma 4 E2B (~3.5 GB GGUF) on first run.
# See "Built-in llama.cpp" section below for all configuration options.

# OpenAI-compatible endpoint (Chat Completions API, /v1/chat/completions)
export HINDSIGHT_API_LLM_PROVIDER=openai
export HINDSIGHT_API_LLM_BASE_URL=https://your-endpoint.com/v1
export HINDSIGHT_API_LLM_API_KEY=your-api-key
export HINDSIGHT_API_LLM_MODEL=your-model-name

# OpenAI-compatible endpoint (Responses API, /v1/responses)
# Same as above but targets /v1/responses instead of /v1/chat/completions.
export HINDSIGHT_API_LLM_PROVIDER=openai-responses
export HINDSIGHT_API_LLM_BASE_URL=https://your-endpoint.com/v1
export HINDSIGHT_API_LLM_API_KEY=your-api-key
export HINDSIGHT_API_LLM_MODEL=your-model-name

# OpenAI Codex (ChatGPT Plus/Pro subscription - uses OAuth, no API key needed)
export HINDSIGHT_API_LLM_PROVIDER=openai-codex
export HINDSIGHT_API_LLM_MODEL=gpt-5.6-luna
# No API key needed - uses OAuth tokens from ~/.codex/auth.json
# For long-running services, set CODEX_HOME to a dedicated auth directory so
# Hindsight doesn't share (and lose) its refresh token with another Codex process.
# See Models docs → "Isolating Codex auth for long-running services".
# export CODEX_HOME=/var/lib/hindsight/codex
# Per-provider override of CODEX_HOME. Set it (with the indexed
# HINDSIGHT_API_LLM_<n>_CODEX_HOME below) to run two independently authorized
# ChatGPT profiles in one process.
# export HINDSIGHT_API_LLM_CODEX_HOME=/var/lib/hindsight/codex-a

# Claude Code (Claude Pro/Max subscription - uses OAuth)
export HINDSIGHT_API_LLM_PROVIDER=claude-code
export HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-5-20250929
# No API key needed - uses `claude auth login` credentials.
# Set HINDSIGHT_API_LLM_API_KEY to a token from `claude setup-token` to authenticate with a key instead.

# Cursor (Cursor subscription - drives the cursor-agent CLI in headless mode)
export HINDSIGHT_API_LLM_PROVIDER=cursor
export HINDSIGHT_API_LLM_MODEL=auto   # or any id from `cursor-agent --list-models`
# No API key needed - uses `cursor-agent login` credentials.
# Set HINDSIGHT_API_LLM_API_KEY (or CURSOR_API_KEY) to authenticate with a key instead.
# An agent CLI turn takes 15-30s, well past the 30s reflect default, so raise the
# timeouts or reflect burns its first iteration on a timeout retry.
export HINDSIGHT_API_LLM_TIMEOUT=300
export HINDSIGHT_API_REFLECT_LLM_TIMEOUT=180
# The CLI exposes no response-format or tool-definition flags, so Hindsight emulates
# structured output and tool calling through the prompt and parses the JSON back. That
# is less reliable than a native schema: prefer a stronger named model over `auto` for
# reflect, which is the operation that leans hardest on tool calls.

# Volcano Engine (ByteDance - OpenAI-compatible)
export HINDSIGHT_API_LLM_PROVIDER=volcano
export HINDSIGHT_API_LLM_API_KEY=your-api-key
export HINDSIGHT_API_LLM_BASE_URL=https://ark.cn-beijing.volces.com/api/v3
export HINDSIGHT_API_LLM_MODEL=doubao-pro-32k

# OpenRouter (OpenAI-compatible, access 100+ models)
export HINDSIGHT_API_LLM_PROVIDER=openrouter
export HINDSIGHT_API_LLM_API_KEY=your-openrouter-api-key
export HINDSIGHT_API_LLM_MODEL=qwen/qwen3.5-9b
# Structured-output calls (fact extraction, consolidation) send
# provider.require_parameters=true, so OpenRouter only routes them to upstreams
# that support response_format. Any "provider" keys you set in
# HINDSIGHT_API_LLM_EXTRA_BODY (e.g. "only") are kept and take precedence.
# That also means every parameter sent must be supported, temperature included.
# Models that reject temperature (e.g. anthropic/claude-haiku-5.5) then fail with
# "No endpoints found that can handle the requested parameters". Fix: omit it.
# export HINDSIGHT_API_LLM_TEMPERATURE=none
# Reflect forces tool calls: tool_choice "required" after its first searches and a
# named tool on the forced steps. Some upstreams accept only "auto" and fail those
# turns with "No endpoints found that support the provided 'tool_choice' value"
# (reported for several gpt-oss-120b hosts, #5832). Pin ones that support it with
# HINDSIGHT_API_LLM_EXTRA_BODY='{"provider": {"only": ["deepinfra", "cerebras"]}}'.

# Requesty (OpenAI-compatible gateway)
export HINDSIGHT_API_LLM_PROVIDER=requesty
export HINDSIGHT_API_LLM_API_KEY=your-requesty-api-key
export HINDSIGHT_API_LLM_MODEL=openai/gpt-4o-mini

# DeepSeek (OpenAI-compatible, https://api.deepseek.com)
export HINDSIGHT_API_LLM_PROVIDER=deepseek
export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
export HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash
# Notes:
# - `deepseek-v4-flash` defaults to thinking mode at the API level (treated as
#   `deepseek-reasoner`). Hindsight handles this transparently; the reflect
#   agent will not crash with "deepseek-reasoner does not support this
#   tool_choice".
# - Use `deepseek-v4-pro` for the higher-quality reasoning route.
# - Use `deepseek-chat` for the non-thinking alias (faster, cheaper).

# z.ai (Zhipu GLM series, OpenAI-compatible, https://z.ai)
export HINDSIGHT_API_LLM_PROVIDER=zai
export HINDSIGHT_API_LLM_API_KEY=your-zai-api-key
export HINDSIGHT_API_LLM_MODEL=glm-4.5-flash  # or glm-4.5-air for the paid tier
# Default base_url: https://api.z.ai/api/coding/paas/v4 (override with HINDSIGHT_API_LLM_BASE_URL if needed)

# opencode-go (OpenAI-compatible)
export HINDSIGHT_API_LLM_PROVIDER=opencode-go
export HINDSIGHT_API_LLM_API_KEY=your-opencode-go-api-key
export HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash
# Default base_url: https://opencode.ai/zen/go/v1 (override with HINDSIGHT_API_LLM_BASE_URL if needed)

# Meta Model API (Muse Spark, OpenAI-compatible, https://ai.developer.meta.com)
export HINDSIGHT_API_LLM_PROVIDER=meta
export HINDSIGHT_API_LLM_API_KEY=your-meta-model-api-key
export HINDSIGHT_API_LLM_MODEL=muse-spark-1.3
# Default base_url: https://api.meta.ai/v1 (override with HINDSIGHT_API_LLM_BASE_URL if needed)
# Muse Spark always reasons: leave HINDSIGHT_API_LLM_REASONING_EFFORT unset or set it to
# minimal/low/medium/high/xhigh. "none" is rejected with HTTP 400. Reasoning tokens are
# billed against the output budget, so keep the per-operation max-token limits generous.
# Muse Spark reasons before every reply, so calls are slow: reflect's 30s default
# deadline is not enough for its final synthesis and the call fails after retries.
# Raise it (and the global deadline) when using this provider:
export HINDSIGHT_API_REFLECT_LLM_TIMEOUT=300
export HINDSIGHT_API_LLM_TIMEOUT=300

# Nous Portal (OpenAI-compatible; no API key — uses your `hermes portal` login)
export HINDSIGHT_API_LLM_PROVIDER=nous
export HINDSIGHT_API_LLM_MODEL=deepseek/deepseek-v4-flash
# No API key needed — reads a rotating JWT from ~/.hermes/auth.json (run `hermes portal` first).
# Default base_url: https://inference-api.nousresearch.com/v1 (override with HINDSIGHT_API_LLM_BASE_URL if needed)
# See the "Nous Portal Setup" section in the Models guide for the login flow.

# SuperGrok subscription (device-code OAuth; no API key). This is the subscription
# lane, NOT xAI API support — for API-key access to api.x.ai use
# HINDSIGHT_API_LLM_PROVIDER=openai with HINDSIGHT_API_LLM_BASE_URL=https://api.x.ai/v1.
export HINDSIGHT_API_LLM_PROVIDER=xai-oauth
export HINDSIGHT_API_LLM_MODEL=grok-4.5
# No API key needed — reads an OAuth grant from ~/.hindsight/xai_oauth.json.
# Log in once: python -m hindsight_api.engine.providers.xai_oauth_auth login
# Default base_url: https://api.x.ai/v1
# See the "SuperGrok Subscription Setup" section in the Models guide for the login flow.

# AWS Bedrock (native support - no API key needed, uses AWS credentials)
export HINDSIGHT_API_LLM_PROVIDER=bedrock
export HINDSIGHT_API_LLM_MODEL=us.amazon.nova-2-lite-v1:0
export AWS_ACCESS_KEY_ID=your-access-key
export AWS_SECRET_ACCESS_KEY=your-secret-key
export AWS_REGION_NAME=us-east-1
# Optional: Use Flex tier for 50% cost savings (with variable latency)
# export HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER=flex

# LiteLLM (100+ providers via LiteLLM SDK)
# Azure OpenAI via LiteLLM
export HINDSIGHT_API_LLM_PROVIDER=litellm
export HINDSIGHT_API_LLM_API_KEY=your-azure-api-key
export HINDSIGHT_API_LLM_MODEL=azure/gpt-4o

# Together AI via LiteLLM
export HINDSIGHT_API_LLM_PROVIDER=litellm
export HINDSIGHT_API_LLM_API_KEY=your-together-api-key
export HINDSIGHT_API_LLM_MODEL=together_ai/meta-llama/Llama-3-70b-chat-hf

# No LLM (chunk storage + semantic search only, no API key needed)
export HINDSIGHT_API_LLM_PROVIDER=none
# Retain automatically uses chunks mode (no fact extraction)
# Recall works normally (semantic search, BM25, graph retrieval)
# Reflect returns HTTP 400 (requires an LLM)
# Consolidation/observations are disabled

/* Detailed source-code truncated for AI context efficiency. */

export HINDSIGHT_API_LLM_PROVIDER=litellmrouter
export HINDSIGHT_API_LLM_LITELLMROUTER_CONFIG='{
  "model_list": [
    {"model_name": "default",  "litellm_params": {"model": "openai/gpt-4o-mini", "api_key": "sk-..."}},
    {"model_name": "fallback", "litellm_params": {"model": "anthropic/claude-sonnet-4-5", "api_key": "sk-ant-..."}}
  ],
  "fallbacks": [{"default": ["fallback"]}]
}'

/* Detailed source-code truncated for AI context efficiency. */

# Primary OpenAI, failover to Groq then Anthropic
export HINDSIGHT_API_LLM_PROVIDER=openai
export HINDSIGHT_API_LLM_API_KEY=sk-...
export HINDSIGHT_API_LLM_1_PROVIDER=groq
export HINDSIGHT_API_LLM_1_API_KEY=gsk-...
export HINDSIGHT_API_LLM_2_PROVIDER=anthropic
export HINDSIGHT_API_LLM_2_API_KEY=sk-ant-...
export HINDSIGHT_API_LLM_STRATEGY='{"mode": "failover"}'

# Weighted round-robin: serve the primary 3x as often as member 1
export HINDSIGHT_API_LLM_STRATEGY='{"mode": "round-robin", "weights": [3, 1]}'

# Failover, re-probing a failed member after 5 minutes (then 10, 20, ... up to 1h)
export HINDSIGHT_API_LLM_STRATEGY='{"mode": "failover", "cooldown_seconds": 300}'

Metadata routing

{"mode": "metadata"} sends each retained item to the chain member its own metadata selects, so one deployment can extract different documents with different models:

export HINDSIGHT_API_LLM_PROVIDER=openai
export HINDSIGHT_API_LLM_API_KEY=sk-...
export HINDSIGHT_API_LLM_1_PROVIDER=ollama
export HINDSIGHT_API_LLM_1_MODEL=qwen3:8b
export HINDSIGHT_API_LLM_STRATEGY='{
  "mode": "metadata",
  "routes": [{"key": "classification", "value": "sensitive", "member": 1}]
}'

/* Detailed source-code truncated for AI context efficiency. */

# Minimal setup (auto-downloads model, uses GPU)
export HINDSIGHT_API_LLM_PROVIDER=llamacpp

# Custom model with tuning
export HINDSIGHT_API_LLM_PROVIDER=llamacpp
export HINDSIGHT_API_LLM_MAX_CONCURRENT=2
export HINDSIGHT_API_LLAMACPP_MODEL_PATH=~/.hindsight/models/my-model.gguf
export HINDSIGHT_API_LLAMACPP_CONTEXT_SIZE=16384
export HINDSIGHT_API_LLAMACPP_NO_GRAMMAR=true  # faster, less reliable JSON
export HINDSIGHT_API_LLAMACPP_EXTRA_ARGS="--n_threads 8"

/* Detailed source-code truncated for AI context efficiency. */

# Default LLM (used as fallback)
export HINDSIGHT_API_LLM_PROVIDER=openai
export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
export HINDSIGHT_API_LLM_MODEL=gpt-4o

# Use GPT-4o for retain (strong structured output)
export HINDSIGHT_API_RETAIN_LLM_MODEL=gpt-4o

# Use faster/cheaper model for reflect
export HINDSIGHT_API_REFLECT_LLM_PROVIDER=groq
export HINDSIGHT_API_REFLECT_LLM_API_KEY=gsk_xxxxxxxxxxxx
export HINDSIGHT_API_REFLECT_LLM_MODEL=llama-3.3-70b-versatile

Example: Tuning Retry Behavior for Rate-Limited APIs

# For Anthropic with tight rate limits (10k output tokens/minute)
export HINDSIGHT_API_LLM_PROVIDER=anthropic
export HINDSIGHT_API_LLM_API_KEY=sk-ant-xxxxxxxxxxxx
export HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514

# Reduce concurrent requests for retain to avoid rate limits
export HINDSIGHT_API_RETAIN_LLM_MAX_CONCURRENT=3

# Fail faster with fewer retries
export HINDSIGHT_API_RETAIN_LLM_MAX_RETRIES=3

# Or increase backoff times to wait out rate limit windows
export HINDSIGHT_API_RETAIN_LLM_INITIAL_BACKOFF=2.0  # Start at 2s instead of 1s
export HINDSIGHT_API_RETAIN_LLM_MAX_BACKOFF=120.0    # Cap at 2min instead of 1min

/* Detailed source-code truncated for AI context efficiency. */

pip install 'hindsight-api-slim[local-onnx]'
# or, in this repository:
uv sync --project hindsight-api-slim --extra local-onnx

CUDA execution is opt-in. Set HINDSIGHT_API_EMBEDDINGS_ONNX_DEVICE=cuda only in an environment that has a compatible onnxruntime-gpu wheel and CUDA/cuDNN runtime. The official images intentionally keep the CPU runtime and do not grow when this feature is unused. For a Docker deployment, use the ready-to-build recipe at docker/docker-compose/cuda-onnx/, which installs the GPU wheel into a private image based on the slim image:

docker compose -f docker/docker-compose/cuda-onnx/docker-compose.yaml up --build

The recipe accepts BASE_IMAGE, ONNXRUNTIME_GPU_VERSION, and HINDSIGHT_API_EMBEDDINGS_ONNX_CUDA_DEVICE_ID for release, runtime, and device selection. Hindsight fails startup when CUDA is requested but the provider is unavailable or the initialized session does not activate it; it does not silently fall back to CPU in that mode. The recipe installs CUDA/cuDNN libraries in the private image; the host needs a compatible NVIDIA driver and GPU passthrough. Individual operators may still run on CPU through normal graph partitioning. See the recipe README for dependency isolation, unreleased-checkout builds, and real GPU tests.

You can either let Hindsight download the model from Hugging Face at startup by setting HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID, or pre-download the ONNX graph and tokenizer files under the Hindsight repository root.

cd /path/to/hindsight
mkdir -p models

MODEL_ID=intfloat/multilingual-e5-small
MODEL_DIR=models/intfloat__multilingual-e5-small

uv run --project hindsight-api-slim --extra local-onnx python - <<'PY'
import os
from huggingface_hub import snapshot_download

snapshot_download(
    repo_id=os.environ["MODEL_ID"],
    local_dir=os.environ["MODEL_DIR"],
    allow_patterns=[
        "onnx/model.onnx",
        "onnx/model.onnx_data",
        "*.json",
        "*.txt",
        "*.model",
    ],
)
PY

Then start Hindsight with paths relative to the repository root:

export HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx
export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH=./models/intfloat__multilingual-e5-small/onnx/model.onnx
export HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH=./models/intfloat__multilingual-e5-small
export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: "
export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: "

For Docker deployments, mount the same model directory and use container paths:

services:
  hindsight:
    volumes:
      - ./models:/app/models:ro
    environment:
      HINDSIGHT_API_EMBEDDINGS_PROVIDER: onnx
      HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH: /app/models/intfloat__multilingual-e5-small/onnx/model.onnx
      HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH: /app/models/intfloat__multilingual-e5-small
      HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS: "384"
      HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX: "query: "
      HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX: "passage: "

Model-specific examples:

# sentence-transformers/all-MiniLM-L6-v2: 384 dimensions, no E5 prefixes
export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=sentence-transformers/all-MiniLM-L6-v2
export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX=""
export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX=""

# intfloat/multilingual-e5-small: 384 dimensions, keep E5 prefixes
export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=intfloat/multilingual-e5-small
export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: "
export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: "

# sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2: 384 dimensions, no E5 prefixes
export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX=""
export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX=""

# BAAI/bge-m3: 1024 dimensions, no E5 prefixes; keep onnx/model.onnx_data next to model.onnx
export HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=BAAI/bge-m3
export HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=1024
export HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX=""
export HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX=""

/* Detailed source-code truncated for AI context efficiency. */

# Local (default) - uses SentenceTransformers
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
export HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5

# Local with custom model requiring trust_remote_code
# WARNING: Only enable trust_remote_code for models you trust (security risk)
# export HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=your-custom-model
# export HINDSIGHT_API_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE=true

# OpenAI - cloud-based embeddings
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=***  # or reuses HINDSIGHT_API_LLM_API_KEY
export HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small  # 1536 dimensions by default
# export HINDSIGHT_API_EMBEDDINGS_OPENAI_DIMENSIONS=384  # optional reduced output size

# OpenAI Codex OAuth - uses existing ChatGPT/Codex login, no API key needed
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai-codex
export HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small  # 1536 dimensions by default
# export HINDSIGHT_API_EMBEDDINGS_OPENAI_DIMENSIONS=384  # optional reduced output size

# Azure OpenAI - embeddings via Azure endpoint
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
export HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=your-azure-api-key
export HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small
export HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=https://your-resource.openai.azure.com/openai/deployments/your-deployment

# TEI - HuggingFace Text Embeddings Inference (recommended for production)
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=tei
export HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://localhost:8080

# OpenRouter - access 100+ embedding models
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=openrouter
export HINDSIGHT_API_EMBEDDINGS_OPENROUTER_API_KEY=your-openrouter-api-key  # or reuses HINDSIGHT_API_LLM_API_KEY
export HINDSIGHT_API_EMBEDDINGS_OPENROUTER_MODEL=perplexity/pplx-embed-v1-0.6b

# ZeroEntropy - zembed-1
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=zeroentropy
export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_API_KEY=your-api-key
export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_MODEL=zembed-1
export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_DIMENSIONS=1280
# export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_ENCODING_FORMAT=base64  # optional
# export HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_LATENCY=fast  # optional

# NVIDIA NIM - asymmetric retrieval embeddings (sends input_type=query/passage)
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=nvidia
export HINDSIGHT_API_EMBEDDINGS_NVIDIA_API_KEY=nvapi-your-key
# export HINDSIGHT_API_EMBEDDINGS_NVIDIA_MODEL=nvidia/llama-nemotron-embed-vl-1b-v2  # default
# export HINDSIGHT_API_EMBEDDINGS_NVIDIA_DIMENSIONS=1024                            # default

# Cohere - cloud-based embeddings
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=cohere
export HINDSIGHT_API_EMBEDDINGS_COHERE_API_KEY=your-api-key
export HINDSIGHT_API_EMBEDDINGS_COHERE_MODEL=embed-english-v3.0  # 1024 dimensions
# Optional: override output dimensions (for Matryoshka-capable models)
# export HINDSIGHT_API_EMBEDDINGS_COHERE_OUTPUT_DIMENSIONS=512

# Azure-hosted Cohere - embeddings via custom endpoint
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=cohere
export HINDSIGHT_API_EMBEDDINGS_COHERE_API_KEY=your-azure-api-key
export HINDSIGHT_API_EMBEDDINGS_COHERE_MODEL=embed-english-v3.0
export HINDSIGHT_API_EMBEDDINGS_COHERE_BASE_URL=https://your-azure-cohere-endpoint.com

# LiteLLM proxy - unified gateway for multiple providers
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=litellm
export HINDSIGHT_API_EMBEDDINGS_LITELLM_API_BASE=http://localhost:4000
export HINDSIGHT_API_EMBEDDINGS_LITELLM_API_KEY=your-litellm-key  # optional
export HINDSIGHT_API_EMBEDDINGS_LITELLM_MODEL=text-embedding-3-small  # or cohere/embed-english-v3.0
# export HINDSIGHT_API_EMBEDDINGS_LITELLM_DIMENSIONS=1536  # optional: declare the width, skip the startup probe

# Google - Gemini API (API key auth)
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=google
export HINDSIGHT_API_EMBEDDINGS_GEMINI_API_KEY=xxxxxxxxxxxx  # or reuses HINDSIGHT_API_LLM_API_KEY
export HINDSIGHT_API_EMBEDDINGS_GEMINI_MODEL=gemini-embedding-001  # 768 dimensions (default)
# export HINDSIGHT_API_EMBEDDINGS_GEMINI_OUTPUT_DIMENSIONALITY=768  # configurable: 256, 512, 768, 1024, etc.

# Google - Vertex AI auth (auto-detected when project ID is set)
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=google
export HINDSIGHT_API_EMBEDDINGS_GEMINI_MODEL=gemini-embedding-001
export HINDSIGHT_API_EMBEDDINGS_VERTEXAI_PROJECT_ID=your-gcp-project-id  # falls back to HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID
# export HINDSIGHT_API_EMBEDDINGS_VERTEXAI_REGION=us-central1  # falls back to HINDSIGHT_API_LLM_VERTEXAI_REGION
# export HINDSIGHT_API_EMBEDDINGS_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/key.json  # falls back to LLM config, or uses ADC

# LiteLLM SDK - direct API access without proxy server (recommended)
export HINDSIGHT_API_EMBEDDINGS_PROVIDER=litellm-sdk
export HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_API_KEY=your-provider-api-key
export HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_MODEL=cohere/embed-english-v3.0
# Optional: request a specific output dimension when the provider supports it
# export HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_OUTPUT_DIMENSIONS=768
# Optional (any provider): the input-token limit inputs are truncated to.
# Defaults to 8192; set it to your model's real limit, or 0 to send text uncapped
# export HINDSIGHT_API_EMBEDDINGS_MAX_INPUT_TOKENS=8192

# Supported LiteLLM SDK embedding providers:
# - cohere/embed-english-v3.0 (1024 dimensions)
# - openai/text-embedding-3-small (1536 dimensions)
# - together_ai/togethercomputer/m2-bert-80M-8k-retrieval
# - huggingface/sentence-transformers/all-MiniLM-L6-v2
# - voyage/voyage-2

/* Detailed source-code truncated for AI context efficiency. */

# Primary: a self-hosted reranker that is not always up.
export HINDSIGHT_API_RERANKER_PROVIDER=tei
export HINDSIGHT_API_RERANKER_TEI_URL=http://workstation:8081

# Member 1: a hosted reranker to fall back on.
export HINDSIGHT_API_RERANKER_1_PROVIDER=cohere
export HINDSIGHT_API_RERANKER_1_COHERE_API_KEY=...

# Member 2: last resort — no reranking, keep the fusion order rather than fail.
export HINDSIGHT_API_RERANKER_2_PROVIDER=rrf

Ending the chain with rrf makes recall fail open: results come back in the order
the retrieval stages produced, exactly as if no reranker were configured, instead of
the request failing. Without an rrf member (or with every member down) the error still
surfaces, which is the default behaviour.

Notes:

• Members are tried in order on every request; there is no circuit breaker, so a member
  that is down costs its own timeout on each request before the next one is tried. Keep
  the timeouts of an unreliable primary short (HINDSIGHT_API_RERANKER_TEI_HTTP_TIMEOUT
  and the per-provider *_TIMEOUT settings).
• A member that fails to initialize at startup is logged, not fatal — it is retried on
  the next request that reaches it.
• Scores are not comparable across providers (some return calibrated [0, 1] relevance,
  others logits), so a request served by a fallback member can score differently from one
  served by the primary. Failovers are logged at WARNING.
• The indexed members are credential fields — never returned by the bank-config API, and
  server-level only (not per-bank configurable).

:::tip Sizing a TEI reranker

TEI reserves one slot per text in a rerank request and rejects the request outright
once its pool is full, so start it with at least
HINDSIGHT_API_RERANKER_TEI_MAX_CONCURRENT × HINDSIGHT_API_RERANKER_TEI_BATCH_SIZE
slots — 1024 for the defaults above, versus TEI's own default of 512:

text-embeddings-router --max-concurrent-requests 2048 ...

Below that threshold, heavy recall traffic makes the reranker reject work it has the
capacity to do. Rejections are retried with backoff, so an undersized pool shows up as
slower recall rather than errors.

:::

# Local (default) - uses SentenceTransformers CrossEncoder
export HINDSIGHT_API_RERANKER_PROVIDER=local
export HINDSIGHT_API_RERANKER_LOCAL_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2

# Local with custom model requiring trust_remote_code (e.g., jina-reranker-v2)
# WARNING: Only enable trust_remote_code for models you trust (security risk)
export HINDSIGHT_API_RERANKER_PROVIDER=local
export HINDSIGHT_API_RERANKER_LOCAL_MODEL=jinaai/jina-reranker-v2-base-multilingual
export HINDSIGHT_API_RERANKER_LOCAL_TRUST_REMOTE_CODE=true

# TEI - for high-performance inference
export HINDSIGHT_API_RERANKER_PROVIDER=tei
export HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081

# OpenRouter - access reranking models via OpenRouter
export HINDSIGHT_API_RERANKER_PROVIDER=openrouter
export HINDSIGHT_API_RERANKER_OPENROUTER_API_KEY=your-openrouter-api-key  # or reuses HINDSIGHT_API_LLM_API_KEY
export HINDSIGHT_API_RERANKER_OPENROUTER_MODEL=cohere/rerank-v3.5

# Cohere - cloud-based reranking
export HINDSIGHT_API_RERANKER_PROVIDER=cohere
export HINDSIGHT_API_RERANKER_COHERE_API_KEY=your-api-key
export HINDSIGHT_API_RERANKER_COHERE_MODEL=rerank-english-v3.0

# Any Cohere-compatible /rerank endpoint (Azure AI Foundry, Jina, Voyage, self-hosted BGE, etc.)
#
# Setting HINDSIGHT_API_RERANKER_COHERE_BASE_URL switches the `cohere` provider
# off the Cohere SDK and onto a plain HTTP client that speaks the standard
# Cohere rerank wire format:
#   Request:  POST {base_url}  (or {base_url}/rerank, depending on host)
#             Authorization: Bearer <api_key>
#             {"model": "...", "query": "...", "documents": [...], "return_documents": false}
#   Response: {"results": [{"index": 0, "relevance_score": 0.9}, ...]}
#
# Any service implementing this contract works here. For Azure AI Foundry the
# base_url is the full invoke URL; for SiliconFlow you can also use the
# dedicated `siliconflow` provider below.
export HINDSIGHT_API_RERANKER_PROVIDER=cohere
export HINDSIGHT_API_RERANKER_COHERE_API_KEY=your-api-key
export HINDSIGHT_API_RERANKER_COHERE_MODEL=rerank-english-v3.0  # whatever model the endpoint serves
export HINDSIGHT_API_RERANKER_COHERE_BASE_URL=https://your-cohere-compatible-endpoint.com

# ZeroEntropy - cloud-based reranking (state-of-the-art accuracy)
export HINDSIGHT_API_RERANKER_PROVIDER=zeroentropy
export HINDSIGHT_API_RERANKER_ZEROENTROPY_API_KEY=your-api-key
export HINDSIGHT_API_RERANKER_ZEROENTROPY_MODEL=zerank-2  # or zerank-2-small
# export HINDSIGHT_API_RERANKER_ZEROENTROPY_BASE_URL=https://your-custom-endpoint.com  # optional

# SiliconFlow - cloud reranking via SiliconFlow's Cohere-compatible /rerank endpoint
export HINDSIGHT_API_RERANKER_PROVIDER=siliconflow
export HINDSIGHT_API_RERANKER_SILICONFLOW_API_KEY=your-api-key
export HINDSIGHT_API_RERANKER_SILICONFLOW_MODEL=BAAI/bge-reranker-v2-m3
# export HINDSIGHT_API_RERANKER_SILICONFLOW_BASE_URL=https://api.siliconflow.cn/v1  # default

# NVIDIA NIM - native reranking API (POST {base}/retrieval/{model}/reranking)
export HINDSIGHT_API_RERANKER_PROVIDER=nvidia
export HINDSIGHT_API_RERANKER_NVIDIA_API_KEY=nvapi-your-key
# export HINDSIGHT_API_RERANKER_NVIDIA_MODEL=nvidia/llama-nemotron-rerank-vl-1b-v2  # default

# TypeSafe - relevance judged by a typed-decision model (Jev), not a /rerank endpoint
export HINDSIGHT_API_RERANKER_PROVIDER=typesafe
export HINDSIGHT_API_RERANKER_TYPESAFE_API_KEY=your-api-key
# export HINDSIGHT_API_RERANKER_TYPESAFE_MODEL=jev-latest              # default
# export HINDSIGHT_API_RERANKER_TYPESAFE_PRUNE_CANDIDATES=true          # also prune irrelevant candidates

# Alibaba Cloud DashScope - qwen3-rerank via Cohere-compatible /reranks endpoint
export HINDSIGHT_API_RERANKER_PROVIDER=alibaba
export HINDSIGHT_API_RERANKER_ALIBABA_API_KEY=your-dashscope-api-key  # or set DASHSCOPE_API_KEY
export HINDSIGHT_API_RERANKER_ALIBABA_MODEL=qwen3-rerank  # default, can omit

# LiteLLM proxy - unified gateway for multiple reranking providers (requires running LiteLLM proxy server)
export HINDSIGHT_API_RERANKER_PROVIDER=litellm
export HINDSIGHT_API_RERANKER_LITELLM_API_BASE=http://localhost:4000
export HINDSIGHT_API_RERANKER_LITELLM_API_KEY=your-litellm-key  # optional
export HINDSIGHT_API_RERANKER_LITELLM_MODEL=cohere/rerank-english-v3.0  # or voyage/rerank-2, together_ai/...

# LiteLLM SDK - direct API access without proxy (recommended for simplicity)
export HINDSIGHT_API_RERANKER_PROVIDER=litellm-sdk
export HINDSIGHT_API_RERANKER_LITELLM_SDK_API_KEY=your-deepinfra-api-key
export HINDSIGHT_API_RERANKER_LITELLM_SDK_MODEL=deepinfra/Qwen3-reranker-8B  # or cohere/rerank-english-v3.0, etc.

# Google Discovery Engine - cloud-based semantic reranking
export HINDSIGHT_API_RERANKER_PROVIDER=google
export HINDSIGHT_API_RERANKER_GOOGLE_PROJECT_ID=your-gcp-project-id
export HINDSIGHT_API_RERANKER_GOOGLE_SERVICE_ACCOUNT_KEY=/path/to/service-account.json  # optional, uses ADC if unset
export HINDSIGHT_API_RERANKER_GOOGLE_MODEL=semantic-ranker-default-004  # or semantic-ranker-fast-004

# Jina MLX - Apple Silicon native reranking (no GPU/cloud required)
# Model (~1.2 GB) is downloaded automatically from HuggingFace Hub on first use.
export HINDSIGHT_API_RERANKER_PROVIDER=jina-mlx

/* Detailed source-code truncated for AI context efficiency. */

uvx --from "optimum-onnx[onnxruntime]" optimum-cli export onnx \
  --model nlpai-lab/KURE-Reranker-nano --task text-classification ./models/kure-reranker-nano

export HINDSIGHT_API_RERANKER_PROVIDER=onnx
export HINDSIGHT_API_RERANKER_ONNX_MODEL_PATH=./models/kure-reranker-nano/model.onnx
export HINDSIGHT_API_RERANKER_ONNX_TOKENIZER_NAME_OR_PATH=./models/kure-reranker-nano

/* Detailed source-code truncated for AI context efficiency. */

export HINDSIGHT_API_DECISION_MODEL_PROVIDER=typesafe
export HINDSIGHT_API_DECISION_MODEL_API_KEY=your-typesafe-key
export HINDSIGHT_API_REFLECT_DECISION_MODEL_MODEL=jev-preview        # reflect only
export HINDSIGHT_API_CONSOLIDATION_DECISION_MODEL_PROVIDER=openai    # consolidation on OpenAI
export HINDSIGHT_API_CONSOLIDATION_DECISION_MODEL_API_KEY=your-openai-key

/* Detailed source-code truncated for AI context efficiency. */

# Enable the built-in API key authentication
export HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension
export HINDSIGHT_API_TENANT_API_KEY=your-secret-api-key

When enabled, all requests must include the API key in the Authorization header:

curl -H "Authorization: Bearer your-secret-api-key" \
  http://localhost:8888/v1/default/banks

Requests without a valid API key receive a 401 Unauthorized response.

If you are enabling Memory Defense, see docs/developer/memory-defense/ for the policy schema, detector catalog, and audit trail.

Forwarding extra headers to extensions

A custom TenantExtension normally only sees the Authorization header, as RequestContext.api_key. That is not enough when the bearer token identifies the proxy rather than the caller — for example behind an authenticating gateway that presents one shared service identity to Hindsight, and carries the per-caller identity in a second header.

List those headers to have them forwarded:

export HINDSIGHT_API_EXTENSION_PASSTHROUGH_HEADERS=x-user-assertion,x-request-origin

/* Detailed source-code truncated for AI context efficiency. */

{
  "content": [
    {"type": "text",  "text": "To reset the VPN, click the button shown:"},
    {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "..."}},
    {"type": "text",  "text": "...then reconnect."},
    {"type": "file",  "source": {"type": "base64", "media_type": "application/pdf", "data": "..."},
                      "filename": "escalation-policy.pdf"}
  ]
}

/* Detailed source-code truncated for AI context efficiency. */

# Fireworks batch retain (50% cost savings, async only)
export HINDSIGHT_API_RETAIN_LLM_PROVIDER=fireworks
export HINDSIGHT_API_RETAIN_LLM_API_KEY=fw_xxxxxxxxxxxx
export HINDSIGHT_API_RETAIN_LLM_MODEL=accounts/fireworks/models/llama-v3p1-8b-instruct
export HINDSIGHT_API_FIREWORKS_ACCOUNT_ID=your-account-id
export HINDSIGHT_API_RETAIN_BATCH_ENABLED=true

/* Detailed source-code truncated for AI context efficiency. */

export HINDSIGHT_API_STORE_DOCUMENT_TEXT=false

/* Detailed source-code truncated for AI context efficiency. */

export HINDSIGHT_API_RETAIN_MISSION="Focus on technical decisions, architecture choices, and team member expertise. Deprioritize social or personal information."

HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbose — more detail per fact

Use when you need richer facts with full context, relationships, and verbosity. Slower and uses more tokens than concise.

HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbatim — store chunks as-is

Each chunk is stored as a single memory unit with its original text preserved exactly — no summarization or rewriting. The LLM still runs to extract entities, temporal information, and location so the chunk is fully indexed and retrievable. Useful for RAG-style indexing, document ingestion pipelines, or benchmarks where you want the original text in memory rather than LLM-generated summaries.

export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbatim

retain_strategies / retain_default_strategy — per-call extraction strategy

Named strategies let you ingest different content types into the same bank using different extraction settings. A strategy is a set of hierarchical field overrides applied on top of the resolved bank config.

Any field in the hierarchical config can be overridden per strategy, including retain_extraction_mode, retain_chunk_size, retain_structured_chunk_size, entity_labels, entities_allow_free_form, retain_mission, etc.

Configure strategies via the bank config API:

{
  "retain_default_strategy": "conversations",
  "retain_strategies": {
    "conversations": {
      "retain_extraction_mode": "concise",
      "retain_chunk_size": 3000,
      "retain_structured_chunk_size": 12000
    },
    "documents": {
      "retain_extraction_mode": "chunks",
      "retain_chunk_size": 800,
      "entity_labels": null,
      "entities_allow_free_form": false
    }
  }
}

Then specify the strategy at retain time:

export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=chunks

export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom
export HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS="ONLY extract facts that are:
✅ Technical decisions and their rationale
✅ Architecture patterns and design choices
✅ Performance metrics and benchmarks

DO NOT extract:
❌ Greetings or social conversation
❌ Process chatter (\"let me check\", \"one moment\")
❌ Anything that would not be useful in 6 months"

# Server default: try iris first, fall back to markitdown if iris fails
export HINDSIGHT_API_FILE_PARSER=iris,markitdown

# Restrict what clients may request (optional — defaults to all registered parsers)
export HINDSIGHT_API_FILE_PARSER_ALLOWLIST=markitdown,iris

// Per-request override (in the JSON body of the file retain endpoint)
{
  "parser": "iris",
  "files_metadata": [
    { "document_id": "report" },
    { "document_id": "fallback_doc", "parser": ["iris", "markitdown"] }
  ]
}

# Configure a dedicated OpenAI-compatible OCR/vision endpoint for MarkItDown OCR
export HINDSIGHT_API_FILE_PARSER=markitdown
export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_ENABLED=true
export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_API_KEY=your-vision-api-key
export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_BASE_URL=https://vision.example/v1
export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_MODEL=ocr-or-vision-model
# Optional custom headers for proxies / request tracing
# export HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_DEFAULT_HEADERS='{"X-Component-Id":"hindsight-ocr"}'

# Use iris as the only parser
export HINDSIGHT_API_FILE_PARSER=iris
export HINDSIGHT_API_FILE_PARSER_IRIS_TOKEN=your-vectorize-token
export HINDSIGHT_API_FILE_PARSER_IRIS_ORG_ID=your-org-id

# Or: try iris first, fall back to markitdown if iris fails or rejects the file type
export HINDSIGHT_API_FILE_PARSER=iris,markitdown

# Use llama_parse as the only parser
export HINDSIGHT_API_FILE_PARSER=llama_parse
export HINDSIGHT_API_FILE_PARSER_LLAMA_PARSE_API_KEY=llx-your-api-key

# Or: try llama_parse first, fall back to markitdown
export HINDSIGHT_API_FILE_PARSER=llama_parse,markitdown

# Increase batch limits for large file imports
export HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE=20
export HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE_MB=500

# Keep files after processing (useful for debugging or re-processing)
export HINDSIGHT_API_FILE_DELETE_AFTER_RETAIN=false

# Native storage is the default — no additional configuration needed
export HINDSIGHT_API_FILE_STORAGE_TYPE=native

# AWS S3
export HINDSIGHT_API_FILE_STORAGE_TYPE=s3
export HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=my-hindsight-files
export HINDSIGHT_API_FILE_STORAGE_S3_REGION=us-east-1
export HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# S3-compatible (MinIO, Cloudflare R2, etc.)
export HINDSIGHT_API_FILE_STORAGE_TYPE=s3
export HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=my-bucket
export HINDSIGHT_API_FILE_STORAGE_S3_REGION=auto
export HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT=https://your-minio.example.com
export HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=minioadmin
export HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=minioadmin

# Tigris (S3-compatible, single global endpoint)
export HINDSIGHT_API_FILE_STORAGE_TYPE=s3
export HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=my-hindsight-bucket
export HINDSIGHT_API_FILE_STORAGE_S3_REGION=auto
export HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT=https://t3.storage.dev
export HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=tid_your_access_key
export HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=tsec_your_secret_key

export HINDSIGHT_API_FILE_STORAGE_TYPE=gcs
export HINDSIGHT_API_FILE_STORAGE_GCS_BUCKET=my-hindsight-files
# Optional: use service account key file (otherwise falls back to ADC)
export HINDSIGHT_API_FILE_STORAGE_GCS_SERVICE_ACCOUNT_KEY=/path/to/key.json

export HINDSIGHT_API_FILE_STORAGE_TYPE=azure
export HINDSIGHT_API_FILE_STORAGE_AZURE_CONTAINER=hindsight-files
export HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_NAME=mystorageaccount
export HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_KEY=base64encodedkey==

HINDSIGHT_API_WORKER_ENABLED=false

export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are broad summaries of project events grouped by week. Each observation should capture what happened, what was decided, and what was blocked — not individual facts. Merge related events into cohesive weekly narratives."

export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are durable facts about specific named people: their preferences, skills, relationships, and behavioral patterns. Only create observations for facts that are stable over time and tied to a named individual."

export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are recurring patterns in customer support interactions: common failure modes, frequently requested features, and pain points that appear across multiple tickets."

# Expose only recall (read-only deployment)
export HINDSIGHT_API_MCP_ENABLED_TOOLS=recall

# Expose recall and reflect only
export HINDSIGHT_API_MCP_ENABLED_TOOLS=recall,reflect

export HINDSIGHT_API_MCP_AUTH_TOKEN=your-secret-token

# Example: instruct MCP to also store assistant actions
export HINDSIGHT_API_MCP_INSTRUCTIONS="Also store every action you take, including tool calls and decisions made."

HINDSIGHT_API_WORKER_CONSOLIDATION_BANK_PRIORITY="shadow-*:10,staging-*:5,*:1"

from hindsight_api import MemoryEngine

memory = MemoryEngine.from_env()
await memory.initialize()

# Enable tracing
export HINDSIGHT_API_OTEL_TRACES_ENABLED=true

# Configure endpoint (example: OpenLIT Cloud)
export HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.openlit.io
export HINDSIGHT_API_OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer olit-xxx"

# Optional: Custom service name and environment
export HINDSIGHT_API_OTEL_SERVICE_NAME=hindsight-production
export HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT=production

./scripts/dev/start-grafana.sh

HINDSIGHT_API_PROFILE='{"every": 60, "top": 20}'

[profile] thread hindsight-loop-            1.41 cores
[profile] thread asyncio_                   0.37 cores
[profile] 1598 functions active in the last 60s
[profile] tottime=0.923 cumtime=1.833 ncalls=15842 <method 'encode' of 'builtins.CoreBPE' objects>
[profile] tottime=0.691 cumtime=7.039 ncalls=1511590

# Point Control Plane to a remote API service
export HINDSIGHT_CP_DATAPLANE_API_URL=http://api.example.com:8888

# Authenticate to an auth-protected API service
export HINDSIGHT_CP_DATAPLANE_API_KEY=my-dataplane-bearer-token

# Protect the Control Plane with an access key
export HINDSIGHT_CP_ACCESS_KEY=my-secret-key

from hindsight_api.config import get_config

# ✅ Access static (infrastructure) fields
config = get_config()
host = config.host  # OK - static field
port = config.port  # OK - static field

# ❌ Attempting to access bank-configurable fields raises an error
chunk_size = config.retain_chunk_size  # ConfigFieldAccessError!

ConfigFieldAccessError: Field 'retain_chunk_size' is bank-configurable and cannot
be accessed from global config. Use ConfigResolver.resolve_full_config(bank_id, context)
to get bank-specific config.

# Internal code that needs bank-specific settings
from hindsight_api.config_resolver import ConfigResolver

# Resolve full config for a specific bank
config = await config_resolver.resolve_full_config(bank_id, request_context)
chunk_size = config.retain_chunk_size  # ✅ Uses bank-specific value

export HINDSIGHT_API_DEFAULT_BANK_TEMPLATE='{"version":"1","bank":{"reflect_mission":"Help support agents remember customer interactions.","retain_extraction_mode":"verbose","disposition_empathy":5},"directives":[{"name":"Be concise","content":"Always respond concisely.","priority":10}]}'

class CustomTenantExtension(TenantExtension):
    async def get_allowed_config_fields(self, context, bank_id):
        # Option 1: Allow all configurable fields
        return None

        # Option 2: Allow specific fields only
        return {"retain_chunk_size", "retain_custom_instructions"}

        # Option 3: Read-only (no modifications)
        return set()

location /hindsight/ {
    proxy_pass http://localhost:8888/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

http:
  routers:
    hindsight:
      rule: "PathPrefix(`/hindsight`)"
      service: hindsight
      middlewares:
        - hindsight-stripprefix

  middlewares:
    hindsight-stripprefix:
      stripPrefix:
        prefixes:
          - "/hindsight"

  services:
    hindsight:
      loadBalancer:
        servers:
          - url: "http://localhost:8888"

# API Service
HINDSIGHT_API_DATABASE_URL=postgresql://hindsight:hindsight_dev@localhost:5432/hindsight
# HINDSIGHT_API_DATABASE_SCHEMA=public  # optional, defaults to 'public'
HINDSIGHT_API_LLM_PROVIDER=groq
HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx

# Authentication (optional, recommended for production)
# HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension
# HINDSIGHT_API_TENANT_API_KEY=your-secret-api-key

# File storage (optional, defaults to PostgreSQL native storage)
# HINDSIGHT_API_FILE_STORAGE_TYPE=s3
# HINDSIGHT_API_FILE_STORAGE_S3_BUCKET=my-hindsight-files
# HINDSIGHT_API_FILE_STORAGE_S3_REGION=us-east-1
# HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
# HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# Control Plane
HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888

git clone https://github.com/vectorize-io/hindsight.git
cd hindsight

uv sync

cd docker && docker-compose up -d postgres

cp .env.example .env

# Database (connects to Docker postgres)
HINDSIGHT_API_DATABASE_URL=postgresql://hindsight:hindsight_dev@localhost:5432/hindsight

# LLM Provider (choose one)
HINDSIGHT_API_LLM_PROVIDER=groq
HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
HINDSIGHT_API_LLM_MODEL=llama-3.1-70b-versatile

./scripts/start-server.sh --env local

# Run all tests
uv run pytest

# Run specific test file
uv run pytest tests/test_retrieval.py

# Run with verbose output
uv run pytest -v

./scripts/generate-clients.sh

./scripts/export-openapi.sh

hindsight/
├── hindsight-api/          # Main API server
│   ├── hindsight_api/
│   │   ├── api/           # HTTP endpoints
│   │   ├── engine/        # Memory engine, retrieval, reasoning
│   │   └── web/           # Server entry point
│   └── tests/
├── hindsight-clients/      # Generated SDK clients
│   ├── python/
│   └── typescript/
├── hindsight-control-plane/ # Admin UI (Next.js)
├── docker/                 # Docker Compose setup
└── scripts/               # Development scripts

docker-compose ps

psql postgresql://hindsight:hindsight_dev@localhost:5432/hindsight

HINDSIGHT_API_PORT=8889 ./scripts/start-server.sh --env local

HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension
HINDSIGHT_API_TENANT_API_KEY=your-secret-key

HINDSIGHT_API_HTTP_EXTENSION=mypackage.ext:MyHttpExtension

HINDSIGHT_API_OPERATION_VALIDATOR_EXTENSION=mypackage.validators:MyValidator

HINDSIGHT_API_MCP_EXTENSION=mypackage.mcp:MyMCPExtension

HINDSIGHT_API_LLM_REQUEST_EXTENSION=mypackage.llms:MyLLMs

HINDSIGHT_API_<TYPE>_EXTENSION=mypackage.module:MyExtensionClass

HINDSIGHT_API_<TYPE>_SOME_CONFIG=value
# Extension receives: {"some_config": "value"}

class MyExtension(TenantExtension):
    def alembic_version_locations(self) -> list[str]:
        return [str(Path(__file__).parent / "alembic" / "versions")]

depends_on = ("a1b2c3d4e5f6",)   # a Hindsight revision id

import jwt
from hindsight_api.extensions import TenantExtension, TenantContext, AuthenticationError

class JwtTenantExtension(TenantExtension):
    def __init__(self, config: dict[str, str]):
        super().__init__(config)
        self.jwt_secret = config.get("jwt_secret")
        if not self.jwt_secret:
            raise ValueError("HINDSIGHT_API_TENANT_JWT_SECRET is required")

    async def authenticate(self, context: RequestContext) -> TenantContext:
        token = context.api_key
        if not token:
            # Optional headers dict is forwarded in HTTP/MCP error responses
            raise AuthenticationError("Bearer token required")

        try:
            payload = jwt.decode(token, self.jwt_secret, algorithms=["HS256"])
            tenant_id = payload.get("tenant_id")
            if not tenant_id:
                raise AuthenticationError("Missing tenant_id in token")
            return TenantContext(schema_name=f"tenant_{tenant_id}")
        except jwt.InvalidTokenError as e:
            raise AuthenticationError(str(e))

raise AuthenticationError(
    "Authorization required",
    headers={"WWW-Authenticate": 'Bearer realm="example"'},
)

HINDSIGHT_API_EXTENSION_PASSTHROUGH_HEADERS=x-user-assertion

async def authenticate(self, context: RequestContext) -> TenantContext:
    assertion = context.extra_headers.get("x-user-assertion")
    if not assertion:
        raise AuthenticationError("x-user-assertion header required")

    user_id = verify_assertion(assertion)  # your verification
    return TenantContext(schema_name=f"tenant_{user_id}")

from fastapi import APIRouter
from hindsight_api.extensions import HttpExtension

class MyHttpExtension(HttpExtension):
    def get_router(self, memory: MemoryEngine) -> APIRouter:
        router = APIRouter()

        @router.get("/hello")
        async def hello():
            return {"message": "Hello from extension!"}

        @router.post("/custom/{bank_id}/action")
        async def custom_action(bank_id: str):
            # Access memory engine for database operations
            pool = await memory._get_pool()
            # ... custom logic
            return {"status": "ok"}

        return router

    def get_root_router(self, memory: MemoryEngine) -> APIRouter | None:
        """Optional: mount routes at the application root (not under /ext/)."""
        router = APIRouter()

        @router.get("/.well-known/my-metadata")
        async def metadata():
            return {"version": "1.0"}

        return router

from hindsight_api.extensions import (
    OperationValidatorExtension,
    ValidationResult,
    PrecheckContext,
    RetainContext,
    RecallContext,
    ReflectContext,
    RetainResult,
)

class MyValidator(OperationValidatorExtension):
    # Pre-body validation (optional)
    async def precheck(self, ctx: PrecheckContext) -> ValidationResult:
        if ctx.content_length is not None and ctx.content_length > 10_000_000:
            return ValidationResult.reject("Payload is too large")
        return ValidationResult.accept()

    # Pre-operation validation (required)
    async def validate_retain(self, ctx: RetainContext) -> ValidationResult:
        # Implement your validation logic
        return ValidationResult.accept()
        # Or reject: return ValidationResult.reject("Reason")

    async def validate_recall(self, ctx: RecallContext) -> ValidationResult:
        return ValidationResult.accept()

    async def validate_reflect(self, ctx: ReflectContext) -> ValidationResult:
        return ValidationResult.accept()

    # Post-operation hooks (optional)
    async def on_retain_complete(self, result: RetainResult) -> None:
        # Log usage, update metrics, send notifications, etc.
        pass

from hindsight_api.engine.search.tags import TagGroupLeaf
from hindsight_api.extensions import OperationValidatorExtension, TagScopeContext


class TeamScopes(OperationValidatorExtension):
    async def resolve_tag_scope(self, ctx: TagScopeContext):
        user = ctx.request_context.api_key_id  # or whatever your tenant extension resolved
        # Dan reads his own memories plus the shared team rules.
        return [TagGroupLeaf(tags=[f"user:{user}", "kind:rule"], match="any_strict")]

class TeamScopes(OperationValidatorExtension):
    async def resolve_write_tag_scope(self, ctx: TagScopeContext):
        user = ctx.request_context.api_key_id
        # Everyone writes their own memories; only Kate writes the team rules.
        return [f"user:{user}", "kind:rule"] if user == "kate" else [f"user:{user}"]

from datetime import datetime, timedelta, timezone

from hindsight_api.extensions import (
    DeferOperation,
    OperationValidatorExtension,
    RetainContext,
    ValidationResult,
)


class QuotaAwareValidator(OperationValidatorExtension):
    async def validate_retain(self, ctx: RetainContext) -> ValidationResult:
        if not await self._quota_available(ctx.bank_id):
            raise DeferOperation(
                exec_date=datetime.now(timezone.utc) + timedelta(minutes=5),
                reason="bank quota window exhausted",
            )
        return ValidationResult.accept()

from mcp.server.fastmcp import FastMCP
from hindsight_api.extensions import MCPExtension
from hindsight_api.engine import MemoryEngine

class MyMCPExtension(MCPExtension):
    async def register_tools(self, mcp: FastMCP, memory: MemoryEngine) -> None:
        @mcp.tool()
        async def custom_search(query: str) -> str:
            """Custom MCP tool for specialized search."""
            # Access memory engine for operations
            pool = await memory._get_pool()
            # ... custom logic
            return f"Results for: {query}"

from hindsight_api.config import LLMMemberConfig
from hindsight_api.extensions import (
    LLMRequestExtension,
    LLMRequestOverrides,
    LLMRequestRefused,
    LLMRequestTarget,
)


class PerCustomerLLMs(LLMRequestExtension):
    def __init__(self, config: dict[str, str]) -> None:
        super().__init__(config)
        self.keys = load_keys(config["keys_file"])  # your own store: {(schema, bank_id): key}

    async def resolve(self, target: LLMRequestTarget) -> LLMRequestOverrides | None:
        key = self.keys.get((target.schema, target.bank_id))
        if key is None:
            # Refuse rather than return None: None sends the call on the server's own LLM.
            raise LLMRequestRefused(f"no LLM key for bank {target.bank_id!r}")
        if target.operation == "reflect":
            # A stronger model for reflect, on the customer's own account.
            return LLMRequestOverrides(llm=LLMMemberConfig(provider="anthropic", model="claude-sonnet-5-5", api_key=key))
        return LLMRequestOverrides(api_key=key, headers={"x-customer": target.bank_id})

FROM ghcr.io/vectorize-io/hindsight:latest

RUN uv pip install --python /app/api/.venv/bin/python --no-cache \
      'PyJWT[crypto]>=2.12.0' 'httpx>=0.27.0'

COPY my_extension /app/extensions/my_extension
ENV PYTHONPATH=/app/extensions

# Fail the build, not the first request, if it isn't importable.
RUN /app/api/.venv/bin/python -c "import my_extension"

# Install Hindsight
pip install hindsight-api

# Install your extension package
pip install ./my-extensions
# or
pip install my-extensions-package

# Configure
export HINDSIGHT_API_TENANT_EXTENSION=my_extensions.auth:JwtTenantExtension
export HINDSIGHT_API_TENANT_JWT_SECRET=your-secret

# Run
hindsight-api

export OPENAI_API_KEY=sk-xxx

docker run -it --pull always --name hindsight --restart unless-stopped --shm-size=1g -p 8888:8888 -p 9999:9999 \
  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
  -v hindsight-data:/home/hindsight/.pg0 \
  ghcr.io/vectorize-io/hindsight:latest

sudo chown -R 1000:1000 $HOME/.hindsight-docker

KeyError: 'getpwuid(): uid not found: 1042'

docker compose -f docker/docker-compose/cuda/docker-compose.yaml up --build

# Standalone (API + Control Plane)
ghcr.io/vectorize-io/hindsight:latest        # Full, latest release
ghcr.io/vectorize-io/hindsight:latest-slim          # Slim, latest release
ghcr.io/vectorize-io/hindsight:0.4.9         # Full, specific version
ghcr.io/vectorize-io/hindsight:0.4.9-slim    # Slim, specific version

# API only
ghcr.io/vectorize-io/hindsight-api:latest
ghcr.io/vectorize-io/hindsight-api:latest-slim

# Control Plane only
ghcr.io/vectorize-io/hindsight-control-plane:latest

cosign verify ghcr.io/vectorize-io/hindsight:<tag> \
  --certificate-identity-regexp '^https://github\.com/vectorize-io/hindsight/\.github/workflows/(sign-images|release)\.yml@.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

# Install with built-in PostgreSQL
helm install hindsight oci://ghcr.io/vectorize-io/charts/hindsight \
  --set api.llm.provider=groq \
  --set api.llm.apiKey=gsk_xxxxxxxxxxxx \
  --set postgresql.enabled=true

# Or use external PostgreSQL
helm install hindsight oci://ghcr.io/vectorize-io/charts/hindsight \
  --set api.llm.provider=groq \
  --set api.llm.apiKey=gsk_xxxxxxxxxxxx \
  --set postgresql.enabled=false \
  --set api.database.url=postgresql://user:[email protected]:5432/hindsight

# Install a specific version
helm install hindsight oci://ghcr.io/vectorize-io/charts/hindsight --version 0.1.3

# Upgrade to latest
helm upgrade hindsight oci://ghcr.io/vectorize-io/charts/hindsight

helm install hindsight oci://ghcr.io/vectorize-io/charts/hindsight \
  --set worker.enabled=true \
  --set worker.replicaCount=3

pip install hindsight-api        # Full — works out of the box
pip install hindsight-api-slim   # Slim — requires external services for embeddings, reranking, and the database

export HINDSIGHT_API_LLM_PROVIDER=groq
export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx

hindsight-api

export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@localhost:5432/hindsight
export HINDSIGHT_API_LLM_PROVIDER=groq
export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx

hindsight-api

hindsight-api --port 9000          # Custom port (default: 8888)
hindsight-api --host 127.0.0.1     # Bind to localhost only
hindsight-api --workers 4          # Multiple worker processes
hindsight-api --log-level debug    # Verbose logging

npx @vectorize-io/hindsight-control-plane --api-url http://localhost:8888

# Run on custom port
npx @vectorize-io/hindsight-control-plane --port 9999 --api-url http://localhost:8888

# Using environment variables
export HINDSIGHT_CP_DATAPLANE_API_URL=http://api.example.com
npx @vectorize-io/hindsight-control-plane

# Production deployment
PORT=80 HINDSIGHT_CP_DATAPLANE_API_URL=https://api.hindsight.io npx @vectorize-io/hindsight-control-plane

pip install hindsight-api

set HINDSIGHT_API_LLM_PROVIDER=openai
set HINDSIGHT_API_LLM_API_KEY=sk-xxx
set HINDSIGHT_API_LLM_MODEL=gpt-4o-mini

hindsight-api

# Install PostgreSQL
winget install PostgreSQL.PostgreSQL.17

# Build pgvector (requires Visual Studio Build Tools)
git clone https://github.com/pgvector/pgvector.git
cd pgvector

# Open "x64 Native Tools Command Prompt for VS" and run:
set PGROOT=C:\Program Files\PostgreSQL\17
nmake /F Makefile.win
nmake /F Makefile.win install

# Create the database and enable the vector extension
psql -U postgres -c "CREATE DATABASE hindsight;"
psql -U postgres -d hindsight -c "CREATE EXTENSION vector;"

pip install hindsight-api

set HINDSIGHT_API_DATABASE_URL=postgresql://postgres@localhost:5432/hindsight
set HINDSIGHT_API_LLM_PROVIDER=openai
set HINDSIGHT_API_LLM_API_KEY=sk-xxx
set HINDSIGHT_API_LLM_MODEL=gpt-4o-mini

hindsight-api

set HF_ENDPOINT=https://hf-mirror.com

set HINDSIGHT_API_LLM_PROVIDER=deepseek
set HINDSIGHT_API_LLM_API_KEY=sk-your-deepseek-key
set HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash
set HINDSIGHT_API_LLM_BASE_URL=https://api.deepseek.com

set HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
set HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5

set HINDSIGHT_API_RERANKER_PROVIDER=flashrank

hindsight-api

pip install hindsight-all        # Full — works out of the box (Linux, Windows, Apple Silicon Macs)
pip install hindsight-all-slim   # Slim — requires external services for embeddings, reranking, and the database

from hindsight import HindsightServer, HindsightClient

with HindsightServer(llm_provider="openai", llm_api_key="sk-xxx") as server:
    client = HindsightClient(base_url=server.url)
    client.retain(bank_id="alice", content="Alice prefers concise answers.")
    results = client.recall(bank_id="alice", query="How should I respond to Alice?")

from hindsight import HindsightEmbedded

client = HindsightEmbedded(llm_provider="openai", llm_api_key="sk-xxx")
client.retain(bank_id="alice", content="Alice prefers concise answers.")
results = client.recall(bank_id="alice", query="How should I respond to Alice?")

hindsight fs mount --bank my-bank

http://localhost:8888/mcp/{bank_id}/

http://localhost:8888/mcp/alice/

export HINDSIGHT_API_MCP_ENABLED=false

export HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension
export HINDSIGHT_API_TENANT_API_KEY=your-secret-key

claude mcp add --transport http hindsight http://localhost:8888/mcp \
  --header "Authorization: Bearer your-secret-key" \
  --header "X-Bank-Id: my-bank"

{
  "mcpServers": {
    "hindsight": {
      "url": "http://localhost:8888/mcp",
      "headers": {
        "Authorization": "Bearer your-secret-key",
        "X-Bank-Id": "my-bank"
      }
    }
  }
}

curl -X POST http://localhost:8888/mcp \
  -H "Authorization: Bearer your-secret-key" \
  -H "X-Bank-Id: my-bank" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc": "2.0", "method": "tools/list", "id": 1}'

export HINDSIGHT_API_MCP_INSTRUCTIONS="Use project:<name> tags for project-specific memories."

{
  "name": "retain",
  "arguments": {
    "content": "User prefers Python over JavaScript for backend development",
    "context": "programming_preferences",
    "tags": ["user:alice", "preferences"]
  }
}

{
  "name": "sync_retain",
  "arguments": {
    "content": "User prefers Python over JavaScript for backend development",
    "context": "programming_preferences",
    "tags": ["user:alice", "preferences"]
  }
}

{
  "name": "recall",
  "arguments": {
    "query": "What are the user's programming language preferences?",
    "tags": ["preferences"],
    "budget": "high"
  }
}

{
  "name": "reflect",
  "arguments": {
    "query": "Based on my past decisions, what architectural style do I prefer?",
    "budget": "mid",
    "tags": ["architecture"]
  }
}

{
  "name": "create_mental_model",
  "arguments": {
    "name": "Current projects",
    "source_query": "Which projects is the user currently working on?",
    "tags": ["projects", "mental-model"],
    "tags_match": "any"
  }
}

{
  "name": "create_mental_model",
  "arguments": {
    "name": "Team Directory",
    "source_query": "Who works here and what do they do?",
    "tags": ["team", "people"],
    "tags_match": "any"
  }
}

{
  "name": "update_mental_model",
  "arguments": {
    "mental_model_id": "team-conventions",
    "trigger": { "refresh_cron": "0 3 * * *", "fact_types": ["observation"] }
  }
}