# Repository: vectorize-io/hindsight # Stars: 9456 ## CLAUDE.md # CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Project Overview Hindsight is an agent memory system that provides long-term memory for AI agents using biomimetic data structures. Memories are organized as: - **World facts**: General knowledge ("The sky is blue") - **Experience facts**: Personal experiences ("I visited Paris in 2023") - **Mental models**: Consolidated knowledge synthesized from facts ("User prefers functional programming patterns") ## Development Commands ### Local Development (API + UI) ```bash # Start both API server and control plane UI ./scripts/dev/start.sh ``` ### API Server (Python/FastAPI) ```bash # Start API server only (loads .env automatically) ./scripts/dev/start-api.sh # Run all tests (parallelized with pytest-xdist) cd hindsight-api-slim && uv run pytest tests/ # Run specific test file cd hindsight-api-slim && uv run pytest tests/test_http_api_integration.py -v # Run single test function cd hindsight-api-slim && uv run pytest tests/test_retain.py::test_retain_simple -v # Lint and format cd hindsight-api-slim && uv run ruff check . cd hindsight-api-slim && uv run ruff format . # Type checking (uses ty - extremely fast type checker from Astral) cd hindsight-api-slim && uv run ty check hindsight_api/ ``` ### Control Plane (Next.js) ```bash ./scripts/dev/start-control-plane.sh # Or manually: cd hindsight-control-plane && npm run dev ``` ### Documentation Site (Docusaurus) ```bash ./scripts/dev/start-docs.sh ``` ### Generating Clients/OpenAPI ```bash # Regenerate OpenAPI spec after API changes (REQUIRED after changing endpoints) ./scripts/generate-openapi.sh # Regenerate all client SDKs (Python, TypeScript, Rust) ./scripts/generate-clients.sh ``` ### Benchmarks ```bash # Accuracy benchmarks ./scripts/benchmarks/run-longmemeval.sh ./scripts/benchmarks/run-locomo.sh # Performance benchmarks ./scripts/benchmarks/run-consolidation.sh ./scripts/benchmarks/run-retain-perf.sh --document # Requires API server running # Results viewer ./scripts/benchmarks/start-visualizer.sh # View results at localhost:8001 ``` ## Architecture ### Monorepo Structure - **hindsight-api-slim/**: Core FastAPI server with memory engine (Python, uv) - **hindsight-control-plane/**: Admin UI (Next.js, npm) - **hindsight-cli/**: CLI tool (Rust, cargo, uses progenitor for API client) - **hindsight-clients/**: Generated SDK clients (Python, TypeScript, Rust) - **hindsight-docs/**: Docusaurus documentation site - **hindsight-integrations/**: Framework integrations (LiteLLM, CrewAI, LangGraph, Pydantic AI, AG2, Claude Code, etc.) - **hindsight-dev/**: Development tools and benchmarks ### Core Engine (hindsight-api-slim/hindsight_api/engine/) - `memory_engine.py`: Main orchestrator for retain/recall/reflect operations - `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, VertexAI, Groq, MiniMax, Ollama, LM Studio, LiteLLM, Claude Code - `embeddings.py`: Embedding generation (local sentence-transformers or TEI) - `cross_encoder.py`: Reranking (local or TEI) - `entity_resolver.py`: Entity extraction and normalization - `query_analyzer.py`: Query intent analysis **retain/**: Memory ingestion pipeline - `orchestrator.py`: Coordinates the retain flow - `fact_extraction.py`: LLM-based fact extraction from content - `link_utils.py`: Entity link creation and management **search/**: Multi-strategy retrieval - `retrieval.py`: Main retrieval orchestrator - `graph_retrieval.py`: Graph retrieval abstract base class - `link_expansion_retrieval.py`: Link expansion graph retrieval - `fusion.py`: Reciprocal rank fusion for combining results - `reranking.py`: Cross-encoder reranking ### API Layer (hindsight-api-slim/hindsight_api/api/) - `http.py`: FastAPI HTTP routers for all REST endpoints - `mcp.py`: Model Context Protocol server implementation Main operations: - **Retain**: Store memories, extracts facts/entities/relationships - **Recall**: Retrieve memories via 4 parallel strategies (semantic, BM25, graph, temporal) + reranking - **Reflect**: Disposition-aware reasoning using memories and mental models. ### Database PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api-slim/hindsight_api/alembic/`. Migrations run automatically on API startup. Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links` ### Adding Database Migrations 1. **Create a new migration file** in `hindsight-api-slim/hindsight_api/alembic/versions/`: - File name format: `_.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`) - Use a unique hex revision ID (12 chars) - Set `down_revision` to the previous migration's revision ID 2. **Migration template**: ```python """Description of the migration Revision ID: f1a2b3c4d5e6 Revises: Create Date: YYYY-MM-DD """ from collections.abc import Sequence from alembic import context, op revision: str = "f1a2b3c4d5e6" down_revision: str | Sequence[str] | None = "" branch_labels: str | Sequence[str] | None = None depends_on: str | Sequence[str] | None = None def _get_schema_prefix() -> str: """Get schema prefix for table names (required for multi-tenant support).""" schema = context.config.get_main_option("target_schema") return f'"{schema}".' if schema else "" def upgrade() -> None: schema = _get_schema_prefix() op.execute(f"CREATE INDEX ... ON {schema}table_name(...)") def downgrade() -> None: schema = _get_schema_prefix() op.execute(f"DROP INDEX IF EXISTS {schema}index_name") ``` 3. **Run migrations locally**: ```bash # Set database URL and run migrations for the base schema plus all tenants uv run hindsight-admin run-db-migration # Run on a specific tenant schema uv run hindsight-admin run-db-migration --schema tenant_xyz ``` ## Key Conventions ### Code Quality **Before writing code, read `.claude/skills/code-review/SKILL.md`** for the full coding standards (Python style, type safety, TypeScript style, general principles). **Always run the lint script after making Python or TypeScript/Node changes:** ```bash ./scripts/hooks/lint.sh ``` **After completing any implementation work, run `/code-review`** to verify your changes against project standards (missing tests, dead code, type safety, etc.). Fix any "must fix" issues before considering the task done. **MANDATORY: Run `/code-review` before pushing code or creating a pull request.** Do not push or create a PR until all "must fix" issues are resolved. ### Memory Banks - Each bank is an isolated memory store (like a "brain" for one user/agent) - Banks have dispositions (skepticism, literalism, empathy traits 1-5) affecting reflect - Banks can have background context - Bank isolation is strict - no cross-bank data leakage ### API Design - All endpoints operate on a single bank per request - Multi-bank queries are client responsibility to orchestrate - Disposition traits only affect reflect, not recall ### Control Plane API Routes When adding or modifying parameters in the dataplane API (hindsight-api), you must also update the control plane routes that proxy to it: 1. **API Routes** (`hindsight-control-plane/src/app/api/`): - `recall/route.ts` - proxies to `/v1/default/banks/{bank_id}/memories/recall` - `reflect/route.ts` - proxies to `/v1/default/banks/{bank_id}/reflect` - `memories/retain/route.ts` - proxies to `/v1/default/banks/{bank_id}/memories/retain` - Other routes follow the same pattern 2. **Client types** (`hindsight-control-plane/src/lib/api.ts`): - Update the TypeScript type definitions for `recall()`, `reflect()`, `retain()` etc. 3. **Checklist when adding new API parameters**: - Add parameter extraction in the route handler (destructure from `body`) - Pass the parameter to the SDK call - Update the client type definition in `lib/api.ts` - Update any UI components that need to use the new parameter ### Adding New Integrations Every new integration in `hindsight-integrations/` must satisfy all of the following before it can be merged: 1. **Tests are required** — tests must simulate or exercise the external system (mock the framework's interfaces and verify the integration actually calls Hindsight correctly). Pure unit tests of helper functions are not sufficient. 2. **CI job** — add a test job in `.github/workflows/test.yml` following the existing pattern (e.g., `test-crewai-integration`). The job must build, install deps, and run `uv run pytest tests -v`. Also add the integration to `detect-changes` outputs so it only runs when its files change. 3. **Release process** — add the integration name to the `VALID_INTEGRATIONS` array in `scripts/release-integration.sh` so it can be released via the standard release workflow. 4. **Follow project code standards** — Python style, type safety, no raw dicts for structured data, no multi-item tuple returns (see `.claude/skills/code-review/SKILL.md`). If any of these are missing, the integration is incomplete and must not be pushed or merged. ### Changelogs Never add "Unreleased" entries to changelogs (e.g. `hindsight-docs/src/pages/changelog/**`). Changelog entries are written by the release script (`./scripts/release-integration.sh`) when a version is actually cut. If a bug fix or feature needs documenting before release, describe it in the PR/commit — the release tooling will surface it in the published changelog section. ### Adding New API Configuration Flags Configuration follows a hierarchical system: **Global (env vars) → Tenant (via extension) → Bank (database)**. Fields must be categorized as either **hierarchical** (can be overridden per-tenant/bank) or **static** (server-level only). #### Adding a New Configuration Field 1. **config.py** (`hindsight-api-slim/hindsight_api/config.py`): - Add `ENV_*` constant for the environment variable name (e.g., `ENV_MY_SETTING = "HINDSIGHT_API_MY_SETTING"`) - Add `DEFAULT_*` constant for the default value - Add field to `HindsightConfig` dataclass with type annotation - **Mark as configurable** by adding to `_CONFIGURABLE_FIELDS` set if the field should be overridable per-tenant/bank via API - Add initialization in `from_env()` method ```python # Configurable field (can be overridden per-tenant/bank via API) _CONFIGURABLE_FIELDS = { ..., "my_setting", # Add here for configurable } # Static field - just don't add to _CONFIGURABLE_FIELDS ``` 2. **main.py** (`hindsight-api-slim/hindsight_api/main.py`): - Add field to the manual `HindsightConfig()` constructor call (search for "CLI override") 3. **Use hierarchical config in MemoryEngine**: ```python # Config is resolved automatically per bank via ConfigResolver config_dict = await self._config_resolver.get_bank_config(bank_id, context) value = config_dict["my_setting"] ``` 4. **Use static config** (non-hierarchical): ```python from ...config import get_config config = get_config() value = config.my_static_field ``` 5. **Documentation** (`hindsight-docs/docs/developer/configuration.md`): - Add to appropriate section table with Variable, Description, Default - Mark if it's hierarchical (can be overridden per-bank) #### Hierarchical vs Static Guidelines **Hierarchical** (per-bank overridable): - LLM settings (provider, model, API key, base URL) - Operation-specific settings (retain mode, chunk size, etc.) - Feature flags that vary by customer/bank **Static** (server-level only): - Infrastructure settings (database URL, port, host) - Global limits (max concurrent operations) - System-wide feature flags ## Environment Setup ```bash cp .env.example .env # Edit .env with LLM API key # Python deps uv sync --directory hindsight-api-slim/ # Node deps (uses npm workspaces) npm install ``` Required env vars: - `HINDSIGHT_API_LLM_PROVIDER`: openai, anthropic, gemini, groq, minimax, ollama, lmstudio - `HINDSIGHT_API_LLM_API_KEY`: Your API key - `HINDSIGHT_API_LLM_MODEL`: Model name (e.g., gpt-4o-mini, claude-sonnet-4-20250514) Optional (uses local models by default): - `HINDSIGHT_API_EMBEDDINGS_PROVIDER`: local (default) or tei - `HINDSIGHT_API_RERANKER_PROVIDER`: local (default) or tei - `HINDSIGHT_API_DATABASE_URL`: External PostgreSQL (uses embedded pg0 by default) - `HINDSIGHT_API_ENABLE_BANK_CONFIG_API`: Enable per-bank config API (default: true) ## README.md
![Hindsight Banner](./hindsight-docs/static/img/hindsight-github-banner.png) [Documentation](https://hindsight.vectorize.io) • [Paper](https://arxiv.org/abs/2512.12818) • [Cookbook](https://hindsight.vectorize.io/cookbook) • [Hindsight Cloud](https://ui.hindsight.vectorize.io/signup) [![CI](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml/badge.svg)](https://github.com/vectorize-io/hindsight/actions/workflows/release.yml) [![Slack Community](https://img.shields.io/badge/Slack-Join%20Community-4A154B?logo=slack)](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![gitcgr](https://gitcgr.com/badge/vectorize-io/hindsight.svg)](https://gitcgr.com/vectorize-io/hindsight) ![PyPI - Downloads](https://img.shields.io/pypi/dm/hindsight-api?label=PyPI) ![NPM Downloads](https://img.shields.io/npm/dm/%40vectorize-io%2Fhindsight-client?logoColor=orange&label=NPM&color=blue&link=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2F%40vectorize-io%2Fhindsight-client)
vectorize-io%2Fhindsight | Trendshift
--- ## What is Hindsight? Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember. It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks. ## Memory Performance & Accuracy Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here: ![Overview](./hindsight-docs/static/img/hindsight-bench.jpg) The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech [Sanghani Center for Artificial Intelligence and Data Analytics](https://sanghani.cs.vt.edu/) and The Washington Post. Other scores are self-reported by software vendors. Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups. ## Adding Hindsight to Your AI Agents The easiest way to use Hindsight with an existing agent is with the LLM Wrapper. You can add memory to your agent with 2 lines of code. That will swap your current LLM client out with the Hindsight wrapper. After that, memories will be stored and retrieved automatically as you make LLM calls. If you need more control over how and when your agent stores and recalls memories, there's also a simple API you can integrate with using the SDKs or directly via HTTP. ![Hindsight Banner](./hindsight-docs/static/img/migration-code.png) --- > 🤖 **Using a coding agent?** Install the Hindsight documentation skill for instant access to docs while you code: > ```bash > npx skills add https://github.com/vectorize-io/hindsight --skill hindsight-docs > ``` > Works with Claude Code, Cursor, and other AI coding assistants. --- ## Quick Start ### Docker (recommended) ```bash export OPENAI_API_KEY=sk-xxx docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \ -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \ -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \ ghcr.io/vectorize-io/hindsight:latest ``` >API: http://localhost:8888 >UI: http://localhost:9999 You can modify the LLM provider by setting `HINDSIGHT_API_LLM_PROVIDER`. Valid options are `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio`, and `minimax`. The documentation provides more details on [supported models](https://hindsight.vectorize.io/developer/models). ### Docker (external PostgreSQL) ```bash export OPENAI_API_KEY=sk-xxx export HINDSIGHT_DB_PASSWORD=choose-a-password cd docker/docker-compose docker compose up ``` >API: http://localhost:8888 >UI: http://localhost:9999 ### Client ```bash pip install hindsight-client -U # or npm install @vectorize-io/hindsight-client ``` #### Python ```python from hindsight_client import Hindsight client = Hindsight(base_url="http://localhost:8888") # Retain: Store information client.retain(bank_id="my-bank", content="Alice works at Google as a software engineer") # Recall: Search memories client.recall(bank_id="my-bank", query="What does Alice do?") # Reflect: Generate disposition-aware response client.reflect(bank_id="my-bank", query="Tell me about Alice") ``` #### Node.js / TypeScript ```bash npm install @vectorize-io/hindsight-client ``` ```javascript const { HindsightClient } = require('@vectorize-io/hindsight-client'); const main = async () => { const client = new HindsightClient({ baseUrl: 'http://localhost:8888' }); await client.retain('my-bank', 'Alice loves hiking in Yosemite'); const results = await client.recall('my-bank', 'What does Alice like?'); console.log(results); } main(); ``` ### Python Embedded (no server required) ```bash pip install hindsight-all -U ``` ```python import os from hindsight import HindsightServer, HindsightClient with HindsightServer( llm_provider="openai", llm_model="gpt-5-mini", llm_api_key=os.environ["OPENAI_API_KEY"] ) as server: client = HindsightClient(base_url=server.url) client.retain(bank_id="my-bank", content="Alice works at Google") results = client.recall(bank_id="my-bank", query="Where does Alice work?") ``` --- ## Use Cases Hindsight is built to support conversational AI agents as well as agents that are intended to perform tasks autonomously. The ideal use case for Hindsight are agents that require a blend of these features such as AI employees that need to handle open-ended tasks, change behavior based on user feedback, and learn to perform complex tasks to automate work at a level that approximates a human work. Hindsight can be used with simple AI workflows like those built with n8n and other similar tools, but may be overkill for such applications. ### Per-User Memories and Chat History One of the simpler use cases you can use Hindsight for is to personalize AI chatbots and other conversational agents by storing and recalling memories associated with individual users. The requirements for this use case usually look something like this: ![Per-User Memories](./hindsight-docs/static/img/per-user-memory-requirements.png) Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories. ![Per-User Memories](./hindsight-docs/static/img/per-user-memory-howto.png) --- ## Architecture & Operations ![Overview](./hindsight-docs/static/img/hindsight-overview.webp) Most agent memory implementations rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works: - **World:** Facts about the world ("The stove gets hot") - **Experiences:** Agent's own experiences ("I touched the stove and it really hurt") - **Mental Models:** Learned understanding of the agent's world formed by reflecting on raw memories and experiences. Memories in Hindsight are stored in banks (i.e. memory banks). When memories are added to Hindsight, they are pushed into either the world facts or experiences memory pathway. They are then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall. Hindsight provides three simple methods to interact with the system: - **Retain:** Provide information to Hindsight that you want it to remember - **Recall:** Retrieve memories from Hindsight - **Reflect:** Reflect on memories and experiences to generate new observations and insights from existing memories. ### Retain The `retain` operation is used to push new memories into Hindsight. It tells Hindsight to _retain_ the information you pass in as an input. ```python from hindsight_client import Hindsight client = Hindsight(base_url="http://localhost:8888") # Simple client.retain( bank_id="my-bank", content="Alice works at Google as a software engineer" ) # With context and timestamp client.retain( bank_id="my-bank", content="Alice got promoted to senior engineer", context="career update", timestamp="2025-06-15T10:00:00Z" ) ``` Behind the scenes, the retain operation uses an LLM to extract key facts, temporal data, entities, and relationships. It passes these through a normalization process to transform extracted data into canonical entities, time series, and search indexes along with metadata. These representations create the pathways for accurate memory retrieval in the recall and reflect operations. ![Retain Operation](hindsight-docs/static/img/retain-operation.webp) ### Recall The recall operation is used to retrieve memories. These memories can come from any of the memory types (world, experiences, etc.) ```python from hindsight_client import Hindsight client = Hindsight(base_url="http://localhost:8888") # Simple client.recall(bank_id="my-bank", query="What does Alice do?") # Temporal client.recall(bank_id="my-bank", query="What happened in June?") ``` Recall performs 4 retrieval strategies in parallel: - Semantic: Vector similarity - Keyword: BM25 exact matching - Graph: Entity/temporal/causal links - Temporal: Time range filtering ![Retain Operation](hindsight-docs/static/img/recall-operation.webp) The individual results from the retrievals are merged, then ordered by relevance using reciprocal rank fusion and a cross-encoder reranking model. The final output is trimmed as needed to fit within the token limit. ### Reflect The reflect operation is used to perform a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world. For example, the `reflect` operation can be used to support use cases such as: - An **AI Project Manager** reflecting on what risks need to be mitigated on a project. - A **Sales Agent** reflecting on why certain outreach messages have gotten responses while others haven't. - A **Support Agent** reflecting on opportunities where customers have questions not answered by current product documentation. The `reflect` operation can also be used to handle on-demand question answering or analysis which require more deep thinking. ```python from hindsight_client import Hindsight client = Hindsight(base_url="http://localhost:8888") client.reflect(bank_id="my-bank", query="What should I know about Alice?") ``` ![Retain Operation](hindsight-docs/static/img/reflect-operation.webp) --- ## Resources **Documentation:** - [https://hindsight.vectorize.io](https://hindsight.vectorize.io) **Clients:** - [Python](http://hindsight.vectorize.io/sdks/python) - [Node.js](http://hindsight.vectorize.io/sdks/nodejs) - [REST API](https://hindsight.vectorize.io/api-reference) - [CLI](https://hindsight.vectorize.io/sdks/cli) **Community:** - [Slack](https://join.slack.com/t/hindsight-space/shared_invite/zt-3nhbm4w29-LeSJ5Ixi6j8PdiYOCPlOgg) - [GitHub Issues](https://github.com/vectorize-io/hindsight/issues) --- ## Star History [![Star History Chart](https://api.star-history.com/svg?repos=vectorize-io/hindsight&type=date&legend=top-left)](https://www.star-history.com/#vectorize-io/hindsight&type=date&legend=top-left) --- ## Contributing See [CONTRIBUTING.md](./CONTRIBUTING.md). ## License MIT — see [LICENSE](./LICENSE) --- Built by [Vectorize.io](https://vectorize.io)