# Repository: MemPalace/mempalace # Stars: 47417 ## CLAUDE.md # CLAUDE.md ## The Mission Memory is identity. When an AI forgets everything between conversations, it cannot build real understanding — of you, your work, your people, your life. MemPalace exists to solve this. It is a memory system — not a search engine, not a RAG pipeline, not a vector database wrapper. It treats every word you have shared as sacred, stores it verbatim, and makes it instantly available. Your data never leaves your machine. We never summarize. We never paraphrase. We return your exact words. 100% recall is the design requirement — the target every search path is measured against. Anything less means forgetting, and forgetting means starting over. The name comes from the ancient "method of loci" — the memory palace technique used for thousands of years to organize and recall vast amounts of information by placing it in imagined rooms of an imagined building. We were also inspired by the Zettelkasten method (created by German sociologist Niklas Luhmann) — small cross-referenced index cards that point to each other. We apply both ideas to AI memory: - **Wings** for broad categories (people, projects, topics) - **Rooms** for time-based groupings (days, sessions) - **Drawers** for full verbatim content (your exact words) - **AAAK compression** for the index layer — a compact symbolic format (via `dialect.py`) that lets an LLM scan thousands of entries instantly and know exactly which drawer to open ## Design Principles These are non-negotiable. Every PR, every feature, every refactor must honor them. - **Verbatim always** — Never summarize, paraphrase, or lossy-compress user data. The system searches the index and returns the original words. If a user said it, we store exactly what they said. This is the foundational promise. - **Incremental only** — Append-only ingest after initial build. Never destroy existing data to rebuild. A crash mid-operation must leave the existing palace untouched. - **Entity-first** — Everything is keyed by real names with disambiguation by DOB, ID, or context. People matter more than topics. - **Local-first, zero API** — All extraction, chunking, and embedding happens on the user's machine. No cloud dependency for memory operations. No API keys required. - **Performance budgets** — Hooks under 500ms. Startup injection under 100ms. Memory should feel instant. - **Privacy by architecture** — The system physically cannot send your data because it never leaves your machine. No telemetry, no phone-home, no external service dependencies for core operations. - **Background everything** — Filing, indexing, timestamps, and pipeline work happen via hooks in the background. Nothing interrupts the user's conversation. Zero tokens spent on bookkeeping in the chat window. ## Contributing We welcome bug fixes, performance improvements, new language support, better entity disambiguation, documentation, and test coverage. We do not accept summarization of user content, cloud storage/sync features, telemetry or analytics, features requiring API keys for core memory, or shortcuts that bypass verbatim storage. ## Setup ```bash pip install -e ".[dev]" ``` ## Commands ```bash # Run tests python -m pytest tests/ -v --ignore=tests/benchmarks # Run tests with coverage python -m pytest tests/ -v --ignore=tests/benchmarks --cov=mempalace --cov-report=term-missing # Lint ruff check . # Format ruff format . # Format check (CI mode) ruff format --check . ``` ## Project Structure ``` mempalace/ ├── mcp_server.py # MCP server — all read/write tools ├── cli.py # CLI dispatcher ├── config.py # Configuration + input validation ├── miner.py # Project file miner ├── convo_miner.py # Conversation transcript miner ├── searcher.py # Semantic search (hybrid BM25 + vector) ├── knowledge_graph.py # Temporal entity-relationship graph (SQLite) ├── palace.py # Shared palace operations ├── palace_graph.py # Room traversal + cross-wing tunnels ├── backends/ # Pluggable storage backends (ChromaDB default) │ ├── base.py # Abstract interface — implement this for new backends │ └── chroma.py # ChromaDB implementation ├── dialect.py # AAAK compression dialect ├── normalize.py # Transcript format detection + normalization ├── entity_detector.py # Auto-detect people/projects from content ├── entity_registry.py # Entity storage and disambiguation ├── layers.py # L0-L3 memory wake-up stack ├── onboarding.py # Interactive first-run setup ├── repair.py # Palace repair and consistency checks ├── dedup.py # Deduplication ├── migrate.py # ChromaDB version migration ├── spellcheck.py # Auto-correct user messages ├── exporter.py # Palace data export ├── hooks_cli.py # Hook management CLI ├── query_sanitizer.py # Prompt contamination prevention ├── split_mega_files.py # Split concatenated transcript files └── version.py # Single source of truth for version hooks/ # Claude Code hook scripts ├── mempal_save_hook.sh # Stop: triggers diary save └── mempal_precompact_hook.sh # PreCompact: saves state before compression ``` ## Conventions - **Python style**: snake_case for functions/variables, PascalCase for classes - **Linter**: ruff with E/F/W rules - **Formatter**: ruff format, double quotes - **Commits**: conventional commits (`fix:`, `feat:`, `test:`, `docs:`, `ci:`) - **Tests**: `tests/test_*.py`, fixtures in `tests/conftest.py` - **Coverage**: 85% threshold (80% on Windows due to ChromaDB file lock cleanup) ## Architecture ``` User → CLI / MCP Server → Storage Backend (ChromaDB default, pluggable) → SQLite (knowledge graph) Palace structure: WING (person/project) └── ROOM (day/topic) └── DRAWER (verbatim text chunk) Index layer (AAAK): Compressed pointers → DRAWER locations Scanned by LLM to find relevant drawers without reading all content Knowledge Graph: ENTITY → PREDICATE → ENTITY (with valid_from / valid_to dates) ``` ## Key Files for Common Tasks - **Adding an MCP tool**: `mempalace/mcp_server.py` — add handler function + TOOLS dict entry - **Changing search**: `mempalace/searcher.py` - **Modifying mining**: `mempalace/miner.py` (project files) or `mempalace/convo_miner.py` (transcripts) - **Adding a storage backend**: subclass `mempalace/backends/base.py`, register in `backends/__init__.py` - **Input validation**: `mempalace/config.py` — `sanitize_name()` / `sanitize_content()` - **Tests**: mirror source structure in `tests/test_.py` ## README.md > [!CAUTION] > **Scam alert.** The only official sources for MemPalace are this > [GitHub repository](https://github.com/MemPalace/mempalace), the > [PyPI package](https://pypi.org/project/mempalace/), and the docs site at > **[mempalaceofficial.com](https://mempalaceofficial.com)**. Any other > domain — including `mempalace.tech` — is an impostor and may distribute > malware. Details and timeline: [docs/HISTORY.md](docs/HISTORY.md).
MemPalace # MemPalace Local-first AI memory. Verbatim storage, pluggable backend, 96.6% R@5 raw on LongMemEval — zero API calls. [![][version-shield]][release-link] [![][python-shield]][python-link] [![][license-shield]][license-link] [![][discord-shield]][discord-link]
--- ## What it is MemPalace stores your conversation history as verbatim text and retrieves it with semantic search. It does not summarize, extract, or paraphrase. The index is structured — people and projects become *wings*, topics become *rooms*, and original content lives in *drawers* — so searches can be scoped rather than run against a flat corpus. The retrieval layer is pluggable. The current default is ChromaDB; the interface is defined in [`mempalace/backends/base.py`](mempalace/backends/base.py) and alternative backends can be dropped in without touching the rest of the system. Nothing leaves your machine unless you opt in. Architecture, concepts, and mining flows: [mempalaceofficial.com/concepts/the-palace](https://mempalaceofficial.com/concepts/the-palace.html). --- ## Install ```bash pip install mempalace mempalace init ~/projects/myapp ``` ## Quickstart ```bash # Mine content into the palace mempalace mine ~/projects/myapp # project files mempalace mine ~/chats/ --mode convos # conversation exports # Search mempalace search "why did we switch to GraphQL" # Load context for a new session mempalace wake-up ``` For Claude Code, Gemini CLI, MCP-compatible tools, and local models, see [mempalaceofficial.com/guide/getting-started](https://mempalaceofficial.com/guide/getting-started.html). --- ## Benchmarks All numbers below are reproducible from this repository with the commands in [`benchmarks/BENCHMARKS.md`](benchmarks/BENCHMARKS.md). Full per-question result files are committed under `benchmarks/results_*`. **LongMemEval — retrieval recall (R@5, 500 questions):** | Mode | R@5 | LLM required | |---|---|---| | Raw (semantic search, no heuristics, no LLM) | **96.6%** | None | | Hybrid v4, held-out 450q (tuned on 50 dev, not seen during training) | **98.4%** | None | | Hybrid v4 + LLM rerank (full 500) | ≥99% | Any capable model | The raw 96.6% requires no API key, no cloud, and no LLM at any stage. The hybrid pipeline adds keyword boosting, temporal-proximity boosting, and preference-pattern extraction; the held-out 98.4% is the honest generalisable figure. The rerank pipeline promotes the best candidate out of the top-20 retrieved sessions using an LLM reader. It works with any reasonably capable model — we have reproduced it with Claude Haiku, Claude Sonnet, and minimax-m2.7 via Ollama Cloud (no Anthropic dependency). The gap between raw and reranked is model-agnostic; we do not headline a "100%" number because the last 0.6% was reached by inspecting specific wrong answers, which `benchmarks/BENCHMARKS.md` flags as teaching to the test. **Other benchmarks (full results in [`benchmarks/BENCHMARKS.md`](benchmarks/BENCHMARKS.md)):** | Benchmark | Metric | Score | Notes | |---|---|---|---| | LoCoMo (session, top-10, no rerank) | R@10 | 60.3% | 1,986 questions | | LoCoMo (hybrid v5, top-10, no rerank) | R@10 | 88.9% | Same set | | ConvoMem (all categories, 250 items) | Avg recall | 92.9% | 50 per category | | MemBench (ACL 2025, 8,500 items) | R@5 | 80.3% | All categories | We deliberately do not include a side-by-side comparison against Mem0, Mastra, Hindsight, Supermemory, or Zep. Those projects publish different metrics on different splits, and placing retrieval recall next to end-to-end QA accuracy is not an honest comparison. See each project's own research page for their published numbers. **Reproducing every result:** ```bash git clone https://github.com/MemPalace/mempalace.git cd mempalace pip install -e ".[dev]" # see benchmarks/README.md for dataset download commands python benchmarks/longmemeval_bench.py /path/to/longmemeval_s_cleaned.json ``` --- ## Knowledge graph MemPalace includes a temporal entity-relationship graph with validity windows — add, query, invalidate, timeline — backed by local SQLite. Usage and tool reference: [mempalaceofficial.com/concepts/knowledge-graph](https://mempalaceofficial.com/concepts/knowledge-graph.html). ## MCP server 29 MCP tools cover palace reads/writes, knowledge-graph operations, cross-wing navigation, drawer management, and agent diaries. Installation and the full tool list: [mempalaceofficial.com/reference/mcp-tools](https://mempalaceofficial.com/reference/mcp-tools.html). ## Agents Each specialist agent gets its own wing and diary in the palace. Discoverable at runtime via `mempalace_list_agents` — no bloat in your system prompt: [mempalaceofficial.com/concepts/agents](https://mempalaceofficial.com/concepts/agents.html). ## Auto-save hooks Two Claude Code hooks save periodically and before context compression: [mempalaceofficial.com/guide/hooks](https://mempalaceofficial.com/guide/hooks.html). --- ## Requirements - Python 3.9+ - A vector-store backend (ChromaDB by default) - ~300 MB disk for the default embedding model No API key is required for the core benchmark path. ## Docs - Getting started → [mempalaceofficial.com/guide/getting-started](https://mempalaceofficial.com/guide/getting-started.html) - CLI reference → [mempalaceofficial.com/reference/cli](https://mempalaceofficial.com/reference/cli.html) - Python API → [mempalaceofficial.com/reference/python-api](https://mempalaceofficial.com/reference/python-api.html) - Full benchmark methodology → [benchmarks/BENCHMARKS.md](benchmarks/BENCHMARKS.md) - Release notes → [CHANGELOG.md](CHANGELOG.md) - Corrections and public notices → [docs/HISTORY.md](docs/HISTORY.md) ## Contributing PRs welcome. See [CONTRIBUTING.md](CONTRIBUTING.md). ## License MIT — see [LICENSE](LICENSE). [version-shield]: https://img.shields.io/badge/version-3.3.0-4dc9f6?style=flat-square&labelColor=0a0e14 [release-link]: https://github.com/MemPalace/mempalace/releases [python-shield]: https://img.shields.io/badge/python-3.9+-7dd8f8?style=flat-square&labelColor=0a0e14&logo=python&logoColor=7dd8f8 [python-link]: https://www.python.org/ [license-shield]: https://img.shields.io/badge/license-MIT-b0e8ff?style=flat-square&labelColor=0a0e14 [license-link]: https://github.com/MemPalace/mempalace/blob/main/LICENSE [discord-shield]: https://img.shields.io/badge/discord-join-5865F2?style=flat-square&labelColor=0a0e14&logo=discord&logoColor=5865F2 [discord-link]: https://discord.com/invite/ycTQQCu6kn