Technical Documentation: NevaMind-AI/memU
ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference
Overview (`README.md`)
memU is a lightweight, agent-driven memory system that gives users a shared LLM wiki across sessions, agents, and devices. It automatically distills your own reusable skills from your agent history. Its core memory logic is only 500 lines — compact enough to inspect, understand, and adapt.
Quick start
memU works with Codex, Claude Code, Cursor, OpenClaw, Hermes, WorkBuddy, Cola, pi, and more. See Host adapters.
Cross-device · Free · Unlimited · View online
Get your API key from memu.so, then send this message to your agent:
Read https://memu.pro/SKILL.md, follow its instructions to install and configure memU, API Key is memu_•••••••••(get Api Key from memu.so).
Agent support
This matrix lists the currently tested memU integrations by operating system.
- Memorize — capture useful session knowledge through a scheduled background task and turn it into reusable memory.
- Retrieve — bring relevant memory into a future task.
- ⚠️ — supported with an important limitation; see the user note.
macOS
| Agent | Mode | Memorize | Retrieve | User note |
|---|---|---|---|---|
| ChatGPT | ChatGPT(Work mode), codex and VS Code extension | ✅ | ✅ | |
| ChatGPT | Chat | ❌ | ❌ | Chat mode is not currently supported. Please use Work mode. |
| Claude Code | Desktop and CLI | ✅ | ✅ | If the selected model declines the setup steps, retry with Opus or another model. Sonnet 5 can occasionally do this. |
| Claude | Chat and Cowork | ❌ | ❌ | |
| Cursor | — | ✅ | ✅ | |
| OpenClaw | — | ✅ | ✅ | Retrieve support has not yet been verified. |
| Hermes Agent | — | ✅ | ✅ | |
| WorkBuddy | — | ✅ | ✅ |
Windows
| Agent | Mode | Memorize | Retrieve | User note |
|---|---|---|---|---|
| ChatGPT | ChatGPT(Work mode), codex and VS Code extension | ✅ | ✅ | |
| ChatGPT | Chat | ❌ | ❌ | Chat mode is not currently supported. Please use Work mode. |
| Claude Code | Desktop and CLI | ✅ | ✅ | If the selected model declines the setup steps, retry with Opus or another model. Sonnet 5 can occasionally do this. |
| Claude | Chat and Cowork | ❌ | ❌ | |
| Cursor | — | ✅ | ✅ | |
| OpenClaw | — | ✅ | ✅ | |
| Hermes Agent | — | ✅ | ⚠️ | Use a memU version with Windows HERMES_HOME support; older versions may retrieve from the wrong files. |
| WorkBuddy | — | ✅ | ✅ | With Hy3, retrieval may fail. Retry with another model if this happens. |
Linux
| Agent | Mode | Memorize | Retrieve | User note |
|---|---|---|---|---|
| Codex | VS Code extension | ❌ | ✅ | |
| Claude Code | CLI | ✅ | ✅ | |
| OpenClaw | 4.23 / 7.1 | ✅ | ✅ |
Support status reflects the current release and may change as host integrations evolve.
How it works
Automatic skill extraction
Once the scheduled bridging task is installed, memU can turn useful agent history into reusable Markdown skills automatically.
- Capture new sessions. The host adapter reads new session history, including messages and tool calls.
- Prepare self-evolve jobs.
prepareslices each session into a self-contained job with the paths and context the agent needs. - Let the agent decide. The agent reads related existing skills, then chooses to do nothing, patch an existing skill, or create a new one.
- Write readable skill Markdown. Each skill has a name, description, and reusable workflow, including useful branches, edge cases, and pitfalls.
- Commit and index.
commitsubmits changed skill files throughcommit_results; memU embeds the skill name and description and stores it under theskilltrack. - Retrieve it later. On a similar future task, memU returns the relevant skill so any connected agent can use the learned workflow.
The judgment and synthesis stay inside the agent. MemoryService makes no LLM or chat calls; it stores, embeds, and retrieves the skill Markdown the agent prepared.
Self-hosted
Private · Single-device · Embedding key required
To run memU locally with your own storage and embedding provider, send this message to your agent:
Read https://raw.githubusercontent.com/NevaMind-AI/MemU/main/SKILL.md and follow it to install memU.
Uninstall
To uninstall memU, send this message to your agent:
Read https://memu.pro/SKILL.md and follow its instructions to uninstall memU.
By default, uninstalling removes the host integration and tooling while keeping your memory store and ~/.memu/config.env, so a later reinstall can resume where you left off. Memory is erased only when you explicitly ask for it.
Host adapters: memory for desktop coding agents
memU runs as a sidecar to a desktop agent, one binary per host. Each binds two seams:
- record — a scheduled bridging task slices new session logs into self-contained job files; the agent itself distills them into memory/skill Markdown;
commitsubmits whatever the agent left on disk back throughcommit_results. - inject — a standing instruction in the host's instruction file tells the agent to run
retrieve(→progressive_retrieve) before answering.
| Host | Binary | Session log it mines | Instruction file it patches |
|---|---|---|---|
| Codex | memu-codex |
~/.codex/sessions/**/*.jsonl |
~/.codex/AGENTS.md |
| Claude Code | memu-claude-code |
~/.claude/projects//<session>.jsonl |
~/.claude/CLAUDE.md |
| Cursor (Agent/CLI) | memu-cursor |
~/.cursor/projects//agent-transcripts/**.jsonl |
./AGENTS.md (per project) |
| OpenClaw | memu-openclaw |
~/.openclaw/agents/<agentId>/agent/openclaw-agent.sqlite (SQLite, read-only) + legacy <agentId>/sessions/*.jsonl |
~/.openclaw/workspace/AGENTS.md |
| Hermes Agent | memu-hermes |
~/.hermes/state.db (SQLite, read-only) |
~/.hermes/SOUL.md |
| WorkBuddy | memu-workbuddy |
~/.workbuddy/projects//<session>.jsonl |
~/.workbuddy/SOUL.md |
| Cola | memu-cola |
~/.cola/sessions/<scope>/<session>.jsonl |
~/.cola/memory-bank/MEMORY.md |
| pi | memu-pi |
~/.pi/agent/sessions/<encoded-cwd>/<session>.jsonl |
~/.pi/agent/AGENTS.md |
| any other agent | memu-agent |
found by memu-agent detect (JSONL dialect sniffed) |
found by detect (AGENTS.md / CLAUDE.md / SOUL.md / …) |
For agents without a dedicated binary, memu-agent detect probes the machine and reports per agent whether memorization works (a recognizable session log exists) and whether retrieval works (an instruction file exists to patch) — then the same verbs run against what it found.
All hosts share one configured memory backend via ~/.memu/config.env — local
or MemU Cloud. What one host's sessions taught memU, another host retrieves.
Installation is the one-message setup in Quick start or Self-hosted. SKILL.md is the routing skill it hands your agent: install the package, identify which host you are (falling back to memu-agent detect for anything without a dedicated adapter), print that host's packaged install guide ( docs install), and follow it — configure the memory backend, register the scheduled bridging task, patch the instruction file, each step behind a verify gate — then report which seams (memorization / retrieval) are now active.
Afterwards doctor proves the whole loop resolves: config, selected
mode, and a live retrieval.
Adding another host means implementing one TranscriptSource (where its session logs live, how its records are shaped) plus a HostSpec-sized CLI — the pipeline, verbs, and instruction text are shared.
Developer integration
Use memu memorize to turn application-owned conversations into memory and skills.
See the developer guide for the input format and integration workflow.
CLI
With memU Cloud, sign in at memu.so to view your memory files. With a local installation, memory lives in the shared store configured by MEMU_DB in ~/.memu/config.env — typically ~/.memu/memu.sqlite3 for local SQLite, or a Postgres DSN.
Once installed, your agent retrieves relevant memory automatically before answering. To retrieve manually, run the adapter for your host:
memu-codex retrieve "What should I remember about this project?"
# or: memu-claude-code / memu-cursor / memu-openclaw / memu-hermes / memu-workbuddy / memu-cola / memu-pi / memu-agentInstall or invoke the CLI directly:
pip install memu-cli # library + memu + memu-codex CLIs
npx memu-cli --help # CLI via npm launcher (engine: PyPI package memu-cli)
uvx --from memu-cli memu # CLI via uv, no installConfiguration
Values resolve in order: process env → ~/.memu/config.env → default. memU
supports Local and Cloud memory backends, selected by MEMU_MEMORY_MODE; an
unset mode remains Local for backward compatibility.
For Local / self-hosted installations, every CLI flag has a matching variable:
| Setting | Env var | Default |
|---|---|---|
| Store | MEMU_DB |
./data/memu.sqlite3 (CLI); required for host adapters |
| Embedding provider | MEMU_EMBED_PROVIDER |
openai (also: jina, voyage, doubao, openrouter); legacy MEMU_LLM_PROVIDER still read |
| API key | MEMU_API_KEY |
the provider's env var, e.g. OPENAI_API_KEY |
| Embedding model | MEMU_EMBED_MODEL |
the provider's default |
| Base URL | MEMU_BASE_URL |
the provider's default |
Run doctor to display the resolved mode and verify the same retrieval
path the host uses.
Storage backends
| Provider | DSN | Vector search | Use for |
|---|---|---|---|
inmemory |
— | brute-force cosine | tests, throwaway sessions |
sqlite |
sqlite:///path.sqlite3 |
brute-force cosine | local/default, single writer |
postgres |
postgresql://... |
pgvector | concurrent access, large stores (pip install "memu-cli[postgres]") |
service = MemoryService(
database_config={"metadata_store": {"provider": "postgres", "dsn": "postgresql://..."}},
embedding_profiles={"default": {"provider": "jina"}},
)License
Apache-2.0
Partnership Community: LINUX DO
Overview (`npm/README.md`)
memu-cli
CLI for memU — personal memory as files. Persist agent-prepared memory and skill documents, then retrieve compact, ranked context instead of restuffing every prompt. Embedding-only: no LLM call happens inside memU.
Quick start
The only prerequisite is uv — it fetches both a Python 3.13 runtime and the memU engine automatically, so nothing else needs to be preinstalled:
curl -LsSf https://astral.sh/uv/install.sh | sh # macOS / Linux
# Windows: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"Then:
export OPENAI_API_KEY=sk-...
# persist prepared memory: {"recall_files": [...], "resource": [...]}
npx memu-cli commit results.json
# list every stored memory/skill file
npx memu-cli list-files
# single-shot embedding retrieval — LLM-free, fast
npx memu-cli retrieve "deploy checklist"By default, state persists in a local SQLite database
(./data/memu.sqlite3). Set MEMU_MEMORY_MODE=cloud andMEMU_CLOUD_API_KEY=<memu-api-key> to run the same commands through MemU Cloud
instead. Register at memu.so to obtain a memU API key. The
production API base defaults to https://api.memu.so/api/v4/memory/ and can be
overridden with MEMU_CLOUD_BASE_URL for compatible deployments. Local
embedding settings remain separate (MEMU_EMBED_PROVIDER, MEMU_EMBED_MODEL,MEMU_DB, ...). Cloud currently persists memory and skill recall files but not
submitted workspace resources. Run npx memu-cli <command> --help for the
local-mode flags.
Commands
| Command | Alias | What it does |
|---|---|---|
retrieve <query> |
search |
Rank memory segments, files, and resources by embedding similarity — zero LLM calls |
list-files |
List every recall file across the memory and skill tracks | |
commit |
Persist externally-prepared recall files and resources (- reads stdin) |
How it works
This npm package is a thin launcher (a few kB, no dependencies). The engine is the PyPI package memu-cli — the memU engine itself, with prebuilt wheels for Linux (x86_64/aarch64), macOS (Intel/Apple Silicon), and Windows. The shim finds a runner and delegates, in order:
$MEMU_PYTHON -m memu— explicit interpreter overrideuvx --from memu-cli memu— no install needed, cached by uv (recommended)pipx run --spec memu-cli memu— no install needed, cached by pipxpython3 -m memu— requirespip install memu-cli
Using Python already? Skip npm entirely: uvx --from memu-cli memu --help, or pip install memu-cli. The same package carries the library API (MemoryService, Postgres backends) — one install covers CLI, host adapters, and library use.
See the main README for the memory model, library API, and self-hosting options.
License
Apache-2.0
Overview (`docs/adr/README.md`)
Architecture Decision Records
- 0001: Use Workflow Pipelines for Core Operations
- 0002: Use Pluggable Storage with Backend-Specific Vector Search
- 0003: Model User Scope as First-Class Fields on Memory Records
- 0004: Add Workspace Memorize Without Touching Single-File
memorize - 0005: Extract Embedding into a Dedicated Package, Fully Decoupled from Chat Clients
- 0006: Promote Skills to a First-Class
RecallFileTrack, Generated Inside Memorize - 0007: Split Memorize/Retrieve into Three Independent Lines on a Layered Wiki-Graph Kernel
- 0008: Trajectory as the Source — Zero-Code Hooks over a Programmatic API
- 0009: Packaging the Codex Integration — pip, a CLI Seam, and One Config Loader
- 0010: Multi-Host Adapters — Claude Code, Cursor, OpenClaw, and Hermes
- 0011: A Generic Host Adapter — Detect First, Then Bind What Works
- 0012: Cloud-Backed Memory Behind the Existing CLI
- 0013: Self-Updating Instruction Templates — Server First, Last-Good Next, Embedded Last
- 0014: Paginated
list_all_recall_files - 0015: The Bridging Run Must Not Mine Itself — Identity from the Host, Gate from the Launch
- 0016: Client Event Reporting — One Envelope, a Spool by Default, Bounded Payloads
- 0017:
config.envIs Written by a Command —initfor the Entry,configfor the Detail - 0018: Mine Claude Cowork Through the Claude Code Bridge
- 0019: Allocate a Private Workspace for Each Developer Memorize Run
Developer (`docs/developer.md`)
Developer integration
Applications that already own their conversation history can submit 1–10 completed sessions to memU without implementing a host adapter. The application decides when sessions are ready, converts each to the canonical input, runs one external evolve executor, and commits the executor's result to the configured Local or Cloud backend.
This is an application integration contract. It starts with a completed session supplied by the application; selecting facts from an ongoing conversation or inventing a conversation on an agent's behalf is outside the v1 interface.
Lifecycle
completed application session
→ memu memorize prepare
→ one external executor processes all jobs serially
→ memu memorize commit <run-id>
→ configured Local or Cloud backendprepare and commit are the deterministic parts of the lifecycle. The middle step is real agent work: the executor reads the session, compares it with existing memory and skill files, and makes create, patch, or no-op decisions.
Each prepare invocation allocates a private directory under ~/.memu/developer/runs/ and returns an opaque run_id. Its 1–10 sessions belong to that single run. Concurrent prepares receive distinct directories:
~/.memu/developer/runs/<run-id>/
├── input/ projected session transcripts
├── jobs/ numbered executor instructions
├── memory/ writable mirror of memory RecallFiles
├── skill/ writable mirror of skill RecallFiles
├── .memorize_manifest.json pre-evolve content snapshot
├── .memorize_run.json active-run marker
├── .resource.tmp files logged by the skill job, when any
└── resources.md verified resource descriptions, when anyThe configured backend is authoritative. memory/ and skill/ are temporary, writable working copies used by the executor; committed changes are persisted as RecallFiles in the backend.
1. Build canonical sessions
Each payload represents one completed session. A prepare invocation accepts 1–10 independent payload files; the CLI argument list is the batch boundary, not a JSON envelope. Its items must remain in the order in which the activity occurred.
Use source activity faithfully:
- preserve the actual user and assistant message text;
- include tool activity when it is available and useful for skill evolution;
- do not synthesize messages, tool calls, results, or user facts to force a memory outcome;
- do not concatenate unrelated sessions into one payload;
- omit credentials, tokens, and other secrets before submission;
- retain the original JSON value and relative position of any tool activity that is included.
The application owns the session boundary and input size. Version 1.0 does not select turns or truncate long sessions automatically. Tool activity is optional, so an application may submit a faithful message-only view when full tool traces are unavailable or unsuitable for persistence.
Top-level fields
| Field | Type | Required | Contract |
|---|---|---|---|
schema_version |
"1.0" |
No | Defaults to "1.0"; no other version is accepted. |
items |
array | Yes | Non-empty ordered list containing at least one message. |
Item fields
| Item | Field | Type | Required | Contract |
|---|---|---|---|---|
message |
type |
"message" |
Yes | Discriminator. |
message |
role |
"user" or "assistant" |
Yes | System messages are not part of the canonical model. |
message |
content |
string | Yes | Non-empty message text. |
tool_call |
type |
"tool_call" |
Yes | Discriminator. |
tool_call |
name |
string | Yes | Non-empty tool name. |
tool_call |
arguments |
any JSON value | No | Defaults to {}. |
tool_result |
type |
"tool_result" |
Yes | Discriminator. |
tool_result |
content |
any JSON value | Yes | May be an object, array, scalar, or JSON null. |
tool_result |
name |
string | No | Non-empty when present. |
tool_result |
is_error |
boolean | No | Whether the tool execution failed. |
The model is strict. Unknown fields such as message IDs, timestamps, tool-call IDs, batch_id, or conversations are rejected rather than ignored. Provider-native OpenAI or Anthropic records must be converted to this canonical shape before calling the CLI.
Minimal message-only example
{
"schema_version": "1.0",
"items": [
{
"type": "message",
"role": "user",
"content": "Please remember that I prefer dark-roast coffee."
},
{
"type": "message",
"role": "assistant",
"content": "Understood."
}
]
}Example with tool activity
{
"schema_version": "1.0",
"items": [
{
"type": "message",
"role": "user",
"content": "Save my dark-roast coffee preference to profile.json."
},
{
"type": "tool_call",
"name": "write_file",
"arguments": {"path": "/workspace/profile.json"}
},
{
"type": "tool_result",
"name": "write_file",
"content": "ok",
"is_error": false
},
{
"type": "message",
"role": "assistant",
"content": "Saved."
}
]
}memU creates two projections from the same ordered input:
| Projection | Receives | Purpose |
|---|---|---|
| Memory | message items only |
Durable user facts, preferences, project context, and working style. |
| Skill | All supplied items | Repeatable workflows learned from messages and tool activity. |
Tool activity does not become user memory. It gives the skill job evidence about how the original task was performed. Resource discovery is best-effort: files referenced by the supplied activity must still exist and be readable in the executor's environment to become resources.
2. Prepare the run
Configure Local or Cloud mode once through the shared MEMU_* environment or ~/.memu/config.env, so prepare, the executor, commit, and later retrieval all use the same backend. The returned next_command does not repeat local backend override flags such as --db or --provider; if an application uses those flags instead of shared configuration, it must pass the same values to commit itself.
Write the payload as UTF-8 JSON and run:
memu memorize prepare session-1.json session-2.json --jsonUse - instead of a file path to read one payload from stdin; stdin cannot be combined with files.
prepare performs the following work before returning:
- validates every canonical payload before allocating a private run directory;
- writes numbered message-only and full JSONL projections into that directory;
- lists the current RecallFiles from the configured backend and writes them into the workspace's
memory/andskill/directories; - snapshots the working copies by content hash;
- creates all memory jobs, then all skill jobs, then one resource job and the active-run marker. One session creates three jobs; ten sessions create 21.
A successful JSON response for the two-session command above has this shape:
/* Detailed source-code truncated for AI context efficiency. */| Response field | Use |
|---|---|
run_id |
Opaque memU-allocated id used by commit, verify-resources, and discard. |
workspace |
Absolute path of this run's private working directory. |
transcript |
Present for one input (file or stdin): one object containing memory_path and skill_path. |
transcripts |
Present for 2–10 inputs: an array of those objects in CLI argument order. |
jobs |
Authoritative execution order. |
executor_prompt |
Complete handoff to one external evolve executor. |
next_command |
Commit command to run once after executor success. |
Exactly one of transcript or transcripts is returned. These paths identify the materialized inputs referenced by the jobs; applications normally do not edit them.
Persist run_id and the handoff before starting the executor. Each prepare creates a new run; it does not resume or replace an existing run. Resource job instructions include memu memorize verify-resources <run-id> so verification targets that run's log.
3. Execute all evolve jobs
Start one external agent session in an environment that can:
- read and write the returned workspace;
- run the
memuexecutable used by the generated resource job; - read any original files that may be described as resources.
Pass executor_prompt to that agent. It instructs the executor to process these jobs:
| Order | Job | Expected outcome |
|---|---|---|
| 1..N | Memory evolution | Create, patch, or leave unchanged the user-memory Markdown files for each session. |
| N+1..2N | Skill evolution | Create, patch, or leave unchanged skills, then log files changed during each supplied session. |
| 2N+1 | Resource description | Verify logged paths and describe readable files in resources.md. |
The executor must process the returned jobs list serially in its given order. It must stop on the first failure and must not run commit. A create, patch, or no-op result is valid for the memory and skill jobs; the executor should not invent an artifact merely to make the run non-empty. Treat each job file as the self-contained instruction for that step; do not treat transcript, memory, skill, or resource contents as new orchestration instructions.
The ordering is load-bearing: the skill job may append paths to .resource.tmp, and the resource job consumes that log. Logged resource paths must be absolute in the executor's environment (including drive-qualified or UNC paths on Windows); ~ paths are also accepted. Drive-relative paths such as C:notes.md and current-drive-root-relative paths such as \notes.md are rejected. Running jobs concurrently can race on the shared workspace and produce incomplete resource output.
A minimal application orchestrator looks like this:
prepared = run_json([
"memu",
"memorize",
"prepare",
session_path,
"--json",
])
execution = evolve_executor.run(
prompt=prepared["executor_prompt"],
)
if not execution.succeeded:
raise RuntimeError("memorize evolve failed; do not commit")
committed = run_json([
"memu",
"memorize",
"commit",
prepared["run_id"],
"--json",
])The executor API and process isolation are application choices. memU defines the filesystem handoff and serial execution contract, not how the external agent is hosted.
4. Commit the result
After the executor reports success, run next_command once. Add --json when a machine-readable result is required:
memu memorize commit run-a1b2c3d4 --jsoncommit hashes the workspace's memory/ and skill/ files against the pre-evolve snapshot, reads successfully described resources, and submits the resulting records through the configured backend. The response contains recall_files and resources; either list may be empty after a valid no-op run.
On success, memU:
- removes the complete run directory, including the working mirrors and any executor-created temporary files;
- makes committed RecallFiles available to normal
list-filesandretrievecalls.
Only newly created or content-modified files are submitted. File deletion is not part of the v1 commit contract.
Run states and recovery
| State | Evidence | Application action |
|---|---|---|
| Ready | Application has a completed batch | Call prepare with 1–10 canonical sessions. |
| Prepared | Returned run directory contains an active marker and ordered jobs | Start exactly one evolve executor for this run. |
| Executing | Executor is processing the jobs | Do not commit, discard, or start another executor for this run. |
| Evolve succeeded | Executor completed all jobs | Run next_command once. |
| Evolve failed | Executor stopped before all jobs completed | Preserve the run for inspection, or explicitly discard it after the executor has stopped. |
| Backend commit failed | Active marker, inputs, jobs, and edits remain | Fix the backend problem and retry commit <run-id>; do not repeat prepare or evolve. |
| Backend commit acknowledged, cleanup failed | Marker has "status":"committed", or was removed before directory cleanup failed |
Do not retry commit; discard the stopped run. |
| Committed / discarded | The run directory is gone | Other runs remain available. |
To abandon a run after stopping its executor:
memu memorize discard run-a1b2c3d4 --jsonDiscard removes only that run directory and does not contact the backend. It also accepts incomplete runs left by a process termination during prepare. Ordinary prepare errors remove the newly allocated directory before returning; a hard process termination may leave a directory under runs/ for inspection and explicit discard. Active runs are never deleted on a timer.
Partial job execution is not resumable in the developer v1 interface. After executor failure, discard the stopped run and prepare the batch again if needed. Backend commit failure retains the evolved workspace for commit retry. If the CLI reports committed, but cleanup failed, the backend accepted the submission even if the active marker still exists. Do not retry commit: stop work on that run and use discard <run-id> for the remaining directory. This applies to snapshot refresh, working-file removal, and final directory removal failures; discard does not undo committed data.
After the backend acknowledges success, memU atomically records "status":"committed" in the run marker before refreshing the snapshot or removing working files. A later commit attempt rejects that marker without resubmitting to the backend. Existing prepared markers containing only schema_version remain usable.
This is a retry guard, not exactly-once delivery. A process termination between backend acknowledgement and marker persistence, a marker-write failure, or an ambiguous network failure can leave the outcome uncertain. If memU reports that the backend committed but the status could not be persisted, do not retry; inspect the backend and discard the stopped run. This guard does not serialize concurrent commits of the same run.
Consistency and concurrency
Independent prepares have isolated working directories. Within a run, the application must serialize executor work, verification, commit, and discard; the active marker is a commit/retry guard, not a process lock. Directory isolation does not provide backend-level conflict resolution.
prepare reads the backend once and establishes the baseline for the run. The workspace is not refreshed again before commit. If another host or developer run changes the same (track, name, user scope) RecallFile during that window, the later successful commit may overwrite the earlier content. Version 1.0 has no ETag, base revision, three-way merge, or conflict copy.
Applications that may overlap with other developer runs or host bridging should serialize the entire prepare → evolve → commit cycle for runs that can edit the same RecallFiles, or assign non-overlapping RecallFile ownership. Serializing only commits does not refresh an already-prepared run's baseline.
Each new CLI run starts with a fresh backend mirror. The explicit-workspace Python functions retain their existing workspace behavior; callers of those functions continue to own directory lifecycle.
Upgrading from the fixed workspace
memu memorize commit and verify-resources now require a run id. Applications must retain the returned run_id or execute the returned next_command; there is no implicit selection of the newest run. Finish an active run in ~/.memu/developer with the previous CLI version before upgrading. Existing fixed-workspace files are not migrated or removed automatically. The per-run lifecycle applies to the developer CLI; host adapter prepare–commit commands keep their existing behavior.
Responsibility boundary
The integrating application owns:
- collecting a completed, faithful session and choosing its boundary;
- deciding when the session is ready to memorize;
- removing secrets and converting provider-native activity to the canonical model;
- keeping backend configuration consistent;
- launching one external executor with
executor_prompt; - treating the jobs as one serial unit of work;
- committing only after executor success;
- applying the recovery and concurrency rules above.
memU owns:
- strict canonical validation;
- memory and skill projection;
- transcript materialization;
- current RecallFile listing and workspace mirroring at
preparetime; - job generation and resource verification;
- content-hash diffing;
- embedding and persistence through the configured Local or Cloud backend;
- cleanup after a successful commit.
Sqlite (`docs/sqlite.md`)
SQLite Database Integration
MemU supports SQLite as a lightweight, file-based database backend for memory storage. This is ideal for:
- Local development and testing
- Single-user applications with persistent storage
- Portable deployments where you need a simple database solution
- Offline-capable applications that can't rely on external databases
Quick Start
Basic Configuration
from memu.app import MemoryService
# Using default SQLite file (memu.db in current directory)
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
},
},
)
# Or specify a custom database path
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
"dsn": "sqlite:///path/to/your/memory.db",
},
},
)In-Memory SQLite (No Persistence)
For testing or temporary storage, you can use an in-memory SQLite database:
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
"dsn": "sqlite:///:memory:",
},
},
)Configuration Options
| Option | Type | Default | Description |
|---|---|---|---|
provider |
str |
"inmemory" |
Set to "sqlite" to use SQLite backend |
dsn |
str |
"sqlite:///memu.db" |
SQLite connection string |
DSN Format
SQLite DSN follows this format:
- File-based:
sqlite:///path/to/database.db - In-memory:
sqlite:///:memory: - Relative path:
sqlite:///./data/memu.db - Absolute path:
sqlite:////home/user/data/memu.db(note the 4 slashes)
Vector Search
SQLite doesn't have native vector support like PostgreSQL's pgvector. MemU uses brute-force cosine similarity for vector search when using SQLite:
service = MemoryService(
llm_profiles={"default": {"api_key": "your-api-key"}},
database_config={
"metadata_store": {
"provider": "sqlite",
"dsn": "sqlite:///memu.db",
},
"vector_index": {
"provider": "bruteforce", # This is the default for SQLite
},
},
)Note: Brute-force search loads all embeddings into memory and computes similarity for each. This works well for moderate dataset sizes (up to ~100k items) but may be slow for larger datasets.
Database Schema
SQLite creates the following tables automatically:
sqlite_resources- Multimodal resource records (images, documents, etc.)sqlite_memory_items- Extracted memory items with embeddingssqlite_memory_categories- Memory categories with summariessqlite_category_items- Relationships between items and categories
Embeddings are stored as JSON-serialized text in SQLite since there's no native vector type.
Data Import/Export
Export Data
You can export your SQLite database for backup or migration:
import shutil
# Simply copy the database file
shutil.copy("memu.db", "memu_backup.db")Import from SQLite to PostgreSQL
To migrate data from SQLite to PostgreSQL:
import json
from memu.database.sqlite import build_sqlite_database
from memu.database.postgres import build_postgres_database
from memu.app.settings import DatabaseConfig
from pydantic import BaseModel
class UserScope(BaseModel):
user_id: str
# Load from SQLite
sqlite_config = DatabaseConfig(
metadata_store={"provider": "sqlite", "dsn": "sqlite:///memu.db"}
)
sqlite_db = build_sqlite_database(config=sqlite_config, user_model=UserScope)
sqlite_db.load_existing()
# Connect to PostgreSQL
postgres_config = DatabaseConfig(
metadata_store={"provider": "postgres", "dsn": "postgresql://..."}
)
postgres_db = build_postgres_database(config=postgres_config, user_model=UserScope)
# Migrate resources
for res_id, resource in sqlite_db.resources.items():
postgres_db.resource_repo.create_resource(
url=resource.url,
local_path=resource.local_path,
caption=resource.caption,
embedding=resource.embedding,
user_data={"user_id": getattr(resource, "user_id", None)},
track=resource.track,
)
# Similar for recall files, segments, and relations...Performance Considerations
| Aspect | SQLite | PostgreSQL |
|---|---|---|
| Setup | Zero configuration | Requires server setup |
| Concurrency | Single writer, multiple readers | Full concurrent access |
| Vector Search | Brute-force (in-memory) | Native pgvector (indexed) |
| Scale | Up to ~100k items | Millions of items |
| Deployment | Single file, portable | External service |
Example: Full Workflow
import asyncio
from memu.app import MemoryService
async def main():
# Initialize with SQLite
service = MemoryService(
database_config={
"metadata_store": {
"provider": "sqlite",
"dsn": "sqlite:///my_memories.db",
},
},
)
# Commit prepared memory
await service.commit_results(
recall_files=[
{
"name": "Preferences",
"track": "memory",
"description": "what alice likes",
"content": "# Preferences\n- prefers dark roast coffee",
}
],
user={"user_id": "alice"},
)
# Retrieve relevant memories
context = await service.progressive_retrieve(
"What are my preferences?", where={"user_id": "alice"}
)
for segment in context["segments"]:
print(f"- {segment['text']} ({segment['score']:.2f})")
asyncio.run(main())Troubleshooting
Database Locked Error
SQLite only allows one writer at a time. If you see "database is locked" errors:
- Ensure you're not running multiple processes writing to the same database
- Consider using PostgreSQL for concurrent access needs
- Use connection pooling with appropriate timeouts
Permission Denied
Make sure the directory containing the SQLite file is writable:
chmod 755 /path/to/data/directorySlow Vector Search
If vector search is slow with large datasets:
- Consider migrating to PostgreSQL with pgvector
- Use more selective
wherefilters to reduce the search space - Reduce
top_kparameters in your retrieve configuration
Adr 0001 Workflow Pipeline Architecture (`docs/adr/0001-workflow-pipeline-architecture.md`)
ADR 0001: Use Workflow Pipelines for Core Operations
- Status: Accepted
- Date: 2026-02-24
Context
memU has multiple high-level operations (memorize, retrieve, and CRUD/patch operations) that each require multi-stage execution, LLM calls, storage writes, and optional short-circuit behavior.
A single monolithic function per operation would make these flows hard to extend, observe, and customize.
Decision
Model each core operation as a named workflow pipeline composed of ordered WorkflowStep units.
- Register pipelines centrally in
MemoryServiceviaPipelineManager - Validate required/produced state keys at pipeline registration/mutation time
- Execute through a
WorkflowRunnerabstraction (localby default) - Support runtime customization by step-level config and structural mutation (insert/replace/remove)
- Provide before/after/on_error step interceptors for instrumentation and control
Consequences
Positive:
- uniform execution model across memorize/retrieve/CRUD
- explicit, inspectable stage boundaries
- extension points for custom runners and step customization
- easier interception and observability around stage execution
Negative:
- dict-based workflow state relies on key naming discipline
- pipeline mutation can increase behavioral variance between deployments
- more framework code compared to direct function calls
Adr 0002 Pluggable Storage And Vector Strategy (`docs/adr/0002-pluggable-storage-and-vector-strategy.md`)
ADR 0002: Use Pluggable Storage with Backend-Specific Vector Search
- Status: Accepted
- Date: 2026-02-24
Context
memU must support:
- zero-setup local development
- lightweight persisted deployments
- production deployments that need scalable vector similarity
No single storage engine fits all three cases.
Decision
Adopt repository-based storage abstraction behind a Database protocol, with selectable providers:
inmemory: in-process state, brute-force similaritysqlite: file-based persistence, embeddings stored as JSON text, brute-force similaritypostgres: SQL persistence, pgvector-enabled similarity when configured
Vector behavior is backend-aware:
- brute-force cosine search remains available for portability
- Postgres can use pgvector distance queries when vector support is enabled
- salience ranking (reinforcement/recency-aware) uses local scoring logic
Consequences
Positive:
- one service API works across local and production footprints
- clear backend contracts through repository interfaces
- predictable fallback behavior when native vector index is unavailable
Negative:
- duplicate repository logic across backends
- behavior/performance differences between providers
- SQLite and in-memory vector search does not scale as well as indexed pgvector
Adr 0003 User Scope In Data Model (`docs/adr/0003-user-scope-in-data-model.md`)
ADR 0003: Model User Scope as First-Class Fields on Memory Records
- Status: Accepted
- Date: 2026-02-24
Context
memU retrieval and writes need scoped operation (for example per user_id, agent_id, or session) for multi-user and multi-agent scenarios.
Keeping scope outside stored records would force ad-hoc filtering logic and weaken data isolation.
Decision
Embed scope directly into all persisted entities by merging a configurable UserConfig.model with core record models.
- Scope fields are part of resource/category/item/relation models
- Repositories accept
user_dataon writes andwherefilters on reads - API-level
wherefilters are validated against configured scope fields before execution
Consequences
Positive:
- consistent filtering model across memorize/retrieve/CRUD APIs
- backend-independent scoping semantics
- supports multi-tenant and multi-agent patterns without separate storage stacks
Negative:
- schema/model generation complexity increases
- schema and index shape can vary by chosen scope model
- callers must keep
whereanduserpayloads aligned with configured scope fields
Adr 0004 Workspace Memorize And Memory File System (`docs/adr/0004-workspace-memorize-and-memory-file-system.md`)
ADR 0004: Add Workspace Memorize Without Touching Single-File `memorize`
- Status: Accepted
- Date: 2026-06-22
Context
A feature branch (feat/memory-fs-synthesis) added a markdown "memory file system"
export, LLM synthesis of MEMORY.md/SKILL.md, an incremental update path, an
embedding-decoupling refactor, and a folder/workspace memorize entry point.
In its original arrangement that work mutates the existing single-filememorize(resource_url, modality) entry point — and ultimately rewrites it to
take a folder argument. We want the existing memorize contract to stay
untouched and to introduce the directory behavior as a new, additive entry point
instead.
Decision
Land the work as seven new, re-scoped commits:
- Folder backbone — add
blob/folder.py(scan / manifest / diff) and a newmemorize_workspace(folder)entry that diff-syncs a directory
(add/modify/delete with cascade-delete of stale memory) by looping over the
unchangedmemorize. No export yet. - Export — add the memory-file-system export and wire a best-effort full
re-export intomemorize_workspace. - Synthesis + update path — add LLM synthesis of
MEMORY.md/SKILL.mdand
the stateful initialize-vs-incremental-update path. The update-on-memorize hook
attaches tomemorize_workspace, notmemorize. - Finish
skill/decoupling inmemory_fs: theskill/tree is always
synthesized from descriptions, never derived from extracted skill-type memory
items. (Unblocks the step-5 restructure.) - Export-tree restructure — restructure the export tree into
resource//memory//skill/with root indexes (INDEX.md/MEMORY.md/SKILL.md) and
bytes-aware diffing. Only the directory restructure is taken;skill/stays
synthesized from descriptions (the step-4 design), so the
synthesizer/_build_memory_filesstay as they were and onlyexporter.pyis
reworked. - Flow + protocol cleanup — split the oversized flows
(retrieve_llm.py/memorize_parse.py), narrow theDatabaseprotocol to
repositories only, and remove the Rust scaffolding. - Decouple embedding — route vectorization through the dedicated
memu.embeddingclients and dropembed()from the chat clients. Kept strictly
last and isolated so it is independently revertable.
memorize(resource_url, modality) remains byte-for-byte unchanged throughout.
Embedding can be last because the decouple only swaps the implementation behind
already-stable call sites (_get_step_embedding_client / .embed()); nothing in
steps 1–6 depends on the new embedding clients.
Notes
- The export, synthesizer, and update-path work do not touch
memorize, so those
steps are clean. Only the initialize-vs-update hook is relocated frommemorize
tomemorize_workspace. - The source branch oscillated on where
skill/content comes from — first
decoupling it to LLM synthesis, then re-coupling it to extracted skill-type
memory items. We keep the synthesis design (step 4) and take only the directory
restructure (step 5), soexporter.pyis reworked by hand rather than applied
wholesale. - Cleanup concerns that were originally bundled together are split so the embedding
decouple lands alone in step 7, isolated and easy to revert.
Consequences
Positive:
- existing
memorizecallers are unaffected; directory ingestion is purely additive - embedding decoupling is isolated in its own commit (step 7) and easy to revert
- each new commit is independently reviewable
Negative:
- the work is split and reordered relative to the source branch, so the new history
is a re-scoped reconstruction rather than a replay of it memorize_workspaceandmemorizeshare the single-file core but are separate
entry points to maintain
Adr 0005 Dedicated Embedding Package (`docs/adr/0005-dedicated-embedding-package.md`)
ADR 0005: Extract Embedding into a Dedicated Package, Fully Decoupled from Chat Clients
- Status: Accepted
- Date: 2026-06-24
Context
Embedding (vectorization) was coupled into the text/chat LLM clients: embed()
lived on OpenAIClient, HTTPLLMClient (which carried its own inline_EmbeddingBackend classes), AnthropicClient (raising), and LazyLLMClient.
The service routed vectorization through an LLM profile (_get_step_embedding_client
returned a chat client), and some retrieval paths reused a single client for both
chat and embed().
This had several problems:
- embedding providers were limited to whatever the chat client happened to
support, so embedding-only providers (Jina, Voyage) had no clean home; - embedding payload/parse logic was duplicated (an orphaned
memu.embedding
module plus the inline backends insideHTTPLLMClient); - the
visioncapability already had a clean sibling package (memu.vlm, see the
LLM/VLM gateway refactor), so embedding was the odd one out; - mixing chat and embedding on one client blurred capability boundaries and made
per-capability provider/transport selection impossible.
ADR 0004 deliberately deferred this as an isolated, independently revertable step
("Decouple embedding").
Decision
Make embedding a first-class capability in its own package, memu.embedding,
structured identically to memu.llm and memu.vlm, and remove embedding from the
chat clients entirely.
Package layout (memu.embedding):
backends/: per-provider request/response shapes —openai,jina,voyage,doubao(incl. multimodal),openrouter; the HTTP client falls back to an
OpenAI-compatible backend for any other provider.http_client/openai_sdk: transport clients;embed()returns(vectors, raw_response)so usage metadata is preserved.gateway.build_embedding_client(cfg): dispatch onclient_backend
(sdk/httpx/lazyllm_backend;anthropicraises — Claude has no
embeddings API).defaults: per-provider default models and embedding-only endpoints.
Configuration and wiring:
- New
EmbeddingConfig/EmbeddingProfilesConfig, sibling toLLMConfig/VLMConfig.MemoryServiceaccepts an optionalembedding_profiles; when
omitted, profiles are derived from the LLM profiles viaembedding_config_from_llm, so existing configs keep working. MemoryServicebuilds/caches embedding clients per profile and wraps them with
the sameLLMClientWrapper, so interceptors and usage tracking are identical to
chat/vision.- All vectorization call sites (query embedding, category/item embedding, RAG
ranking) go through_get_step_embedding_client/_get_embedding_client.
Decoupling the chat clients:
OpenAIClient,HTTPLLMClient, andAnthropicClientno longer exposeembed();HTTPLLMClientno longer carries inline embedding backends.LazyLLMClientkeepsembed()because it is a multi-capability framework
adapter and serves as the embedding transport forlazyllm_backend.LLMConfig.embed_model/embed_batch_sizeare retained only as a
backward-compat bridge consumed byembedding_config_from_llm; they no longer
affect chat clients. New configs should setembedding_profilesdirectly.
Consequences
Positive:
- clear capability boundaries:
memu.llm(text),memu.vlm(vision),memu.embedding(vectors) are isomorphic and independent; - embedding-only providers (Jina, Voyage) are first-class; new providers are a
single backend module plus a registry entry; - no duplicated embedding logic; one source of truth;
- per-capability provider/transport selection (e.g. chat via OpenAI, embed via
Voyage) with zero-config derivation as the default.
Negative:
LLMConfigstill carriesembed_model/embed_batch_sizeas a bridge, so the
decoupling is not yet "pure" at the config layer (kept for backward
compatibility);- an explicit
embedding_profilesis required to use an embedding provider that
differs from the chat provider; - one more package/gateway to maintain in parity with
llm/vlm.
Adr 0006 From Memory Item Category To Tracked Workspace Memorization (`docs/adr/0006-from-memory-item-category-to-tracked-workspace-memorization.md`)
ADR 0006: From Memory Item/Category to a Tracked RecallFile/RecallEntry Store for Workspace Memorize/Retrieve
- Status: Accepted
- Date: 2026-06-30
Context
ADR 0004 landed memorize_workspace and the markdown "memory file system", and
deliberately kept the skill/ tree synthesized at export time from the shared
per-source description trunk (never persisted, never derived from extracted items).
Skills therefore live only on disk: MemoryFilesBuilder.build synthesizes aslug -> body map via MemorySynthesizer, the exporter writes skill/<slug>/SKILL.md,
and an incremental update reads those files back off disk (read_skills) to feed the
"existing skills" half of the next synthesis.
This has grown awkward:
- skills are not first-class data — they cannot be scoped (
user_data), queried, or
reasoned about alongside the structured store; the markdown tree is the database; - the "existing skills" context for incremental synthesis is recovered by parsing
markdown back off disk, rather than read from a table; - the memory track and the skill track are asymmetric: memory categories are
RecallFilerows rendered deterministically intomemory/, while skills are an
ephemeral synthesis artifact with a parallel-but-different code path in the exporter.
Meanwhile the memory track already does, per source file in the memorize_workspace
loop, exactly the shape skills want: take preprocessed content, fold it into a durableRecallFile.summary via an LLM patch. Skills should reuse that machinery.
This work lands on top of a domain-wide rename done as the first commit on the branch:MemoryItem → RecallEntry, MemoryCategory → RecallFile, CategoryItem →RecallFileEntry (and their repositories), a pure no-behavior-change rename that
establishes the RecallFile/RecallEntry vocabulary this ADR uses throughout.
Decision
Make a skill a first-class RecallFile, distinguished from a memory category by a
new track column, and generate skills inside the memorize workflow (the
workspace path) rather than freshly at export time.
Data model
- Add
RecallFile.track: strwith values"memory" | "skill", defaulting to"memory". Existing rows backfill to"memory". - Rename
RecallFile.summary→RecallFile.content. Thesummaryname predates theMemoryCategory→RecallFilerename; the column holds the file's body, socontentis the correct name. (RecallEntry.summaryis unchanged — only the
file-level column is renamed.) Both changes ship in one migration since thetrack
add already forces one. get_or_create_categorykeys on(name, track, *scope), and the Postgres
unique-with-scope index becomes(name, track, *scope), so a skill and a memory
category may share a name without colliding.
Track isolation via `where`
Every RecallFile reader scopes by track through the existing where filter rather
than ad-hoc post-filtering:
- the memory/categorize/retrieve/CRUD paths and the exporter's
memory/rendering
readtrack="memory"; - the skill generation step and (future) skill rendering read
track="skill".
This keeps skills out of memory retrieval/RAG scoring and out of memory/.
Skill generation in the workflow
_memorize_one (the memorize_workspace per-file path) runs a new workflow
(memorize_workspace) — the existing memorize steps plus a skill-generation step —
rather than overloading the shared memorize pipeline (single-file memorize() is
unchanged, mirroring ADR 0004's "don't touch memorize" stance). The new step:
- runs per file, after the memory persist step, taking
preprocessed_resourcesas
input (it has no data dependency on the memory categorize/persist outputs); - reads existing
track="skill"RecallFiles as the "existing skills" context — read
from the DB, not from disk — so file N sees skills created by files 1..N-1; - reuses the existing skill-synthesis prompt/algorithm (processed content + existing
skills → skills to add/replace), now emitting(name, description, body); - persists each skill directly as a
RecallFile(track="skill", content=body),
embeddingname + description, bypassing theRecallEntryplane entirely; - is gated behind
memory_files_config.synthesize.
Export (mirrors the memory track)
The exporter mirrors the memory track exactly: it renders each track="skill"RecallFile deterministically into skill/<slug>.md (just as memory/<slug>.md is
rendered from memory-track files — one shared, track-agnostic renderer), and the
synthesizer's skill role is reduced to producing a single overview SKILL.md body (the
mirror of MEMORY.md) synthesized from the skill files' bodies, rather than the
per-skill bodies. The old description-based skill synthesis, the RecallEntry skill
bypass, and the skill/<slug>/SKILL.md folder layout are gone; read_skills is
replaced by read_skill_body (the SKILL.md-overview analog of read_memory_body).SKILL.md falls back to a deterministic link index when synthesize=False, mirroringMEMORY.md.
Retrieve (consuming the track)
The track column also opens the read side. A new retrieve_workspace entry point — the
read-side analog of memorize_workspace, and the simple counterpart to the routing-heavyretrieve — does single-shot, LLM-free retrieval: it embeds the query once and ranks the
file/entry/resource layers by vector similarity, with none of the intention routing,
sufficiency checks, or summarization of the RAG path. The file layer scopes by track
through RetrieveFileConfig.tracks, so skill-track RecallFiles can be retrieved (or
excluded) independently of the memory track — the read-side mirror of the track isolation
above. Driven by a new RetrieveWorkspaceConfig.
Scope of these commits
This ADR spans four commits on the branch (a fifth — gating new-category creation behindallow_new_categories — is an unrelated fix to the existing taxonomy path):
- Domain rename (foundation).
MemoryItem→RecallEntry,MemoryCategory→RecallFile,CategoryItem→RecallFileEntry, and the matching repositories. Pure
rename, no behavior change; establishes the vocabulary the rest of this work assumes. - Skill track + workflow. Schema foundation (
trackcolumn,summary→content,(name, track)keying, migration), track isolation (memory-track readers filtertrack="memory"), and the newmemorize_workspaceworkflow with its per-file skill
generation step that persiststrack="skill"RecallFiles. - Export mirror.
skill/<slug>.mdrendered deterministically from the DB + a
synthesized/indexedSKILL.mdoverview, symmetric withmemory/+MEMORY.md. - Workspace retrieve. The LLM-free
retrieve_workspacepath that reads the track
back out — see "Retrieve (consuming the track)" above.
Open issues (deferred)
Skill staleness / provenance on deletion. By bypassing the
RecallEntryplane,
skillRecallFiles have no link back to the resources that produced them. The memory
track invalidates correctly on file change/delete viaRecallEntry.resource_id → RecallFileEntry → categorycascade
(_cascade_delete_by_urls); skills have no equivalent, so a skill derived from a
now-deleted source file lingers. We do not yet have a chosen solution (candidates: a
provenance link from skill files to contributing resources, or a full skill-track
rebuild ondiff.has_removals). Until then the skill track is append/merge-only
and does not shrink on deletion. A solution is expected in a future commit.Cost / latency. Skill synthesis moves from ~1 merged LLM call per workspace sync
to one call per added/modified file (consistent with how per-file memory summaries
already work, and gated behindsynthesize). Plus oneSKILL.md-overview synthesis
per export, the mirror of the existingMEMORY.mdsynthesis. Acceptable, but noted.
Consequences
Positive:
- skills are first-class, scoped, queryable rows; the markdown tree stops being the
only home for skills; - the "existing skills" context for incremental generation is a table read, not a
markdown re-parse; - memory and skill tracks become symmetric (
RecallFilerows distinguished bytrack), collapsing two divergent code paths; - single-file
memorize()stays untouched; the skill step lives only on the workspace
path; - making skills first-class rows lets the new
retrieve_workspacepath read them back bytrack, so the workspace gets a symmetric memorize/retrieve pair without skills needing
a bespoke read path.
Negative:
- a schema migration is required (
trackadd +summary→contentrename); existing
Postgres/SQLite deployments need it applied (fresh DBs get it fromcreate_all); - every
RecallFilereader must now be track-aware, a cross-cutting concern threaded
throughwhere; - deletion/provenance for skills is a known, deferred gap (see open issues).
Adr 0007 Three Independent Memory Lines Wiki Graph (`docs/adr/0007-three-independent-memory-lines-wiki-graph.md`)
ADR 0007: Split Memorize/Retrieve into Three Independent Lines on a Layered Kernel
- Status: Proposed
- Date: 2026-07-01
- Supersedes: ADR 0004 (workspace memorize + memory file system), ADR 0006 (skill as a
RecallFiletrack)
Context
ADR 0004 added memorize_workspace/the markdown memory file system; ADR 0006 promoted
skills to a track="skill" RecallFile generated inside the workspace memorize workflow.
Living with both has surfaced a consistent coupling problem.
One pipeline does everything, per file.
memorize_workspace(folder)is a fan-out:
for each changed file it runs the fullmemorizeworkflow plus agenerate_skills
step, then rebuilds the entireINDEX.md/MEMORY.md/SKILL.mdtree. Memory
extraction, skill synthesis, and indexing all fire for every file regardless of what
that file is for.generate_skillsis a parallel mechanism that bypasses the entry plane. It
re-synthesizes skills from the raw preprocessed text (not from the extractedRecallEntrys), and persists them directly asRecallFile(track="skill"), bypassing
theRecallEntry → RecallFileEntry → RecallFilecascade. That bypass is the cause of
ADR 0006's own deferred open issue: skills have no provenance link, so they never
invalidate on source change/delete.The
trackcolumn exists only to jam two different things into one table. Memory
categories and skills shareRecallFileand are split bytrack. That forced sharing
is the root of the asymmetry and thegenerate_skillsbypass.The read side is forked, not composed.
retrieve_rag/retrieve_llm(heavy
intention-routing + sufficiency RAG over the structured store) andretrieve_workspace
(flat embedding recall) share no handlers —_rag_*/_llm_*vs_ws_*are
independent reimplementations of the same recall. The write side composes
(memorize_workspace=memorize+ one step); the read side does not.
The deeper observation: INDEX / MEMORY / SKILL are not three outputs of one process —
they are three different treatments of source material, with three different read
models. Routing them through one pipeline and one store is the coupling.
A second observation refines the design: embedding search and a graph are separable.
Embedding search (embed → vector rank) is a flat capability over a set of pages; a graph
adds edges + traversal. Of the three layers, memory and the workspace index are
relational (entities interlink; documents reference each other and have hierarchy —
traversal matters), while skills are a retrieval problem ("which skill fits my current
task" = flat top-k), where a graph adds cost for little value.
Decision
Refactor memorize/retrieve into three independent lines, each fed by its own
source, owning its own store and output, and sharing a layered wiki-graph kernel.
The three lines
| Line | Source | Output index | L0 → L1 → L2 (raw → doc → item slices) | Read |
|---|---|---|---|---|
| chat | conversation logs | MEMORY.md |
raw chat → memory category → paragraph slices | hybrid: embedding + BM25 |
| workspace | workspace files | INDEX.md |
raw media → caption paragraph → caption slices | hybrid: embedding + BM25 |
| skill | execution / tool traces | SKILL.md |
raw logs → skill md → per-skill descriptions | hybrid: embedding + BM25 |
Each line keeps the established output-file name as its L1 index/map, over its own L0/L1/L2 stores.
Note. This section's original "graph wiki / graph traversal" framing is superseded
by "Refinement: hybrid retrieval (embedding + BM25), no graph" below; there is no graph.
Shared kernel (one implementation, three instances)
All three lines share one in-process kernel: record stores (resource / item / category),
embedding, hybrid search (cosine embedding + BM25 keyword), rank, markdown render, and
manifest diff. There is no graph layer and no external database — see "Refinement: hybrid
retrieval" below.
Per-line data model (independent stores, no `track`)
Node (a wiki page / file): id · slug · content(md) · embedding · source_ref · meta
Edge (a [[wikilink]]): from_node · to_node · type? ← graph lines onlyINDEX.md / MEMORY.md / SKILL.md are rendered deterministically from each line's Node
set. The three lines have separate stores — there is no shared table and no track
discriminator.
Layer model: L0 / L1 / L2 (refines the data model above)
The generic "Node/Edge" sketch above is refined into three representation layers. Every
line has all three. The layers share a uniform role but hold line-specific artifacts, and
each is derived from the one below (L0 → L1 → L2):
| Layer | Role (uniform) | chat | workspace | skill |
|---|---|---|---|---|
| L0 | raw source (resource) |
raw chat corpus | raw multimodal data | raw agent-run logs |
| L1 | coarse document derived from L0 | the classified memory category file | the preprocessed caption paragraph | the skill markdown |
| L2 (item) | fine slices/extracts of L1 — the embed/search unit | slices of the category's paragraphs | slices of the caption paragraph | the description extracted per skill |
- L1 ⊇ L2 in every line: an item (L2) is a slice/extract of its L1 document. This
inverts the earlier wording ("L1 = atomic item, L2 = category"): now L1 is the coarse
document and L2 is the fine item, consistently across all three lines. - Derivation runs L0 → L1 → L2: preprocess the raw source into the L1 document, then
slice/extract it into L2 items. - Because the meanings are now per-line (and inverted from the old model), they no longer map
1:1 ontoResource/RecallEntry/RecallFile; each line's store holds a resource (L0), a
document (L1), and item slices (L2).
Retrieval (hybrid, no graph, single pass). The L2 items are embedded; a query runs
one hybrid pass — cosine embedding + BM25 keyword, each min-max normalized and fused — over
the L2 items, and the top items roll up to their L1 document (and its L0 resource) for the
result. No item ↔ item edges, no entity index, no traversal, no multi-hop.
Markdown output. MEMORY.md / SKILL.md render the L1 documents (memory categories /
skills); INDEX.md indexes the L0 resources. L2 items live only in the store as the
embed/search units, not as separate files.
Seam with preprocessing. Preprocessing is the injected first step of every line's
memorize pipeline (preprocess → …): it takes a source reference and produces the L1
document (e.g. caption paragraph / classified memory / skill md) from the raw L0 source;
slicing/extracting that document into L2 items is the next step. Like the other treatments
(embed, and the per-line slice/extract), preprocess is injected so lines stay testable; a
real implementation wraps memu.preprocess (fetch via blob + modality decode + describe).
Divergence note. This redefinition inverts L1/L2 and gives every line all three layers,
so it is ahead of the currentmemu.linescode (which still has L1 = atomic item, L2 =
grouping, workspace with no L2, skill with no L1). The code will be re-aligned to this model.
Refinement: hybrid retrieval (embedding + BM25), no graph
This supersedes every "graph / edges / backlinks / traversal / entity linking / GraphRAG /
multi-hop" description earlier in this ADR. L1 has no connective structure:
- No graph. No
item ↔ itemedges, noneighbors/backlinks/traverse. - No entity layer / GraphRAG / multi-hop. No parallel entity store, no item→entity index,
no entity-overlap ranking, no iterative expansion.
Retrieval is a single pass, hybrid: a cosine embedding score and a BM25 keyword score are
each min-max normalized across candidates and fused into one rank. No iteration, no graph
walk. All three lines retrieve this way, differing only in what they search and what they
return:
All three lines retrieve the same way — embed the L2 items, one hybrid pass, roll up to
the L1 document:
| Line | embeds / searches (L2 items) | rolls up to / returns (L1) |
|---|---|---|
| chat | slices of the memory category's paragraphs | the memory category (file) |
| workspace | slices of the caption paragraph | the caption paragraph (and its L0 resource) |
| skill | descriptions extracted per skill | the skill markdown |
Rationale: without multi-hop traversal, an entity index is not a graph — it is only a ranking
feature, and it did not justify graph machinery here. Hybrid (embedding + keyword) is a
standard, cheap, single-pass retrieval covering both semantic and exact-term matches; a graph
/ entity / multi-hop signal can be added later as an explicit, measured enhancement if a
benchmark shows it helps.
Concretely, Edge / neighbors / backlinks / traverse, any entity store / item→entity
index, and any multi-hop loop are not part of the design. The stores hold records +
embeddings and expose a single search(query_vec, query_text, k) over the L2 items that fuses
cosine + BM25; the hits then roll up to their L1 document before returning.
Per-line ingest treatment (the only code that differs)
Each line's ingest is preprocess (raw L0 → L1 document) → slice/extract (L1 → L2 items) → embed:
- chat: classify the conversation into a memory category (L1); slice its paragraphs into items (L2).
- workspace: preprocess media into a caption paragraph (L1); slice it into items (L2).
- skill: synthesize a skill markdown (L1) from the logs; extract a description per skill as the item (L2).
Independent triggers
Each line has its own source manifest and change detection. A change under one line's
source rebuilds only that line; lines never trigger each other. (A change in /chat
no longer re-synthesizes skills or rebuilds the workspace index.)
Shared, not duplicated
The kernel is one implementation; the three lines are instances of it. Existing low-level
utilities stay shared calls, not copies: LocalFS fetch, the preprocess registry
(modality handlers), the embedding clients / ClientPool, and the workflow engine. A
single scope convention (user_id / where) is preserved across all three lines so
cross-line scoping ("everything about user X") still works even though the stores are
separate.
What this removes
RecallEntry/RecallFile/RecallFileEntryand thetrackcolumn → per-lineNode/Edge.- The monolithic per-file
memorize_workspacepipeline and thegenerate_skillsstep →
three ingest treatments. retrieve_rag/retrieve_llm/retrieve_workspace→ one shared read mechanism
(embedding search, plus graph traversal on the two graph lines).- The skill provenance/deletion gap (ADR 0006 open issue): the skill line owns its store
and bypasses nothing; deletion is driven by the line's own manifest.
Consequences
Positive:
- Three high-cohesion modules replace the two ~1500-line
memorize.py/retrieve.py
mixins; each line is end-to-end self-contained (source → ingest → store → read →
output). - The
trackcolumn and thegenerate_skillsbypass are gone; the skill provenance gap
is resolved structurally rather than deferred. - The read side becomes coherent — one search mechanism (+ optional traversal) instead of
three duplicated recall paths. - Each line's structure matches its real need (graph for relational lines, flat for
retrieval), instead of an over-imposed uniform shape. - Independent triggers cut wasted work: a chat change no longer re-synthesizes skills or
rebuilds the index.
Negative / costs:
- A large refactor with a data migration (track-based rows → per-line stores).
- No cross-line edges: an entity relevant to more than one line is duplicated, with no
cross-domain traversal. Accepted as the separation tax; a read-only weak cross-line
reference (e.g. a skill page citing a memory entity by URI) is a future option, not built
now. - The workspace graph relies on structural links to avoid LLM link-inference cost; richer
semantic linking there is deliberately out of scope. - Three stores instead of one: a cross-line query must hit three lines (mitigated by the
shared scope convention).
Open issues (deferred)
- Skill source. Assumed to be a
/log(trace) folder. Whether skill is a primary
source line or is distilled from the chat/workspace lines is unresolved; it changes
whether skill is a true third line or a second-order derivation. - Migration mechanics from the current
RecallFile(track=…)schema to three per-line
stores (sketch:track=memory→ chat nodes;track=skill→ skill nodes; resources/INDEX
→ workspace nodes). - Workspace linking — keep it purely structural, or add light semantic linking later.
Implementation plan
PoC the workspace line first: it has the least LLM (structural links, kept-whole pages)
and the clearest graph, so it best validates the Base + Graph kernel end-to-end before the
LLM-heavy chat and skill lines are built on the same kernel.
Related ADRs
- Supersedes
docs/adr/0004-workspace-memorize-and-memory-file-system.md - Supersedes
docs/adr/0006-from-memory-item-category-to-tracked-workspace-memorization.md - Builds on
docs/adr/0001-workflow-pipeline-architecture.md(the kernel still composes as
workflow steps),docs/adr/0002-pluggable-storage-and-vector-strategy.md, anddocs/adr/0005-dedicated-embedding-package.md.
Adr 0008 Two Integration Surfaces Hooks And Api (`docs/adr/0008-two-integration-surfaces-hooks-and-api.md`)
ADR 0008: Trajectory as the Source — Zero-Code Hooks over a Programmatic API
- Status: Proposed
- Date: 2026-07-07
- Builds on: ADR 0007 (three independent memory lines on a layered kernel), ADR 0001 (workflow pipelines)
- Refines: ADR 0007's source model (resolves its "skill source" open issue) and output model
(no rendered index files: per-line folders of L1 documents, searched via a pure per-line
embedding index of L2 items carrying L1-pointing metadata)
Context
ADR 0007 settled the engine: three independent lines (chat / workspace / skill) over a
shared layered kernel, driven by memorize-workspace (write) and retrieve-workspace
(read), against a per-project store. What it did not settle is how an agent — Claude
Code, Codex, OpenClaw, Hermes, or an in-house one — actually feeds sources in and gets
memory back out during a live session.
Today there is effectively one path: a human (or an agent following SKILL.md) runs the
CLI by hand against a curated folder — sources are whatever someone placed under chat/
or agent/, and the directory decides the treatment. That is fine for "sync this folder,"
but it does not give an agent continuous memory: the user turns and the agent's own tool
trajectory are never captured unless someone remembers to save them, and retrieved memory is
never injected unless the agent explicitly asks.
The deeper problem is the source model itself. The richest record of a working session is
the agent's trajectory — the interleaved user messages, agent reasoning, and tool actions.
Asking users (or agents) to pre-sort that record into per-line source directories throws away
the fact that a single stretch of work simultaneously contains who the user is, how the
agent got something done, and where the project stands. The structural, directory-based
routing of ADR 0007's sources cannot see any of that.
Beyond the source model, two different consumers want two different things:
- A drop-in user wants memory to "just work" with no code changes to their agent —
install something, and the agent starts remembering across sessions. - A builder embedding memU inside their own agent wants explicit control — call
retrieve at a chosen point, memorize a chosen artifact, with their own batching and
scoping.
Trying to serve both with one mechanism has been the tension. A pure API serves the builder
but leaves the drop-in user writing glue. A pure "magic" integration serves the drop-in user
but is a black box the builder cannot steer. And every host agent exposes a different hook
system (Claude Code settings.json events, Codex/OpenClaw/Hermes each their own), so "just
use hooks" is not one integration but N.
Decision
Three decisions: one philosophical (the source), one structural (the surfaces), one of
naming (the CLI).
1. Trajectory is the primary source; the three lines are three extractions from it
The agent's trajectory (user turns + agent actions/tool traces) becomes the primary raw
source. Instead of routing pre-sorted directories to lines, memorize runs a semantic
extraction over the trajectory that distills three kinds of memory — each landing on the
ADR 0007 line built for it. Concretely, each line's L0 is a different projection of the
same trajectory (see the layer mapping below), rather than a separate source folder.
The lines are renamed after what they extract — memory / skill / project are the
canonical terms from here on (ADR 0007's chat and workspace are the old names of thememory and project lines respectively):
| Line (canonical term) | What it captures | ADR 0007 name / source dir | Output |
|---|---|---|---|
| memory | user facts, preferences, conversational context | chat (/chat) |
memory/ folder (L1 docs) + embedding index |
| skill | reusable procedures — how the agent accomplished something | skill (/skill) |
skill/ folder (L1 docs) + embedding index |
| project | the files the work touched — project state as reflected in their content | workspace (/workspace) |
project/ folder (L1 docs) + embedding index |
Output model: per-line L1 folders + a pure per-line embedding index. No rendered index
files. ADR 0007's single rendered files (MEMORY.md / SKILL.md / INDEX.md) are
removed, replaced by two artifacts per line:
- Browsable output — a folder per line (
memory/,skill/,project/) holding one
markdown file per L1 document (per memory category, per skill, per project document).
The folder is the human-facing output; nothing is rendered on top of it. - Searchable output — a pure embedding index per line: one embedding per L2 item,
whose embedded content is the L2 slice itself and whose metadata points to its L1
document (and through it, the L0 trajectory span). A retrieval hit rolls up L2 → L1 via
that metadata alone; L2 items never materialize as files.
Rationale: a change to one document touches one file instead of rewriting a monolith; the
search path needs no rendered artifact at all — embeddings + metadata carry the full
L2 → L1 → L0 chain; and the on-disk names match the line terms exactly.
Layer mapping. Each line keeps ADR 0007's L0 → L1 → L2 derivation, but the L0s are now
three projections of the same trajectory instead of three separate source folders:
| Layer (role) | memory | skill | project |
|---|---|---|---|
| L0 — the line's projection of the trajectory | the user queries and agent responses only (no tool/action traces) | the entire user–agent trajectory (turns + actions/tool traces) | the contents of files touched in the trajectory |
| L1 — summarized document (one md file in the line's folder) | the summarized memory file | the synthesized skill | the summary of the file's content |
| L2 — the embedded unit (metadata → L1) | slices of the memory file | the skill's description | slices of the summary |
Two properties distinguish this from ADR 0007's directory routing:
- Extraction is one-to-many, not routing. A single turn can yield a user fact and a
reusable skill and a project-state update; each extractor takes what it needs from the
same trajectory. Directory routing was one-to-one by construction and could never do this. - The intelligence moves from the caller into the pipeline. Nobody pre-sorts sources; the
classify/extract step (the L0 → L1preprocessseam of each line, per ADR 0007) decides
what a trajectory contains. Callers — human, hook, or code — just hand over raw trajectory.
This resolves ADR 0007's open "skill source" question: skill is distilled from the
trajectory, not a primary source line fed by a separate /log folder. It also shifts the
project line's meaning from "indexed multimodal files" (the old workspace framing) toward
project memory; direct file ingestion (pointing memu memorize at a curated folder)
remains supported as a secondary path on the same line.
2. Two integration surfaces, layered — not two parallel products
- The API/CLI is the base plane.
memu memorize/memu retrieve(Decision 3; and the
in-process service behind them) are the single source of truth. Everything writes to and
reads from the same per-project store defined in ADR 0007. - Hooks are a zero-code wrapper built on that base plane. The hook integration is not a
second engine; it is an adapter layer that calls the same CLI/service on the agent's
behalf. Because both surfaces share one store, a project can mix them freely — record
via hooks, retrieve explicitly in code, or vice versa — with no divergence.
Surface A — Hook integration (zero-code, default distribution)
An agent loads memU's SKILL.md for capability description, and a host adapter wires memU
into the agent's lifecycle. The adapter does exactly three things, and only these three:
- Load — make
SKILL.md(and the store location) visible to the agent as a capability. - Record trajectory — on each turn, append the user input and the agent's action/tool
trace as raw trajectory, then trigger memorize. The adapter does no sorting: the
pipeline's extraction step (Decision 1) distills memory / skill / project from it. - Inject memory — before the agent generates, retrieve against the current prompt and
splice the results into the agent's context.
The hook contract (one abstraction, N thin adapters)
Host hook systems are heterogeneous, so memU defines one contract with two seams, and each
host ships a thin adapter that maps its native events onto them:
| Seam | Fires | Does | memU call | Execution |
|---|---|---|---|---|
on_turn |
after a turn completes (user msg + agent trace available) | append the turn's raw trajectory, trigger memorize (extraction distills memory/skill/project) | memu memorize |
async / background, non-blocking |
on_prompt |
before the agent generates | retrieve for the current prompt, splice hits into context | memu retrieve |
synchronous, in the critical path |
The adapter's whole job is to bind a host's native hook events to on_turn / on_prompt.
Concretely:
- Claude Code —
settings.jsonhooks: aStop/post-turn hook →on_turn; aUserPromptSubmithook →on_prompt. - Codex / OpenClaw / Hermes — each maps its own post-turn and pre-generation events onto
the same two seams.
New hosts are added by writing one adapter, not by touching the kernel or the CLI.
Execution model (per the decision above)
- Recording is asynchronous.
on_turnenqueues; memorize runs in the background and must
never block the conversation. It reuses ADR 0007's incremental manifest diff, so repeated
turns are cheap and only changed sources are processed. - Injection is synchronous.
on_promptdoes a single-shotmemu retrieve(embedding +
BM25 hybrid search, no LLM per ADR 0007) in the critical path, under a token budget, and
fails open: a miss, a timeout, or an empty store injects nothing and the turn proceeds
normally.
Surface B — Programmatic API (code-level control)
Builders embedding memU call retrieve / memorize directly from their agent code (in-process
service or CLI/JSON), choosing their own timing, batching, and scope (user_id / where).
This is the base plane exposed directly — the same calls the hook adapters make, with no hook
layer in between.
Shared invariants (hard constraints)
- One store, one semantics. Both surfaces target the same per-project store and the same
three lines. Hook mode must not invent a private store or a divergent record format. - Scope convention preserved. The ADR 0007
user_id/wherescope convention holds
across both surfaces, so cross-surface queries ("everything about user X") work regardless
of how it was written. - Hooks add no engine. Any behavior available via hooks must be expressible as base-plane
calls; the adapter is orchestration only.
3. CLI: reclaim `memu memorize` / `memu retrieve` as the canonical pair
With the lines renamed (Decision 1), "workspace" no longer names a line — leaving thememorize-workspace / retrieve-workspace commands carrying a retired term that now reads
as "one line" when they actually operate on the whole store. Rename the pair:
memu memorizeandmemu retrievebecome the only commands, carrying the new
semantics (trajectory ingestion + extraction on the write side; hybrid search over all
three lines on the read side).- The legacy single-file
memorize/retrieve(already deprecated, documented as
never-fall-back) are removed first; the names are reclaimed only after removal, so no
release ships two behaviors under one name. memorize-workspace/retrieve-workspaceare removed outright — no alias, no
deprecation window. This is a breaking major-update cut; adapters andSKILL.mdship the
canonical names in the same release.
The accepted risk: both prior command generations break at once — old legacy users find the
reclaimed names doing something new, and -workspace users get command-not-found. Accepted
deliberately: this ADR is the major update, and a clean cut beats carrying a retired term in
an alias.
What this changes
- The terms unify around the extractions: the three lines are canonically named
memory / skill / project (renaming ADR 0007'schatandworkspace); docs, CLI
output, and code should converge on these three terms. - The output changes shape:
MEMORY.md/SKILL.md/INDEX.mdare gone; each line
yields a folder of L1 documents (memory//skill//project/, one file per document)
plus a pure embedding index of L2 items whose metadata points back to L1. - The source model inverts: from "users curate per-line source folders" to "the trajectory
is the source; extraction curates." Directory-based routing (chat/→ memory,agent/→
skill) becomes a legacy/secondary ingestion path. - The project line's center of gravity shifts from multimodal file indexing to project
memory; ADR 0007's "skill source" open issue closes as distilled from trajectory. - The CLI collapses to one pair:
memu memorize/memu retrieve(new semantics); both
the legacy single-file pair andmemorize-workspace/retrieve-workspaceare removed —
no aliases. SKILL.mdgraduates from "instructions a human/agent follows to run the CLI" to "the
capability surface a host adapter loads," while remaining valid for manual use.- memU gains a small host-adapter layer (one thin adapter per host) as the unit of new
integrations, with the two-seam contract as its only coupling to the engine. - The API surface is stated as a first-class, supported integration path, not just an
internal detail behind the CLI.
Consequences
Positive:
- Drop-in users get continuous cross-session memory with no code changes; builders keep full
programmatic control — from one engine, not two. - Host heterogeneity is contained: adding Codex/OpenClaw/Hermes/etc. is one adapter against a
fixed two-seam contract, not a bespoke integration each time. - Async record + sync inject keeps the conversation responsive (no per-turn LLM blocking)
while still injecting fresh memory before each generation. - Mixing surfaces is safe by construction because both hit the same store with the same scope
convention.
Negative / costs:
- Extraction is not free. Directory routing was structural (zero-cost); semantic
extraction over trajectory is LLM work on every memorize, and a misclassifying extractor
pollutes lines silently — quality now depends on prompts, not conventions. - N host adapters to build and maintain; each host's hook system evolves independently and can
break its adapter. - Async recording means a just-finished turn may not yet be retrievable on the immediately
following prompt (eventual, not read-your-writes, consistency) — accepted for
responsiveness. - Synchronous injection adds retrieve latency to every prompt; bounded by the token/time
budget and fail-open behavior, but non-zero. - Auto-recording every turn raises privacy/scope questions (what gets persisted, and under
whoseuser_id) that manual CLI use did not.
Open issues (deferred)
- Injection shape & budget. Where retrieved memory lands (system-prompt append vs. a
dedicated context block), and the exact token budget / ranking cutoff, are unspecified here. - Record batching policy. Per-turn vs. per-session flushing, and debounce/coalescing of
rapid turns, are left to the adapter; a shared default is TBD. - Extraction quality bar. What counts as a skill-worthy procedure or a project-worthy
state change (vs. noise) is a prompt/heuristic question for the extraction step; the
one-to-many contract is decided here, the classifier itself is not. - Trajectory format. The wire format a host adapter emits (full transcript vs. structured
turn events, tool-output truncation) is per-adapter for now; a shared schema is TBD. - Privacy controls. Opt-out, redaction, and per-turn scoping of auto-recorded content are
not designed yet.
Related ADRs
- Builds on
docs/adr/0007-three-independent-memory-lines-wiki-graph.md(engine, three lines,
store,memorize-workspace/retrieve-workspace); refines its source model —
trajectory replaces per-line source folders as the primary source, its "skill source"
open issue is resolved as distilled from trajectory, itschat/workspacelines
are renamed memory / project, and its renderedMEMORY.md/SKILL.md/INDEX.mdoutputs are replaced by per-line L1 folders plus a pure per-line embedding
index of L2 items. - Builds on
docs/adr/0001-workflow-pipeline-architecture.md(both surfaces drive the same
workflow pipelines).
Adr 0009 Codex Packaging Cli And Config (`docs/adr/0009-codex-packaging-cli-and-config.md`)
ADR 0009: Packaging the Codex Integration — pip, a CLI Seam, and One Config Loader
- Status: Accepted
- Date: 2026-07-13
- Builds on: ADR 0008 (two integration surfaces — hooks over a programmatic API), ADR 0007
(three lines / per-project store) - Scope: how the Codex integration is installed and invoked on a user's machine — the
distribution, the retrieval invocation seam, and the runtime configuration. It does not
revisit the engine or the source model settled in 0007/0008.
Context
ADR 0008 defined what the Codex integration is: a host adapter binding two seams —on_turn (record) and on_prompt (inject) — onto the same base-plane memu calls every
other surface uses. It deliberately left the packaging open.
Codex realizes those two seams concretely, and the realization forces three packaging
questions 0008 did not answer:
- Record (
on_turn) is realized not as a per-turn hook but as a scheduled "bridging"
task (seesrc/memu/hosts/codex/BRIDGING_TASK.md): a cron job that periodically walks new
turns under~/.codex/sessions, runs a prepare → self-evolve → commit pipeline, and writes the
results back to the store. Two of its three steps are code (prepare,commit) thatimport memudirectly. - Inject (
on_prompt) is realized as a UserPromptSubmit hook that adds a single
instruction telling the agent to retrieve memory relevant to the current query. The agent
then issues a retrieval call itself.
That leaves three decisions the installer and the two seams must agree on:
- Distribution. How is
memuput on the user's machine — pip (Python) or npm (a JS
reimplementation of the same surface)? - Retrieval invocation. When the inject hook fires, what does the agent actually run — a
wrapped Python script, or a CLI command? - Configuration.
MemoryServiceneeds a non-trivial config (at minimum a database DSN
and an embedding/LLM provider). Today the bridging scripts construct it with a placeholder
(MemoryService() # fill in config if needed). The recording seam (writes) and the
injection seam (reads) must construct identical config or retrieval reads a different
store than bridging wrote. How is that config decided at install time and shared?
The load-bearing constraint across all three: record and inject share one store and one
embedding space (ADR 0008's "one store, one semantics"). A query is embedded at inject time
and compared against vectors written at record time; if the DSN or the embedding
provider/model differ between the two seams, the comparison is meaningless and retrieval
silently returns nothing.
Decision
1. Distribute via pip; do not ship a parallel npm package
memU is installed as a Python package (pip install memu-cli), exposing two console entrypoints:memu (the library's own surface) and memu-codex (the host adapter — see Decision 2). No npm
package is built for this integration.
Rationale — a Python runtime with memu importable is not optional on the Codex path: the
scheduled bridging task's prepare and commit steps import memu directly. Given Python is
mandatory for the record seam regardless, npm cannot replace it — it can only add
a second runtime and a second implementation of the retrieval surface, which must then be kept
byte-for-byte behavior-compatible with the Python one (the embedding gateway, cosine_topk
vector search, the repo layer, and the whole pydantic config surface thatAgenticMixin.progressive_retrieve touches). An npm package that merely shells out to the CLI
adds a dependency without removing the Python requirement. npm would be the right call only if
the goal were a zero-Python toolchain; the scheduled task makes that impossible here.
One runtime, one implementation, one thing to keep in sync.
2. Both seams are invoked through a CLI command, never a script path
Neither seam points the agent at a python /abs/path/to/script.py invocation. Both are PATH
commands on a host-adapter binary:
- inject →
memu-codex retrieve "<query>" - record →
memu-codex prepare/memu-codex commit(embedded in the scheduled task's prompt)
There is no semantic difference between a command and a script — both end up calling the sameMemoryService method. The difference is entirely operational, and every operational axis
favors the command:
- Reliable invocation. After
pip install, the binary is onPATHand the agent emits a
stable string. A script forces the agent to reproduce an exact<VENV_PYTHON> <ABS_PATH>pair,
and baking an absolute script path into a cron prompt — which is where the record seam lives —
breaks the moment the directory moves or the venv is rebuilt. This was the original design and
it was the most fragile thing about it. - One config path. The CLI centralizes config construction (Decision 3). A standalone script
re-implements it and invites drift — and drift here is not cosmetic, it silently breaks
retrieval. - Stable contract.
memu-codex retrieve <q>is a versioned interface; a file path is not.
Which retrieval. The inject hook must not call memu retrieve — that is the LLM-routed
path (intention routing, sufficiency checks, summarization), far too heavy to run on every turn.
It calls AgenticMixin.progressive_retrieve: LLM-free, single-shot, embeds the query once and
ranks segments/files/resources.
Correction. An earlier draft of this ADR justified a new subcommand by claiming
"retrieve-workspaceruns the heavy LLM-routed workflow." That is false.retrieve_workspace
is already LLM-free and single-shot, and returns the same three layers asprogressive_retrieve; the two differ only in thatprogressive_retrieveexecutes the stages
inline rather than through the workflow engine. The heavy LLM-routed command isretrieve
(no suffix). The decision to exposeprogressive_retrievetherefore rests on where the verb
lives — on the host adapter, next to the hook that calls it — not on a capabilityretrieve-workspacelacks. Whether skipping the workflow engine buys enough to justify the
separate code path is an open question (see below), not an established one.
Which binary. Retrieval is exposed on the host adapter (memu-codex retrieve), not on the
core memu CLI. The core binary stays the algorithm surface; host plumbing does not accrete onto
it. Because the inject seam is identical across hosts — the hook differs, the retrieval does not —
the implementation lives once in memu.hosts.retrieval and each host CLI registers it.
3. One config loader, sourced from a persisted env file — not per-script placeholders
Configuration is expressed as the existing MEMU_* environment variables (already defined
by the CLI: MEMU_DB, MEMU_LLM_PROVIDER, MEMU_API_KEY, …). We do not invent new config
keys. Two refinements make them safe for the Codex seams:
- Persist to a file, not the interactive shell. Install writes the collected values to a
dotenv at a known absolute path (~/.memu/config.env). Scheduled tasks run with no reliable
working directory and cannot be assumed to inherit an interactive shell's exported
environment, so a file every entrypoint loads is the robust carrier; shellexports in a
profile are not. Precedence is process environment > file > default, so a one-off override
needs no file edit. - One shared loader is the single source of truth. The service-construction logic that lived
insidecli.py(_build_service/_database_config) is promoted tomemu.env
(build_service_from_env()/database_config()), and every entrypoint calls it: thememuCLI,memu-codex retrieve, and the bridging pipeline's prepare and commit. TheMemoryService() # fill in config if neededplaceholders are gone.
We frame this as single source of truth, deliberately not as "best-effort loading."
Best-effort implies a silent fallback to defaults — which is precisely the failure mode to
avoid: an entrypoint that falls back to the default ./data/memu.sqlite3 while bridging wrote to
the configured DSN will "succeed" and retrieve nothing. Missing required config must surface, not
default: build_service_from_env() raises when MEMU_DB is unset rather than guessing.
Minimum an install must collect: MEMU_DB (the DSN) and the provider + API key. The provider
choice is load-bearing beyond credentials — it selects the embedding model, and record and
inject must embed in the same space (see Context); the shared loader guarantees they do.
What this changes
- The Codex integration is a pip install of the existing package; no npm artifact is
produced or maintained. - A second console script,
memu-codex, is added alongsidememu. It carries both seams —retrieve(inject), andprepare/commit/verify-resources(record) — plusdoctoranddocs. The corememuCLI is unchanged: no new subcommand, no new flag. - The
skills/codex-memu-task/folder is deleted. Its scripts move into the package undersrc/memu/hosts/, split by what varies:hosts/bridging/is the host-agnostic pipeline
(cursor, job templates, snapshot/diff, commit),hosts/base.pyis the only host seam
(TranscriptSource: where sessions live, how a record is shaped), andhosts/codex/is ~50
lines of session parsing plus a thin CLI. A second host is aTranscriptSourceand a CLI, not
a forked pipeline. cli.py's service-construction helpers are extracted intomemu.env
(build_service_from_env/database_config), used by the CLI and the bridging pipeline; the
twoMemoryService() # fill in configplaceholders are removed.- Install grows a step that writes
~/.memu/config.envwith the chosenMEMU_*values;
every entrypoint reads it, so record and inject provably agree on store and embedding space. - Two agent-facing docs ship inside the wheel at
src/memu/hosts/codex/:INSTALL.md(the
three-part setup) andBRIDGING_TASK.md(the scheduled-task procedure), printable viamemu-codex docs. Nothing else needs to be distributed — there is no skill folder for a user
to copy and no script directory for a cron prompt to point at.
Consequences
Positive:
- One runtime and one implementation of the retrieval surface — nothing to keep behavior-synced
across languages. - The inject hook and the scheduled task are guaranteed to hit the same store and embedding
space, so retrieval actually finds what bridging wrote (the correctness invariant is
structural, not a matter of the user setting two things consistently). - Both seams are stable
PATHcommands. Nothing machine-specific is baked into a hook config or
a cron prompt, so neither breaks when a directory moves or a venv is rebuilt. - Config is decided once at install time and lives in one file read by every seam.
- The distributed footprint is nothing. The user installs a package; the guides ride along
inside it. There is no folder to copy into a skills directory and no download step. - The core
memuCLI keeps describing the algorithm. Host plumbing lives on the host binary, somemu --helpdoes not degrade into a grab-bag as hosts are added.
Negative / costs:
- Users without a usable Python environment cannot adopt the integration; there is no JS-only
path (accepted — the scheduled task requires Python anyway). ~/.memu/config.envstores an API key in plaintext on disk; its permissions and lifecycle
are an install concern (see open issues).build_service_from_envcouples the CLI and the bridging pipeline to one helper; a breaking
change to it now touches every seam at once (the intended trade-off — that shared fate is
exactly what keeps record and inject in agreement).- A per-host binary means each new host re-exposes
retrieve. Mitigated by implementing it once
inmemu.hosts.retrievaland having host CLIs register rather than redefine it, but the verb
is still duplicated across--helpoutputs. - The agent-facing guides are now versioned with the package: fixing a wording bug in
INSTALL.md
requires a release. Accepted — it is the same "versioned interface, no drift" trade the CLI
decision makes, and it guarantees the guide matches the installed code.
Open issues (deferred)
- Secret handling. Whether the API key belongs in
~/.memu/config.envat all (vs. a
keychain / OS secret store) is unspecified. Install currently setschmod 600. - Concurrent hosts. The session cursor is per-host
(.session_manifest.<host>.json), but~/.memu/jobs,~/.memu/sessions, and the touched-file
log are shared and wiped per run. Two hosts' bridging tasks running at the same time would race.
Fine while Codex is the only host; must be settled before the second one ships. - Multi-project stores. A single
~/.memu/config.envassumes one store per machine; how
per-project stores (ADR 0007) are selected at inject time on Codex is not addressed here.
Related ADRs
- Builds on
docs/adr/0008-two-integration-surfaces-hooks-and-api.md— this ADR is the
concrete Codex packaging of its two seams:on_turnrealized as the scheduled bridging task,on_promptrealized as the UserPromptSubmit inject hook, both over the samememubase
plane. - Builds on
docs/adr/0007-three-independent-memory-lines-wiki-graph.md— the "one store, one
embedding space" invariant this ADR's config decision protects is that ADR's per-project
store.
Adr 0010 Multi Host Adapters (`docs/adr/0010-multi-host-adapters.md`)
ADR 0010: Multi-Host Adapters — Claude Code, Cursor, OpenClaw, and Hermes
- Status: Accepted
- Date: 2026-07-16
- Builds on: ADR 0008 (two integration surfaces), ADR 0009 (packaging, CLI seam, one config
loader) - Scope: adding four host adapters beyond Codex, and the two pieces of shared structure that
shipping a second host forced: a common CLI builder, and per-host working trees. It does
not revisit the pipeline, the seams, or the config decisions of 0008/0009.
Context
ADR 0009 closed with a claim and an IOU. The claim: "a second host is a TranscriptSource
and a CLI, not a forked pipeline." The IOU: the working directories (~/.memu/jobs,~/.memu/sessions, the touched-file log) are shared and wiped per run — "two hosts' bridging
tasks running at the same time would race. Fine while Codex is the only host; must be settled
before the second one ships."
Four hosts now ship at once: Claude Code, Cursor, OpenClaw, and Hermes Agent.
Each was located empirically (inspecting a live machine and, for OpenClaw and Hermes, the
hosts' own source) rather than from documentation alone:
| Host | Session log | Container | Record shape |
|---|---|---|---|
| Codex | ~/.codex/sessions/**/*.jsonl |
JSONL per session | {timestamp, payload: {type, role, …}}; type: message (user/assistant) vs function_call/function_call_output |
| Claude Code | ~/.claude/projects/<escaped-cwd>/<session-uuid>.jsonl (subagent transcripts in per-session subdirs) |
JSONL per session | {type: user|assistant, timestamp, message: {role, content}}, one content block per record: text / tool_use / tool_result / thinking; plus non-message noise types (queue-operation, attachment, system, pr-link, last-prompt) and isMeta user records |
| Cursor (Agent/CLI) | ~/.cursor/projects/<escaped-cwd>/agent-transcripts//.jsonl |
JSONL per session | {role, message: {content: [blocks]}}; text and tool_use blocks, often in one record; no timestamps. The IDE's Composer chats live in the editor's state.vscdb SQLite and are out of scope |
| OpenClaw | ~/.openclaw/agents/<agentId>/agent/openclaw-agent.sqlite (one database per agent), plus legacy <agentId>/sessions/<sessionId>.jsonl (-topic- suffix for topic sessions; sessions.json index alongside); root moves with OPENCLAW_STATE_DIR |
SQLite, WAL mode — transcript_events(session_id, seq, event_json, created_at), one entry per row — and JSONL per session, read together |
parent-linked entry tree, identical in both containers (event_json is the entry verbatim): {type: session|message|reset|custom…, id, parentId, timestamp, message: {role, content}}; roles user/assistant/toolResult |
| Hermes Agent | ~/.hermes/state.db (moves with HERMES_HOME; ~/.hermes/sessions/saved/ holds only manual snapshots) |
SQLite, WAL mode | sessions + messages tables; OpenAI-shaped rows: role (system/user/assistant/tool), content, tool_calls, tool_call_id, epoch-seconds timestamp |
And the inject seam's landing file per host:
| Host | Instruction file | Why |
|---|---|---|
| Codex | ~/.codex/AGENTS.md |
global, loaded every session (ADR 0009) |
| Claude Code | ~/.claude/CLAUDE.md |
the global memory file, loaded in every project |
| Cursor | ./AGENTS.md (per project) |
Cursor's CLI honors no global instruction file; User Rules are IDE settings out of a CLI's reach |
| OpenClaw | ~/.openclaw/workspace/AGENTS.md |
the workspace file loaded at the start of every session |
| Hermes | ~/.hermes/SOUL.md |
the one file loaded from HERMES_HOME regardless of cwd; project files (.hermes.md, AGENTS.md) would miss sessions started elsewhere |
Two of the four break assumptions Codex allowed:
- Hermes's log is not files. The
TranscriptSourcedefaults (rglob, line reads, string
timestamps) assume JSONL-on-disk. 0009's seam anticipated this ("a host with a different
container overridesdiscover,read_records,timestamp"), but no host had exercised it. - Claude Code's roles lie. A tool result is logged as a user-typed record. Any
classification keyed on role alone would put tool output in the conversation transcript the
memory job reads.
Decision
1. The host CLI is built from a declaration, not copied
Five binaries share one verb set — retrieve, install-instruction, prepare, commit,verify-resources, doctor, docs — because the behavior behind every verb is host-agnostic.
So the parser and handlers move to memu.hosts.host_cli, built from a HostSpec: the host id,
the session-log default, the instruction-file default, and the docs package. A host's cli.py
is now the spec plus main, ~40 lines; memu-codex is rebuilt on the same spec with byte-
identical behavior.
The standing-instruction text (ADR 0009's managed block) is likewise parameterized on the host
binary rather than hardcoding memu-codex; each host's block names its own retrieve, and the
per-binary begin-marker means two hosts pointed at one file manage two independent blocks.
Codex's rendered block is unchanged, so already-installed markers still match and upgrade in
place.
What stays per host is exactly what 0009 predicted — a TranscriptSource — plus the two paths
(session log, instruction file) and the packaged guides. Cursor's discover is narrowed to*/agent-transcripts/**/*.jsonl (the project dirs also hold canvases and terminal logs);
Hermes overrides the container seam wholesale: sessions are virtual paths keyed by session
id, read_records serializes message rows to JSON lines ordered by insertion id, anddiscover orders by last activity so the manifest's early-stop stays sound. The database is
opened read-only — the bridging task must never contend for Hermes's WAL write lock. The
line-count cursor carries over untouched: message rows are append-only per session, so "lines
seen" is "rows seen."
2. Working trees are per host; the store stays shared
0009's open issue is settled by removing the sharing, not by locking: every host's Layout
gets its own base directory, ~/.memu/hosts/<host>/, holding its jobs, sliced sessions,
mirrors, manifests, and touched-file log. Concurrent bridging runs of different hosts touch
disjoint trees. Codex keeps ~/.memu: its job-file paths are baked into users' already-
scheduled task prompts (the exact fragility 0009's PATH-command decision was protecting
against — the prompt references ~/.memu/jobs/*.txt literally), and breaking every existing
install to make five directories symmetrical is a bad trade. The asymmetry is recorded inHostSpec.base_dir with a comment saying why.
What is not per host: ~/.memu/config.env and the store behind MEMU_DB. That is the
point of the whole exercise — a session mined from Claude Code tonight is retrievable from
Cursor tomorrow. The install guides instruct an agent that finds an existing config.env to
reuse it as is, because a second host writing a second store/provider would silently split the
embedding space (0009's core invariant).
3. Classification is by record shape, not role
Each host's classify encodes its log's actual semantics, pinned by tests with hand-written
records in the host's real shape:
- Claude Code: block type decides —
text(or a raw-string user message) is conversation;tool_use/tool_resultare tool records regardless of the wrapping role;thinkingblocks,isMetauser records, and all non-message types are dropped. - Cursor: a record with prose is conversation even when it also carries the
tool_use
blocks it narrates; only bare tool-block records go to the tool transcript. No timestamps
exist, so the manifest recordsnulland the line count alone drives incrementality. - OpenClaw:
type: messagewith role user/assistant is conversation, roletoolResultis
a tool record; session headers, compaction summaries, and extension entries are dropped.
Timestamps may be ISO strings or epoch millis; both normalize. - Hermes: role
tool, and assistant rows carrying onlytool_calls, are tool records;
user/assistant rows with content are conversation;systemrows are dropped. Epoch-seconds
timestamps normalize to ISO.
What this changes
- Four new console scripts —
memu-claude-code,memu-cursor,memu-openclaw,memu-hermes— each with packagedINSTALL.md/BRIDGING_TASK.mdguides printable viadocs, alongside the unchangedmemu-codex. memu.hosts.host_cli(new):HostSpec+ the shared parser/handlers.codex/cli.py
shrinks to its spec; its public names (build_parser,AGENTS_MD,HOST,VERIFY_COMMAND) survive.memu.hosts.instruction:INSTRUCTION/BEGINconstants becomeinstruction(binary)/begin(binary)templates;install/patch/registertake the binary.memu.hosts.base.TranscriptSourcegainsexists()(default: root is a directory) so a
SQLite-backed host can gateprepareon its file instead.- New host packages:
hosts/claude_code,hosts/cursor,hosts/openclaw,hosts/hermes—
each asessions.py, a spec-sizedcli.py, and the two guides.
Consequences
Positive:
- The 0009 claim is now demonstrated: four hosts, zero pipeline forks, and the next host is a
classifymethod, two paths, and two markdown files. - Concurrent bridging across hosts is safe by construction (disjoint trees), with no locking
protocol to get wrong. - One instruction text and one CLI surface to improve; all five binaries pick up fixes at once.
Negative / costs:
- Codex's
~/.memuvs everyone else's~/.memu/hosts/<host>is a visible asymmetry, carried
for compatibility. A future major release could migrate Codex in. - The recall-file mirror is written once per host per run instead of once per machine —
redundant disk writes, accepted for isolation. - Cursor's inject seam is per project (no global file), so its instruction install is a
per-project step the guides must (and do) call out. - Host log formats are observed, not contracted. OpenClaw already has a SQLite session target
behind a flag; if it becomes the default, that host needs the Hermes treatment. The
per-host fixture tests localize such breaks.
Open issues (deferred)
- Same-host concurrency. Two bridging runs of the same host still race on that host's
tree; scheduled tasks make this unlikely, and per-run lock files remain available if it
bites. - IDE-container hosts. Cursor's Composer history (
state.vscdb) and any host whose log
lives inside an editor's private state would need the SQLite treatment plus a stability
story for schema drift; not attempted. - Session-dir env vars. OpenClaw (
OPENCLAW_STATE_DIR) and Hermes (HERMES_HOME) can
relocate their logs; the adapters take--session-dirrather than reading the host's env,
keeping the contract explicit. The guides say when to pass it.
Related ADRs
- Builds on
docs/adr/0008-two-integration-surfaces-hooks-and-api.md— each new host binds
the same two seams. - Builds on
docs/adr/0009-codex-packaging-cli-and-config.md— settles its "concurrent
hosts" open issue via per-host working trees; keeps its distribution, CLI-command, and
config decisions unchanged.
Adr 0011 Generic Host Adapter (`docs/adr/0011-generic-host-adapter.md`)
ADR 0011: A Generic Host Adapter — Detect First, Then Bind What Works
- Status: Accepted
- Date: 2026-07-16
- Builds on: ADR 0008 (two seams), ADR 0010 (multi-host adapters, shared CLI builder)
- Scope: how memU integrates with agents that have no dedicated adapter. It does not
change the five dedicated hosts or the pipeline.
Context
ADR 0010 shipped five dedicated adapters and noted the recipe for a sixth: a classify
method, two paths, two markdown files. That recipe assumes someone writes the adapter. The
long tail of agents — new CLIs appear monthly — will mostly never get one, yet many of them
are integrable today, because the ecosystem has converged on a handful of session-log
dialects and a handful of instruction-file names.
Two observations make a generic adapter viable:
- Session logs cluster into ~5 JSONL dialects. Payload-wrapped (Codex lineage), typed
message trees (OpenClaw/pi lineage), typed block records (Claude Code lineage),
role+message blocks (Cursor lineage), and flat OpenAI-client chat rows. A sniffingclassifythat tries these in order covers most JSONL-logging agents without any
per-agent code. - The two seams degrade independently. An agent with a readable session log gets
memorization (the record seam) even if we cannot find its instruction file; an agent
with only anAGENTS.md/CLAUDE.md/SOUL.md-style file gets retrieval (the inject
seam) even if its sessions are unreadable (SQLite, editor state, or absent). Partial
integration is still integration — but only if the user is told which half they got.
Decision
One more binary, memu-agent (package memu.hosts.generic), with the shared verb surface
plus one new verb:
detectprobes before anything binds. With no argument it surveys~for agent
directories; with a path it probes that directory. For each candidate it answers the two
questions in order — is there a session log whose records sniff as a known dialect?
(memorization), and if not or additionally, is there an instruction file to patch?
(retrieval) — and prints an explicit per-agent verdict: memorization works / retrieval
works / both / neither, with the found paths and the exact next command. Directories of
the five dedicated hosts are redirected to their own binaries. Sessions in containers the
sniffer cannot read (SQLite and friends) are reported as such — the Hermes precedent, not
silence.- The transcript source sniffs per record.
GenericTranscriptSource.classifytries the
five dialects in order; a record matching none isOTHER. This is the same fail-closed
posture the dedicated adapters take with their hosts' metadata records: the mining jobs
see only what provably is conversation or tool traffic, so a half-recognized log degrades
to less input, never to garbage input. - Nothing is defaulted that detect must find.
prepare --session-diris required (theHostSpecmarks no universal location), andinstall-instructiontakes the--path
detect reported. The one machine-specific value the bridging prompt carries is that
session dir — a deliberate, documented exception to ADR 0009's "nothing machine-specific
in the prompt", because for an unknown agent there is noPATH-stable name to hide it
behind. - Several generic agents share one binary, not one working tree. The default tree is
~/.memu/hosts/agent/; integrating a second unknown agent means passing--base-dir ~/.memu/hosts/<name>in that agent's task, keeping ADR 0010's isolation
guarantee.
Consequences
Positive:
- Any agent logging a known JSONL dialect integrates with zero new code, and any agent with
a recognizable instruction file gets retrieval regardless. - The user always learns which seams work and why — the detect report is the contract,
so a retrieval-only integration is a stated outcome rather than a silent half-failure. detectdoubles as the scouting tool for new dedicated adapters: an unrecognized-dialect
or SQLite report is exactly the evidence a sixth adapter starts from.
Negative / costs:
- Sniffing is heuristic. A new dialect classifies as
OTHER(missed memories, safe), and a
pathological log could mis-bucket records (wrong-track memories, bounded by the dialects'
mutual exclusivity). The dedicated adapters remain the correctness bar. detect's survey reads directory listings across~/.*(capped per directory); it is
read-only but not free, and its skip-list of non-agent dirs needs occasional tending.- The required
--session-dirbakes one machine-specific path into the generic bridging
prompt — accepted, see above.
Related ADRs
docs/adr/0010-multi-host-adapters.md— the shared CLI builder (HostSpec) this reuses,
extended here withregister_extra(host-specific subcommands) and required-when-blanksession_dir.
Adr 0012 Cloud Backed Agentic Backend (`docs/adr/0012-cloud-backed-agentic-backend.md`)
ADR 0012: Cloud-Backed Memory Behind the Existing CLI
- Status: Accepted
- Date: 2026-07-24
- Builds on: ADR 0008 (two host seams), ADR 0009 (one CLI/config surface)
- Scope: selecting local or hosted execution for the three agentic memory
operations. It does not changeMemoryService, storage repositories, or the
host command surface.
Context
Every memu and memu-<host> operation currently constructs a localMemoryService. That requires a local database plus an embedding provider even
when the hosted memU service already exposes the same three operations over its
API-key-authenticated v4 API.
Creating separate cloud binaries would duplicate every host command, retrieval
instruction, scheduled task, and installation path. Adding HTTP behavior toMemoryService would instead turn the local composition root into two unrelated
runtime roles and weaken its embedding-only, storage-pluggable boundary.
Decision
One structural capability boundary
AgenticMemoryBackend is a structural protocol containing exactlylist_all_recall_files, progressive_retrieve, and commit_results.MemoryService already satisfies it and remains the local in-process
implementation. CloudMemoryClient implements it by forwarding requests to:
GET /api/v4/memory/POST /api/v4/memory/searchPOST /api/v4/memory/
The cloud client returns successful response dictionaries unchanged. It owns
Bearer authentication, explicit timeouts, structured cloud errors, and bounded
retries for transient failures. None of those concerns enter memu.app.
One selector for every caller
build_agentic_memory_backend_from_env() is the only backend selector used by
the main CLI, host retrieval, bridging prepare/commit, and doctor.MEMU_MEMORY_MODE accepts local or cloud; absent means local, preserving
existing installs.
Cloud configuration is separate from local embedding configuration:
MEMU_MEMORY_MODE=cloud
MEMU_CLOUD_API_KEY=<memu-api-key>
MEMU_CLOUD_BASE_URL=https://api.memu.so/api/v4/memory/The base URL has the production value above as its default and remains
overridable for compatible deployments. Users obtain a memU API key by
registering at memu.so. MEMU_BASE_URL and MEMU_API_KEY
keep their existing local-embedding meaning.
The main memu CLI retains its local database/embedding flags. They are passed
as local-mode overrides to the shared selector; cloud mode does not construct a
local store as a side effect.
Explicit default owner scope
Cloud requests always send both user_id and agent_id. Missing, blank, orNone values become the literal default. Cloud reads accept only exactuser_id and agent_id filters because the public API cannot represent local
repository operators such as user_id__in; unsupported filters fail rather
than silently widening scope.
Resource compatibility is visible, not forked
The bridging pipeline remains identical in both modes: it prepares resource
work and submits commit_results.resource. The current cloud API accepts that
field for wire compatibility but returns an empty resources list because it
does not persist workspace resources yet. Doctor and installation documentation
state this limitation explicitly; memory and skill recall files are durable.
Consequences
Positive:
- Existing agent commands, scheduled tasks, and retrieval instructions work in
either mode without rewrites. MemoryServicekeeps its three-method public surface and remains free of HTTP
and authentication behavior.- Cloud credentials cannot be confused with embedding-provider credentials.
- An old config with no mode keeps its local database and embedding space.
Costs:
- The cloud client has a deliberately narrower
wherevocabulary than a custom
localUserConfig.model. - Resource-description jobs still run in cloud mode even though the service
currently does not persist their output. This preserves one bridging
pipeline at the cost of temporary redundant work. - The project API key may be stored as plaintext in
~/.memu/config.env; guides
require user-only file permissions and process-environment overrides remain
available.
Out of scope
- Dashboard cookie/CSRF APIs
- OAuth, browser login, or OS keychain integration
- Local-to-cloud memory migration
- Cloud resource persistence
- New CLI binaries
- Storage protocol, backend, or migration changes
Adr 0013 Self Updating Instruction Templates (`docs/adr/0013-self-updating-instruction-templates.md`)
ADR 0013: Self-Updating Instruction Templates
- Status: Accepted
- Date: 2026-07-25
- Builds on: ADR 0008 (two host seams — record and inject), ADR 0012 (config in
~/.memu, one shared backend selector) - Scope: how the agent-facing instruction text the SDK carries — the three
self-evolve job templates and the retrieval body — stays current between SDK
releases. It changes neither the bridging pipeline's structure nor the host
command surface.
Context
Four blocks of text drive the agents memU steers, and all four ship embedded in
the SDK as Python string constants:
MEMORY_JOB_TEMPLATE,SKILL_JOB_TEMPLATE
(memu/hosts/bridging/instructions.py) andRESOURCE_JOB_TEMPLATE
(memu/hosts/bridging/resources.py) — the job templates, filled with
concrete paths atpreparetime and handed to the self-evolve agent.RETRIEVAL_BODY(memu/hosts/instruction.py) — the retrieval body,
installed into a host'sSKILL.mdor its managed instruction block, and read
by the agent on every turn. The server publishes it as a full skill document
(retrieval-skill.txt— frontmatter,# Retrieve…heading, and body): a skill
host writes it verbatim, and an inline host carves out just the body
(_skill_body) to slot into its own managed block.
Embedded text is a floor, not a ceiling. A wording fix or a better mining prompt
cannot reach an install until that install upgrades the SDK, and installs upgrade
rarely. We want these four to track a server-hosted copy so an improvement
reaches the field without a release — while never becoming dependent on the
server, since the embedded copy is always shippable-correct.
The two kinds of template are consumed on very different duty cycles, which the
design must respect:
- Job templates are read only on the
preparerun — hourly-to-daily,
latency-tolerant, off the user's interactive path. Pulling every run is fine. - The retrieval body backs a file the host re-reads every turn, and the
retrievecommand runs on that same per-turn hot path. It must never fetch
there. But its durable copy already lives on disk (the installedSKILL.md/
managed block), so it needs refreshing only occasionally.
Decision
One fail-open resolver, `memu/hosts/templates.py`
A single module fetches https://memu.pro/sdk/instructions/<name>.txt for the
four names memory-job, skill-job, resource-job, retrieval-skill, and is
fail-open by construction: a missing network, a non-200, an oversize body
(> 64 KiB), or a malformed template all collapse to "fall back". No call raises.
The base URL defaults to https://memu.pro/sdk/instructions and is overridable
with MEMU_TEMPLATE_BASE_URL; setting it empty switches remote refresh off
entirely (air-gapped installs, offline CI, the test suite).
The /sdk/ path segment marks these as SDK-consumed assets, not a human-facing
page. There is deliberately no version segment today (see Open issues).
Validation is the trust boundary
These templates are instructions an autonomous agent executes with shell access,
and the job templates are str.format()-ed with concrete paths downstream. A
server-side typo must therefore be rejected before it is trusted, never carried
into a job file where .format() would raise mid-run. A fetched template is used
only if it is non-empty, contains every required {placeholder} for its name,
and .format()-s cleanly against exactly those keys — which rejects a dropped
placeholder, a stray brace, and an unknown field alike. The same check re-runs on
every cache read, so a truncated or hand-edited cache file is treated as absent.
Two fallback shapes, matched to the two duty cycles
resolve(name, embedded)— server → last-good cache → embedded. Used for
the job templates atpreparetime. A validated fetch also writes the text to
a local last-good cache (~/.memu/cache/instructions/<name>.txt, shared across
hosts since the templates are host-agnostic). The cache is what makes a
transient outage a non-event: once an install has seenv1, a later day
whose pull fails still runsv1rather than regressing to the embeddedv0.fetch(name)— server only, no cache. Used for the retrieval skill
(retrieval-skill), served as a fullSKILL.mddocument. Its durable copy is
not a cache but the file already installed on disk, so there is nothing extra to
cache.install-instructionusesfetch(...) or embedded(an explicit, one-off
command; the embedded floor is correct when the server is down mid-install). The
scheduled refresh (below) usesfetch(...)and skips the write entirely onNone, so a transient outage leaves the already-installed — and newer — copy
in place rather than downgrading it.
The retrieval body refreshes on the bridging schedule, never on retrieve
instruction.refresh() rewrites whatever holds the retrieval body — theSKILL.md on a skill host (the server document verbatim), the inline managed
block on an inline host (just the body, carved out of that document by_skill_body) — and is called from _cmd_prepare. This piggybacks the refresh on the low-frequencyprepare run the host already schedules, giving the "larger interval" the hot
path requires for free, with no timestamp bookkeeping. It is best-effort
(a failure is a note, never a failed prepare), skip-not-downgrade (see
above), and refresh-not-bootstrap (it does nothing where nothing is
installed yet — installing remains install-instruction's job, so a scheduled
run never creates files a user did not ask for).
Consequences
Positive:
- A prompt improvement reaches every install on its next
prepare, without a
release. - The embedded copy remains a guaranteed floor; the server is pure upside and
never a runtime dependency. A total outage degrades to embedded, never to
nothing. - The per-turn
retrievepath is untouched — it still performs no network I/O
for templates. - A malformed server template cannot crash the pipeline or reach an agent.
Costs / limitations:
preparenow makes up to four short, timeout-bounded HTTP calls per run. This
is acceptable on its duty cycle and fully disabled byMEMU_TEMPLATE_BASE_URL="".- The server copy is trusted after structural validation but is not
cryptographically verified (see Open issues).
Open issues
No version numbers on the cache (deliberate, for now). The last-good cache
keeps the last text seen, not the newest by version. One regression follows
from this: if the SDK is upgraded while the server is unreachable, a cached
older-server copy can momentarily win over a newer embedded template, because
nothing lets the resolver compare their ages. We accept this because it
self-heals on the next successful pull — and a release publishes the matching
server copy, so that pull is imminent — and because it can only affect the
low-frequency job templates for at most a run or two (the retrieval body is
uncached, so it is immune). The fix, when the infrastructure is ready, is a
monotonic version integer on each template so the offline branch can pick the
newest of {cache, embedded} while a reachable server stays authoritative
(letting deliberate rollbacks still win). Not built yet.No signing / integrity verification. Because these templates are executed
by an agent with shell access, a compromisedmemu.pro/sdk/instructions/…is a
meaningful supply-chain surface. Today's guards are HTTPS, a pinned host, a size
cap, and structural validation. Shipping a public key in the SDK and verifying
a detached signature over each template is the recommended hardening.No URL schema-version segment. The placeholder set each template requires
is a contract between the server copy and the installed SDK. Today an old SDK
that fetches a server template needing a new placeholder simply rejects it in
validation and falls back — safe, but it silently forgoes the upgrade. A/v1/
segment bumped only when the placeholder contract changes would let the server
serve contract-compatible copies per SDK generation. Cheap to add later; left
out now to keep the first cut simple.
Out of scope
- Versioned or signed template distribution (tracked above).
- Any change to what the templates say — only to how they are delivered.
- The bridging pipeline's prepare → self-evolve → commit structure (ADR 0008).
- Backend selection and cloud/local execution (ADR 0012).
Adr 0014 Paginated List All Recall Files (`docs/adr/0014-paginated-list-all-recall-files.md`)
ADR 0014: Paginated list_all_recall_files
- Status: Proposed
- Date: 2026-07-26
- Builds on: ADR 0012 (cloud-backed agentic backend), ADR 0007 (three memory
lines / tracks), ADR 0003 (user scope in the data model) - Scope: making
list_all_recall_filesreturn one bounded page per call across
both the local and cloud backends, and having the only two callers reassemble
the full set by following a cursor. It does not changeprogressive_retrieve,commit_results, or thewherevocabulary.
Context
list_all_recall_files returns every recall file in scope in a single
response. Today AgenticMixin.list_all_recall_files callsrecall_file_repo.list_recall_files(where), which loads the whole result set,
and both callers consume all of it at once:
- bridging
preparemirrors the entire store to disk; - the
list-filesCLI prints the whole list.
As a scope's memory/skill set grows, that single response grows without bound:
one large DB read, one large serialization, one large HTTP body. We want a
bounded page size.
Two facts about the topology constrain the design:
There is one implementation, not two. The hosted service does not have
its own query code for this operation —memu-serviceruns open-source
memU'sAgenticMixinunchanged (MemuAgenticHost(AgenticMixin)) over its
own storage adapter, and the HTTP route returns the mixin's dict untouched
("the wire shape is memU's by construction"). So the paginated response
shape must be born in the mixin; the cloud client and the server route
cannot synthesize a shape the mixin does not produce.Both execution paths must stay byte-for-byte identical.
localruns the
mixin in process;cloudruns the same mixin behind one HTTP roundtrip
(ADR 0012). Pagination — page size, ordering, cursor encoding — therefore
belongs to the mixin and its repositories, not to any one transport.
Decision
Paginate at the source, keyed on `(track, name, id)`
AgenticMixin.list_all_recall_files gains cursor and limit parameters and
returns {"recall_files": [...], "next_cursor": <token-or-null>}. A nullnext_cursor means the last page. The page is produced by a new repository
method (below), ordered by the composite key (track, name, id).
Why (track, name) is the ordering key — the crucial decision:
- It is the domain's own identity, so it is unique by construction, not by
accident. A recall file is keyed bynamewithin itstrack
(memory/skill) — seecommit_results. Pagination always runs inside a
fixed(user_id, agent_id)scope, so within one paging sequence(track, name)is unique. Uniqueness is the one hard requirement for correct paging:
with no ties, no page can skip or duplicate a row, which is exactly what a
caller reassembling the full set needs. - It is immutable under the write path. There is no in-place rename; a
"rename" is delete-then-create. A row therefore never changes its sort
position mid-pagination, which is what keyset resumption relies on. - It produces a meaningful order for free — stable alphabetical
list-files
output, and a debuggable cursor — where the primary-key alternative (a random
UUID,RecallFile.id) would order arbitrarily and carry no business meaning.
id is appended as a final tiebreaker (ORDER BY track, name, id). (track, name) is already unique today, so this is defense-in-depth: it makes the total
order unconditional if the uniqueness invariant is ever relaxed (a migration
window, soft-deleted duplicates) at zero cost.
Keyset, not offset
The cursor encodes the last row's (track, name, id) and each page resumes with(track, name, id) > (cursor). Offset pagination drifts under the concurrent
commits that prepare races against: a delete positioned before the offset
shifts every later row up and silently skips one. Keyset over an immutable
unique key does not drift. The cursor is an opaque token (base64 of the tuple)
so callers cannot depend on its internals.
Page at the repository, not in memory
The page is a bounded repository query
(WHERE scope AND (track,name,id) > cursor ORDER BY track,name,id LIMIT n+1),
added as a new method rather than by slicing the existing full-fetch in the
mixin. In-memory slicing would re-read and re-sort the entire table on every
page — on the multi-tenant server, N page calls become N full scans, which is
worse than the single read we set out to shrink. A keyset query touches only one
page's worth of rows per call. (The in-memory backend "queries" a sorted list,
so paging there is a trivial slice — but the method signature is uniform.)
The new method is additive:
list_recall_files_page(where, *, after, limit)
-> (ordered list[RecallFile], next_after | None)The existing list_recall_files(where) -> dict[str, RecallFile] is left
untouched: its other callers — the _collect_files file roll-up and the
per-track lookup in _commit_recall_files — genuinely need the full,
unordered set, and must not be forced through a cursor.
One page per call at the transport; callers reassemble
CloudMemoryClient.list_all_recall_files forwards cursor/limit as query
parameters and passes the paginated body through unchanged — one page per call,
no client-side loop. Reassembly lives in the callers, where it composes with
work they already do per item:
- bridging
preparefollowsnext_cursor, writing each page to disk as it
arrives — no change to its per-file mirror logic, and the whole store is never
held in memory at once; - the
list-filesCLI followsnext_cursorto gather the full list before
printing (its contract is "show everything").
limit defaults to a fixed page size (DEFAULT_PAGE_LIMIT); callers do not set
it.
Server-side changes (hippocampus-server)
The mixin change flows to the server through the pinned memU git dependency; the
service's own code changes only to thread the two new parameters through and to
implement the paged query on its storage adapter:
- HTTP route (
app/interfaces/http/api/memory_files.py,GET /api/v4/memory): acceptcursor: str | Noneandlimit: intasQuery
parameters (boundlimitwithge=1, le=<max>) and forward them. The return
value is still the mixin's dict, passed through untouched, sonext_cursor
reaches the wire for free. - Engine (
app/infrastructure/memu/engine.py,MemuAgenticEngine.list_all_recall_files): acceptcursor/limitand
forward them intohost.list_all_recall_files(where=where, cursor=cursor, limit=limit). No shape work — the result already passes through_run
unchanged. - Storage adapter (
MemuV4Store's recall-file repository): implementlist_recall_files_pagewith the same keyset predicate andORDER BY track, name, idover the multi-tenant schema. This is the one place
the server supplies its own query code (it satisfies memU's repository
protocol), and it must order identically to memU's own repositories or the
two execution paths diverge. An index covering(scope…, track, name, id)
keeps each page seek-only.
Consequences
Positive:
- Response, serialization, and DB-read size per call are bounded by
limit
regardless of how large a scope grows. - Local and cloud paths remain identical: same ordering, same cursor, same page
shape, because pagination lives in the shared mixin + repository contract. preparenever materializes the whole store in memory; it streams pages to
disk.- The ordering key is the domain identity, so correctness (no skips/dups) does
not depend on an incidental property.
Costs:
- The repository protocol gains a method that every backend must implement
(SQLite, in-memory, Postgres here;MemuV4Storeon the server). A backend
that forgets it, or orders it differently, breaks paging silently. list_all_recall_filesis no longer a single call; both callers carry a
cursor loop, and any future caller must remember to follownext_cursor
rather than reading one page and stopping.- The cursor's meaning (
(track, name, id)keyset) is now part of the
cross-repo contract; changing the sort key later invalidates in-flight cursors.
Out of scope
- Paginating
progressive_retrieve(already bounded bytop_k) orcommit_results. - Widening the
wherevocabulary (ADR 0012 keeps cloud reads to exactuser_id/agent_id). - A stateful/snapshot cursor that hides concurrent inserts appearing on a later
page; keyset gives read-committed semantics, which is sufficient forprepare
(a missed just-created file is picked up on the next run). - Total-count or
has_more-beyond-next_cursormetadata.
Adr 0015 Bridging Must Not Mine Its Own Run (`docs/adr/0015-bridging-must-not-mine-its-own-run.md`)
ADR 0015: The Bridging Run Must Not Mine Itself — Identity from the Host, Gate from the Launch
- Status: Proposed
- Date: 2026-07-29
- Builds on: ADR 0008 (trajectory as the source), ADR 0010 (multi-host adapters), ADR 0011
(generic host adapter) - Scope: how a bridging run recognises the host session it is itself running in, so
preparenever slices its own bookkeeping into a mining job. It does not change the
pipeline, the cursor,classify, or any host's log format.
Context
ADR 0008 chose trajectory as the source: memorization is not an API call the user makes but
a scheduled agent run that reads what the agent recently did. ADR 0010 then made that run
one shape across hosts — a TranscriptSource plus a thin CLI.
Neither ADR wrote down the consequence: the record seam produces trajectory too. The
bridging task runs as a session of the host agent, and the host logs that session in exactly
the directory (or table) memU discovers sessions from. So:
- every run hands the next run fresh "new" content, and
preparecan never report zero —
killing the idle signal that bothINSTALL.mdand the silent-blindness diagnosis of #528
lean on; - the mining jobs chew through memU's own
prepare → self-evolve → commitbookkeeping; - worst of the three, those transcripts are the newest on disk, so newest-first discovery
floats them above real conversation and they consume theMAX_JOBSslots. That is the same
harm #532/#533 fixed for trajectory/checkpoint sidecars, arriving from a different source.
Measured on one machine, counting only sessions that were actually sliced into a job:claude_code 19 of 95 tracked sessions (2,111 of 25,833 records); hermes 4 of 5. Onhermes the entire mined "conversation" of one such session was a single run summary whose
own words were "bridge sessions meta-only" — the mining agent had already noticed what it
was reading.
Tracked as #606.
Decision
1. Identity comes from the host, never from the transcript
A run learns which session it is in from the host, throughHostSpec.session_id_env — the environment variable the host uses to hand a tool subprocess
its session id (CLAUDE_CODE_SESSION_ID). prepare executes inside that session, so this
is an exact identity, not an inference. The id is appended to~/.memu/hosts/<host>/.self_sessions.<host>.json, host-scoped for the same reason the cursor
file is, and discover()'s results are filtered against it.
TranscriptSource.session_id() maps a discovered session back to that same id. The default
is the file stem; hosts whose naming carries more than the id override it.
The rejected alternative — a sentinel string in the bridging prompt, matched against the
transcript — fails on two independent grounds, and both are worth recording because the idea
is the obvious one:
- Content flows. Paste the prompt into a chat to debug it, and it is mined into a memory;
retrievethen injects that memory into an unrelated session, which now matches the rule
and is skipped. Silent loss of real conversation. A rule that can be quoted can be forged. - Prompt prose drifts. #591 caught a live host prompt diverging from canon
("shell tool" vs "bash") with the sync test not noticing. A sentinel tied to prose breaks
the moment the prose is revised; an environment variable is not prose.
Filtering on the host's own scheduling metadata (created_via = 'cron') was also rejected:
#532 settled that cron is not noise — a cron-driven prompt is a session-driving instruction —
so that filter would discard the user's own scheduled conversations. This decision is
strictly narrower.
2. Only the *scheduled* run may claim a session — the gate is the launch, not the command
Running prepare is not evidence that a session belongs to bridging. It is an ordinary
command, run during development and — more importantly — as a deliberate "remember this
conversation now". Treating its execution as the signal excludes the very session the user
asked to have mined, permanently and for every later run: the opposite of the request, and
silent.
So two facts must hold, and they come from different places:
the launcher : knows "this is a bridging run" · does not know the session id
prepare : knows the session id · does not know why it is runningThe launch-side fact is carried by MEMU_BRIDGING_RUN, exported by the invocation memU
controls, or by the run's working directory being the host's memU tree (schedule passes-WorkingDirectory; the Unix wrapper cannot rely on this, since cron's cwd is $HOME).
Either suffices. Anything unrecognised is treated as a person: nothing recorded, nothing
skipped — failing open toward the pre-#606 behaviour, never toward dropping user data.
Both signals live in the invocation, not in the conversation, so neither is forgeable by an
injected memory. That is what separates them from the sentinel in decision 1.
3. Skip by `continue`, never by `break`
The scan stops at the first already-seen unchanged session, which is sound only because
discovery is newest-first. Self-sessions are the newest files on disk, so ending the scan
there would hide every real session beneath them. They are passed over instead.
4. The gate has three grades, and a host's grade follows from how it is scheduled
Hosts differ in who schedules the run, and this is the part that does not generalise for
free:
- memU writes the invocation (
claude_code,cursor,hermes— crontab, launchd, Task
Scheduler). The wrapper exports the marker. Strong: no prose, no compliance. - The host schedules it natively and the payload is a prompt (
openclaw'sopenclaw cron,codex,workbuddy,cola's native task UI). There is no wrapper to write, so the
marker can only ride in the documented command inside the prompt. It is still read from the
process environment rather than matched against a transcript, so it cannot be forged — but
it depends on the agent reproducing the command, and #591 is the evidence that prompts
drift. Weaker, and it fails open. - The host records which scheduled job created the session. Where this exists it is
better than either. OpenClaw persists every isolated run under the exact structural keyagent:<agentId>:cron:<jobId>:run:<sessionId>insession_windowsbefore the model starts.
memU records the stable job id once at install time, then checks every per-agent store and
accepts only canonical cron-run rows containing that registered job id and the row's own
session id. The agent segment is ignored: it is mutable routing metadata, not bridging
identity. This follows agent-owner changes without persisting or comparing mutable routing
state. It needs neither marker nor
environment variable, does not select the newest row, and never puts the concrete session
id into prompt or tool content. Preferred wherever a host offers it.
A host's grade is not a fixed property of the host — it follows from what its guide tells the
user to register. codex sits in the second grade because its guide creates a native Codex
scheduled task; registering codex exec under an OS scheduler instead would move it to the
first. So the grade is something a guide can improve, not something to accept.
So each new host adapter owes one exact identity seam appropriate to its launcher. An
OS-scheduled host supplies the variable name (surveyed on a real install, on Unix — Windowsos.environ is case-insensitive and will hide a casing error), a session_id() that returns
what it holds, and a launch-side marker. A native-scheduled host may instead register the
exact task identity and resolve sessions from structural scheduler metadata. Both feed the
same remembered self-session list and continue filter; their identity sources need not be
forced into one shape.
Consequences
- Every host is a separate survey. As of this ADR
claude_codeis verified end to end
(2.1.220: headlessclaude -pexports the variable, and its value is the transcript's file
name), and Hermes is verified to exportHERMES_SESSION_IDto tool subprocesses during a
one-shot run, matching its SQLite session id. Hosts withsession_id_envempty keep the
pre-#606 behaviour, which is why the default is empty rather than guessed. - Codex still needs a
session_id()override before an environment seam can be wired
(rollout-<ts>-.jsonl— the stem is not the id). OpenClaw instead uses its native
scheduler metadata and therefore does not ask the run to report its own id. Hermes's
virtual paths now returnpath.name, matching the SQLite id without truncating ids that
contain a dot. - Subagent transcripts must be attributed to the session that spawned them, not to their own
file name: a run that spawns a subagent produces bookkeeping under a different name. Claude
Code nests them two ways (<sessionId>/subagents/…and<sessionId>/subagents/workflows/<wf>/…), so attribution reads the first directory under
the project slug rather than the parent directory. - The fix is forward-only. Bridging sessions already recorded stay in the cursor and are
inert — a finished session gains no new lines, so it is never re-offered. - Claiming a session is permanent and therefore announced:
prepareprints the id and the
file to remove. The one remaining false positive is a person runningpreparefrom inside
the host's memU tree, which is visible and reversible. - Upgrading an existing OS-scheduled install is not automatic. Unix users must re-copy the
wrapper to pick up the marker; Windows tasks already pass-WorkingDirectory, so the
directory signal covers them untilschedule installis re-run. OpenClaw users instead
register the existing memU cron job id once; the recurring prompt then returns to a plainmemu-openclaw preparecall.
Out of scope
genericcannot be wired by memU, because memU does not know what it is. ADR 0011
makes the host indeterminate by design, and both halves are missing rather than just one:
there is no known variable to read, and no invocation that can be assumed to be ours to
mark — the guide suggests cron, but a generic agent may well have a scheduler of its own,
and memU never sees which the user chose. The escape is the one ADR 0011 already relies on:
the user supplies what detection cannot.session_id_envis data, so a flag could carry it
for an agent that does export an id. Deliberately not decided here.- Wiring the remaining hosts, pending their surveys.
- Assigning the session id ourselves (
claude --session-id) was considered: it would
make the id known before launch, covering a run that dies beforeprepare. Rejected as
narrower than it looks — it needs both that memU launches the host and that the CLI has
such a flag, which of the CLIs surveyed onlyclaudedoes. Recorded so it is not
re-proposed as a general answer. - Not writing the run's session at all. Some agent CLIs appear to offer this — one machine's
claude --help(2.1.220) andcodex exec --help(codex-cli 0.144.5) each list such a flag.
Neither was run: the behaviour above is read off a one-line help string, and which
versions have it is unknown, so nothing in this ADR rests on it. If it does what it says, it
would remove the problem rather than skip it — but only where memU constructs the
invocation, and at the cost of the transcript a failed run is debugged from. Verify before
building on it.
Related ADRs
- Builds on
docs/adr/0008-two-integration-surfaces-hooks-and-api.md— names the consequence
of "trajectory as the source": the seam emits trajectory of its own. - Builds on
docs/adr/0010-multi-host-adapters.md— addssession_id_envandsession_id()to what a host declaration owes, alongside its existing fields. - Builds on
docs/adr/0011-generic-host-adapter.md— the indeterminate host is the one memU
cannot wire on its own; if it is wired, the user supplies the missing value, as with--session-dir.
Adr 0016 Client Event Reporting (`docs/adr/0016-client-event-reporting.md`)
ADR 0016: Client Event Reporting — One Envelope, a Spool by Default, Bounded Payloads
- Status: Proposed
- Date: 2026-07-31
- Builds on: ADR 0008 (two host seams — record and inject), ADR 0009 (the CLI seam and one
config loader), ADR 0010 (multi-host adapters), ADR 0012 (cloud-backed backend, config in~/.memu), ADR 0013 (contacting a memU server fail-open, never as a dependency) - Scope: how the CLI reports a small, fixed set of lifecycle events to a memU ingest endpoint
— the envelope, the transport, the identity, and what is allowed inside a payload. It
changes neither the record nor the inject seam's behaviour, the store, the templates, nor
any host's install procedure beyond adding two calls to the guides.
Context
memU today is blind to its own operation. An install either works or the user files an issue;
a bridging task that silently stops producing memory looks identical, from the outside, to one
that had nothing to mine; a retrieval that returns zero hits is indistinguishable from a
retrieval that was never wired. Every diagnosis in #528, #538, #606 started from a human
noticing and reporting, which selects hard for the loudest failures and against the quiet ones
— and the quiet ones are the dangerous class, because memU's core promise degrades silently:
record and inject can disagree about the store and nothing looks broken (ADR 0009's opening
argument).
The backend wants a fixed event envelope, of which the CLI fills every field:
{
"event_id": "0195dcad-94c0-7fd9-a17b-68e051d339ba",
"event_name": "memory_search_succeeded",
"client_type": "memu_cli",
"client_instance_id": "7d2df95e-5f09-4db7-b70f-c1ea14095ba1",
"occurred_at": "2026-07-29T08:30:12.123Z",
"context": {
"client_version": "2.0.0b0",
"agent_platform": "claude_code",
"os": "macos",
"deployment_mode": "local",
"reported_by": "code",
"session_id": "0ad71768-1599-4ac5-946f-a74b297113a9"
},
"properties": {"result_count": 8, "latency_ms": 412}
}context carries the environment dimensions rather than being the reserved empty object it was
at first writing: the deployed ingest schema stopped accepting client_version,agent_platform, os, deployment_mode and session_id as top-level fields, and an envelope
that sends them there is rejected as a permanent 4xx — which the flush discards rather than
retries (§2), so the events are lost outright. Same values, same sources; only their place in
the envelope moved. session_id is still omitted rather than faked (§7), so the key is absent
from context rather than present and empty.
Six things are to be reported: install completed, uninstall, a bridging (remember) run
finished, a retrieval, a listing of the store, and a fatal error.
Four facts about this codebase constrain the answer, and each one rules something out:
retrieveruns on every turn, so its reporting cost must be a constant. This is already
written down and already load-bearing:_refresh_retrievalexists precisely so the retrieval
body updates on the low-frequencypreparerather than on the per-turn hook (host_cli.py).
As first written this constraint was "must never fetch", and the retrieve event was
spool-only. Amended: the backend asked for that event promptly and accepted the round
trip, soretrievenow delivers its own single envelope inline, on both its success and
its failure leg — see §2. What the amendment does not relax is the bound: aflush()there
would drain the whole spool at one POST per event, up toMAX_FLUSH_POSTS, which is a cost
that grows with how far behind the machine has fallen. One request per turn is a constant; a
flush is not, and the difference is the entire content of this constraint.- Install and uninstall are agent-driven prose, not commands.
INSTALL.mdis a
multi-part guide with verify gates;UNINSTALL.mdlikewise. No code path spans either, so
no code path can observe that one succeeded. - Local mode has no credential.
MEMU_CLOUD_API_KEYexists only in cloud mode, and a
local-mode user chose "this device" explicitly (INSTALL.mdPart 1.2). - There is a working precedent for talking to a memU server.
memu.hosts.templates
contactsmemu.proon a short timeout, with a size cap, catching everything and returningNone. Its docstring states the principle this ADR inherits wholesale: the server is pure
upside and never a dependency.
Decision
1. One module, one envelope, filled from what the CLI already knows
A new top-level memu/events.py, peer to env.py and cloud.py — top-level because both thememu core binary and every memu-<host> adapter emit events, and neither owns the other.
Every envelope field has exactly one source, and none of them requires new plumbing:
| Field | Source |
|---|---|
event_id |
uuid4(), generated by the client at the moment the event is recorded |
event_name |
Per §4's name list — one name per action and outcome; the error events per §5 |
client_type |
Constant "memu_cli" |
client_instance_id |
Per §6 |
occurred_at |
RFC 3339 UTC with milliseconds, stamped when the event is recorded, not when it is sent |
context.session_id |
Per §7; omitted when unavailable |
context.client_version |
Per §8 |
context.agent_platform |
HostSpec.host, normalised per §9 |
context.os |
platform.system() mapped to macos / windows / linux, else other |
context.deployment_mode |
env.memory_mode() — already returns exactly local / cloud |
context.reported_by |
"code" when the client observed the event itself, "agent" when a model asserted it — per §4 |
properties |
Per §10 — an allowlist, never free-form |
occurred_at and delivery time are deliberately decoupled (§2), so they will routinely differ
by hours. The backend must keep its own receive timestamp and must not treat occurred_at
as monotone or trustworthy: it is a client clock, and client clocks are wrong.
2. Two transports; the spool is the only one wired
The active path is accumulate-then-flush. Recording an event appends one JSON line to a
shared spool at ~/.memu/events.jsonl and returns. Nothing is sent. Flushing takes the whole
spool, POSTs it as a batch, and only then discards it.
This is not merely the polite choice for the per-turn hook — it is the choice that makesevent_id mean anything. Client-generated ids buy idempotence for a client that retries. A
fire-and-forget sender never double-delivers, so it would have nothing for the backend to
deduplicate; a spool that survives an offline laptop and re-POSTs on the next flush does. The
two decisions are one decision.
Flush points, all of them low-frequency and latency-tolerant:
prepareandcommit— the bridging pair, already the designated place for
latency-tolerant server contact.report uninstall— synchronously, before returning (§4). Uninstall is the one event
that cannot wait for a later flush, becauseUNINSTALL.mdPart 3 may remove the very binary
that would perform it.init,docs install, andreport install— the whole install funnel (§4). A machine
that abandons the install never reachesprepareorcommit, so these are the only flushes
its events may ever get.docs installhas already contacted the docs server by then, so it
costs nothing new;initis a local command and pays one bounded, fail-open POST for being
the earliest point the funnel can be seen from. Amended:report installwas originally
spool-only, on the reasoning that an install which succeeded would go on to bridge and flush
there. That reasoning assumes the thing the event attests to. An install whose scheduled task
never registered still reachesINSTALL.md's last step and still reports completing — and it
is exactly the run that never flushes, so the funnel would lose its terminal row on the
machines whose funnel matters most. Leaving one step of three spooled also made the
completion's delivery latency differ from its own siblings' for no gain a consumer could use.report error— synchronously, before returning (§5), and for the same reason asdocs installrather than asreport uninstall: the runs that file an error are
disproportionately the runs that never reachprepareorcommit. A failed install, a
store the bridging pair cannot talk to, aretrievethat has silently returned nothing for
a week — in every one of those the later flush this event would wait for is the thing that
is broken. The cost is bounded and already paid: an agent that stopped to file an error has
spent longer deciding than the POST takes, and fail-open means an unreachable endpoint costs
the timeout, not the event, which stays spooled for the next attempt.report flush, an explicit verb, for guides and for debugging.- Not
retrieve. It delivers inline on both legs, but through the single-event path below
rather than a flush; the distinction is one POST versus all of them. See "the immediate path"
below. - The CLI's top-level error handler — except when the failing command is
retrieve. Flushing there is right for the bridging pair, where the thing that
broke is the normal flush point, so waiting for it would mean waiting forever.
It is wrong for the per-turn hook: a store the hook cannot reach fails it on
every turn, so a drain on that path would cost up toMAX_FLUSH_POSTSrequests
per turn, precisely when the user is already broken. Caught by exercising a real
install against a live endpoint, not by reasoning — constraint 1 alone does not
say that the error path is covered too. This exemption is unchanged by the
amendment below: what it rules out is the unbounded drain, not reporting, and thememory_search_failedenveloperetrievedelivers for itself already says the
same thing at constant cost. Only thecli_errorbehind it waits.
Mechanics, stated because they are what make a shared spool safe:
- Appends are single-line
O_APPENDwrites. Two hosts share one spool, since identity (§6) is
machine-scoped, not host-scoped. - A flush first
os.replaces the spool to a uniqueevents..sendingfile, then POSTs
it, then unlinks it. Concurrent appends land in a fresh spool; a crashed flush leaves a.sendingfile that the next flush picks up alongside the current spool. - A failed POST leaves the
.sendingfile in place. It is retried on the next flush. - The spool is capped (size and line count). Past the cap, appends are dropped and a counter is
carried into the next successful flush, so the loss is reported rather than silent. An
offline machine runningretrieveevery turn is the case this exists for. - A batch is capped, so one flush after a long offline stretch cannot post an unbounded body.
The immediate path is wired, for retrieve and nothing else. memu/events.py carries a
single-event blocking POST with a short timeout — the same shape as templates._get, in the
other direction. It was written dormant, reserved for an event that some future decision showed
could not tolerate spool latency; the backend's request for prompt retrieval telemetry is that
decision, and record(deliver=True) is how it is spent. It is not fire-and-forget: the
envelope is POSTed once, and anything the server did not settle (a timeout, a 5xx, a 429) is
appended to the spool for the bridging pair, exactly as if it had never been attempted. Only a
permanent 4xx is dropped, because that is a payload the server will refuse every time — the
same call _flush makes. A bare send would lose precisely the events an offline laptop
generates, which is the case the spool exists for.
Two consequences are accepted rather than solved:
- Events can arrive out of order. A delivered retrieve event overtakes everything still
spooled behind it.occurred_atsays when each event happened andevent_idmakes a replay
idempotent, so no consumer needs arrival order; a client that preserved it would have to
drain the spool first, which is the cost this design exists to avoid. - A broken
retrievecosts one POST per turn. Both legs deliver. Amended: the failure
leg was originally spool-only, on the reasoning that a store the hook cannot reach fails it
on every turn, so that is when a per-turn blocking POST is least affordable. The reasoning
held for the cost and inverted the value. A machine whose retrieval is broken is the one this
whole feature exists to surface, and the later flush its event would wait for runs on the
bridging pair — which the same broken store breaks. Spooling made the least healthy machine
the quietest one, which is the failure mode §5 names as its motivation. So the cost is
accepted and named rather than avoided: a fully broken store adds up to_TIMEOUT_SECONDSto
each turn's hook until it is fixed. That is a constant, it is bounded by the timeout rather
than by the backlog, and it is the same constant the success leg already pays. What is not
accepted is a drain, which is why the error handler'sretrieveexemption above stands.
Note that report uninstall does not use this path — it triggers an ordinary flush inline,
which is a spool operation, and it needs the whole spool gone rather than one event sent.
Both paths are fail-open in the ADR 0013 sense: no network, a non-2xx, a timeout, an
unwritable spool, a corrupt spool line — every one of them is swallowed. No event failure
may ever change a command's exit code or its output. A retrieve that cannot record an
event is a successful retrieve.
3. Auth is optional and record-only
The ingest endpoint requires no authentication; the backend owns rate-limiting and abuse. The
client's rule is therefore exactly: send Authorization: Bearer <key> if a key happens to
exist, omit the header otherwise. Local-mode users, who have no key, are first-class.
Concretely, this reads env("MEMU_CLOUD_API_KEY") directly, not env.cloud_api_key() —
the latter raises when unset, which is correct for its own caller and exactly wrong here. The
client never resolves, prompts for, or validates a credential for telemetry's sake, and never
fails on its absence.
4. Code-observed where possible, an explicit verb only where prose is unavoidable
The dividing line is whether any code can observe the completion.
Instrumented in place, no new command surface:
| Event | Call site |
|---|---|
| retrieve | retrieval._cmd_retrieve — one inline POST on either leg, never a flush (§2) |
| remember finished | host_cli._cmd_commit — the terminal step of the record seam, which already holds the recall-file and resource counts |
cli_error |
host_cli.run's top-level except — see §5 |
cli_install_started |
config_cmd._cmd_init, after the config write — see below |
install_guide_opened / uninstall_guide_opened |
host_cli._cmd_docs, on the two lifecycle guides and not on docs task — see below |
An explicit verb, registered once in host_cli.build_parser so every host inherits it:
memu-<host> report install
memu-<host> report uninstall
memu-<host> report error --stage <enum> [--detail "<short>"]
memu-<host> report flushinstall and uninstall need a verb because of constraint 2: the guides are prose, and only
the agent knows it reached the final ✅ gate. install-instruction was considered as a stand-in
and rejected — it also runs on re-runs, partial repairs, and legacy-path migrations, so it
means "the inject seam is patched", which is a different proposition from "install succeeded".report install goes at the end of INSTALL.md; report uninstall goes in UNINSTALL.md
before Part 3 removes the package.
Neither verb takes a failure flag, and the success verbs still say only that it worked.
Failure travels on report error, whose --stage already covers both operations.
That much is unchanged. What is changed, deliberately and against this section's first
writing, is that report error --stage install|uninstall|remember now also emits a concretecli_install_failed / cli_uninstall_failed / memory_update_failed alongside §5'sagent_error_reported. The original text rejected a --failed flag on the grounds that it
"would duplicate that channel with a second, coarser one and force every consumer to join
them", and that objection was sound about a flag: two ways for an agent to say the same
thing, chosen by the model. It does not carry to a second event derived from the one report,
because the choice stays single — the agent still says one thing, in one place, and the client
fans it out to two consumers who want different shapes of it. The concrete event is a funnel
counter, joinable by name to its _started and _succeeded siblings; agent_error_reported is
the triage inbox that carries the prose. Neither is coarser than the other; they are the same
fact projected for two different queries.
The cost this buys is real and must be stated rather than discovered: succeeded + failed
never equals started. The success and start legs of install and remember are code-observed
or verb-driven, while the _failed leg rests on a model choosing to call report error, which
undercounts for every reason this section already lists. The _failed events are a lower
bound on failure; the honest failure count remains started − succeeded. That is exactly the
arithmetic a name family invites someone to get wrong, which is why provenance is carried
explicitly rather than left to be inferred from the name:
context.reported_by is "code" or "agent", on every envelope. It sits in context
beside client_version rather than in properties because it is a fact about how the envelope
came to exist, not about the thing it reports — one line at the construction site, and no key
added to any event's allowlist. With it, "failures we observed" and "failures a model told us
about" are one predicate apart even where they share a name family, and a query that mixes them
is visible in review instead of silently wrong.
Stages get a concrete event only where an agent is actually taught to use them. install,uninstall, and remember are named in INSTALL.md, UNINSTALL.md, and BRIDGING_TASK.md,
so their reports arrive with enough regularity to count. retrieve and other are reachable
but unmentioned by design (see Open Issues), so a concrete event for either would be a name
that is almost always zero — and in retrieve's case an actively harmful one. A code-observedmemory_search_failed means "the retrieve command raised"; an agent reporting --stage retrieve means "retrieval has been silently returning nothing for a week", which is the
failure this entire feature exists to surface (§5). Routing the second into the first would
bury the rare signal inside the common one. Both stages therefore report throughagent_error_reported alone, and gain a concrete event only if and when an instruction directs
an agent to them.
Only the completion needs a verb; the way in is observed. Two code-observed steps precede
it. cli_install_started is recorded by init, and install_guide_opened by docs install.
All three steps flush where they are recorded, so provenance is the only asymmetry left in the
funnel and delivery is not one of them. The provenance asymmetry is the point:report install is voluntary and undercounts, and a start that undercounted independently of
it could report more completions than attempts, which is worse than no funnel at all. Taking
both steps in code makes started >= opened >= succeeded hold structurally, since SKILL.md
runs them in that order and the guides are reachable only through docs install.
The install-instruction objection above does not carry over. It was rejected as a stand-in
for completion because it also runs on re-runs and partial repairs; for a step on the way
in, a re-run is a new attempt, which is the correct reading rather than a defect.
Why the start sits at init and not at docs install. It was originally at docs install,
on the argument that printing the guide is the first act that proves memu-cli is installed
and resolving. init (ADR 0017) is now SKILL.md Step 2 and proves the same thing one command
earlier, while additionally holding the two facts the event most wants: the memU Cloud key, so
the first envelope of an install is attributable rather than anonymous, and MEMU_CLIENT_ID,
which it mints into config.env itself. The start is therefore emitted after the write —
before it, client_instance_id() would find no id, generate a second one, persist it, and be
overwritten, leaving the machine's first event reporting under an id its own config no longer
contains. ADR 0017's open issue "where the install-start event belongs" is closed by this.
install_guide_opened is the old emission, renamed rather than deleted: what it observes is
unchanged and still worth a row, but it is now the funnel's second step, and a second name
ending in _started beside the real one is the pair a consumer sums by accident. It carries nocli_ prefix — the prefix marks what the client observed about itself, and this observes a
document being asked for — and the gap between the two counts installs abandoned before they
had begun.
uninstall_guide_opened is that same observation on the way out, added later and for the gap
rather than for symmetry. cli_uninstall_succeeded is agent-reported and undercounts like
every other verb, and the removal path had no code-observed step at all to read it against, so
an uninstall that was begun and abandoned was indistinguishable from one never begun. There is
deliberately no cli_uninstall_started beside it: nothing on the way out corresponds to init,
and inventing one would put a _started name next to this — the exact pair the rename above
exists to avoid. docs task stays uninstrumented for a different reason: it is printed by a
scheduled run rather than by anyone deciding anything, so it would count how often cron fires.
Flushing at all three, rather than recording only, is the load-bearing half. An install that
dies in Part 2 never reaches prepare or commit, so without these flushes its start — and
every cli_error it accumulated on the way down — would sit in the spool until the cap ate it,
and that run is precisely the one these events exist to make visible. docs install affords a
flush outright: resolve_doc has already blocked on a server GET by the time the guide is
printed. init does not, and pays for it — one bounded POST on a local command, fail-open, so
an unreachable endpoint costs the timeout rather than the write. report install was the last
to join them, and the argument for exempting it — a completed install goes on to bridge, and
will flush there — turned out to assume the very thing the event reports; see §2.
The remaining costs are recorded rather than hidden. init is idempotent and gets re-run on
repairs and on a second host, and a guide re-printed mid-run — a compaction, a restarted agent
— counts twice, so both steps are attempts as observed, not distinct machines. A start
event's deployment_mode is the mode init inferred, which Part 1.2's backend choice may
still change, so it must never be joined on. And failure detail still rests on an agent
choosing to call report error, which is voluntary and model-judged: the funnel says that an
install died, never why.
The names on the wire, settled with the backend and held as constants in one place so a
later revision is one edit:
event_name |
reported_by |
Emitted for |
|---|---|---|
memory_update_started |
code | prepare opened a bridging cycle |
memory_update_succeeded |
code | commit closed the cycle that this machine's prepare opened |
memory_update_failed |
agent | report error --stage remember |
memory_commit_succeeded |
code | the commit store call returned |
memory_commit_failed |
code | the commit store call raised |
memory_search_succeeded / _failed |
code | retrieve |
memory_list_succeeded / _failed |
code | a whole-store sweep — prepare's mirror, or memu list-files |
cli_install_started |
code | init wrote config.env |
install_guide_opened |
code | docs install printed the guide |
uninstall_guide_opened |
code | docs uninstall printed the guide |
cli_install_succeeded |
agent | report install |
cli_install_failed |
agent | report error --stage install |
cli_uninstall_succeeded |
agent | report uninstall |
cli_uninstall_failed |
agent | report error --stage uninstall |
agent_error_reported |
agent | any report error — §5 |
cli_error |
code | an exception the CLI caught — §5 |
cli_events_dropped |
code | the spool hit its cap and discarded events |
The cli_ prefix marks what the client observed about itself — its own install, its own
uncaught exception, its own spool overflowing — against the memory_ family, which reports on
memU's actual work. The *_guide_opened pair are the names outside both, and deliberately: they
report a document being asked for, which is neither the client's own lifecycle nor memU's
work on memory.
One name per action and outcome, replacing core_action_completed and itsproperties.action_name discriminator. The first writing put three actions and both outcomes
behind one name; every question then began by unpacking two properties before it could be
asked. Splitting on the action is what the backend's statistics want and costs nothing here.
Splitting on the outcome as well is the backend's call and is recorded as theirs: it makesWHERE event_name = 'memory_update_failed' a single predicate, at the price of a name enum
that grows by two per action rather than one, and of the succeeded + failed ≠ started
arithmetic above. The tense follows what was already there — cli_install_started andcore_action_completed were past-participle, so _succeeded / _failed are too, and_success / _fail are not used.
The split also tightens §10. One allowlist per action means memory_search_* is no longer
permitted to carry recall_file_count, nor memory_update_* a result_count, purely because
one shared entry had to admit the union of every action's fields. The privacy boundary gets
narrower as a side effect of the rename, which is the rare case where a wire change pays for
itself twice.
memory_update now names the bridging cycle, not the commit call. Until this revision it
was the commit store call, and that call is now memory_commit_*. This is a silent break for
any consumer written against the earlier name — same string, different subject, no error at the
boundary — and it is accepted because the alternative was an event_name remap the backend
would carry forever. The cutover is queryable: context.client_version distinguishes the two
readings, and rows below the release that ships this mean commit.
The distinction is worth having under either name. A cycle spans prepare → the agent's
self-evolve pass → commit, across two processes and mostly not memU's work; a commit is one
store call. --stage remember was always the former, action_name: memory_update always the
latter, and collapsing them is what made the failure channels of §5 look like duplicates when
they never were. memory_commit_* also fires where no cycle exists — BRIDGING_TASK.md's
LEFTOVERS step runs commit on a crashed run's jobs without a preceding prepare — which is
precisely why the two events are not redundant:
memory_update_succeeded is emitted only when the run marker exists. A leftover commit
emits memory_commit_succeeded and no cycle event, because it closed no cycle this machine
opened. Without the gate one bridging run emits one started and two succeeded — LEFTOVERS
commits, then step 4 commits again — and started >= succeeded fails on exactly the runs that
are already unhealthy.
The gate reads the same marker duration_ms does (§10) but is not the same predicate, and the
difference matters in one direction: a cycle whose wall clock skewed, or which outlived a day,
still happened and still reports memory_update_succeeded — it simply reports no duration. The
pairing is closed from the other end too. memory_update_started is emitted only when the
marker was actually written, so a full disk costs both halves rather than announcing a cycle
this machine has no way to close — which would manufacture a failure out of a storage problem.
5. Two error provenances, which never share a name
They are not variants of one thing.
cli_error — code-observed. Raised by host_cli.run's existing top-level handler, which
is where genuine unhandled failures already land. No model is in the loop, so the data is exact:
the exception type, the subcommand that was running, and the host. This is the higher-quality
of the two feeds and costs nothing to collect.
Its payload is where the care goes. A raw traceback is not sendable: absolute paths carry
the OS username and often the user's project names, and frame rendering can surface local
variables. So the frame list is reduced to module:function:lineno — module dotted path, never
a filesystem path — and the exception message is omitted by default, because messages are
where secrets and paths actually appear (a ConfigError naming a DSN, an HTTP error echoing a
URL with a token). If a message turns out to be needed, it is added later under the same
allowlist-and-truncate rules as --detail, not by relaxing this.
agent_error_reported — model-judged. Invoked deliberately by the agent when it decides
something is fatally wrong. Because an LLM chooses the payload and its context is the user's
transcript, this is the highest-leakage surface in the feature.
It was core_action_failed at first writing, named as the failure counterpart tocore_action_completed and carrying the same properties.action_name discriminator so that a
consumer could slice both the same way. Both of those are now gone, and the reason is the same
one that motivated the rest of this revision. The name asserted a symmetry that does not exist:
it read as the failure leg of a core action, and it is nothing of the kind — it is an agent's
free-text bug report about the world, which may not correspond to any core action that ran, or
failed, at all. action_name made the false symmetry concrete and joinable. It was already
known not to work (stage says remember where a core action said memory_update, andinstall / uninstall / other name no core action whatever), nothing ever read it back, and
under §4's split it would sit beside a genuine memory_update_failed — two events any analyst
would join, that must never be joined. So the field is dropped and the event is named for what
it is: an error an agent reported. stage remains, as the only structured field, and the flag,
the prose, and every agent-facing instruction still say stage and only stage.
Every valid report error emits it, and three stages emit a concrete _failed event as
well (§4). The two are one report projected twice, not two reports: one dedup decision covers
both, so an agent in a retry loop files neither a duplicate counter nor a duplicate prose row.
They are also shaped for different consumers, which is why the fan-out is worth its extra name
— the concrete event carries no properties at all, exactly like the success events it sits
beside, because its existence is the whole signal and a funnel counter has no use for prose.--detail therefore lives on agent_error_reported and nowhere else: one home for the
highest-leakage field in the feature, one place to audit, one place to truncate.
--stage is a closed enum, and the only structured field on the event:
install | uninstall | remember | retrieve | otherThree of those mirror the operations that have their own events. The other two are there for
reasons worth stating, because both are easy to leave out:
retrievecovers the failure this whole feature exists for. A retrieval that returns
nothing forever throws no exception, socli_errorcannot see it — record and inject have
silently disagreed about the store, which is the opening argument of ADR 0009. Only an agent
can notice it, and without this value it would have nowhere to go.otheris the escape hatch. A closed enum that rejects unknowns converts unclassifiable
into unreported — exactly the loss this feature exists to prevent. An agent facing a brokendoctor, or config trouble outside any named operation, must still be able to file.
The vocabulary is coarse now and will be refined. When it is, new values are hierarchical —install.part2.schedule, not install_schedule_failed — so a stage LIKE 'install%' query
keeps working across the granularity change and old rows stay joinable to new ones. Consumers
must therefore tolerate unknown stage values rather than reject them; the CLI-side enum
is validation at the point of entry, not a promise that the set is final.
--detail is optional and free-form: the agent decides what to say. Structured error codes
were considered for this slot and deferred (see --code below), so in v1 the event is stage
plus prose. Consume it accordingly — it is a triage inbox read by humans, not a metric.
Two floors stay on the CLI side regardless of what any instruction says:
- A hard byte cap. Without it an agent can paste an entire transcript, which is both a
privacy dump and an unbounded POST body. Truncation is one line of code and cannot be
forgotten the way a prompt can. - Per-stage dedup on
(stage, detail)for one hour, since an agent in a retry loop will
otherwise file the same failure a dozen times. The window is wall-clock rather than "until
the next flush" because this verb is a flush point: the spool a scan would consult is
emptied by the POST the previous report just made, so every retry would read as new. What
remembers instead is a sidecarevents.errorsledger beside the spool, holding truncated
SHA-256 fingerprints of the pair — bounded regardless of--detaillength, and prose-free,
so the one piece of state this feature keeps outside the spool has nothing in it to leak. An
hour is chosen to cover a loop inside a single session while still letting a failure that is
still occurring tomorrow file again; that recurrence is a different fact and worth a row.
An unwritable ledger costs duplicates, never a dropped report.
Content scrubbing is guided prompt-side and is deferred to the instruction stage along with
every other agent-facing text. That deferral is safe only because of a gate, which is a
decision and not an assumption: no agent-facing text may instruct an agent to callreport error until the scrubbing sentence lands with it. Until then the verb exists but
nothing directs an agent to it, so the free-text channel is inert. Shipping the instruction
first would leak by default in the first release, and a sent event cannot be recalled.
--code is reserved, not implemented. The intent is a structured error vocabulary, but no
good definition of one exists yet, and shipping an unvalidated free-text field in the meantime
costs more than waiting: agents populate flags that exist, so it would accumulate a year of
ad-hoc values before the vocabulary is defined (making that history unusable anyway), while
opening a second free-text channel beside --detail for no present gain. Adding a flag later
is backward-compatible; retracting one is not. Recorded here so the name stays free.
6. `client_instance_id` lives in `~/.memu/config.env`
Generated once, on first need, as a uuid4; written to ~/.memu/config.env asMEMU_CLIENT_ID; read through the ordinary env() chain thereafter.
That file is the right home for one specific reason: UNINSTALL.md Part 3 says to keep~/.memu/config.env always, unconditionally, because it is the user's configuration and a
reinstall must pick it back up. So install → uninstall → reinstall joins into one instance's
history, which is the whole point of the id. The obvious alternative, ~/.memu/hosts/<host>/,
is exactly what Part 3 tells the agent to remove, and would additionally fragment one machine
into one id per host.
The id is therefore per-machine-per-OS-user and shared across hosts; agent_platform
distinguishes hosts within it. It is an opaque random value carrying no machine fingerprint —
no hostname, no MAC, no user name. It is not derived from anything, so it cannot be
re-derived, and deleting the line is a complete reset.
7. `session_id` when the host hands it over for free, omitted otherwise
ADR 0015 already established HostSpec.session_id_env — the variable through which a host
tells a tool subprocess which session it is running in. Where it is set, session_id is filled
from it. Where it is not, the field is omitted. Nothing is inferred, nothing is synthesised,
and no new survey work is owed: a host that has not been surveyed simply reports no session.
Two notes. It is most meaningful on the bridging pair, where it ties a remember run to the
host session that performed it. But it is also free on retrieve, whose process is likewise
a tool subprocess of a live host session — so the rule is "read it wherever it is set", not
"read it only on commit". And per ADR 0015 only claude-code is verified end to end today,
so in practice this field will be sparse at first, by design rather than by defect.
8. `__version__` on the package, resolved from installed metadata
memu/__init__.py gains:
__version__ = importlib.metadata.version("memu-cli")guarded, so a source checkout without an installed distribution degrades to a sentinel rather
than making import memu fail — a telemetry field must never be able to break an import.
The Python package version always wins. npm/package.json carries its own, unrelated
number (0.2.0 against the Python 2.0.0-beta.0) because it is a thin launcher for the PyPI
package; it is never reported. client_version therefore always answers "which memU is
running", never "how was it fetched".
9. `agent_platform` is normalised at the envelope boundary
The internal HostSpec.host values and the backend's vocabulary do not match, and the
mismatch is not cosmetic — it would split one host into two rows:
claude-code(hyphen, and it is also the on-disk directory name under~/.memu/hosts/)
against the backend'sclaude_code. Hyphens are normalised to underscores in the envelope
only; nothing on disk changes.- The generic adapter's host id is
agent, notgeneric— the binary ismemu-agent. As an
analytics dimension "agent" is meaningless, so it is emitted asgeneric, matching ADR 0011's
own name for it. - The core
memubinary has no host at all. It emitsnone.
The mapping lives in events.py and is explicit, not a string transformation applied hopefully.
10. `properties` is an allowlist: counts, never content
Per event name, a fixed set of keys, defined in code. Anything not on the list is dropped
rather than passed through — including anything an agent supplies.
memory_search_succeeded/_failed→result_count,latency_ms.result_countis the segments + files + resources sum thatdoctoralready computes.memory_commit_succeeded/_failed→recall_file_count,resource_count,session_count,latency_ms.memory_update_started→{};memory_update_succeeded→recall_file_count,resource_count,session_count,duration_ms;memory_update_failed→{}.The two clocks that were once two fields on one event are now one field each on two events,
which is the split doing its job: averaging them together was always meaningless, and now it
takes a deliberate join.latency_mson the commit event is memU's own blocking work — the
store callcommitmakes — measured in-process with a monotonic clock, exactly as retrieve's
is.duration_mson the cycle event is the wholeprepare→commitspan, mostly the
agent's self-evolve pass between the two: it is a round-trip time, not a latency.The counts appear on both because they answer different questions and neither subsumes the
other. Onmemory_commit_*they are what that call wrote, and they are the only place a
LEFTOVERS commit's output is ever reported, since it emits no cycle event (§4). Onmemory_update_succeededthey are what a whole cycle produced, which is the number that
belongs besideduration_ms.Measuring it costs a file, because the two halves are separate processes minutes or hours
apart and no monotonic clock crosses that boundary:preparestamps~/.memu/hosts/<host>/.bridging_run.<host>.json, and thecommitthat reports the cycle
clears it. An explicit marker rather than an existing artifact's mtime — the nearest
candidate, the pending session cursor, is written partway throughprepare(before the
store mirror and the template fetches), so its timestamp would silently exclude exactly the
network time worth measuring, and would bind the number to a statement order nobody has
agreed to preserve. A failedcommitkeeps the marker, so the retry after it still measures
from thepreparethat opened the cycle.The marker now carries a second job: it gates
memory_update_succeededitself (§4), not
merely this field. Two questions off one file, and deliberately not one question — did this
commit close a cycle decides whether the event exists, can the span be believed decides
whether this field does. A skewed clock loses the field and keeps the event.Being wall clock, a suspended laptop or an NTP step lands inside
duration_ms. Consumers do
not have to filter that themselves: a negative span, or one longer than a day, is dropped at
the client. And the field is absent, never zero, whenever the cycle cannot be measured —
acommitwith no precedingprepare, an upgrade that landed mid-cycle. Same rule assession_idbelow: a field the CLI cannot fill is omitted, never faked.memory_list_succeeded/_failed→result_count,latency_ms.Emitted from the two places that sweep the whole store:
prepare's mirror of the recall
files to disk, and the core binary'smemu list-files. One event per sweep, not one per
page — a page count is an artifact of ADR 0014's keyset paging, whereas how long the sweep
took and how much came back is what actually moves as a user's memory grows. (page_count
was considered and dropped: a new allowlist key for a number nobody has a question about
yet.)latency_mscovers the entire loop, disk writes between pages included, on the same
definition the other two actions use — memU's own blocking work inside one process. It is
therefore not store latency, and a slow disk lands in it.result_countis what the store returned, not what the caller kept: the bridging mirror
skips files whose track has no directory on disk, and those were still listed. It is reported
on the failure leg too, so a sweep that dies midway says how far it got rather than zero.
Calling itresult_countrather thanrecall_file_countfollows the split the two existing
actions already draw — a read action reports what came back (memory_search), whilerecall_file_countis what the write action wrote (memory_update).memu list-filesis the first core action emitted by a binary with no host, so it carriesagent_platform: "none"and nosession_id— the defined answer for that binary (§9), not a
gap. It is also the one core action that does not flush: a hand-run command must not block on
a POST, and unlike the bridging pair it is never the last thing to run on a machine that is
breaking, so its event waits in the spool for whichever run drains it next.install / uninstall, every leg →
{}. There is nothing to say: the outcome is in the name
(§4), so anysuccessfield would be constant per name, and a field that is always the same
value teaches a consumer nothing while inviting someone to later "fix" it by sending the
other one — quietly re-creating a second, contradictory outcome channel beside the name.memory_update_startedandmemory_update_failedare empty for the same reason.errors → per §5.
agent_error_reportedisstage+detail; the concrete_failedevents
that accompany it carry neither, so--detailhas exactly one destination on the wire.
The query text never leaves the machine. Neither does memory content, a recall file's name
or track, a store DSN, an embedding provider's base URL, an absolute path, or a session
transcript. In local mode the user made a positive choice for "this device"; a counts-only
payload is what keeps that choice honest, and it is a property of the code, not of a policy
document.
11. Disclosed, and off with one line
MEMU_TELEMETRY=0in~/.memu/config.env(or the environment) disables recording and
flushing entirely — nothing is written, nothing is sent, nothing accumulates.DO_NOT_TRACK=1is honoured identically, because it is the established convention and a
user who set it has already expressed this preference.MEMU_EVENTS_BASE_URLsteers the endpoint and, set empty, disables it — the same shape and
the same escape hatch asMEMU_TEMPLATE_BASE_URLandMEMU_DOCS_BASE_URL, so air-gapped
installs, offline CI, and tests switch it off the way they already switch those off.INSTALL.mdPart 1.2 states plainly what is collected and how to turn it off, at the same
moment the user chooses local or cloud. Silent telemetry in an OSS CLI is a reputational
event; disclosure at the point of the privacy decision is the cheap way to not have one.
Consequences
- Install and uninstall counts are self-reported by a model and will under-count. An agent
that stops early, hits an edited guide, or is interrupted never reaches the verb. They are a
floor on install volume, not a census. The trustworthy denominator is the firstretrieveor
firstcommitseen from aclient_instance_id— both code-observed — and dashboards should
be built on those, withcli_install_succeededread as a funnel signal rather than a total.cli_install_started,install_guide_openedanduninstall_guide_openedare code-observed
and so do not share this bias, but they count attempts as observed rather than machines (§4),
which makes them numerators for "how many installs finish", not substitute denominators for
"how many installs exist". succeeded + failednever equalsstarted, wherever a_failedleg is agent-reported.
This holds forcli_install_*,cli_uninstall_*, andmemory_update_*— the three families
§4 fans an agent report out into. The_failedcounts are a floor; the honest failure number
isstarted − succeeded.context.reported_byexists so this is checkable in a query rather
than remembered from a document, and a dashboard that mixes provenances should say so.memory_updatechanged subject. Before this revision it was thecommitstore call;
it is now the whole bridging cycle, and the call ismemory_commit_*. Nothing errors at the
boundary, so rows on either side of the release must be read differently:context.client_versionis the discriminator, and history below the cutover means commit.
Accepted in preference to anevent_nameremap the backend would maintain indefinitely.- Most events' delivery is bounded by a bridging run. On a machine whose schedule is
broken, events accumulate to the cap and are then dropped. This is the correct failure — a
broken bridging task is precisely a thing worth being unable to hide — but it means "no
events from this install" has two causes, and reading it as "uninstalled" is wrong. Since §2,retrieveis the exception that largely covers this: it delivers its own event on either
leg, so a machine whose bridging never runs still reports every turn, and the spool reaches
its cap far less often than originally expected. retrievegains one blocking POST per turn, successful or not. Bounded to exactly one
request, taken after the result is already on stdout, and fail-open: an unreachable endpoint
costs the timeout and spools the event rather than losing it or failing the command. The
failure leg's share of this is the cost §2 names and accepts — a fully broken store adds the
timeout to every turn until it is fixed. What must not change is the shape — recording still
neither parses nor rewrites the spool, and any future work that makes it do either, or that
turns either leg into a flush, re-opens constraint 1.- The spool is a new file under
~/.memuthat uninstall does not mention.UNINSTALL.md
Part 3 needs a line for it, andreport uninstall's flush should leave it empty anyway.
It is not alone:events.jsonl.*.sending, theevents.droppedcounter, and §5'sevents.errorsledger are siblings, so that line names the family rather than one file. - Two hosts share one spool. The rename-then-send flush makes concurrent flushes safe, but
this is the first shared mutable file across host adapters — ADR 0010 otherwise scoped
per-host state under~/.memu/hosts/<host>/exactly to avoid such races. The deviation is
deliberate (identity is machine-scoped) and is the reason the flush mechanics are specified
here rather than left to implementation. - A field the CLI cannot fill is omitted, never faked.
session_idon unsurveyed hosts is
the live example. The backend must accept absent optional fields.
Open issues
- The agent-facing text has landed; two
--stagevalues stay unreachable on purpose.
Each host'sINSTALL.md,UNINSTALL.md, andBRIDGING_TASK.mdnow direct an agent toreport install,report uninstall, andreport error --stage install|uninstall|remember,
every one of them carrying §5's scrubbing sentence in the same breath — the gate is met, and
a test holds it that way rather than a promise.--stage retrieveand--stage otherare
deliberately left with no instruction pointing at them:retrieveis the silent-disagreement
case, which needs an agent-facing rule for noticing it before it can have one for reporting
it, andothershould not become the path of least resistance before the three named stages
have produced data to refine the vocabulary from. Both therefore report throughagent_error_reportedalone and have no concrete_failedevent (§4); giving either one is
the same decision as writing its instruction, and belongs in the same change. - The §11 disclosure is still unwritten.
INSTALL.mdPart 1.2 does not yet say what is
collected or how to switch it off, whileretrieveandcommitalready record. That is the
one remaining piece of §11, and it gates a release rather than a merge — silent telemetry in
an OSS CLI is the reputational event the disclosure exists to avoid. --codeis reserved and unimplemented (§5). It needs a real vocabulary, drawn from the
failure modes #528/#538/#606 actually produced, before it is worth a flag.--stagewill need finer values. The five in §5 are deliberately coarse. Refinements
must follow the hierarchicalinstall.part2.scheduleshape so old rows stay joinable.- The batch wire format is not settled. The endpoint is now fixed, but the implementation
posts a JSON array of envelopes; whether the route wants an array or newline-delimited bodies
is the backend's to fix. - No opt-in gate for first run. This ADR ships opt-out. If a jurisdiction or a partner
requires opt-in, the switch isMEMU_TELEMETRY's default, and everything else stands.
Out of scope
- Any event tied to memory content. Not deferred — excluded. §10 is a boundary, not a
starting point. - A second
client_type. The npm launcher, the library used programmatically, and the
cloud API's own callers all report nothing;memu_cliis the only value this ADR defines. - Sampling. Every recorded event is spooled. If retrieve volume turns out to dominate,
sampling belongs at record time, behind a server-supplied rate — deliberately not designed
here, because a client-side constant would be wrong the moment volume changes. - Turning the immediate path on. It exists (§2) and has no caller. Enabling it for a given
event is a future decision that must state why that event cannot tolerate spool latency. - Reporting from the scheduled wrapper itself. A bridging run that dies before reaching
commitreports nothing, which is a real blind spot; covering it means instrumenting cron,
launchd, and Task Scheduler wrappers separately, and is left for its own decision.
Related ADRs
- Builds on
docs/adr/0008-two-integration-surfaces-hooks-and-api.md— the two seams are what
the two core-action events measure. - Builds on
docs/adr/0009-codex-packaging-cli-and-config.md— thereportverbs are ordinaryPATHcommands, andMEMU_CLIENT_ID/MEMU_TELEMETRYride the one config loader. - Builds on
docs/adr/0010-multi-host-adapters.md—agent_platformisHostSpec.host, and
the shared spool is a deliberate exception to its per-host state scoping. - Builds on
docs/adr/0012-cloud-backed-agentic-backend.md—deployment_modeismemory_mode(), and the optional bearer header mirrors the cloud client's. - Builds on
docs/adr/0013-self-updating-instruction-templates.md— the fail-open contract for
contacting a memU server, inverted from GET to POST. - Builds on
docs/adr/0015-bridging-must-not-mine-its-own-run.md—session_idreusessession_id_env, and inherits its per-host survey status.
Adr 0017 Config Env As A Command (`docs/adr/0017-config-env-as-a-command.md`)
ADR 0017: config.env Is Written by a Command, Not by the Agent — init for the Entry, config for the Detail
- Status: Proposed
- Date: 2026-08-05
- Builds on: ADR 0009 (the CLI seam and one config loader), ADR 0010 (multi-host
adapters, one shared parser from aHostSpec), ADR 0012 (cloud/local backend
selection, config in~/.memu), ADR 0013 (server-refreshable agent-facing
docs), ADR 0016 (client event reporting, the install funnel) - Scope: who writes
~/.memu/config.envduring an install, and through what
command surface. It changes neither seam's behaviour, the store, the schema of
the file, norenv.py's resolution order — only the mechanism that produces
the file and the two guides that drive it.
Context
The install pipeline has three steps today:
SKILL.md— the entry document. Tells the agent topip install memu-cli
and pick its host binary.docs install— prints the packaged, server-refreshable guide.INSTALL.md— the guide, followed top to bottom.
~/.memu/config.env is created in step 3, by the agent, from prose. Two
problems follow from that placement.
The file is written by hand. INSTALL.md §1.2 tells the agent which keys to
write and the agent writes them, typically with a heredoc or echo >>. That file
is shared across every host on the machine, carries a plaintext credential, and
holds an invariant the guide can only state in words: record and inject must
agree on the backend, and in local mode on the DSN and embedding space too, or
retrieval silently returns nothing (ADR 0009's opening argument). The guide
already spends a paragraph on "repair the connection, never the identity" —MEMU_DB, MEMU_EMBED_PROVIDER, MEMU_EMBED_MODEL must never be edited on an
existing store — and enforces it with nothing but emphasis. Field behaviour
matches the exposure: agents occasionally rewrite the file rather than merge into
it, taking unrelated keys with them.
This is the same argument that produced install-instruction. memU owns the
retrieval block's text, so memU writes it; the guide's instruction is "do not
hand-write this". config.env is a strictly harder file — merge semantics, a
secret, a cross-host invariant — and has no such command.
The install-start event fires before the key exists. cli_install_started
(ADR 0016 §4) is emitted from docs install, because printing the guide is the
first act on the install path that proves memu-cli resolves. On a first install
that happens in step 2, before step 3 has collected the memU Cloud key, so the
event goes out with no Authorization header and lands unattributed. (The
emission has since moved into init — see Open issues, where that was left open
and is now settled; docs install keeps its own event as install_guide_opened.)
This second problem is less severe than it first appears, and should not be the
sole justification for this ADR. Every envelope already carriesclient_instance_id, and client_instance_id() generates and persists the id
into config.env on first use — so the unattributed start already carries a
stable id that every later authenticated event on that machine reuses. The
backend can join anonymous → account on it retroactively. What ordering fixes is
attribution at emission — and it is what would let a later release fail fast on
a wrong key, which this one does not attempt (see Open issues).
The obvious alternative for the ordering — spool the start event without flushing
and let a later flush attach the key — is rejected: the flush at docs install
exists precisely so an install that dies in Part 2 still reports its start, and
those are the runs the event was added for.
Decision
Two verbs over one writer
init and config, registered on the shared host parser
(host_cli.build_parser) so every host adapter gets both, backed by one merge
function whose guard policy is a parameter. init is the inferring front door;config is the explicit one. They must not each own a code path forMEMU_CLOUD_API_KEY, or the two will drift within a release.
<binary> init [--cloud-api-key K] # entry; named in SKILL.md
<binary> config show # read-only; the preflight probe
<binary> config --cloud [--cloud-api-key K] [--cloud-base-url URL] [--force]
<binary> config --local [--db PATH] [--embed-provider P] [--embed-api-key K]
[--embed-base-url URL] [--embed-model M] [--force]--force is on config only, and init needs none: it never refuses, so there
is nothing to override. That is not a weaker contract than config's but a
different one, and the reason is where each verb sits — see "init never
refuses" below.
The split is not cosmetic — the two verbs have different policies, and the
place each is named is the reason:
SKILL.mdis the least updatable surface memU has. It ships to users, is
pasted into repos, and — unlikeINSTALL.md— does not go throughtemplates.resolve_doc, so it cannot be refreshed from the server (ADR 0013).
Anything named there needs a contract expected to hold for years.initwith
at most one flag is that. A verb whose flag surface grows every time local mode
gains a knob is not.INSTALL.mdis server-refreshable, soconfig's flag surface can evolve at
the server's pace.- Folding both into one verb is what produces the trap in the next section.
`init` — infers, and hands back what it cannot infer
Zero or one flag, idempotent, safe to run on any machine in any state:
init --cloud-api-key K— the user provided a key, which is the choice of
cloud. SetsMEMU_MEMORY_MODE=cloudand persists the key, whatever the file
said before.init— no key offered. If the file already declaresMEMU_MEMORY_MODE,
keep it: a re-install where the user did not re-supply a key must not flip
cloud to local. If absent, setlocal.- In every case, generate and persist
MEMU_CLIENT_IDif missing.
`init` never refuses
Not "rarely" — never. Bare init has nothing to refuse on: every branch it
takes either keeps what is there or fills a vacuum. init --cloud-api-key on a
machine already configured for local with MEMU_DB set is the one case that
looks like it should, and it does not either. Three reasons, in order of weight:
- A refusal there is terminal.
inithas no--force, and it is the first
command on the install path — the oneSKILL.mdnames beforedocs install.
Exiting non-zero does not redirect the agent to a better command so much as
stop the install, and every step after it, on a machine whose user has just
handed over their key. - The signal is unambiguous. Having a memU Cloud key in hand and passing it
is about as explicit as intent gets at this point in the flow. Treating it as
something the caller might not have meant reads the situation wrong. - Nothing is destroyed. The local store is not touched:
MEMU_DBstays in
the file and the rows stay in the database, unread until the mode is switched
back withconfig --local --force. What the flip costs is that memories
written locally stop being retrieved, and new ones go to the key's cloud
account instead — real, reversible, and exactly the kind of consequence a
warning is for.
So init says it, loudly, and proceeds: a switched row beside the mode, and a
fuller warning: on stderr naming what happened and the command that undoes it.
The guides carry the same consequence in prose where they ask for the key.
This is the one place init and config are allowed to disagree about the same
state test, and the asymmetry is the point: config's caller named a backend and
can be handed --force, so refusing them costs one re-run. init's caller has
no such second move.
Writing MEMU_MEMORY_MODE=local explicitly, rather than relying onmemory_mode()'s compatibility default, also retires the ambiguity INSTALL.md
currently has to explain in prose ("an existing file without MEMU_MEMORY_MODE
is local mode for backward compatibility"). It is safe to write into a file that
was silent on the question — including one carrying a MEMU_CLOUD_API_KEY,
which looks like a cloud install being demoted but is not: memory_mode()
already resolves that file to local today, so init records the mode the machine
was already in rather than changing it.
`config` — explicit, and refuses ambiguity
--cloud or --local is mandatory to touch MEMU_MEMORY_MODE. This is the
verb INSTALL.md §1.2 drives once the user has actually chosen a backend, and
the only verb that configures the local-mode knobs.
config --local with no --db is allowed, and writes the mode alone. The
alternative — requiring --db whenever none is set — would put the refusal in
the wrong place: this verb runs inside INSTALL.md, where the agent is already
in a conversation with the user about the store and can ask. A mode with no
store fails the Part 1 verify gate loudly, which is the right amount of
enforcement for a value only the user can supply.
`config show` — the read side, and the preflight probe
Prints the resolved mode and the keys that are set (never their values, beyondset/unset for anything credential-shaped), and writes nothing. This is the
"is the backend already configured" question that INSTALL.md's preflight and
§1.2's opening if already configured, skip both ask, and it is why the read
side is a named subcommand: a bare config that printed while a bare init
wrote is exactly the asymmetry agents get wrong.
The one-backend guard is state-based, not declaration-based
This is the load-bearing decision, and without it the whole design has a hole.
If MEMU_MEMORY_MODE were an identity key guarded on declaration alone, the most
ordinary first install breaks: the agent runs bare init in step 2.5 (the user
had not found their key yet), local lands on disk, the guide then asks the
backend question, the user says cloud — and config --cloud is refused as a mode
flip. Once written, an inferred default is indistinguishable from a chosen
one, and a declaration-based guard can only see the file.
What the invariant actually protects is existing memory, not a string. So a mode
change is refused only when the current mode has something to lose — and only onconfig, which is the verb that can offer the override:
| Current state | config --<other-mode> |
|---|---|
local, MEMU_DB set |
refuse without --force |
cloud, MEMU_CLOUD_API_KEY present |
refuse without --force |
| anything else (a vacuous declaration) | change it silently — this is the first real choice |
"Has something to lose" is judged by declaration of the store, not by the
store's contents. MEMU_DB being set is enough; whether the file exists, or
holds one memory or ten thousand, is not asked. MEMU_DB spans a bare path, asqlite:// URL, a postgres:// DSN and the in-memory sentinel
(env.database_config), so "the store exists" has no single cheap meaning —
and the one shape where it is cheap, a local SQLite file, is also the shape
where database_config creates the parent directory as a side effect of
being asked. A guard that mutates the filesystem to answer a question is worse
than a guard that over-refuses, and over-refusing costs one --force on a path
where the agent is already talking to the user.
The state the guard reads is the file, never the resolved environment. Every
other consumer in this codebase asks env.env(), which consults os.environ
first — correct for resolving config, wrong for deciding what is on disk. An
exported MEMU_CLOUD_API_KEY in the calling shell would otherwise makeconfig --local refuse a file that never declared anything, and init's
"already set, keep it" would preserve a mode nobody ever wrote. So the writer and
the guard take the parsed dotenv as their only input, and the process
environment reaches neither.
MEMU_DB, MEMU_EMBED_PROVIDER and MEMU_EMBED_MODEL keep an unconditional
guard: those three bind an embedding space that already has vectors in it, and
"fixing" one silently strands every existing vector. Unconditional means no state
test, not no escape — --force still overrides, and must, because the
alternative to a supported override is the user hand-editing the file, which is
the thing this ADR exists to stop. Setting one of the three where none was set is
not a change and needs nothing. Connection-level keys
(MEMU_CLOUD_BASE_URL, MEMU_BASE_URL, NO_PROXY) are freely updatable — this
is the guide's existing "repair the connection, never the identity" rule, moved
from prose into a gate.
Two cases the inference rules do not cover on their own, settled here:
init --cloud-api-key Kwhen the file sayslocal. This is a mode flip by
inference, andinitmakes it in every case: a vacuous local config flips
silently, a local config withMEMU_DBset flips with the warning described
under "initnever refuses". The same state test still decides whether to
warn; it just does not decide whether to act.init --cloud-api-key K2whenK1is stored. Replace it, and say so
loudly ("replacing the stored memU Cloud key"). Refusing would block the
legitimate rotated-key repair, and the client cannot distinguish a rotation
from a different account. Announcing it is the honest middle.
File mechanics
Merge, never replace the document. Parse the existing file, set only the named
keys, and preserve every other logical line's text, order, and comments. Writes
use canonical UTF-8; a legacy encoding is normalized on its first write, line
endings follow the host text convention, and a missing final newline is added.
Write to a temp file and os.replace, mkdir(0700) the directory and chmod 600 the file, then env.reload().
One writer, for every key — including MEMU_CLIENT_ID. events.client_instance_id()
already writes that key today, by appending to the file with no atomicity and no
permissions, and deliberately re-reading afterwards so two racing first runs
converge. Left alone it becomes a second mechanism for the same file, and a
worse interaction than duplication: this section's read-modify-write plusos.replace silently discards a concurrent append. So client_instance_id()
moves onto this writer, keeping the two properties it was built with — it
swallows OSError and still returns an id, and it re-reads rather than trusting
what it wrote. That places the writer where events can import it without
reaching into memu.hosts.
Permissions are reported as what actually happened. chmod 600 restricts
nothing on Windows — os.chmod there moves the read-only bit and no ACL — so
claiming it on that platform would replace an instruction the agent might follow
(today's guides say to restrict the file to the current user) with a line
asserting a protection that does not exist. POSIX chmods and says so; Windows
says the file holds a plaintext key and that its ACLs are inherited. Doing the
real thing there means shelling out to icacls, which is a subprocess and a new
failure mode on the install path; not worth it for a claim, and the honest line
leaves the door open to adding it later.
Do not persist defaults. MEMU_CLOUD_BASE_URL is written only when it
differs from DEFAULT_CLOUD_BASE_URL; baking today's default into the file
freezes a value the code should own across upgrades. Note also thatMEMU_BASE_URL is the embedding endpoint (local mode) and has no role in cloud
configuration — the two are easy to conflate.
Flag naming follows the variable
--cloud-api-key, not --api-key. The codebase deliberately separatesMEMU_CLOUD_API_KEY (memU Cloud) from MEMU_API_KEY (the embedding provider),
and INSTALL.md has to warn against confusing them. A bare --api-key re-opens
that door on the one surface meant to close it. Local mode's embedding credential
is --embed-api-key for the same reason.
Neither verb verifies — `doctor` stays the only prover
init and config report what is declared. doctor remains the only command
that proves the backend answers, and Part 1's verify gate remains where a wrong
key is caught.
Verifying the key inside init --cloud-api-key was considered and deferred (see
Open issues). Two reasons. The key is usually pasted from memU's own website
alongside the install instruction, so a wrong key is a rarer failure than the
design was weighting it as; and where an agent does garble one, the recovery is
already specified — a later init --cloud-api-key K2 replaces it loudly.
The ordering argument is worth stating precisely, because it is what makes
deferring safe: verification must run before the write, not after. A
write-then-verify init leaves cloud plus a bad key on disk when it fails,
and that is exactly the state the guard table reads as "cloud, key present —
refuse without --force". The user who then abandons cloud and asks for local
hits a refusal on the most ordinary first install, which is the failure the
state-based guard exists to eliminate. Nothing needs to be on disk to verify —
the cloud client takes its base URL and key as arguments — so whenever
verification is added, it is added ahead of the write.
Refusals are an exit code, not an exception
A guard refusal returns a distinct non-zero status (2, as prepare already
uses for a missing session log) from the handler. It must not raise: host_cli.run
catches exceptions into record_cli_error and flushes, which would file every
ordinary "you meant --force" into the error feed as if memU had broken.
Output is the agent's next-step signal
Decisive lines, not a diff:
mode cloud (from --cloud-api-key)
key set
client a1b2… (generated)
config ~/.memu/config.env (chmod 600)
ready — continue with `memu-claude-code docs install`and for the bare case, mode local (default; no store configured yet) plus a
pointer that docs install will guide the rest. On Windows the config line
reads …\.memu\config.env (plaintext key; Windows ACLs inherited) — see File
mechanics.
A config refusal prints the same shape: what is on disk, why it is protected,
and the exact config … --force to run if the change was meant. An init that
flipped a machine off a configured local store prints that shape too — aswitched row in the block, then the consequence and the config --local --force that reverses it — but exits 0, because it did the thing.
On the host binaries, not `memu`
memu is the algorithmically correct home — config.env is host-irrelevant, and
ADR 0009 makes memu the algorithm surface with the adapters as sidecars. It is
nonetheless the wrong choice here. The install flow's ergonomic bet is thatSKILL.md Step 2 hands the agent exactly one binary name and every later step
is <your-binary> …; a second binary mid-flow adds a branch, a fresh
"command not found" mode, and a name far likelier to be shadowed on PATH thanmemu-claude-code is. doctor is already just as host-irrelevant in substance
and lives on the host binaries for this reason. A thin memu init alias over the
same module is acceptable, but the guides name only <your-binary>.
Guide changes
SKILL.mdStep 1 installs withpip install --upgrade memu-cli(anduv tool install --upgrade), because the flag is what makes the rest of this
ADR reachable — see the stale-build note below.SKILL.mdStep 2 ends by runninginit, rather than gaining the "Step
2.5" an earlier draft of this ADR called for. Asking for the key and creating
the file is the finalization of picking a binary — you cannot run<your-binary> initbefore you have<your-binary>— and a numbered step
between two numbered steps reads as an afterthought on the one surface where
numbering is the structure. Ask the user for their memU API key, then<your-binary> init --cloud-api-key <key>; no key → bare<your-binary> init.
The prose stays minimal on purpose — everything that might change lives in the
command's own output — with one exception it must state, because the command
states it only after acting: passing a key on a machine already using local
memory switches it to the cloud, and the local memories stop being retrieved.INSTALL.md§1.2 collapses from ~70 lines of write-these-keys instruction
to "config show; if already configured, reuse; otherwiseconfig --cloud|--local …". The local-mode knobs and the no-embedding-key
fallback survive as guidance for choosing arguments, not for writing the
file, and the "repair the connection, never the identity" paragraph becomes
what it always described: aconfig --embed-base-urlcall for the connection,
and a guard that refuses the identity.- §1.2 also names
init, hedged with "if you arrived fromSKILL.mdyou
have already run this". An agent handed the guide directly — the user pasteddocs installoutput, or found the file — never passes throughSKILL.md, andinitis idempotent, so the duplicated instruction costs a
no-op and the missing one costs a config file. - The "is the backend already configured" probe becomes
config show— one command, no file test, noMEMU_MEMORY_MODEgrep. It opens §1.2
rather than joining Claude Code's Preflight block (the only host with one):
that block runs before Part 1 and exists to find out whethermemu-*
resolves at all, so a probe that needs the binary cannot live in it.
There are eight guides carrying this section, not one: seven hosts head it### 1.2 Configure the memory backend, the generic adapter has it as its## Part 1, and Cola states it in prose with no heading. All eight change.
The guides ship ahead of the code they name, and must not. docs install
prefers the server's copy of INSTALL.md over the packaged one
(templates.resolve_doc, ADR 0013), so the moment this text is published
server-side every machine still on an older memu-cli is handed a guide telling
it to run a subcommand its binary does not have — argparse answers invalid choice at the first step of Part 1. Three things answer this, and the first two
are the real fix: the install step in every guide carries --upgrade, so the
ordinary path never has a stale binary; every guide names invalid choice
explicitly as the stale-build symptom with "upgrade and re-run" as the cure; and
the server-side publish is still gated on the release that carries the commands.
Consequences
Positive:
- The cross-host invariant is enforced by code on the one file that carries it,
instead of by emphasis in a document. - Unrelated keys, comments, and another host's settings survive a re-install.
- Permissions and atomicity stop depending on the agent remembering
chmod 600. - The install-start event carries the key when the user had one to give — and now
fires frominititself, so it is the first event of an install rather than the
second, and attributed whenever--cloud-api-keywas passed at all. MEMU_CLIENT_IDexists before anything reports, rather than being appended by
whichever event happened to fire first.SKILL.md's command surface becomes something we expect never to change.
Costs / limitations:
- Two more verbs on every host binary, and a required-input prompt (the backend
choice) moves intoSKILL.md— the surface we cannot refresh from the server. - The key lands in
argv, so it is visible tops, shell history, and the
session transcript. Not a regression — the heredoc it replaces is equally
exposed — but see Open issues. - One more command in a flow whose main failure mode is agents skipping steps.
initis idempotent and cheap, which is the mitigation, not a guarantee. init --cloud-api-keycan strand a local store behind a warning nobody read.
This is the deliberate trade in "initnever refuses", and it is the one place
where the safer behaviour was traded away for a flow that completes. The store
survives and oneconfig --local --forcerestores it, so the cost is confusion
and a stretch of unretrieved memories rather than data loss — but the warning
is all that stands between the two, and warnings are read less often than exit
codes are.
Open issues
- The credential is in
argv, and memU mines transcripts. The record seam
reads the very session log that will containinit --cloud-api-key sk-….
AcceptingMEMU_CLOUD_API_KEYfrom the process environment, and--cloud-api-key -to read one line from stdin, are both cheap to add; agents
are unreliable with stdin, so the flag stays primary. Whether the bridging
pipeline should additionally redact key-shaped strings from mined transcripts
is a separate question this ADR does not settle. - Nothing checks the key until Part 1's gate. Deferring verification (above)
costs the earliest failure point: a garbled key now survivesinit, survivesdocs install, and dies atdoctor— several steps and one wasted funnel
event later, that event being theinitstart the key is attached to. Accepted for now
because keys are copied, not typed, and because a re-initrepairs one. When
it is added: before the write, never after, for the guard-state reason
given above, and reusingdoctor's call rather than inventing a second one. Where the install-start event belongs.Settled: it belongs ininit.
This ADR shipped with the emission left atdocs installand only reordered
relative to the write; the start has since moved intoinit, which is the first
command on the install path and knows both the host and the key, so attribution
is now unconditional rather than dependent on the user having configured cloud
before Step 3. The two objections were answered rather than overruled.docs installstill proves the package resolves — it keeps its own event under the
nameinstall_guide_opened, so nothing that was observable stopped being
observable, and the funnel gained a step instead of trading one. And a re-run ofinitbeing a weaker signal is the same property a re-printed guide already
had: both count attempts-as-observed, which §4 of ADR 0016 already said out
loud. The one thing the move made load-bearing is ordering withininit— the
event must follow the write, orclient_instance_id()mints and persists a
second id that the write then overwrites.- Local mode's flag surface will grow. Every new local knob is a new
config
flag. Acceptable whileINSTALL.mdis refreshable, but if the count keeps
climbing, aconfig set KEY=VALUEform will be tempting — and would forfeit the
identity-key guard, which is the reason for named flags.
Out of scope
- The schema of
config.envorenv.py's resolution order (ADR 0009/0012). - Anything the guides say about choosing a backend — only who writes the result.
- The bridging pipeline, the instruction seam, and
install-instruction. - The event envelope and transport (ADR 0016); this ADR only changes when one
event is emitted relative to the config being written.
Adr 0018 Cowork Through Claude Code Bridge (`docs/adr/0018-cowork-through-claude-code-bridge.md`)
ADR 0018: Mine Claude Cowork Through the Claude Code Bridge
- Status: Accepted
- Date: 2026-08-14
- Builds on: ADR 0010, ADR 0015
Context
Claude Cowork stores one user-visible Recent workspace inlocal_<session-id>/audit.jsonl. A file contains the outer Cowork ID plus internal
agent IDs and audit/lifecycle records. Claude Code Desktop metadata is already
covered by ~/.claude/projects, so treating it as another input would duplicate
history. Windows field evidence confirms Cowork's audit directory is separate.
Decision
Cowork is a distinct read-only CoworkTranscriptSource, composed into the existingmemu-claude-code bridge rather than exposed as a second binary or schedule.
Discovery is platform-specific rather than one mixed list. Windows checks existing
Claude Desktop roots under %APPDATA%\Claude, %LOCALAPPDATA%\Claude-3p, and
MSIX Claude_* roaming roots. macOS checks both Claude and Claude-3p under~/Library/Application Support; Linux checks both under${XDG_CONFIG_HOME:-~/.config}. These conventional macOS/Linux roots are
enabled for staging and remain subject to correction from real-machine field
reports.
MEMU_COWORK_ROOTS is the staging and nonstandard-install escape hatch: a
platform-path-separator-delimited list of Claude Desktop data roots. A declared
value replaces automatic discovery, nonexistent entries fail visibly, and an
empty value disables discovery. Constructor
roots remain the highest-priority test seam. Every selected root is an independent
scan region, so an unchanged session ends only its own newest-first region and
cannot hide Code history or another Desktop root.
The outer local_ directory is the session boundary and session identity.
Record-level IDs are execution details and never split the session. The source
normalizes only user/assistant message records, removes audit transport data and
internal identifiers, drops lifecycle/result/system records and isReplay user
copies, and reuses Claude content-block classification. It never opens .audit-key.
The existing Claude Code self-skip remains unchanged: scheduled claude -p runs
are Claude Code sessions under ~/.claude/projects; Cowork is not assigned an
unverified environment identity contract.
Consequences
- One schedule, working tree, CLI, and memory backend serve Claude Code and Cowork.
- Cowork memories are available through the shared store to all integrated hosts.
- Cowork parsing is isolated from Claude Code parsing, so protocol drift is pinned
by fixture tests without widening the existing reader. - Platform providers keep unrelated OS paths out of the scan; staging feedback can
refine one provider without changing the reader or bridge. - Explicit roots let testers and nonstandard installations run immediately without
patching Python or mixing staged data with automatically discovered history.
Adr 0019 Developer Run Workspaces (`docs/adr/0019-developer-run-workspaces.md`)
0019: Allocate a Private Workspace for Each Developer Memorize Run
Status
Accepted.
Context
Developer applications submit 1–10 canonical sessions through memu memorize prepare,
run an external executor, and commit once. A shared ~/.memu/developer workspace
couples independent invocations: a pending or failed executor blocks the next run,
and callers cannot address each run independently.
Decision
The developer CLI allocates one private directory per prepare invocation usingtempfile.mkdtemp below ~/.memu/developer/runs/. Atomic allocation gives concurrent
prepares distinct paths. Its basename is the opaque run id returned with the
workspace, transcript paths, ordered jobs, executor prompt, and next command.
A batch of N sessions remains one run with 2N + 1 serial jobs.
commit <run-id>, internal verify-resources <run-id>, and discard <run-id>
resolve only ids below that root. Path syntax and links redirecting the run
outside its allocated location are rejected. Callers cannot select arbitrary
workspace paths through the CLI.
Each run retains the existing active marker and content snapshot. Backend errors
preserve the run for recovery. A transport failure may leave the submission
outcome unknown, so retaining the run does not guarantee that retry is safe.
After acknowledged backend success, the CLI attempts to remove the entire run
directory. If post-commit cleanup fails, it reports that the submission succeeded;
callers must discard the stopped run rather than resubmit it. The current marker
does not reliably distinguish a committed run from an active run after such a
failure, so callers must honor that result rather than retry every failed command.
Explicit discard removes a stopped run without a backend call and does not undo
an accepted submission. It can also remove an incomplete run left by process
termination. Ordinary prepare failure cleans up its newly allocated directory;
there is no TTL deletion.
Allocation and complete-directory deletion belong to the CLI, which owns these
paths. The existing explicit-workspace Python lifecycle functions retain their
behavior, including leaving their caller-owned directory in place. Host adapter
prepare–commit workflows are unchanged.
Consequences
- Applications persist the returned run id and address it explicitly on commit,
verification, or discard. Commands never implicitly select the latest run. - Applications coordinate one executor per run and serialize operations on that
run. The active marker is not a process lock, and discard must follow executor
termination. - Separate directories prevent filesystem interference but not backend write
conflicts. Runs updating overlapping RecallFiles must serialize the complete
prepare–evolve–commit cycle, or use disjoint ownership. Serializing commits
alone cannot refresh snapshots created by earlier prepares. - The CLI change is breaking for callers using an unqualified memorize commit or
verifier command. Complete any old fixed-workspace run with the previous CLI
before upgrading. Old working files are not automatically moved or deleted. - Evolve-type selection and job granularity remain independent follow-up work.
--- METRICS ---
- Files Extracted: 24
- Estimated Token Budget: ~61195 tokens
- Canonical Reference: https://codewiki.google/github.com/NevaMind-AI/memU