File: README.md
The Store
The hard problem with local-first apps is synchronization. If each device has
its own SQLite file, how do you keep them in sync?
Epicenter's answer: an application is one Yjs document, replayed in full
before any handle exists, and the surface over it is synchronous. A read is a
property access, not a round trip, so nothing is awaited and nothing needs cache
invalidation or race protection.
import { openDevice } from '@epicenter/data/browser';
import { defineTable, defineWorkspace } from '@epicenter/workspace';
const notes = defineTable({
title: 'string',
pinned: 'boolean',
folderId: 'string|null = null',
});
const notesWorkspace = defineWorkspace({
namespace: 'com.example.notes',
tables: { notes },
});
// Every verb returns a Result; the error arm is elided here for length.
const { data: db } = await openDevice(notesWorkspace); // the only await
db.tables.notes.create({ title: 'Hello', pinned: false }, { document: ['body'] });
const listed = db.tables.notes.list(); // synchronous: { rows, nonconforming }
const stop = db.tables.notes.subscribe(read); // fires with the row ids a commit touchedA workspace is one application's declaration of its durable data: pure JSON,
arktype expression strings, no storage and no lifecycle of its own. It is
release-local and never migrates your data. A row it cannot read is reported
beside the rows it can, with the reason and the raw values intact, and an
ordinary write repairs it.
Prose merges per character: a row is allocated a document container when it is
created, and db.tables.notes.document(id)?.get('body') hands an editor a live type to
bind to. Epicenter never looks inside it.
Sync is one Cloudflare Durable Object per (account, application). Being signed
in on two devices is the entire sharing model: nothing is paired, invited, or
approved.
Read the data package docs | What it replaced, and why
Build With The Toolkit
The developer toolkit is MIT: build anything on it, including closed-source and commercial products, and you own what you build, with no obligation back to Epicenter. @epicenter/data, @epicenter/workspace, and @epicenter/ui are the packages meant to leave this repo. They are pre-1.0 and tuned for our own apps, so treat them as fork-and-own rather than a stability-guaranteed SDK for now.
Status
There is one runtime: a desktop SPA in a WebView, over a store the client owns.
A host serves bundles and brokers credentials; it owns no application data. A
hosted web runtime with a host-owned replica is refused, and so are third-party
installed apps, for now.
Honeycrisp is the app running on the store, and its
README is the worked example.
Whispering, vocab, skills, and the Epicenter host do not compile right now. The
superseded data stack was deleted before they were migrated, deliberately: the
new store has no row-document HTTP path and no multi-process observation
carrier, so there was nowhere for them to move until those refusals landed. Data
they held on the old stack is accepted as lost; there is no importer.
Matter edits user-owned Markdown folders directly and keeps a
disposable matter.sqlite query mirror beside them. Local
Books and Local Mail are headless mirrors
that pull a hosted account into local SQLite. Those three do not use the store.
Trust Boundaries
Pick the trust model you want.
| Path | What leaves your device |
|---|---|
| Signed out | Nothing. The store is complete on the machine it opened on, and every read comes from a document already in memory. |
| Signed in | Your application's document, as opaque update bytes, to one authority per account. |
| Hosted Epicenter | That authority is ours, along with account and session data and any hosted feature you enable. |
| Self-hosted instance | You control the server, secrets, deployment, and infrastructure boundary. |
| A provider an app calls | Whatever that app sends it: transcript text to an LLM, audio to a transcription provider. Epicenter servers are not in that path. |
Signed-in sync sends your data to a trusted server that reads it in plaintext. On hosted Epicenter the authority is ours, so that data sits inside our trust boundary; self-hosting puts it on infrastructure you control, so Epicenter never holds it. See the trust model for the details, including where this is heading with the anchor.
Repo Map
Apps
| App | Status | Notes |
|---|---|---|
| Honeycrisp | Runs on the store | Local-first notes. Folders and notes are rows; a note's prose is a rich-text type inside its row. |
| Matter | Runs, separately | Typed grid over user-owned Markdown folders. It edits ordinary .md files directly; matter.sqlite is a disposable query mirror. |
| Local Books, Local Mail | Run, separately | Headless CLI mirrors that pull QuickBooks and Gmail into local SQLite. |
| API | Hosted infrastructure | Personal cloud Worker. Owns the store authority binding, hosted-only billing, and the dashboard. |
| Self-host | Reference deployable | Community-supported single-partition instance without hosted billing. |
| Whispering, vocab, skills, Epicenter | Do not compile | Awaiting migration onto the store. See Status. |
| Other app folders | Research and prototypes | Useful history and experiments, not the current product lineup. |
Packages
These packages carry the main architecture.
| Package | Role | License |
|---|---|---|
@epicenter/data |
The store: one Yjs document per application, a synchronous surface over it, and the transport that carries it. | MIT |
@epicenter/workspace |
The inert workspace declaration vocabulary: arktype JSON, row addresses, and nonconformance. | MIT |
@epicenter/sqlite |
Neutral embedded-SQLite driver with Browser, Bun, and Durable Object adapters. It owns no product schema. | MIT |
@epicenter/sync |
The WebSocket subprotocol vocabulary both halves of a handshake must agree on. | MIT |
@epicenter/ui |
Shared Svelte component library used by multiple apps. | MIT |
@epicenter/server |
Shared Hono server library composed by the hosted API and the self-host reference deployable. | AGPL-3.0-or-later |
Architecture
The server side is split into one shared library and two deployable folders:
packages/server
shared Hono library
route composition for auth, sessions, store sync, blobs,
and provider-backed inference and transcription
apps/api
hosted personal Cloudflare Worker
composes packages/server with a Better Auth principal resolver
owns hosted-only dashboard and billing code
apps/self-host
self-hosted single-partition instance reference deployable
composes packages/server with the instance principal resolver
community-supported
no hosted billing surfaceFull architecture walkthrough | Trust model
Development
Use Bun in this repo.
git clone https://github.com/EpicenterHQ/epicenter.git
cd epicenter
bun installEvery app starts from the repo root. bun dev:<app> runs every process the app needs; for apps that talk to the hosted API, that includes the API worker on localhost:8787. bun dev:<app>:ui runs the app's frontend alone when that split exists, and bun dev:api runs just the backend. Bare bun dev is bun dev:honeycrisp, and bun run with no arguments lists every target.
| Command | Starts | App port |
|---|---|---|
bun dev:honeycrisp |
API + Honeycrisp desktop | 5175 |
bun dev:honeycrisp:ui |
Honeycrisp in the browser, no API or Tauri shell | 5175 |
bun dev:api |
Hosted API worker alone | 8787 |
bun dev:api-dashboard |
API + dashboard UI | 5178 |
bun dev:landing |
Landing site, standalone | 4321 |
bun dev:matter |
Matter desktop, standalone | 5180 |
bun dev:posthog-reverse-proxy |
PostHog reverse proxy Worker | wrangler default |
bun dev:self-host |
Self-host server (needs INSTANCE_TOKEN) |
8787 |
bun dev:whispering, bun dev:vocab, bun dev:skills, and bun dev:epicenter still exist, but those apps do not compile until they are migrated onto the store.
The API needs local Postgres and Infisical; see apps/api/README.md. Rust is needed for Tauri apps such as Honeycrisp and Matter. Local Books and Local Mail run their own multi-process dev flows; their READMEs document them.
bun run check is the gate. It runs lint, typecheck, every workspace test, and the structural checks, and it is the same gate CI runs, so a green local run predicts a green pull request. Formatting is handled separately by the autofix workflow.
bun run checkRun the pieces on their own while you work:
bun run format # rewrite formatting (CI autofixes this for you)
bun run lint:check
bun run typecheck
bun run test
bun run check:structure # doc paths, catalog pins, API paths, licenses, UI boundary, boot purityTwo checks sit outside the gate on purpose. bun run check:doc-hygiene flags specs and ADRs that time has made stale, so it belongs to review rather than to merge. bun run smoke:local boots the API against local services.
Design Notes
Durable decisions and their reasoning live in docs/adr/. Specs in specs/ are in-flight design scaffolding rather than current truth; when a spec and an ADR disagree, the ADR wins. Start with docs/README.md.
Contributing
Contributions are welcome. Good entry points are docs, local-first infrastructure, Svelte interfaces, migrating a broken app onto the store, and small changes that make the repo easier to understand.
Contributors coordinate in Discord.
License
Epicenter uses a two-tier split by how you use the code:
- MIT for code you build with: the toolkit roots (
@epicenter/data,@epicenter/workspace,@epicenter/ui) and the toolkit-internal contracts they carry (@epicenter/field,@epicenter/sqlite,@epicenter/sync,@epicenter/identity,@epicenter/agent-protocol,@epicenter/chat). - AGPL-3.0 or later for code we ship or run: every app, the shared server library, and the rest of the internal packages.
- There is no proprietary tier today. Revenue is intended to come from hosting and services, not from selling closed licenses.
Every dependency of the toolkit packages is MIT-compatible, enforced by bun run check:licenses. The license split follows the same broad pattern as Plausible and PostHog for hosted open-source services, and Yjs for MIT core libraries with copyleft server pieces.
See the root LICENSE, FINANCIAL_SUSTAINABILITY.md, and the licensing strategy for the full model.
File: apps/posthog-reverse-proxy/README.md
PostHog Reverse Proxy
A Cloudflare Worker that acts as a reverse proxy for PostHog for our landing page. This proxy routes requests to PostHog through our own domain (proxy.epicenter.so).
Development
bun dev:posthog-reverse-proxyThis starts Wrangler from the repo root.
File: apps/matter/README.md
Matter
A grid view over a folder of markdown files. Each .md file is a row, its
frontmatter is the typed columns (declared in matter.json), and the body is the
one rich field.
One line: Rust is a race-free, schema-blind streamer of file changes over a
caller-supplied channel; TypeScript owns all meaning and funnels every change,
external or self-inflicted, through a single path into the grid and a disposable
SQLite mirror.
The catch: you are not the only writer
Agents, git, and your own text editor all edit these files too. So the app
cannot "read the folder once", it has to stay live as the folder changes
underneath it. The naive read-then-watch approach has a famous bug:
read folder โโโโโถ [โ someone edits a file HERE] โโโโโถ start watching
(this change happened after the read and
before the watch: it is lost, silently)Read first, watch second, and any edit in that gap vanishes. The grid is then
permanently wrong until some unrelated edit happens to refresh it.
How Rust closes the race (`src-tauri/src/watch.rs`)
watch_folder does the three steps in the safe order, inside one command:
watch_folder(path, channel):
1. ARM the watcher โ nothing can slip through after this point
2. SCAN the folder โ the "seed": current contents
3. SEND the seed on the channel
โ return a watch idArming before scanning means any edit during the scan is already captured. The
race is gone by construction, not by luck.
The key choice: the Channel is passed into watch_folder. The frontend
hands Rust a pipe, and Rust pushes both the seed and every later change down that
same pipe. There is no separate "subscribe" step that could run too late and miss
the seed.
This is also why the IPC layer uses
ts-rs(type-only codegen) and nottauri-specta: tauri-specta cannot typeChannel<T>, and switching to
broadcast events would reopen exactly this subscribe-then-emit race.
After the seed, Rust is a dumb, faithful byte-streamer. On a change it:
- waits 100ms (debounce) to let a burst settle,
- dedups by filename,
- reads each changed file fresh,
- sends a
FileDelta[]batch, where each delta is self-contained:content(the bytes),removed, orunreadable.
Rust never learns what a "column" or a "schema" is. All meaning lives in
TypeScript. FileDelta is the one wire contract, defined once in Rust and
generated into src/lib/bindings/FileDelta.ts by ts-rs (run cargo test insrc-tauri to regenerate), so the two sides cannot drift.
How TypeScript consumes it (`src/lib/vault.svelte.ts`)
Every batch, from the seed and from every later change, flows through one
function:
channel batch โโถ applyDeltas(deltas)
โโ mutate the `files` SvelteMap โ grid repaints instantly
โโ setTimeout(reconcileMirror, 0) โ rebuild matter.sqlite after paintOne funnel, one store, two projections of it. The view is the in-memory,
reactive, editable one (read: a pure $derived that classifies each row against
the loaded model); it drives the grid. The mirror (matter.sqlite) is the
on-disk, disposable, queryable one you can run raw SQL against, from an
out-of-process reader (a coding agent) or in-app (see Querying).
The mirror is rebuilt as a full DROP + CREATE + INSERT so it is always a pure
function of the folder, never a drifting incremental cache, and deferred off the
paint task because a large projection should never delay the grid.
The twist: the app's own edits
When you edit a cell, the app writes the file to disk. Originally it then waited
for that write to echo back through the 100ms watcher before the grid updated, a
visible lag. Now it short-circuits:
your edit โโถ atomic write โโถ on success: feed the bytes into applyDeltas yourself (instant)
the watcher's later echo re-applies identical bytes (harmless)So the mental model is clean:
- Your edits are applied immediately by the writer itself.
- Everyone else's edits arrive through the watcher.
- Both land in the same
applyDeltasfunnel, so the grid and the SQLite mirror
never care where a change came from.
Writes are read-modify-write (read the freshest bytes, transform in JS, write
atomically) so a concurrent external edit to a different field is preserved, not
clobbered. Writes to one file are serialized (serializeWrite) so two quick
edits to the same file cannot interleave their read-modify-write and drop one.
Querying the mirror
The mirror is read-only from the app's side: one Rust command, query_mirror, opensmatter.sqlite SQLITE_OPEN_READ_ONLY, runs a caller-supplied SELECT, and returns
generic { columns, rows } (capped by an optional limit, or every row when omitted).
Read-only means a query can never mutate the disposable mirror; busy_timeout lets it
wait out an in-flight rebuild. Rust stays schema-blind: it runs the statement and hands
back values it never interprets.
Two surfaces ride that one command:
- The WHERE filter (folder header): you type a predicate like
status = 'ready' and word_count > 500; the page callsvault.matchingNames(clause)
(which runsSELECT "name" FROM "<folder>" WHERE <your clause>) and hands the grid the
matched row names, which the grid intersects to narrow its live rows, still typed, still
editable. The filter is owned by the page (where the live vault is), not the grid,
so the grid never holds a query engine, it just renders the rows it is told to. It
matches only valid rows (the only ones in the mirror); invalid rows are the separate
"needs attention" axis. - A SQL console (planned): the full
SELECT ...shown as a raw result table,
for aggregations and for seeing exactly what an agent sees.
The filter is debounced and re-runs when the data changes, so a small lag exists
between editing a value and a row leaving or entering the filtered set: the value
updates instantly, only set membership waits for the reconcile plus the re-query.
Invariants worth protecting
- The grid never blocks on
matter.sqlite; the index is a pure side channel. - The store equals disk after every settled write (apply only on success).
- Switching folders never lands a write for the wrong folder (the path is
captured per vault). matter.sqliteis a pure function of the folder's valid rows; an unmodeled
folder has no typed table.
Where things live
| Concern | File |
|---|---|
| Live vault: funnel, store, writes, queries, watch lifecycle | src/lib/vault.svelte.ts |
| Folder watcher + race-free seed (Rust) | src-tauri/src/watch.rs |
| Atomic read/write of one entry (Rust) | src-tauri/src/entry.rs |
matter.sqlite mirror: write + read-query executor (Rust) |
src-tauri/src/mirror.rs |
| Wire contract, generated from Rust | src/lib/bindings/FileDelta.ts |
| Classify rows against the model | src/lib/core/folder.ts |
| Project valid rows to SQL | src/lib/core/sqlite.ts |
Parse one .md entry / the model |
src/lib/core/parse.ts |
| The grid UI | src/lib/components/FolderGrid.svelte |
| Page: vault lifecycle + the WHERE filter | src/routes/+page.svelte |
Developing
bun run tauri dev # desktop app, empty (open a folder via the picker)
cargo test # in src-tauri: runs Rust tests AND regenerates FileDelta.ts
bun run typecheckMatter writes your edits back to disk, so the dev loop runs against a disposable
sandbox, never your real files. dev:fixture copies the sample fixture into a
gitignored sandbox (apps/matter/.dev-vault) and launches the app, so every edit
and the matter.sqlite mirror it drops land on throwaway files:
bun run dev:fixture # copy the sample fresh, then open Matter
bun run dev:fixture --keep # reuse the existing sandbox (keep what you typed)Open the printed sandbox path in the folder picker. The app persists open folders
by path, so you only pick it once: the stable sandbox path keeps working across
resets and reloads. By default the script re-copies on every launch, so you always
start from known state. The sample (examples/matter/sample-vault/drafts) covers
every conformance category (valid, invalid, unparseable, no-frontmatter), so one
fixture is enough.
A real folder opens the exact same way: pick it in the picker (it watches one
flat folder with its own matter.json, edits write to those actual files, and it
persists across reloads, so you only pick it once). The app needs no dev-only mode
to tell the two apart, the sandbox is just a folder you opened.
Verifying the adopt flow
dev:fixture opens the sample's drafts folder, which is already a table (it
has a matter.json), so it never shows the "adopt" empty state. To see and test
adopt you must open an unmarked folder: one with no matter.json and no marked
subfolders. That empty state, with its "Adopt this folder as a table" button, only
appears when vault.tables.length === 0.
The result of adopt is provable without the GUI, because adopt only writes a {}
marker and loadPath is what reads it back:
T=$(mktemp -d)/notes && mkdir -p "$T"
printf -- '---\ntitle: First note\n---\nhello\n' > "$T/note-a.md"
printf -- '---\ntitle: Second note\n---\nworld\n' > "$T/note-b.md"
bun run check "$T" # BEFORE: "0 tables" (unmarked, not a table)
printf '{}' > "$T/matter.json"
bun run check "$T" # AFTER: "1 untyped (1 table, 2 rows)"That confirms the classification half. The GUI adds one thing on top: the root
watcher must re-scan live when the button writes the marker. To verify that end
to end:
Create a throwaway unmarked folder with a couple of notes (no
matter.json):shmkdir -p ~/matter-adopt-test printf -- '---\ntitle: First note\n---\nhello\n' > ~/matter-adopt-test/note-a.md printf -- '---\ntitle: Second note\n---\nworld\n' > ~/matter-adopt-test/note-b.mdWith the app running (
bun run devorbun run dev:fixture), open~/matter-adopt-testin the folder picker. It is unmarked with no marked
children, so it shows "Not a table yet" with an "Adopt this folder as a
table" button.Click adopt. The app writes
matter.json({}); the root watcher sees the new
top-level marker, re-scans, and the folder appears live as one untyped table
withnote-aandnote-bas rows and atitlecolumn. No reload needed.Confirm the marker landed:
shcat ~/matter-adopt-test/matter.json # -> {}
If the table does not appear after the click, run step 4 anyway to localize it:
matter.jsonis{}but the grid stays empty -> the write works, the live
re-scan did not fire. Reopen the folder; if the table shows after reopen, that
is a root-watcher bug (the non-recursive watch missed the marker create).matter.jsonis absent and an error shows under the button -> the write
failed; the message under the button is the cause (permissions, path).
Empty-folder variant: an unmarked folder with no .md adopts the same way and
shows a table with zero rows. Clean up with rm -rf ~/matter-adopt-test.
File: apps/local-mail/src-tauri/README.md
local-mail desktop shell
The smallest native wrapper around the Bun mail engine (local-mail app). It
owns the window and the child engine's lifetime, and nothing else. Gmail OAuth,
token refresh, the reconciler that is Gmail's only writer, the SQLite mirror, the
durable intent.db, and the loopback API bearer all stay in Bun end to end
(ADR-0116, ADR-0199). Rust never sees a Gmail token or the bearer.
How it works
setup()spawnsbun src/bin.ts appfrom the engine package dir.- The engine prints the loopback origin it serves (
http://127.0.0.1:)
to stdout, once, only after it is listening. That single line is both the
discovery signal (which port) and the readiness signal (it is up). - The shell opens a
WebviewUrl::Externalwindow at that origin. The engine's
own response headers (frame-deny,no-store, CSP) are the security boundary;
Tauri does not inject CSP into external pages, and the webview never callsinvoke(), so the shell registers no commands and needs no remote-IPC
allowlist. - Closing the window quits the app, which kills the engine (
RunEvent::Exit).
The engine never outlives the window: no background mail service (ADR-0116).
This is the ADR-0084 sidecar recipe (spawn the engine, read the port from
stdout, point a window at it), the pattern that spec named for the Tauri wrap.
Run it (dev)
Prerequisites: bun on PATH, and the UI built once so the engine has something
to serve:
bun run --cwd apps/local-mail/ui buildThen, from apps/local-mail:
bun run app:desktop # == tauri dev: cargo runs the shell, which spawns the engineA connected account is required (bun run src/bin.ts connect once). The shell
opens no window until the engine reports its origin, so a blank moment at launch
is the engine starting, not a hang. Watch the terminal for [sync ...] and thelistening on http://127.0.0.1:... hint.
Only bun is called: no port is pinned, so this never collides with a separatelocal-mail app you already have open. That instance keeps the per-account
reconciler lock, so it is the only one delivering to Gmail; this one serves reads
and can still record triage assertions, which the lock holder delivers on its
next wake (ADR-0199). Both run under their own ephemeral origins.
Build a desktop bundle
Set the public Google Desktop client identity that belongs to the distribution,
then build:
LOCAL_MAIL_DISTRIBUTION_GMAIL_CLIENT_ID=... \
LOCAL_MAIL_DISTRIBUTION_GMAIL_CLIENT_SECRET=... \
bun run desktop:buildbeforeBuildCommand runs bun run build:desktop first, which builds the SPA
(ui/dist) and compiles the engine sidecar
(src-tauri/binaries/local-mail-engine-<target-triple>, a bun build --compile
binary, ~60MB). Tauri then bundles both intosrc-tauri/target/release/bundle/macos/Local Mail.app.
Cargo compiles that pair into the native shell. At launch, the shell passes it
to the Bun engine as public application configuration; OAuth, tokens, mail, and
the SQLite mirror still never transit Rust. An inheritedGMAIL_CLIENT_ID / GMAIL_CLIENT_SECRET pair remains the machine-wide override
and wins atomically. A source build without a compiled distribution identity
must provide that override.
How the packaged app differs from tauri dev:
- The engine is a compiled sidecar, not
bun src/bin.ts. A packaged app has
no repo beside it andbunmay be absent from the user's PATH (ADR-0116 keeps
the whole mail engine in Bun, so it ships as one binary).bundle.externalBin
registersbinaries/local-mail-engine; Tauri strips the target-triple suffix
and copies it intoContents/MacOS/local-mail-engine.engine_commandbranches
oncfg!(debug_assertions): dev spawnsbun src/bin.ts app; release resolvescurrent_exe().parent()/local-mail-engine. - The SPA is a bundled resource, not a source sibling. A
bun build --compile
binary'simport.meta.diris a virtual path with noui/distbeside it, so
the SPA ships viabundle.resources(../ui/dist->Contents/Resources/ui-dist)
and the shell passesLOCAL_MAIL_UI_DIST(resolved fromresource_dir()) when
it spawns the sidecar. The engine's serving code is identical to dev; only the
root path differs. Sameappentrypoint, same loopback contract.
Rust still owns only the window and the child's lifetime. The sidecar is the
same Bun engine; no Gmail token, sync, mirror, or bearer transits Rust
(ADR-0116).
Signing and notarization (the one deferred item)
The produced .app is unsigned (ad-hoc). That is fine for a locally built
app run on the same machine: it is not quarantined, so Gatekeeper does not block
it. It is not distributable as-is; a downloaded copy would be quarantined and
refused without a right-click-open.
To sign and notarize, no code changes are needed, only credentials and config:
- An Apple Developer ID Application certificate in the login keychain.
bundle.macOS.signingIdentityset (or theAPPLE_SIGNING_IDENTITYenv var),
plusbundle.macOS.hardenedRuntime: trueand an entitlements file if any
hardened-runtime exceptions are needed.- Notarization credentials (
APPLE_ID,APPLE_PASSWORD,APPLE_TEAM_ID) sotauri buildcan staple the notarization ticket.
Blocker, not a design gap: this machine has no Apple Developer credentials
configured, so the wave ships an unsigned local bundle and leaves the signing
config to whoever holds the certificate. Crib apps/honeycrisp for the
hardened-runtime block.
Deferred niceties (not correctness gates)
- Graceful engine shutdown. Exit kills the engine with SIGKILL, which is
safe by design (the reconciler lock is a kernel-released fcntl lock; presence
is stale-safe). A SIGINT/SIGTERM path (so the engine clears its presence file
and releases the lock cleanly) is a nice-to-have. Undelivered work is not at
risk either way: an assertion is committed tointent.dbwhen the act returns,
and a kill between a Gmail call and its retirement leaves the row, so the next
pass redelivers it and Gmail no-ops the repeat (ADR-0198, ADR-0199). - Orphan on hard kill. If the app is SIGKILLed, the engine is reparented and
keeps running (holding the reconciler lock). Harmless and self-healing (the
lock is crash-safe; the next launch or a manualkillclears it), but worth a
parent-death watchdog if it ever bites.
File: apps/local-mail/test-support/README.md
Local Mail write-path test support
Local-only harness for exercising Local Mail's triage path (archive, undo, label,
mark read/unread, star, trash) against a mock Gmail backend and a throwaway
copy of your data dir, so you can verify triage UX without ever touching real
Gmail or your real mirror and intent store.
A triage act is a local write and the reconciler is the only thing that reaches
Gmail (ADR-0199), so the harness covers both halves: the act, which must be
visible to the next read immediately, and the pass that delivers it.
This is developer tooling, not bun test. The smoke needs a real connected
mirror to copy from; check-gmail-discovery.ts makes a live Google call. Neither
is hermetic, so both stay out of the offline suite.
Gmail API drift check
check-gmail-discovery.ts fetches Gmail's Discovery document and asserts that
every method and schema field src/gmail-client.ts / src/schema.ts rely on is
still present and still the type we expect. Our schemas are deliberately partial
and permissive (they tolerate unknown Gmail fields, and every read field is
optional, so a removed field passes Value.Check and reaches a reader asundefined), which is exactly what makes Gmail removing, moving, or
retyping something we depend on invisible until a live sync misbehaves.
The schema-side contract is not re-listed: the check walks the actual schema.ts
TypeBox objects (JSON Schema at runtime), so schema.ts stays the single source
of the fields we read. The only hand-maintained pieces are the small set of
methods we call (the client builds those paths as string templates, nothing to
derive) and the root-schema name map.
bun run --cwd apps/local-mail check:gmail-driftIt runs weekly (and on demand) via .github/workflows/local-mail.gmail-drift.yml,
not per-PR: it is a network call and drift is slow-moving. Exits non-zero listing
each drift.
Safety model
Four independent guarantees keep this from touching anything real:
- A throwaway copy, never the real data dir.
setup-copy.shcopies the
real app dir (~/Library/Application Support/so.epicenter/apps/local-mail)
intoLM_TEST_DIR/apps/local-mail(default root/tmp/local-mail-harness)
and pointslocal-mail appat that root viaEPICENTER_DATA_DIR. The copiedintent.dbis dropped: it holds triage the real account still owes Gmail, and
the copy starts owing nothing. - Forged credentials, so no Google contact. The copy's
credentials.json
is rewritten with a dummy access token whose expiry is the year 2099. The
token manager only refreshes near expiry (src/token-manager.ts), so the app
reuses the dummy bearer forever and never calls Google's token endpoint. - A local mock, not
gmail.googleapis.com.LOCAL_MAIL_GMAIL_API_BASE
points the client atmock-gmail.tson127.0.0.1. The mock services onlymessages.modify(logged) and a no-ophistory.list; every other route
returns a non-retryable 403, which the pull phase treats as a hard failure
rather than a signal to run a FULL pull. So the mock can never wipe even the
copy. - A fingerprint proof.
fingerprint.shhashes the real data dir's durable
files (credentials.json, everymail.v<version>.dbincluding predecessors,
and everyintent.db); capture it before and after and diff to confirm
nothing real changed.intent.dbmatters most: the mirror is re-pullable from
Gmail and undelivered triage is not (ADR-0198).
Files
| file | what it is |
|---|---|
mock-gmail.ts |
Mock Gmail REST server. Reads the copy's SQLite to know current labels, applies the modify, logs it, 403s everything else. |
setup-copy.sh |
Copies the real app dir into LM_TEST_DIR/apps/local-mail, drops the copied intent store and lock, and forges dummy credentials. |
fingerprint.sh |
Hashes the real app dir's durable state, for the before/after safety proof. |
boot.ts |
Shared boot used by smoke.ts (and any manual session): stands up copy + mock + the app on ephemeral ports and hands back the launch coordinates. The one owner of the safety-critical wiring. |
smoke.ts |
Headless one-shot: records one real act through /api/accounts/:account/messages/assert, proves the next read already reflects it, reconciles, asserts the delivery hit the mock, and asserts the real data dir is unchanged. |
check-gmail-discovery.ts |
Gmail API drift check: fetches the live Discovery doc and asserts the methods (hand-listed) + schema fields (walked from schema.ts) the client relies on are still present and correctly typed. |
Runtime artifacts (the copy, the modify log, server logs) live underLM_TEST_DIR, never inside the repo.
Automated smoke (no browser)
Proves the full server โ mock write path end to end and tears itself down:
bun run apps/local-mail/test-support/smoke.tsOn success it prints SMOKE PASS, the mock log line for the delivery, and
confirms the real data dir's fingerprint is unchanged. Exits non-zero on any
failure.
Manual write-UX check (browser)
The affordances the API smoke can't assert (undo toast, keyboard triage, the
pending chip, the shortcuts overlay) are verified by hand. boot.ts exportsbootHarness(), which stands up the same safe stack (copy + mock + the app) and
returns a launch URL; call it from a scratch script or the REPL, open the URL,
and poke the SPA. Build the SPA first or the page is blank:
bun run --cwd apps/local-mail/ui buildWatch the deliveries land:
tail -f /tmp/local-mail-harness/modify-log.jsonlRead-only smoke against your real mirror
No copy, no writes; a dead Gmail base no-ops the reconcile loop and every action
button is disabled:
bun run --cwd apps/local-mail/ui build
LOCAL_MAIL_READ_ONLY=1 LOCAL_MAIL_GMAIL_API_BASE=http://127.0.0.1:9 \
LOCAL_MAIL_PORT=4181 bun run apps/local-mail/src/bin.ts appProving the real mirror was untouched
bash apps/local-mail/test-support/fingerprint.sh > /tmp/lm-before.txt
# ...run smoke.ts or a manual bootHarness session...
diff /tmp/lm-before.txt <(bash apps/local-mail/test-support/fingerprint.sh) # must be emptysmoke.ts does this automatically. Note the fingerprint covers only durable
files; the read-only smoke above legitimately syncs the real mirror, so run that
one separately from a fingerprint window.
Environment knobs
| var | default | meaning |
|---|---|---|
LM_TEST_DIR |
/tmp/local-mail-harness |
the throwaway Epicenter data root the copy and the logs live under; EPICENTER_DATA_DIR points the app at it |
LOCAL_MAIL_REAL_DIR |
the real app dir under the macOS Epicenter data root | the app dir to copy/fingerprint |
LOCAL_MAIL_ACCOUNT |
the sole connected account | which account to forge (required if you have more than one) |
File: apps/local-mail/README.md
local-mail
Headless Gmail reader and triage ledger for local tools and agents. It pulls a
Gmail account into a private SQLite mirror with full pulls plus incrementalhistory.list polling, records triage acts as durable local assertions, and
delivers them back to Gmail from one writer per account. CLI queries, a stdio
MCP server, and a loopback SPA all read the same two files.
Two files per account carry that, and the difference between them is the whole
design:
| File | Holds | If you lose it |
|---|---|---|
mail.v<version>.db |
Gmail's facts, mirrored | Nothing but quota; re-pull |
intent.db |
What the user asked for, undelivered | Real work, unrecoverably |
Design authority lives in ADRs first, then the current code:docs/adr/0081-*.md (a per-device grant and mirror), docs/adr/0082-*.md
(push-free history.list polling), docs/adr/0098-*.md (state round-trips
through Gmail), docs/adr/0116-*.md (desktop-first, one Bun engine),docs/adr/0188-*.md (who owns the Google application identity, and the
device-only credential boundary), docs/adr/0196-*.md (the mirror is a reader
and one full message fetch is its entire budget), docs/adr/0197-*.md (the
artifact is named by its corpus version), docs/adr/0198-*.md (what a durable
write is), and docs/adr/0199-*.md (who delivers it). The mirror is Gmail-owned
cache data. Human-meaningful mail state must round-trip through Gmail, not a
local-only table; an undelivered assertion is not an exception to that, because
it names a Gmail label id and exists in order to stop existing.
The write model
There is one model, and this is it (docs/adr/0198-*.md, docs/adr/0199-*.md).
A triage act writes a durable assertion, not a Gmail call. Acting on a
message records one row per touched label in a durable intent.db, a per-account
partial map from (Gmail message id, Gmail label id) to want or do-not-want. Ids
only, never label names: Gmail's label ids are immutable, its names are editable
display strings. The map holds opinions only about labels the user touched, so it
never claims a whole desired label set, and it is keyed so that re-asserting a
pair overwrites it. Archive, then un-archive, then archive is one row.
One reconciler per account is the only thing that writes to Gmail. It drains
the account's assertions, retires each one Gmail confirms, then pulls Gmail's
facts. Running a pass requires the per-account lock as a value, not as a promise
the caller made: reconcileAccount takes the capability acquireReconcileLock
mints, so writing and pulling cannot interleave and a second writer cannot be
written by accident.
It wakes on app start, the poll interval, a coalesced local write, and an
explicit reconcile, and never after the app closes.
Reversal is a new assertion. Undo asserts the opposite want on the same key.
Nothing retires an assertion by comparing it to the mirror: the mirror lags Gmail
and can be incomplete, so a "the mirror already agrees" rule would drop real
writes on stale evidence. Only provider confirmation retires an assertion, and
only the reconciler does it.
Undelivered work is visible and discardable. Status reports the undelivered
count, the age of the oldest, and, in the running app, the current reconcile
failure as one message. local-mail discard --all abandons every undelivered
assertion so none is sent; it cannot un-send one already delivered. There is no
attempt counter, no per-row error, no retry schedule, no dead-letter tier, and
nothing expires. Retries are bounded by a human, which is what discard is for.
Refused, deliberately: a generic queue or event log, a payload or operation
table, a provider-independent request abstraction, standing policy over a thread
or query, cross-account atomicity, messages.batchModify, background delivery
after the app closes, and browser-memory optimism beside the durable ledger. Each
one is a decision with a record; the full list, with its reasons, is under
Refused at the end of this file.
Shape
Runtime: Bun.
bun:sqlitestores the mirror, built-infetchcalls Gmail,oauth4webapihandles OAuth.Local Mail's one directory is
<epicenter-root>/apps/local-mail, partitioned
by account underaccounts/. The app computes no OS application-data path of
its own: Epicenter owns exactly one root andEPICENTER_DATA_DIRis the only
override, so a CLI run from a terminal and the desktop host operate on the same
mailbox (ADR-0201). The directory is a place, not an inter-app API: no other
app receives a path or a SQLite handle into it, andquery's read-only SQL is
this app's surface over the file this app wrote.One SQLite artifact per connected account, under
<app-dir>/accounts/<accountEmail>/,
namedmail.v<version>.dbafterMIRROR_VERSIONinsrc/db.ts: the version
of the corpus contract this build stores, not the app's release version.src/db-file.tsowns the naming, the two opening modes, the version listing,
and pattern-scopeddeleteOlderVersions; the rest of the app owns the version
constant, DDL, ingestion, cursors, locking, file permissions, readiness, and
deletion timing (ADR-0247). Opening is non-destructive and never falls
back to a lower version: a bump is a new filename, not a migration, so nothing
is dropped or unlinked on open and the predecessor is retained until something
reclaims it. There is no schema version stamped in the file. See ADR-0197.
BumpingMIRROR_VERSIONcosts one full re-pull of the mailbox, and until that
re-pull finishesstatusreports the artifact asbuilding, neverready.
The refresh token lives in a separate0600 credentials.jsonat the app-dir
root, never inside the mirror db.One durable
intent.dbper account, beside the mirror at<app-dir>/accounts/<accountEmail>/intent.db. It holds the account's undelivered label
assertions and is the only irreplaceable local state in the app. Deliberately
not inside the mirror: the mirror is version-named and reclaimable, so a corpus
bump or areclaimPredecessors()would silently discard pending work. It is
outside ADR-0197's<name>.v<version>.dbfilename grammar, so reclamation
cannot name it and neither opening mode touches it.The per-account layout is therefore:
text<root>/apps/local-mail/credentials.json 0600 refresh tokens <root>/apps/local-mail/provider.json 0600 cached client-identity override <root>/apps/local-mail/accounts/<accountEmail>/ mail.v<version>.db 0600 Gmail facts, disposable, re-pullable intent.db 0600 undelivered assertions, durableThe account segment is the address Gmail reports, which Google does not
promise to keep: a renamed Workspace account would strand its partition and
start a second one. Adopting the OpenIDsubclaim is the fix, and it costs
one re-consent per account (ADR-0201); it has not shipped.The app directory and every directory below it are
0700. The mirror artifact, its-waland-shmsidecars,intent.db, andcredentials.jsonare0600.Tables:
messages,labels, and_meta. Thread facts are derived from live
messages instead of stored in a separatethreadstable.messages.resourceholds the parsedmessages.get(format=full)payload
verbatim. It is deliberately not calledraw:format=rawis Gmail's name for
the base64url RFC 5322 blob this app never fetches. Everything SQLite can
project from the resource is a generated column; the only stored derivations
aresubject,sender, andbody_text, which exist because triage pushes its
filter and sort into SQL andjson_extractcannot reach a header array or a
base64url MIME tree. Everything else is derived at read time. See ADR-0196.messages.body_textis decoded at ingest fromtext/plainMIME parts, with
strippedtext/htmlas a fallback. That makes SQL and MCP useful for body
questions without adding FTS yet.The mirror is a disposable offline reader, not an archive. One
messages.get(format=full)is the entire per-message budget: noformat=raw,
nomessages.attachments.get, no attachment or media bytes on disk in any
form. Attachments are metadata only, enough to say "this message has a 42 KB
PDF" and not enough to open it; opening it is Gmail's job. When Gmail returns a
body part as anattachmentIdinstead of inline data, the reader says it has
no offline body for that message rather than spending a second call. See
ADR-0196.The app detail pane can render formatted email from the stored Gmail payload,
but only behind the SPA sanitizer boundary. DOMPurify strips executable markup
and remote assets before the single{@html}sink renders the body on a light
email canvas; plain text stays available as the fallback view.
Commands
Connect once. A packaged distribution can supply its Google OAuth Desktop
client identity through the native shell. The official Epicenter client is not
provisioned in this repository yet, so source builds remain bring-your-own: setGMAIL_CLIENT_ID / GMAIL_CLIENT_SECRET, then run the connect flow.
bun run src/bin.ts connectOne client identity serves every Gmail account connected on this machine. An
explicit GMAIL_* pair is the machine-wide override and wins over the identity
supplied by a packaged distribution. Because Google binds a refresh token to
the client that minted it, changing the effective pair means reconnecting
every account here; a stored grant is never reinterpreted under a new client,
so a mismatch fails loudly on the next refresh instead of surfacing as Google's
opaque invalid_grant. The client id and secret are public application
configuration, not user secrets: the Gmail access and refresh tokens are the
secrets, and they never leave this device. Epicenter Cloud and self-hosted
Epicenter instances are not in the Gmail path at all, so which instance you
select has no effect on any of this.
Provide the pair however you keep local secrets for the first connect:
an export, a local .env, or a secrets manager such asinfisical run --path=/apps/local-mail -- bun run src/bin.ts connect. On the
first successful connect or refresh, Local Mail caches an explicit override to
a 0600 <app-dir>/provider.json sibling to credentials.json. The
distribution identity is never copied into that user-owned file. Every later
run and every other git worktree on this machine reads the override from there,
so no per-worktree config is needed. See .env.example for the
canonical override names.
Local Mail requests gmail.modify so label changes can round-trip through Gmail.
Trash and untrash are in scope and built: the app UI moves a message to Trash and
undoes it, delivered through Gmail's own messages.trash / messages.untrash
verbs rather than as a label delta, though it is stored as an ordinary TRASH
assertion like any other label. Although Google grants send at the same OAuth
layer, Local Mail exposes no send, reply, compose, or drafts today, and
permanent delete is refused outright: messages.delete is deliberately never
wired. The difference matters when reading the two lists at the end of this
file. Send and drafts are unbuilt; permanent delete is a decision.
Headless bootstrap is still available. The refresh token is redeemed
immediately (one refresh grant plus a profile read), so a dead token fails
here rather than on the first sync, and the account email comes from the
Gmail profile instead of being typed:
bun run src/bin.ts seed-token <refresh-token>Run a reconcile pass: drain this account's assertions, then pull Gmail's facts.
bun run src/bin.ts reconcile --full
bun run src/bin.ts reconcile --watchThere is no separate sync verb, because a pass is drain-then-pull and the pull is
its second half. If the open app already owns this account's reconciler, a
headless pass yields to it rather than becoming a second writer.
Check connection and mirror state:
bun run src/bin.ts statusQuery the mirror:
bun run src/bin.ts query "SELECT subject, sender FROM messages ORDER BY internal_date DESC LIMIT 10"Triage messages. Each verb records label assertions for the ids you name; the
account's reconciler delivers them. Output is human-readable; add --json for
the typed outcome MCP and the app HTTP API share.
bun run src/bin.ts archive
bun run src/bin.ts unarchive
bun run src/bin.ts mark-read
bun run src/bin.ts mark-unread
bun run src/bin.ts trash
bun run src/bin.ts untrash
bun run src/bin.ts label --add Work --remove Promotionsarchive, mark-read, and friends are the triage vocabulary; label is the
transparent primitive they desugar to (archive is label --remove INBOX).
They are spellings, not separate write paths: archive, read, star, trash, and a
custom label are one assertion with a different label id, so every verb here
reaches the same single assert. LOCAL_MAIL_READ_ONLY refuses every write while
leaving query/status and the pull half of a pass available.
An assertion is durable the moment the verb returns, so a triage act survives an
offline machine, a killed process, and a reboot. It is not a Gmail write yet:status reports how many assertions are undelivered and how old the oldest is,
and discard is how you abandon them before they are sent.
bun run src/bin.ts discard --all--all is mandatory and there is no partial form. Discard is the only thing in
the system that drops a recorded change without delivering it, and it abandons
rather than recalls: an assertion the reconciler already delivered is Gmail's,
and getting back is a new act like any other.
It takes the account's reconcile lock, so it refuses while the app is open or a
pass is running, and exits nonzero saying so. That is what makes its report true:
a reconciler snapshots what is pending when its drain begins, so a discard
landing mid-pass would delete rows that pass is still about to send, and the
message would claim the change was abandoned while Gmail was hearing the
opposite. Unlike a busy reconcile, which exits 0 because the owner does that
work for you, a busy discard is work that did not happen.
Serve the triage UI and its API from one loopback process:
bun run src/bin.ts appapp is the desktop runtime host: it owns this account's reconciler and serves the triage
SPA (ui/) plus a same-origin /api on 127.0.0.1, then prints the origin to
open (http://127.0.0.1:PORT). It launches no browser; opening the window is the
host's job (a terminal today, Tauri later), not the engine's. Security: every
request is Host-checked first (the DNS-rebinding kill switch); the web UI carries
a per-launch local API bearer that the host injects into the served HTML aswindow.__LOCAL_MAIL__ = { origin, bearer } before the SPA boots (no URL
fragment, no session-exchange endpoint, no sessionStorage), and that HTML is
served no-store and frame-denied so a rotated bearer is never cached and a
cross-origin page cannot frame the auto-authenticated SPA. The bearer is a
loopback credential, never a Gmail token. In app mode the host loads every
connected account and serves them under one origin: GET /api/accounts lists
the loaded accounts, and reads/writes live under /api/accounts/:account/*.
The write surface is two routes over the same core the CLI verbs and MCP tools
use:
| Route | Does |
|---|---|
POST /api/accounts/:account/messages/assert |
Record label assertions for concrete ids and return |
POST /api/accounts/:account/reconcile |
Wake the reconciler now: drain, then pull |
That is the whole write door, and the invariant lives there: a request asserts or
it wakes the reconciler, and nothing else reaches Gmail. GET /status carries the
aggregate reading (undelivered count, age of the oldest, and this host's current
reconcile failure as one message) and never a per-assertion status, because there
is not one. There is deliberately no route that enumerates individual assertions:
the vocabulary the product has for pending work is a count and an age, and
abandoning it is the CLI's discard --all, which is the whole ledger rather than
a selection.
Archive, read, star, trash, and custom labels all desugar to assertions
client-side; trash is an ordinary TRASH assertion here and routes to Gmail'smessages.trash / untrash only at delivery. The old direct-write vocabulary
(POST .../messages/modify, POST .../messages/trash, POST .../sync, and theModifyMessageLabelsOutcome shape with its folded and aborted fields) is
deleted, not deprecated: it described a request that talked to Gmail, and no
request does that any more. LOCAL_MAIL_READ_ONLY disables writes end to end;--port <n> pins the server port (LOCAL_MAIL_PORT is the env fallback).
Bulk and thread actions enumerate concrete message ids from the mirror at the
moment you act. A message that arrives in that thread afterward is untouched,
because nothing durable refers to the thread.
Develop the UI against a running app:
LOCAL_MAIL_PORT=4177 bun run src/bin.ts app # serves /api + writes the presence file
bun run --cwd ui dev # Vite proxies /api to the host and injects its bearer from the presence fileThe dev server reads the host's 0600 presence file (runtime.json) for the
per-launch bearer and injects it on each proxied /api request (SvelteKit's dev
HTML pipeline bypasses Vite's HTML transform, so the prod window.__LOCAL_MAIL__
injection is a proxy-side header in dev). No token is typed by a human. Start the
host first; a host restart rotates the bearer, so restart the dev server to pick
it up. Pin LOCAL_MAIL_PORT if the host uses an ephemeral port.
Serve tools to an MCP host:
bun run src/bin.ts mcpWhen more than one account is connected, local-mail app serves all of them
under one origin with an account switcher. Set LOCAL_MAIL_ACCOUNT only for
headless reconcile, query, and mcp, which operate on one account per
process.
Tools:
query: read-only SQL over the mirror, capped at 1000 returned rows. The read
connection attaches the account'sintent.dbread-only and defines aneffective_labels(message_id, label_id)view over both, so a query can see
Gmail's stored labels and the effective set that overlays this machine's
undelivered assertions on them. Join the view;messages.label_idsis Gmail's
last word, not what the user has already asked for.status: account, cursor, row counts, and the write ledger: undelivered count
and the age of the oldest. Aggregate only, by design. The current reconcile
failure is not here, because only a running reconciler has one and this tool
serves a process that is not one.reconcile: one pass, drain then pull. Yields if another process owns this
account's reconciler.assert_labels: record label assertions on messages by id, adding or removing
labels by id or exact name. It returns what it stored, not what Gmail did,
because delivery has not happened yet; an agent that needs to know its write
landed callsreconcileor readsstatusafterward rather than assuming.
Unlisted underLOCAL_MAIL_READ_ONLY. The CLI triage verbs above are the
human-facing form of this one tool.
Config
GMAIL_CLIENT_ID/GMAIL_CLIENT_SECRET: the machine-wide Google OAuth
Desktop client override. The resolver reads the pair atomically from the
environment first, then<app-dir>/provider.json. A packaged distribution
identity is the fallback and is never persisted as an override. Missing
credentials in a source build fail loudly naming the exact variables.
Changing the effective client requires reconnecting every account on this
machine.LOCAL_MAIL_ACCOUNT: optional account override forreconcile,query, andmcp. Required only when more than one account is connected.EPICENTER_DATA_DIR: the one Epicenter data root. This app lives at<root>/apps/local-mail; there is no per-app override.LOCAL_MAIL_TOKEN_FILE: token file override.LOCAL_MAIL_GMAIL_API_BASE: test plumbing only; points the Gmail client at
a mock server in the MCP subprocess test.LOCAL_MAIL_PORT: fallback for pinning theappserver port; prefer--port <n>for normal use. Also names the port the Vite dev proxy targets
when the host uses an ephemeral port.
The pre-Epicenter-root layout
Local Mail used to keep its data in its own application-data directory
(~/Library/Application Support/local-mail on macOS). That directory is not
read, moved, or referenced by any code here. Connect the account again and the
mirror re-pulls; the old directory is inert bytes you can delete whenever you
notice it.
Testing
Run from this package:
bun test
bun run typecheckThe tests use real bun:sqlite temp files for DB and intent-store behavior, a
fake GmailClient for delivery and sync folding, mock HTTP endpoints for OAuth,
and a real MCP stdio subprocess for the agent-facing protocol surface.
Not built yet
- Remote image loading and Gmail-perfect HTML fidelity. Formatted bodies render
as sanitized inline HTML on a light canvas; remote assets stay stripped, and
there is no show-images proxy yet. - Signed distribution. The Tauri desktop shell (
src-tauri/, run withbun run app:desktop) spawns this engine, reads the origin it prints, and
opens aWebviewUrl::Externalwindow at it, owning only the window and the
engine's lifetime (Rust never touches Gmail tokens, mail data, or the bearer).bun run desktop:buildproduces a local unsigned macOS.appwith the compiled
Bun sidecar and bundled SPA resources. The shell can compile in the
distribution's public Google client identity, but no official Epicenter
identity is provisioned in this repository. Signing, notarization, and the
official Google client still need release configuration, as described insrc-tauri/README.md. - Discard from the SPA.
local-mail discard --allexists; the app shows the
pending count and age but offers no button for it yet. - Account disconnect. There is no disconnect verb. When one is built it must
report the account's undelivered count first and discard on confirmation, and a
reauthorization is not a disconnect (ADR-0199). - Enumeration surfaces. Thread and bulk acts, and asserting
UNREADoff when a
message is opened. The semantics are already fixed: enumerate concrete message
ids from the mirror at the moment the user acts, never at delivery time. - Gmail-backed drafts, send, reply, and compose. Allowed in principle only for
drafts, which round-trip (ADR-0098); none is built, and shipping send would
widen the OAuth grant pastgmail.modify(ADR-0188). - FTS5.
LIKEoverbody_textis enough for the current mirror size.
Refused
These are decisions, not gaps. Each has a record, and adding one back is an ADR
rather than a pull request.
- Local-only mail state: snooze, send-later, and a local tag or read flag
that never reaches Gmail (ADR-0098). A pending assertion is not an exception:
it names a Gmail label id and exists in order to stop existing. - Permanent delete and spam reporting.
messages.deleteis never wired, and
a deferred, discardable, silently-retried intent is the wrong shape for an
irreversible act (ADR-0198). - A locally held draft as durable intent. A Gmail-backed draft is a different
thing and is merely unbuilt; a draft that lives only on this device is a second
durable write shape (ADR-0198). - Attachment and media bytes on disk, in any form. One
messages.get(format=full)is the entire per-message budget (ADR-0196). - Thread-level modify.
threads.modifyapplies to the thread as Gmail sees
it at delivery, which would silently include messages that arrived after the
user acted. Triage is message-level for that reason, not for lack of effort
(ADR-0199). messages.batchModify. Its response body is empty, so it proves nothing
about which assertions landed. The reopen condition is named and specific in
ADR-0199; the quota saving alone does not meet it.- A generic queue, event log, or provider-independent operation abstraction,
and any per-assertion attempt counter, error column, retry schedule, or
dead-letter tier (ADR-0198, ADR-0199). - Background delivery after the app closes, and any always-on warm-mirror
daemon (ADR-0116, ADR-0199). - Push/Pub/Sub, and any LAN or remote exposure: the server binds
127.0.0.1
(ADR-0082, ADR-0116).
File: apps/local-books/README.md
local-books
Your QuickBooks books, as a private SQLite database on your own computer that you (or any AI agent) can grill.
local-books keeps a faithful local copy of a QuickBooks Online company and keeps it current. Once it is synced, you ask questions with plain SQL ("who owes me money?", "where did the money go last quarter?", "what still needs categorizing?"), run live financial statements, and push a category fix back to QuickBooks. Your financial data stays on your machine; nothing goes to a cloud service.
See it work in 10 seconds (no QuickBooks account needed)
local-books demoThis builds a small sample company in a local file and immediately answers a few real questions against it, then prints the exact local-books query commands so you can poke at it yourself. Everything below is how you point it at your own books.
How it works
sync pulls your whole company into one SQLite file: one table per record type (invoices, customers, bills, vendors, purchases, accounts, ...), each row carrying the verbatim QuickBooks JSON in raw plus a few extracted columns for easy filtering and joins. It stays current with incremental Change Data Capture, so a re-sync only fetches what changed.
That file is the product. You grill it three ways:
local-books query "<sql>"runs read-only SQL over the local copy. Writes are rejected at the connection, so this is always safe.local-books report <Name>runs a live computed statement from QuickBooks (ProfitAndLoss,BalanceSheet,CashFlow,AgedReceivables,AgedPayables,TrialBalance). These are read live because QuickBooks owns the math, and there is no change feed to keep a cached copy honest.local-books recategorize --to <accountId>moves an expense to a different account in QuickBooks, then updates the local copy. It is the one command that writes; everything else only reads.
For natural-language questions, point an AI coding agent you already use (Claude Code, Codex) at the file and just ask: it writes the SQL for you. See "Grill it with an AI agent" below.
Connect your company
You need an Intuit app for the API keys (this is a one-time developer step Intuit requires; your books are still yours). At https://developer.intuit.com, create an app, open Keys & credentials, and register http://localhost:8765/callback as a redirect URI. Intuit issues two key sets that are not interchangeable: Development keys connect sandbox (test) companies; Production keys connect your real company and are issued only after Intuit's go-live review. Start with Development and a sandbox company.
Bring your own keys. The name carries the environment, so the sandbox (Development)
keyset is QB_SANDBOX_* and production is QB_PRODUCTION_*; --qb-env picks which
one is read (see .env.example and ADR-0108). Start with sandbox:
export QB_SANDBOX_CLIENT_ID=...
export QB_SANDBOX_CLIENT_SECRET=...
local-books auth # opens a browser; log into your sandbox company (--qb-env sandbox is the default)
local-books sync --full # build the local copy
local-books status # see what is connected and syncedauth captures the company id from the sign-in, so you never pass it by hand. From here on, sync refreshes everything and never needs the browser again.
Grill it
local-books query "SELECT display_name, balance FROM customers WHERE balance > 0 ORDER BY balance DESC"
local-books report ProfitAndLoss --start 2026-01-01 --end 2026-03-31
local-books recategorize Purchase 7011 --to 61 --to-name "Software & Subscriptions"The line-level category lives in raw, so reach it with SQLite JSON, e.g.:
local-books query "SELECT json_extract(line.value, '\$.AccountBasedExpenseLineDetail.AccountRef.name') AS category,
SUM(json_extract(line.value, '\$.Amount')) AS spent
FROM purchases p, json_each(p.raw, '\$.Line') line
WHERE p.deleted = 0 GROUP BY category ORDER BY spent DESC"Grill it with an AI agent
The local copy is just a SQLite file, so any agent that can run SQL can answer questions about your finances without your data leaving the machine. Print its path and point your agent at it:
local-books status # prints the exact path of the local copyThen open Claude Code or Codex in that folder and ask in plain English. To reach it from your phone or another machine, expose your box over a private mesh like Tailscale and drive the agent there; the books still never leave the box. Set LOCAL_BOOKS_READ_ONLY=1 to disable recategorize while you let an agent explore (both reads stay available).
Hand it to an agent over MCP
local-books mcp runs a Model Context Protocol server over stdio, so a coding agent drives the books through real tools instead of hand-written SQL: it gets query, status, report, sync, and, unless LOCAL_BOOKS_READ_ONLY is set, recategorize. The agent spawns the server as a local subprocess that reads the same SQLite file, so the data still never leaves the machine, and the financial data never touches a network (this is why the exposure is local stdio, not a hosted server).
Register it with Claude Code, passing the company through the environment (and the Epicenter data root only if yours is not the default):
claude mcp add local-books \
--env LOCAL_BOOKS_QB_REALM=<company-id> \
-- local-books mcpAdd --env LOCAL_BOOKS_READ_ONLY=1 to hand over the reads without the one write. Then ask in plain English: "what were my three biggest expenses last month?" runs a query, and "re-sync my books" runs sync. From the monorepo (no installed binary), the launch command is bun run /abs/path/apps/local-books/src/bin.ts mcp.
Inspect the server without an agent using the MCP Inspector:
npx @modelcontextprotocol/inspector local-books mcpOpen it in your browser
local-books appapp keeps the mirror fresh and serves a dense browser UI plus a same-origin /api on 127.0.0.1, then prints http://127.0.0.1:PORT/#token=... and opens it. The tab is the app: browse every record type with counts, page a table, inspect a row's extracted fields and raw QuickBooks JSON, run read-only SQL, pull a live report, and (unless read-only) recategorize an expense. The same src/books/* cores back the CLI, the MCP tools, and this UI, so all three agree.
Security is the loopback shell local-mail app established: the server binds 127.0.0.1 only; every request is Host-checked first (the DNS-rebinding kill switch); a single-use bootstrap token rides in the URL fragment and is exchanged once at POST /api/session for a per-launch session bearer (kept in sessionStorage); every other /api call carries that bearer. LOCAL_BOOKS_READ_ONLY disables the one QuickBooks write (recategorize) end to end and hides its UI. --no-open prints the URL without launching a browser; --port <n> pins the server port; LOCAL_BOOKS_NO_OPEN=1 and LOCAL_BOOKS_PORT are env fallbacks. Pick the company with --realm when more than one is authenticated.
Develop the UI against a running app:
LOCAL_BOOKS_DEV=1 LOCAL_BOOKS_TOKEN=devtoken LOCAL_BOOKS_PORT=4178 local-books app
LOCAL_BOOKS_TOKEN=devtoken bun run --cwd ui dev # same token: Vite proxies /api to app, injecting this bearerThe bearer gate is never disabled in any mode; dev just moves the credential server-side into the Vite proxy so no token reaches the browser.
Keep it fresh
local-books sync # refresh now; full vs incremental is chosen automatically
local-books sync --interval 30m # refresh now, then every 30 minutes until Ctrl-Csync is safe to stop and restart anytime (its position lives in the file). To keep it running across logout or reboot, wrap it in a launchd agent (macOS) or a systemd user service.
Connect your real company (production)
Once a sandbox works, switch to production. Set the Production keyset (QB_PRODUCTION_CLIENT_ID / QB_PRODUCTION_CLIENT_SECRET) and select the production API with --qb-env production (set LOCAL_BOOKS_QB_ENV=production once to avoid repeating it). The minted token records its environment, so a later sync or status refuses if you point the wrong --qb-env at it:
local-books auth --qb-env production
local-books sync --qb-env production --fullIntuit production rejects http://localhost redirect URIs, so the one-time auth needs a public HTTPS URL. This is only for auth; once tokens are saved, sync never touches the redirect URI again.
- Start a tunnel to the local callback port (
cloudflaredneeds no account):Copy theshcloudflared tunnel --url http://localhost:8765https://<name>.trycloudflare.comURL it prints. - On the Intuit app's Production redirect URIs, add
https://<name>.trycloudflare.com/callback. - Run
authwith the tunnel as the public redirect, pointing the local listener back at 8765:After tokens land, stop the tunnel;shLOCAL_BOOKS_QB_REDIRECT_URI=https://<name>.trycloudflare.com/callback \ LOCAL_BOOKS_CALLBACK_PORT=8765 \ local-books auth --qb-env productionsyncandstatusneed nothing further.
Inside the Epicenter monorepo
The Intuit keys live in your personal Infisical project at /apps/local-books
under their environment-qualified names (QB_SANDBOX_* andQB_PRODUCTION_*, ADR-0108). This is per-person bring-your-own provider
configuration, not Epicenter-hosted infrastructure. Because the name carries the
QuickBooks target, both keysets sit in prod for a single injection. The
app-local .infisical.json points at the personal project; copy.infisical.json.example to .infisical.json and keep the real file local.
That file is ignored because each operator has a different personal Infisical
project. The committed apps/api and ops configs point at Epicenter's hosted
project instead. The monorepo root has no Infisical config. --qb-env alone
picks the keyset. Instead of exporting keys by hand, wrap a command with
Infisical from this app directory:
infisical run --path=/apps/local-books -- bun run src/bin.ts auth --qb-env sandbox
infisical run --path=/apps/local-books -- bun run src/bin.ts sync --qb-env sandbox --fullThe invocations are wrapped as scripts named for the QuickBooks account they touch, so you do not assemble the flags by hand:
bun run auth:sandbox
bun run sync:production --entity Invoice --full
bun run status:productionWhere things live
<epicenter-root>/apps/local-books/
companies/<company-id>/
books.v<version>.db # record-type tables + sync state; `status` prints the path
credentials.json # OAuth tokens (0600), never inside a company's db
config.json # optional: entities, environment, scheduleEpicenter owns one data root on the machine and gives local-books one directory below it, so a CLI run from a terminal and the Epicenter desktop app see the same books. The root is ~/Library/Application Support/so.epicenter on macOS, %APPDATA%\so.epicenter on Windows, and $XDG_DATA_HOME/so.epicenter (or ~/.local/share/so.epicenter) elsewhere; EPICENTER_DATA_DIR moves it, and it is the only such override.
Tokens live in a 0600 credentials.json at the app-directory root, never inside a company's db, so the read-only query surface can never read them. Override the token path with LOCAL_BOOKS_TOKEN_FILE. That file is also the record of which companies are connected: there is no separate index. With one connected company the verbs find it on their own; with more than one, pass --realm <company-id> or set LOCAL_BOOKS_QB_REALM.
Everything under the app directory belongs to local-books. No other Epicenter app receives a path or a database handle into it: a company's books reach another app only through a verb local-books publishes (the MCP tools above) or through a fact you promote into Epicenter yourself.
The pre-Epicenter-root layout
local-books used to keep its data in its own application-data directory (~/Library/Application Support/local-books on macOS), with a companies.json index beside the tokens. That directory is not read, moved, or referenced by any code here. Run local-books auth again and sync re-pulls the mirror; the old directory is inert bytes you can delete whenever you notice it.
Build a single binary
bun run build:binary # -> dist/local-booksDevelop
bun test # boots a mock QuickBooks server and drives the real command paths
bun run typecheck
bun run src/bin.ts demo # the sample company, end to end, offlineDesign
Standalone CLI by deliberate decision: see ADR-0072 (the daemon/chat surface is deferred, with a trigger to revisit), ADR-0061 (the three capabilities), and the sync engine spec specs/20260621T100000-local-books-cli-sync-engine.md.
File: apps/landing/src/content/blog/crdt-filesystem-pattern.md
title: 'The vault pattern: Why folders beat databases for personal tools' pubDate: 2025-01-29
I've been building personal tools for years. Every time, I start with SQLite. Every time, I regret it.
Not because SQLite is bad. It's fantastic. But because personal tools have different constraints than web apps, and databases optimize for the wrong things.
The problem with traditional databases
I built a journaling app in 2020. Used SQLite, like everyone said I should. Then I wanted to sync between devices. My options:
- Build a sync server (now I'm running infrastructure)
- Use a sync service (now I'm vendor-locked)
- Try to sync the SQLite file (corruption city)
I went with option 1. Six months later, I shut down the server. My journal died with it.
Why CRDTs change everything
Last year I discovered Yjs. It's a CRDT library that makes distributed data structures feel like magic. Here's the key insight that took me too long to realize:
With CRDTs, sync isn't a feature you build. It's a property of the data structure.
// This is all the sync logic you need
doc1.on('update', update => {
Y.applyUpdate(doc2, update)
})That's it. No conflict resolution. No sync servers. No three-way merges.
The vault pattern
Once you have CRDTs, you don't need a database. You need a place to put files. Here's the pattern:
~/YourApp/
โโโ service1/
โ โโโ data.yjs # The CRDT document
โ โโโ assets/ # Binary files (images, etc)
โ โโโ config.json # Service-specific settings
โโโ service2/
โ โโโ data.yjs
โโโ README.md # Explain the structureEach service owns a folder. Each folder contains:
- One or more Yjs documents (the magic)
- Regular files for assets
- Human-readable configs
One document per logical unit
The first time I used Yjs, I made one giant document for everything. Bad idea. Here's why:
- Large documents = slow initial sync
- Every change updates every peer
- Merge conflicts become tangled
Instead, use one document per logical unit:
- One doc per note
- One doc per email thread
- One doc per project
Now syncs are fast, conflicts are isolated, and you can garbage collect old data.
The implementation
Here's actual code from Epicenter's note service:
class NoteService {
constructor(vaultPath) {
this.notesDir = path.join(vaultPath, 'notes')
this.docs = new Map()
}
async createNote(title) {
const doc = new Y.Doc()
const noteId = generateId()
// Define the structure
const note = doc.getMap('note')
note.set('title', title)
note.set('content', new Y.Text())
note.set('created', Date.now())
// Save to disk
const filePath = path.join(this.notesDir, `${noteId}.yjs`)
await fs.writeFile(filePath, Y.encodeStateAsUpdate(doc))
this.docs.set(noteId, doc)
return noteId
}
async loadNote(noteId) {
const filePath = path.join(this.notesDir, `${noteId}.yjs`)
const data = await fs.readFile(filePath)
const doc = new Y.Doc()
Y.applyUpdate(doc, data)
this.docs.set(noteId, doc)
return doc
}
}Simple, right? No SQL schemas. No migrations. Just data structures and files.
Handling binary data
CRDTs are great for text and structured data. They're terrible for binary blobs. So don't put binaries in CRDTs. Put them in the filesystem:
async addAttachment(noteId, file) {
// Hash the content for deduplication
const hash = await hashFile(file)
const ext = path.extname(file.name)
// Store in assets folder
const assetPath = path.join(this.notesDir, 'assets', `${hash}${ext}`)
await fs.copyFile(file.path, assetPath)
// Reference in the CRDT
const doc = this.docs.get(noteId)
const attachments = doc.getArray('attachments')
attachments.push([{
hash,
name: file.name,
size: file.size,
type: file.type
}])
}Now your attachments are just files. You can browse them with Finder. You can back them up with rsync. They're not trapped in a database blob column.
Making it fast
CRDTs can get slow if you're not careful. Here's what works:
- Enable garbage collection:
doc.gc = true - Snapshot periodically: Save the full state every 1000 updates
- Load incrementally: Start with the snapshot, then apply recent updates
- Index separately: Keep a lightweight index for search
Here's our snapshotting logic:
class DocManager {
constructor(filePath) {
this.filePath = filePath
this.snapshotPath = filePath + '.snapshot'
this.updatesPath = filePath + '.updates'
this.updateCount = 0
}
async save(doc, update) {
// Always append the update
await fs.appendFile(this.updatesPath, update)
this.updateCount++
// Snapshot every 1000 updates
if (this.updateCount >= 1000) {
const state = Y.encodeStateAsUpdate(doc)
await fs.writeFile(this.snapshotPath, state)
await fs.unlink(this.updatesPath)
this.updateCount = 0
}
}
async load() {
const doc = new Y.Doc()
// Load snapshot if it exists
if (await exists(this.snapshotPath)) {
const snapshot = await fs.readFile(this.snapshotPath)
Y.applyUpdate(doc, snapshot)
}
// Apply recent updates
if (await exists(this.updatesPath)) {
const updates = await fs.readFile(this.updatesPath)
// Updates are concatenated, need to parse them
let offset = 0
while (offset < updates.length) {
const updateSize = readVarInt(updates, offset)
offset += getVarIntSize(updateSize)
const update = updates.slice(offset, offset + updateSize)
Y.applyUpdate(doc, update)
offset += updateSize
}
}
return doc
}
}The payoff
Here's what you get with the vault pattern:
- True offline-first: No sync server to be offline from
- Debuggable: It's just files. Use standard Unix tools
- Portable: Copy the folder to move your data
- Extensible: Third-party tools can read/write the format
- Future-proof: Even if Yjs dies, your data is in documented formats
When not to use this
The vault pattern isn't always right. Skip it if:
- You need complex queries (use SQLite)
- You have millions of items (use a real database)
- You need ACID transactions (CRDTs are eventually consistent)
- You're building a web service (this is for personal tools)
Try it yourself
I extracted the core vault pattern into a library: vault-crdt. It handles the boring parts (snapshotting, corruption recovery) so you can focus on your app.
Or just steal the ideas. That's why I'm writing this.
Personal tools should be personal. Your data should be yours. And folders of files is the most personal, portable data format we've invented.
Time to stop renting databases for our own thoughts.
File: apps/landing/src/content/blog/hello-world.md
title: 'Hello World' pubDate: 2025-01-28
Welcome to the Epicenter blog. This is where I'll share thoughts on building tools that matter.
What's coming
I'll be writing about:
- Building tools with a focus on user ownership
- Technical deep dives into projects
- Lessons learned from open source development
Stay tuned.