## File: README.md --- ## The Store The hard problem with local-first apps is synchronization. If each device has its own SQLite file, how do you keep them in sync? Epicenter's answer: **an application is one Yjs document, replayed in full before any handle exists, and the surface over it is synchronous.** A read is a property access, not a round trip, so nothing is awaited and nothing needs cache invalidation or race protection. ```typescript import { openDevice } from '@epicenter/data/browser'; import { defineTable, defineWorkspace } from '@epicenter/workspace'; const notes = defineTable({ title: 'string', pinned: 'boolean', folderId: 'string|null = null', }); const notesWorkspace = defineWorkspace({ namespace: 'com.example.notes', tables: { notes }, }); // Every verb returns a Result; the error arm is elided here for length. const { data: db } = await openDevice(notesWorkspace); // the only await db.tables.notes.create({ title: 'Hello', pinned: false }, { document: ['body'] }); const listed = db.tables.notes.list(); // synchronous: { rows, nonconforming } const stop = db.tables.notes.subscribe(read); // fires with the row ids a commit touched ``` A *workspace* is one application's declaration of its durable data: pure JSON, arktype expression strings, no storage and no lifecycle of its own. It is release-local and never migrates your data. A row it cannot read is reported beside the rows it can, with the reason and the raw values intact, and an ordinary write repairs it. Prose merges per character: a row is allocated a document container when it is created, and `db.tables.notes.document(id)?.get('body')` hands an editor a live type to bind to. Epicenter never looks inside it. Sync is one Cloudflare Durable Object per (account, application). Being signed in on two devices is the entire sharing model: nothing is paired, invited, or approved. [Read the data package docs](packages/data/README.md) | [What it replaced, and why](docs/the-store-and-what-it-replaced.md) ## Build With The Toolkit The developer toolkit is MIT: build anything on it, including closed-source and commercial products, and you own what you build, with no obligation back to Epicenter. [`@epicenter/data`](packages/data), [`@epicenter/workspace`](packages/workspace), and [`@epicenter/ui`](packages/ui) are the packages meant to leave this repo. They are pre-1.0 and tuned for our own apps, so treat them as fork-and-own rather than a stability-guaranteed SDK for now. ## Status There is one runtime: a desktop SPA in a WebView, over a store the client owns. A host serves bundles and brokers credentials; it owns no application data. A hosted web runtime with a host-owned replica is refused, and so are third-party installed apps, for now. [Honeycrisp](apps/honeycrisp) is the app running on the store, and its README is the worked example. Whispering, vocab, skills, and the Epicenter host do not compile right now. The superseded data stack was deleted before they were migrated, deliberately: the new store has no row-document HTTP path and no multi-process observation carrier, so there was nowhere for them to move until those refusals landed. Data they held on the old stack is accepted as lost; there is no importer. [Matter](apps/matter) edits user-owned Markdown folders directly and keeps a disposable `matter.sqlite` query mirror beside them. [Local Books](apps/local-books) and [Local Mail](apps/local-mail) are headless mirrors that pull a hosted account into local SQLite. Those three do not use the store. ## Trust Boundaries Pick the trust model you want. | Path | What leaves your device | | --- | --- | | Signed out | Nothing. The store is complete on the machine it opened on, and every read comes from a document already in memory. | | Signed in | Your application's document, as opaque update bytes, to one authority per account. | | Hosted Epicenter | That authority is ours, along with account and session data and any hosted feature you enable. | | Self-hosted instance | You control the server, secrets, deployment, and infrastructure boundary. | | A provider an app calls | Whatever that app sends it: transcript text to an LLM, audio to a transcription provider. Epicenter servers are not in that path. | Signed-in sync sends your data to a trusted server that reads it in plaintext. On hosted Epicenter the authority is ours, so that data sits inside our trust boundary; self-hosting puts it on infrastructure you control, so Epicenter never holds it. See the [trust model](docs/trust-model.md) for the details, including where this is heading with the anchor. ## Repo Map ### Apps | App | Status | Notes | | --- | --- | --- | | [Honeycrisp](apps/honeycrisp) | Runs on the store | Local-first notes. Folders and notes are rows; a note's prose is a rich-text type inside its row. | | [Matter](apps/matter) | Runs, separately | Typed grid over user-owned Markdown folders. It edits ordinary `.md` files directly; `matter.sqlite` is a disposable query mirror. | | [Local Books](apps/local-books), [Local Mail](apps/local-mail) | Run, separately | Headless CLI mirrors that pull QuickBooks and Gmail into local SQLite. | | [API](apps/api) | Hosted infrastructure | Personal cloud Worker. Owns the store authority binding, hosted-only billing, and the dashboard. | | [Self-host](apps/self-host) | Reference deployable | Community-supported single-partition instance without hosted billing. | | [Whispering](apps/whispering), [vocab](apps/vocab), [skills](apps/skills), [Epicenter](apps/epicenter) | Do not compile | Awaiting migration onto the store. See [Status](#status). | | Other app folders | Research and prototypes | Useful history and experiments, not the current product lineup. | ### Packages These packages carry the main architecture. | Package | Role | License | | --- | --- | --- | | [`@epicenter/data`](packages/data) | The store: one Yjs document per application, a synchronous surface over it, and the transport that carries it. | MIT | | [`@epicenter/workspace`](packages/workspace) | The inert workspace declaration vocabulary: arktype JSON, row addresses, and nonconformance. | MIT | | [`@epicenter/sqlite`](packages/sqlite) | Neutral embedded-SQLite driver with Browser, Bun, and Durable Object adapters. It owns no product schema. | MIT | | [`@epicenter/sync`](packages/sync) | The WebSocket subprotocol vocabulary both halves of a handshake must agree on. | MIT | | [`@epicenter/ui`](packages/ui) | Shared Svelte component library used by multiple apps. | MIT | | [`@epicenter/server`](packages/server) | Shared Hono server library composed by the hosted API and the self-host reference deployable. | AGPL-3.0-or-later | ## Architecture The server side is split into one shared library and two deployable folders: ```txt packages/server shared Hono library route composition for auth, sessions, store sync, blobs, and provider-backed inference and transcription apps/api hosted personal Cloudflare Worker composes packages/server with a Better Auth principal resolver owns hosted-only dashboard and billing code apps/self-host self-hosted single-partition instance reference deployable composes packages/server with the instance principal resolver community-supported no hosted billing surface ``` [Full architecture walkthrough](docs/architecture.md) | [Trust model](docs/trust-model.md) ## Development Use Bun in this repo. ```bash git clone https://github.com/EpicenterHQ/epicenter.git cd epicenter bun install ``` Every app starts from the repo root. `bun dev:` runs every process the app needs; for apps that talk to the hosted API, that includes the API worker on `localhost:8787`. `bun dev::ui` runs the app's frontend alone when that split exists, and `bun dev:api` runs just the backend. Bare `bun dev` is `bun dev:honeycrisp`, and `bun run` with no arguments lists every target. | Command | Starts | App port | | --- | --- | --- | | `bun dev:honeycrisp` | API + Honeycrisp desktop | 5175 | | `bun dev:honeycrisp:ui` | Honeycrisp in the browser, no API or Tauri shell | 5175 | | `bun dev:api` | Hosted API worker alone | 8787 | | `bun dev:api-dashboard` | API + dashboard UI | 5178 | | `bun dev:landing` | Landing site, standalone | 4321 | | `bun dev:matter` | Matter desktop, standalone | 5180 | | `bun dev:posthog-reverse-proxy` | PostHog reverse proxy Worker | wrangler default | | `bun dev:self-host` | Self-host server (needs `INSTANCE_TOKEN`) | 8787 | `bun dev:whispering`, `bun dev:vocab`, `bun dev:skills`, and `bun dev:epicenter` still exist, but those apps do not compile until they are migrated onto the store. The API needs local Postgres and Infisical; see [apps/api/README.md](apps/api/README.md). Rust is needed for Tauri apps such as Honeycrisp and Matter. Local Books and Local Mail run their own multi-process dev flows; their READMEs document them. `bun run check` is the gate. It runs lint, typecheck, every workspace test, and the structural checks, and it is the same gate CI runs, so a green local run predicts a green pull request. Formatting is handled separately by the autofix workflow. ```bash bun run check ``` Run the pieces on their own while you work: ```bash bun run format # rewrite formatting (CI autofixes this for you) bun run lint:check bun run typecheck bun run test bun run check:structure # doc paths, catalog pins, API paths, licenses, UI boundary, boot purity ``` Two checks sit outside the gate on purpose. `bun run check:doc-hygiene` flags specs and ADRs that time has made stale, so it belongs to review rather than to merge. `bun run smoke:local` boots the API against local services. ## Design Notes Durable decisions and their reasoning live in [docs/adr/](docs/adr). Specs in [specs/](specs) are in-flight design scaffolding rather than current truth; when a spec and an ADR disagree, the ADR wins. Start with [docs/README.md](docs/README.md). ## Contributing Contributions are welcome. Good entry points are docs, local-first infrastructure, Svelte interfaces, migrating a broken app onto the store, and small changes that make the repo easier to understand. [Read the Contributing Guide](CONTRIBUTING.md) Contributors coordinate in [Discord](https://go.epicenter.so/discord). ## License Epicenter uses a two-tier split by how you use the code: - [MIT](licenses/LICENSE-MIT) for code you build with: the toolkit roots (`@epicenter/data`, `@epicenter/workspace`, `@epicenter/ui`) and the toolkit-internal contracts they carry (`@epicenter/field`, `@epicenter/sqlite`, `@epicenter/sync`, `@epicenter/identity`, `@epicenter/agent-protocol`, `@epicenter/chat`). - [AGPL-3.0](licenses/LICENSE-AGPL-3.0) or later for code we ship or run: every app, the shared server library, and the rest of the internal packages. - There is no proprietary tier today. Revenue is intended to come from hosting and services, not from selling closed licenses. Every dependency of the toolkit packages is MIT-compatible, enforced by `bun run check:licenses`. The license split follows the same broad pattern as Plausible and PostHog for hosted open-source services, and Yjs for MIT core libraries with copyleft server pieces. See the root [LICENSE](LICENSE), [FINANCIAL_SUSTAINABILITY.md](FINANCIAL_SUSTAINABILITY.md), and the [licensing strategy](docs/licensing/licensing-strategy.md) for the full model. --- --- ## File: apps/posthog-reverse-proxy/README.md # PostHog Reverse Proxy A Cloudflare Worker that acts as a reverse proxy for PostHog for our landing page. This proxy routes requests to PostHog through our own domain (`proxy.epicenter.so`). ## Development ```bash bun dev:posthog-reverse-proxy ``` This starts Wrangler from the repo root. - [PostHog Proxy Documentation](https://posthog.com/docs/advanced/proxy/cloudflare) - [Cloudflare Workers Documentation](https://developers.cloudflare.com/workers/) --- ## File: apps/matter/README.md # Matter A grid view over a folder of markdown files. Each `.md` file is a row, its frontmatter is the typed columns (declared in `matter.json`), and the body is the one rich field. > **One line:** Rust is a race-free, schema-blind streamer of file changes over a > caller-supplied channel; TypeScript owns all meaning and funnels every change, > external or self-inflicted, through a single path into the grid and a disposable > SQLite mirror. ## The catch: you are not the only writer Agents, `git`, and your own text editor all edit these files too. So the app cannot "read the folder once", it has to stay live as the folder changes underneath it. The naive read-then-watch approach has a famous bug: ``` read folder ────▶ [✏ someone edits a file HERE] ────▶ start watching (this change happened after the read and before the watch: it is lost, silently) ``` Read first, watch second, and any edit in that gap vanishes. The grid is then permanently wrong until some unrelated edit happens to refresh it. ## How Rust closes the race (`src-tauri/src/watch.rs`) `watch_folder` does the three steps in the safe order, inside one command: ``` watch_folder(path, channel): 1. ARM the watcher ← nothing can slip through after this point 2. SCAN the folder ← the "seed": current contents 3. SEND the seed on the channel → return a watch id ``` Arming before scanning means any edit during the scan is already captured. The race is gone by construction, not by luck. **The key choice: the `Channel` is passed _into_ `watch_folder`.** The frontend hands Rust a pipe, and Rust pushes both the seed and every later change down that same pipe. There is no separate "subscribe" step that could run too late and miss the seed. > This is also why the IPC layer uses `ts-rs` (type-only codegen) and not > `tauri-specta`: tauri-specta cannot type `Channel`, and switching to > broadcast events would reopen exactly this subscribe-then-emit race. After the seed, Rust is a dumb, faithful byte-streamer. On a change it: - waits **100ms** (debounce) to let a burst settle, - dedups by filename, - reads each changed file fresh, - sends a `FileDelta[]` batch, where each delta is self-contained: `content` (the bytes), `removed`, or `unreadable`. Rust never learns what a "column" or a "schema" is. All meaning lives in TypeScript. `FileDelta` is the one wire contract, defined once in Rust and generated into `src/lib/bindings/FileDelta.ts` by `ts-rs` (run `cargo test` in `src-tauri` to regenerate), so the two sides cannot drift. ## How TypeScript consumes it (`src/lib/vault.svelte.ts`) Every batch, from the seed and from every later change, flows through one function: ``` channel batch ─▶ applyDeltas(deltas) ├─ mutate the `files` SvelteMap → grid repaints instantly └─ setTimeout(reconcileMirror, 0) → rebuild matter.sqlite after paint ``` One funnel, one store, two projections of it. The **view** is the in-memory, reactive, editable one (`read`: a pure `$derived` that classifies each row against the loaded model); it drives the grid. The **mirror** (`matter.sqlite`) is the on-disk, disposable, queryable one you can run raw SQL against, from an **out-of-process** reader (a coding agent) or in-app (see [Querying](#querying-the-mirror)). The mirror is rebuilt as a full `DROP + CREATE + INSERT` so it is always a pure function of the folder, never a drifting incremental cache, and deferred off the paint task because a large projection should never delay the grid. ## The twist: the app's own edits When you edit a cell, the app writes the file to disk. Originally it then waited for that write to echo back through the 100ms watcher before the grid updated, a visible lag. Now it short-circuits: ``` your edit ─▶ atomic write ─▶ on success: feed the bytes into applyDeltas yourself (instant) the watcher's later echo re-applies identical bytes (harmless) ``` So the mental model is clean: - **Your edits** are applied immediately by the writer itself. - **Everyone else's edits** arrive through the watcher. - Both land in the same `applyDeltas` funnel, so the grid and the SQLite mirror never care where a change came from. Writes are **read-modify-write** (read the freshest bytes, transform in JS, write atomically) so a concurrent external edit to a _different_ field is preserved, not clobbered. Writes to one file are **serialized** (`serializeWrite`) so two quick edits to the same file cannot interleave their read-modify-write and drop one. ## Querying the mirror The mirror is read-only from the app's side: one Rust command, `query_mirror`, opens `matter.sqlite` `SQLITE_OPEN_READ_ONLY`, runs a caller-supplied `SELECT`, and returns generic `{ columns, rows }` (capped by an optional `limit`, or every row when omitted). Read-only means a query can never mutate the disposable mirror; `busy_timeout` lets it wait out an in-flight rebuild. Rust stays schema-blind: it runs the statement and hands back values it never interprets. Two surfaces ride that one command: - **The WHERE filter** (folder header): you type a predicate like `status = 'ready' and word_count > 500`; the page calls `vault.matchingNames(clause)` (which runs `SELECT "name" FROM "" WHERE `) and hands the grid the matched row names, which the grid intersects to narrow its live rows, still typed, still editable. The filter is owned by the page (where the live vault is), not the grid, so the grid never holds a query engine, it just renders the rows it is told to. It matches only valid rows (the only ones in the mirror); invalid rows are the separate "needs attention" axis. - **A SQL console** (planned): the full `SELECT ...` shown as a raw result table, for aggregations and for seeing exactly what an agent sees. The filter is debounced and re-runs when the data changes, so a small lag exists between editing a value and a row leaving or entering the filtered set: the value updates instantly, only set membership waits for the reconcile plus the re-query. ## Invariants worth protecting - The grid never blocks on `matter.sqlite`; the index is a pure side channel. - The store equals disk after every settled write (apply only on success). - Switching folders never lands a write for the wrong folder (the path is captured per vault). - `matter.sqlite` is a pure function of the folder's valid rows; an unmodeled folder has no typed table. ## Where things live | Concern | File | | --- | --- | | Live vault: funnel, store, writes, queries, watch lifecycle | `src/lib/vault.svelte.ts` | | Folder watcher + race-free seed (Rust) | `src-tauri/src/watch.rs` | | Atomic read/write of one entry (Rust) | `src-tauri/src/entry.rs` | | `matter.sqlite` mirror: write + read-query executor (Rust) | `src-tauri/src/mirror.rs` | | Wire contract, generated from Rust | `src/lib/bindings/FileDelta.ts` | | Classify rows against the model | `src/lib/core/folder.ts` | | Project valid rows to SQL | `src/lib/core/sqlite.ts` | | Parse one `.md` entry / the model | `src/lib/core/parse.ts` | | The grid UI | `src/lib/components/FolderGrid.svelte` | | Page: vault lifecycle + the WHERE filter | `src/routes/+page.svelte` | ## Developing ```sh bun run tauri dev # desktop app, empty (open a folder via the picker) cargo test # in src-tauri: runs Rust tests AND regenerates FileDelta.ts bun run typecheck ``` Matter writes your edits back to disk, so the dev loop runs against a **disposable sandbox**, never your real files. `dev:fixture` copies the sample fixture into a gitignored sandbox (`apps/matter/.dev-vault`) and launches the app, so every edit and the `matter.sqlite` mirror it drops land on throwaway files: ```sh bun run dev:fixture # copy the sample fresh, then open Matter bun run dev:fixture --keep # reuse the existing sandbox (keep what you typed) ``` Open the printed sandbox path in the folder picker. The app persists open folders by path, so you only pick it once: the stable sandbox path keeps working across resets and reloads. By default the script re-copies on every launch, so you always start from known state. The sample (`examples/matter/sample-vault/drafts`) covers every conformance category (valid, invalid, unparseable, no-frontmatter), so one fixture is enough. A **real** folder opens the exact same way: pick it in the picker (it watches one flat folder with its own `matter.json`, edits write to those actual files, and it persists across reloads, so you only pick it once). The app needs no dev-only mode to tell the two apart, the sandbox is just a folder you opened. ## Verifying the adopt flow `dev:fixture` opens the sample's `drafts` folder, which is **already a table** (it has a `matter.json`), so it never shows the "adopt" empty state. To see and test adopt you must open an **unmarked** folder: one with no `matter.json` and no marked subfolders. That empty state, with its "Adopt this folder as a table" button, only appears when `vault.tables.length === 0`. The result of adopt is provable without the GUI, because adopt only writes a `{}` marker and `loadPath` is what reads it back: ```sh T=$(mktemp -d)/notes && mkdir -p "$T" printf -- '---\ntitle: First note\n---\nhello\n' > "$T/note-a.md" printf -- '---\ntitle: Second note\n---\nworld\n' > "$T/note-b.md" bun run check "$T" # BEFORE: "0 tables" (unmarked, not a table) printf '{}' > "$T/matter.json" bun run check "$T" # AFTER: "1 untyped (1 table, 2 rows)" ``` That confirms the classification half. The GUI adds one thing on top: the root watcher must re-scan **live** when the button writes the marker. To verify that end to end: 1. Create a throwaway **unmarked** folder with a couple of notes (no `matter.json`): ```sh mkdir -p ~/matter-adopt-test printf -- '---\ntitle: First note\n---\nhello\n' > ~/matter-adopt-test/note-a.md printf -- '---\ntitle: Second note\n---\nworld\n' > ~/matter-adopt-test/note-b.md ``` 2. With the app running (`bun run dev` or `bun run dev:fixture`), open `~/matter-adopt-test` in the folder picker. It is unmarked with no marked children, so it shows **"Not a table yet"** with an **"Adopt this folder as a table"** button. 3. Click adopt. The app writes `matter.json` (`{}`); the root watcher sees the new top-level marker, re-scans, and the folder appears **live as one untyped table** with `note-a` and `note-b` as rows and a `title` column. No reload needed. 4. Confirm the marker landed: ```sh cat ~/matter-adopt-test/matter.json # -> {} ``` If the table does not appear after the click, run step 4 anyway to localize it: - `matter.json` is `{}` but the grid stays empty -> the **write works, the live re-scan did not fire**. Reopen the folder; if the table shows after reopen, that is a root-watcher bug (the non-recursive watch missed the marker create). - `matter.json` is absent and an error shows under the button -> the **write failed**; the message under the button is the cause (permissions, path). Empty-folder variant: an unmarked folder with **no** `.md` adopts the same way and shows a table with zero rows. Clean up with `rm -rf ~/matter-adopt-test`. --- ## File: apps/local-mail/src-tauri/README.md # local-mail desktop shell The smallest native wrapper around the Bun mail engine (`local-mail app`). It owns the window and the child engine's lifetime, and nothing else. Gmail OAuth, token refresh, the reconciler that is Gmail's only writer, the SQLite mirror, the durable `intent.db`, and the loopback API bearer all stay in Bun end to end (ADR-0116, ADR-0199). Rust never sees a Gmail token or the bearer. ## How it works 1. `setup()` spawns `bun src/bin.ts app` from the engine package dir. 2. The engine prints the loopback origin it serves (`http://127.0.0.1:`) to stdout, once, only after it is listening. That single line is both the discovery signal (which port) and the readiness signal (it is up). 3. The shell opens a `WebviewUrl::External` window at that origin. The engine's own response headers (frame-deny, `no-store`, CSP) are the security boundary; Tauri does not inject CSP into external pages, and the webview never calls `invoke()`, so the shell registers no commands and needs no remote-IPC allowlist. 4. Closing the window quits the app, which kills the engine (`RunEvent::Exit`). The engine never outlives the window: no background mail service (ADR-0116). This is the ADR-0084 sidecar recipe (spawn the engine, read the port from stdout, point a window at it), the pattern that spec named for the Tauri wrap. ## Run it (dev) Prerequisites: `bun` on PATH, and the UI built once so the engine has something to serve: ```sh bun run --cwd apps/local-mail/ui build ``` Then, from `apps/local-mail`: ```sh bun run app:desktop # == tauri dev: cargo runs the shell, which spawns the engine ``` A connected account is required (`bun run src/bin.ts connect` once). The shell opens no window until the engine reports its origin, so a blank moment at launch is the engine starting, not a hang. Watch the terminal for `[sync ...]` and the `listening on http://127.0.0.1:...` hint. Only `bun` is called: no port is pinned, so this never collides with a separate `local-mail app` you already have open. That instance keeps the per-account reconciler lock, so it is the only one delivering to Gmail; this one serves reads and can still record triage assertions, which the lock holder delivers on its next wake (ADR-0199). Both run under their own ephemeral origins. ## Build a desktop bundle Set the public Google Desktop client identity that belongs to the distribution, then build: ```sh LOCAL_MAIL_DISTRIBUTION_GMAIL_CLIENT_ID=... \ LOCAL_MAIL_DISTRIBUTION_GMAIL_CLIENT_SECRET=... \ bun run desktop:build ``` `beforeBuildCommand` runs `bun run build:desktop` first, which builds the SPA (`ui/dist`) and compiles the engine sidecar (`src-tauri/binaries/local-mail-engine-`, a `bun build --compile` binary, ~60MB). Tauri then bundles both into `src-tauri/target/release/bundle/macos/Local Mail.app`. Cargo compiles that pair into the native shell. At launch, the shell passes it to the Bun engine as public application configuration; OAuth, tokens, mail, and the SQLite mirror still never transit Rust. An inherited `GMAIL_CLIENT_ID` / `GMAIL_CLIENT_SECRET` pair remains the machine-wide override and wins atomically. A source build without a compiled distribution identity must provide that override. How the packaged app differs from `tauri dev`: - **The engine is a compiled sidecar, not `bun src/bin.ts`.** A packaged app has no repo beside it and `bun` may be absent from the user's PATH (ADR-0116 keeps the whole mail engine in Bun, so it ships as one binary). `bundle.externalBin` registers `binaries/local-mail-engine`; Tauri strips the target-triple suffix and copies it into `Contents/MacOS/local-mail-engine`. `engine_command` branches on `cfg!(debug_assertions)`: dev spawns `bun src/bin.ts app`; release resolves `current_exe().parent()/local-mail-engine`. - **The SPA is a bundled resource, not a source sibling.** A `bun build --compile` binary's `import.meta.dir` is a virtual path with no `ui/dist` beside it, so the SPA ships via `bundle.resources` (`../ui/dist` -> `Contents/Resources/ui-dist`) and the shell passes `LOCAL_MAIL_UI_DIST` (resolved from `resource_dir()`) when it spawns the sidecar. The engine's serving code is identical to dev; only the root path differs. Same `app` entrypoint, same loopback contract. Rust still owns only the window and the child's lifetime. The sidecar is the same Bun engine; no Gmail token, sync, mirror, or bearer transits Rust (ADR-0116). ## Signing and notarization (the one deferred item) The produced `.app` is **unsigned** (ad-hoc). That is fine for a locally built app run on the same machine: it is not quarantined, so Gatekeeper does not block it. It is not distributable as-is; a downloaded copy would be quarantined and refused without a right-click-open. To sign and notarize, no code changes are needed, only credentials and config: 1. An Apple Developer ID Application certificate in the login keychain. 2. `bundle.macOS.signingIdentity` set (or the `APPLE_SIGNING_IDENTITY` env var), plus `bundle.macOS.hardenedRuntime: true` and an entitlements file if any hardened-runtime exceptions are needed. 3. Notarization credentials (`APPLE_ID`, `APPLE_PASSWORD`, `APPLE_TEAM_ID`) so `tauri build` can staple the notarization ticket. Blocker, not a design gap: this machine has no Apple Developer credentials configured, so the wave ships an unsigned local bundle and leaves the signing config to whoever holds the certificate. Crib `apps/honeycrisp` for the hardened-runtime block. ## Deferred niceties (not correctness gates) - **Graceful engine shutdown.** Exit kills the engine with SIGKILL, which is safe by design (the reconciler lock is a kernel-released fcntl lock; presence is stale-safe). A SIGINT/SIGTERM path (so the engine clears its presence file and releases the lock cleanly) is a nice-to-have. Undelivered work is not at risk either way: an assertion is committed to `intent.db` when the act returns, and a kill between a Gmail call and its retirement leaves the row, so the next pass redelivers it and Gmail no-ops the repeat (ADR-0198, ADR-0199). - **Orphan on hard kill.** If the app is SIGKILLed, the engine is reparented and keeps running (holding the reconciler lock). Harmless and self-healing (the lock is crash-safe; the next launch or a manual `kill` clears it), but worth a parent-death watchdog if it ever bites. --- ## File: apps/local-mail/test-support/README.md # Local Mail write-path test support Local-only harness for exercising Local Mail's triage path (archive, undo, label, mark read/unread, star, trash) against a **mock Gmail backend** and a **throwaway copy** of your data dir, so you can verify triage UX without ever touching real Gmail or your real mirror and intent store. A triage act is a local write and the reconciler is the only thing that reaches Gmail (ADR-0199), so the harness covers both halves: the act, which must be visible to the next read immediately, and the pass that delivers it. This is developer tooling, not `bun test`. The smoke needs a real connected mirror to copy from; `check-gmail-discovery.ts` makes a live Google call. Neither is hermetic, so both stay out of the offline suite. ## Gmail API drift check `check-gmail-discovery.ts` fetches Gmail's Discovery document and asserts that every method and schema field `src/gmail-client.ts` / `src/schema.ts` rely on is still present and still the type we expect. Our schemas are deliberately partial and permissive (they tolerate unknown Gmail fields, and every read field is optional, so a removed field passes `Value.Check` and reaches a reader as `undefined`), which is exactly what makes Gmail *removing*, *moving*, or *retyping* something we depend on invisible until a live sync misbehaves. The schema-side contract is not re-listed: the check walks the actual `schema.ts` TypeBox objects (JSON Schema at runtime), so `schema.ts` stays the single source of the fields we read. The only hand-maintained pieces are the small set of methods we call (the client builds those paths as string templates, nothing to derive) and the root-schema name map. ```sh bun run --cwd apps/local-mail check:gmail-drift ``` It runs weekly (and on demand) via `.github/workflows/local-mail.gmail-drift.yml`, not per-PR: it is a network call and drift is slow-moving. Exits non-zero listing each drift. ## Safety model Four independent guarantees keep this from touching anything real: 1. **A throwaway copy, never the real data dir.** `setup-copy.sh` copies the real app dir (`~/Library/Application Support/so.epicenter/apps/local-mail`) into `LM_TEST_DIR/apps/local-mail` (default root `/tmp/local-mail-harness`) and points `local-mail app` at that root via `EPICENTER_DATA_DIR`. The copied `intent.db` is dropped: it holds triage the real account still owes Gmail, and the copy starts owing nothing. 2. **Forged credentials, so no Google contact.** The copy's `credentials.json` is rewritten with a dummy access token whose expiry is the year 2099. The token manager only refreshes near expiry (`src/token-manager.ts`), so the app reuses the dummy bearer forever and never calls Google's token endpoint. 3. **A local mock, not `gmail.googleapis.com`.** `LOCAL_MAIL_GMAIL_API_BASE` points the client at `mock-gmail.ts` on `127.0.0.1`. The mock services only `messages.modify` (logged) and a no-op `history.list`; **every other route returns a non-retryable 403**, which the pull phase treats as a hard failure rather than a signal to run a FULL pull. So the mock can never wipe even the copy. 4. **A fingerprint proof.** `fingerprint.sh` hashes the real data dir's durable files (`credentials.json`, every `mail.v.db` including predecessors, and every `intent.db`); capture it before and after and diff to confirm nothing real changed. `intent.db` matters most: the mirror is re-pullable from Gmail and undelivered triage is not (ADR-0198). ## Files | file | what it is | |--------------------|------------| | `mock-gmail.ts` | Mock Gmail REST server. Reads the copy's SQLite to know current labels, applies the modify, logs it, 403s everything else. | | `setup-copy.sh` | Copies the real app dir into `LM_TEST_DIR/apps/local-mail`, drops the copied intent store and lock, and forges dummy credentials. | | `fingerprint.sh` | Hashes the real app dir's durable state, for the before/after safety proof. | | `boot.ts` | Shared boot used by `smoke.ts` (and any manual session): stands up copy + mock + the app on ephemeral ports and hands back the launch coordinates. The one owner of the safety-critical wiring. | | `smoke.ts` | Headless one-shot: records one real act through `/api/accounts/:account/messages/assert`, proves the next read already reflects it, reconciles, asserts the delivery hit the mock, and asserts the real data dir is unchanged. | | `check-gmail-discovery.ts` | Gmail API drift check: fetches the live Discovery doc and asserts the methods (hand-listed) + schema fields (walked from `schema.ts`) the client relies on are still present and correctly typed. | Runtime artifacts (the copy, the modify log, server logs) live under `LM_TEST_DIR`, never inside the repo. ## Automated smoke (no browser) Proves the full server → mock write path end to end and tears itself down: ```sh bun run apps/local-mail/test-support/smoke.ts ``` On success it prints `SMOKE PASS`, the mock log line for the delivery, and confirms the real data dir's fingerprint is unchanged. Exits non-zero on any failure. ## Manual write-UX check (browser) The affordances the API smoke can't assert (undo toast, keyboard triage, the pending chip, the shortcuts overlay) are verified by hand. `boot.ts` exports `bootHarness()`, which stands up the same safe stack (copy + mock + the app) and returns a launch URL; call it from a scratch script or the REPL, open the URL, and poke the SPA. Build the SPA first or the page is blank: ```sh bun run --cwd apps/local-mail/ui build ``` Watch the deliveries land: ```sh tail -f /tmp/local-mail-harness/modify-log.jsonl ``` ## Read-only smoke against your real mirror No copy, no writes; a dead Gmail base no-ops the reconcile loop and every action button is disabled: ```sh bun run --cwd apps/local-mail/ui build LOCAL_MAIL_READ_ONLY=1 LOCAL_MAIL_GMAIL_API_BASE=http://127.0.0.1:9 \ LOCAL_MAIL_PORT=4181 bun run apps/local-mail/src/bin.ts app ``` ## Proving the real mirror was untouched ```sh bash apps/local-mail/test-support/fingerprint.sh > /tmp/lm-before.txt # ...run smoke.ts or a manual bootHarness session... diff /tmp/lm-before.txt <(bash apps/local-mail/test-support/fingerprint.sh) # must be empty ``` `smoke.ts` does this automatically. Note the fingerprint covers only durable files; the read-only smoke above legitimately syncs the real mirror, so run that one separately from a fingerprint window. ## Environment knobs | var | default | meaning | |-----------------------|---------|---------| | `LM_TEST_DIR` | `/tmp/local-mail-harness` | the throwaway Epicenter data root the copy and the logs live under; `EPICENTER_DATA_DIR` points the app at it | | `LOCAL_MAIL_REAL_DIR` | the real app dir under the macOS Epicenter data root | the app dir to copy/fingerprint | | `LOCAL_MAIL_ACCOUNT` | the sole connected account | which account to forge (required if you have more than one) | --- ## File: apps/local-mail/README.md # local-mail Headless Gmail reader and triage ledger for local tools and agents. It pulls a Gmail account into a private SQLite mirror with full pulls plus incremental `history.list` polling, records triage acts as durable local assertions, and delivers them back to Gmail from one writer per account. CLI queries, a stdio MCP server, and a loopback SPA all read the same two files. Two files per account carry that, and the difference between them is the whole design: | File | Holds | If you lose it | | -------------------- | ------------------------------------ | -------------------------- | | `mail.v.db` | Gmail's facts, mirrored | Nothing but quota; re-pull | | `intent.db` | What the user asked for, undelivered | Real work, unrecoverably | Design authority lives in ADRs first, then the current code: `docs/adr/0081-*.md` (a per-device grant and mirror), `docs/adr/0082-*.md` (push-free `history.list` polling), `docs/adr/0098-*.md` (state round-trips through Gmail), `docs/adr/0116-*.md` (desktop-first, one Bun engine), `docs/adr/0188-*.md` (who owns the Google application identity, and the device-only credential boundary), `docs/adr/0196-*.md` (the mirror is a reader and one full message fetch is its entire budget), `docs/adr/0197-*.md` (the artifact is named by its corpus version), `docs/adr/0198-*.md` (what a durable write is), and `docs/adr/0199-*.md` (who delivers it). The mirror is Gmail-owned cache data. Human-meaningful mail state must round-trip through Gmail, not a local-only table; an undelivered assertion is not an exception to that, because it names a Gmail label id and exists in order to stop existing. ## The write model There is one model, and this is it (`docs/adr/0198-*.md`, `docs/adr/0199-*.md`). **A triage act writes a durable assertion, not a Gmail call.** Acting on a message records one row per touched label in a durable `intent.db`, a per-account partial map from (Gmail message id, Gmail label id) to want or do-not-want. Ids only, never label names: Gmail's label ids are immutable, its names are editable display strings. The map holds opinions only about labels the user touched, so it never claims a whole desired label set, and it is keyed so that re-asserting a pair overwrites it. Archive, then un-archive, then archive is one row. **One reconciler per account is the only thing that writes to Gmail.** It drains the account's assertions, retires each one Gmail confirms, then pulls Gmail's facts. Running a pass requires the per-account lock as a value, not as a promise the caller made: `reconcileAccount` takes the capability `acquireReconcileLock` mints, so writing and pulling cannot interleave and a second writer cannot be written by accident. It wakes on app start, the poll interval, a coalesced local write, and an explicit reconcile, and never after the app closes. **Reversal is a new assertion.** Undo asserts the opposite want on the same key. Nothing retires an assertion by comparing it to the mirror: the mirror lags Gmail and can be incomplete, so a "the mirror already agrees" rule would drop real writes on stale evidence. Only provider confirmation retires an assertion, and only the reconciler does it. **Undelivered work is visible and discardable.** Status reports the undelivered count, the age of the oldest, and, in the running app, the current reconcile failure as one message. `local-mail discard --all` abandons every undelivered assertion so none is sent; it cannot un-send one already delivered. There is no attempt counter, no per-row error, no retry schedule, no dead-letter tier, and nothing expires. Retries are bounded by a human, which is what discard is for. **Refused, deliberately:** a generic queue or event log, a payload or operation table, a provider-independent request abstraction, standing policy over a thread or query, cross-account atomicity, `messages.batchModify`, background delivery after the app closes, and browser-memory optimism beside the durable ledger. Each one is a decision with a record; the full list, with its reasons, is under [Refused](#refused) at the end of this file. ## Shape - Runtime: Bun. `bun:sqlite` stores the mirror, built-in `fetch` calls Gmail, `oauth4webapi` handles OAuth. - Local Mail's one directory is `/apps/local-mail`, partitioned by account under `accounts/`. The app computes no OS application-data path of its own: Epicenter owns exactly one root and `EPICENTER_DATA_DIR` is the only override, so a CLI run from a terminal and the desktop host operate on the same mailbox (ADR-0201). The directory is a place, not an inter-app API: no other app receives a path or a SQLite handle into it, and `query`'s read-only SQL is this app's surface over the file this app wrote. - One SQLite artifact per connected account, under `/accounts//`, named `mail.v.db` after `MIRROR_VERSION` in `src/db.ts`: the version of the corpus contract this build stores, not the app's release version. `src/db-file.ts` owns the naming, the two opening modes, the version listing, and pattern-scoped `deleteOlderVersions`; the rest of the app owns the version constant, DDL, ingestion, cursors, locking, file permissions, readiness, and deletion timing (ADR-0247). Opening is non-destructive and never falls back to a lower version: a bump is a new filename, not a migration, so nothing is dropped or unlinked on open and the predecessor is retained until something reclaims it. There is no schema version stamped in the file. See ADR-0197. Bumping `MIRROR_VERSION` costs one full re-pull of the mailbox, and until that re-pull finishes `status` reports the artifact as `building`, never `ready`. The refresh token lives in a separate `0600 credentials.json` at the app-dir root, never inside the mirror db. - One durable `intent.db` per account, beside the mirror at `/accounts//intent.db`. It holds the account's undelivered label assertions and is the only irreplaceable local state in the app. Deliberately not inside the mirror: the mirror is version-named and reclaimable, so a corpus bump or a `reclaimPredecessors()` would silently discard pending work. It is outside ADR-0197's `.v.db` filename grammar, so reclamation cannot name it and neither opening mode touches it. - The per-account layout is therefore: ``` /apps/local-mail/credentials.json 0600 refresh tokens /apps/local-mail/provider.json 0600 cached client-identity override /apps/local-mail/accounts// mail.v.db 0600 Gmail facts, disposable, re-pullable intent.db 0600 undelivered assertions, durable ``` The account segment is the address Gmail reports, which Google does not promise to keep: a renamed Workspace account would strand its partition and start a second one. Adopting the OpenID `sub` claim is the fix, and it costs one re-consent per account (ADR-0201); it has not shipped. - The app directory and every directory below it are `0700`. The mirror artifact, its `-wal` and `-shm` sidecars, `intent.db`, and `credentials.json` are `0600`. - Tables: `messages`, `labels`, and `_meta`. Thread facts are derived from live messages instead of stored in a separate `threads` table. - `messages.resource` holds the parsed `messages.get(format=full)` payload verbatim. It is deliberately not called `raw`: `format=raw` is Gmail's name for the base64url RFC 5322 blob this app never fetches. Everything SQLite can project from the resource is a generated column; the only stored derivations are `subject`, `sender`, and `body_text`, which exist because triage pushes its filter and sort into SQL and `json_extract` cannot reach a header array or a base64url MIME tree. Everything else is derived at read time. See ADR-0196. - `messages.body_text` is decoded at ingest from `text/plain` MIME parts, with stripped `text/html` as a fallback. That makes SQL and MCP useful for body questions without adding FTS yet. - The mirror is a disposable offline reader, not an archive. One `messages.get(format=full)` is the entire per-message budget: no `format=raw`, no `messages.attachments.get`, no attachment or media bytes on disk in any form. Attachments are metadata only, enough to say "this message has a 42 KB PDF" and not enough to open it; opening it is Gmail's job. When Gmail returns a body part as an `attachmentId` instead of inline data, the reader says it has no offline body for that message rather than spending a second call. See ADR-0196. - The app detail pane can render formatted email from the stored Gmail payload, but only behind the SPA sanitizer boundary. DOMPurify strips executable markup and remote assets before the single `{@html}` sink renders the body on a light email canvas; plain text stays available as the fallback view. ## Commands Connect once. A packaged distribution can supply its Google OAuth Desktop client identity through the native shell. The official Epicenter client is not provisioned in this repository yet, so source builds remain bring-your-own: set `GMAIL_CLIENT_ID` / `GMAIL_CLIENT_SECRET`, then run the connect flow. ```sh bun run src/bin.ts connect ``` One client identity serves every Gmail account connected on this machine. An explicit `GMAIL_*` pair is the machine-wide override and wins over the identity supplied by a packaged distribution. Because Google binds a refresh token to the client that minted it, changing the effective pair means reconnecting *every* account here; a stored grant is never reinterpreted under a new client, so a mismatch fails loudly on the next refresh instead of surfacing as Google's opaque `invalid_grant`. The client id and secret are public application configuration, not user secrets: the Gmail access and refresh tokens are the secrets, and they never leave this device. Epicenter Cloud and self-hosted Epicenter instances are not in the Gmail path at all, so which instance you select has no effect on any of this. Provide the pair however you keep local secrets for the first connect: an export, a local `.env`, or a secrets manager such as `infisical run --path=/apps/local-mail -- bun run src/bin.ts connect`. On the first successful connect or refresh, Local Mail caches an explicit override to a 0600 `/provider.json` sibling to `credentials.json`. The distribution identity is never copied into that user-owned file. Every later run and every other git worktree on this machine reads the override from there, so no per-worktree config is needed. See [`.env.example`](.env.example) for the canonical override names. Local Mail requests `gmail.modify` so label changes can round-trip through Gmail. Trash and untrash are in scope and built: the app UI moves a message to Trash and undoes it, delivered through Gmail's own `messages.trash` / `messages.untrash` verbs rather than as a label delta, though it is stored as an ordinary `TRASH` assertion like any other label. Although Google grants send at the same OAuth layer, Local Mail exposes no send, reply, compose, or drafts today, and permanent delete is refused outright: `messages.delete` is deliberately never wired. The difference matters when reading the two lists at the end of this file. Send and drafts are unbuilt; permanent delete is a decision. Headless bootstrap is still available. The refresh token is redeemed immediately (one refresh grant plus a profile read), so a dead token fails here rather than on the first sync, and the account email comes from the Gmail profile instead of being typed: ```sh bun run src/bin.ts seed-token ``` Run a reconcile pass: drain this account's assertions, then pull Gmail's facts. ```sh bun run src/bin.ts reconcile --full bun run src/bin.ts reconcile --watch ``` There is no separate sync verb, because a pass is drain-then-pull and the pull is its second half. If the open app already owns this account's reconciler, a headless pass yields to it rather than becoming a second writer. Check connection and mirror state: ```sh bun run src/bin.ts status ``` Query the mirror: ```sh bun run src/bin.ts query "SELECT subject, sender FROM messages ORDER BY internal_date DESC LIMIT 10" ``` Triage messages. Each verb records label assertions for the ids you name; the account's reconciler delivers them. Output is human-readable; add `--json` for the typed outcome MCP and the `app` HTTP API share. ```sh bun run src/bin.ts archive bun run src/bin.ts unarchive bun run src/bin.ts mark-read bun run src/bin.ts mark-unread bun run src/bin.ts trash bun run src/bin.ts untrash bun run src/bin.ts label --add Work --remove Promotions ``` `archive`, `mark-read`, and friends are the triage vocabulary; `label` is the transparent primitive they desugar to (`archive` is `label --remove INBOX`). They are spellings, not separate write paths: archive, read, star, trash, and a custom label are one assertion with a different label id, so every verb here reaches the same single assert. `LOCAL_MAIL_READ_ONLY` refuses every write while leaving `query`/`status` and the pull half of a pass available. An assertion is durable the moment the verb returns, so a triage act survives an offline machine, a killed process, and a reboot. It is not a Gmail write yet: `status` reports how many assertions are undelivered and how old the oldest is, and discard is how you abandon them before they are sent. ```sh bun run src/bin.ts discard --all ``` `--all` is mandatory and there is no partial form. Discard is the only thing in the system that drops a recorded change without delivering it, and it abandons rather than recalls: an assertion the reconciler already delivered is Gmail's, and getting back is a new act like any other. It takes the account's reconcile lock, so it refuses while the app is open or a pass is running, and exits nonzero saying so. That is what makes its report true: a reconciler snapshots what is pending when its drain begins, so a discard landing mid-pass would delete rows that pass is still about to send, and the message would claim the change was abandoned while Gmail was hearing the opposite. Unlike a busy `reconcile`, which exits 0 because the owner does that work for you, a busy discard is work that did not happen. Serve the triage UI and its API from one loopback process: ```sh bun run src/bin.ts app ``` `app` is the desktop runtime host: it owns this account's reconciler and serves the triage SPA (`ui/`) plus a same-origin `/api` on `127.0.0.1`, then prints the origin to open (`http://127.0.0.1:PORT`). It launches no browser; opening the window is the host's job (a terminal today, Tauri later), not the engine's. Security: every request is Host-checked first (the DNS-rebinding kill switch); the web UI carries a per-launch local API bearer that the host injects into the served HTML as `window.__LOCAL_MAIL__ = { origin, bearer }` before the SPA boots (no URL fragment, no session-exchange endpoint, no sessionStorage), and that HTML is served `no-store` and frame-denied so a rotated bearer is never cached and a cross-origin page cannot frame the auto-authenticated SPA. The bearer is a loopback credential, never a Gmail token. In app mode the host loads every connected account and serves them under one origin: `GET /api/accounts` lists the loaded accounts, and reads/writes live under `/api/accounts/:account/*`. The write surface is two routes over the same core the CLI verbs and MCP tools use: | Route | Does | | --------------------------------------------- | --------------------------------------------------- | | `POST /api/accounts/:account/messages/assert` | Record label assertions for concrete ids and return | | `POST /api/accounts/:account/reconcile` | Wake the reconciler now: drain, then pull | That is the whole write door, and the invariant lives there: a request asserts or it wakes the reconciler, and nothing else reaches Gmail. `GET /status` carries the aggregate reading (undelivered count, age of the oldest, and this host's current reconcile failure as one message) and never a per-assertion status, because there is not one. There is deliberately no route that enumerates individual assertions: the vocabulary the product has for pending work is a count and an age, and abandoning it is the CLI's `discard --all`, which is the whole ledger rather than a selection. Archive, read, star, trash, and custom labels all desugar to assertions client-side; trash is an ordinary `TRASH` assertion here and routes to Gmail's `messages.trash` / `untrash` only at delivery. The old direct-write vocabulary (`POST .../messages/modify`, `POST .../messages/trash`, `POST .../sync`, and the `ModifyMessageLabelsOutcome` shape with its `folded` and `aborted` fields) is deleted, not deprecated: it described a request that talked to Gmail, and no request does that any more. `LOCAL_MAIL_READ_ONLY` disables writes end to end; `--port ` pins the server port (`LOCAL_MAIL_PORT` is the env fallback). Bulk and thread actions enumerate concrete message ids from the mirror at the moment you act. A message that arrives in that thread afterward is untouched, because nothing durable refers to the thread. Develop the UI against a running `app`: ```sh LOCAL_MAIL_PORT=4177 bun run src/bin.ts app # serves /api + writes the presence file bun run --cwd ui dev # Vite proxies /api to the host and injects its bearer from the presence file ``` The dev server reads the host's `0600` presence file (`runtime.json`) for the per-launch bearer and injects it on each proxied `/api` request (SvelteKit's dev HTML pipeline bypasses Vite's HTML transform, so the prod `window.__LOCAL_MAIL__` injection is a proxy-side header in dev). No token is typed by a human. Start the host first; a host restart rotates the bearer, so restart the dev server to pick it up. Pin `LOCAL_MAIL_PORT` if the host uses an ephemeral port. Serve tools to an MCP host: ```sh bun run src/bin.ts mcp ``` When more than one account is connected, `local-mail app` serves all of them under one origin with an account switcher. Set `LOCAL_MAIL_ACCOUNT` only for headless `reconcile`, `query`, and `mcp`, which operate on one account per process. Tools: - `query`: read-only SQL over the mirror, capped at 1000 returned rows. The read connection attaches the account's `intent.db` read-only and defines an `effective_labels(message_id, label_id)` view over both, so a query can see Gmail's stored labels and the effective set that overlays this machine's undelivered assertions on them. Join the view; `messages.label_ids` is Gmail's last word, not what the user has already asked for. - `status`: account, cursor, row counts, and the write ledger: undelivered count and the age of the oldest. Aggregate only, by design. The current reconcile failure is not here, because only a running reconciler has one and this tool serves a process that is not one. - `reconcile`: one pass, drain then pull. Yields if another process owns this account's reconciler. - `assert_labels`: record label assertions on messages by id, adding or removing labels by id or exact name. It returns what it stored, not what Gmail did, because delivery has not happened yet; an agent that needs to know its write landed calls `reconcile` or reads `status` afterward rather than assuming. Unlisted under `LOCAL_MAIL_READ_ONLY`. The CLI triage verbs above are the human-facing form of this one tool. ## Config - `GMAIL_CLIENT_ID` / `GMAIL_CLIENT_SECRET`: the machine-wide Google OAuth Desktop client override. The resolver reads the pair atomically from the environment first, then `/provider.json`. A packaged distribution identity is the fallback and is never persisted as an override. Missing credentials in a source build fail loudly naming the exact variables. Changing the effective client requires reconnecting every account on this machine. - `LOCAL_MAIL_ACCOUNT`: optional account override for `reconcile`, `query`, and `mcp`. Required only when more than one account is connected. - `EPICENTER_DATA_DIR`: the one Epicenter data root. This app lives at `/apps/local-mail`; there is no per-app override. - `LOCAL_MAIL_TOKEN_FILE`: token file override. - `LOCAL_MAIL_GMAIL_API_BASE`: test plumbing only; points the Gmail client at a mock server in the MCP subprocess test. - `LOCAL_MAIL_PORT`: fallback for pinning the `app` server port; prefer `--port ` for normal use. Also names the port the Vite dev proxy targets when the host uses an ephemeral port. ## The pre-Epicenter-root layout Local Mail used to keep its data in its own application-data directory (`~/Library/Application Support/local-mail` on macOS). That directory is not read, moved, or referenced by any code here. Connect the account again and the mirror re-pulls; the old directory is inert bytes you can delete whenever you notice it. ## Testing Run from this package: ```sh bun test bun run typecheck ``` The tests use real `bun:sqlite` temp files for DB and intent-store behavior, a fake `GmailClient` for delivery and sync folding, mock HTTP endpoints for OAuth, and a real MCP stdio subprocess for the agent-facing protocol surface. ## Not built yet - Remote image loading and Gmail-perfect HTML fidelity. Formatted bodies render as sanitized inline HTML on a light canvas; remote assets stay stripped, and there is no show-images proxy yet. - Signed distribution. The Tauri desktop shell (`src-tauri/`, run with `bun run app:desktop`) spawns this engine, reads the origin it prints, and opens a `WebviewUrl::External` window at it, owning only the window and the engine's lifetime (Rust never touches Gmail tokens, mail data, or the bearer). `bun run desktop:build` produces a local unsigned macOS `.app` with the compiled Bun sidecar and bundled SPA resources. The shell can compile in the distribution's public Google client identity, but no official Epicenter identity is provisioned in this repository. Signing, notarization, and the official Google client still need release configuration, as described in `src-tauri/README.md`. - Discard from the SPA. `local-mail discard --all` exists; the app shows the pending count and age but offers no button for it yet. - Account disconnect. There is no disconnect verb. When one is built it must report the account's undelivered count first and discard on confirmation, and a reauthorization is not a disconnect (ADR-0199). - Enumeration surfaces. Thread and bulk acts, and asserting `UNREAD` off when a message is opened. The semantics are already fixed: enumerate concrete message ids from the mirror at the moment the user acts, never at delivery time. - Gmail-backed drafts, send, reply, and compose. Allowed in principle only for drafts, which round-trip (ADR-0098); none is built, and shipping send would widen the OAuth grant past `gmail.modify` (ADR-0188). - FTS5. `LIKE` over `body_text` is enough for the current mirror size. ## Refused These are decisions, not gaps. Each has a record, and adding one back is an ADR rather than a pull request. - **Local-only mail state**: snooze, send-later, and a local tag or read flag that never reaches Gmail (ADR-0098). A pending assertion is not an exception: it names a Gmail label id and exists in order to stop existing. - **Permanent delete and spam reporting.** `messages.delete` is never wired, and a deferred, discardable, silently-retried intent is the wrong shape for an irreversible act (ADR-0198). - **A locally held draft as durable intent.** A Gmail-backed draft is a different thing and is merely unbuilt; a draft that lives only on this device is a second durable write shape (ADR-0198). - **Attachment and media bytes on disk, in any form.** One `messages.get(format=full)` is the entire per-message budget (ADR-0196). - **Thread-level modify.** `threads.modify` applies to the thread as Gmail sees it at delivery, which would silently include messages that arrived after the user acted. Triage is message-level for that reason, not for lack of effort (ADR-0199). - **`messages.batchModify`.** Its response body is empty, so it proves nothing about which assertions landed. The reopen condition is named and specific in ADR-0199; the quota saving alone does not meet it. - **A generic queue, event log, or provider-independent operation abstraction**, and any per-assertion attempt counter, error column, retry schedule, or dead-letter tier (ADR-0198, ADR-0199). - **Background delivery after the app closes**, and any always-on warm-mirror daemon (ADR-0116, ADR-0199). - **Push/Pub/Sub**, and any LAN or remote exposure: the server binds `127.0.0.1` (ADR-0082, ADR-0116). --- ## File: apps/local-books/README.md # local-books Your QuickBooks books, as a private SQLite database on your own computer that you (or any AI agent) can grill. `local-books` keeps a faithful local copy of a QuickBooks Online company and keeps it current. Once it is synced, you ask questions with plain SQL ("who owes me money?", "where did the money go last quarter?", "what still needs categorizing?"), run live financial statements, and push a category fix back to QuickBooks. Your financial data stays on your machine; nothing goes to a cloud service. ## See it work in 10 seconds (no QuickBooks account needed) ```sh local-books demo ``` This builds a small sample company in a local file and immediately answers a few real questions against it, then prints the exact `local-books query` commands so you can poke at it yourself. Everything below is how you point it at your own books. ## How it works `sync` pulls your whole company into one SQLite file: one table per record type (`invoices`, `customers`, `bills`, `vendors`, `purchases`, `accounts`, ...), each row carrying the verbatim QuickBooks JSON in `raw` plus a few extracted columns for easy filtering and joins. It stays current with incremental Change Data Capture, so a re-sync only fetches what changed. That file is the product. You grill it three ways: - **`local-books query ""`** runs read-only SQL over the local copy. Writes are rejected at the connection, so this is always safe. - **`local-books report `** runs a live computed statement from QuickBooks (`ProfitAndLoss`, `BalanceSheet`, `CashFlow`, `AgedReceivables`, `AgedPayables`, `TrialBalance`). These are read live because QuickBooks owns the math, and there is no change feed to keep a cached copy honest. - **`local-books recategorize --to `** moves an expense to a different account in QuickBooks, then updates the local copy. It is the one command that writes; everything else only reads. For natural-language questions, point an AI coding agent you already use (Claude Code, Codex) at the file and just ask: it writes the SQL for you. See "Grill it with an AI agent" below. ## Connect your company You need an Intuit app for the API keys (this is a one-time developer step Intuit requires; your books are still yours). At https://developer.intuit.com, create an app, open **Keys & credentials**, and register `http://localhost:8765/callback` as a redirect URI. Intuit issues two key sets that are not interchangeable: **Development** keys connect sandbox (test) companies; **Production** keys connect your real company and are issued only after Intuit's go-live review. Start with Development and a sandbox company. Bring your own keys. The name carries the environment, so the sandbox (Development) keyset is `QB_SANDBOX_*` and production is `QB_PRODUCTION_*`; `--qb-env` picks which one is read (see [`.env.example`](.env.example) and ADR-0108). Start with sandbox: ```sh export QB_SANDBOX_CLIENT_ID=... export QB_SANDBOX_CLIENT_SECRET=... local-books auth # opens a browser; log into your sandbox company (--qb-env sandbox is the default) local-books sync --full # build the local copy local-books status # see what is connected and synced ``` `auth` captures the company id from the sign-in, so you never pass it by hand. From here on, `sync` refreshes everything and never needs the browser again. ### Grill it ```sh local-books query "SELECT display_name, balance FROM customers WHERE balance > 0 ORDER BY balance DESC" local-books report ProfitAndLoss --start 2026-01-01 --end 2026-03-31 local-books recategorize Purchase 7011 --to 61 --to-name "Software & Subscriptions" ``` The line-level category lives in `raw`, so reach it with SQLite JSON, e.g.: ```sh local-books query "SELECT json_extract(line.value, '\$.AccountBasedExpenseLineDetail.AccountRef.name') AS category, SUM(json_extract(line.value, '\$.Amount')) AS spent FROM purchases p, json_each(p.raw, '\$.Line') line WHERE p.deleted = 0 GROUP BY category ORDER BY spent DESC" ``` ### Grill it with an AI agent The local copy is just a SQLite file, so any agent that can run SQL can answer questions about your finances without your data leaving the machine. Print its path and point your agent at it: ```sh local-books status # prints the exact path of the local copy ``` Then open Claude Code or Codex in that folder and ask in plain English. To reach it from your phone or another machine, expose your box over a private mesh like [Tailscale](https://tailscale.com) and drive the agent there; the books still never leave the box. Set `LOCAL_BOOKS_READ_ONLY=1` to disable `recategorize` while you let an agent explore (both reads stay available). ### Hand it to an agent over MCP `local-books mcp` runs a [Model Context Protocol](https://modelcontextprotocol.io) server over stdio, so a coding agent drives the books through real tools instead of hand-written SQL: it gets `query`, `status`, `report`, `sync`, and, unless `LOCAL_BOOKS_READ_ONLY` is set, `recategorize`. The agent spawns the server as a local subprocess that reads the same SQLite file, so the data still never leaves the machine, and the financial data never touches a network (this is why the exposure is local stdio, not a hosted server). Register it with Claude Code, passing the company through the environment (and the Epicenter data root only if yours is not the default): ```sh claude mcp add local-books \ --env LOCAL_BOOKS_QB_REALM= \ -- local-books mcp ``` Add `--env LOCAL_BOOKS_READ_ONLY=1` to hand over the reads without the one write. Then ask in plain English: "what were my three biggest expenses last month?" runs a `query`, and "re-sync my books" runs `sync`. From the monorepo (no installed binary), the launch command is `bun run /abs/path/apps/local-books/src/bin.ts mcp`. Inspect the server without an agent using the MCP Inspector: ```sh npx @modelcontextprotocol/inspector local-books mcp ``` ## Open it in your browser ```sh local-books app ``` `app` keeps the mirror fresh and serves a dense browser UI plus a same-origin `/api` on `127.0.0.1`, then prints `http://127.0.0.1:PORT/#token=...` and opens it. The tab is the app: browse every record type with counts, page a table, inspect a row's extracted fields and raw QuickBooks JSON, run read-only SQL, pull a live report, and (unless read-only) recategorize an expense. The same `src/books/*` cores back the CLI, the MCP tools, and this UI, so all three agree. Security is the loopback shell `local-mail app` established: the server binds `127.0.0.1` only; every request is Host-checked first (the DNS-rebinding kill switch); a single-use bootstrap token rides in the URL fragment and is exchanged once at `POST /api/session` for a per-launch session bearer (kept in sessionStorage); every other `/api` call carries that bearer. `LOCAL_BOOKS_READ_ONLY` disables the one QuickBooks write (`recategorize`) end to end and hides its UI. `--no-open` prints the URL without launching a browser; `--port ` pins the server port; `LOCAL_BOOKS_NO_OPEN=1` and `LOCAL_BOOKS_PORT` are env fallbacks. Pick the company with `--realm` when more than one is authenticated. Develop the UI against a running `app`: ```sh LOCAL_BOOKS_DEV=1 LOCAL_BOOKS_TOKEN=devtoken LOCAL_BOOKS_PORT=4178 local-books app LOCAL_BOOKS_TOKEN=devtoken bun run --cwd ui dev # same token: Vite proxies /api to app, injecting this bearer ``` The bearer gate is never disabled in any mode; dev just moves the credential server-side into the Vite proxy so no token reaches the browser. ## Keep it fresh ```sh local-books sync # refresh now; full vs incremental is chosen automatically local-books sync --interval 30m # refresh now, then every 30 minutes until Ctrl-C ``` `sync` is safe to stop and restart anytime (its position lives in the file). To keep it running across logout or reboot, wrap it in a launchd agent (macOS) or a systemd user service. ## Connect your real company (production) Once a sandbox works, switch to production. Set the Production keyset (`QB_PRODUCTION_CLIENT_ID` / `QB_PRODUCTION_CLIENT_SECRET`) and select the production API with `--qb-env production` (set `LOCAL_BOOKS_QB_ENV=production` once to avoid repeating it). The minted token records its environment, so a later `sync` or `status` refuses if you point the wrong `--qb-env` at it: ```sh local-books auth --qb-env production local-books sync --qb-env production --full ``` Intuit production rejects `http://localhost` redirect URIs, so the one-time `auth` needs a public HTTPS URL. This is only for `auth`; once tokens are saved, `sync` never touches the redirect URI again. 1. Start a tunnel to the local callback port (`cloudflared` needs no account): ```sh cloudflared tunnel --url http://localhost:8765 ``` Copy the `https://.trycloudflare.com` URL it prints. 2. On the Intuit app's **Production** redirect URIs, add `https://.trycloudflare.com/callback`. 3. Run `auth` with the tunnel as the public redirect, pointing the local listener back at 8765: ```sh LOCAL_BOOKS_QB_REDIRECT_URI=https://.trycloudflare.com/callback \ LOCAL_BOOKS_CALLBACK_PORT=8765 \ local-books auth --qb-env production ``` After tokens land, stop the tunnel; `sync` and `status` need nothing further. ## Inside the Epicenter monorepo The Intuit keys live in your personal Infisical project at `/apps/local-books` under their environment-qualified names (`QB_SANDBOX_*` and `QB_PRODUCTION_*`, ADR-0108). This is per-person bring-your-own provider configuration, not Epicenter-hosted infrastructure. Because the name carries the QuickBooks target, both keysets sit in `prod` for a single injection. The app-local `.infisical.json` points at the personal project; copy `.infisical.json.example` to `.infisical.json` and keep the real file local. That file is ignored because each operator has a different personal Infisical project. The committed `apps/api` and `ops` configs point at Epicenter's hosted project instead. The monorepo root has no Infisical config. `--qb-env` alone picks the keyset. Instead of exporting keys by hand, wrap a command with Infisical from this app directory: ```sh infisical run --path=/apps/local-books -- bun run src/bin.ts auth --qb-env sandbox infisical run --path=/apps/local-books -- bun run src/bin.ts sync --qb-env sandbox --full ``` The invocations are wrapped as scripts named for the QuickBooks account they touch, so you do not assemble the flags by hand: ```sh bun run auth:sandbox bun run sync:production --entity Invoice --full bun run status:production ``` ## Where things live ``` /apps/local-books/ companies// books.v.db # record-type tables + sync state; `status` prints the path credentials.json # OAuth tokens (0600), never inside a company's db config.json # optional: entities, environment, schedule ``` Epicenter owns one data root on the machine and gives local-books one directory below it, so a CLI run from a terminal and the Epicenter desktop app see the same books. The root is `~/Library/Application Support/so.epicenter` on macOS, `%APPDATA%\so.epicenter` on Windows, and `$XDG_DATA_HOME/so.epicenter` (or `~/.local/share/so.epicenter`) elsewhere; `EPICENTER_DATA_DIR` moves it, and it is the only such override. Tokens live in a `0600` `credentials.json` at the app-directory root, never inside a company's db, so the read-only query surface can never read them. Override the token path with `LOCAL_BOOKS_TOKEN_FILE`. That file is also the record of which companies are connected: there is no separate index. With one connected company the verbs find it on their own; with more than one, pass `--realm ` or set `LOCAL_BOOKS_QB_REALM`. Everything under the app directory belongs to local-books. No other Epicenter app receives a path or a database handle into it: a company's books reach another app only through a verb local-books publishes (the MCP tools above) or through a fact you promote into Epicenter yourself. ## The pre-Epicenter-root layout local-books used to keep its data in its own application-data directory (`~/Library/Application Support/local-books` on macOS), with a `companies.json` index beside the tokens. That directory is not read, moved, or referenced by any code here. Run `local-books auth` again and `sync` re-pulls the mirror; the old directory is inert bytes you can delete whenever you notice it. ## Build a single binary ```sh bun run build:binary # -> dist/local-books ``` ## Develop ```sh bun test # boots a mock QuickBooks server and drives the real command paths bun run typecheck bun run src/bin.ts demo # the sample company, end to end, offline ``` ## Design Standalone CLI by deliberate decision: see [ADR-0072](../../docs/adr/0072-local-books-ships-as-a-standalone-cli-the-daemon-surface-is-deferred.md) (the daemon/chat surface is deferred, with a trigger to revisit), [ADR-0061](../../docs/adr/0061-local-books-reads-facts-from-the-mirror-reports-live-and-writes-through-one-approved-verb.md) (the three capabilities), and the sync engine spec `specs/20260621T100000-local-books-cli-sync-engine.md`. --- ## File: apps/landing/src/content/blog/crdt-filesystem-pattern.md --- title: 'The vault pattern: Why folders beat databases for personal tools' pubDate: 2025-01-29 --- I've been building personal tools for years. Every time, I start with SQLite. Every time, I regret it. Not because SQLite is bad. It's fantastic. But because personal tools have different constraints than web apps, and databases optimize for the wrong things. ## The problem with traditional databases I built a journaling app in 2020. Used SQLite, like everyone said I should. Then I wanted to sync between devices. My options: 1. Build a sync server (now I'm running infrastructure) 2. Use a sync service (now I'm vendor-locked) 3. Try to sync the SQLite file (corruption city) I went with option 1. Six months later, I shut down the server. My journal died with it. ## Why CRDTs change everything Last year I discovered Yjs. It's a CRDT library that makes distributed data structures feel like magic. Here's the key insight that took me too long to realize: With CRDTs, sync isn't a feature you build. It's a property of the data structure. ```javascript // This is all the sync logic you need doc1.on('update', update => { Y.applyUpdate(doc2, update) }) ``` That's it. No conflict resolution. No sync servers. No three-way merges. ## The vault pattern Once you have CRDTs, you don't need a database. You need a place to put files. Here's the pattern: ``` ~/YourApp/ ├── service1/ │ ├── data.yjs # The CRDT document │ ├── assets/ # Binary files (images, etc) │ └── config.json # Service-specific settings ├── service2/ │ └── data.yjs └── README.md # Explain the structure ``` Each service owns a folder. Each folder contains: - One or more Yjs documents (the magic) - Regular files for assets - Human-readable configs ## One document per logical unit The first time I used Yjs, I made one giant document for everything. Bad idea. Here's why: - Large documents = slow initial sync - Every change updates every peer - Merge conflicts become tangled Instead, use one document per logical unit: - One doc per note - One doc per email thread - One doc per project Now syncs are fast, conflicts are isolated, and you can garbage collect old data. ## The implementation Here's actual code from Epicenter's note service: ```javascript class NoteService { constructor(vaultPath) { this.notesDir = path.join(vaultPath, 'notes') this.docs = new Map() } async createNote(title) { const doc = new Y.Doc() const noteId = generateId() // Define the structure const note = doc.getMap('note') note.set('title', title) note.set('content', new Y.Text()) note.set('created', Date.now()) // Save to disk const filePath = path.join(this.notesDir, `${noteId}.yjs`) await fs.writeFile(filePath, Y.encodeStateAsUpdate(doc)) this.docs.set(noteId, doc) return noteId } async loadNote(noteId) { const filePath = path.join(this.notesDir, `${noteId}.yjs`) const data = await fs.readFile(filePath) const doc = new Y.Doc() Y.applyUpdate(doc, data) this.docs.set(noteId, doc) return doc } } ``` Simple, right? No SQL schemas. No migrations. Just data structures and files. ## Handling binary data CRDTs are great for text and structured data. They're terrible for binary blobs. So don't put binaries in CRDTs. Put them in the filesystem: ```javascript async addAttachment(noteId, file) { // Hash the content for deduplication const hash = await hashFile(file) const ext = path.extname(file.name) // Store in assets folder const assetPath = path.join(this.notesDir, 'assets', `${hash}${ext}`) await fs.copyFile(file.path, assetPath) // Reference in the CRDT const doc = this.docs.get(noteId) const attachments = doc.getArray('attachments') attachments.push([{ hash, name: file.name, size: file.size, type: file.type }]) } ``` Now your attachments are just files. You can browse them with Finder. You can back them up with rsync. They're not trapped in a database blob column. ## Making it fast CRDTs can get slow if you're not careful. Here's what works: 1. **Enable garbage collection**: `doc.gc = true` 2. **Snapshot periodically**: Save the full state every 1000 updates 3. **Load incrementally**: Start with the snapshot, then apply recent updates 4. **Index separately**: Keep a lightweight index for search Here's our snapshotting logic: ```javascript class DocManager { constructor(filePath) { this.filePath = filePath this.snapshotPath = filePath + '.snapshot' this.updatesPath = filePath + '.updates' this.updateCount = 0 } async save(doc, update) { // Always append the update await fs.appendFile(this.updatesPath, update) this.updateCount++ // Snapshot every 1000 updates if (this.updateCount >= 1000) { const state = Y.encodeStateAsUpdate(doc) await fs.writeFile(this.snapshotPath, state) await fs.unlink(this.updatesPath) this.updateCount = 0 } } async load() { const doc = new Y.Doc() // Load snapshot if it exists if (await exists(this.snapshotPath)) { const snapshot = await fs.readFile(this.snapshotPath) Y.applyUpdate(doc, snapshot) } // Apply recent updates if (await exists(this.updatesPath)) { const updates = await fs.readFile(this.updatesPath) // Updates are concatenated, need to parse them let offset = 0 while (offset < updates.length) { const updateSize = readVarInt(updates, offset) offset += getVarIntSize(updateSize) const update = updates.slice(offset, offset + updateSize) Y.applyUpdate(doc, update) offset += updateSize } } return doc } } ``` ## The payoff Here's what you get with the vault pattern: 1. **True offline-first**: No sync server to be offline from 2. **Debuggable**: It's just files. Use standard Unix tools 3. **Portable**: Copy the folder to move your data 4. **Extensible**: Third-party tools can read/write the format 5. **Future-proof**: Even if Yjs dies, your data is in documented formats ## When not to use this The vault pattern isn't always right. Skip it if: - You need complex queries (use SQLite) - You have millions of items (use a real database) - You need ACID transactions (CRDTs are eventually consistent) - You're building a web service (this is for personal tools) ## Try it yourself I extracted the core vault pattern into a library: [vault-crdt](https://github.com/EpicenterHQ/epicenter/vault-crdt). It handles the boring parts (snapshotting, corruption recovery) so you can focus on your app. Or just steal the ideas. That's why I'm writing this. Personal tools should be personal. Your data should be yours. And folders of files is the most personal, portable data format we've invented. Time to stop renting databases for our own thoughts. --- ## File: apps/landing/src/content/blog/hello-world.md --- title: 'Hello World' pubDate: 2025-01-28 --- Welcome to the Epicenter blog. This is where I'll share thoughts on building tools that matter. ## What's coming I'll be writing about: - Building tools with a focus on user ownership - Technical deep dives into projects - Lessons learned from open source development Stay tuned.