# Technical Documentation: trycompai/crm > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/trycompai/crm) ### Overview (`README.md`)

Powered by Context

stars

CRM

Comp AI CRM is an open source, CRM designed for AI agents.
Agentic-first CRM.

The agent · Connections · Stack · Quick start · Configuration · Deploying · Contributing

MIT licence Built with eve Built with Bun Postgres

The companies list with an account open on its Agent tab

--- ## What this is Most CRMs are a database with a form in front of it. The AI ones bolt a chat box onto the side of that form. Both leave the actual work — finding out what is true, and writing it down — to a human who has better things to do. This is built the other way round. **The agent is not a feature of the CRM; the CRM is where the agent keeps its notes.** It runs on its own deployment, on its own schedule, against its own work queue. It decides what to look at next, books its own follow-ups, spends a research budget, and stops when the budget runs out. Nothing about it is request-response: close the browser and it keeps going. The rule the agent itself never breaks: **nothing about a person is guessed.** No tool accepts a confidence score, because a model asked to grade its own certainty will, and it will be wrong in the direction that makes it look useful. Tools report what they *observed* — `crm.signature-block`, `github.account-identity` — and a ledger prices the evidence. Strong evidence writes to the record. Weak evidence becomes a suggestion a human settles. A confidently wrong fact about a customer is worse than a blank field, because nobody can tell it is wrong. ## Screenshots
Agents that automate your CRM: a composer describing an agent in one sentence, with suggested actions beneath it What it actually does: records fill themselves in, agents that build agents, it books its own follow-ups, and a question box on every record
## The agent [`apps/agent`](./apps/agent) is its own deployment, built on [**eve**](https://eve.dev) — Vercel's filesystem-first framework for durable agents. A tool is a file, a skill is a markdown file, a schedule is a file, and the runtime handles the durable part: sessions that survive a redeploy, work that resumes where it stopped. | | | | --- | --- | | **18 authored tools** | `read_crm_history`, `search_crm`, `identify_contact`, `research_person`, `enrich_company`, `record_fact`, `schedule_recheck`… | | **4 skills** | `evidence.md`, `identity-matching.md`, `data-boundaries.md`, `writing-a-brief.md` — prose the agent reads, versioned like code | | **1 schedule** | `dispatch.ts`, which decides nothing. It leases what is due and starts a session per row. | | **A sandbox** | `bash`, `grep`, `glob` and a `/workspace`, with **`deny-all` egress** | **It runs itself.** `lib/tasks.ts` is the work queue: `claimDue` leases rows with `FOR UPDATE SKIP LOCKED`, so two dispatchers take disjoint work and a run that dies frees its row when the lease expires. Anything that looks like "every N minutes, the oldest ten contacts" belongs in a task's `dueAt`, not in a cron expression. When the agent wants another look at somebody it calls `schedule_recheck` and says why — and the reason is shown to the rep, because an agent that cannot say why it will be back in fourteen days does not have a reason, it has a default. **Every outside source is optional, and it is designed to run with none of them.** With no API keys at all it still works: `read_crm_history` reads your own threads, meetings and signature blocks, which is free and is the best evidence there is — no data vendor can sell you a reply from the person's own address. Each key opens one more place to look. It is told at the start of every session which ones this install has, so it plans around what it actually has rather than discovering the gaps one failed call at a time, and it prints the list at startup: ``` [agent] off Web research (PERPLEXITY_API_KEY) [agent] on Company brand data (Settings → General) [agent] on LinkedIn (Settings → General) ``` **Company brand data and LinkedIn are both [Context](https://link.context.dev/crm)**, and both come from the same key. The first is the logo, the colours, the industry and the real name behind a domain, which is the difference between an account that arrives as itself and one that arrives as a grey square with its initials in it. The second reads a person back from a LinkedIn URL already on their record — their real name, current title, employer and every earlier role — which is what settles who an address belongs to. It is the one key that is asked for rather than configured: it lives in a row, the onboarding asks for it, and **Settings → General** changes it afterwards, because a self-hoster's admin cannot redeploy to set an environment variable. **The sandbox has no network and no database.** Turning it on is what gives the model a shell — the difference between a tool-caller and something that can keep a dossier, diff this month's profile against last month's, and grep a thread for a signature block. `deny-all` egress costs nothing, because `web_fetch` runs in the app runtime and `web_search` at the model provider. What it removes is the only path by which a customer's email body could leave through a shell command. The other half of that rule is an absence: **the sandbox is never given `DATABASE_URL`.** A shell with credentials and egress is exfiltration-shaped even in an internal tool; a shell with neither is a text processor. **You can talk to it, and watch it work.** Every contact, company and deal has an **Agent** tab — the steps as it takes them, the leads it throws away and why, and its questions answered in place when it cannot decide between two people. Conversations are durable and survive a reload; the record travels in a signed token rather than being bolted onto the front of your message. Set `AGENT_BRIDGE_SECRET` to the same value in both processes to turn it on. Without it the tab reports that it is not configured, and the agent carries on running its own schedule. [`docs/agent.md`](./docs/agent.md) is the full write-up. ## Connections and agent access Open **Settings → Connections** to connect services for the workspace. Only an owner or admin can connect, reconnect, review, or disconnect a service. The add dialog shows only supported CRM connections. Each supported service opens inside CRM. Choose **Read only** when agents must not change external data. Choose **Read and write** when approved agent tools can make external changes. The connection page shows every discovered tool and its access type. New or changed tools require review before an agent can use them. A connection gives the workspace a maximum access level. Each deployed agent receives a smaller grant with exact tool names and fingerprints. The connection page lists every deployed agent that uses the service. Disconnecting a service blocks the next agent call. It also removes every discovered tool from active use. Reconnect the service and review its current tools before agents use it again. ## The stack A [Turborepo](https://turborepo.dev) monorepo on [Bun](https://bun.com), deployed on [Vercel](https://vercel.com). | | | | --- | --- | | **Agent** | [eve](https://eve.dev) — durable sessions, tools, skills, schedules, sandboxes | | **Model** | [Vercel AI Gateway](https://vercel.com/docs/ai-gateway) — no provider SDK, and OIDC on Vercel means no key to manage | | **Sandbox** | [Vercel Sandbox](https://vercel.com/docs/vercel-sandbox) in production, Docker or microsandbox locally | | **Front end** | [Next.js](https://nextjs.org) App Router · [shadcn/ui](https://ui.shadcn.com) · [nuqs](https://nuqs.dev) for URL state | | **API** | [NestJS](https://nestjs.com) with [nestjs-trpc](https://nestjs-trpc.io) — HTTP, auth, tRPC, mailbox sync | | **Data** | [Prisma](https://prisma.io) · Postgres ([Neon](https://neon.tech)) · optional Redis ([Upstash](https://upstash.com)) | | **Auth** | [Better Auth](https://better-auth.com) — Google, Microsoft, or your own IdP; one allow-list | | **Files** | [Vercel Blob](https://vercel.com/docs/vercel-blob) — mirrors profile pictures so they survive the source going away | | **Tooling** | [Biome](https://biomejs.dev) · TypeScript everywhere | The app talks to the API over **tRPC**, and the router type is generated from the NestJS routers — so the front end is type-safe from the Prisma row to the table cell. List state (filters, sort, page) lives in the URL, so copying the address bar reproduces the view. ### Layout | Path | | | --- | --- | | `apps/agent` | The research agent — tools, skills, schedules, sandbox | | `apps/app` | Next.js front end · :3000 | | `apps/api` | NestJS API — HTTP, auth, tRPC, mailbox sync · :3001 | | `packages/db` | Prisma schema, migrations, shared Postgres client | | `packages/auth` | Better Auth config and the sign-in allow-list | | `packages/ui` | shadcn/ui components, the Tailwind theme | | `packages/env` | Finds and loads the root `.env` | ### Three rules the codebase holds to Written up where the work happens, not in a style guide: - **Intelligence never lives in the API** ([docs/api.md](./docs/api.md)). Nest reports that something happened; the agent decides what it means. Two copies of an identity matcher once drifted until one matched every employer on earth. - **`packages/ui` is the only source of UI** ([docs/design.md](./docs/design.md)). No overriding styles at the call site. - **There are no organizations.** Single tenant, deliberately. An `organizationId` that is always the same value is a column, an index and a permissions check that buys nothing and reads like a real one at review time. ## Quick start You need [Bun](https://bun.com) and Docker. ```sh git clone https://github.com/trycompai/crm.git && cd crm cp .env.example .env # then fill in the values below bun install docker compose up -d # Postgres on :5432 bun run db:deploy # apply migrations bun run db:seed # optional: a believable pipeline to look at bun run dev ``` The app is on [localhost:3000](http://localhost:3000), the API on [localhost:3001](http://localhost:3001). That clone gives you `release`, the default branch and the last tagged release — what you want if you are running this. `main` is where unreleased work lands: green, but not cut yet. If you are here to send a pull request, `git switch main` first and read [CONTRIBUTING](./CONTRIBUTING.md). ### The values to set Open `.env` and set these. Everything else in the file is optional and commented out. | Variable | What to put in it | | ------------------------------------------ | -------------------------------------------------------------------- | | `BETTER_AUTH_SECRET` | `openssl rand -base64 32` | | `ALLOWED_SIGN_IN` | Your email domain, e.g. `acme.com`. Or one address, e.g. `you@gmail.com`. | | `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET`| A Google OAuth client — 2 minutes, below. Both or neither. | | `MICROSOFT_CLIENT_ID` / `MICROSOFT_CLIENT_SECRET` | A Microsoft Entra app registration — below. Both or neither. | **Pick at least one of Google and Microsoft**, or add your own identity provider on **Settings → SSO** once you are in. Setting both is fine and common: the sign-in page offers both buttons, and each rep's mail is read from whichever they signed in with. `DATABASE_URL` already matches the `docker compose` Postgres, so leave it alone unless you brought your own.
Getting the Google OAuth client 1. [Google Cloud console](https://console.cloud.google.com/apis/credentials) → **Credentials** → **Create credentials** → **OAuth client ID** → **Web application**. 2. Under **Authorised redirect URIs**, add `http://localhost:3001/api/auth/callback/google`. 3. Enable the [Gmail API](https://console.cloud.google.com/apis/library/gmail.googleapis.com) and the [Calendar API](https://console.cloud.google.com/apis/library/calendar-json.googleapis.com) for the project. 4. Copy the client ID and secret into `.env`. Google is the sign-in method a clone starts with, and the same client reads Gmail and Calendar — so most installs want it. The API will nonetheless boot without it: an install that signs in with Microsoft, or through its own identity provider added on **Settings → SSO**, leaves both empty and gets no Google button and no Gmail sync. Set them together or not at all; half a pair is a button that fails at Google. If your account is on a Google Workspace domain, set the consent screen to **Internal** and nobody outside your org can even reach the prompt.
Getting the Microsoft app registration 1. [Azure portal](https://portal.azure.com) → **Microsoft Entra ID** → **App registrations** → **New registration**. Microsoft's own walkthrough is [Register an application](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app#register-an-application). 2. Under **Supported account types**, pick **Accounts in this organizational directory only** if everyone signing in is on your tenant. Anything wider is fine too — `ALLOWED_SIGN_IN` still decides who gets an account — but the narrow choice turns outsiders away at Microsoft rather than at our door. 3. Set the **Redirect URI** to **Web** and `http://localhost:3001/api/auth/callback/microsoft`. In production this is `https:///api/auth/callback/microsoft` — the API's origin, not the app's, because the API is what serves `/api/auth/*`. 4. **API permissions** → **Add a permission** → **Microsoft Graph** → **Delegated permissions** → tick **User.Read** and **Mail.Read**. `offline_access` is requested at sign-in and needs no entry here. If your tenant requires admin consent, click **Grant admin consent** so reps are not each asked. 5. **Certificates & secrets** → **New client secret**. Copy the **Value**, not the Secret ID — the value is shown once and never again. 6. **Overview** → copy the **Application (client) ID**. 7. Put the two into `.env` as `MICROSOFT_CLIENT_ID` and `MICROSOFT_CLIENT_SECRET`. `Mail.Read` is read-only: the CRM can list and read messages and can never send, reply, move or delete. Reading is forward-only — the first check records the current time and imports nothing, so connecting an old mailbox does not dump years of mail into the CRM. To refuse other tenants at Microsoft rather than relying on `ALLOWED_SIGN_IN`, set `MICROSOFT_TENANT_ID` to your tenant's GUID (**Overview** → **Directory (tenant) ID**). It defaults to `common`, which accepts any work, school or personal Microsoft account and leaves the allow-list to sort them out. `organizations` is the middle setting: any work or school account, no personal ones. Microsoft client secrets expire — 24 months at most, and 6 months by default. Note the expiry somewhere, because the symptom of a lapsed one is every rep's mail quietly failing to sync.
`ALLOWED_SIGN_IN` is the entire authorisation model — an unset value means nobody can sign in, which is the safe direction to fail. It takes whole domains, individual addresses, or a mix: ```sh ALLOWED_SIGN_IN="acme.com" # everyone at your company ALLOWED_SIGN_IN="acme.com,contractor@gmail.com" # …plus one outsider ALLOWED_SIGN_IN="you@gmail.com" # a one-person install ``` ## Configuration **There is one `.env`, at the root of the repo**, read by all three processes. Real environment variables always win, so on a hosting platform you configure it there and the file is purely a local convenience. Beyond the values above, everything is optional and the app runs without any of it. [`.env.example`](./.env.example) is the full list with a note on each; the short version: | | | | --- | --- | | `API_URL` / `APP_URL` | Where the two halves are served. Only needed off localhost. | | `PERPLEXITY_API_KEY` | Lets the agent search the open web, with citations. | | `AGENT_BRIDGE_SECRET` | Lets a rep talk to the agent from a contact's **Agent** tab. | | `CHAT_EVENT_ENCRYPTION_KEY` | Encrypts builder chat history. Generate it with `openssl rand -base64 32`. | | `MCP_ENCRYPTION_KEY` | Encrypts MCP OAuth tokens. Generate it with `openssl rand -base64 32`. | | `REDIS_URL` | A shared cache. Without it, per-instance and in-memory. | | `CRON_SECRET` | Guards the mailbox sync route. Required to use it. | | `CRM_TELEMETRY_DISABLED` | Set to `1` and this install reports nothing. `DO_NOT_TRACK` too. | ## Tasks | Command | | | --- | --- | | `bun run dev` | Prepare the local database, then run everything in dependency-aware watch mode | | `bun run build` | Build all apps and packages | | `bun run test` | Run the test suite | | `bun run check-types` | `tsc --noEmit` everywhere | | `bun run lint` / `format` | [Biome](https://biomejs.dev) | | `bun run db:migrate` | Create and apply a migration | | `bun run db:seed` | Top up the demo pipeline (idempotent) | | `bun run db:studio` | Prisma Studio | | `bun run --filter=api trpc:generate` | Regenerate the `AppRouter` type | | `bun run --filter=api dev:session` | Print a session cookie for a local user | Scope any of them with a Turborepo filter: `bun run dev --filter=api`. Because sign-in goes through an identity provider, there is no way to get a session from a terminal — `dev:session` writes the rows Better Auth would have written and prints the cookie it would have set. It refuses to run with `NODE_ENV=production`. ## Deploying Three deployments and a Postgres: the Next.js app, the NestJS API, and the agent. They are independent, and the only thing they must agree on is `DATABASE_URL` and `BETTER_AUTH_SECRET` — the API mints the session cookie and the app verifies it, so a mismatch is a redirect loop rather than an error. Set `API_URL` and `APP_URL` to the real origins, and if the two are on different subdomains of one parent, set `AUTH_COOKIE_DOMAIN` to the parent so one cookie covers both. Add `http://your-api-host/api/auth/callback/google` — and, if you use Microsoft, `http://your-api-host/api/auth/callback/microsoft` — to the provider's redirect URIs. Set `CRON_SECRET` and point a scheduler at `POST /internal/sync/mailboxes` to keep the mailbox sync running. `apps/api/src/generated/server.ts` is committed and `build` must never regenerate it — the generator needs a newer GLIBC than most build images have. Regenerate locally and commit it with the router change that caused it. ## Contributing We'd rather have a paragraph you wrote than a pull request an agent wrote. See [CONTRIBUTING.md](./CONTRIBUTING.md). Security issues go through [SECURITY.md](./SECURITY.md), privately, not a public issue. ## Licence [MIT](./LICENSE). --- ### Overview (`adrs/README.md`) # adrs Proposals live here. One file per idea, `.md` or `.txt`, named after the thing you want to change. Write it yourself, in your own words, at whatever length the idea actually needs — a paragraph is a perfectly good ADR. See [CONTRIBUTING.md](../CONTRIBUTING.md) for why we'd rather have that than a pull request full of generated code. Roughly: - what you want to change - why the current behaviour is a problem, ideally with the case that made you notice - what you'd do instead, and what it would break Nothing to record after the fact. These are for deciding, and they stay as the record of what was decided. --- ### Overview (`apps/api/README.md`) # `api` NestJS API for the CRM. Runs on Bun, backed by `@crm/db` and `@crm/auth`. ## Running it ```sh cp .env.example .env # at the repo root — one file for the whole monorepo openssl rand -base64 32 # -> BETTER_AUTH_SECRET bun run dev # watch mode on http://localhost:3001 bun run test bun run build && bun run start:prod ``` Three values are required and the process refuses to boot without them, naming the one it is missing: `DATABASE_URL`, `BETTER_AUTH_SECRET` and `ALLOWED_SIGN_IN`. `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` are the fourth value almost every install wants — they are both the sign-in button and the Gmail and Calendar sync — but they are optional and set as a pair, because an install that signs in through its own identity provider on **Settings → SSO** needs neither. With them, register `http://localhost:3001/api/auth/callback/google` as an authorised redirect URI. `src/config/env.validation.ts` is the full list of what this process reads; [`docs/environment.md`](../../docs/environment.md) explains where the file is found. Bun is the runtime, not just the package manager: `@crm/db` and `@crm/auth` ship TypeScript sources, so `tsc`/`node` cannot run this app directly. `tsc` is used for type checking only (`bun run check-types`). ## Routes | Route | Auth | Notes | | ---------------- | ---------- | --------------------------------------------- | | `/api/auth/*` | anonymous | Mounted by `@thallesp/nestjs-better-auth` | | `/auth/me` | required | Cached profile of the signed-in user | | `/auth/session` | optional | Whether the caller is signed in | | `/health` | anonymous | 200 with a database round-trip, 503 otherwise | | `/internal/sync/google` | `CRON_SECRET` bearer | Vercel Cron entrypoint for Gmail/Calendar sync. Fails closed when the secret is unset. | ## How auth is wired This process owns authentication. It mounts `/api/auth/*` and is the only one that writes session cookies; the Next.js app in `apps/app` reads those sessions straight from Postgres via `@crm/auth` and calls the routes above with `credentials: "include"`. `AuthModule.forRoot({ auth })` mounts the Better Auth handler and registers a **global** `AuthGuard`, so every route is protected unless it opts out: ```ts @Get('public') @AllowAnonymous() // no session required @OptionalAuth() // session optional; @Session() may be undefined ``` It also calls `enableCors({ origin: trustedOrigins, credentials: true })` for the whole app, which is what lets the browser at `localhost:3000` talk to it — `APP_URL` is the single knob for that, comma-separated if the app is served from more than one origin. `main.ts` creates the app with `bodyParser: false` — Better Auth needs the raw request body, and the library installs its own parsers around the auth routes. `AuthHooksService` uses `@AfterUpdate("user")` to drop a cached profile the moment its row changes. Database hooks require `databaseHooks: {}` in the Better Auth options and endpoint hooks require `hooks: {}` (both set in `@crm/auth`); the library throws at startup without them. ## Caching `AppCacheModule` registers `@nestjs/cache-manager` globally. It uses `REDIS_URL` when set and otherwise falls back to a per-instance in-memory store — fine for local development, not for more than one API instance. `AuthService.getProfile` is the reference pattern: read through the cache, write with an explicit TTL, invalidate on change. ## Notes - Better Auth stores rate limits in the database (`rateLimit.storage`), so every request to `/api/auth/*` needs a reachable Postgres. Moving this to `secondaryStorage` backed by Redis would remove that dependency. - Environment variables are validated at boot by `src/config/env.validation.ts`. The process refuses to start on a bad config. --- ### Overview (`apps/app/README.md`) This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app). ## Getting Started First, run the development server: ```bash npm run dev # or yarn dev # or pnpm dev # or bun dev ``` Open [http://localhost:3000](http://localhost:3000) with your browser to see the result. You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file. This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel. ## Learn More To learn more about Next.js, take a look at the following resources: - [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API. - [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial. You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome! ## Deploy on Vercel The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js. Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details. --- ### Overview (`packages/auth/README.md`) # `@crm/auth` [Better Auth](https://better-auth.com) configuration for the monorepo, backed by `@crm/db`. Enabled: Google sign-in, account linking, cookie-cached sessions, and database-backed rate limiting. This is an internal, single-tenant app — there are no organizations, and email + password is switched off so the only way in is a Google account. ## Topology The **NestJS API** (`apps/api`, port 3001) mounts `/api/auth/*` via `@thallesp/nestjs-better-auth` and is the only process that writes session cookies. The **Next.js app** (`apps/app`, port 3000) imports this package on the server to *read* sessions straight from Postgres, and points its browser client at the API for sign-in and sign-out. Both processes therefore need the same `BETTER_AUTH_SECRET` and `DATABASE_URL`, or the cookie one writes will not verify in the other. ## Usage ### Server ```ts import { auth } from "@crm/auth/auth"; // Session for an incoming request const session = await auth.api.getSession({ headers: request.headers }); ``` `auth.handler` is a standard `(Request) => Promise` function, so it mounts anywhere. In a Next.js App Router route (`app/api/auth/[...all]/route.ts`): ```ts import { auth } from "@crm/auth/auth"; export const GET = auth.handler; export const POST = auth.handler; ``` ### Client ```ts import { signIn, signOut, useSession } from "@crm/auth/client"; await signIn.social({ provider: "google", callbackURL: "/" }); ``` `NEXT_PUBLIC_API_URL` decides which origin the client talks to. It must point at whichever process mounts the handler — the NestJS API. The Next.js app inlines it at build time from `API_URL`, in `next.config.ts`, so there is one variable rather than two spellings of one origin. Unset, the client uses the current origin. The client plugin list mirrors the server plugin list. Keep them in sync or the inferred client API will drift from the routes the server exposes. ## Setup ```bash cp .env.example .env # at the repo root — there is no per-package env file openssl rand -base64 32 # -> BETTER_AUTH_SECRET ``` This package is a library and never loads an env file of its own; it reads whatever `process.env` its host process has, and `src/env.ts` imports `@crm/env/load` so the repo-root `.env` is picked up even when the Better Auth CLI loads `auth.ts` directly. See [`docs/environment.md`](../../docs/environment.md). Create an OAuth client in the Google Cloud console and add `/api/auth/callback/google` — `http://localhost:3001/api/auth/callback/google` in development — as an authorised redirect URI. `ALLOWED_SIGN_IN` decides who may sign in, and an empty value admits nobody. It is the whole authorisation model: there are no roles and no organizations, so `src/workspace.ts` is worth reading before you change anything here. ## Changing the schema Adding a plugin or an additional user field changes the database schema. After editing `src/auth.ts`: ```bash bun run auth:generate # rewrites packages/db/prisma/schema.prisma bun run db:migrate # create the migration ``` ## Notes - **JIT package.** `exports` point at TypeScript sources, which keeps Better Auth's inferred types intact — declaration emit tends to break them. Turbopack transpiles workspace packages automatically, so a Next.js app needs no `transpilePackages` entry. - **Next.js server actions** need the `nextCookies()` plugin (from `better-auth/next-js`) as the *last* entry in the plugin array. It is omitted here so the package stays framework-agnostic; add it — along with `next` as a dependency — once an app relies on setting cookies from server actions. - **Cross-origin cookies.** On localhost the API and the app differ only by port, which cookies ignore. Deployed on separate subdomains they need `AUTH_COOKIE_DOMAIN` set to the shared parent (`.example.com`). --- ### Overview (`packages/db/README.md`) # `@crm/db` PostgreSQL access for the monorepo: the Prisma schema, migrations, and a shared `PrismaClient` instance. ## Usage ```ts import { db } from "@crm/db"; const users = await db.user.findMany({ take: 10 }); ``` Types and query helpers come from the same entrypoint: ```ts import { Prisma, type User } from "@crm/db"; ``` ## Setup ```bash docker compose up -d # Postgres matching the DATABASE_URL in .env.example cp .env.example .env # at the repo root bun run db:generate # generate Prisma Client bun run db:deploy # apply the migrations ``` `DATABASE_URL` comes from the **repo-root `.env`**, loaded by `@crm/env` — see [`docs/environment.md`](../../docs/environment.md). `src/client.ts` imports `@crm/env/load` before reading it, and `prisma.config.ts` does the same so the CLI works without any app running. ## Scripts | Script | Purpose | | ------------- | -------------------------------------------------------- | | `build` | `prisma generate` — cached by Turborepo, runs via `^build` | | `dev:prepare` | Apply pending local migrations, reject drift, and generate Prisma Client | | `db:generate` | Regenerate Prisma Client | | `db:migrate` | Create and apply a migration (development) | | `db:deploy` | Apply pending migrations (CI / production) | | `db:push` | Push the schema without a migration (prototyping only) | | `db:reset` | Drop and recreate the database | | `db:seed` | Run `prisma/seed.ts` | | `db:studio` | Open Prisma Studio | Each is also exposed at the repo root (`bun run db:migrate`) and routed through `turbo run`. The root `bun run dev` command runs `dev:prepare` before any service starts and reruns it when the schema or migrations change. Services that hold a Prisma client restart only after preparation finishes, so they cannot retain an older generated model map. ## Notes - **Prisma 7 + driver adapter.** There is no query engine binary; the client talks to PostgreSQL through `@prisma/adapter-pg`. See `src/client.ts`. - **Generated code is not committed.** `prisma generate` writes to `src/generated/`, which is gitignored and declared as the `build` task's output so Turborepo caches it. - **JIT package.** `exports` point at TypeScript sources; the consumer compiles them. Turbopack transpiles workspace packages automatically, so a Next.js app needs no `transpilePackages` entry. Non-bundler consumers need a TypeScript runtime — the NestJS API runs on Bun for exactly this reason. - **Auth models are generated.** `User`, `Session`, `Account`, `Verification` and `RateLimit` come from `@better-auth/cli`. Do not hand-edit them — change the Better Auth config in `@crm/auth` and re-run `bun run auth:generate`. The generator is additive: it adds models and fields a plugin needs but never removes the ones a dropped plugin left behind, so removing a plugin means deleting its models from the schema by hand. --- ### Overview (`.agents/skills/nuqs/README.md`) # nuqs Best Practices for Next.js A comprehensive best practices skill for using [nuqs](https://nuqs.dev) - type-safe URL query state management - in Next.js applications. ## Overview This skill provides 42 rules across 8 categories to help AI agents and developers write correct, performant, and maintainable code when using nuqs for URL state management. ## Getting Started ```bash # Install dependencies (if any) pnpm install # Build AGENTS.md from references pnpm build # Validate the skill pnpm validate ``` ## Categories | Priority | Category | Impact | Rules | |----------|----------|--------|-------| | 1 | Parser Configuration | CRITICAL | 8 | | 2 | Adapter & Setup | CRITICAL | 5 | | 3 | State Management | HIGH | 7 | | 4 | Server Integration | HIGH | 6 | | 5 | Performance Optimization | MEDIUM | 5 | | 6 | History & Navigation | MEDIUM | 4 | | 7 | Debugging & Testing | LOW-MEDIUM | 3 | | 8 | Advanced Patterns | LOW | 4 | ## Creating a New Rule 1. Create a new file in `references/` with the pattern `{prefix}-{slug}.md` 2. Use the template from `assets/templates/_template.md` 3. Include YAML frontmatter with `title`, `impact`, `impactDescription`, `tags` 4. Add **Incorrect** and **Correct** code examples 5. Run `pnpm build` to regenerate AGENTS.md 6. Run `pnpm validate` to check for errors ## Rule File Structure ```markdown --- title: Rule Title impact: CRITICAL|HIGH|MEDIUM|LOW impactDescription: Quantified impact (e.g., "2-10× improvement") tags: prefix, keyword1, keyword2 --- ## Rule Title Explanation of WHY this matters. **Incorrect (annotation):** \`\`\`tsx // Bad code example \`\`\` **Correct (annotation):** \`\`\`tsx // Good code example \`\`\` Reference: [Link](url) ``` ## File Naming Convention - Rule files: `{prefix}-{descriptive-slug}.md` (e.g., `parser-use-typed-parsers.md`) - Prefix must match a section defined in `references/_sections.md` - Use lowercase with hyphens ## Impact Levels | Level | Description | |-------|-------------| | CRITICAL | Causes build failures, runtime errors, or major performance issues | | HIGH | Significant bugs, performance degradation, or maintenance problems | | MEDIUM | Noticeable issues or suboptimal patterns | | LOW-MEDIUM | Minor improvements or edge case handling | | LOW | Nice-to-have optimizations or advanced patterns | ## Scripts - `pnpm build` - Regenerate AGENTS.md from references - `pnpm validate` - Check skill against quality guidelines ## Contributing 1. Check existing rules to avoid duplication 2. Follow the rule template structure 3. Include production-realistic code examples 4. Quantify impact where possible 5. Run validation before submitting ## References - [nuqs Documentation](https://nuqs.dev) - [nuqs GitHub](https://github.com/47ng/nuqs) - [Next.js Documentation](https://nextjs.org/docs) --- ### Overview (`.agents/skills/vercel-composition-patterns/README.md`) # React Composition Patterns A structured repository for React composition patterns that scale. These patterns help avoid boolean prop proliferation by using compound components, lifting state, and composing internals. ## Structure - `rules/` - Individual rule files (one per rule) - `_sections.md` - Section metadata (titles, impacts, descriptions) - `_template.md` - Template for creating new rules - `area-description.md` - Individual rule files - `metadata.json` - Document metadata (version, organization, abstract) - **`AGENTS.md`** - Compiled output (generated) ## Rules ### Component Architecture (CRITICAL) - `architecture-avoid-boolean-props.md` - Don't add boolean props to customize behavior - `architecture-compound-components.md` - Structure as compound components with shared context ### State Management (HIGH) - `state-lift-state.md` - Lift state into provider components - `state-context-interface.md` - Define clear context interfaces (state/actions/meta) - `state-decouple-implementation.md` - Decouple state management from UI ### Implementation Patterns (MEDIUM) - `patterns-children-over-render-props.md` - Prefer children over renderX props - `patterns-explicit-variants.md` - Create explicit component variants ## Core Principles 1. **Composition over configuration** — Instead of adding props, let consumers compose 2. **Lift your state** — State in providers, not trapped in components 3. **Compose your internals** — Subcomponents access context, not props 4. **Explicit variants** — Create ThreadComposer, EditComposer, not Composer with isThread ## Creating a New Rule 1. Copy `rules/_template.md` to `rules/area-description.md` 2. Choose the appropriate area prefix: - `architecture-` for Component Architecture - `state-` for State Management - `patterns-` for Implementation Patterns 3. Fill in the frontmatter and content 4. Ensure you have clear examples with explanations ## Impact Levels - `CRITICAL` - Foundational patterns, prevents unmaintainable code - `HIGH` - Significant maintainability improvements - `MEDIUM` - Good practices for cleaner code --- ### Overview (`.agents/skills/vercel-react-best-practices/README.md`) # React Best Practices A structured repository for creating and maintaining React Best Practices optimized for agents and LLMs. ## Structure - `rules/` - Individual rule files (one per rule) - `_sections.md` - Section metadata (titles, impacts, descriptions) - `_template.md` - Template for creating new rules - `area-description.md` - Individual rule files - `src/` - Build scripts and utilities - `metadata.json` - Document metadata (version, organization, abstract) - __`AGENTS.md`__ - Compiled output (generated) - __`test-cases.json`__ - Test cases for LLM evaluation (generated) ## Getting Started 1. Install dependencies: ```bash pnpm install ``` 2. Build AGENTS.md from rules: ```bash pnpm build ``` 3. Validate rule files: ```bash pnpm validate ``` 4. Extract test cases: ```bash pnpm extract-tests ``` ## Creating a New Rule 1. Copy `rules/_template.md` to `rules/area-description.md` 2. Choose the appropriate area prefix: - `async-` for Eliminating Waterfalls (Section 1) - `bundle-` for Bundle Size Optimization (Section 2) - `server-` for Server-Side Performance (Section 3) - `client-` for Client-Side Data Fetching (Section 4) - `rerender-` for Re-render Optimization (Section 5) - `rendering-` for Rendering Performance (Section 6) - `js-` for JavaScript Performance (Section 7) - `advanced-` for Advanced Patterns (Section 8) 3. Fill in the frontmatter and content 4. Ensure you have clear examples with explanations 5. Run `pnpm build` to regenerate AGENTS.md and test-cases.json ## Rule File Structure Each rule file should follow this structure: ```markdown --- title: Rule Title Here impact: MEDIUM impactDescription: Optional description tags: tag1, tag2, tag3 --- ## Rule Title Here Brief explanation of the rule and why it matters. **Incorrect (description of what's wrong):** ```typescript // Bad code example ``` **Correct (description of what's right):** ```typescript // Good code example ``` Optional explanatory text after examples. Reference: [Link](https://example.com) ## File Naming Convention - Files starting with `_` are special (excluded from build) - Rule files: `area-description.md` (e.g., `async-parallel.md`) - Section is automatically inferred from filename prefix - Rules are sorted alphabetically by title within each section - IDs (e.g., 1.1, 1.2) are auto-generated during build ## Impact Levels - `CRITICAL` - Highest priority, major performance gains - `HIGH` - Significant performance improvements - `MEDIUM-HIGH` - Moderate-high gains - `MEDIUM` - Moderate performance improvements - `LOW-MEDIUM` - Low-medium gains - `LOW` - Incremental improvements ## Scripts - `pnpm build` - Compile rules into AGENTS.md - `pnpm validate` - Validate all rule files - `pnpm extract-tests` - Extract test cases for LLM evaluation - `pnpm dev` - Build and validate ## Contributing When adding or modifying rules: 1. Use the correct filename prefix for your section 2. Follow the `_template.md` structure 3. Include clear bad/good examples with explanations 4. Add appropriate tags 5. Run `pnpm build` to regenerate AGENTS.md and test-cases.json 6. Rules are automatically sorted by title - no need to manage numbers! ## Acknowledgments Originally created by [@shuding](https://x.com/shuding) at [Vercel](https://vercel.com). --- ### Overview (`apps/agent/agent/sandbox/workspace/README.md`) # /workspace Scratch space for research in progress. Nothing in here is the record — the CRM is the record, and it is reached through tools, not through this filesystem. Useful for: - **Dossiers.** `dossiers/.json` — dump a profile you fetched, then `grep`/`jq` it instead of re-reading it into context. - **Diffs.** Keeping last month's profile next to this month's is how a job change becomes visible without a second lookup. - **Working notes** while you assemble a brief across several sources. Two things this filesystem must never hold: 1. **Anything out of a mailbox.** Email bodies and meeting notes stay in the turn, in the app runtime. `read_crm_history` gives you the whole message because it is our own data; writing it here moves it somewhere with a different lifetime and a different set of eyes on it. 2. **Credentials.** There are none here, and there is no reason to create any. There is no network from this sandbox. `web_fetch` and `web_search` still work — they run outside it. --- ### Agent Panel (`docs/agent-panel.md`) # The Agent panel — read when touching the record sheet's Agent tab `lib/agent-record.ts` maps a record kind to everything downstream: the header sent, the claim minted, the field a conversation is filed under, the empty-thread questions. `AgentConversation` holds only the handle (session id + cursor); the transcript is in `AgentEvent` from the audit hook. - **Load with `session.snapshot()`, never by hand** — one call returns the event prefix, the cursor, and a continuation token *iff* eve will accept another turn. Hand-rolling the stream produced every panel bug this thing has had. - **The token is the authority on whether a message can be sent**, not our reading of the events. - **`streamIndex: 0` on resume** — the saved index is where the last *reader* stopped. - **The snapshot is cached for a minute, not refetched on every mount.** A settled thread cannot change under us — `ready` is waiting on the rep, `ended` is over — and the snapshot is the whole session from index 0, tens of megabytes on a real thread. A live thread stays live through `refetchInterval`, which ignores `staleTime`. - **`loadThread` rethrows an abort.** Swallowing it caches a false `offline` for the life of the `staleTime`; only a genuinely unreachable agent may return one. - **The archive is fetched only when the snapshot fails.** It is the offline fallback and nothing else, so loading it up front bought a serial round trip on every open. Nothing renders until it lands — `useEveAgent` reads `initialEvents` once, at mount. - **It ages with the snapshot, and is never `staleTime: Infinity`.** A session's archive is not immutable: the agent appends to it every turn. Held forever, an agent that goes down, comes back, and goes down again renders the first outage's transcript and silently drops what happened in between. Both reads mirror the same session, so both expire at `SETTLED_TTL_MS`. - **Never key the transcript on the archive.** A refetch that lands while it is mounted is ignored on purpose; remounting to pick it up would abort a live answer, which is the same failure `keepMounted` exists to prevent. Correcting a stale fallback is the next mount's job. - **The audit hook does not archive `reasoning.appended`.** Every delta carries `reasoningSoFar`, the whole reasoning to that point, so the archive grew as the square of the tokens — one insert per token, 90% of `AgentEvent`, for events the transcript never renders. `reasoning.completed` still lands. eve's durable stream keeps its own copy either way; this is only what we file. - **Quiet for 90 seconds is over, not working** — restarted agents leave sessions with no closing boundary, which would lock the thread forever. - **An unreachable agent is `offline`, not `working`** — fall back to the `AgentEvent` archive and keep the composer usable. - **An ended thread gets a Start-a-new-conversation button, not a locked box**; `composerState()` keeps ended and working apart. - **Nothing mounts until the list has loaded**, or a new session starts and remounts. - **The landed thread is captured once** (`resolveThread`), or the first save swaps the open conversation out from under a live answer. It lives in `?thread=`. - **`keepMounted` on the tab descriptor** (`detail-sheet.tsx`) — Radix drops an inactive tab, aborting the stream mid-answer. - **`autoScroll` and nothing else**; `scrollAnchor` stops it following the bottom. - **One `MessageScrollerItem` per message, not per part**; ids prefer `toolCallId`. - **Scoped to the rep** — a session id in a body decides which row, never whose. This lives in the API and is not a breach of rule one: listing history decides nothing. --- ### Agent (`docs/agent.md`) # Agent — `apps/agent` An [eve](https://eve.dev/docs) app, its **own deployment**, owning every piece of intelligence in this repo. Read with `api.md`, whose first rule is that none of this may move into the API. Local dev, the bridge env vars and the manual dispatch command are in `docs/setup.md`. **Read the eve guide before writing eve code** — `apps/agent/node_modules/eve/docs/README.md` matches the installed version; `.agents/skills/eve` is the skill. Guessing typechecks, builds, then misbehaves. ## Model Default `zai/glm-5.2-fast`; `DEFAULT_AGENT_MODEL` in `@crm/db/settings` because the agent and the API both need it. - **A row (`AppSetting`), not an env var**, via `defineDynamic` on `session.started`. Open conversations keep their model — prompt caches are per model. - **`lib/model.ts` always sends `modelContextWindowTokens`**; eve never inherits it. - **A failed read logs and keeps the compiled fallback.** Never throws. - **The chooser offers only `tool-use` models** (`ModelCatalogService`). - **Not a frontier model, deliberately** — refusing wrong answers is enforced by the tools and evidence model, not model strength. ## Pictures are copied, never linked `mirror()` copies bytes to Vercel Blob; the record points at our copy. Lives in **`@crm/db/blob`** — writers are `lib/brand-images.ts`, `lib/portrait.ts`, `FaviconService`, `ImageMirrorService`, `prisma/seed.ts`. - **The key hashes the bytes** — idempotent, and a redesigned mark gets a new URL. - **`COMPANY_IMAGE_FIELDS` (`@crm/db/images`) is the one list of picture columns.** - **Fetch through `@crm/db/safe-fetch`** — vendor URLs are SSRF vectors. - **No `BLOB_READ_WRITE_TOKEN` means no photographs**; logos keep the origin URL. - **`isOptimizable` (`@crm/db/images`) is the whole rule**: `next.config.ts` allow-lists only our Blob host (a wildcard makes us an open image proxy), and a mirrored **SVG is still refused**. - **Faces are not optimized** — `AvatarImage` skips `` because Radix probes the URL itself, doubling fetches. - **A photograph only comes from a source already tied to this person** — `lib/portrait-sources.ts`: their LinkedIn, their GitHub, their employer's team page, each keyed on an identifier already on the record. - **There is no image search by name, and there must never be.** Nobody audits a face. **Guess where to look, never what you will find.** ## Two lanes `schedules/dispatch.ts`, split by `DIRECT_KINDS` in `@crm/db/agent-tasks`. | | Kinds | How | Per tick | | --- | --- | --- | --- | | **Visible** | `brand`, `portrait` | Directly — no `receive`, no model | 60, six at a time | | **Research** | everything else | One eve session per row | 12 | **Neither visible kind has anything to decide**, and through a session they queued behind sixty LLM runs for 25 minutes (`test/lanes.integration.spec.ts`). **The row says what the work is; the lane only says whether it needs a conversation.** **Priority**: `brand` 900 · `portrait` 800 · `workspace` 500 · `requested` 300 · `meeting` 200 · `identify` 100 · `sweep` 50 · `companyProfile` 40 · `recheck` 0. The top two are what a rep reads *before* deciding what to open. **`claimDue` sorts what it claims** — Postgres does not order `UPDATE … RETURNING` by its sub-select's `ORDER BY`. ### Dispatch on demand `POST /internal/crm/dispatch` drains **both lanes**; `AgentTriggerService.poke()` calls it after writing any `AgentTask`. - **Fire-and-forget, never awaited** — the row is still the message. - **Both lanes.** Visible-only made them diverge under `eve dev`, where there is no cron: logos resolved instantly while `identify` sat at `attempts = 0` forever. - **Calls the channel's own `send`, not `receive`** — it is already on the crm channel. Principal from `APP_AUTH` (`lib/app-auth.ts`); `taskAuth()` keeps schedule and route from drifting. - **`drainAll` collapses** via `collapsing()` (`lib/pool.ts`) — forty new contacts poke forty times, and `claimDue` hands each a disjoint batch. Per-process; cross-process overlap is leases and `FOR UPDATE SKIP LOCKED`. - **An abandoned sweep is still in flight, and `dispatchHealth()` says so.** The timeout aborts the lanes, which stop between items, but the sweep only leaves `unsettledSweeps` when it truly settles. Until then health reports `running: true` and the endpoint answers `503`, and a new sweep is refused rather than stacked on top of stuck work. - **A slow `start(task)` is reconciled, never failed.** Past `sweep.startTimeoutMs` the lane stops waiting, the record stays *Researching*, and the late session id is attached by `noteSession` when the send lands. Only a send that actually rejects settles the record `FAILED`; a send that never lands is retired by `retireExhausted` after `MAX_ATTEMPTS`. A session id that never reaches the row after `research.link.attempts` raises `unlinkedSessions` in health, so a task running under a session nobody recorded is visible. - **`AGENT_BRIDGE_SECRET` unset refuses rather than opens.** ### `POST /internal/crm/verify-key` Probes a candidate Context key → `valid`/`invalid`/`unknown`. No session, no model, no task row; exists because the API may not call Context. - **The probe is free and chosen to be** — a free-provider address gets a `422` before billable resolution. **Do not point it at a real domain**: ten credits per typo. - **`classifyKey` rejects on `401` and nothing else.** - **The candidate key, never the stored one.** ### Blank fields are filled on the dispatch tick `sweepBlankFacts` (`lib/blank-facts.ts`) applies every pending suggestion whose field is still empty and clears the ones that have stopped saying anything. It runs at the top of `schedules/dispatch.ts`, every minute, over **every contact in the workspace** — it is a database pass with no session, no model, no task row and no credits, so there is nothing to ration and nobody to scope it to. - **Scans 2000 suggestions and fills at most 500 a pass**, and reports what it did not reach (`unscanned`) rather than a clean sweep it did not make. - **Idempotent** — a second pass fills nothing, because those fields are no longer blank. - **The suggestions left are conflicts**, every one against a value already on the record. That number should stay small and reads as work for a rep. - **It does not belong on sign-in, and that is not a preference.** It lived there for one release and never once ran. Two reasons, either fatal: `onSignedIn` fires on `session.create`, so a rep already signed in never triggers it; and `BackfillService` does its work in a detached `void (async () => …)()` after the response, on a Nest API that is a Vercel **serverless function** (`apps/api/api/index.ts`) — the tail of that chain is not guaranteed to run at all. A cron in the agent is the only trigger here that is a fact rather than a hope. ### Stale rows are closed on the dispatch tick `reconcileStaleTasks` (`lib/stale-tasks.ts`) runs before `drainAll`, every minute. Like the blank-field pass it is a database pass with no session, no model and no credits. It closes rows that are provably done or provably dead, so the queue a rep reads is the work that is actually happening. - **It only looks at work that is due.** A `recheck` booked ninety days out is not stale, it is scheduled, and closing it would delete the next look at that contact. - **A live lease is never touched.** That is work in flight, whatever the record says. - **A record that completed after the task started closes the task** with a plain outcome. That is the row the settle path failed to close: research landed, facts were written, and the task sat open holding the contact on *Researching*. - **A dead lease with attempts left goes back to waiting**, and the lease is cleared so the row reads as waiting rather than as running. Exhausted rows are retired by `retireExhausted`, which is the one rule for giving up. - **Idempotent** — a second pass in the same minute closes nothing, because the rows it closed have a `finishedAt` and the rows it released have no lease. - **Scans 200 rows a pass** and reports `unscanned` rather than a clean sweep it did not make. The counts are in `dispatchHealth().staleTasks`, so a growing number of stale rows is visible at `GET /internal/crm/dispatch-health`. **It is a safety net, not the fix.** The settle path still fails to close some rows; this pass keeps that from reaching a rep, and the count it reports is how you see the underlying bug getting worse. ### Backfills Sign-in sweep covers records never looked up (10 credits/company); `ImageMirrorService` in the same sweep re-hosts off-site pictures (free); `backfill:images` fixes enriched records missing only pictures (free); `backfill:facts` is the blank-field sweep above run by hand, with `--dry` to read it first — the cron covers it, so this is for a machine pointed at another database. - **The image sweep keeps "every picture is ours" true**, not true-since-Tuesday. 25 rows/table/sweep. - **Nobody clicks a suggestion into a blank field.** The rule is enforced at the write path, so no queue forms; the sweep is for rows written before it existed and for the field a rep clears while a suggestion is pending. **An unreachable agent leaves the suggestions where they are** — the API cannot apply one itself. - **A finished `portrait` task stands that contact down for thirty days** — that third source costs credits and usually finds nothing. - **No button, deliberately** — a rep cannot know which records predate a resolver. - **The trigger is signing in**, via `databaseHooks.session.create.after` in `packages/auth`; `BackfillService` subscribes with `onSignedIn` and has no router (`@crm/auth` must not import a Nest provider). Five-minute stand-down, and `auto()` returns before work starts. - **The 500-row cap is on the pass, not each query in it.** Deduplicate the union, cut to 500, count `remaining` against the union. ## Evidence, not confidence **No tool accepts a confidence, a score, or a `sourceUrl` offered as proof.** Tools report what they *observed* (`crm.signature-block`, `github.account-identity`) and `lib/evidence.ts` prices it. A model asked to grade its own certainty will, and will be wrong in the direction that looks useful. - **`lib/facts.ts` is the only write path to a contact's fields.** Applies at `VERIFIED`, proposes below it, and enforces three things a prompt cannot: never overwrite a human, never re-offer a dismissal, never write without a primary source. - **A band decides only when there is something to lose.** An empty field is filled by whatever cleared the floor for keeping, whatever the band — approving a sourced guess into a blank is a click that can only say yes, and a rep with four hundred contacts reads none of them. `fillsBlank` is the whole rule: no human value in the way and nothing already found. **`PROBABLE` still means *a rep decides* when the field is already filled**, which is the case where a wrong answer costs something. The dispatch tick applies the same rule to rows that predate it — `sweepBlankFacts`, above. - **Applying settles the field's other suggestions.** They were all offers to fill the same blank, and the sheet shows one at a time — so left alone, accepting one reveals the next, forever. The same rule holds when a rep accepts one (`decideFact`). - **The same value is never offered twice.** A second `PROPOSED` row with a value already waiting is refused at the write path, not deduplicated on read. - **"The same value" is `sameValue`, and a URL is compared as an address, not a string.** `canonicalValue` lowercases, collapses whitespace, and for `http(s)` drops the scheme, `www.`, a trailing slash and the query, and reads `twitter.com` as `x.com`. Compared byte for byte, `…/in/pogrebs/` is not `…/in/pogrebs`, so the record showed a suggestion offering back the URL already in the field — the exact click this whole rule exists to remove. **The stored value is still what the source said**; only the comparison normalises. **The refusal is for offers only** — the check runs after `applies` is decided, so evidence that has reached `VERIFIED` since the offer was made still lands, settles the suggestion it matches, and replaces the older value. Refusing it there left a weaker value on the record with the answer sitting unread beneath it. - **A new fact field goes in `FIELDS` (`lib/facts.ts`) *and* `FACT_COLUMNS`** (`apps/api/src/contacts/contacts.service.ts`). ## Optional by default `lib/capabilities.ts` is the single place that knows what is set: prints it at boot, states it in the session instructions, and gives tools a shared "not configured, retrying will not help" result — **checked before the research budget is charged**. A missing key removes a place to look. **Never an error, never throws.** **`capabilities()` is async** because the Context key is a row; `capabilitiesFrom()`/`markdownFor()` are the pure halves. `contextDevKey()` is the only resolver, and `lib/context-dev.ts` memoises its client on the key string. ## Budget and scheduling - `lib/focus.ts` — per-session budget in `defineState`; running out is a normal ending. **A unit is one metered vendor call, not one credit.** `spend(2)` is what a billable lookup costs: a brand lookup is 10 Context credits, a person enrich is 20. Both charge 2, because the budget rations calls per contact and a session with a budget of 4 must still be able to make two of them. - `lib/tasks.ts` — `claimDue` leases with `FOR UPDATE SKIP LOCKED`. - **`schedules/dispatch.ts` is the only schedule and decides nothing.** "Every N minutes, the oldest ten contacts" belongs in a `dueAt`. - `tools/schedule_recheck.ts` — its `reason` is shown to the rep. ## Three records, no dead ends **Every read hands back the ids of neighbouring records.** Breaking this made the agent ask a rep who had a company open, contacts on screen, to paste an email. | Read (all free) | Hands back | | --- | --- | | `read_crm_history` | the contact's **company id**, their deals, their colleagues | | `read_company_history` | **every contact with id**, deals, threads, meetings, notes | | `read_deal_history` | stage clock and history, **people with ids**, last reply | | `search_crm` | contacts, companies and deals matching typed text | **A preamble or tool result naming a record without its id is a bug** — the only recovery is asking the human. Ambiguity is fine: four Marchettis is four rows with titles. **`search_crm` does no fuzzy matching** — "Marchetti" reaching "Marchetta" is a wrong record about a real person. ### Preambles `lib/preamble.ts` varies on **which record** (each names its neighbours with ids and points at the read to start from) and **who opened it** — a dispatched task is a research pass with a budget, a rep in the sheet is a conversation. Told neither, the agent answered a question with a work plan. `taskKind` is the tell. `task.ts` is the resolver and owns one side effect: seeding `lib/focus.ts`, without which the audit hook files events against nothing. **A fourth record kind** = `sessionPreamble` entry + a read + a `TOOL_VERBS` line (`apps/app/lib/agent-transcript.ts`) + a `COPY` entry (`lib/agent-record.ts`). ### Every session knows who *we* are `composeClosing()` puts a **Who we are** block before the capabilities in every preamble; `lib/workspace.ts` is the only renderer. - **Tiny, enforced by the write path** — `MAX_NARRATIVE` (320) and `MAX_LINE` in `@crm/db/workspace`. It is prompt-cached and precedes every question. - **It says what the context is for** — fit, competitor, partner, or nothing — and **never a pitch**, or the model sells our own product back to us. - **No profile still gets the name line**, plus *do not guess at what we sell*. - **The profile dies with its website** — `readWorkspaceIdentity` returns it only while `website` matches. - **Not a `Company` row** — that needs excluding from every list, facet and join. One `WorkspaceProfile` keyed on `WORKSPACE_ID`. The pass is a `workspace-profile` task using `web_fetch` (no credits), filed only via `write_workspace_profile`, queued by `WorkspaceService.update` on a website change. **A finished attempt stands the sweep down for seven days.** ## What may be read, and what may leave It may read **everything**, including full email bodies — internal single-tenant tool, and a signature block is the best source of a job title there is. The boundary is egress: 1. No customer text in a third-party query. Derived questions only. 2. Nothing from a mailbox into `/workspace` — different lifetime. 3. Nothing sensitive logged. Reading is not logging. `skills/data-boundaries.md` is the agent's copy. Keep them in step. ## Sandbox `agent/sandbox/sandbox.ts`: `bash`, file tools, `/workspace`, **`deny-all` egress on the backend factory** so it cannot be forgotten per session. Costs nothing — `web_fetch` runs in the app runtime, `web_search` at the provider. **Never give the sandbox `DATABASE_URL`.** CRM access is authored tools. A shell with credentials and network is exfiltration-shaped; with neither it is a text processor. ## Team-agent builder and runner `agent_builder` and `agent_runner` are declared subagents with independent instructions, tools and deny-all sandboxes. They inherit nothing from the root. The root built-in `agent` copy tool is disabled; these two named specialists are the only delegation paths for custom agents. - **Creation requires the current `CREATE_AGENT` turn.** Every builder tool checks the purpose and command type in session auth. A normal builder chat cannot create a draft by prompt alone. - **Builder clarification is durable HITL.** The specialist calls eve's built-in `ask_question` directly; descendant input requests are proxied to the root channel, and the same child turn resumes when the user answers. The authored `tools/ask_question.ts` disable override must stay absent. Builder task output is typed as `draft_ready` and carries the immutable version ids only after save. - **Empty never means all.** A version chooses `SELECTED` or `WORKSPACE` record scope. Selected scope requires at least one record tagged in that private conversation; workspace scope is an explicit grant and cannot also list selected records. - **Connections are executable permissions.** Only `google:gmail` and `google:calendar` are accepted, only when connected, and the runner does not query their synced tables unless the deployed manifest includes that source. - **Actions are structured permissions.** `crm.activity.create` separately names `NOTE`, `TASK`, or both. Runtime enforcement never infers a grant from the action's prose summary. Every action is ledgered before execution and keyed by eve's call id for replay safety. - **CRM events are shared domain contracts.** `@crm/db/crm-events` owns the event vocabulary, record kind, and builder-facing description used by the API, builder, and worker. API writes enqueue a durable `agent-event` task; the agent worker alone matches live triggers and creates runs. Do not duplicate event lists in prompts or feature code. - **Versions can have multiple triggers.** The manifest stores a `triggers` array and each entry becomes its own `AgentTrigger` row. Deployment enables every trigger on the approved version, so one agent can react to several independent lifecycle moments without polling or duplicate agents. - **Deployment is the human approval boundary.** Saving produces a private READY version and never deploys it. The review screen shows its triggers, scope, actions, access and exact files. A user's Deploy action pins that immutable version for the team. Scheduled runner sessions use task mode and therefore cannot pause for a per-action approval; the deployed permission and idempotent runtime checks are the boundary. - **Approved instructions are system context.** The runner resolves the pinned version instructions at `session.started`, then calls `inspect_run` for the manifest and current run state. Every runner tool also checks the `team-agent` purpose and revalidates scope and action permission. - **Stopping a run is a row, not a signal.** `agents.cancelRun` settles the `AgentRun` to `CANCELLED` inside one transaction — terminal event, outstanding `AgentAction` rows, audit entry — and *then* pokes `POST /internal/crm/cancel-run`, which calls eve's `cancel({ continuationToken })` for `run:`. **The row is what stops the work**: every run tool refuses a run that is not `RUNNING`, so a cancel that never reaches the agent still means nothing further is written. The poke only stops it spending tokens, and an unset `AGENT_BRIDGE_SECRET` costs exactly that and nothing more. - **Cancelling is not undoing.** eve keeps side effects that already completed, so a note, task or Slack message the run already made stays. The dialog says so. - **Whoever started it can stop it**, plus the creator and workspace admins. `canCancel` is computed server-side per run and is what the button reads, so the control and the 403 cannot disagree. - **`turn.cancelled` settles the run too**, keyed off `run:` in the continuation token, so a cancel from any other path still reaches `finishedAt` rather than leaving the run reading *Running* forever. - **No generic execution surface.** Both specialists disable shell, file, arbitrary web and todo built-ins. The runner also disables direct questions; the builder keeps only `ask_question` for durable clarification. CRM access exists only through their small authored tool sets. Tool code runs in the trusted app runtime; the sandbox remains isolated and deny-all. Runner manifests fail closed when either the explicit record-scope mode or an activity type grant is missing. Versions created before these typed permissions were introduced must be revised and deployed again before they can run. `bun run --filter=agent eval` runs the eve end-to-end builder eval against the real HTTP channel. It creates an isolated private conversation, dispatches the builder, asserts the declared subagent path, verifies a READY side-effect-free manifest, and cleans up its rows. It skips visibly when the database, bridge secret or model credential is unavailable. ## The bridge ``` browser → /eve/v1/* (same origin, session cookie, x-crm-contact header) → apps/app/app/eve/v1/[...path]/route.ts checks the Better Auth session, strips the cookie, mints a 2-minute HS256 token naming the rep + record → AGENT_URL/eve/v1/* → channels/eve.ts repFromCrm() → instructions/task.ts reads attributes.contactId ``` - **The record travels in the token, never in the message.** - **Mounted at `/eve/v1/*`** because that is where `useEveAgent()` looks — no `host`, no CORS, no cross-site cookie. - **The proxy is an enforcement point, not a passthrough** — the agent never sees the session cookie, so if that route did not check, nothing would. - **eve's `jwtHmac()` resolves to `principalType: "service"`** — wrong for a person, and `lib/approval.ts` reads exactly those fields to decide whether to pause for a human. `repFromCrm` maps the subject to a real user principal (`test/channel-auth.spec.ts`). - **`AGENT_BRIDGE_SECRET` unset skips the auth entry rather than opening it.** ### The panel `lib/agent-record.ts` maps a record kind to everything downstream. The panel's own rules — snapshot loading, composer state, thread capture, scrolling — are in **`docs/agent-panel.md`**. It lives in the API and is not a breach of rule one: listing history decides nothing. ## Continuation tokens are namespaced **eve prefixes them with the channel name.** `channels/crm.ts` mints `task:`; `session.waiting` returns `crm:task:`. Minting the prefix ourselves meant matching against `crm:crm:task:` and returning before `completeTask` — research ran, facts were written, sessions looked clean, but **no task reached `finishedAt`**, so contacts sat on "Researching" forever. Hidden because the archived event's `data.continuationToken` is *un-namespaced* while `channel.continuationToken` is not. ### Conversations are kept A record accumulates conversations, and they survive a reload. `AgentConversation` holds the *handle* — the durable eve session id plus its cursor — while the transcript itself is already in `AgentEvent`, written by the audit hook. Nothing is stored twice. - **Resuming.** The panel passes the saved cursor as `initialSession`, so reopening a contact continues last week's thread rather than starting another. eve keeps sessions for 30 days. - **Replay from the start.** `streamIndex: 0` on resume, deliberately — the saved index is where the *last reader* stopped, and a reopened thread should show what was said in it, not only what has happened since. - **Which thread is open lives in the URL** (`?thread=`), like every other view state in the sheet, so a refresh keeps your place and a conversation is a link. It is cleared when the record or the tab changes, by the same rule that drops a half-typed quick-add form. - **Nothing mounts until the list has loaded.** Rendering a thread while the history is still in flight starts a *new* eve session and then remounts onto the real one — which presents as "the history only appears if I refresh". - **The thread the panel landed on is captured once.** Re-deriving "the latest" as the list changes would swap the open conversation out from under a live answer the moment the first save adds a row. `resolveThread` in `lib/agent-transcript.ts` holds the rule, and it is tested. - **The panel is not unmounted when you switch tabs.** It holds a live stream, and Radix drops an inactive tab by default — which aborts the stream mid-answer, so the reply landed in the durable session with nothing attached to receive it. That is the "I went to another tab and the answer never came back" bug, and no amount of re-reading state on the way in could fix it, because the events had been dropped. `keepMounted` on the tab descriptor (`detail-sheet.tsx`) keeps it alive; it renders nothing until the tab is opened once, so flicking through records costs nothing. - **A thread is loaded with `session.snapshot()`, not by hand.** One call returns the complete event prefix, the cursor that continues from it, and a continuation token *if and only if* eve will accept another turn — about 30ms against a hundred-event thread. `lib/agent-session.ts` is the whole of it. What it replaced is worth remembering, because every panel bug of the last day came out of it: a raw `fetch` of `…/stream?startIndex=-1`, parsing the last line into a state machine. The endpoint *follows*, so awaiting the body never returned. The stream opens with a bare newline, so "the first line" was empty — which failed closed to "busy" and locked **every** reopened conversation with "still working on the last question", including ones parked with a perfectly good token. And a read that failed reported the *session* as working rather than reporting itself as broken, so it could never recover. The framework had a documented answer to the exact question that code was asking. Read the guide before hand-rolling the protocol. - **The token is the authority on whether a message can be sent**, not our reading of the events. eve returns one only when the captured prefix ends parked, which is precisely the condition under which the next send lands. - **A turn that has gone quiet for 90 seconds is over, not working.** A restarted agent leaves sessions with no closing boundary; they never park, and treating them as in-flight locks that thread forever. - **An unreachable agent is `offline`, not `working`.** One is a fact about us and the other a claim about the session; stated as the latter it is both untrue and unrecoverable, since the read fails identically next time. The transcript then comes from our own `AgentEvent` archive — which is also what makes a thread older than eve's 30-day retention still readable — and the composer stays usable. - **An ended thread gets a button, not a locked box.** Ended and working both disable the composer and mean completely different things: one is a wait of seconds, the other is permanent. `composerState()` keeps them apart, and an ended thread offers **Start a new conversation**, which moves the picker to a new thread. The transcript stays on screen throughout, and the save hook treats the fresh session as a new conversation by comparing session ids rather than by whether the panel started empty. - **`autoScroll` and nothing else.** The scroller is a state machine (`following-bottom`, `free-scrolling`, `anchored-to-message`) and `scrollAnchor` selects the third, which *stops it following the bottom* — the answer then streams below the fold while the modes fight over each new row. Left alone, `autoScroll` follows the tail while the reader is at the bottom and releases the moment they scroll away, which lights the jump-to-end button. - **One `MessageScrollerItem` per message, not per part.** The row is what the scroller measures; a row per tool call adds a boundary every few hundred milliseconds during an answer. Part ids prefer `toolCallId`, which is stable across a call's streaming states. - **A thread nobody has spoken in is loaded from nothing.** The snapshot query is disabled without a conversation; a brand-new thread mounts with no session and no events, and its first message creates both. - **Scoped to the rep.** Two people asking about the same contact are having two conversations. `ConversationsService` filters on the caller, and a session id in a request body decides which row, never whose. - **Cached the way `api.md` prescribes**: read through `cache-manager` (Redis when `REDIS_URL` is set), write on miss, explicit invalidation on every save. The list is read on every sheet open and changes only when somebody sends a message, which is the shape a cache is for. This lives in the API rather than the agent, and that is not a breach of rule one: listing a record's history researches nothing, scores nothing and decides nothing. The agent owns judgement; the data surface owns filing. ### Turning it on Same value in both processes, from the one root `.env`: ```sh AGENT_URL="http://127.0.0.1:2000" # the default AGENT_BRIDGE_SECRET="$(openssl rand -base64 32)" ``` Then `bun run dev` (the agent serves on `:2000`) and open any contact. **If the Agent tab errors:** | Symptom | Cause | | --- | --- | | `503`, "not configured for this install" | `AGENT_BRIDGE_SECRET` is unset in the app's process | | `401` | The two processes hold *different* secrets | | `502`, "not reachable" | The agent is not running, or `AGENT_URL` is wrong | `eve dev` takes about five seconds to bind, and it listens on **IPv4 only**. That is why `AGENT_URL` defaults to `http://127.0.0.1:2000` rather than `http://localhost:2000`: Node resolves `localhost` to `::1` first, so the `localhost` form fails to connect on a machine where the agent is plainly running — and reports itself as "not reachable", which sends you looking in the wrong place. A variable in `.env` is not enough on its own: Turbo runs in strict env mode, so `apps/app/turbo.json` and `apps/agent/turbo.json` both declare the pair in `passThroughEnv`. Adding a variable and not declaring it produces exactly the `401` above. ### Checking it without a browser `localDev()` accepts anything on loopback, so a bare `curl` to `127.0.0.1` proves nothing about the bridge. Send a non-loopback `Host` to make that entry skip: ```sh curl -s -o /dev/null -w '%{http_code}\n' \ -H 'Host: agent.example.com' \ http://127.0.0.1:2000/eve/v1/info # 401 curl -s -H 'Host: agent.example.com' \ -H "authorization: Bearer $TOKEN" \ http://127.0.0.1:2000/eve/v1/info | jq '.tools.available | length' ``` `GET /eve/v1/info` is the whole inventory — tools, skills, schedules, channels, sandbox, and a `diagnostics` count that is the fastest way to find a file eve silently ignored. ## Watching it work `eve dev` shows every tool call, every result and every token in its interactive TUI, and it is the agent package's default `dev` command. The package's Turbo task is explicitly interactive, so select the agent pane and press Enter to hand stdin to eve. Type `/traces` to open the live trace viewer; Ctrl+Z returns input to Turbo. `dev:headless` keeps `eve dev --no-ui` available when a terminal cannot render the nested TUI. It starts the same server on the same port with the same routes and watcher. In that mode `hooks/activity.ts` narrates the session instead: a line per tool call with its arguments, a line per result with how long it took, the finish reason and token spend of each step, and any failure with its code. - **The lines go to stderr, not stdout.** The TUI's default log mode is `stderr` and it keeps stdout buffered and hidden, so a `console.log` here would be invisible in the mode it exists to serve. Written to stderr the same lines show under `--no-ui`, under the TUI, and in `eve logs`. - **Contents print outside production; the shape prints everywhere.** Which tool ran, whether it worked, what it cost — none of that is anybody's data, so it logs wherever the agent runs. Arguments and replies carry names, addresses and whatever a rep typed, which is the "nothing sensitive logged" rule above, so they are gated on `NODE_ENV`. In production the durable record is an `AgentEvent` row, not a log drain. - **It is not the audit trail.** `hooks/audit.ts` writes every event to `AgentEvent` whatever this prints, and the panel's transcript is read back from there. A change to one is not a change to the other. - **A call is timed by remembering it, because the result event does not carry the tool name or a duration.** The map of in-flight calls is bounded rather than trusted: a turn that dies between request and result would otherwise leak an entry per call, forever, in a process that stays up for days. `eve logs` reads the full record back, but only for an **interactive** `eve dev` — that is the process that writes `.eve/logs/`. Under `--no-ui` the pane is the record, so keep the turbo scrollback rather than going looking for a file that was never written. Two consequences of opting into `dev:headless`, both worth recognising rather than debugging: - **A headless dev process cannot reconnect.** An interactive `eve dev` reconnects to a local server that is already up; `dev:headless` rejects it and exits non-zero. `A dev server is already running for this eve agent` means exactly what it says: use the terminal it is in, or stop it before starting another. - **An orphaned agent holds the port.** If turbo dies without reaping its child, nothing on screen says so and every later `dev:headless` fails the same way. `lsof -nP -iTCP:2000 -sTCP:LISTEN` names the process to kill. ### Nothing is researching, and the queue only grows **`eve dev` never fires schedules on their cron cadence.** It is one line in eve's own [schedules guide](../apps/agent/node_modules/eve/docs/schedules.mdx), and it used to be the single most confusing thing about working on this agent, because every visible part of the loop worked: the Research button wrote its `AgentTask` row, the sheet said *Queued*, the toast promised the page would update — and `schedules/dispatch.ts`, the only thing that turns a row into a session, was never called. Twenty rows sat with `attempts = 0` and `AgentEvent` was empty. Nothing was broken and nothing reported a problem, because nothing ran. **The poke is what makes dev behave like production now**, which is most of why it was widened to both lanes — see [dispatch on demand](#dispatch-on-demand). A row written by the API is dispatched immediately whatever the clock is doing, so the schedule is a backstop rather than the only door. **It only does that when `AGENT_BRIDGE_SECRET` is set**, and that variable is optional: `poke()` reads it first and returns without sending anything when it is unset. So an install that has not set it is back at the paragraph above with no cron behind it either — rows queue, nothing dispatches, and the queue looks exactly like a slow agent. Set it, or run the dispatch below by hand. That leaves two cases where the clock's absence bites even with the poke working, and both look identical to the above: **a task the API did not write** — `schedule_recheck`, which books its own `dueAt` weeks out — and **anything queued while the agent was down**, since a missed poke is not retried. For those, the dev server mounts a one-shot route that runs the exact dispatch path production cron uses: ```sh bun run --filter=agent dispatch # {"scheduleId":"dispatch","sessionIds":["wrun_01KZ…", …]} ``` It drains **both lanes**, exactly as the cron does: up to `VISIBLE_BATCH` (60) `brand` and `portrait` rows six at a time, handled in the process with no session at all, and `RESEARCH_BATCH` (12) research rows, one session each. So the `sessionIds` it prints are the research rows only — a run that resolved forty logos prints an empty list and was not idle. Either way it spends real credits, a vendor call per visible row and a model session per research one; that is the point of it, and the reason it is a command you run rather than a ticker somebody leaves on. Watch the agent pane; the session ids it returns are also streamable at `GET /eve/v1/session/:id/stream`. `eve start` on a built app *does* run the schedule, and so does Vercel, where each `defineSchedule` becomes a Cron Job. Dev is the only place the clock is missing. ### The continuation token you write is not the one you read **eve namespaces a continuation token with the channel's name.** `channels/crm.ts` mints `task:`; by the time `session.waiting` hands it back on the channel context it is `crm:task:`. The `eve` channel's own sessions read back as `eve:` for the same reason. This is worth stating because of how it failed, which was silently and completely. `taskToken()` used to mint `crm:task:` itself, so the handler was matching `startsWith("crm:task:")` against `crm:crm:task:`, getting `null`, and returning before `completeTask`. Nothing errored. The research ran, facts were written, briefs were saved, and the agent pane showed a clean session — but **no task ever reached `finishedAt`**, so every contact sat on "Researching" forever and the sweep re-queued work that had already been done. Twenty-eight tasks, zero finished. Two things made it survive a reading: - **The event data and the channel accessor disagree.** `session.waiting`'s `data.continuationToken` carries the token *as stored* — un-namespaced — while `channel.continuationToken` carries it namespaced. Debugging from the archived event says the token is fine, because from that angle it is. - **Nothing downstream depends on settling.** The record's status is the only thing that notices, and "still researching" is indistinguishable from "researching slowly" until you look at `attempts` in the table. So `taskFromToken` keys on the `task:` marker rather than a fixed prefix, which reads correctly whoever namespaced it and still settles sessions parked before the fix. `test/crm-token.spec.ts` pins all three forms. The general rule: **a channel handler must not assume the token it receives is byte-identical to the one it sent.** Parse for your own marker. ## Tests `bun run --filter=agent test`. The integration specs need `DATABASE_URL` and run against a real Postgres, which is the point — "never overwrite a human" is only true if the transaction says so. The `test/e2e` scripts are separate and you run them by hand. Three of them cost money or change something outside the database, so each one is off unless you switch it on: | Variable | What it does | | --- | --- | | `E2E_LIVE_MODEL=1` | `live-run.e2e.ts` calls the real model. It spends credits. | | `E2E_SLACK_JOIN=1` | `slack-join.e2e.ts` joins real Slack channels. It does not leave them. | | `E2E_LOAD_COUNT` | How many tasks `load.e2e.ts` queues. The default is 300. | Stop the local agent before you run an e2e script against a dev database. A running agent leases the rows the script seeds and retires exhausted tasks the script did not create. **`taskFromToken` keys on the `task:` marker, not a fixed prefix** (`test/crm-token.spec.ts`). **A channel handler must not assume the token it receives is byte-identical to the one it sent.** --- ### Api (`docs/api.md`) # API Rules ## Logging `apps/api/src/logging`. `new Logger(Thing.name)` picks up `ContextLogger`. **Never `console.log`.** Format follows `NODE_ENV` and is not configurable. - **One object, not extra arguments** — `logger.log({ message: "Saved", userId })`; Nest prints a line per argument. - **Errors pass the stack second** — `logger.error({ message }, err.stack)`. Passing the error object drops the trace. - **Never log headers, query strings, or bodies** — cookies and personal data. - **`LoggingModule` stays first in `AppModule`'s imports**, and Better Auth routes log via its own `middleware` option — it mounts before `MiddlewareConsumer`, so `/api/auth/*` never reaches ours. - `requestId` from `RequestLoggerMiddleware` via `AsyncLocalStorage`; `UserContextInterceptor` adds `userId`. Prisma statements are opt-in (`PRISMA_LOG_QUERIES`). ## Intelligence never lives in the API The API serves HTTP, auth, tRPC and the mailbox sync. It does **not** research, enrich, score, summarise, match identities or decide anything about a person or company — not as a fallback, not behind a flag. That is the eve agent in `apps/agent`, which owns the vendor clients, the confidence model and the writes. Nest's half is to report *that something happened*: `AgentTriggerService` writes an `AgentTask` row. A row, not an HTTP call — the agent already leases from that table, so the row survives the agent being down. About to add a vendor client to `apps/api`? You want `apps/agent/agent/lib`. One documented exception, for timing: the exchange-rate fetcher, below. ## One organization, and it is not a tenancy boundary Single tenant. No org header, no org interceptor, no org-scoped cache keys, **no `organizationId` on any CRM record.** A **singleton workspace** exists — Better Auth's `organization` plugin, one row with id `WORKSPACE_ID` (the literal `workspace`, in `@crm/db`, re-exported by `@crm/auth` so the agent needn't depend on it). It answers only: what are we called, who works here, what do we sell. - **The id is a constant, never a parameter.** A function taking an `organizationId` has turned the plugin into tenancy plumbing. - **Signing in is the join; no invite flow.** `ensureWorkspaceMembership` runs in `databaseHooks.session.create.before` and **degrades, never throws** — a throw fails the session create and locks everyone out. The plugin's `invitation` table is unused. - **First account is owner**, and the hook enrols pre-existing users, oldest first. - **Permissions come from `@crm/auth`** — `canRenameWorkspace`, `canChangeRole`, `canConfigureSso`, `canManageCurrency` — enforced by the service *and* used to disable the UI control, so the button and the 403 cannot disagree. `WorkspaceService` adds one invariant: **the last owner cannot be demoted**, with `FOR UPDATE` on the owner rows before counting. - **Reads and writes go through tRPC**, not `authClient.organization.*`. - **Name and website are required at onboarding and cannot be skipped**, in the form *and* in `updateWorkspaceInput`, posting the same `workspace.update` as settings. - **Onboarded state is `onboardedAt` inside the plugin's `metadata` blob**, not a column; `isOnboarded`/`markOnboarded` (`@crm/db/workspace`) are the only accessors, and `markOnboarded` preserves every other key. - **The name starts as `DEFAULT_WORKSPACE_NAME` (`CRM`), a placeholder not an answer.** The header renders ` CRM`, so `workspaceLabel` tests the name rather than comparing to the default. - **The website queues the agent's `workspace-profile` task** and goes through `normalizeDomain`, rejecting null. Stored canonical, so re-saving uncanonically counts as a change and re-queues research. ### Gates in `proxy.ts` Onboarding, then `/onboarding/research` for the Context key. Asked server-side every request. - **`getSessionCookie()` decides signed-in**; pages still resolve the real session via `requireMailboxAccess()`. - **Nothing is cached in a cookie** — both facts revert on a database reset while a year-long marker insists the gate passed. Cache in the API if cost ever matters. - **Both reads run concurrently**, but order decides which is *asked* — the research read is never made while onboarding is open. - **An unreachable API fails open** (`unknown` lets the request through). - **`/sign-in`, `/grant-access`, `/eve` are ungated.** `/sign-in` is the only path a stranger may read; `/` joins it only when `IS_MARKETING` is set. - **There is no way past the key gate but to answer** — Skip stranded installs, every later company sitting `PENDING` with nothing saying so. ### The name is also the URL Served under the workspace slug (`/comp-ai/companies`). **Cosmetic, not tenancy** — every query still resolves through `WORKSPACE_ID`. - **The slug is the plugin's column**, written by `workspaceSlug(name)` (`@crm/db/workspace`) on rename and create. **Never derive it on read.** - `ensureWorkspaceMembership` reconciles it; `RESERVED_SLUGS` prevents collision with a real route (a collision gets `-crm`). - **The proxy is the only thing that puts the slug on.** Missing or stale slugs are redirected with the query string intact, not 404'd; `[slug]/layout.tsx` is the backstop. - `appPath` in `proxy.ts` is the one place `/` resolves for a signed-in rep, which keeps every `callbackURL` correct without knowing about slugs. - **Renaming moves the URL**, so `workspace-form.tsx` replaces the location onto the slug `workspace.update` returns. ## SSO is a row, not a deployment An `ssoProvider` row via Better Auth's `sso` plugin, on Settings → SSO, because a self-hoster's admin cannot redeploy. - **OpenID Connect only** — issuer, client id, secret; endpoints from discovery. No SAML UI: it needs an X.509 cert and SP signing key we have nowhere to keep. - `SsoService` passes `WORKSPACE_ID`, never an input. - **Management is tRPC (`sso.*`); signing in is `authClient.signIn.sso()`.** - **`sso.signInOptions` is the one public procedure in the app.** Every other `sso.*` takes `AuthMiddleware` at the *method*, which is what leaves it open. A client secret is never read back out. - **It is the API's answer, not the app's** — the API serves `/api/auth/*`. - **An install with no Google, no Microsoft and no provider says so**, naming the variables; a read that *fails* falls back to offering Google. - **A provider hides the social buttons, it does not disable them** — `/sign-in?method=google` and `?method=microsoft` still work, so a mistyped issuer cannot lock an admin out. - **Signing in with an IdP does not cost you Gmail.** `needsMailboxGrant` (`@crm/auth`) walls only an account whose sign-in rows are *all* mailbox providers — Google, Microsoft, or both — and none of them granted. `mailboxGrantsNeeded` returns which, so `/grant-access` offers the button they can actually use. - `ALLOWED_SIGN_IN` still decides who gets an account, in `databaseHooks.user.create.before`, for SSO sign-ups too. - `organizationProvisioning: { disabled: true }` — `ensureWorkspaceMembership` already does the join. ## tRPC is the data surface; REST is auth and health only - **One router per module**, `*.router.ts` (the codegen glob), with `@Router({ alias })` and `@UseMiddlewares(AuthMiddleware)`. **No `AuthMiddleware` means public — there is no other guard.** - **Routers are thin**: zod in, service call out; Prisma lives in `*.service.ts`. - Services throw Nest's `HttpException` family; `DomainErrorMiddleware` maps them. - **Filter, sort and paginate in Prisma.** List procedures take `listInput` and return `{ rows, total, facetCounts }`. Never filter a whole table in the browser; never interpolate `sort` into a field name — use `resolveOrderBy`. - **`src/generated/server.ts` is generated *and committed*, and `build` must never regenerate it** — the generator needs GLIBC 2.39, newer than Vercel's build image. Only `check-types` and `dev` run it. If the app cannot see a new procedure, it has not run. ## The OpenAPI document is built at runtime, not committed `GET /openapi.json` serves one document: Nest's own controllers plus a REST bridge under `/rest` generated from every tRPC procedure. Swagger UI renders it at `/`. `createApp` builds both halves and merges them, so nothing is generated at build time and no file is checked in — the document is whatever the routers are. `SwaggerModule.setup` runs **before** `app.init()`, because it registers its Express routes synchronously and Nest's own routing would otherwise shadow them. The factory form defers building the document to the first request, which is what lets it read the tRPC router that only exists after init. Two rules follow for the serverless build: - `@nestjs/swagger` stays in `EXTERNALS` in `apps/api/scripts/build-func.mjs`, because `swagger-ui-dist` resolves its assets from disk at runtime and cannot be bundled. - Anything external must be **vendored**, and vendoring follows non-optional `peerDependencies`, not only `dependencies`. Nest packages declare their runtime needs as peers, so following `dependencies` alone ships a function that throws `MODULE_NOT_FOUND` on the first request. Adding a name to `EXTERNALS` without checking it lands in `.vercel/output` breaks production, and the build stays green. ## Two mail providers, one pipeline `apps/api/src/mailbox` is everything neither Google nor Microsoft owns: `MailboxApiClient` (bearer GET, and the one place a status code becomes an outcome), `SyncStateService` (the `MailboxSync` row), `MailboxTokenService`, `MailboxMatchService`, `participants.ts`, `message-text.ts`, and `ThreadWriterService`. - **`ThreadWriterService.store` is the only writer of `EmailThread`, `EmailMessage` and the `EMAIL` activity.** Gmail and Outlook each parse their own wire format down to one `IncomingMessage` and hand it over; matching, threading, counting and stamping happen once. A second copy of that is how a rule like *reply before you create a company* comes to be true in one inbox and not the other. - **A thread is keyed by RFC message id, not by the provider's thread id.** Root comes from `References` → `In-Reply-To` → own `Message-ID`, so a rep on Gmail and a rep on Outlook land on the same `EmailThread` for the same conversation. Graph only returns `internetMessageHeaders` when `$select`ed and not for every message, so Outlook falls back to `outlook-conversation:` — threading that still holds inside Outlook, just not across to Gmail. - **`MailboxSync.source` is the discriminator** — `calendar`, `gmail`, `outlook`. Each provider's module only ever sees its own, and `sync/mailbox-sync.service.ts` is the one place that dispatches. One cron, one budget: `POST /internal/sync/mailboxes` (`/google` is kept as an alias so an existing deployment's cron keeps working). - **Gmail is forward-only from a `historyId`, Outlook from a timestamp.** Graph has no mailbox-wide delta, so the Outlook cursor is the last `receivedDateTime` seen, re-read with a one-second overlap; `rfcMessageId` is unique, so the overlap costs a duplicate fetch and never a duplicate row. - **Microsoft has no token-revocation endpoint.** `revoke` clears the columns and the UI says the consent itself is removed in the user's Microsoft account. Google's still posts to `oauth2.googleapis.com/revoke` and refuses to clear if that fails. ## Not every address on a thread is a person `externalParticipants` (`mailbox/participants.ts`) is the one gate, discarding **us** (allow-list domains, `User` table), **rep decisions** (`SuppressedContact`, `SuppressedDomain`), and **addresses no human reads**. - **`isMachineDomain` (`companies/domain.ts`) sits beside `FREE_EMAIL_DOMAINS`**; `domainFromEmail` returns null for both, and `companyForEmail` is the only path from address to company — so a caller ignorant of the rule still cannot create one. `.calendar.google.com` covers shared calendars, rooms and ICS feeds. - **Matches the host, never a substring** — `calendar.acme.com` is a real company. - **`isMachineAddress` also catches opaque local parts** (24 hex chars, UUIDs), deliberately narrow: a false positive is a real customer never filed. - **It leaves no row** — a rep may still type these into quick-add; only the *inbox* is barred from deciding. `syncAttendees` filters the same addresses beside `attendee.resource`. - **`isAutomatedAddress` is a separate list about the local part** (`sales@`, `noreply@`), which is why `support@acme.com` never becomes a lead. ## People on a deal `DealContact` is the join, and `deals.attachContact` / `detachContact` / `setContactRole` are the only ways to write it. `deals.contactOptions` is what the picker reads. - **A contact on a deal works at that deal's company**, enforced in the service and not merely by the picker — the same rule as `companies.setPrimaryContact`. - **Attaching is an upsert and re-attaching keeps the role already there**, so a double click cannot blank what somebody typed. - **Detaching removes the row, never the contact.** They stay in the CRM, on the company, with their history. - **`role` is blanked to null, never stored as `""`** — `blankToNull`, as everywhere else. ## Deleting a record is archive first, purge later `contacts.archive`, `companies.archive`, `deals.archive` set `archivedAt`. Nothing else changes — no suppression, no cascade cleanup, no stamp recompute — because the row is still there, just filtered out of every list. - **`archivedFilter(input.archived)` is in every `buildWhere` and every `facetCounts` where.** A list call defaults to `archived: false`; the Archived view is the same procedure with `archived: true`. There is no third "everything" mode — mixing active and archived rows in one table is exactly what this feature removes. - **`archive`/`restore` are a bare `update({ archivedAt })`, nothing more.** They do not touch `AgentTask`, `SuppressedContact`, or `lastActivityAt` — the record is unchanged, only hidden. - **`purge` is the old `delete`.** Same transaction, same suppression, same `AgentTask`/`AgentEvent` cleanup, same `ActivityStampService.recomputeAfterDelete`. Read the rest of this section as `purge`'s contract, not `archive`'s. - **A rep can `purge` straight from the Archived view — retention is a ceiling, not a wait.** `bulkPurge`/the per-record "Delete forever" action call it directly, no different from the cron. **Pruning**: `ArchiveRetentionController` (`internal/archive/prune`, `CRON_SECRET`-gated, `apps/api/vercel.json`) reads `AppSetting.archiveRetentionDays` (`readArchiveRetentionDays`, `@crm/db/settings`, default 180) and calls each service's `purgeExpired(before)` — `findMany({ archivedAt: { lte: before } })` capped at `ARCHIVE.prune.maxBatch` (`archive/archive-config.ts`), then `purge` per row through the ordinary `runBulk`. Settings → General has the day count (`settings.archiveRetention` / `setArchiveRetention`). Below is `purge`'s contract — everything that used to be `delete`'s: - **A purged contact is suppressed by address**, or the sync recreates them from the next thread. `ContactsService.purge` writes `SuppressedContact`, and `externalParticipants` drops it like a `SuppressedDomain` — one filter covering contact creation, company auto-creation and attribution. - **Keyed lower case.** `normalizeEmail` (`crm/values.ts`) is the one canonicaliser, on `contacts.create`, `.update` and the suppression; conflict checks and `allowAgain` match case-insensitively. - **The address comes from the delete itself** (`tx.contact.delete({ select: { email: true } })`), not a read before it — and the 404 is that statement's own `P2025` through `translate`. - **Adding them back lifts the suppression** via `allowAgain` **inside the write's transaction**. Never automatic. - **Purging a company does not suppress its domain** — its people survive with no company, and domain suppression stays the explicit Settings → Connections control. - **Clear `AgentTask` and `AgentEvent` yourself** — they carry `contactId`/`companyId` with no foreign key, so nothing cascades. - **Recompute `lastActivityAt` on exactly the records the purge reached.** `ActivityStampService.targetsOf(where)` collects them *inside* the transaction (the evidence is what gets deleted); `recomputeMany` restamps. A company's `where` must follow its deals: `{ OR: [{ companyId }, { deal: { companyId } }] }`. `recomputeAll()` is for a purge only. - **Recompute after commit, logging rather than throwing** — the row is already gone, and a raised error makes the browser skip invalidation and retry into a 404. ## Money A deal is sold in one currency and reported in another, and **only `baseAmount` may ever be summed**. The rules — `baseCurrency`, `countedWhere`/`pendingWhere`, frozen rates, the supported currencies, the keyless feed, and why the fetcher is the one documented exception to *no intelligence in the API* — are in **`docs/currency.md`**. Read it before touching any amount, total, chart or rate. ## Freshness: invalidate the query, don't disable the cache - **Invalidate in `onSuccess` through `useCrmCache()`** (`lib/trpc/cache.ts`), never by listing keys at the call site. Say what changed — `cache.deal(id)`, `cache.company(id)`, `cache.contact(id)`, `cache.activity()`. **A new mutation adds a call there, not a new list of keys.** - **A deletion is `cache.removed(ref)`** — one wide fan-out, and the only place `refetchType: "none"` is right: the deleted record's `byId` query is still mounted while the sheet animates shut, so refetching reads a 404 into the closing sheet, while leaving it alone serves 30s of a dead record from cache. - **`{ settle: "record" }`** for inline editors, so the field's spinner clears without waiting for the table. - **Infinite queries need `pathKey()`, not `queryKey()`** — the latter stamps `{ type: "query" }` and silently cannot match `{ type: "infinite" }`. `activities.timeline` is read both ways. - **`cache-manager` is per-value and opt-in**, not an interceptor; `AuthService.getProfile` is the model. - **Background writes need polling, not invalidation** — `refetchInterval` while `PENDING`/`RUNNING`, via `isEnriching()` and `ENRICHMENT_POLL_MS`. **Lists poll too, not just the sheet.** --- ### Connections (`docs/connections.md`) # Connections — rules for AI agents Covers `/settings/connections` and everything under it: the index, per-connection pages, the add-connection picker, and identity matching. Read `docs/design.md` first. Read `docs/api.md` before touching `apps/api`, and `docs/agent.md` before touching `apps/agent`. The Paper file is `app.paper.design/file/01KZ72S23CM00WXN6S1MP68M03/1-0`, page **crm - grim**. ## The one rule that governs everything here **A connection is a capability. An automation is an agent.** A connection page shows what a connection can do, who it is wired to, and which agents use it. It never owns, authors, or edits an automation. Automations are built in the agent builder chat by tagging the integration, and they live at `/agents/:agentId`. This is already true in code: `builderResource.kind` in `apps/api/src/conversations/conversations.contracts.ts` is `["integration", "company", "contact", "deal"]`, and the composer picker in `apps/app/components/agent-builder/agent-composer.tsx` is labelled "Tag CRM records and integrations" with ids shaped `google:calendar`, `google:gmail`. A new integration becomes taggable by extending that enum's id space, not by building a second authoring surface. If you find yourself adding a toggle, a trigger builder, or an "Edit automation" form to a settings page, stop — that work belongs in the builder. ## Removing a connection is an owner or admin decision A connection is shared. One person disconnecting Slack stops every agent that posts to it and clears the cached channel list for everybody, so `SlackConnectionService.disconnect` asks `canManageConnections` (`@crm/auth`) after `AgentAccessService.assertMember`, and `slack.status` returns `canManage` so the button is disabled for the same people the service refuses. Being signed in is not authorisation: `AuthMiddleware` only proves a session exists. Everything additive — joining a channel, creating one, refreshing people — stays open to any workspace member, because the agent builder needs it and Slack can undo it. ## Connecting Slack is refused at the OAuth endpoints, not in the UI Connecting is the same decision as disconnecting, because `replaceSlackConnection` deletes every other Slack account row: a second person connecting *replaces* the workspace's Slack, and every deployed agent then reads from and posts to whichever Slack they installed. Hiding the button is not enough — `authClient.oauth2.link` is one POST. `slackConnectGuard` (`packages/auth/src/slack-connect.ts`) is Better Auth's `hooks.before`, and it asks the same `canManageConnections` the API does. It covers all three doors: `/oauth2/link`, `/sign-in/oauth2` (Slack is a connection, never a sign-in method, and that endpoint needs no session) and `/oauth2/callback/slack`. The callback is the one that matters — refusing there happens **before the code is exchanged**, so a refused attempt writes no `SlackWorkspaceGrant` user token and deletes no bot token. Google and Microsoft sign in on different paths and never reach the guard. A workspace with no owner and no admin lets any member connect. There is nobody left to ask, and a fresh install must not be locked out of its first connection. ## Direction is the organising idea Every connection declares what it **brings in** and what it **sends**. Use those two words. Not arrows, not "inbound/outbound", not scope tokens. `Sends` earns its place by being reassuring when it is empty. HubSpot reads "Nothing, so nothing here can change HubSpot" — that is the guarantee a migration needs, stated where someone will look for it. ## Destinations: derived or chosen When an agent sends something, its destination is one of two kinds. Getting this distinction wrong is what makes an automation feel like it works by magic. **Derived** — follows from the record, so never ask. The deal's owner. The channel that belongs to this deal. Say so explicitly in the UI ("The person who booked, at the email they booked with. Nothing to choose.") so the absence of a control reads as deliberate. **Chosen** — cannot be inferred, so must be asked exactly once. A standing channel like `#sales`. A naming pattern for channels the agent creates (`#deal-{company}`). A billing contact. Chosen destinations are collected **in the builder conversation** with the Questionnaire, not in a settings form. See "The Questionnaire" below. ## Slack needs two grants, not one A bot token cannot add itself to a private channel. No Slack scope grants that. The workspace must also hand over a **user token** (`xoxp-`), which the app then uses to act as the person who connected Slack. - **Public channel** — the bot self-joins with `conversations.join`. Bot token only. - **Private channel** — only a user token can invite the bot. Both tokens. The bot token lives on `Account`. The user token lives on `SlackWorkspaceGrant`, keyed by Slack team id, because the grant belongs to the workspace and not to the person who clicked Connect. `packages/auth/src/slack-grant.ts` writes it, and `apps/agent/agent/lib/slack-connection.ts` is the only place that reads either. A missing user grant is a capability that is off, not an error. The connection page names it, and the private-channel row falls back to asking a human. **The cached row does not choose the token.** `joinSlackChannel` (`apps/agent/agent/lib/slack-membership.ts`) reads the channel from Slack with `conversations.info` before it picks a path. `slackChannel.isPrivate` and `isMember` are a cache, and a cache is wrong the moment somebody adds the bot in Slack — or the moment a new column arrives with a default, which classifies every existing row as public and non-member until the inventory next runs. A bot that cannot see the channel at all is read as private and not a member, which is the only safe reading. When Slack and the row disagree the row is corrected and a `slack-people-match` task is queued, so the connection page stops showing the stale classification too. **Nothing else calls Slack to read the channel list.** The agent builder reads the cached rows and, when they are older than `SLACK.inventory.staleMs`, queues the same background task rather than waiting on a round trip (`apps/agent/agent/lib/slack-people.ts`). A builder chat that blocks on Slack is a builder chat that is as slow as Slack is. ## Permissions are shown in groups, not one line each Sixteen scopes read as noise. Group them by what they touch — people, channels it can read, messages it can send, channels it can change — and give each group a count of how many reach the whole workspace. `SLACK_SCOPE_GROUPS` in `packages/auth/src/slack-scopes.ts` owns both the grouping and the plain wording. The catalogue is the single source. `auth.ts` builds its scope request from it and the page renders from it, so the screen cannot promise something the app never asked for. ## Identity matching is connection-level "Message whoever owns the deal" needs a CRM user *and* a Slack member. Those are two identities. Match on email, show the result, and let a human fix it. An unmatched person must be allowed. The automation stops and says so rather than guessing at a similar name — and the connection page surfaces the count so the gap is visible before it bites. ## The Questionnaire The strongest pattern in this work. When the builder cannot derive something, it asks in chat rather than failing or guessing. Reuse `packages/ui/src/components/questionnaire.tsx` — `Questionnaire`, `QuestionnaireItem`, `QuestionnaireTitle`, `QuestionnaireDescription`, `QuestionnaireChoices`, `QuestionnaireChoice`, `QuestionnaireActions`, `QuestionnaireSubmit`. It is wired through `agent-clarification-composer.tsx` and `builderQuestionResponseInput`. Rules for a question: - Ask only what cannot be derived. Say which parts you already worked out. - Title is the question in plain words. Description says why it is being asked ("This one is a standing channel, so it is the same every time"). - Choices carry a fact that helps decide — member counts, last activity — not decoration. - Always offer an escape ("Search for another channel"). - Footnote the constraint that shapes the list ("Only channels Slack has been added to"). - The submit label restates the answer: "Use #sales", not "Continue". ## Failure is on the surface Nobody owns these automations day to day, and a broken one currently sits for a week or two. So every connection surface leads with liveness, not configuration: "Last call arrived 31 minutes ago", "1 call needs a look", "3 of 4 matched". An automation you cannot see rot in is one nobody will own. ## Irreversible things are bounded in writing Anything that messages a customer states its own limits next to the switch: "It stops the moment Stripe reports the payment, and never messages the same deal twice in a day." "It runs once per deal." An agent that DMs people about money needs its guardrails legible to whoever turns it on. ## Layout Page metrics are tokens. Never a literal. | Token | Value | Use | | --- | --- | --- | | `--spacing-page-top` | 40px | Top bar to first line of content | | `--spacing-page-bottom` | 40px | Below the last block | | `--spacing-page-inline` | 24px | Viewport gutters | | `--spacing-page-gap` | 24px | Between top-level blocks | | `--spacing-block-inline` | 20px | Inset on every block, **including the page title** | | `--container-page` | 820px | Content column | | `--container-page-wide` | 1120px | Two-column or table pages | | `--container-narrow` | 560px | Empty states, dialogs | | `--container-sheet` | 640px | Edit sheet over a page | | `--height-tool-panel` | 400–640px | Scrollable tool inventories | `--spacing-block-inline` is the one doing real work: a card sits flush at the column edge and carries 20px of its own padding, so card text lands in the same lane as the title on a flat page. The two page types stay aligned without matching each other's chrome. Short single-decision pages (empty state, pre-connect, match people) centre vertically. List pages top-align so nothing jumps when a row is added. ## Cards versus flat Cards are for **discrete, independently-actionable objects** — an agent you can pause on its own, a connection row with its own Manage button, a choice in a picker, a Questionnaire. Flat sections separated by a 1px `--color-dark-border` rule are for **a page describing one thing** — the pre-connect page. The split is by what the block *is*, not by page. Do not card a section; do not flatten a list of separately-actionable objects. ## Brand marks `packages/ui/src/components/brand-logos/` has claude, eve, github, google, microsoft, nextjs, slack, stripe, vercel. **Stripe is there** — use it. Missing from the repo and living only in Paper: **hubspot** (`hubspot icon` `L5E-0`), **docusign** (`docusign logo` `L5Z-0`), **ergo** (`ergo logo` `KJJ-0`). Extract their path data into `brand-logos/*.tsx` shaped like `stripe.tsx` when building these screens. Until a mark exists, `EntityLogo` falls back to an initials monogram on `bg-muted` — that is correct behaviour, not a placeholder to design around. Only icon marks are used at these sizes; the wordmarks stay unused. Docusign is spelled with a lowercase s since its 2024 rebrand. The note-taker is **Ergo**. ## MCP connections Clerk, Cloudflare, Linear, Neon, PlanetScale, Prisma, Supabase, and Vercel use the shared MCP connection flow. An owner or admin chooses read-only or read-and-write access. The API stores OAuth tokens in encrypted form. The API never returns tokens to the browser. The agent builder lists only connected platforms. Each deployed version stores exact tool names and fingerprints. The deployment review shows each tool and its call kind. Changed tools stop until a new version receives review. Disconnecting a platform blocks the next tool call. The agent resolves MCP tools from the deployed version at session start. The model never receives disconnected, changed, unreviewed, or undeployed tools. Model-facing results stop at 16 KiB. Archived results stop below 32 KiB. The chat compiles every connected provider result into the shared provider card. The compiler handles arrays, common collection objects, records, and short scalar results. The compiler removes secret fields, large text fields, unsafe links, and raw JSON walls. Known marks render automatically. Unknown marks use the approved initials fallback. Provider-specific result components are not required for new catalog entries. Supabase read-only access also uses the hosted server's `read_only=true` mode. Treat SQL execution, logs, keys, deployments, secrets, billing, and branch changes as risky. Vercel accepts only reviewed MCP clients. Keep the Vercel connection unavailable in production until Comp AI CRM receives approval. --- ### Crm Plan (`docs/crm-plan.md`) # Plan — Agentic CRM (HubSpot replacement) A lightweight, opinionated CRM for our sales team. Companies are the top-level object; Contacts and Deals hang off them; everything that happens is an **Activity** on a timeline. Company knowledge is filled in by an agent using [Context.dev](https://docs.context.dev) rather than typed by a human. This is the build plan: decisions, data model, contracts, conventions and phased delivery. Read it with [`AGENTS.md`](../AGENTS.md), [`design.md`](./design.md) and [`api.md`](./api.md). --- ## 1. The decisions that stop this becoming a nightmare Five choices carry the whole design. Everything else follows from them. 1. **Core fields are real columns. Custom fields are one JSONB column.** Not EAV. A table showing eight custom columns must not become eight joins. §5. 2. **One property registry drives columns, filters, forms and the detail sheet.** System fields and custom fields are described by the *same* metadata, so adding a column to the CRM is a data change, not four code changes. §5. 3. **Permissions are enforced in the tRPC middleware and again in the service.** The UI hides buttons; it never guards anything. Row-level ownership is a service concern because no statement grammar can express it. §6. 4. **Sheets, not inner pages.** A row click opens a sheet keyed by a URL param. Back button closes it, links are shareable, and we never build a `/companies/[id]/contacts/[id]/edit` route tree. §10. 5. **Everything that lists data is the same `DataTable`.** Ported verbatim from `mvp`, driven by nuqs, paginated server-side. §10. ### Choice table | Decision | Choice | Why | | --- | --- | --- | | Tenancy | Single tenant, no organizations | Repo is already built this way (`20260731160000_remove_organizations`); internal tool, Google-only sign-in. | | Hosting | Both apps on Vercel | §2 — with the serverless consequences handled explicitly, not discovered later. | | Data fetching | tRPC end to end, prefetched in RSC | Type-safe Prisma→cell; first paint is already filtered. | | Table state | nuqs, server-side page/sort/filter | Shareable URLs; 100k rows stay fast. | | Data table | mvp's `DataTable` into `packages/ui` | `design.md` makes `/packages/ui` the source of truth; `simple-table` and `card-table` already live there. | | Custom fields | JSONB + `PropertyDefinition` registry | §5. | | Associations | Many-to-many, three typed join tables, labels + primary flags | HubSpot parity; a contact really can sit on two accounts. §4. | | Lifecycle / lead status | First-class enum columns on Company and Contact | They drive filters and reporting; HubSpot parity. §4. | | Last activity date | Denormalised `lastActivityAt` column, written with the activity | It's a sortable column on every list; a subquery per row would be the slowest thing here. §4. | | Saved views | `SavedView` row storing a serialised query string | Falls out of URL-as-state for almost nothing. §11. | | RBAC | Better Auth `admin` plugin + `createAccessControl` | §6. No orgs, so the org plugin's role model doesn't apply. | | Attachments | Vercel Blob, client uploads, presigned reads | §7. | | Speed | Redis for sessions, property defs and aggregates + RSC prefetch | §8. | | Forms | `react-hook-form` + `zodResolver` + `@crm/ui` Field primitives | §11 — matches mvp exactly. | | Effects | Banned, lint-enforced, `useMountEffect` escape hatch | §12. | | Comments | None in new code | §12. | ### Judgment calls worth your veto - **`UNQUALIFIED_TO_BUY` is terminal, not linear.** You listed it between two active stages, but it's a disqualification. Treated like `CLOSED_LOST`: excluded from pipeline value and forecast, shown under a Closed tab. Say the word if you want it mid-funnel. - **Reps read everything, write only what they own.** Sales teams need visibility; letting anyone edit anyone's deal is how data rots. Activities are the exception — anyone can log an activity against any record. - **`Company.domain` and `Contact.email` are unique.** Duplicates are the #1 way a CRM rots, and it makes the HubSpot import idempotent. Cost: two people behind one `info@` need separate records. - **Not everything is a custom property.** The sales-critical fields are real columns with real constraints. Custom properties exist for the team's own additions. Making everything dynamic is the Salesforce trap. - **Many-to-many associations everywhere, with labels and primary flags.** Your call, over my earlier suggestion to flatten them. It costs a join on every company column and turns the company timeline into a union (§4) — both handled, neither free. In exchange the model matches how you actually sell, and we never hit the wall where a contact genuinely sits on two accounts. - **No Tickets, Quotes, Subscriptions, Payments, Orders or Payment links.** The record sidebars in your screenshots are mostly empty `(0)` cards. Rebuilding those is rebuilding HubSpot. §15. ### Repo issues found while surveying - **`docs/review.md` does not exist** but `AGENTS.md` says "Always review" it. Every agent session currently starts by failing to read a required file. - **`docs/api.md` describes a multi-tenant world this repo doesn't have.** The org-context and per-org cache-generation sections came from the MVP; there are no organizations here. The logging rules apply verbatim and must stay. Trim the rest in Phase 0. - `apps/app/make-session.tmp.ts` is a stray scratch file. - The existing codebase is heavily commented. The no-comments rule applies to new code; existing comments are left alone rather than churned. --- ## 2. Hosting on Vercel, and what it costs us Vercel is a serverless platform. Three things about this stack break there if we don't design for them, and all three are cheap to handle now and expensive to retrofit. **The API runtime changes from Bun to Node.** `apps/api` currently builds with `bun build --target=bun` and runs `bun src/main.ts`. Vercel has no Bun runtime. NestJS deploys as a single Node serverless function via `@vercel/node`, with a cached bootstrap so warm invocations reuse the Nest app: ```ts let cached: Promise | null = null; export default async function handler(req: IncomingMessage, res: ServerResponse) { cached ??= bootstrap(); const app = await cached; app.getHttpAdapter().getInstance()(req, res); } ``` Keep the Bun scripts for local dev; add Node build/start scripts for Vercel. `bun test` stays as-is for local test runs. **Fire-and-forget dies on serverless.** The function freezes once the response is sent, so `void this.enrichment.enqueue(id)` silently never finishes. Use `waitUntil` from `@vercel/functions` — the same primitive mvp already relies on per `docs/api.md`. Enrichment is a ~7–18s Context.dev call, comfortably inside the function budget. When volume outgrows it, move to a durable queue; the `EnrichmentService` interface stays the same either way, which is the point of putting it behind a service in the first place. **Postgres connections exhaust.** Every concurrent invocation is a new client. Use a pooled connection string for the app and a direct one for migrations: ``` DATABASE_URL="postgres://...pooler...?pgbouncer=true&connection_limit=1" DIRECT_URL="postgres://...direct..." ``` with `directUrl` in the Prisma datasource. Neon or Supabase both give you this; so does Prisma Accelerate. Getting this wrong produces "too many connections" under load and nothing before it. **Redis must be serverless-friendly.** Provision Upstash through the Vercel Marketplace. `@keyv/redis` (already a dependency) speaks the Redis protocol, which Upstash supports over TLS; Fluid Compute keeps the connection alive across invocations. ### Deployment shape Two Vercel projects from one repo: `crm-app` (root `apps/app`) and `crm-api` (root `apps/api`). Turborepo remote caching on. `apps/app` rewrites `/api/*` to the API deployment, so the browser stays same-origin and the session cookie just works. --- ## 3. Architecture ``` browser │ same-origin /api/trpc ▼ apps/app (Next.js 16, RSC) Vercel project crm-app ├─ app/api/[...path]/route.ts ──proxy──▶ crm-api ├─ server components: getServerTrpc() → prefetch → HydrationBoundary └─ client: useTRPC() + TanStack Query, nuqs for all table + sheet state │ apps/api (NestJS, Node function) ▼ Vercel project crm-api ├─ /api/trpc routers: companies contacts deals activities │ properties attachments users dashboard ├─ /api/auth/* Better Auth (Google) + admin plugin (RBAC) ├─ /api/blob/upload client-upload token handshake ├─ services Prisma, property validation, ownership guards └─ enrichment Context.dev via waitUntil │ │ │ Postgres (pooled) Redis (Upstash) Vercel Blob packages/db sessions, defs, attachments aggregates ``` ### Dependencies to add | Package | Where | Version | | --- | --- | --- | | `@trpc/server`, `zod` | `apps/api` | `^11.18.0`, `^4.4.3` | | `@trpc/client`, `@trpc/server`, `@trpc/tanstack-react-query` | `apps/app` | `^11.18.0` | | `@tanstack/react-query`, `@tanstack/react-query-devtools` | `apps/app` | `^5.101.2` | | `nuqs` | `apps/app`, `packages/ui` | `^2.8.9` | | `zod`, `@hookform/resolvers` | `apps/app` | `^4.4.3`, latest | | `@vercel/blob`, `@vercel/functions` | `apps/api`, `apps/app` | latest | | `keyv`, `@keyv/redis` | `packages/auth` | matching `apps/api` | | `context.dev` | `apps/api` | latest | `nestjs-trpc@^2.13.0`, `@keyv/redis`, `cache-manager`, `react-hook-form` and `better-auth@^1.6.25` are already present. --- ## 4. Core data model Auth models are owned by Better Auth and regenerated by `bun run auth:generate` — never hand-edited. `User` gains only back-relations (`CompanyOwner`, `ContactOwner`, `DealOwner`, `ActivityAuthor`, `AttachmentUploader`, `SavedViewAuthor`) plus the `admin` plugin's fields (§6). `SavedView` below references `PropertyEntity`, which is declared in §5 — Prisma schemas are order-independent, so the split across sections is presentational only. It is all one `schema.prisma`. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` **`Activity.companyId` is denormalised, and many-to-many complicates it.** It is stamped with the *primary* company of whatever the activity hangs off, so the common case — one company, one timeline — stays a single indexed range scan. But under M2M an activity on a deal shared by two companies belongs on both timelines, and one stamped column cannot say that. The company timeline is therefore a union: ```sql WHERE "companyId" = $1 OR "dealId" IN (SELECT "dealId" FROM "companyDeal" WHERE "companyId" = $1) OR "contactId" IN (SELECT "contactId" FROM "companyContact" WHERE "companyId" = $1) ``` Every branch is indexed, and a company with a few hundred associated records resolves in single-digit milliseconds. The denormalised column still earns its place: it keeps the overwhelmingly common single-company read on one index, and it is what `lastActivityAt` propagation walks. If a company ever accumulates thousands of associated records, the fix is an `ActivityCompany` fan-out table written alongside the activity. Do not build it now — measure first. Noting it here so the upgrade path is deliberate rather than a rewrite. **`lastActivityAt` is denormalised too, and this one is not optional.** Your HubSpot screenshots show *Last activity date* as a visible, sortable column on every list and in every record's Data highlights. Computed as a correlated `MAX(activity.createdAt)` subquery it would run once per row per page — the single easiest way to make this CRM feel slow. It is a real indexed column, written by the same service method that writes the activity, and backfilled by the seed. Same argument for the `createdAt` indexes: *Create date* is the default sort on the companies list. **Lifecycle stage and lead status are first-class, not custom.** Both objects carry `lifecycleStage`; contacts and companies carry `leadStatus`. They drive filters, pipeline reporting and the Data highlights strip, so they get enum columns with indexes rather than JSONB. Changing either writes a `LIFECYCLE_CHANGE` activity and stamps `lifecycleStageChangedAt`, which is what lets the record header say "Customer for 21 hours" and offer history — the timeline *is* the history, so there is no second audit table to keep in sync. **Associations are many-to-many by default**, matching HubSpot — a contact can sit on several companies, a deal can span several, and a deal has as many contacts as it has humans in the room. This was your call over my earlier recommendation, and it is the right one for a company selling to groups and subsidiaries; the rest of this section is what it takes to do it without paying for it later. **Three typed join tables, not one polymorphic edge table.** A generic `Association { fromType, fromId, toType, toId }` looks tempting and is the same mistake as EAV, one level up: no foreign keys, no cascade deletes, no Prisma relations, and every query hand-written. There are exactly three pairings, that number is bounded, so `CompanyContact`, `CompanyDeal` and `DealContact` each get real columns and real integrity. **There are two independent notions of "primary", and conflating them is a bug.** HubSpot only models the first: - `isPrimaryCompany` — *this contact's* (or *this deal's*) main company. Drives the Primary badge and the "Company" column on list views. - `isPrimaryContact` — *this company's* main point of contact, which you asked for in the original brief. Same for a deal's lead contact. They point in opposite directions across the same row, so `CompanyContact` carries both flags. **Postgres enforces one primary per side; the application does not.** Prisma cannot express partial unique indexes, so these go in a hand-written migration and are not optional — "two primary companies" is exactly the corruption that makes a CRM untrustworthy: ```sql CREATE UNIQUE INDEX company_contact_one_primary_company ON "companyContact" ("contactId") WHERE "isPrimaryCompany"; CREATE UNIQUE INDEX company_contact_one_primary_contact ON "companyContact" ("companyId") WHERE "isPrimaryContact"; CREATE UNIQUE INDEX company_deal_one_primary_company ON "companyDeal" ("dealId") WHERE "isPrimaryCompany"; CREATE UNIQUE INDEX deal_contact_one_primary_contact ON "dealContact" ("dealId") WHERE "isPrimaryContact"; ``` Promoting a new primary is a two-statement transaction: demote the old, promote the new. The index turns a race into a failed write rather than silent duplication. **`Deal.companyId` was required and is now gone**, so the "every deal belongs to a company" invariant moves into the service: creating a deal requires at least one company association, and removing the last one is rejected. A schema-level `NOT NULL` was doing real work here; losing it means the service has to. **Association labels are a managed vocabulary**, not free text. `AssociationLabel` holds the options per pairing — Decision maker, Champion, Billing contact, Parent company — editable in Settings next to custom properties, validated server-side on write. Free text would give us four spellings of "champion" inside a month. **What this costs, honestly.** Every list that shows a company column now reads through a join filtered to `isPrimaryCompany`, and every per-company count is a join count rather than a foreign-key scan. Both are indexed and fine at this scale. The real cost is the timeline, below. **Timeline semantics.** Past = `occurredAt` set, or `completedAt` set, or a `NOTE`/`STAGE_CHANGE`/`ENRICHMENT` row using `createdAt`. Upcoming = `dueAt > now()` and `completedAt IS NULL`. Overdue = `dueAt < now()` and `completedAt IS NULL`. **Pipeline math.** Open stages are `DEMO_BOOKED`, `QUALIFIED_TO_BUY`, `DECISION_MAKER_BOUGHT_IN`, `CONTRACT_SENT`. Closed are `CLOSED_WON`, `CLOSED_LOST`, `UNQUALIFIED_TO_BUY`. Pipeline value sums `amount` over open stages only. Moving to a closed stage stamps `closedAt` and requires `closedReason` on the two losing outcomes. --- ## 5. Custom properties The part that decides whether this is maintainable in two years. ### Why JSONB and not EAV An entity-attribute-value table is the textbook answer and the wrong one here. Rendering a table with eight custom columns means eight joins or a pivot, on every page of every list. Values live in one `customFields` JSONB column on the row we already selected: zero extra queries, atomic writes, and a GIN index that makes containment filters fast. The trade — no referential integrity on values — is bought back by validating every write against the definition registry server-side. ### The registry ```prisma enum PropertyEntity { COMPANY CONTACT DEAL } enum PropertySource { SYSTEM CUSTOM } enum PropertyType { TEXT LONG_TEXT NUMBER CURRENCY DATE BOOLEAN SELECT MULTI_SELECT URL EMAIL PHONE USER } model PropertyDefinition { id String @id @default(cuid()) entity PropertyEntity source PropertySource @default(CUSTOM) key String label String type PropertyType description String? options Json? required Boolean @default(false) showInTable Boolean @default(false) filterable Boolean @default(true) sortable Boolean @default(false) order Int @default(0) archivedAt DateTime? createdAt DateTime @default(now()) updatedAt DateTime @updatedAt @@unique([entity, key]) @@index([entity, archivedAt]) @@map("propertyDefinition") } ``` `SYSTEM` rows describe the real columns from §4 — `name`, `domain`, `stage`, `amount`, `owner`. They are seeded from a versioned TypeScript constant and upserted on boot, never hand-edited, and their `key`/`type` are immutable. Only `label`, `showInTable`, `order` are editable on a system row. `CUSTOM` rows describe keys inside `customFields`. ### Your existing HubSpot custom fields validate the design The deal record in your screenshots carries *Deal Type*, *Term*, *Deal Segment*, *Forecast Context*, *Forecast Context (manual)*, *Competitors Mentioned*, *Compliance Frameworks* and *Compliance Timeline*; the company carries *VAT ID*, *Record source* and *Frameworks*; the contact carries *Buying Role*, *Assessment*, *assessment-source* and the *Original Traffic Source* drill-downs. Every one of these is a `CUSTOM` definition in the registry — no schema migration, no code change. Two are worth calling out. *Compliance Frameworks* renders as removable tag chips, which is the `MULTI_SELECT` type. And `assesment-source` is misspelled in HubSpot today: the import maps it to a correctly-spelled `key` with the old name preserved as an alias, so we carry the data across without carrying the typo forward forever. Seed these as part of the migration so day one in the new CRM has the same fields the team already uses. ### One registry, four consumers `properties.list({ entity })` returns both kinds in one array. From it: | Consumer | Function | Produces | | --- | --- | --- | | Table | `buildColumns(defs, renderers)` | `DataTableColumn[]` | | Filters | `buildFacets(defs)` | `DataTableFacet[]` for every `SELECT` with `filterable` | | Forms | `buildFormSchema(defs)` | the Zod object for the custom section | | Detail sheet | `buildProperties(defs, row)` | `DetailSheetProperty` rows | Adding a property becomes a row in one table. No code change touches four files, which is the entire point. ### Rules that keep it safe - **`key` is immutable after creation.** Renaming the label is free; renaming the key orphans every stored value. The API rejects key changes outright. - **Delete is archive.** `archivedAt` hides the property everywhere and leaves the JSONB untouched, so an accidental delete is a one-click restore. Hard purge is a separate, explicit, owner-only action that strips the key from every row in a transaction. - **Every write is validated server-side** against a Zod schema compiled from the definitions and cached in Redis, invalidated when a definition changes. Unknown keys are rejected, not silently stored — silent acceptance is how you end up with four spellings of the same field. - **Type changes are restricted to safe widenings** (`TEXT`→`LONG_TEXT`, `NUMBER`→`CURRENCY`). Anything else requires archive-and-recreate, so no write can land against a type that no longer matches the stored data. - **`USER` type values store a user id** and are resolved through the same owner-picker component as system owner fields. ### Filtering and sorting Filtering uses Prisma's JSON path filters against the GIN index: ```ts where: { customFields: { path: [def.key], equals: value } } ``` Sorting is the sharp edge: Prisma cannot `orderBy` into a JSON column. The list service takes a raw `ORDER BY "customFields" #>> ARRAY[$1]` escape hatch, with the key validated against the definition registry before it reaches SQL — an allowlist derived from data we control, never the raw query string. Only properties flagged `sortable` are offered in the sort menu. Free-text `q` searches system text columns in v1. Extending it across custom text properties wants `pg_trgm`; that is a later, measured change. --- ## 6. Role-based access control Better Auth's **`admin` plugin** with `createAccessControl`, imported from the dedicated paths for tree-shaking. The org plugin's role model does not apply — there are no organizations. Roles live on `user.role`. ```ts // packages/auth/src/permissions.ts import { createAccessControl } from "better-auth/plugins/access"; import { defaultStatements, adminAc } from "better-auth/plugins/admin/access"; export const statement = { ...defaultStatements, company: ["read", "create", "update", "delete"], contact: ["read", "create", "update", "delete"], deal: ["read", "create", "update", "delete", "reassign"], activity: ["read", "create", "update", "delete"], attachment: ["read", "create", "delete"], property: ["read", "manage"], member: ["read", "manage"], } as const; export const ac = createAccessControl(statement); ``` | Role | Intent | | --- | --- | | `owner` | Everything, including user management and hard-purging properties. | | `manager` | All records, reassign deals, manage the property schema. No user management. | | `rep` | Read everything; create anything; update and delete only records they own. | | `readonly` | Read everything. No writes except nothing at all. | ### Three layers, and why each exists 1. **tRPC middleware.** `PermissionsMiddleware` reads `meta: { permissions: { deal: ["update"] } }` off the procedure and checks it against the role — mvp's pattern, minus the org lookup. Coarse-grained, and it catches an entire router that someone forgot to guard. 2. **Service-level ownership.** No statement grammar expresses "only rows where `ownerId = me`". A shared `assertCanMutate(record, user)` runs inside the service, before the write. This is the layer that actually protects a rep's deal from another rep. 3. **UI.** `roleCan()` hides buttons a user cannot use. Presentation only. Anything enforced solely here is not enforced. ### Two operational details that bite later - **Role changes lag the cookie cache.** `session.cookieCache` is on with a 5-minute `maxAge`, and the cached user object carries the role. Revoke the user's sessions when their role changes so the next request re-reads from the database. Otherwise a demotion takes up to five minutes to bite. - **Re-run `bun run auth:generate` after adding the plugin.** It adds `role`, `banned`, `banReason`, `banExpires` to `User` and `impersonatedBy` to `Session`. Skipping this produces runtime adapter errors, not type errors. First Google sign-in bootstraps as `owner` if no users exist; everyone after defaults to `rep`, promoted from Settings → Members. --- ## 7. Attachments on Vercel Blob `vercel blob create-store crm-attachments` writes `BLOB_READ_WRITE_TOKEN` into the project. **Not yet created** — see §14. ```prisma model Attachment { id String @id @default(cuid()) pathname String @unique filename String contentType String size Int companyId String? company Company? @relation(fields: [companyId], references: [id], onDelete: Cascade) contactId String? contact Contact? @relation(fields: [contactId], references: [id], onDelete: Cascade) dealId String? deal Deal? @relation(fields: [dealId], references: [id], onDelete: Cascade) activityId String? activity Activity? @relation(fields: [activityId], references: [id], onDelete: Cascade) uploadedById String uploadedBy User @relation("AttachmentUploader", fields: [uploadedById], references: [id]) createdAt DateTime @default(now()) @@index([companyId]) @@index([dealId]) @@map("attachment") } ``` **Client uploads, not server uploads.** Vercel caps a serverless request body at 4.5 MB; a signed contract PDF blows straight through that. Use `upload()` from `@vercel/blob/client` against a `handleUpload` route that authenticates the session, checks `attachment: ["create"]`, and only then issues a token. Server-side `put()` is reserved for small files we generate ourselves. **Store the pathname, not the URL.** Blob URLs carry an unguessable random suffix, which is obscurity, not authorisation — and a CRM holds customer contracts. Downloads go through an API route that checks RBAC and then issues a presigned URL (the CLI exposes `blob presign` and `blob signed-token`; confirm the SDK flag names when implementing). Row payloads never contain a directly fetchable URL. **Deletes are two-phase.** Delete the row, then delete the blob in `waitUntil`. A dangling blob costs pennies; a dangling row is a broken download. The `@crm/ui` `attachment.tsx` component already models the idle/uploading/processing/error/done states the upload flow needs. --- ## 8. Speed: Redis and rendering Three distinct uses of Redis. Conflating them is how caches go stale. 1. **Better Auth secondary storage.** Port mvp's `packages/auth/src/secondary-storage.ts` verbatim. Every RSC render validates a session; on Vercel that is a Postgres round trip per render unless sessions live in Redis. Biggest single win available. 2. **Application cache** via the existing `AppCacheModule`. Cache the things that are hot and rarely change: property definitions, the user list, and dashboard aggregates. Keys are namespaced `crm:v{gen}:{entity}:{hash}`; a write bumps that entity's generation counter rather than deleting keys. 3. **Enrichment dedupe.** A short-lived `enrich:lock:{domain}` key stops two concurrent creates of the same domain from both burning 10 credits. Do **not** cache per-record reads behind Redis in v1. TanStack Query already dedupes and caches client-side, and record freshness after a write matters more than shaving a query. Cache aggregates and registries; leave records live. **Rendering.** Every list page prefetches on the server through `getServerTrpc()` and hydrates: ```tsx const queryClient = getServerQueryClient(); await queryClient.prefetchQuery(trpc.companies.list.queryOptions(params)); return ( ); ``` Parsed nuqs values feed the prefetch, so the first paint is already filtered and sorted — no loading flash on a shared link. Subsequent filter changes stay shallow and are served by TanStack Query. --- ## 9. The agentic layer `apps/api/src/enrichment/` — a typed `ContextDevClient` over `https://api.context.dev/v1` plus an `EnrichmentService` invoked through `waitUntil`. | Trigger | Call | Cost | | --- | --- | --- | | Company created or given a new `domain` | `POST /brand/retrieve` `{type:"by_domain"}` | 10 credits | | Manual re-enrich | same, `maxAgeMs: 0` | 10 credits | | Contact created with a work email, no company | `{type:"by_email"}` → find-or-create Company, link as primary | 10 credits | | CSV import | `POST /utility/prefetch` per row, then retrieve | prefetch free on paid plan | | Research this company | `POST /web/extract` with our schema → `ENRICHMENT` activity | 10 credits | **Field mapping.** `title`→`name` (only when the record was created domain-first and unnamed); `description`/`slogan`→`description`; `logos[]` picked by `mode`+`type`, never `logos[0]` — light+`logo`→`logoUrl`, dark+`logo`→ `logoDarkUrl`, `icon`→`iconUrl`; `colors[0].hex`→`brandColor` (key off `hex`, the name is generated); `industries.eic[0]`→`industry`/`subIndustry`; `address`→`city`/`stateCode`/`country`/`countryCode`; `socials[]`→ the social URLs by `type`; `links`→`pricingUrl`/`careersUrl`. Whole payload → `CompanyEnrichment.raw`, so re-deriving a field never costs credits again. **Error handling.** Every field is nullable — never overwrite a human-entered value with `null`. `400 NOT_FOUND` and `WEBSITE_ACCESS_ERROR` are `SKIPPED`, not `FAILED`, and are not billed. `422` on an email lookup (free or disposable) is `SKIPPED` with no company created. `403` on prefetch logs once and falls through to a direct retrieve. `408`/`429` get one backoff retry, then `FAILED` with the reason in `enrichmentError`. Cold lookups run p50 ≈ 7s, p90 ≈ 18s, so `timeoutMS: 60000` and never in the request path. **Progress in the UI** is a TanStack Query `refetchInterval` that runs while `enrichmentStatus` is `PENDING` or `RUNNING` and returns `false` once it settles — not a `setInterval` in an effect. `CONTEXT_DEV_API_KEY` lives in the API's environment, declared `@IsOptional()` in `env.validation.ts`. No key means enrichment is disabled, logged once at boot, and companies stay `PENDING`. The key is server-side only. ### Deal intelligence (Phase 11) Your deal record shows a *Deal Score* card and an AI-written *Forecast Context* that reads like a summary of the timeline. That is the second agent, and it is the one that makes this "agentic" rather than "enriched": - **Deal score** — a 0–100 health number with a one-paragraph rationale, derived from stage age, activity recency and cadence, contact coverage (do we have a champion *and* an economic buyer?), and what the notes actually say. Adds `dealScore`, `dealScoreSummary`, `dealScoredAt`, `forecastContext` and `forecastContextManual` to `Deal` in a Phase 11 migration — deliberately not in the §4 schema, so the core model ships without waiting on the model work. Recomputed on stage change and on a nightly cron, never in the request path. - **Forecast context** — a rolling summary of the deal's timeline, regenerated when the timeline changes materially. Their record keeps *Forecast Context* and *Forecast Context (manual)* side by side; we do the same, and `forecastContextManual` always wins when set. - **Stalled-deal detection** — an open deal with no activity in N days raises a task for its owner. This is the cheapest high-value automation in the whole product and needs no model at all. Use Claude via the Anthropic API for these; the scoring prompt reads the timeline the CRM already stores, so there is nothing new to integrate. --- ## 10. API contract Every list endpoint speaks one shape, so one table component drives everything. ```ts export const listInput = z.object({ q: z.string().default(""), sort: z.string().default(""), dir: z.enum(["asc", "desc"]).default("asc"), page: z.number().int().min(1).default(1), pageSize: z.number().int().min(1).max(100).default(25), filters: z.record(z.string(), z.string()).default({}), }); ``` Returning `{ rows, total, facetCounts }`, where `facetCounts` feeds the counts in the filter dropdowns. | Router | Procedures | | --- | --- | | `users` | `me`, `list`, `setRole`, `ban` | | `companies` | `list`, `byId`, `create`, `update`, `remove`, `enrich` | | `contacts` | `list`, `byId`, `create`, `update`, `remove` | | `deals` | `list`, `board`, `byId`, `create`, `update`, `setStage`, `remove` | | `activities` | `timeline`, `create`, `update`, `complete`, `remove`, `myTasks` | | `properties` | `list`, `create`, `update`, `archive`, `restore`, `purge`, `reorder` | | `associations` | `list`, `link`, `unlink`, `setLabel`, `setPrimary`, `labels` | | `savedViews` | `list`, `create`, `update`, `remove`, `reorder` | | `attachments` | `list`, `createUploadToken`, `download`, `remove` | | `dashboard` | `summary` | One `associations` router serves all three pairings, taking `{ pairing, fromId, toId }` rather than one endpoint per direction — the alternative is nine near-identical procedures. `setPrimary` runs the demote-then-promote transaction from §4, and `unlink` refuses to remove a deal's last company. `listInput.filters` carries facet values as strings. Date-range facets encode as either a named window (`last_7_days`, `this_month`, `last_quarter`) or an explicit `from..to` pair of ISO dates; the service resolves both to a Prisma `gte`/`lte` on the mapped column. Named windows are resolved server-side so a shared URL means the same thing tomorrow as it does today. Every router carries `@UseMiddlewares(AuthMiddleware, PermissionsMiddleware)` with per-procedure `meta.permissions`. `deals.setStage` writes the `STAGE_CHANGE` activity and stamps `stageChangedAt`/`closedAt` in one transaction — a stage change that doesn't appear on the timeline is a bug, so it cannot be two calls. --- ## 11. Frontend ### Routes — flat, because sheets carry the depth ``` apps/app/app/(app)/ page.tsx dashboard companies/page.tsx contacts/page.tsx deals/page.tsx settings/properties/page.tsx settings/members/page.tsx ``` There are no `[id]` routes. A record opens in a sheet driven by a URL param: `?company=cmp_123`, `?deal=deal_456`, `?new=company`. The back button closes it, the URL is shareable, and we never grow a nested route tree. Deep links render the list server-side and open the sheet on top. ### Sheets Port `responsive-sheet.tsx` and `detail-sheet.tsx` from mvp — `Sheet` on desktop, `Drawer` on mobile, behind one API. `DetailSheet` supplies `Header` / `Stats` / `Properties` / `Section` / `Body` / `Footer`, which maps onto the HubSpot record layout almost one to one — their three-column page becomes one vertical sheet: - **Header** — logo, name, domain link, owner, enrichment chip, then the composer row your screenshots put under the record title: **Note · Email · Call · Task · Meeting · More**. Same five, same order. - **Stats** — `DetailSheetStats` is their *Data highlights* strip verbatim: **Create date · Lifecycle stage · Last activity date**. On a deal, stage replaces lifecycle. - **Properties** — their *About this company* / *Details* panel. System properties first in registry order, then custom, all from §5. Inline-editable in place; no separate edit mode. - **Sections** — the right rail, flattened: Companies, Contacts, Deals, Attachments, Timeline. Each shows a count and an **Add** action, and each renders a `SimpleTable` rather than the full `DataTable` — these are short embedded lists, not paginated views. The associated-deal rows carry an inline stage dropdown, matching the contact record in your screenshot. Because associations are many-to-many, each row also carries a **Primary** badge and an **association label** chip, both editable in place, exactly as your screenshots show. Sections cap at five rows with a *View all associated …* link that opens the full list filtered to that record — a company with forty contacts must not render forty rows inside a sheet. Adding an association is a `cmdk` search over existing records with a create-new fallback, so linking never means leaving the sheet. - **Footer** — save, delete, re-enrich. **Timeline.** Grouped by month with day sub-headers, newest first, matching their *Recent activities*. It gets its own search box, an activity-type filter, a date-window filter and a collapse-all toggle. Upcoming and overdue items pin above the history with a completion checkbox. ### Create buttons live in the page shell header Every list page puts its primary action in `PageShellActions`, which the shell already positions top-right: ```tsx Companies Every account we sell to. ``` The button sets `?new=company`; the same sheet component handles create and edit, distinguished by whether an id is present. One component, one form schema, two entry points. ### The data table, and its rules Ported from `/Users/lewiscarhart/mvp/apps/app/components/data-table/`, rewriting `@trycompai/ui` → `@crm/ui`: | Source | Destination | | --- | --- | | `data-table.tsx` | `packages/ui/src/components/data-table.tsx` | | `table-pagination.tsx`, `numbered-pagination.tsx` | `packages/ui/src/components/` | | `use-table-query.ts`, `build-search.ts` | `apps/app/components/data-table/` | Everything it needs already exists in `@crm/ui`: `table`, `dropdown-menu`, `input-group`, `button`, `spinner`, `lib/row-accent`. The hook stays app-side because it is routing state, not UI. `nuqs` becomes a `packages/ui` dependency since the table owns the `expand` and `hide` params. Behaviour carried over unchanged: - `table-fixed` layout, `overflow-hidden` cells, so long values truncate instead of blowing out the table - sticky header with `bg-background` - search in an `InputGroup` with a leading `Search` icon - one dropdown per facet, radio-group single-select, `all` as the reset option - a tab dropdown for the primary partition, with counts from `facetCounts` - a Sort dropdown listing sortable columns plus asc/desc, and clickable sortable headers with an arrow indicator and `aria-sort` - a Columns dropdown toggling `hideable` columns, persisted in `?hide=` - the whole filter row collapses behind a Filters button under `sm`, with an active-filter count - `useDeferredValue` on rows so typing stays smooth - `ROW_ACCENT` on clickable rows; expandable rows use `ROW_ACCENT_EXPANDABLE` - pagination footer showing range and total Five CRM-specific additions, four of them driven by the HubSpot screenshots: 1. **Columns come from the property registry**, not a literal array (§5). 2. **A row click sets the record's URL param** instead of navigating. 3. **Date-range facets.** mvp's `DataTable` only has single-select radio facets, but every HubSpot list filters on *Create date*, *Last activity date* and *Close date*. Add a `type` discriminator to `DataTableFacet`: `select` keeps today's behaviour, `dateRange` renders a preset list (Today, Last 7 days, This month, Last quarter, Custom) over `@crm/ui`'s existing `calendar` and `popover`. This is the one real extension to the ported component; everything else is reuse. 4. **Saved views as tabs above the table.** HubSpot's "Active Customers", "Intent", "Closed Won Deals". Because the entire table state already lives in the URL, a saved view is a stored query string — `savedViews.create` snapshots `query.search`, and clicking a tab replaces the search params. A `+` tab saves the current filter set. This is nearly free given the architecture and it is the feature the team will notice missing on day one. 5. **Standard cell renderers**, so every list looks like one product: `OwnerCell` (avatar + name, `No owner` when null), `RelativeDateCell` ("Today at 10:07", "Yesterday at 3:20 PM", then absolute), `CurrencyCell` (right-aligned, tabular numerals), `StageBadge`, `LifecycleBadge`, and `@crm/ui`'s existing `empty-cell` rendering `--` for every null — matching HubSpot exactly. ### Default columns Mirroring your screenshots, so the team's muscle memory survives the move. | List | Columns | | --- | --- | | Companies | Name (icon + link), Owner, Industry, Lifecycle stage, Create date, Phone, Last activity, City | | Contacts | Name, Email, Company, Title, Lead status, Lifecycle stage, Owner, Last activity | | Deals | Name, Stage, Close date, Owner, Amount, Forecast category, Last activity | Companies default-sort by Create date descending; Deals by Close date ascending. Everything beyond the first four columns is `hideable`, so the Columns dropdown is genuinely useful rather than decorative. The Company column renders the primary association, with a `+N` chip when a record has more — one line per row, no wrapping, and the full set is one click away in the sheet. `AssociationCell` owns this so all three lists agree. ### Forms `react-hook-form` + `zodResolver` + `@crm/ui` `Field` primitives (`Field`, `FieldGroup`, `FieldLabel`, `FieldContent`, `FieldDescription`, `FieldError`), matching mvp's `person-sheet.tsx`. - **One schema per form, shared with the tRPC input** where the shapes match. - **`Controller` for non-native controls** (Select, Switch, Combobox); `register` for plain inputs. - **`""` means clear, `undefined` means leave alone.** The submit handler maps empty strings to explicit `null`. Collapsing the two makes "clear this field" silently a no-op — mvp learned this the hard way. - **Submit through `useMutation`** with an optimistic row patch, `toast.error` on failure, invalidate on settle. - **The custom-property section is generated** from `buildFormSchema(defs)` and rendered by a `PropertyField` switch on `type`. - **No effect resets the form when the selected record changes.** The sheet body is keyed: ``. --- ## 12. Conventions ### No `useEffect` Banned in app and feature code. The five replacements: | Instead of an effect for | Use | | --- | --- | | Deriving state from state or props | compute inline during render | | Fetching | TanStack Query / tRPC | | Reacting to a user action | the event handler | | Resetting state when a prop changes | `key` on the component | | One-time external sync on mount | `useMountEffect` | `packages/ui/src/hooks/use-mount-effect.ts` already exists and is the only sanctioned wrapper. Concretely in this codebase: sheet open state is derived from nuqs, never synced; search debounce is nuqs' `limitUrlUpdates`, not a timer; enrichment polling is `refetchInterval`, not `setInterval`; form reset is a `key`, not a dependency array. Enforce it with Biome — `noRestrictedImports` banning the `useEffect` named import from `react`, scoped to `apps/app/**` and `packages/ui/src/hooks/**`. `packages/ui/src/components` is already Biome-excluded, so vendored shadcn files (`calendar`, `sidebar`, `carousel`) are unaffected. Verify the `importNames` syntax against Biome 2.5 when wiring it. ### No code comments New code carries no comments. Names, types and small functions do the explaining; anything that genuinely needs prose goes in `docs/`. Existing comments stay — churning them adds diff noise and no value. `biome-ignore` directives and the auto-generated Prisma header are not comments in this sense. Add both rules to `AGENTS.md` so every agent session picks them up. --- ## 13. Phases Each phase ends green on `bun run check-types` and `bun run lint`. **Phase 0 — Plumbing.** Dependencies; `apps/api/src/trpc/` (module, context, auth + permissions + logging middleware, error handler) from the mvp reference; `nestjs-trpc generate` + `exports["./app-router"]`; the `/api/[...path]` proxy; `lib/trpc/*`; `NuqsAdapter` + `TRPCReactProvider`; the data table into `packages/ui`; Biome no-`useEffect` rule; housekeeping on `docs/review.md`, `docs/api.md` and the stray temp file. *Verify: the dashboard renders `users.me` through a server-prefetched tRPC query.* **Phase 1 — Auth, RBAC and hosting.** `admin` plugin + access control in `packages/auth`; `bun run auth:generate`; Redis secondary storage; owner bootstrap; `PermissionsMiddleware`; two Vercel projects; pooled `DATABASE_URL` + `DIRECT_URL`; Upstash; blob store. *Verify: a `readonly` user gets 403 from a mutation, and the button is hidden.* **Phase 2 — Schema, registry and seed.** Models from §4 and §5; one migration; system property definitions seeded from the versioned constant; seed ~15 companies with real domains, ~40 contacts, ~25 deals across stages, ~150 activities including overdue and upcoming. *Verify: `db:migrate`, `db:seed`, inspect in Studio.* **Phase 3 — Companies.** `companies` + `properties` routers; the standard cell renderers; the `dateRange` facet type; list page with the registry-driven DataTable and the default columns from §11; company sheet with header, composer row, Data highlights, properties, contacts, deals; lifecycle stage and lead status editable inline; create button in `PageShellActions`. *Verify: filter by owner and a date window, sort, page; copy the URL into a new tab and get the same view with the sheet open.* **Phase 3b — Associations.** `associations` router; the three join tables with their partial unique indexes; `AssociationLabel` vocabulary and its settings screen; `AssociationCell`; the add/link/label/set-primary UI in the sheet sections. Lands here because Companies is the first surface that needs it and every later phase assumes it. *Verify: a contact on two companies shows Primary on exactly one, and a concurrent double-promote fails the write instead of producing two.* **Phase 4 — Contacts.** `contacts` router; list; sheet; multi-company associations with labels; lifecycle stage, lead status and buying role. Replaces the placeholder page. **Phase 5 — Deals.** `deals` router; list with stage tab, owner facet and close date range; inline stage change writing `STAGE_CHANGE`; forecast category; deal sheet with stage stepper, amount, associated companies and contacts with labels; the at-least-one-company service invariant. *Verify: a stage change appears on the deal and on the timelines of every associated company, and `lastActivityAt` moves on all affected records.* **Phase 5b — Saved views.** `savedViews` router; the tab strip above every table; save-current-filters, rename, reorder, share. *Verify: a saved view survives a reload and a different browser.* **Phase 6 — Activity, tasks, dashboard.** Timeline component and composer; `activities` router; complete and snooze; dashboard with pipeline by stage, closing this month, overdue tasks, recent activity. **Phase 7 — Attachments.** Blob client uploads; `handleUpload` token route; RBAC-checked presigned downloads; attachment sections on all three sheets. **Phase 8 — Custom properties UI.** Settings → Properties: create, edit, reorder, archive, restore; live preview against the table. **Phase 9 — The agent.** `ContextDevClient`, `EnrichmentService`, `waitUntil`, field mapping, status polling, contact-email → company, research brief. *Verify: create a company with domain `stripe.com` and watch logo, description, industry, socials and address populate within ~20s without a reload.* **Phase 10 — Migration and polish.** HubSpot CSV import for companies, contacts, deals, their custom fields **and their associations with labels and primary flags** — HubSpot exports associations as separate files, so the importer runs records first, associations second, and reconciles primaries in a third pass. Dedupe on `domain`/`email`, lifecycle and lead-status value mapping, `assesment-source` typo fix, `lastActivityAt` backfill, prefetch warming, dry-run report first. Then deal board view (`?view=board`) and the `cmdk` quick switcher. **Phase 11 — Deal intelligence.** Deal score, forecast context, stalled-deal tasks (§9). Last because it is worth nothing until the timeline has real data in it. --- ## 14. Open items - **Vercel Blob store — done.** `crm-attachments` (`store_pXY7wYOD7ccPGUNP`), region `iad1`, **private access**, in the `comp-ai-test` team. Not yet attached to a project, because the repo has no Vercel projects yet; attaching it in Phase 1 is what populates `BLOB_READ_WRITE_TOKEN`. Private access confirms §7's design — the CLI requires `--access public|private` and exposes `blob presign`, so RBAC-checked presigned downloads are the supported path, not a workaround. - **Postgres provider** for pooled connections — Neon and Supabase both work; pick one and I'll wire `DATABASE_URL`/`DIRECT_URL`. - **`docs/review.md`** — write it or drop the `AGENTS.md` reference. ## 15. Deliberately out of scope Email sync, calendar sync, sequences and cadences, workflow automation, a reporting builder, and territory rules. Plus every object sitting at `(0)` in your record sidebars: Tickets, Quotes, Subscriptions, Payments, Orders, Payment links and Products. Multiple pipelines are out too — your deals list shows an "All Pipelines" selector over a single "Deals pipeline", so one opinionated pipeline is the honest version of what you actually run. Each of these is a HubSpot feature we are choosing not to rebuild. If sales asks for one, it is a new plan, not a phase. --- ### Currency (`docs/currency.md`) # Currency — read when touching deal amounts, totals, or rates Two amounts, and only one is ever summed. (`_sum: { amount: true }` once added euros to dollars and printed `$2.0M`, silently.) - **`amount` + `currency`** is what the customer pays. Never converted in place. - **`baseAmount` is the only column any total, chart, average or sort may touch**; `fxRate`/`fxRateAt` record how it got there. - **`baseCurrency` says what `baseAmount` is denominated in** — without it a figure converted against a stale base is indistinguishable from a correct one. - **`countedWhere(base)` filters every money aggregate.** Counts are deliberately *not* filtered, so stage groups read counts and sums from separate queries. - **`pendingWhere(base)`** = null `baseAmount` *or* wrong `baseCurrency`, and it **matches null explicitly** — `{ not: base }` is `NULL`, not true, for a null column, so such rows were invisible everywhere. - **Compose with `AND`, never spread** — it contains an `OR`, and so does the deals list's own `where`. - **Every writer of `baseAmount` writes `baseCurrency` in the same statement**: `ConversionService` and `prisma/seed.ts`. - **The rate is resolved once and frozen.** `create`/`update` call `ConversionService.dealFields` when `amount` *or* `currency` changes, reading the unchanged one back in the same call. Converting on read makes a closed quarter change value every morning. - **A missing rate is a null, disclosed not zeroed** — it falls out of `_sum` automatically, and `unconverted` counts those rows so the UI can say *3 deals in CHF are not included*. - **`fillMissing()` never touches a converted deal**; `rerateAll()` is the only thing that overwrites a frozen rate, and only on a reporting-currency change. **`ExchangeRate.rate` = units of `baseCurrency` per unit of `quoteCurrency`**, so `baseAmount = amount × rate` everywhere. The feed quotes the other way; `RatesService` inverts on ingest. - **`MANUAL` beats `FETCHED`** (unique on `(base, quote, source)`), which makes the fetcher optional — Settings → Currencies is the manual path. `resolveRate` refuses a rate ≤ 0. - **Re-rating deduplicates codes through a `Set`** — `currency` was free text, so ` usd ` and `USD` were two groups each updating every variant. - **`MAX_AMOUNT_CENTS`** (`deals.contracts.ts`) is what `Decimal(14, 2)` holds; `baseAmount` is `Decimal(24, 4)` so amount × rate still fits. - **Reporting currency is `AppSetting.reportingCurrency`**, read only through `readReportingCurrency`. - **Codes are validated against `isCurrencyCode` (`@crm/db/currency`), not a regex** — `z.string().length(3)` accepted `ZZZ`. `isWellFormedCurrency` is separate because `Intl` throws on non-three-letter input. - **`CURRENCIES` is eleven currencies and that is all this CRM supports** — USD, EUR, JPY, GBP, CNY, AUD, CAD, CHF, HKD, SGD, ZAR, in array order. `isCurrencyCode` is the single gate for the picker, the feed filter and the stored setting. A refresh **prunes** `FETCHED` rows outside the list and leaves `MANUAL` alone. - **`applyRate` rounds to the *reporting* currency's `minorUnitsOf`**, not to two. The ×100 cents transport cannot represent a three-decimal minor unit; nor can `Decimal(14, 2)`. **Feed: `open.er-api.com`, keyless**, two attempts at 6s. **Check `result`, not just the status** — an unsupported base returns HTTP 200 with `{"result":"error"}`. An unreachable provider warns and returns `{ ok: false }`; only the interactive refresh throws. **The fetcher is in the API deliberately**: a daily rate decides nothing, and `DealsService` needs it *synchronously* to write `baseAmount` in the same transaction. Any judgement about it belongs in `apps/agent`. `POST /internal/sync/rates` is guarded by `CRON_SECRET` (`timingSafeEquals`) and **fails closed when unset**. **A route is not a schedule** — add it to `apps/api/vercel.json` in the same change or nothing runs it. --- ### Design (`docs/design.md`) # Design — Rules for AI Agents - /packages/ui is the single source of truth for all UI. - Always use shared shadcn components from /packages/ui. - Do not override component styles with className. - Do not introduce custom border radii, spacing, colours, shadows, or other visual deviations. - Corners are rounded, from the scale only: `rounded-sm` (4px) for the smallest controls, `rounded-md` (5px) for buttons, inputs and segments, `rounded-lg` (8px) for surfaces that contain controls — popovers, dialogs, menus, table shells. Never a literal radius at the call site. - `rounded-none` is still correct in one case: an element that must join its neighbour edge to edge. The input inside an input group, the middle cells of a selected date range, and the drawer handle are the existing examples. - If a component needs a new variant or style, implement it in /packages/ui so the entire application stays consistent. ## Colour Flat white, neutral greys, and one brand green (`#006B4F`). The greys are untinted on purpose: there is no scene to tint them toward, and a tinted grey without a reason reads as indecision. **Only two things are filled**: `primary` for the action you want, `destructive` for the one you cannot undo. Everything else — secondary, outline, ghost — is a white chip in light and a dark chip in dark. That is what keeps a rep's eye landing on *go* or *stop* and skimming past the rest. `--primary` and `--destructive` hold the **same value in both themes**. A brand colour that changes per theme is not one colour, it is two, and both then need maintaining. The single exception is `--ring`, which lightens in dark: a fill carries the brand, but a ring only has to be seen, and `#006B4F` is too close to the dark background to register. --- ### Environment (`docs/environment.md`) # Environment Setup, DB commands, Google Cloud and the `vercel env pull` hazard: `docs/setup.md`. ## One `.env`, at the repo root `.env.example` **is the documentation** — every variable the repo reads, with a note, and nothing that is not read. `packages/env` walks up to the workspace root and reads `.env`, then `.env.local` on top. - **Real environment variables always win** — the loader never overwrites `process.env`, so Vercel/Docker/CI takes precedence. - **Never add a per-package `.env`.** Four once existed with duplicate `DATABASE_URL`/`BETTER_AUTH_SECRET`; when they drifted the API minted a cookie the app could not verify and the browser bounced between `/sign-in` and `/` forever. - **The root marker is a `package.json` declaring `workspaces`** — stopping at the first `turbo.json` resolves the API's root to `apps/api`. ## A new variable has three homes, not two `.env.example` and — if the API reads it — `env.validation.ts` are the two people remember. The third is **`globalPassThroughEnv` in the root `turbo.json`**, and it is the one that bites: Turborepo hides an undeclared variable from every task it runs, so a deployment that sets the variable perfectly still hands the code `undefined`, and nothing anywhere says so. That is how `MICROSOFT_CLIENT_ID` shipped with the sign-in button quietly missing. **`passThroughEnv`, never `env`** — a secret in `env` is a cache key, which means a cache miss on every rotation and the secret in the cache metadata. The root file's comment has the whole account. ## Required `DATABASE_URL`, `BETTER_AUTH_SECRET`, `ALLOWED_SIGN_IN`. Everything else has a localhost default or is genuinely optional. **`GOOGLE_CLIENT_ID` + `GOOGLE_CLIENT_SECRET`** are the sign-in button *and* the Gmail/Calendar sync — optional, so an SSO-only install needn't create a Google project, but **set together or not at all** (`packages/auth/src/env.ts` throws on one). **`MICROSOFT_CLIENT_ID` + `MICROSOFT_CLIENT_SECRET`** are the same bargain for Entra ID: the other sign-in button *and* the Outlook mail sync, one app registration, the same pair rule. **`MICROSOFT_TENANT_ID`** defaults to `common` and is the only one of the three that is genuinely optional on its own — set it to your tenant's GUID to refuse other tenants at Microsoft instead of at `ALLOWED_SIGN_IN`. There is **no Microsoft equivalent of `hd`**: `tenantId` is the whole of it. **Neither pair is required, but an install wants one of them or an SSO provider** — with none, the sign-in page says so by name rather than rendering nothing. **`ALLOWED_SIGN_IN`** — comma-separated whole domains or single addresses (bare addresses exist for a solo self-hoster, where `gmail.com` would be an open door). **One list, read by the sign-in guard *and* the sync's "which side is external" decision** — if they drifted a colleague would be refused at the door or filed as a lead. **An empty list fails closed.** Parsed on demand. `packages/auth/src/workspace.ts`. ## Where things are - **`API_URL`** (`:3001`) mints session cookies and serves `/api/auth/*`; `next.config.ts` republishes it as `NEXT_PUBLIC_API_URL`, so one variable does both sides. `BETTER_AUTH_URL` is a legacy fallback. - **Editing a file under `packages/` does not restart the API. Restart it by hand.** `bun --watch src/main.ts` refuses to watch outside its project directory and says so once at boot: `File ... is not in the project directory and will not be watched`. So a change to `packages/auth` or `packages/db` leaves the API serving the old module until someone kills it. This cost an hour once: the Slack OAuth scope list was correct in source and stale in the process, and every reconnect kept asking Slack for the old scopes. Running the API from the repo root fixes the watch and breaks Nest, which resolves its tsconfig paths from the current directory and then cannot build its dependency graph. There is no fix in the dev script today. - **`APP_URL`** (`:3000`) is also the trusted-origin and `callbackURL` allow-list. - **Every OAuth `redirect_uri` is built from `API_URL`, never `APP_URL`.** Better Auth serves `/api/auth/*` at `baseURL`, and `baseURL` is `apiUrl`. A redirect built from `APP_URL` points at the web app, where `/api/auth/callback` does not exist, and the provider rejects it with "redirect_uri did not match". This is invisible until someone sets `APP_URL` to a tunnel or a LAN host, at which point the redirect silently becomes that host. `ssoCallbackBase()` is the pattern; `slackRedirectUri` in `auth.ts` once was not. - **`AUTH_COOKIE_DOMAIN`** only for API and app on different subdomains of one parent. - **`AGENT_URL`** is the agent's deployment, server-side only, and **must include the scheme** — validated at boot, or it throws when a task is queued instead. - **`AUTH_COOKIE_PREFIX` is `crm`** (`@crm/auth/cookies`), set on **both** `advanced.cookiePrefix` in `auth.ts` and `getSessionCookie(request, { cookiePrefix })` in `proxy.ts` — one alone redirects every signed-in request. Better Auth's default collides with any neighbour on a shared parent domain, silently: sign-in completes, the row is written, every reader resolves `null`. **Changing it signs everybody out.** ## `IS_MARKETING` — landing page flag, off by default `"true"` serves `app/(landing)` at `/`; anything else sends a signed-out visitor to `/sign-in`, because the page markets *this* product. - **Only the literal `true`** (same shape as `PRISMA_LOG_QUERIES`). - **It decides one thing**: what a stranger at `/` sees. - **`isMarketing()` (`apps/app/lib/env.ts`) reads per request**, so a config change needs no rebuild. Declared in `apps/app/turbo.json` `passThroughEnv`. ## Typed, validated env `apps/api/src/config/env.validation.ts` runs via `ConfigModule.forRoot({ validate })`, and lists every variable the API reads and nothing else. - **Validation runs while `AppModule` is evaluated** — a test must set variables before importing it (see the dynamic `import()` in `test/auth.e2e.spec.ts`). - **The schema is the API's, not the repo's** — `@crm/auth` and the agent read their own. ## Builder chat encryption `CHAT_EVENT_ENCRYPTION_KEY` encrypts new builder chat events before database storage. Generate 32 bytes with `openssl rand -base64 32`. The API and agent must use the same value. Without it, builder chat stores the sanitized public event as plaintext. Removing it hides encrypted replay data without stopping the agent or API. ## Optional: what the agent can do Every outside source is optional and the agent runs with none. A missing key removes a place to look; **never an error, never throws**. `agent/lib/capabilities.ts` is the single place that knows what is set. | Variable | What it adds | | --- | --- | | `PERPLEXITY_API_KEY` | Open-web research with citations; finds a LinkedIn slug | | `GITHUB_TOKEN` | Raises the GitHub rate limit from 60/hour | | `BLOB_READ_WRITE_TOKEN` | Mirrors logos and photos into Blob | | `AI_GATEWAY_API_KEY` | The model. Not needed on Vercel (OIDC) | | `AGENT_BRIDGE_SECRET` | The rep-facing Agent panel — see `agent.md` | `BLOB_READ_WRITE_TOKEN` is also in `env.validation.ts` and `apps/api/turbo.json` because the API and the seed write pictures too. The Next.js app is deliberately excluded — recognising our URL for the image optimizer needs no token. ### The Context key is asked for, not configured **`CONTEXT_DEV_API_KEY` is not a variable here and must not become one.** The key lives in `AppSetting`, is asked for at `/onboarding/research`, and changes on Settings → General — an admin who cannot redeploy cannot set a variable. - **It buys two places to look, not one.** Company brand data by domain, and a person read back from a LinkedIn URL already on their record. Both capabilities in `agent/lib/capabilities.ts` turn on and off with this one key. - **An install that had the variable is asked again**: no migration, no fallback, and **the gate cannot be dismissed**. - **Nothing is lost while waiting.** A keyless `brand` task settles `SKIPPED` *before* anything marks the row `RUNNING`, and `settle` only overwrites `RUNNING` — so the company stays `PENDING`, which the sweep re-queues (`test/keyless-brand.integration.spec.ts`). - **Saving the key runs the company sweep immediately** (fire-and-forget). - **`readContextDevKey` (`@crm/db/settings`) is the only reader**, read live with no cache. An unreadable database is a capability that is off, not an exception. - **The key is never read back** — only whether one is set, and its last four. - **The agent checks it, not the API** (a vendor client in the API is a bug): `settings.setResearchKey` calls `POST /internal/crm/verify-key` and writes unless the answer is *invalid*. **`401` is the only answer meaning the key is wrong**, and **a check that cannot be made is not a failed check** — `unknown` saves anyway and logs it unverified. ## Mailbox sync Always on, on whichever social provider is configured, so there is no extra redirect URI beyond the sign-in one. Scopes are requested at sign-in and gated by `requireMailboxAccess()`, because granular consent lets a user untick one and still sign in. **An SSO rep is not gated** — `needsMailboxGrant` (`@crm/auth`) walls only an account whose sign-in rows are *all* mailbox providers. It cannot be "has the scopes": an SSO rep has no Google or Microsoft account to grant on, and `revoke()` keeps the `account` row, so trying the optional feature and revoking would lock them out. They connect from Settings → Connections, posting the same `linkSocial` call. **One granted mailbox is enough.** A rep with both providers linked who granted Google is not asked for Outlook; `mailboxGrantsNeeded` names the ones still outstanding and `/grant-access` offers exactly those buttons. **Microsoft's granted scopes come back fully qualified** — `https://graph.microsoft.com/Mail.Read`, not `Mail.Read`. `parseScopes` is the one canonicaliser and strips that prefix, so the comparison is against the bare permission everywhere. **Sync is forward-only** — Gmail records the current `historyId` on its first pass and imports nothing, Calendar reads from `now`, and Outlook records `now` as its cursor. **`CRON_SECRET`** (min 16 chars) guards `POST /internal/sync/mailboxes` and `/internal/sync/rates`; both **fail closed when unset**. `/internal/sync/google` is kept as an alias of the first, so an existing deployment's cron does not break on deploy. **Crons live in `apps/api/vercel.json`** — mailboxes `*/5 * * * *`, rates daily. Minute-level schedules need a Pro plan; on Hobby it silently becomes daily. Deliberate absences: **no `GOOGLE_SYNC_ENABLED`** (a switch that can disable a mandatory feature is only ever wrong), **no `GOOGLE_WORKSPACE_DOMAIN`** (`ALLOWED_SIGN_IN` already says who is internal — two sources is how a colleague becomes a lead), **no `GMAIL_BACKFILL_DAYS`**, **no `OUTLOOK_BACKFILL_DAYS`**, **no rate provider variable**. ## Telemetry is on, and turning it off is one variable `CRM_TELEMETRY_DISABLED="1"` — or `DO_NOT_TRACK=1`, honoured identically — and nothing is sent. No client is constructed, so there is no queue waiting to flush later. - **Server side only**, `posthog-node` in the API and the agent. **`posthog-js` appears once, on the `trycrm.ai` landing page**, and nowhere a record can be reached: autocapture on a CRM would lift contact names and deal amounts out of somebody else's database. That one import is gated on `window.location.hostname`, not on `IS_MARKETING` — turning the landing page on for your own domain never loads it. `docs/telemetry.md`. - **There is no variable for the destination.** The project key and host are constants in `packages/telemetry/src/project.ts`. A `phc_` key is write-only — it can send events and read nothing back — so making it configurable would only imply it were a secret. Edit the constants to point somewhere else. - **The install ID is a row, not a file** — `install`, one row, UUID written by the migration. Vercel's filesystem is ephemeral, so `~/.crm/telemetry-id` would count containers. - Declared in `env.validation.ts` as optional, like everything else here. Every event and the never-sent list are in **`docs/telemetry.md`**. ## Not env vars - **Cache TTL** — `DEFAULT_TTL_MS` (60s) in `cache.module.ts`; `CACHE_TTL_MS` overrides. - **Redis** — optional; without `REDIS_URL` the cache is per-instance in-memory, which is wrong for multi-instance. - **Sign-in method** — Google and Microsoft are in code; an IdP is a row (SSO, in `api.md`). --- ### List Building Roadmap (`docs/list-building-roadmap.md`) # List-Building Roadmap A plan to answer the Director of Sales Development's read on the CRM: filtering looks thin for real BDR list-building. Integrations are out of scope, by his own note — this is about fields, filters, and views inside the CRM itself. ## 1. What he saw > "As it stands it looks like a research/automation layer more than a system of > record — I'd be concerned about moving off HubSpot/Salesforce-caliber > filtering without seeing those gaps closed first." > > — Director of Sales Development, first-look feedback He is right about the current state. Today's filtering is this small: - **3** Company filters — owner, industry, enrichment - **2** Contact filters — owner, company - **1** value per filter — no "is any of" yet - **0** saved or shared list views ## 2. Why this is cheaper than it looks Two systems already in the CRM cover most of the fields on his list. Neither is exposed as a filter yet — that is the actual gap, not missing intelligence. **Already built — admin fields.** An admin-configurable field system already backs every record sheet and table: `FieldDefinition` + `FieldOption` + `FieldValue`, with a `SELECT` type for picklists and a `USER` type for a second owner. Account type, segment/tier, territory, lifecycle stage, and lead source all fit this today — no database migration required. **Already built — agent inference.** The agent already scores `seniority` and `function` (persona) per contact from evidence, the same pipeline that fills title and LinkedIn today. It just isn't wired to a column, so it never reaches the table or a filter. ## 3. The gap map Every field from his note, what backs it today, and the plan to close it. ### Companies view All seven rows below are already seeded as `FieldDefinition` rows (`seedCompanyFields` in `packages/db/prisma/seed.ts`) with real option sets. The gap is not the field — it is that none of them show on the table or reach the filter bar yet. | Field he wants | Where it stands today | Plan | Effort | | --- | --- | --- | --- | | Account type (prospect / customer / partner / churned) | Seeded `SELECT` field — Prospect, Customer, Partner, Churned | Turn on table + filter | Small | | Inside Sales / BDR owner (separate from AE / account owner) | Seeded `USER` field, sits beside Owner | Turn on table + filter | Small | | Segment / tier | Seeded `SELECT` field — Enterprise, Mid-Market, SMB | Turn on table + filter | Small | | ICP fit score | Seeded `NUMBER` field; agent scoring is a natural next step, not a v1 requirement | Turn on table + filter | Small | | Territory | Seeded `SELECT` field — AMER, EMEA, APAC | Turn on table + filter | Small | | Lifecycle stage | Seeded `SELECT` field — Lead, MQL, SQL, Opportunity, Customer, … | Turn on table + filter | Small | | Lead source (IB / OB / Event) | Seeded `SELECT` field — Inbound, Outbound, Event, kept separate from the `source` enum that records how a row entered the CRM | Turn on table + filter | Small | ### Contacts view | Field he wants | Where it stands today | Plan | Effort | | --- | --- | --- | --- | | Title filter | Column exists, shown on the table, no filter | Add it as a filter | Small | | Seniority | The agent already infers and scores this per contact — never synced anywhere filterable | Sync into a real column the same way title already syncs; then filter on it | Medium | | Persona | Agent infers this as "function" — same situation as seniority | Same fix as seniority, same pipeline | Medium | | Engagement / activity recency | Last-activity date is tracked and shown, not filterable | Add an "active within 7 / 30 / 90 days" filter | Medium | | Sequence status | No source of truth — this CRM does not run sequences | Needs an outreach-tool connection. Deferred, per your note to set integrations aside. | Deferred | ## 4. The part that actually blocks list-building Adding every field above still will not give a BDR a HubSpot-style list-builder. The filter bar itself is one value at a time. This is the real foundation, and it comes before the fields above pay off. 1. **Filters are single-select** *(Medium)* — a BDR cannot say "VP or Director" or "SF Bay Area or NYC" in one filter today — each filter holds exactly one value. 2. **Admin fields are not filterable at all** *(Medium)* — every custom field — present or future — shows up as a column only. None of it reaches the filter bar without new plumbing. 3. **No saved or shared views** *(Large)* — a working list a BDR builds today lives only in the URL. There is no way to name it, save it, or hand it to a teammate. ## 5. Sequence Four phases, in order — each phase's fields are inert without the phase before it. ### Phase 1 — Filter engine foundation *(ships first)* Multi-select ("is any of") on every filter. Any admin-defined `SELECT` or `USER` field becomes a filter automatically. An activity-recency filter goes in here too. Resolves: unlocks every field below · engagement / activity recency ### Phase 2 — Company segmentation fields Turn on table and filter visibility for the seven Company fields from the gap map — account type, BDR owner, segment/tier, territory, lifecycle stage, lead source, ICP fit score. All seven are already seeded; nothing new to create. Resolves: account type · BDR owner · segment/tier · territory · lifecycle stage · lead source · ICP fit score ### Phase 3 — Contact intelligence, surfaced Sync the agent's existing seniority and persona facts into real, filterable fields. Add the title filter. Resolves: seniority · persona · title filter ### Phase 4 — Saved & shared views *(later, on request)* Name a filter combination, save it, share it with the team. Sequence status waits here too, tied to an outreach-tool connection if that scope reopens. Resolves: reusable BDR lists · sequence status (needs an integration) ## 6. Issues Reported in full, including risks this plan does not remove. 1. RISK — New fields alone do not fix list-building. Filters stay single-select without phase 1. Fix: build multi-select filtering first. Sequence, above, already orders this first. 2. NOT DONE — Custom fields are not wired into any filter today. They render as table columns only. Fix: not done. Planned in phase 1 as automatic facet generation. 3. NOT DONE — Sequence status has no source of truth inside this CRM. Fix: not done. Needs an outreach-tool integration — out of scope now, per your note. 4. RISK — A BDR owner shipped as an admin field will not appear in owner-based reports until someone decides it should. Fix: ship as an admin field first. Promote to a first-class column only if reporting needs it later. 5. UNKNOWN — No user role distinguishes an AE from a BDR today, so no field can default itself by role. Fix: not investigated. Confirm before phase 2 whether role-based defaults matter to him. --- Scoped to the CRM's own data model and filter UI. Integrations, per your note, are out of scope for this plan. --- ### Setup (`docs/setup.md`) # Setup and local development Operational detail moved out of the rule docs. `api.md`, `agent.md` and `environment.md` are what agents read before changing code; this is what a person reads once. ## First run ```sh cp .env.example .env # fill DATABASE_URL, BETTER_AUTH_SECRET, ALLOWED_SIGN_IN docker compose up -d # Postgres, matching .env.example bun run db:migrate && bun run db:seed bun run dev # app :3000, api :3001, agent :2000 ``` Prisma from the repo root: `db:generate`, `db:migrate`, `db:push`, `db:reset`, `db:seed`, `db:studio`, `db:deploy`. `dev` depends on `^dev:prepare`, so every start applies pending migrations and regenerates the Prisma client before a single server boots. That is why the first run needs `db:migrate` only for the seed that follows it. When the database and `schema.prisma` have diverged past what `migrate deploy` can reconcile, `dev:prepare` stops the whole run rather than starting servers against a schema they do not match — reconcile with `db:migrate`, or `db:reset` when the divergence is an edited migration that has already been applied. ## Google Cloud - **Enable the Gmail API and the Google Calendar API** on the project. - **Set the consent screen to User type: Internal** if you are on Workspace. `gmail.readonly` is a *restricted* scope, so an External app needs OAuth verification plus an annual CASA assessment. Going External later means the full review — a decision, not a checkbox. ## The agent bridge ```sh AGENT_URL="http://127.0.0.1:2000" # 127.0.0.1, not localhost: eve dev is IPv4-only AGENT_BRIDGE_SECRET="$(openssl rand -base64 32)" ``` | Agent tab error | Cause | | --- | --- | | `503` | `AGENT_BRIDGE_SECRET` unset in the app's process | | `401` | The two processes hold different secrets, **or** `passThroughEnv` in `apps/app/turbo.json` / `apps/agent/turbo.json` is missing the pair (Turbo is strict-env) | | `502` | Agent not running, or `AGENT_URL` wrong | `localDev()` accepts any loopback request, so `curl 127.0.0.1` proves nothing about the bridge — send `-H 'Host: agent.example.com'`. `GET /eve/v1/info` is the whole inventory, including a `diagnostics` count that finds files eve silently ignored. ## Running the agent The agent package's default `dev` command is interactive `eve dev`. The root Turbo task marks it interactive, so select the agent pane and press Enter before using the eve TUI. Run `turbo run dev:headless --filter=agent` when a terminal cannot render the TUI; that uses `eve dev --no-ui`, and the Turbo pane is the record because only interactive development writes `.eve/logs/` for `eve logs`. Reach for the Turbo task rather than `bun run --filter=agent dev:headless`: the package script alone skips `dev:prepare`, so the agent would start against unmigrated tables. `hooks/activity.ts` is the replacement narration, **to stderr** (the TUI hides stdout), printing shape everywhere and argument contents outside production only. It is **not the audit trail** — `hooks/audit.ts` writes `AgentEvent` regardless. - A second `bun run dev` fails the whole turbo run. - An orphaned agent holds the port: `lsof -nP -iTCP:2000 -sTCP:LISTEN`. ### Nothing is researching, and the queue only grows **`eve dev` never fires schedules on their cron cadence**, and everything visible still works — the row is written, the sheet says *Queued*, and `dispatch.ts` is never called. The poke covers this **only when `AGENT_BRIDGE_SECRET` is set**; unset, `poke()` returns silently and the queue looks exactly like a slow agent. Tasks the API did not write (`schedule_recheck`) and anything queued while the agent was down still need a manual run: ```sh bun run --filter=agent dispatch # exact production path, both lanes, real credits ``` Its printed `sessionIds` are research rows only, so a run that resolved forty logos prints an empty list and was not idle. `eve start` and Vercel do run the schedule. ## `vercel env pull` writes `.env.local`, which wins `.env.local` is the override the loader reads *last*, and `vercel env pull` writes **production** credentials there by default. Pull once and every process silently points at production — not as an error, but as `bun run dev` working perfectly against the live database. On 2026-08-01 eleven migrations landed on Neon from a laptop. 1. **Pull somewhere inert**: `vercel env pull .env.vercel`. 2. **`packages/db/scripts/require-local-db.ts` guards `db:migrate`, `db:push`, `db:reset`, `db:seed`** and takes `ALLOW_REMOTE_DB=1`. `db:deploy` is unguarded on purpose. It reads the root files directly rather than `process.env`, because Bun auto-loads the working directory's `.env` while Prisma's CLI only sees `@crm/env/load`. ## Migrations run on the production deploy, and nowhere else `apps/api/scripts/build-func.mjs` runs `prisma migrate deploy` during the crm-api build, gated on `VERCEL_ENV === "production"`. The schema therefore moves when the release pull request merges and `release` deploys — with the code that needs it, and once rather than once per branch. Preview deploys share the production database: `DATABASE_URL` is a single value across production, preview and development. Until that changes, **a preview of a branch that adds a migration runs against a database without those tables** — it builds, and the pages that touch them fail. Test schema changes locally, where `bun run dev` migrates for you. Before the gate existed the reverse was true and worse: every preview applied its own migrations to the production database, so on 2026-08-07 the live schema ran six migrations ahead of the live code all day. ### `migrate deploy` is not proof the schema is right The build follows the deploy with `prisma migrate diff --exit-code` against `schema.prisma` and shouts in the build log when they disagree. **`No pending migrations to apply` only means `_prisma_migrations` has a row for every file** — it says nothing about what the tables actually look like. They came apart once. A `prisma db push` shaped production from a laptop, the migration rows were recorded as applied without their SQL ever running, and `agentConversationAttachment` went live without its `position` column. Every deploy reported nothing pending, for days, while `conversations.builderById` returned 500. The tell is an object in the database that no migration defines — there was an `agentConversationAttachment_submissionId_createdAt_idx` that appears in no migration file, only in a `db push` of an older schema. Reconciling is one command, and it is worth reading before running: ```sh DATABASE_URL="…" bunx prisma migrate diff \ --from-config-datasource --to-schema prisma/schema.prisma --script ``` ## Secrets hygiene `.gitignore` ignores `.env` and `.env.*` with one negation for `.env.example`, so `.env.bak` is ignored too. `.env.example` ships no secret — placeholders are empty strings, asserted by `packages/env/test/root.spec.ts`. **Generate your own secret**; never reuse one from an example, a tutorial, or another environment. ## Tests ```sh bun run --filter=api test bun run --filter=agent test # integration specs need DATABASE_URL + real Postgres ``` ### The test database rebuilds itself when it drifts `bun run db:test` creates `crm_test` and runs `migrate deploy` on it. The database name must end in `_test`; the suite deletes rows it expects to put back, so it refuses anything else. **`migrate deploy` only applies migrations that are missing. It never removes a table, a column or a constraint the database has and the schema does not.** A `crm_test` built on a branch that was later abandoned therefore keeps that branch's objects forever, and `db:test` used to report `already exists` and move on. The extra objects are invisible until one of them rejects a write, and then the failure names a constraint that appears in no migration and in no schema — a stray `trackedEvent_visitorId_fkey` once failed seven tracking specs this way, on every branch, for as long as the database survived. So `db:test` now checks the database it found and rebuilds it when either is true: - **It holds a migration this branch does not have.** The database came from another branch. The name of the first one is printed. - **It no longer matches `schema.prisma`**, by `prisma migrate diff`. Something was pushed or altered by hand. A rebuild drops the database and re-runs every migration, and it says which of the two reasons fired. Force one with `bun run db:test --reset`. Nothing else in the repo may drop a database, and this may only because the `_test` suffix is checked first. --- ### Telemetry (`docs/telemetry.md`) # Telemetry What this install reports about itself, in full. The decisions behind it are in `adrs/telemetry.md`; this file is the list. **Turn it off** with `CRM_TELEMETRY_DISABLED="1"` or `DO_NOT_TRACK=1` in the root `.env`, then restart. This file is the full account of what is sent; there is no settings page for it, so this is the thing to read. **Where it goes** is not configurable and deliberately so: the project key and host are constants in `packages/telemetry/src/project.ts`. A `phc_` key is write-only — it sends events and reads nothing back — so a variable would only have made a public identifier look like a secret. Point a fork somewhere else by editing that file. Nothing is sent while `NODE_ENV` is `test`, so a test run cannot emit. ## The shape of it - **Server side only, in everything you run.** `posthog-node` in `apps/api` and `apps/agent`. No autocapture, no session replay and nothing client-side anywhere a record can be reached. Autocapture on a CRM would lift contact names, email addresses, company domains and deal amounts out of your database; the simplest way not to hold that is not to collect it. The single exception is the marketing site at `trycrm.ai`, which is a web page and not an install — **The landing page**, below, is the whole of it, and it cannot run on your hostname. - **One identifier**, a UUID generated by the migration that created your database. It is attached to nothing else. `identify()` is never called, there are no person properties, and no user, email or `ALLOWED_SIGN_IN` value is ever sent. - **One event a day**, plus eight one-shot setup events and one event per failure. - **No IP address.** Every event sets `$ip: null` and disables geoip, *and* the receiving project has `anonymize_ips` on so the address is dropped at ingestion. Both are needed: PostHog stamps `$ip` from the incoming connection whatever the SDK sends, and reads a null as "not provided" rather than "do not keep". Point this at a different project and you must set it there too. - **Never blocks and never throws.** Every send is fire-and-forget behind a swallowed handler; a telemetry failure logs at debug and is invisible everywhere else. - **Off leaves no marks.** With telemetry disabled the rollup route returns before the funnel sweep runs: no milestone is recorded, no counter is drained, no rollup day is claimed. Turn it back on and the funnel is still whole, because nothing was quietly consumed while it was off. - **Nothing is sent from a test run.** `NODE_ENV=test` disables it, and `apps/api` and `apps/agent` additionally set `CRM_TELEMETRY_DISABLED=1` in their `test` scripts — because a spec is allowed to reassign `process.env.NODE_ENV`, and one that does was enough to let a real event out. - **An allowlist, in code.** `ALLOWED_PROPERTIES` in `packages/telemetry/src/allowlist.ts` names every property that may be sent. Anything else is dropped before the event is built — not by review, not by convention. ## Events ### `install_daily` Once per install per day, from the API itself. Counts and distributions from grouped queries. Never a value from a row. **It runs in-process and needs no cron.** `TelemetryService` rolls up on boot and then hourly, and the day claim below makes all but the first of those a no-op. That covers both shapes an install comes in: a serverless deployment rolls up on a cold start, a long-running container on the timer. `POST /internal/telemetry/rollup` still exists for a platform cron that would rather drive it, and still refuses without `CRON_SECRET` — it is an anonymous route, so an unauthenticated one would let a stranger drive any install's aggregation. Nothing depends on it: an install that never sets `CRON_SECRET` reports exactly the same. The day is claimed under a row lock on `install` before anything is gathered, so two crons that fire at once produce one event rather than two — and the second does not find the counters already drained. A rollup that fails to send hands the day and the counters back, so tomorrow's run is not suppressed by a send that never happened. **The event carries a deterministic `uuid`**, `stableUuid(install.uuid, "install_daily", day)` — a SHA-256 digest in a v8 UUID — and PostHog ingests a repeat of one it has already seen only once. RFC 4122 defines v5 as SHA-1, and CodeQL rightly refuses to see a hash of the install identity and a weak algorithm in the same expression. Nothing here needs v5, so it is v8, the slot RFC 9562 leaves for a derivation of one's own. The lock stops two processes racing; the id stops the *same* process sending twice, which the release path can otherwise cause. Counting installs survives a duplicate either way, because PostHog's unique math is per install per day, but `tool_calls_total` and the other summed properties would be counted twice, and now are not. **A send reports failure conservatively.** `posthog-node` does not reject on a failed send — its `sendImmediate` catches the error and emits it on the client instead — so `client.ts` enqueues and awaits `flush()`, which does throw, and additionally checks the client's error counter, which any capture in flight can move. Either signal is treated as a failure. That errs toward re-sending a rollup that actually landed, which the deterministic id makes free, rather than consuming a day whose event never arrived, which cannot be recovered. #### The install | Property | What it is | | --- | --- | | `crm_version` | The root `package.json` version, re-read on every boot | | `git_commit_sha` | `VERCEL_GIT_COMMIT_SHA` when there is one; omitted otherwise | | `days_since_install` | Whole days since the first migration ran | | `is_vercel` | Whether `VERCEL` is set | | `node_version` | Major only, e.g. `22` | | `postgres_version` | Major only, e.g. `17` | | `members_bucket` | How many people work here, in bands | | `agent_model_id` | The model chosen on Settings → General, e.g. `zai/glm-5.2-fast` | | `agent_model_context_window` | Its context window in tokens | | `seed_only` | True when every contact came from `bun run db:seed` | #### Capabilities — booleans, never values `cap_perplexity`, `cap_context_dev`, `cap_blob`, `cap_github`, `cap_redis`, `cap_agent_bridge`, `cap_cron_secret`, `cap_ai_gateway`, `cap_google_oauth`, `cap_sso_provider`, `cap_tracking`, `is_marketing`. Each is only whether the key is set. `cap_context_dev` is whether an `AppSetting` row holds one, `cap_sso_provider` whether an `ssoProvider` row exists, `cap_tracking` whether a tracking site id has been minted — never the id itself. No key, value or last-four is sent. `cap_context_dev` covers both of the agent's Context capabilities: company brand data by domain, and reading a person off a LinkedIn URL. They are one key, so they are one boolean. #### The agent | Property | What it is | | --- | --- | | `tool_calls` | Calls in the last 24h, keyed by tool name | | `tool_calls_total` | Their sum | | `tool_errors` | Of those, the ones that returned an error, keyed by tool name | | `sandbox_used` | Whether any sandbox tool (`bash`, `grep`, `glob`, file reads and writes) ran | | `sessions_started` / `sessions_completed` / `sessions_failed` | Distinct sessions in the window | | `tools_per_session_mean` | Tool calls divided by sessions that made one | | `tasks_claimed` / `tasks_completed` / `tasks_retired` | `AgentTask` counts keyed by `kind` | | `task_attempts_mean` / `task_attempts_max` | Attempts per kind | | `budget_exhausted` | Sessions that stopped because the research budget ran out. Once per session, not once per blocked tool call | | `recheck_scheduled` | `recheck` tasks queued in the window | | `recheck_interval_days` | Their intervals, in day bands | | `agent_conversations` | How many `AgentConversation` rows exist | Tool names are matched against the authored tools in `apps/agent/agent/tools/` and eve's own builtins. Anything else is counted as `other`. Task kinds are matched against `TASK_KINDS`. Tool counts are read from the `agentEvent` rows the audit hook already writes — the event `type` and the tool name only. `AgentEvent.data` itself is never sent. #### The evidence ledger | Property | What it is | | --- | --- | | `facts_by_status` | `ContactFact` counts by `APPLIED` / `PROPOSED` / `DISMISSED` / `SUPERSEDED` | | `facts_by_band` | Counts by `VERIFIED` / `PROBABLE` / `POSSIBLE` | | `facts_by_method` | Counts by the `method` label a tool recorded | | `facts_by_evidence_kind` | Counts by evidence kind, from the `WEIGHTS` map in `lib/evidence.ts` | | `fact_dismissal_rate` | The share of decided facts a human rejected | | `fact_decision_median_hours` | Median hours from `observedAt` to `decidedAt` | | `facts_superseded_within_7_days` | Facts the agent changed its mind about | Evidence kinds are matched against the eleven in `lib/evidence.ts`; anything else is `other`. `method` is a label a tool writes, so it is sent only when it matches a strict lowercase dotted-slug shape under 40 characters — a name, an address or a sentence cannot pass, and is counted as `other` instead. The fact's `value`, `field`, `evidence` and `sourceUrl` are never sent. #### The CRM | Property | What it is | | --- | --- | | `contacts_bucket`, `companies_bucket`, `deals_bucket`, `activities_bucket` | Sizes, in bands | | `contacts_by_source`, `companies_by_source` | Counts by `MANUAL` / `IMPORT` / `EMAIL` / `CALENDAR` | | `deals_by_stage` | Counts by `DealStage`. Stages, never amounts | | `activities_by_type` | Counts by `ActivityType` | | `mailbox_sync_configured` | Whether any `MailboxSync` row exists | | `mailbox_sync_status` | Counts by `GoogleSyncStatus` | | `threads_ingested`, `messages_ingested` | How many arrived in the window. Counts only | | `enrichment_by_status` | `Company` counts by `EnrichmentStatus` | | `suppressed_domains`, `suppressed_contacts` | Counts. Never the domains or addresses | | `workspace_profile_written` | Whether a `WorkspaceProfile` row exists | #### Website tracking | Property | What it is | | --- | --- | | `tracking_domains` | How many domains are on the allow list, in bands. Never a host | | `tracking_page_views` | Page views recorded in the window. A count, never a path | | `tracking_forms` | Form submissions stored in the window. A count, never a field | | `tracking_contacts_created` | Contacts created **by** tracking in the window — `Contact.source = TRACKING`, not submissions that attached to somebody already filed | | `tracking_capped` | Submissions the hourly contact cap refused. Matched against `CONTACT_CAP_REASON` (`@crm/db/tracking`) exactly, so the wording is a constant two files share rather than prose one of them may reword | | `tracking_paused` | Whether collection is paused | ### The setup funnel Eight events, each sent once per install, ever. Each carries the timestamp of the thing it records rather than the moment it was noticed, so the funnel reads true even though a nightly job is what sends it. `migrations_applied` → `first_sign_in` → `google_oauth_configured` → `first_mailbox_sync` → `first_non_seed_contact` → `first_agent_task_claimed` → `first_agent_task_completed` → `first_fact_applied` They carry no properties of their own beyond the common ones — version, days since install, whether this is Vercel. `google_oauth_configured` is the one step with no row to read a time off: two environment variables leave no timestamp behind. It is stamped from the first Google `account` row when one exists — configuration must have come before the account did — and otherwise from the moment the sweep first saw the variables set. That sweep runs on every API boot, not only on the cron, so an install that arrives configured is stamped on its first boot. `first_fact_applied` is the earliest `decidedAt` of any fact a human accepted, or the earliest `observedAt` of any fact the agent applied itself, whichever came first. Facts that were applied and later superseded still count — being replaced later does not make the first application not have happened. **The step is claimed before it is sent, and handed back if the send fails.** The claim is the insert itself — `telemetryMilestone.step` is the primary key, so of two processes sweeping at the same instant exactly one is told it landed the row, and only that one sends. A failed send deletes the row, so the next sweep tries again rather than losing the step forever. This is the opposite of the ordering the funnel shipped with, which sent first and recorded after. That was chosen on the grounds that a duplicate is a rounding error while a lost step cannot be recovered — but it made a duplicate the *likely* outcome rather than a rare one, because both the boot sweep and the rollup call `sweep()`. One install sent `first_fact_applied` six times. Claiming first costs the case the old ordering was protecting against: a crash between the claim and the send drops the step. Each event also carries a deterministic `uuid`, `stableUuid(install.uuid, step)`, so a step that is genuinely sent twice is still ingested once. Nothing is recorded at all while telemetry is off. ### Errors Error **class** and where it came from. Never a message, never a stack trace, never a payload: our errors would carry contact fields. | Event | Properties | | --- | --- | | `agent_error` | `error_class`, `tool`, `task_kind`, `error_source` (`tool` / `turn` / `session`) | | `sync_error` | `error_class`, `sync_source` (`gmail` / `calendar` / `outlook`), `error_source` (`google_sync` / `microsoft_sync` / `mailbox_sync`) | | `api_error` | `error_class`, `route`, `status_code` | | `model_error` | `error_class`, `model_id` | `sync_error`'s `error_source` is derived from `sync_source` rather than passed in, so one mailbox pipeline that dispatches to both providers cannot report an Outlook failure as a Google one. A source that is not one of the three is sent as `other`, and its `error_source` as `mailbox_sync`. `route` is the **route pattern** — `/internal/sync/google`, `/trpc/contacts.list` — never a URL with parameters in it. Anything that does not match a strict path shape is sent as `other`, and so is any error class that is not a plain identifier. ## The landing page `trycrm.ai` is the one place `posthog-js` runs: web analytics and session replay of strangers reading a pitch. It is a website, not an install — there is no database behind that page and no record within reach of it. Everything above still holds for the CRM itself. **It cannot run on your hostname.** `analyticsAllowed` (`apps/app/lib/analytics.ts`) compares `window.location.hostname` against two literals — `trycrm.ai` and `www.trycrm.ai` — and `posthog-js` sits behind a dynamic `import()` on the far side of that check. Turn `IS_MARKETING="true"` on and serve the same page from a domain of your own and the SDK is never fetched: not sent and then dropped, never downloaded. The gate is the hostname rather than `IS_MARKETING`, because `IS_MARKETING` is precisely the flag a self-hoster is invited to set. PostHog refuses it from the other side too. `recording_domains` on the receiving project names the same two origins, so replay and heatmaps are declined for any other page that presents the key. **It shares the install's project, key and host**, and the two do not mix: the landing page sends `$pageview`, `$pageleave`, `$autocapture`, `$web_vitals` and the two CTA events below about a browser, an install sends `install_daily` and its funnel about a database. Install events set `$process_person_profile: false`, so they never become a person and never join one. ### The two CTA events | Event | When | | --- | --- | | `setup_prompt_copied` | The setup prompt reached the clipboard — after the write resolves, so a refused clipboard is not counted as a copy | | `github_star_clicked` | The **Star on GitHub** button was clicked. A click on the way to GitHub, not a star: whether one was given happens on a page we cannot see | Both carry one property, `cta_location` — `hero` or `closing` — because both buttons appear twice on the page and which one a stranger reaches is the only interesting thing about them. Nothing else is attached. They go through `captureLanding` in `components/landing/analytics.tsx`, behind the same hostname gate and the same dynamic `import()` as `init`. On any other hostname the SDK is never fetched, so the click is not sent and then dropped — there is nothing there to send it. Both sends are fire-and-forget behind a swallowed handler; neither delays the copy or the navigation. **`anonymize_ips` stays on**, and that costs something here: PostHog drops the address before geoip runs, so the landing page has no country, region or city. That setting is what makes the no-IP promise above true for every install, and an install's anonymity outranks a marketing map. A separate project is the only way to have both. `ALLOWED_PROPERTIES` does not reach these events — it guards what `packages/telemetry` builds, and these are the browser SDK's own. What it sends is ordinary web analytics: URL, referrer, UTM parameters, browser, device, screen size, a session ID. Replay masks every input by default and the page has no field to type in. ## What is never sent - Any `Contact` or `User` name, email, title or photo - Any `Company` name, domain, industry or location - `ContactFact.value`, `.sourceUrl`, `.evidence`, `.field` - `AgentTask.reason` and `AgentTask.outcome` — both free text the agent wrote about a named person - `ContactBrief` content, `WorkspaceProfile.website`, `.narrative`, `.sections` - `EmailThread` and `EmailMessage` subjects or bodies, `CalendarEvent` titles, `CalendarAttendee` rows - `Deal` names and amounts. Stage distribution is fine; amounts are not. - `AgentEvent.data`, `AgentConversation` content, prompts, completions, reasoning traces - `ALLOWED_SIGN_IN`, `AppSetting.contextDevApiKey`, any key, secret, token or connection string - `SuppressedDomain` and `SuppressedContact` values — counts only - **IP address.** Set `$ip: null` and disable geoip. n8n collects IP and has to caveat their anonymity claim because of it. We do not need it and we would rather the claim be unqualified. On that last line: the two properties alone do not achieve it, because PostHog fills `$ip` in from the connection during ingestion. `anonymize_ips` is enabled on the receiving project, which is what actually drops it. See `adrs/telemetry.md`. ## Where it lives | | | | --- | --- | | `packages/telemetry` | The only place `posthog-node` is imported. Allowlist, install accessor, event builders | | `install` table | The UUID, the version, and when the last rollup went. One row, made by the migration | | `telemetryMilestone` table | Which funnel steps have fired | | `telemetryCounter` table | The one runtime number no other row records — `budget_exhausted` — drained by each rollup and put back if the send fails | | `packages/telemetry/src/project.ts` | The key, the ingest host and the UI host. Three constants, no imports, so the browser can read them too | | `apps/api/src/telemetry` | The daily rollup, the funnel sweep, the boot-and-hourly timer, the optional cron route | | `apps/agent/agent/hooks/telemetry.ts` | Tool, turn and session failures | | `apps/app/lib/analytics.ts` | The two hostnames the landing page may report from | | `apps/app/components/landing/analytics.tsx` | The only `posthog-js` import in the repo, behind that gate and a dynamic `import()`. `init` on mount, `captureLanding` for the two CTA events | Nothing is instrumented inside `apps/agent/agent/sandbox`, which runs `deny-all` egress by design. A capture in there would hang and then fail. ## Adding a property 1. Add its name to `ALLOWED_PROPERTIES`. Until it is there, it is dropped. 2. Add a row to the table above. 3. If it is an open set of keys — a tool, a method, a route — give it a `permitted*` function so the keys are constrained too, not just the property name. --- ### Tracking (`docs/tracking.md`) # Website tracking A first-party script on the customer's own marketing site, a collector in the API, and one rule about what it is for: **a form submission becomes a contact.** Page views exist to give that contact a story, not to be a web-analytics product. Everything here is one install's own website. There is no second tenant, no shared pixel, and no vendor: the script is served from the same origin as the app, the cookie is first-party, and the only thing that ever leaves the browser is a POST to `/api/t/e` on the install's own API. ## Two scripts, and why | | | | --- | --- | | `apps/app/lib/tracking/loader.ts` → `/t/crm.js` | The tag a rep pastes. Reads the site id from `data-site` or from `?site=`, checks the shape, injects the second script. Immutable and cached for a year at the edge | | `apps/app/lib/tracking/tracker.ts` → `/t/.js` | The tracker itself, with the config **baked into the source** rather than fetched. Cached for five minutes | The split is the whole cache design. The tag never changes, so it is `immutable` and free forever; the config does change, so the file that carries it is the one with the short life. Baking the config in also means a page view costs one request, not a request and then a config fetch before anything can be recorded. - **The site id has two carriers, and a tag manager is the reason.** Google Tag Manager's Custom HTML injector rebuilds the script element and keeps only the URL — `data-site`, `async` and `defer` are all dropped on the way in. An attribute-only tag therefore loads `/t/crm.js`, finds no site, and returns: the loader is in the network tab, the tracker never is, and the install looks installed while recording nothing. `?site=` rides in the `src`, which no injector can strip. **`data-site` still wins when both are present**, so a pasted tag beats a stale URL. Either way **the loader stays config-free** — baking the install's own id into it would put a year-immutable edge cache in front of the rotate kill switch. - **Five minutes is a promise.** Pause tracking and every browser stops within `CONFIG_MAX_AGE_SECONDS`. That is why `/t/[site]` carries **no `stale-while-revalidate`** — a revalidation window is exactly a licence to keep executing the old config after the pause, and 24 hours of it once cost this guarantee entirely. - **Both routes are anonymous**, listed in `proxy.ts` as `ANONYMOUS = ["/t"]`. A stranger's browser on the customer's marketing site has no session and must not be redirected to `/sign-in`. - **The tracker has a size budget**, asserted in `apps/app/test/tracking-bundle.spec.ts`: 1 KB brotli for the loader, 4 KB for the tracker, because the settings page promises a number. A change that busts it fails the test rather than the promise. ### Writing tracker source It is a string of ES5 in a template literal, minified by hand, and it runs on somebody else's page. That imposes rules nothing else in this repo has: - **Never throw.** An uncaught error on line 40 means no listeners are installed and the page records nothing at all. Every `decodeURIComponent`, `JSON.parse`, `new URL` and `history` call is already wrapped; a malformed UTM parameter must read as *absent*, never as *fatal*. - **Constants come from `@crm/db/tracking`**, interpolated in — `MAX_BODY_BYTES`, `MAX_EVENTS_PER_BATCH`, `LINKER_MAX_AGE_SECONDS`, `COOKIE_NAME`. A literal at the call site is a client and a server that disagree, and the disagreement is silent. - **The client stays inside the body limit the collector enforces.** `flush()` measures the packed batch and splits it, then trims a single oversized form's fields, rather than posting a body the collector will destroy — a rejected POST is not retried, so the whole batch would simply be lost. ## The collector `POST /api/t/e`, anonymous, 204, in `TrackingController`. It answers nothing: a tracker that could read a response is a tracker whose failures a stranger can probe. **It is the one route that sets `Cross-Origin-Resource-Policy: cross-origin`**, and it must. `helmet()` puts `same-origin` on every response, which is right for an API only its own app calls — but this one is called by a `no-cors` beacon on somebody else's marketing site, so Chrome blocks the reply with `ERR_BLOCKED_BY_RESPONSE.NotSameOrigin` and logs a failure under every page view. The header is set on the response, not switched off in `helmet`, so the exception stays with the route that needs it. The gauntlet, in order, in `TrackingIngestService.accept`: 1. **User agent** — the `BOT` pattern. 2. **Site id** — must be a live `cmp_` id, and `forSite` refuses a rotated one. 3. **Origin** — `originAllowed`. A missing `Origin` header is refused **even with the allow list off**, because the header is the only thing tying the POST to a browser on a page. 4. **Visitor id** — 8–64 of `[a-zA-Z0-9_-]`, or the batch is dropped. 5. **`scripted()`** — three or more events sharing one timestamp is a replay. It only fires when **every** event carries a numeric `at`; missing timestamps are no signal, and treating them as proof discarded honest traffic. 6. **Host** — each event's own host against the allow list. The batch is filtered, not refused, because one bad host in twenty is a bug in somebody's SPA. 7. **Rate** — `EVENTS_PER_MINUTE` charged **per accepted event, atomically**, through `TrackingCounter`. > **The rate limit counts events, not requests, and it counts them in the > database.** A per-request counter with a full batch behind it admits twenty times > the number on the constant, and a read-then-write in `cache-manager` admits > however many requests are in flight. `TrackingCounterService.take(key, limit, > amount)` is one statement — an `INSERT … ON CONFLICT DO UPDATE` whose `WHERE` > carries the limit — so the check and the charge cannot be separated, and **a > refused batch spends nothing**. Increment-then-compare looks equivalent and is > not: the refused batch still burns its own size out of the window, so one > oversized batch locks out the honest small ones behind it for the rest of the > minute. It fails **closed**: a counter it cannot reach refuses the write. `MAX_BODY_BYTES` is enforced by destroying the request as it streams, before any parse. Keep it in step with what the tracker can produce — a form is forty fields of five hundred characters, and a limit under that silently drops the submission this whole feature exists to catch. ### What is never stored - **No query strings.** `normalizePath` strips `?` and `#` from every path, and `stripQuery` does the same to every referrer, on the event row *and* inside the `firstTouch` / `lastTouch` blobs. A password-reset link in a referrer is a password-reset link in the database. UTM parameters are captured as their own fields, so nothing is lost by it. - **No sensitive fields.** `clean()` drops any field whose *name* matches `SENSITIVE`, anything shaped like a card number, and every `password`, `hidden` and `file` input — the last three never leave the browser at all. - **No IP address**, anywhere, ever. ## Attribution `packages/db/src/attribution.ts`, and it is pure: no database, no config, one function. `classifyTouch` turns UTM parameters and a referrer into a `Touch`. - **An explicit `utm_source` beats the referrer**, because the marketer said so. - **The referrer host is matched on DNS labels, never on a substring**, and the provider's labels must sit at the registrable name — one tail label, or two when the first is a second-level suffix **and the last is a two-letter country code** (`co.uk`, `com.au`, `co.jp`). So `images.google.de` and `scholar.google.co.jp` are Google, while `notgoogle.com`, `google.evil.com`, `google.com.phish.example` and `google.com.example` are all plain referrals. A substring match hands a phisher a trusted source name in the rep's report, and so does `com.`: the country code is what stops `SECOND_LEVEL` from waving through every `com.` lookalike. **It is still a heuristic, not the public suffix list** — a provider under an unusual two-label suffix reads as a referral, which is the safe way to be wrong. - **Webmail is checked before search.** `mail.google.com` contains `google.`, so order alone decides whether a campaign's own click-throughs read as *email* or as *organic search* — and reading your newsletter as Google organic is how a marketing report lies. - **`Direct` is a source, not a null.** Everything lands in one of the seven `MEDIUMS`; an unrecognised medium is `other`. ## Filing: from a submission to a contact `TrackingFilingService.file` is the only path from a form to a `Contact`, and it reuses the mailbox pipeline's judgement rather than inventing a second one — `isMachineAddress`, `isAutomatedAddress`, `isMachineDomain`, `workspaceDomains`, `SuppressedContact`, `SuppressedDomain`, `companyForEmail`, `splitName`. **A rule that holds for the inbox holds here.** A second copy is how *deleted contacts stay deleted* comes to be true of email and not of forms. - **Every submission is stored; filing is separate and may decline.** `skipReason` says why, and the row stays for a rep to look at. - **A refusal is a `skipReason`, never a lost row**, and `CONTACT_CAP_REASON` is a shared constant because `rollup.service.ts` matches on it. Telemetry that substring-matches prose breaks the first time somebody improves the wording. - **`CONTACTS_PER_HOUR` bounds the blast radius** of a scripted form. It is charged only when a *new* contact would be created — an existing contact costs nothing, because attaching to somebody already in the CRM cannot flood it — and **a create that loses the race hands its slot back**, so duplicate delivery of one form cannot eat the hour's quota for a contact that was only ever made once. - **The email race is handled, not hoped away.** Two submissions for one address land at once, one `create` loses on the unique index, and the loser attaches to the contact that won. Left as a raw `P2002` it becomes a stored submission that is never filed and never retried. - **`attach` claims the row before it writes anything.** It is an `updateMany` on `filedAt: null`, and a caller that loses the claim returns without writing — otherwise two concurrent deliveries of one submission each leave the contact a `Submitted a form on …` note. - **Duplicate delivery retries an unfiled row.** `dedupeKey` collapses a resend inside one minute, but if the first attempt died before it filed — no `filedAt`, no `skipReason` — the resend is the retry. - **The agent is told, never asked.** `agent.contactCreated` writes an `AgentTask`. No enrichment, no scoring and no identity matching happens here; see `docs/api.md`. ## Retention `POST /internal/tracking/retention`, nightly at 04:00 via `apps/api/vercel.json`, `CRON_SECRET` or nothing. - **The cutoff is a whole UTC day**, `EVENT_RETENTION_DAYS` back and then truncated. A mid-day cutoff splits one calendar day across two nightly runs, and `trackedPageDaily` keeps the larger half of a day it saw twice — so the boundary day is quietly under-reported forever. Roll whole days or do not roll. - **Roll before you delete.** `TrackingRollupService` aggregates `page_view` rows only; a click is not a page view, and counting visitors over both produces a row reading `views = 0, visitors = 3`. - **Deleting is batched and bounded** — `SWEEP_BATCH` × `MAX_SWEEP_PASSES`. A sweep that hits the ceiling **says so**, in the log and in the response, because silently leaving events behind reads exactly like having deleted them. - **A visitor outlives their events only if a contact points at them.** An anonymous visitor with nothing left is removed. ## Settings, and who may change them Settings → Tracking & Analytics. `canManageTracking` (`@crm/auth`) gates every mutation **in the service**, and the same flag disables the control — the button and the 403 cannot disagree, as everywhere else. `tracking.sources` is manager-only too, so the page must not prefetch it for everybody: a member's render would fire a request that can only be refused. - **Verify reports on the page it actually fetched.** `safeFetch` follows redirects, so the host in the result comes from `fetched.url`, not from what the rep typed — otherwise `acme.com` reports the allow-list status of a page that lives on `www.acme.com`. - **Verify reads Tag Manager containers, not only the HTML.** A tag a container injects is not in the response, so an HTML-only check calls a working install broken — and it did. When the HTML carries no tag, `verify` takes the `GTM-…` ids out of it and reads `gtmContainerUrl(id)`, at most `MAX_CONTAINERS` of them. **The URL is built from the id we matched, never from anything a rep typed**, so the one host this can ever reach is Google's. A container holding the attribute form is reported as *found and broken*, with the fix, because that is precisely what it is — and `missing` names the containers it read, so a rep can tell *not installed* from *not looked at*. - **`MAX_VERIFY_BYTES` is measured against real marketing pages.** The old 512 KB cut was smaller than one homepage this repo's own company ships, and a tag past the cut reads exactly like a tag that is not there. - **Rotating the site id is the kill switch for a stolen snippet.** The old id stops resolving at `forSite` within the cache TTL. - **The compiled config is cached for five minutes and invalidated on every write.** Two things guard the cache, because one is not enough. `invalidate()` bumps a **per-process generation** before it deletes, which stops this replica's own in-flight read putting the pre-pause config back. That counter means nothing to a second replica, so nothing is written to the cache unless its hash still equals `AppSetting.trackingConfigHash` — **the shared version, in the database, that every replica can see**. A read that raced a change computes the old hash, finds it no longer current, and declines to cache it. Pausing sets the column to `null`, so while tracking is paused **no config can validate and none is cached at all**. ## Where it lives | | | | --- | --- | | `packages/db/src/tracking.ts` | Every constant and every pure helper both sides share. **The single source of truth** — a literal copied out of here is a future divergence | | `packages/db/src/attribution.ts` | `classifyTouch` and the source tables. No imports, no database | | `apps/app/lib/tracking/` | The loader and the tracker source | | `apps/app/app/t/` | The two public routes | | `apps/api/src/tracking/` | Collector, config cache, ingest, filing, counters, rollup, retention, tRPC router | | `apps/app/app/(app)/[slug]/settings/tracking/` | The settings page | | `apps/app/components/crm/website-activity.tsx` | The record-sheet section, which renders its own heading so it can render nothing at all | --- ### Plan Contact Intelligence Agent (`docs/plan/contact-intelligence-agent.md`) # Plan — Contact intelligence agent (v2) Turn the enrichment worker we have into an agent that **decides**: what to research, how hard, when to look again, and how sure it is. Every claim it writes carries its evidence; the contact sheet shows the person, the background, and how much of it to believe — live, while the agent works. Read it with [`AGENTS.md`](../../AGENTS.md), [`api.md`](../api.md), [`design.md`](../design.md), [`crm-plan.md`](./crm-plan.md), [`gmail-calendar-plan.md`](./gmail-calendar-plan.md) and [`people-enrichment-agent.md`](./people-enrichment-agent.md), which is v1 and whose §0 findings still hold. Conventions there are assumed, not repeated. **Two rules from the top, and everything below obeys them:** 1. **Never at the API level.** This is an agentic-first platform. NestJS serves HTTP, auth, tRPC and the Google sync. It does not research, enrich, score, summarise or decide anything about a person — not as a fallback, not "just the cheap bit", not behind a feature flag. §2, §10. 2. **The agent may read everything we have.** This is a single-tenant internal tool. Email bodies, meeting notes, attendee lists and every activity on a timeline are legitimate evidence and the agent gets them in full. The one boundary is egress, not access. §4. **This document is not static either.** Each phase closes by appending what it measured to §14, the way v1's §0 rewrote v1's architecture. --- ## 0. Where we actually are Measured against the tree on 2026-08-01, not remembered. ### Two enrichment stacks, one job | | Lines | Owns | | --- | --- | --- | | `apps/api/src/enrichment/` | ~1,720 | Context.dev company brand + research, LinkDAPI contact identity, avatar mirroring, an in-process queue | | `apps/agent/agent/` | ~2,140 | LinkDAPI identity again, Perplexity research, summaries, socials | Two LinkedIn clients (`enrichment/linkdapi.client.ts`, `agent/lib/linkdapi.ts`). Two identity matchers. `looksLikeSameCompany`, `nameMatchesLocalPart` and `searchTerms` exist verbatim in both trees — and **have already drifted**: the agent's `looksLikeSameCompany` guards `(!b && !c)`, the API's does not, so with an empty company name the API copy evaluates `a.includes("")` and matches every employer on earth. It is unreachable today because the only caller passes a domain fallback. That is the point: nobody noticed, because nobody was looking at the copy they were not working in. Triggers are scattered to match: `companies.service.ts:305,364,389` for companies, `google-match.service.ts:335` for contacts, two eve schedules for everything else. ### The "agent" is a worker with a numbered list Both schedules are step lists — `research-new-contacts` steps 1–5, `profile-contacts` steps a–f. The model chooses nothing that matters. It does not pick who to work on (a `take: limit` does), how deep to go (a boolean flag does), or when to look again (a cron expression does). Ten tools, each one API call. That is a script with a language model inside it. It is also using almost none of the framework. No sandbox, so no `bash`, no `web_fetch`, no `web_search`, no workspace. No `defineState`, no `defineDynamic`, no hooks, no channels, no subagents, no evals. Ten hand-written tools, several of which re-implement things eve ships. ### Confidence is computed, then thrown away `get_linkedin_profile` returns `high | medium | low`. `update_contact` refuses anything below `high` and returns a sentence. Nothing is stored. A `medium` match — the right person, one weak signal short — is discarded so completely that the next run pays to rediscover it and discards it again. v1 §5 said `medium` becomes a suggestion. Never built. ### There is no provenance store, and no way to see the agent think v1 §6's `ContactEnrichment` model never landed. The only field-level source in the schema is `Contact.summarySourceUrl`, shipped today. `Contact` has no `enrichmentStatus`, so the sheet cannot even show work in flight. Everything the agent does happens in a cron job nobody can watch, explain or interrupt. --- ## 1. What "better than HubSpot and Salesforce" has to mean Not more fields. They will always have more fields. Four structural advantages: | They optimise for | We do instead | | --- | --- | | **Enrichment as a purchase.** Credits buy a record; the unit is the record and the vendor's confidence is not your business. | **Enrichment as an argument.** Every claim carries evidence, method and date, and the UI shows them. A rep can disagree with one field for one reason. | | **Per-seat, per-credit economics** across millions of tenants, so re-reading every contact monthly is a line item they must charge for. | **One tenant, our Postgres, our keys.** Re-reading costs cents. That turns enrichment from a snapshot into a *feed*, which is the only way job-change detection exists at all. | | **Generic B2B data.** The same firmographics every customer of the platform gets. | **Our own conversation history as evidence** — in full. We sync Gmail and Calendar; a signature block settles a job title outright, and a reply on a thread is proof of identity no data vendor can sell us (§4). | | **A chat panel bolted onto a record.** Ask a question, get an answer, nothing persists. | **A resident agent with durable sessions.** It works the record on its own clock, and when a rep opens the sheet they can watch it, interrupt it and redirect it — same session, same memory, next week. | The fourth is the one that is hard to copy, and eve is the reason it is cheap for us: durable sessions, a streamable event log and a browser client are framework features, not a project. **The test:** a rep opens a contact five minutes before a call and knows who they are, what they do, how long they have done it, what we have already said to each other, and which parts a machine inferred. Then they type "is he still at Fleetio?" into the same panel and watch it check. --- ## 2. Decisions that carry the design 1. **Never at the API level.** Not a fallback, not a fast path. The API's enrichment directory is deleted, not deprecated — §0 shows what a second copy does even when nobody is calling it. Nest emits *events*; the agent decides what they mean. §10. 2. **The agent runs in a sandbox and uses the harness.** `bash`, `web_fetch`, `web_search`, `glob`, `grep`, `todo`, `ask_question` and a `/workspace` come with eve. Authored tools are for the three things the harness cannot do: vendor APIs that need keys and verification, CRM reads and writes, and decisions. Every tool we write that wraps a public HTTP GET is a tool we are maintaining for nothing. §4. 3. **Confidence is data, not a gate.** Every fact is stored with evidence, score, method and source — including the ones too weak to apply. Discarding a `medium` match throws away the most valuable thing the run produced: a candidate a human could settle in three seconds. §5. 4. **Background runs propose; people approve.** A markdown schedule is task mode and **cannot park for a person** — so an approval gate in a cron run is not a pause, it is a failure. Background work below the bar writes a *proposal*. Approval prompts and `ask_question` are for sessions a human started. §5, §7. 5. **The record is the interface to the agent.** The Next app mounts eve's routes with `withEve` and the contact sheet renders a live session with `useEveAgent`. The rep watches the same event stream the runtime persists. No second app, no chat silo. §6. 6. **The agent may read the whole CRM, including bodies.** Internal tool, single tenant, our own data. The boundary is *egress*: raw customer text never leaves as a third-party search query, and never reaches the sandbox, which has network access. §4, §8. 7. **Budget is an input the agent spends,** held in `defineState` and enforced in the tool — a VP on a live deal earns a deep pass, a newsletter signup earns one lookup. §7. 8. **Humans outrank the agent, permanently.** A field a person typed is never overwritten; a dismissed proposal is never re-proposed. Enforced in the fact store, not in a prompt. ### Judgment calls worth your veto - **Deleting the API path outright.** The safe-sounding option is to keep it as a fallback. That is how we got two of everything. Enrichment has never been on a request path; if the agent is down, contacts wait. - **Giving the agent a shell.** A sandbox with `bash` and web egress is a real capability and eve's own docs tell you to review it before production. The mitigation is not "no shell", it is **no database credentials in the sandbox** (§8): CRM access is authored tools in the app runtime, never `psql`. - **Numeric confidence, shown as words.** Storing `0.82` and rendering "probable" invites "why not 0.79?". The honest answer is that the number is calibrated by evals (§9), not a probability. Three opaque bands cannot be calibrated at all. - **A live agent panel on the record.** More UI, and it puts model output in front of reps constantly. That is the intent: an agent nobody watches is an agent nobody corrects. - **The agent schedules itself.** Bounded by lease and budget, but it decides to come back in fourteen days. Anything less and "not static" is a slogan. --- ## 3. Data model Current values stay denormalised on `Contact` so lists stay one query. The new tables are the *reasoning*, not the read path. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` `AgentEvent` is close to free: a single `defineHook` on `*` writing `insert … on conflict (id) do nothing`, exactly the pattern eve's streaming doc recommends, and ULIDs keep it append-ordered. It gives us the agent's full history — which tool ran, what it returned, what it decided — without inventing an audit format. `Contact` also gains `enrichmentStatus`/`enrichedAt`/`enrichmentError` matching `Company`, so the sheet reuses `isEnriching()` and `ENRICHMENT_POLL_MS` (`api.md`) for the non-streaming case. `ContactFact` subsumes `summarySourceUrl`, `summaryUpdatedAt` and `socialsCheckedAt` shipped today; those columns migrate into facts in Phase 1. **Job changes fall out for free:** a new `employer` fact superseding an applied one *is* the event. No diffing machinery, which is why v1 put it last and this plan does not have to. --- ## 4. Capabilities: what eve gives us, what we write ### From the harness (a sandbox is all it takes) | Built-in | What it replaces | | --- | --- | | `web_search`, `web_fetch` | Half of `lib/perplexity.ts`. Perplexity stays for *cited synthesis*; plain retrieval is a built-in. | | `bash`, `read_file`, `write_file`, `glob`, `grep` | A `/workspace` dossier per contact: dump the profile JSON, grep it, diff this month's against last month's. Job-change detection becomes a `diff`. | | `todo` | Durable per-session task list. Multi-step research stops living in the prompt. | | `ask_question` | The agent asks the rep — "two Marchettis at Fernhill, which one?" — and parks. Worth more than any heuristic we could write for that case. | | `agent` (root-only) | Parallel research fan-out. Meeting prep with four attendees is four children, not four sequential passes. | | `load_skill` | Procedures load on demand instead of bloating every turn. | Enabled deliberately, per eve's own guidance to review the defaults: `bash`, file tools and `web_fetch` stay on because a research agent without retrieval is a chatbot; the sandbox gets **no database credentials** (§8). ### Authored tools — only where the harness cannot go | Group | Tools | Why authored | | --- | --- | --- | | **CRM read** | `read_contact`, `read_crm_history`, `read_company` | Database access belongs in the app runtime, not the sandbox. `read_crm_history` returns threads, meetings, attendance **and message bodies** — the strongest identity evidence we own. | | **Vendor** | `get_linkedin_profile`, `get_work_history`, `research_person` | Keys, rate limits, and LinkDAPI's `200 {success:false}` envelope. | | **Judge** | `verify_identity`, `verify_social`, `corroborate_from_crm` | These fetch their own evidence and rule in code. `verify_social` exists and is the template: no `confidence` argument, no model-supplied `sourceUrl`. | | **Decide** | `plan_research`, `schedule_recheck`, `set_priority` | The tools that make it an agent. `schedule_recheck` takes a date **and a reason**, and the reason is shown to the rep. | | **Write** | `apply_fact`, `propose_fact`, `write_brief`, `record_job_change` | Approval policy lives here (§5). | Two invariants: **a tool never accepts the model's own confidence**, and **"looked, found nothing" is recorded with a date** — the generalisation of today's `socialsCheckedAt`, without which every run re-litigates the same absence. ### Context control `instructions.md` shrinks to identity and the one rule. Everything else becomes a skill loaded on demand — `identity-matching` (exists), `confidence-scoring`, `writing-a-brief`, `meeting-prep`, `job-change`, `data-boundaries`. `load_skill` adds instructions, never execution surface, so this is free discipline. `defineDynamic` then makes the surface itself situational, resolved at `session.started`: a contact with no company gets no company tools; a task on a live deal loads the deal-brief skill; ambiguous identity work can resolve a stronger model. (Model resolution stays at `session.started` — switching mid-session re-ingests the conversation at uncached prices.) ### The egress boundary The agent reads everything. What leaves is controlled: - **Never send raw customer text to a third-party search.** Perplexity and `web_search` get derived questions ("what did Acme announce in 2026?"), never a pasted thread. - **Nothing from a mailbox reaches `/workspace`.** The sandbox has network access; email bodies stay in the app runtime, in the model's context. - **`api.md` still stands:** never *log* headers, bodies or query strings. Read is not log. --- ## 5. The confidence model ### Evidence, weighted Starting values, to be calibrated (§9), not truths: | Evidence | Weight | Primary? | | --- | --- | --- | | Exact email match on a profile | 0.95 | ✔ | | LinkedIn: employer matches **and** name is consistent with the address | 0.85 | ✔ | | **They replied on a thread we synced** | 0.85 | ✔ | | GitHub API: the account's own `name` or `company` matches | 0.8 | ✔ | | **A signature block in a thread states the title** | 0.8 | ✔ | | They attended a meeting on our calendar | 0.7 | ✔ | | Perplexity states it with a citation | 0.4 | | | Handle is a construction of their name | 0.35 | | | A search engine cites the profile for their name + employer | 0.35 | | | Employer matches, name does not | 0.2 | | The two bolded rows are ours alone, and they are near the top. That is the argument in §1 turned into arithmetic. Combined as `1 − Π(1 − wᵢ)` over *independent* items, capped at 0.99, with two rules a score cannot express: 1. **No auto-apply without a primary source.** Three weak signals multiply to a confident-looking number and remain three weak signals. 2. **Contradiction floors the score.** A profile saying one employer and a mail header saying another is not 0.6, it is unresolved, and the fact is held. ### Bands | Band | Score | Behaviour | In the UI | | --- | --- | --- | --- | | `VERIFIED` | ≥ 0.85 + primary | Applied automatically | The value, plain. Dotted underline reveals the source. | | `PROBABLE` | 0.55–0.85 | Stored as a proposal; field stays empty | A line under the field: *"LinkedIn suggests Head of Security · accept · dismiss"* | | `POSSIBLE` | 0.3–0.55 | Stored, never shown as a value | Only in the provenance panel, as what was considered | | below | < 0.3 | Not stored | — | ### Enforced by the approval policy, not the prompt eve's docs are explicit that model behaviour alone must not guard sensitive writes, and a rubric in a system prompt *is* model behaviour. The policy also has to know **who** is driving, because a task-mode schedule cannot park for a person: ```ts title="agent/tools/apply_fact.ts (policy sketch)" approval: ({ session, toolInput }) => { const auth = session.auth.current; const automated = auth?.authenticator === "app" && auth.principalId === "eve:app" && auth.principalType === "runtime"; if ((toolInput?.score ?? 0) >= 0.85 && toolInput?.hasPrimarySource) { return "not-applicable"; // verified: just write it } // A cron turn cannot wait for a human, so asking would hang the run. return automated ? { type: "denied", reason: "Below the bar — call propose_fact instead." } : "user-approval"; // a rep is here; ask them }; ``` Same tool, two behaviours, decided by the principal rather than by a flag the model can set. Sensitive fields (`email`, `companyId`, anything that re-parents a record) sit a band higher regardless of score. --- ## 6. The contact sheet All of it inside `design.md`: components from `/packages/ui`, no new radii or colours, no `className` overrides on shared components, new variants land in the package. ### The Background section Above Details, because it is what the sheet is *for*: ``` BACKGROUND refreshed 2 days ago Lewis Carhart is the CEO and co-founder of Comp AI. He previously led growth at Fleetio and spent four years at Deloitte in risk advisory. Current role CEO & Co-founder · Comp AI · 2 yrs 3 mos Previously Head of Growth, Fleetio · Risk Advisory, Deloitte Seniority Founder / C-level Function Executive Based London, UK WE KNOW THEM ← from our own data 12 emails · last reply 4 days ago · 3 meetings · next: Thursday 14:00 Also here Sarah Chen (CTO) · Tom Reyes (Security Lead) ``` The second block is the one no incumbent renders, because it is built from the Gmail and Calendar sync rather than a purchased record. ### How confidence reads `status-indicator.tsx` argues against pill chips in its own comment and it is right: a sheet speckled with coloured badges is a sheet you read the badges of. - **Verified values render as plain text.** No chrome. Decoration on everything is noise, and noise is how the one uncertain field gets missed. - **Agent-written values carry a dotted underline.** Hover or focus reveals a `ProvenanceTooltip` — source, method, date, the evidence that scored it. One new component in `/packages/ui`. - **Proposals are not values.** The field stays empty; the suggestion sits beneath with accept and dismiss. A guess is never formatted like a fact. - **`StatusIndicator`** already covers in-flight and failed. Accept and dismiss write `ContactFact.status` with `decidedById` — labelled data for §9, produced by the people best placed to judge, when they care. ### The live panel `withEve()` in `apps/app/next.config.ts` mounts eve's routes on the app's own origin — no CORS, no URL env var. A `useEveAgent()` panel in the sheet then gives the record something neither incumbent has: - **Watch it work.** `actions.requested` / `action.result` render as a plain list of steps: *searched LinkedIn · read profile · checked GitHub API · rejected `github.com/lewis`*. The rejections are the most trust-building thing on the page. - **Interrupt and redirect.** "Is he still at Fleetio?" continues the same durable session, with the same history, on the same record. - **Answer its questions.** `ask_question` renders inline: two Marchettis at Fernhill, pick one. The answer is worth more than any tiebreak heuristic. - **Replay.** Sessions are durable for 30 days and the stream is rewindable, so "why does it think that?" is a question with an answer. **Tables get none of this.** The contacts list stays plain text; scanning is its whole job. Confidence and agency belong where a decision gets made. --- ## 7. Not static: how the agent chooses ### One dispatcher, not two cron prompts A single `defineSchedule({ cron: "* * * * *", run })` claims due `AgentTask` rows under an atomic lease and calls `receive(...)` per row with `appAuth`, following eve's documented dynamic-scheduling pattern. Prompts stop being numbered lists; the row carries the objective, the budget and the reason. ### The agent writes its own rows `schedule_recheck` takes a date *and* a reason: - champion on an open deal → 14 days ("job change here moves a live deal") - named contact, no open deal → 90 days - nothing found twice → 365 days, low priority - `support@`, `no-reply@` → never Today's `cron: "20 * * * *"` over `take: 10 ORDER BY createdAt` treats the CEO of a live opportunity and a bounced alias identically. ### Events jump the queue A calendar event tomorrow with an unknown external attendee inserts a high-priority `meeting-prep` task the moment the sync sees it. Delivery is a custom channel (`agent/channels/crm.ts`) exposing routes the sync posts domain events to — "thread ingested", "attendee unknown" — never "enrich this". The agent decides what an event means; that is the whole point of rule 1. ### Depth is a decision, budget is state `plan_research` reads the contact, the deal context and the budget, then picks the passes. The budget is a `defineState` slot decremented by the tools that spend it — eve's own budget example, applied to vendor calls: ```ts const research = defineState("crm.research-budget", () => ({ spent: 0, cap: 8 })); ``` A $40k opportunity earns more calls than a newsletter signup, and the agent — not a boolean — makes that call. ### Fan-out for briefs Meeting prep with four external attendees delegates four children through the built-in `agent` tool with non-overlapping write scopes, then writes one brief. If the orchestration gets genuinely complex, the opt-in `Workflow` tool lets the model author that fan-out as one durable step; it stays off until a brief needs it, because a capped subagent budget is easier to reason about than model-authored JavaScript. --- ## 8. What stays deterministic Widening discretion is only safe if the boundary is code: - **Identity verification.** `verify_*` fetch their own evidence and rule. - **The approval policy.** §5, keyed on principal, in the tool definition. - **Never overwrite a human; never re-propose a dismissal.** In the fact store. - **No database credentials in the sandbox.** The sandbox has `bash` and egress; CRM access is authored tools in the app runtime. This is the single most important line in the section. - **No raw customer text leaves as a search query.** §4. - **Rate limits and budgets.** A token bucket per vendor; a per-task budget the agent spends but cannot raise. - **Business context only.** Name, title, employer, tenure, public profile. No special categories regardless of what an endpoint returns (v1 §9 stands). Dynamic in *what to do*. Fixed in *what is allowed*. --- ## 9. Evals, or the confidence number means nothing `evals/*.eval.ts` with `defineEval`, driving real sessions over the same HTTP surface the app uses; `mockModel` for deterministic fixtures where the point is the plumbing rather than the judgement. | Metric | Target | Why | | --- | --- | --- | | Precision at `VERIFIED` | ≥ 0.98 | An auto-applied wrong fact is the failure that costs a deal. | | Precision at `PROBABLE` | ≥ 0.75 | Below this, reps learn to dismiss without reading. | | Recall | tracked, not gated | Coverage is a cost decision; precision is a trust one. | | Cost per contact | tracked | Where "budget as an input" meets reality. | The fixture set is real CRM addresses with known answers, **including ones whose correct answer is nothing** — a matcher that never abstains scores well on a set where everybody is findable. Accept/dismiss decisions from §6 feed the fixtures, which is the flywheel: the more it is corrected, the better calibrated the number the corrections are shown against. Calibration is the deliverable: when a band says 0.85, roughly 85 of 100 should be right. The weights in §5 move until that holds. --- ## 10. Retiring the API path Order matters — no window with two writers: 1. **Facts first.** `ContactFact` and the `AgentEvent` hook land; the agent writes through them. 2. **Invert the triggers.** `companies.service.ts` and `google-match.service.ts` stop calling enrichment and post a domain event to the agent's CRM channel. Nest learns nothing about enrichment; it reports what happened. 3. **Move company enrichment.** Context.dev brand mapping becomes agent tools. `brand-mapping.ts` moves as-is — pure and well-tested. 4. **Delete `apps/api/src/enrichment/`,** minus `avatar.service.ts` (Blob mirroring belongs to whoever writes the URL). ~1,600 lines out of the API, ~200 of them moving rather than dying. One LinkedIn client in the repo, one identity matcher, one place to fix a bug. 5. **Move the tests.** `enrichment.spec.ts`, `contact-enrichment.spec.ts` and the integration spec go to `apps/agent/test/`. They encode findings, not implementation. What Nest keeps: HTTP, auth, tRPC, the sync, and the ability to say "this happened". What it stops doing: deciding anything about people. --- ## 11. Phases | Phase | Scope | Done when | | --- | --- | --- | | **1 — Provenance spine** | `ContactFact`, `ContactBrief`, `AgentEvent` hook, `Contact.enrichmentStatus`; `ProvenanceTooltip` in `/packages/ui` | Every agent-written value on the sheet traces to a source and a date without opening a database. | | **2 — Harness** | Sandbox on, built-ins reviewed and gated, instructions split into skills, `defineState` budget, `defineDynamic` surface | The agent researches with `web_search`/`web_fetch`/`bash`; three authored tools are deleted as redundant. | | **3 — Confidence** | Evidence ledger, bands, principal-aware approval policy, proposals inline with accept/dismiss | A `medium` match becomes a suggestion a rep settles in three seconds instead of a discarded log line. | | **4 — Agent-only** | §10 steps 2–5 | `apps/api/src/enrichment/` is gone. | | **5 — Background panel** | Structured sections, tenure, "we know them" from Gmail/Calendar, colleagues | The five-minutes-before-a-call test in §1 passes. | | **6 — Live panel** | Same-origin proxy, `useEveAgent` on the contact sheet, `ask_question` inline, step list | A rep watches a rejection happen and trusts the accepted ones more. | | **7 — Dynamic scheduling** | `AgentTask`, dispatcher, `plan_research`, `schedule_recheck`, CRM channel, subagent fan-out | Two contacts researched on different cadences, for reasons the agent wrote and the rep can read. | | **8 — Meeting prep** | Calendar-triggered briefs for external attendees | Thursday's 14:00 has a brief on it by Wednesday night. | | **9 — Job changes** | Superseding facts raise an activity and a task for the owner | A champion moves and their owner knows that week. | Phases 1–4 make the data trustworthy; 5–9 are what a rep would miss if you took it away. One ordering constraint is non-negotiable, as v1 also said: a brief built on unverified identities is confidently wrong *in front of a customer*. --- ## 12. Risks - **eve is in preview.** APIs may change before GA. `apps/agent/lib/` is where the vendor-shaped code lives; keep framework surface in the authored files thin enough to move. - **A sandbox with egress is a real capability.** §8's no-credentials rule is the mitigation, and it only holds if nobody "temporarily" adds `DATABASE_URL` to the sandbox env. - **Task-mode schedules cannot pause.** Any design that wants a human mid-cron is wrong by construction; that is why §5's policy denies rather than asks. - **Live streams show reasoning.** eve's docs flag the privacy implications of rendering reasoning events. Internal tool, so acceptable — but it is a decision, and it is recorded here as one. - **Single-provider LinkedIn risk** (v1 §9) is unchanged. --- ## 13. Open questions 1. **Score on the record, or only in the tooltip?** Leaning: band in the tooltip, raw number only in the provenance panel. 2. **Do proposals expire?** A suggestion nobody has judged in 90 days is noise. Auto-dismiss, or keep and stop re-scoring? 3. **Who owns the budget dial?** A constant, or a per-account setting a rep can turn up? 4. ~~Does the agent see message bodies?~~ **Decided: yes, in full.** Internal tool, single tenant. The boundary is egress (§4), not access. 5. **Company facts too?** The model generalises, but companies have one source and no identity problem. Probably later, possibly never. 6. **A rep edits a field by hand — what happens to the fact?** Superseded and kept as history is right, but the store must then accept writes that did not come from the agent. 7. **Does the live panel belong on companies and deals too?** Same machinery, three times the surface. Contacts first. --- ## 14. What each phase actually taught us Appended as phases land, in v1 §0's spirit — measurements that changed the design, not a changelog. ### Phases 1–4 (2026-08-01) **The weights in §5 do not behave the way the table implies, and a test caught it.** `crm.signature-block` at 0.8 is below the `VERIFIED` floor, so a title read out of a signature was going to become a suggestion rather than a fact — not what §1 promises. The realistic case saves it: a signature is read *on a thread they replied to*, so the evidence is `crm.thread-reply` + `crm.signature-block` → 0.97, and it writes. The lesson is that weights must be tested against the **combinations that actually occur**, not one at a time. **Four weak signals do not reach 0.85** (they reach 0.797), so the "pile of weak evidence" scenario the primary-source rule exists to stop was not reachable by arithmetic anyway. The rule is still right — it has to hold under re-calibration — but the test for it now asserts `bandFor(0.99, false)` directly rather than pretending the weights get there. **Approval gates were wrong for `record_fact`.** The plan had one on the write tool. Building it showed the flaw: if approval denies the call, `execute` never runs, so the *proposal is never stored* — the gate would have deleted the feature it was protecting. Band logic inside the same function that scores the evidence is strictly stronger, because there is no path around it. Approval moved to where blast radius is bigger than one field: `record_job_change`, which can re-parent a contact. **Verification functions should return evidence, not verdicts.** `verifyGithub` used to decide `accepted: true` on *any* of name-match, company-match, or domain-link. That meant an account sharing only an employer — a colleague — was written with the same authority as one naming the person. Returning evidence kinds instead put both through the ledger, and the colleague case now scores 0.2 and is dropped. Same code, one fewer way to be confidently wrong. **The "delete three redundant tools" prediction was wrong in detail and right in outcome.** Nothing became redundant *because of the harness* — `research_person` and the two candidate-finders still earn their keep. Six tools went anyway (`update_contact`, `write_contact_summary`, `write_briefing`, `list_contacts_to_research`, `list_contacts_to_profile`, plus the API's two enrichment services), because the fact store replaced the write path and one `list_outstanding_work` replaced the "which queue am I" tools. **The sandbox needs no network, which was not obvious until the harness table was read properly.** `web_fetch` runs in the app runtime and `web_search` at the provider — only `bash` and the file tools run in the sandbox. So `deny-all` costs nothing and removes the only shell-shaped path out of the building for the email bodies this agent now reads in full. It is set on the backend factory rather than in `onSession`, so it cannot be forgotten per session. **`packages/db` and `packages/auth` imported `@crm/env` without declaring it.** Hoisting hid it until `bun add` in `apps/agent` created a local `node_modules` that shadowed the root, and then every DB-touching test failed at once. Two one-line `package.json` fixes; worth recording because the symptom ("Cannot find module" in a package nobody edited) looks nothing like the cause. ### Phase 6 (2026-08-01) **`withEve` was the wrong tool, and the constraint made the design better.** The agent stays its own deployment, so mounting it inside the Next build was never available. What replaced it is the pattern the app already uses for the API: a **same-origin proxy**, at `app/eve/v1/[...path]/route.ts`, mounted at exactly the path `useEveAgent()` defaults to. The hook needs no `host`, the browser never learns the agent has its own origin, and there is no CORS and no cross-site cookie anywhere in it. **The proxy is an enforcement point, not a passthrough**, and that is the one structural difference from the API proxy beside it. The agent never sees the session cookie — it is stripped — so if the route did not check the session, nothing downstream would. It checks, then mints a two-minute HS256 token naming the rep. **eve's `jwtHmac()` helper labels the caller a service, and that would have been a silent bug.** Verified through the helper, a rep's token resolves to `principalType: "service"` with `principalId: "crm-app:user_123"` — sensible for a machine credential, wrong for a person. `lib/approval.ts` decides whether to pause for a human by reading exactly those fields, so a sensitive write would have been *refused* with "not something to do unattended" while the rep sat watching. `repFromCrm` wraps `verifyJwtHmac` and maps the subject to a real user principal instead. Found by printing the verifier's output rather than trusting the shape, and now pinned by `channel-auth.spec.ts`. **Both halves of the token are tested against each other**: the app mints and eve's own verifier accepts (`apps/app/test/agent-bridge.spec.ts`), and forged, expired, wrong-audience, wrong-issuer and subject-less tokens are all rejected by the real auth function (`apps/agent/test/channel-auth.spec.ts`). A shared secret is the whole security of the panel, so neither side is asserted from memory. **`AGENT_BRIDGE_SECRET` unset skips the entry rather than opening it.** No signer, no panel, everything else unchanged — the agent still runs its own schedule. That is the correct failure for an optional capability whose absence must never widen access. **One duplication survived on purpose.** `domainOf` in the agent and `domainFromEmail`/`normalizeDomain` in the API do the same job. Both apps genuinely need it, and a shared package for two pure functions is more machinery than the risk justifies — but it is the same shape as the drift in §0, so it is written down here rather than left to be rediscovered. --- ### Plan Crm Plan (`docs/plan/crm-plan.md`) # Plan — Agentic CRM (HubSpot replacement) A lightweight, opinionated CRM for our sales team. Companies are the top-level object; Contacts and Deals hang off them; everything that happens is an **Activity** on a timeline. Company knowledge (logo, description, industry, socials, address) is filled in by an agent using [Context.dev](https://docs.context.dev) rather than typed by a human. This document is the build plan: decisions, data model, contracts, and phased delivery. Read it with [`AGENTS.md`](../AGENTS.md), [`design.md`](./design.md) and [`api.md`](./api.md). --- ## 1. Decisions up front | Decision | Choice | Why | | --- | --- | --- | | Tenancy | **Single tenant, no organizations** | The repo is already built this way (`20260731160000_remove_organizations`). It's an internal tool; Google sign-in is the only door. | | Data fetching | **tRPC (nestjs-trpc) end to end** | Type-safe from Prisma to the table cell, and the pattern is proven in `/Users/lewiscarhart/mvp`. | | Table state | **nuqs**, server-side pagination/sort/filter | URLs are shareable and survive reload; the API does the work so 10k rows stay fast. | | Data table | Port `mvp`'s `DataTable` into **`packages/ui`** | `design.md` makes `/packages/ui` the single source of truth; `simple-table` and `card-table` already live there. Same code as mvp, different address. | | Enrichment | **Context.dev**, fire-and-forget on write, cached in Postgres | 10 credits per company; we never pay twice for the same domain. | | Activity model | **One `Activity` table**, denormalised `companyId` | A company timeline must show its deals' and contacts' events. One indexed query beats three joins. | | Deal owner | `Deal.ownerId` → `User`, **required** | Every deal has a name against it. Companies and contacts have optional owners. | ### Judgment calls worth your veto - **`UNQUALIFIED_TO_BUY` is terminal, not linear.** You listed it between "qualified to buy" and "decision maker bought-in", but it's a disqualification, not a step forward. Plan treats it like `CLOSED_LOST`: excluded from pipeline value and forecast, shown in a "Closed" tab rather than the open pipeline. Say the word if you want it mid-funnel instead. - **`Contact.email` is unique.** Matches HubSpot's own constraint and makes the CSV import idempotent. The cost: two people behind one `info@` address need separate records with one email each. - **`Company.domain` is the enrichment key and is unique.** Two records for the same domain is the #1 way a CRM rots, so we block it at the database. ### Repo inconsistencies found while surveying - **`docs/review.md` does not exist** but `AGENTS.md` says "Always review" it. Either write it or drop the reference — right now every agent starts by failing to read a required file. - **`docs/api.md` describes a multi-tenant world this repo doesn't have.** The org-context (`x-organization-slug`, `OrgContextInterceptor`) and per-org cache generation sections were carried over from the MVP; there are no organizations here. The **logging** rules apply verbatim and should stay. Recommend trimming api.md to logging + a short "no orgs, no org cache keys" note in Phase 0. - `apps/app/make-session.tmp.ts` is a stray scratch file, safe to delete. --- ## 2. Architecture ``` browser │ same-origin /api/trpc (cookies just work) ▼ apps/app (Next.js 16, RSC) ├─ app/api/[...path]/route.ts ──proxy──▶ apps/api:3001 ├─ server components: prefetch via getServerTrpc() → HydrationBoundary └─ client components: useTRPC() + TanStack Query, nuqs for table state │ apps/api (NestJS on Bun, port 3001) ▼ ├─ /api/trpc nestjs-trpc routers (companies, contacts, deals, activities, users) ├─ /api/auth/* Better Auth (Google only) ├─ services Prisma queries, timeline writes └─ enrichment Context.dev client + queue ──▶ api.context.dev │ packages/db (Prisma) ◀────────────────────────┘ ``` Two things the CRM adds to the existing skeleton: 1. **`apps/app/app/api/[...path]/route.ts`** — a catch-all proxy (ported from `mvp`) so the browser calls `/api/trpc` same-origin. Today the app sends the browser straight to `:3001`; that works for auth redirects but makes every tRPC call a cross-origin, credentialed request. The proxy removes that whole class of problem and streams SSE through untouched. 2. **`apps/api` exports its router type**: `"exports": { "./app-router": "./src/generated/server.ts" }`, generated by `nestjs-trpc generate`. The app imports `type { AppRouter } from "api/app-router"`. ### Dependencies to add | Package | Where | Version (matching mvp) | | --- | --- | --- | | `@trpc/server`, `zod` | `apps/api` | `^11.18.0`, `^4.4.3` | | `@trpc/client`, `@trpc/server`, `@trpc/tanstack-react-query` | `apps/app` | `^11.18.0` | | `@tanstack/react-query`, `@tanstack/react-query-devtools` | `apps/app` | `^5.101.2` | | `nuqs` | `apps/app` + `packages/ui` | `^2.8.9` | | `zod` | `apps/app` | `^4.4.3` | | `context.dev` | `apps/api` | latest | `nestjs-trpc` (`^2.13.0`) and `@keyv/redis`/`cache-manager` are already in `apps/api`. Note `nestjs-trpc` ships a binary that may need the `postinstall` chmod shim mvp uses — check on first install. --- ## 3. Data model New models in `packages/db/prisma/schema.prisma`. Auth models are untouched. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` `User` gains the back-relations (`CompanyOwner`, `ContactOwner`, `DealOwner`, `ActivityAuthor`). It is otherwise untouched — Better Auth owns that model and `bun run auth:generate` regenerates it. ### Timeline semantics - **Past** = `occurredAt` set, or `completedAt` set, or a `NOTE`/`STAGE_CHANGE`/`ENRICHMENT` row (use `createdAt`). - **Upcoming** = `dueAt > now()` and `completedAt IS NULL`. - **Overdue** = `dueAt < now()` and `completedAt IS NULL`. - A company's timeline is `WHERE companyId = ?`, which picks up deal and contact activity for free because we stamp `companyId` on write. ### Pipeline math - **Open stages**: `DEMO_BOOKED`, `QUALIFIED_TO_BUY`, `DECISION_MAKER_BOUGHT_IN`, `CONTRACT_SENT`. - **Closed stages**: `CLOSED_WON`, `CLOSED_LOST`, `UNQUALIFIED_TO_BUY`. - Pipeline value = `SUM(amount)` over open stages. Setting a closed stage stamps `closedAt` and prompts for `closedReason` on the two losing outcomes. --- ## 4. The agentic layer (Context.dev) `apps/api/src/enrichment/` — a `ContextDevClient` (thin typed wrapper over `https://api.context.dev/v1`) plus an `EnrichmentService` that owns a small in-process queue (concurrency 2, one retry with backoff, per-domain dedupe). **Trigger points** | Event | Call | Cost | | --- | --- | --- | | Company created/updated with a new `domain` | `POST /brand/retrieve` `{type:"by_domain"}` | 10 credits | | Manual "Re-enrich" button | same, `maxAgeMs: 0` | 10 credits | | Contact created with a work email and no company | `POST /brand/retrieve` `{type:"by_email"}` → find-or-create Company | 10 credits | | CSV import (bulk) | `POST /utility/prefetch` per row, then retrieve | prefetch free (paid plan) | | "Research this company" action | `POST /web/extract` with our schema → `ENRICHMENT` activity | 10 credits | **Field mapping** from the `brand` object: - `title` → `name` (only when the record was created domain-first and unnamed) - `description`, `slogan` → `description` - `logos[]` → pick by `mode`/`type`, never `logos[0]`: `logoUrl` = light+`logo`, `logoDarkUrl` = dark+`logo`, `iconUrl` = `icon` - `colors[0].hex` → `brandColor` (key off `hex`; the `name` is generated) - `industries.eic[0]` → `industry` / `subIndustry` - `address` → `city`, `stateCode`, `country`, `countryCode` - `socials[]` → `linkedinUrl` / `twitterUrl` / `githubUrl` by `type` - `links` → `pricingUrl`, `careersUrl` - `email`, `phone` → same fields - Whole payload → `CompanyEnrichment.raw` **Handling**, per the Context.dev skill: - Every field is nullable — never overwrite a human-entered value with `null`. - `400 NOT_FOUND` / `WEBSITE_ACCESS_ERROR` → `SKIPPED`, not `FAILED` (and not billed). - `422` on email lookup (free/disposable) → `SKIPPED`, no company created. - `403 FORBIDDEN` on prefetch → log once, fall through to direct retrieve. - `408`/`429` → one backoff retry, then `FAILED` with `enrichmentError`. - Cold lookups are p50 ≈ 7s, p90 ≈ 18s. Always background; never block the create mutation. Set `timeoutMS: 60000`. **Research brief** (`POST /web/extract`) uses a fixed JSON Schema — `{ positioning, pricingModel, targetCustomer, notableCustomers[], recentNews[] }` — and posts the result as an `ENRICHMENT` activity on the company timeline so a rep sees it in context instead of in a side panel. **Config**: `CONTEXT_DEV_API_KEY` in `apps/api/.env`, declared `@IsOptional()` in `env.validation.ts`. Absent key ⇒ enrichment disabled, logged once at boot, companies stay `PENDING`. The key never reaches the browser; all calls are server-side. **Caching note**: `docs/api.md`'s org-scoped generation keys don't apply (no orgs). tRPC responses aren't served by the HTTP response cache; freshness comes from TanStack Query invalidation on mutation. Enrichment finishing is a *background* write the client can't see, so the company detail page polls (`refetchInterval` while `enrichmentStatus` is `PENDING`/`RUNNING`) and stops when it settles. --- ## 5. tRPC contract Every list endpoint speaks the same shape so one `DataTable` drives all three modules. ```ts // apps/api/src/trpc/list-input.ts export const listInput = z.object({ q: z.string().default(""), sort: z.string().default(""), dir: z.enum(["asc", "desc"]).default("asc"), page: z.number().int().min(1).default(1), pageSize: z.number().int().min(1).max(100).default(25), }); // → { rows: T[]; total: number; facetCounts: Record> } ``` | Router | Procedures | | --- | --- | | `users` | `me`, `list` (owner pickers) | | `companies` | `list` (+ facets `owner`, `industry`, `enrichment`), `byId`, `create`, `update`, `remove`, `setPrimaryContact`, `enrich` | | `contacts` | `list` (+ facets `owner`, `company`), `byId`, `create`, `update`, `remove` | | `deals` | `list` (+ tab `stage`, facets `owner`, `closing`), `board`, `byId`, `create`, `update`, `setStage`, `setContacts`, `remove` | | `activities` | `timeline` ({ companyId? contactId? dealId?, filter, cursor }), `create`, `update`, `complete`, `remove`, `myTasks` | | `dashboard` | `summary` (pipeline by stage, closing this month, overdue tasks, recent activity) | All routers carry `@UseMiddlewares(AuthMiddleware)` — ported from mvp minus the org and permissions middleware, which have no meaning here. `deals.setStage` writes the `STAGE_CHANGE` activity and stamps `stageChangedAt`/`closedAt` in the same transaction. Filtering, sorting and pagination happen **in Prisma**, not in the browser. `q` searches name/domain/email with a case-insensitive `contains`; add a `pg_trgm` index if the tables ever get big enough to notice. --- ## 6. Frontend ### Routes ``` apps/app/app/(app)/ page.tsx dashboard companies/page.tsx DataTable companies/[companyId]/page.tsx detail + timeline contacts/page.tsx DataTable (placeholder exists today) contacts/[contactId]/page.tsx detail + timeline deals/page.tsx DataTable (+ ?view=board) deals/[dealId]/page.tsx detail + timeline ``` `AppIconRail` gains Companies and Deals entries alongside the existing Overview (`Dashboard`) and Contacts (`UserMultiple`) — Carbon glyphs imported one at a time through the `Icon` wrapper, per the README. Pick the glyph names off the [Carbon icon library](https://carbondesignsystem.com/libraries/icons/) and confirm the file exists in `@carbon/icons-react/es/` before importing; the package isn't installed in this checkout yet, so the names are unverified. ### The data table Port these from `/Users/lewiscarhart/mvp/apps/app/components/data-table/`, rewriting `@trycompai/ui` → `@crm/ui`: | Source | Destination | | --- | --- | | `data-table.tsx` (631 lines) | `packages/ui/src/components/data-table.tsx` | | `table-pagination.tsx`, `numbered-pagination.tsx` | `packages/ui/src/components/` | | `use-table-query.ts`, `build-search.ts` | `apps/app/components/data-table/` | Everything it depends on already exists in `@crm/ui`: `table`, `dropdown-menu`, `input-group`, `button`, `spinner`, and `lib/row-accent`. The hook stays in the app because it's routing state, not UI. `nuqs` becomes a dependency of `packages/ui` (the table owns the `expand` and `hide` params). Behaviour we're keeping verbatim: sticky header, `table-fixed` with truncating cells, sort/columns/facet dropdowns, mobile filter collapse, expandable rows, `useDeferredValue` so typing stays smooth, and the pagination footer. ### nuqs conventions - `NuqsAdapter` from `nuqs/adapters/next/app` wraps the root layout, alongside `TRPCReactProvider`. - Shared parsers live in a `*-search-params.ts` per module, importing from **`nuqs/server`** so the page and the client component agree. - Server components read them with `createLoader`, and pass the parsed values into the tRPC prefetch so the first paint is already filtered. - The search box uses `limitUrlUpdates` (nuqs' built-in debounce) rather than a hand-rolled timer. - `clearOnDefault` everywhere, so a default view has a clean URL. - Table state stays **shallow** (default): TanStack Query refetches; we don't round-trip the RSC on every keystroke. Detail sheets (`?company=`, `?deal=`) are shallow too. ### Server rendering Each list page prefetches through `getServerTrpc()` + `getServerQueryClient()` and wraps the client panel in `` — the exact mvp pattern, minus the org slug. ### Company detail page The page that has to be good, because it's where reps live: - **Header** — logo (`iconUrl`, falling back to initials), name, domain link, industry badge, owner avatar, and an enrichment state chip that shows a spinner while the agent works. - **Left column** — properties card (inline-editable: name, domain, website, phone, address, owner), primary contact card, socials row. - **Right column** — the timeline: composer at the top (Note · Call · Email · Meeting · Task), then Upcoming/Overdue pinned, then history grouped by day. Filter tabs via nuqs (`?tab=all|notes|upcoming|done`). - **Below** — Contacts table and Deals table, both `SimpleTable` (not the full `DataTable`; these are short embedded lists). Per `design.md`: shared components only, no bespoke radii/spacing/colours, no rounded corners. `brandColor` is used as a small accent swatch, never as a surface — brand colours would fight the design tokens. --- ## 7. Phases Each phase ends green on `bun run check-types` and `bun run lint`. ### Phase 0 — Plumbing (no product surface) - Add the dependencies in §2; wire `apps/api/src/trpc/` (module, context, `AuthMiddleware`, error handler, logging middleware) from the mvp reference. - `nestjs-trpc generate` script + `exports["./app-router"]`; add the watch task to `dev`. - `apps/app/app/api/[...path]/route.ts` proxy; `lib/trpc/{server.ts,client.tsx,query-client.ts}`. - `NuqsAdapter` + `TRPCReactProvider` in the root layout. - Port the data table into `packages/ui`. - Housekeeping: resolve `docs/review.md`, trim the org sections from `docs/api.md`, delete `make-session.tmp.ts`. - **Verify**: dashboard renders `users.me` through tRPC, server-prefetched. ### Phase 1 — Schema and seed - The models in §3, one migration, `User` back-relations. - Extend `packages/db/prisma/seed.ts`: ~15 companies (real domains so enrichment has something to chew on), ~40 contacts, ~25 deals spread across stages, ~150 activities including overdue and upcoming tasks. - **Verify**: `bun run db:migrate && bun run db:seed`, inspect in Studio. ### Phase 2 — Companies - `companies` router + service; list page with DataTable — columns: logo+name, domain, industry, owner, contacts, open deals, last activity; facets owner / industry / enrichment status; search on name+domain. - Company detail page per §6, minus the timeline (Phase 5 stub). - Create sheet: name + domain + owner, nothing else. The agent fills the rest. - **Verify**: filter + sort + page, copy the URL into a new tab, same view. ### Phase 3 — Contacts - `contacts` router; list DataTable (name, title, email, company, owner, last activity); detail page; company link; "make primary contact" action. - Replace the existing `contacts/page.tsx` placeholder. ### Phase 4 — Deals - `deals` router; list DataTable (name, company, stage badge, amount, owner, expected close, last activity); tab = open/closed/all, facets owner + stage. - Inline stage change from the row (dropdown) writing `STAGE_CHANGE`. - Deal detail: stage stepper, amount/currency/close date, associated contacts, timeline slot. - **Verify**: stage change appears on the deal *and* the company timeline. ### Phase 5 — Activity, tasks and dashboard - Timeline component + composer; `activities` router; complete/snooze on tasks. - Dashboard: pipeline value by stage, deals closing this month, my overdue tasks, recent activity feed. - **Verify**: a note on a deal shows on the company timeline; an overdue task shows on the dashboard. ### Phase 6 — The agent - `ContextDevClient`, `EnrichmentService`, queue, field mapping, status polling. - Auto-enrich on company create; manual re-enrich; contact-email → company. - Research brief action writing an `ENRICHMENT` activity. - **Verify**: create a company with domain `stripe.com` → logo, description, industry, socials and address populate within ~20s without a reload. ### Phase 7 — Migration and polish - HubSpot CSV import for companies/contacts/deals, dedupe on `domain`/`email`, `utility/prefetch` warming before the retrieve pass, dry-run report first. - Deal board view (`?view=board`, stage columns, drag to change stage). - `cmdk` quick switcher over companies/contacts/deals. --- ## 8. What this plan deliberately leaves out Email sync, calendar sync, sequences/cadences, quotes, custom properties, multiple pipelines, reporting builder, and permissions/roles. Every one of them is a HubSpot feature we're choosing not to rebuild. If the sales team asks for one, it's a new plan, not a phase. --- ### Plan Dynamic Fields Build (`docs/plan/dynamic-fields-build.md`) # Plan — Dynamic fields (build) Build instructions for the agent implementing dynamic fields on companies, contacts and deals. [`dynamic-fields.md`](./dynamic-fields.md) is the **design record** — it owns the schema and the reasoning behind every decision. This document is the **build order**: what to type, where, and in what sequence. Where they disagree, the design doc wins on *why* and this one wins on *how*. Two constraints govern everything below. 1. **The Paper design is fixed.** Match it exactly. Do not improve it, round it, or reword it. 2. **shadcn components only.** No hand-rolled buttons, inputs, switches, chips or menus. If a component does not exist, add it to `packages/ui` with the shadcn CLI or add a variant there — never at the call site. --- ## 0. Read before you type, and say what you read | Before touching | Read | | --- | --- | | Anything | [`AGENTS.md`](../../AGENTS.md), [`design.md`](../design.md), [`dynamic-fields.md`](./dynamic-fields.md) | | `apps/api` | [`api.md`](../api.md), `.agents/skills/nestjs-trpc/SKILL.md` | | `packages/db` | `.agents/skills/prisma-database-setup/SKILL.md` | | `apps/app` | `apps/app/AGENTS.md` → then the relevant guide in `apps/app/node_modules/next/dist/docs/01-app/` | | URL state | `.agents/skills/nuqs/SKILL.md` | | Any UI | `.agents/skills/shadcn/SKILL.md` and `rules/styling.md`, `rules/composition.md`, `rules/forms.md` | | `apps/agent` | `.agents/skills/eve/SKILL.md` → then `apps/agent/node_modules/eve/docs/README.md` | | A telemetry event | [`telemetry.md`](../telemetry.md) | ## 1. The design Paper file **CRM**, page **crm - lewis**: `https://app.paper.design/file/01KZ72S23CM00WXN6S1MP68M03/5-1` | Artboard | What it specifies | | --- | --- | | `/companies?record=company:abcd` | Cog in the DETAILS header; custom fields inline; a pending agent suggestion | | `/contacts?record=contact:abcd` | Cog placement only | | `/deals?record=deal:abcd` | Cog placement only | | `…&manageFields=company` | The fields sheet — list state | | `…&manageFields=company (first run)` | Empty state | | `…&manageFields=company&manageField=new` | Create a field | | `…&manageFields=company&manageField=runs_on` | Edit a field, with coverage | | `…&manageFields=company&manageField=runs_on (archive)` | Archive confirmation | **Take values from the file, not from screenshots.** `get_jsx`, `get_computed_styles`, `get_node_info`. A screenshot will not tell you whether a gap is 8 or 10. Every user-visible string is in §7. They are final. ## 2. Schema Copy the Prisma models from **[`dynamic-fields.md` §3](./dynamic-fields.md)** verbatim — `FieldEntity`, `FieldType`, `FieldDefinition`, `FieldOption`, `FieldValue`. That section also explains why values are typed columns rather than JSON, why one value table carries three nullable FKs, and why `key` is immutable. Do not redesign it; if you think it is wrong, say so before writing the migration. On top of that: - Migration name: `dynamic_fields`. Run through `bun run db:migrate` in `packages/db` — never `db:push` for a schema that ships. - Add the back-relations on `Company`, `Contact` and `Deal` (`fieldValues FieldValue[]`). - `agentFilled Boolean @default(true)` — the default is the feature. See [`dynamic-fields.md` §5](./dynamic-fields.md). - No `organizationId`. Single tenant. ## 3. `@crm/db/fields` — the shared core Everything else in this plan is a caller of this module. It is what makes the feature API-driven *and* agent-driven without two implementations drifting apart. Create `packages/db/src/fields.ts` and add `"./fields": "./src/fields.ts"` to the `exports` map in `packages/db/package.json` — the same shape as `@crm/db/currency` and `@crm/db/workspace`, which are the precedents. It owns: - `FIELD_TYPES` and the type→storage-column map (`TEXT`→`text`, `NUMBER`→ `number`, `DATE`→`date`, `CHECKBOX`→`bool`, `SELECT`→`optionId`, `USER`→ `userId`, …). - `fieldKeyFromLabel(label)` — slug, collision-checked by the caller against `@@unique([entity, key])`. - `valueSchemaFor(definition)` — returns the Zod schema for one field's value. - `readValues(entity, recordId)` / `writeValues(tx, entity, recordId, values)` — the only two functions that touch `FieldValue`. - `serializeField(definition)` — the wire shape used by tRPC *and* by agent tools, so both see identical field metadata. **Rule: no field logic anywhere else.** Not in a router, not in a service, not in a component, not in an eve tool. A validator written twice is a bug with a delay on it. ## 4. API — `apps/api/src/fields/` Four files, mirroring `apps/api/src/companies/`: `fields.contracts.ts`, `fields.router.ts`, `fields.service.ts`, `fields.module.ts`. Register `FieldsModule` in `app.module.ts` imports; `FieldsRouter` **must** be in the module's `providers` or it silently does not exist at runtime. `@Router({ alias: "fields" })` + `@UseMiddlewares(AuthMiddleware)` on the class. No middleware means public — there is no other guard. | Procedure | Input | Notes | | --- | --- | --- | | `fields.list` | `{ entity, includeArchived? }` | Definitions with options, `position` order | | `fields.byKey` | `{ entity, key }` | For the editor screen | | `fields.create` | label, type, options, `agentFilled`, `agentBrief`, placement | Key derived from label | | `fields.update` | id + same, minus key | Reject a `key` change with `BAD_REQUEST` | | `fields.reorder` | `{ entity, ids }` | One transaction, positions rewritten from the array | | `fields.archive` / `fields.restore` | `{ id }` | Archive is the normal path | | `fields.delete` | `{ id }` | Loud, and separate from archive | | `fields.backfill` | `{ id }` | Writes **one** `AgentTask`, `kind: "field-backfill"` — never one per record | Rules that are not optional: - **Routers are thin.** Zod in, service call out. Prisma lives in the service. - **Values do not get their own router.** They ride the record procedures: `companies.byId` gains `fields`, `companies.update` accepts `fields: Record`. Same for contacts and deals. One write, one invalidation, and the existing inline editors keep their mutation. - Services throw Nest's `HttpException` family; `DomainErrorMiddleware` maps them. - **Regenerate and commit** `src/generated/server.ts` (`bun run trpc:generate`). If the app cannot see `trpc.fields.*`, this is why. - Tests: `apps/api/test/fields.spec.ts`, `bun:test`, modelled on `apps/api/test/bulk.spec.ts` (real database, `TEST_RUN_ID`-suffixed data, cleaned up in `afterAll`). Cover: key collision, key immutability, value coercion per type, archive keeps values, delete cascades, reorder is atomic. ## 5. Agents — `apps/agent` The API is one caller of `@crm/db/fields`; the agent is the other. Add `apps/agent/agent/lib/fields.ts` as a thin wrapper, in the shape of `lib/facts.ts`, then these tools in `apps/agent/agent/tools/`: | Tool | Does | | --- | --- | | `list_fields` | Every field for an entity with its key, type, options and brief — so the agent can discover the schema instead of being told it | | `set_field_value` | Write a value the agent is certain of | | `propose_field_value` | Offer a value with evidence — routes through the fact/proposal path, not a direct write | | `create_field` / `update_field` | Manage definitions, so "add a field for X" works in chat | | `archive_field` | Behind `lib/approval.ts`. Destructive schema changes get a human | Constraints: - The `agentBrief` is the instruction the agent works from. `list_fields` must return it. - Tool descriptions carry the same voice as the existing tools — read `record_fact.ts` and `search_crm.ts` first. - **Nest still writes rows, not intelligence.** `fields.backfill` writes an `AgentTask`; the agent decides fan-out, batching and priority. - Missing capability ⇒ missing capability, never a throw. **MCP later, for free.** Every capability is a tRPC procedure over `@crm/db/fields`, and every tool is a thin call to the same module. An MCP server is then a mechanical wrapper over `fields.*` with no new logic. That only stays true if you keep logic out of the tools. Do not build MCP now. ## 6. App — `apps/app` ### URL state (nuqs) Two params, added to the **existing shared parser object** in `apps/app/components/crm/record-sheet/record-stack.ts` — do not start a second parser file: ```ts fields: parseAsStringLiteral(RECORD_KINDS), field: parseAsString, ``` - `fields` is the entity whose fields are open; `field` is either the create sentinel `new` or a field key. Add nothing else to the pasted lines — AGENTS.md forbids code comments, and these go straight into `record-stack.ts`. - **`new` is a reserved key**, held back by `RESERVED_KEYS` in `packages/db/src/fields-shape.ts`, so a field labelled *New* slugs to `new_field` and its editor can still be opened. Never let the sentinel and a derivable key collide. - Both cleared by the existing `write()` alongside `tab`, `add`, `thread`. - `useQueryStates` for the pair; `null` to clear; `history: "replace"` for sheet state, matching what `record-stack.ts` already does. - The cog sets `fields` to the record's own kind, so the sheet opens on the entity you came from. ### Components — use these, build nothing | Element in Paper | Use | | --- | --- | | Fields sheet shell | `DetailSheet` (`components/detail-sheet.tsx`) at a new `md` size | | Sheet header, back + title + close | `DetailSheetHeader` — it already does back-at-left, close-at-right | | Entity switcher | `Tabs` / `TabsList` / `TabsTrigger`, default variant | | Standard / Archived disclosure rows | `Collapsible` — **add it**: `bunx --bun shadcn@latest add collapsible` in `packages/ui` | | Type chip (`Select`, `Date`, `Number`, `Checkbox`) | `Badge` — **add it**, then add a `mono` variant in `packages/ui` | | Row overflow menu | `Button variant="ghost" size="icon-xs"` + `DropdownMenu` | | Drag handle | `Button variant="ghost" size="icon-xs"` carrying the dnd listeners, Carbon `Draggable` icon | | Form labels and helper text | `Field`, `FieldLabel`, `FieldDescription`, `FieldGroup` (`packages/ui/src/components/field.tsx`) | | Label / brief inputs | `Input`, `Textarea` | | Type picker | `Select` | | Option rows | `InputGroup` with a leading handle addon and a trailing remove `Button` | | "Add option" | `Button variant="ghost" size="sm"` | | Agent toggle | `Switch` | | Placement toggles | `Checkbox` + `FieldLabel` | | Footer actions | `Button` (primary) and `Button variant="outline"` | | Coverage block | Mirror `DetailSheetPending`'s muted panel; dot is `StatusIndicator` | | Archive confirm | `AlertDialog`, action button destructive | | Empty state | `Empty` / `EmptyHeader` / `EmptyMedia` / `EmptyTitle` / `EmptyDescription` / `EmptyContent` | | The cog itself | `Button variant="ghost" size="icon-sm"`, Carbon `Settings`, in `DetailSheetSection`'s existing `action` slot | Three additions to `packages/ui`, because that is where they belong: 1. **`md` sheet size.** `sheetContentVariants` has `sm | lg | xl | 2xl` and nothing near 460px. Add `md: "data-[side=left]:sm:max-w-[460px] data-[side=right]:sm:max-w-[460px]"`. The width comes from the artboards, and 1:1 outranks landing on a Tailwind step. 2. **`Badge` + a `mono` variant** for the type chip. 3. **`SortableList`** over `@dnd-kit/core` + `@dnd-kit/sortable`. The repo has no drag-and-drop library today. The artboards show drag handles on both field rows and option rows, so dragging is in scope and a Move up / Move down menu is *not* an acceptable substitute here. `EmptyDescription` also has no wrap class — add `text-balance` there so the empty state does not orphan its last word. Fix it in the component, not at the call site. ### Rendering fields on a record In `company-sheet.tsx` (then contacts, then deals), custom fields render **inline inside the existing `Details` section**, after the standard properties, in `position` order. No "Custom fields" heading, no separate section. One map from `FieldType` to the editor that already exists: | Type | Component | | --- | --- | | `TEXT`, `URL`, `EMAIL`, `PHONE` | `InlineField` (with `type`) | | `LONG_TEXT` | `InlineTextArea` | | `NUMBER` | `InlineField` | | `DATE` | `InlineDateField` | | `SELECT`, `USER` | `InlineSelectField` | | `CHECKBOX` | `Checkbox` in a `DetailSheetProperty` | Agent-sourced values get `SOURCED_VALUE` (dotted underline) with `SourcedValue` for the hover source. Pending proposals render with the existing `Suggestion` component. Both are built — use them, do not restyle them. ### Table columns `showOnTable` is opt-in per field. A workspace with thirty fields must not get a thirty-column table. Wire it into the existing column definitions in `companies-table.tsx` and friends. ### Cache Add `fields(entity?)` to `CrmCache` in `apps/app/lib/trpc/cache.ts`. A definition change invalidates `fields.list` plus every list and record query for that entity. Value edits keep `{ settle: "record" }` so the inline spinner clears without waiting for the table. **A new mutation adds a call there, not a new list of keys at the call site.** ## 7. Copy — verbatim **Fields sheet** - Title `Fields` - Subtitle `This shapes every company in your CRM.` (company / contact / deal) - Tabs `Companies` `Contacts` `Deals` - `Standard fields` · `8 · reorder and hide only` - `Custom fields` · `Drag to order` - `Archived` · `2 · values kept, hidden everywhere` - Footer `New field` · `Order here is the order on the sheet` **Empty state** - `No custom fields yet` - `Create dynamic fields that your agents can research and pre-fill.` - `New field` **Editor** - `Label` - `Key` · `What the API and your agents call it. Set from the label, fixed once saved — renaming the label later never breaks a caller.` - `Let your agents fill this` · `They propose a value with a source, and never overwrite yours.` - `What counts as an answer` · `Leave it empty and your agents work from the label and type alone.` - `Type` · `Select — one of a fixed list` - `Options` · `Add option` - `Show on the company sheet` · `Offer as a column on the Companies table` - `Create field` · `Cancel` · `Save changes` · `Archive` **Coverage** - `Filled on 41 of 64 companies` - `Last looked 2 days ago · you accepted 38 of 41` - `Fill the rest` **Archive dialog** - `Archive Runs on?` - `Hidden everywhere. Its 41 values are kept.` - `Cancel` · `Archive field` Never "eve" in the product. It is *your agents*. ## 8. Order of work Each phase ends green — typecheck, lint, tests — before the next begins. 1. **Schema.** Models, migration, back-relations. `bun run db:migrate`. 2. **`@crm/db/fields`.** Types, key derivation, value schemas, read/write. Unit tests here; it is pure and cheap to test. 3. **API.** Contracts, service, router, module, registration. Regenerate and commit `server.ts`. `fields.spec.ts` green. 4. **Values on records.** `byId` returns `fields`, `update` accepts them, all three entities. 5. **`packages/ui` additions.** Sheet `md`, `Badge` + mono, `Collapsible`, `SortableList`, `EmptyDescription` balance. 6. **The sheet.** List → empty → create → edit → archive, in that order, checking each against its artboard before moving on. 7. **The cog and the record rendering.** Companies first, then contacts and deals. 8. **Table columns.** 9. **Agent lib and tools.** ## 9. Do not - Do not deviate from the artboards, including the copy. - Do not write a custom button, input, chip, switch or menu. - Do not override component colours or typography with `className`. Layout only. - Do not put field logic in a router, a component or a tool — it goes in `@crm/db/fields`. - Do not put a vendor client, scoring or enrichment in `apps/api`. - Do not add code comments. - Do not add a `Co-Authored-By` trailer. - Do not add a per-package `.env`. - Do not hand-edit `apps/api/src/generated/server.ts` — regenerate it. - Do not build the MCP server. Keep the seam clean so it stays a wrapper. - Do not make the agent toggle default off. ## 10. Decisions already made — do not relitigate - Agent-first: `agentFilled` defaults true, the brief sits above the type in the editor, and only manual fields are badged. [`dynamic-fields.md` §5](./dynamic-fields.md). - Archive over delete for definitions, with delete kept as a separate loud action. - Standard fields appear in the list, reorderable and hideable, never deletable. - Custom fields render inline in DETAILS with no grouping of their own. ## 11. Open — decide with Lewis, do not guess **Generalising `ContactFact` to `RecordFact`** (three nullable FKs plus an optional `fieldId`) so agent proposals work for custom fields on all three entities, instead of a parallel proposal table. `ContactFact` is contact-only and keys on `field String`. Agent-first makes this load-bearing rather than optional, and it is the one decision that gets more expensive after launch. Raise it before phase 9 — earlier if phase 1 would touch it. --- ### Plan Dynamic Fields (`docs/plan/dynamic-fields.md`) # Plan — Dynamic fields Fields a workspace defines for itself, on companies, contacts and deals, edited from one sheet that opens from any record. The visual design is in Paper, file **CRM**, page **crm - lewis**, artboards `Dynamic fields — cog placement`, `/companies?record=company:abcd&manageFields=company`, `Dynamic fields — new field & empty` and `Dynamic fields — on the record`. This document is the half Paper cannot hold: the model, the API and the agent. This document is the design record — what we decided and why. The build order — files, procedures, component mapping, phases — is [`dynamic-fields-build.md`](./dynamic-fields-build.md), which points back here for the schema in §3 rather than copying it. Read it with [`AGENTS.md`](../../AGENTS.md), [`api.md`](../api.md) and [`design.md`](../design.md). --- ## 1. One sheet, one cog The cog goes in the **DETAILS section header**, on all three sheets. That is the `action` slot `DetailSheetSection` already has, so no new primitive and no per-sheet variation — the same element in the same place whether DETAILS is the full width of the contact sheet or the 320px rail of the company sheet. Not the header bar beside Re-enrich and Close: that bar is about *this record*, and the cog is not. It changes every company at once. Which is the one thing the sheet has to say out loud. It opens with **"This shapes every company in your CRM"**, because a control reached from Northwind Labs' sheet will otherwise be read as belonging to Northwind Labs. The sheet stacks over the record sheet at 460px, dimming it, and carries an entity switcher — Companies / Contacts / Deals — so the same sheet reached from a deal can still fix a contact field without closing anything. State lives in the URL beside `record`, in `record-stack.ts`: ``` /companies?record=company:abcd&manageFields=company ``` so it is shareable, Escape and Back close it in the right order, and the record underneath survives a refresh. ## 2. Standard fields are in the list The sheet shows two groups: **Standard** (Name, Domain, Website, Phone, Email, City, Country, Owner) and **Custom fields**. Standard fields are real rows — they can be reordered and hidden, never deleted or retyped. A rep who opens the sheet looking for "Domain" finds it and learns the rule in the same second, which is cheaper than any explanation of why it is missing. Custom fields then render **inline in DETAILS**, in the order set here. No "Custom fields" box. A field is a field; the grouping was an implementation detail and does not belong on the record. ## 3. The model ``` /* Detailed source-code truncated for AI context efficiency. */ ``` `User` carries the other half of that relation: ```prisma fieldValues FieldValue[] @relation("FieldValueUser") ``` Six decisions worth defending: - **Typed columns, not a `Json` value.** `api.md` says filter, sort and paginate in Prisma. A JSON blob cannot be indexed or ordered honestly, and the moment someone wants the pipeline sorted by *Renewal date* the answer would be to pull the table into the browser. One column per storage class keeps that promise; the type on the definition says which one is live. - **One value table with three nullable FKs, not three tables.** Every FK cascades, so deleting a company takes its values with it. `AgentTask` carries `contactId`/`companyId` with no FK and `api.md` documents the chore that creates — "clear `AgentTask` and `AgentEvent` yourself". Not repeating it. - **Options are rows.** Renaming "Google Cloud" must not orphan every record pointing at it, and a JSON array of strings makes rename indistinguishable from delete-plus-create. - **`key` is immutable, `label` is not.** The key is what the API and the agents say; the label is what the rep reads. Renaming the label of a field an agent has filled for a month cannot break the brief that taught it. - **The record columns are indexed on their own, not only inside the uniques.** A record read is `where: { companyId }` with no `fieldId` — `FieldsService.valuesFor` opens every sheet that way — and a unique starting `fieldId` cannot serve it. The composite uniques stay for the one-value-per- field-per-record rule; the standalone indexes are what the reads use. - **`userId` is a foreign key, like every other id on the row.** A `USER` field holds a workspace user, so a deleted rep must not leave an id on the record that resolves to nobody. `SetNull` matches `Company.owner` and `Contact.owner`: the value empties, and it never blocks the delete. **Archive, not delete.** A record delete is hard in this CRM by design, but a *definition* delete destroys a column of typed data across every record at once. Archiving keeps the values and hides the field everywhere; a true delete stays available behind a separate destructive confirm that names the count. No `organizationId`, on any of the three. Single tenant. ## 4. The API A new module, `apps/api/src/fields/`, one router as the codegen glob expects: | Procedure | Notes | | --- | --- | | `fields.list({ entity, includeArchived })` | Definitions with options, in `position` order. | | `fields.create` / `fields.update` | Key derived from label on create, rejected on update. | | `fields.reorder({ entity, ids })` | One transaction, positions rewritten from the array. | | `fields.archive` / `fields.restore` / `fields.delete` | Delete is the loud one. | **Values do not get their own router.** They ride the record procedures that already exist: `companies.byId` returns `fields`, `companies.update` accepts `fields: Record`. One write per record, one invalidation, and the inline editors keep using the mutation they already use. The value validator is derived from the definition at runtime — a shared `crm/fields.ts` that both the router's zod schema and the service use, so a `NUMBER` field cannot be written a string by either path. `useCrmCache()` gains `cache.fields(entity)`. A definition change is a schema change: it fans out to every list and record query for that entity. Rare enough that the width does not matter, and `{ settle: "record" }` still covers the inline value edits. `src/generated/server.ts` is regenerated by `dev`/`check-types` and committed, or the app cannot see any of this. ## 5. Agent-first, and what that costs This was the open question, and the answer is **agent-first**: `agentFilled` defaults to **true**. A new field is something your agents keep up to date unless you say otherwise, and "Manual only" is the exception a rep opts into. That is a real difference, not a default flipped for effect. It changes three things: - **Order in the editor.** The brief — *What counts as an answer* — sits directly under the label, **above** the type. The second thing you write is the instruction, not the shape. - **What the list shows.** Each row's subtitle is its brief, truncated. Agent upkeep is the unmarked normal case; only the exceptions read "Manual only". Nothing is badged for doing the expected thing. - **What an empty brief means.** Not "never touched" — that was the old, opt-in reading. Agents work from the label and the type alone; the brief sharpens what would count. Opting out is `agentFilled: false`, one switch. **Never "eve" in the product.** UI copy says *your agents*. `eve` is the runtime in `apps/agent`, an implementation detail a rep should never have to learn, and a self-hoster may well point this at their own. The word appears in this document and in the code; it does not appear on screen. Nest still writes rows and fills nothing. Creating or editing a definition with `agentFilled` writes **one** `AgentTask` (`kind: "field-backfill"`, `reason` = the key) — one row per field, not one per record. The agent leases it and decides the fan-out, batching and priority itself, which is the point of the rule in `api.md` and the reason a workspace with 5,000 companies does not get 5,000 rows from one click. `agentBrief` is the whole instruction, in the rep's words: > Which cloud they run production on. Their docs, status page or engineering job > ads say it outright — infer nothing from a logo on a customer wall. The self-hoster's answer is unchanged and still needs no special case: with no agent running, nothing leases the task, nothing proposes, and every field is one you type into. A missing capability removes a capability. It does not throw. **Proposed, not written** — and this is what makes agent-first safe to default on. A value an agent finds arrives the way a `ContactFact` arrives: `PROPOSED`, with a score, a band and evidence, rendered by the `Suggestion` component that already exists, accepted or dismissed by a human. A dotted underline (`SOURCED_VALUE`) then marks what an agent put there, and hover gives the source. Nothing a rep typed is ever overwritten. Turning this on by default would be reckless if agents wrote straight into the field. They do not, so the cost of a wrong default is a suggestion someone dismisses — against a rep never discovering the feature at all, which is the cost of the opt-in version. That behaviour is already built; the fork is that `ContactFact` is contact-only and keys on `field String`. Recommendation: **generalise `ContactFact` to `RecordFact`** with the same three nullable FKs and an optional `fieldId`, rather than growing a parallel proposal table for custom fields. It is the larger migration and the smaller system. This is the one decision in this plan that should be made before any of it is built, because it is the only one that gets more expensive later — and agent-first makes it load-bearing rather than optional. ## 6. What the UI needs that does not exist Everything in the design is already in `packages/ui` — Sheet, Input, Select, Switch, Checkbox, Button, Suggestion, the Inline\* editors — with one exception. **There is no drag-and-drop library in the repo.** The design shows a drag handle on every row. Either: 1. add `@dnd-kit/core` + `@dnd-kit/sortable` to `packages/ui` and build a `SortableList` there, which is where a shared interaction belongs; or 2. ship reordering as Move up / Move down in the row's overflow menu, and add dragging later. Option 2 is a smaller first cut and the ordering it produces is identical. Option 1 is what the artboards show. Worth deciding explicitly rather than by accident. ## 7. Deliberately out of scope - **Filtering and sorting the tables by a custom field.** The columns exist (`showOnTable`), the indexes support it, `resolveOrderBy` would need to learn about field keys. Phase two. - **Quick-add.** New fields do not appear in quick-add unless `required`. - **Per-record fields.** A field belongs to an entity, never to one company. - **Formula, rollup and relation types.** Not in the first cut. --- ### Plan Gmail Calendar Plan (`docs/plan/gmail-calendar-plan.md`) # Plan — Gmail & Calendar Put the conversation back in the CRM. Three features, in the order they should be built: 1. **Meetings on the record.** A calendar event with someone from Acme shows on Acme's timeline — before it happens, and after. 2. **Email threads in-line.** A thread with someone from Acme shows on Acme's timeline, collapsed to a subject and a snippet, expandable to the messages. 3. **Auto-created companies and contacts.** Booking a meeting or holding a two-way thread with a new domain creates the company and the contact, so nobody types what Google already knows. Read it with [`AGENTS.md`](../../AGENTS.md), [`api.md`](../api.md), [`design.md`](../design.md) and [`crm-plan.md`](../crm-plan.md). The conventions there are assumed, not repeated: single tenant, tRPC as the data surface, services throw `HttpException`, freshness via `useCrmCache()`, all UI from `/packages/ui`. --- ## 1. The decisions that carry the design Six choices. Everything else follows. 1. **Gmail and Calendar are granted at sign-in, and using the CRM is conditional on them.** This is an internal, single-tenant tool where seeing the conversation on a record *is* the product, so a rep with an account and no mailbox is a half-configured account nobody notices is broken. The scopes go on the provider config; because Google's granular consent lets someone untick one and still finish signing in, the app also gates on what was actually granted. §3. 2. **Google's per-mailbox ids are not identities.** Gmail `threadId` is scoped to one mailbox, so two reps on one thread would create two CRM threads. Identity is the RFC 822 `Message-ID` for email and `iCalUID` for calendar, both of which are global. §4.2. 3. **Email and calendar get real tables; the timeline gets a projection.** One `Activity` row per thread and per event, linked by FK. The timeline query, its indexes, its filters and its cursor paging are untouched. §4. 4. **Nothing is stored unless it is relevant.** A message is persisted only if a participant resolves to a company we track, or qualifies to create one. A rep's inbox contains their doctor. §5. 5. **Auto-creation needs two-way engagement.** A thread the rep replied to, or a meeting on the calendar. Not a received newsletter. §6. 6. **Sync is cron-driven and its cursor lives in Postgres.** The API is a serverless function; there is no worker to hold state, and the existing in-process `EnrichmentQueue` deliberately drops work on shutdown. §7. ### Choice table | Decision | Choice | Why | | --- | --- | --- | | Connect flow | Scopes on the provider config, requested at sign-in | §3.1. Internal tool — mailbox access is a condition of having an account. | | Enforcement | `requireGoogleAccess()` gate in the app shell | §3.1. Granular consent means "signed in" never implies "granted". | | Repair path | `authClient.linkSocial({ provider: "google", scopes })` | §3.1. Re-consent for a partial grant or a pre-existing account. | | Connected? | `Account.scope` contains both scopes | §3.1. The grant itself is the state — nothing to keep in sync. | | Token refresh | `auth.api.getAccessToken({ providerId: "google", userId })` | Better Auth refreshes on demand. No hand-rolled refresh. | | Disconnect | There isn't one — only purge-and-re-import, or revoke | §3.4. Syncing is mandatory, and a local disconnect would be undone by the next reconcile anyway. | | Scopes | `gmail.readonly`, `calendar.readonly`, read-only | Nothing writes to Google in v1. Sending mail is a separate decision. | | Google app type | **Internal** (Workspace-only) | Skips OAuth verification entirely. §3.3. | | Email identity | `Message-ID` header, unique | Gmail `threadId` is per-mailbox. §4.2. | | Thread identity | Root of `References` / `In-Reply-To`, unique | Same conversation, one CRM thread, whoever synced it. | | Event identity | `(iCalUID, originalStartTime)`, unique | `iCalUID` is shared across attendees *and* recurrence instances. §4.2. | | Timeline | One projected `Activity` per thread/event, FK-linked | Keeps the one-index-scan timeline in `activities.service.ts`. | | Relevance | Participant domain must resolve to a `Company`, or qualify to create one | §5. Privacy and junk prevention are the same rule. | | Bodies | Stored, plain text, quoted history stripped | "In-line" means readable. Never logged, never in list payloads. | | Sync trigger | Vercel Cron → authenticated internal route | §7. Pub/Sub push is a later upgrade behind the same handler. | | Sync state | `MailboxSync` row per account per source | Serverless has no memory between invocations. | | Backfill | **None.** Forward-only from first sight of a mailbox | §7.3. Nobody asked for three months of old mail in their CRM, and it deletes a whole class of machinery. | | Feature flag | None. Sync is simply on | §13. It gates nothing that can genuinely be absent, and defaults to off — so it is only ever wrong. | | Freshness | Poll while syncing, mirroring `enrichment-status` | A background write the browser did not cause. `api.md`. | | New UI | Added to `/packages/ui` | `design.md` — thread disclosure and attendee list are new primitives. | ### Judgment calls worth your veto - **Calendar ships before Gmail.** Smaller API, cleaner dedupe key, no body parsing, no privacy surface worth arguing about, and it proves the whole connect → cron → project → render pipeline on the easy case. Gmail then adds only Gmail's problems. If you want threads first, say so — but Phase 2 is about two weeks cheaper than Phase 3 for most of the perceived value. - **Read-only, both APIs.** No sending, no event creation, no "log this email" button writing back. It halves the scope review and removes every way this feature can damage a customer relationship. Writing is a v2 conversation. - **Auto-create fires without an approval queue.** The guardrails in §6 are tight enough that a review queue would mostly be a list of correct rows waiting for someone to click Accept — and approval queues rot. Instead every auto-created record is tagged with its provenance and is filterable, so a bad rule is visible and reversible in bulk. Say the word if you want the queue. - **We store message bodies.** The alternative — metadata plus a deep link to Gmail — is more private and much less useful, because the point is reading the thread without leaving the record. §10 covers what that obliges us to do. - **Personal mail is dropped at ingest, not stored-then-hidden.** Slightly more work at sync time, and the only version of this that survives someone reading the database. --- ## 2. What exists that we build on | Thing | Where | Use | | --- | --- | --- | | Google OAuth, Better Auth | `packages/auth/src/auth.ts` | Add `scope` and `accessType: "offline"` here. Scope strings live in `packages/auth/src/scopes.ts`, imported by the API and the app. | | `Account.accessToken` / `refreshToken` / `scope` | `schema.prisma` | Better Auth already persists per-provider tokens. | | `domainFromEmail()` + free-domain blocklist | `apps/api/src/companies/domain.ts` | The matching backbone. Already rejects gmail.com et al. | | `EnrichmentService.companyForEmail()` | `apps/api/src/enrichment/enrichment.service.ts:197` | Find-or-create company by domain, then enqueue enrichment. Auto-creation reuses this verbatim. | | `ActivityType.EMAIL` / `MEETING` | `schema.prisma` | The enum values already exist and are already in `NOTE_TYPES`. | | Timeline, cursor-paged, filter tabs | `apps/api/src/activities/activities.service.ts` | Extended, not replaced. | | Enrichment polling pattern | `apps/app/components/crm/enrichment-status.tsx` | Copy the shape for sync status. | | Cache fan-out | `apps/app/lib/trpc/cache.ts` | Add `cache.google()`; do not list keys at call sites. | --- ## 3. Connecting an account ### 3.1 Scopes at sign-in, and the gate behind them The scopes go on the provider config in `packages/auth/src/auth.ts`, so every sign-in asks for them: ```ts socialProviders.google = { ...env.google, scope: [...SYNC_SCOPES], // gmail.readonly, calendar.readonly accessType: "offline", // so Google issues a refresh token }; ``` Four behaviours make this work, all read out of the installed `better-auth@1.6.25` rather than assumed: - **Scopes merge, they don't replace.** The Google provider builds `["email", "profile", "openid"]`, then appends `options.scope`, then appends any per-call `scopes`. - **Incremental authorisation is already on.** The provider hardcodes `additionalParams: { include_granted_scopes: "true" }`, so a token issued for a widened grant covers the union rather than only the new scopes. - **Re-consenting updates the account in place.** The OAuth callback finds the existing account by `(providerId, accountId)` and updates the tokens, both expiries and `scope`. That makes the repair path idempotent. - **`Account.scope` is the connection state.** The granted scopes are persisted as a string, so "did they actually grant it?" is a column read. **Requesting is not the same as granting.** Google's granular consent lets someone untick a scope and finish signing in anyway, and any account created before these scopes were required still carries the old grant. Either way the person lands inside a CRM that looks signed-in and silently syncs nothing — so asking at sign-in is necessary but not sufficient, and the app gates on the result: - `requireGoogleAccess()` (`apps/app/lib/session.ts`) reads `Account.scope`, `cache()`d so it costs one query per request, and redirects to `/grant-access` when either scope is missing. - The `(app)` layout calls it instead of `requireSession()`, so the gate covers every page in the shell at once. - `/grant-access` lives outside the `(app)` group deliberately — a page inside it would be redirected by the very layout doing the gating. It explains what is read and why, and offers `linkSocial` to re-consent, plus a sign-out for anyone who would rather not. `prompt: "consent"` is deliberately **not** set: it is a provider-level option, so it would show the consent screen on every sign-in forever. It is not needed, because a first sign-in — and any sign-in asking for a scope not yet granted — prompts anyway, and it is that prompt which mints the refresh token. The missing-refresh-token case is detected (`hasRefreshToken`) and repaired rather than pre-empted. ### 3.2 Reading and refreshing tokens Never touch `Account.accessToken` directly. A `GoogleTokenService` wraps: ```ts const { accessToken } = await auth.api.getAccessToken({ body: { providerId: "google", userId }, }); ``` which refreshes if expired (within five seconds of expiry, per the implementation) and persists the new token. Confirmed present in 1.6.25 as `/get-access-token`. If it throws, the connection is dead — mark the `MailboxSync` row `NEEDS_RECONNECT` and surface a reconnect prompt; do not retry in a loop. ### 3.3 Google Cloud setup (Phase 0, and it gates everything) - One OAuth client, existing GCP project, add the two scopes. - Set the consent screen **User type: Internal**. This is the whole ball game. `gmail.readonly` is a **restricted** scope (Google's own classification, not a judgement call), and an External app using it needs OAuth verification *plus* a CASA security assessment, renewed annually — months of work. For an Internal app Google's documentation is explicit: *"scopes aren't listed on the consent screen and use of restricted or sensitive scopes doesn't require further review by Google."* This CRM is already internal-only behind Google sign-in, so the exemption costs us nothing. Two caveats worth writing down now, because both are expensive later: the exemption is tied to the **Internal user type specifically** — an External app that merely happens to target Workspace users does not get it — and going External at any point in the future means the full restricted-scope review. If this CRM will ever be sold or opened to another domain, that decision belongs here, not in a year. - Enable the Gmail API and the Google Calendar API on the project. - The callback is the **existing** `…/api/auth/callback/google`. Extending the grant adds no new redirect URI, which is one fewer thing to get wrong across preview environments. ### 3.4 There is no disconnect Making mailbox access a condition of having an account settles this section: there is no supported state where a signed-in rep is not syncing, so there is no disconnect button. That is not only a policy choice, it is what the mechanics allow. A local disconnect would delete the `MailboxSync` rows, and the next reconcile — which derives those rows from the grant — would rebuild them within five minutes. A button that appears to work and silently undoes itself is worse than no button. Better Auth also **refuses to unlink the last account** (`FAILED_TO_UNLINK_LAST_ACCOUNT`), and Google is our only sign-in, so unlinking was never available either. What the settings page offers instead: - **Delete and re-import** (`google.purgeSyncedData`) — deletes every thread and event this mailbox contributed, cascading to the projected activities, then resets the cursors so the window re-imports from scratch. This is the answer to "get my mail out of the CRM", and it should not need a migration. - **Revoke Google access** — calls Google's revoke endpoint. One OAuth client means one grant per user, so this ends *CRM access itself*, not just syncing: the scopes disappear, the gate catches the next request, and the person lands on `/grant-access`. Worded as such and behind a confirm, never as a default. The only design with independent revocation — mailbox access off, sign-in intact — is a **separate GCP OAuth client** for sync. Under a mandatory-sync policy that state is not wanted, so the second client stays unbuilt. §15. **The missing-refresh-token edge case** still applies: if Google decides not to re-prompt, no refresh token is issued and `getAccessToken` fails an hour later. `hasRefreshToken` detects it and the settings page shows a reconnect banner rather than the sync dying quietly. --- ## 4. Data model ### 4.1 New models ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### 4.2 Why those keys, specifically Both were checked against Google's docs rather than assumed, because getting either wrong produces duplicates that are painful to unpick later. - **Gmail `threadId` is mailbox-scoped.** If two of our reps are on one thread, each mailbox reports a different `threadId` for it. Deduping on `threadId` would put the same conversation on Acme's timeline twice. The RFC 822 `Message-ID` header is globally unique per message, so `EmailMessage` deduplicates on that, and the thread is keyed by the root message id — the first entry of `References`, else `In-Reply-To`, else the message's own `Message-ID`. - **`iCalUID` is shared across attendees' copies** — that is its purpose, per RFC 5545 and Google's own reference — **and it is also shared by every instance of a recurring series**, where only the `id` differs. So the key is `(iCalUid, originalStartTime)`: `originalStartTime` identifies the instance even if that instance has been moved. ### 4.3 Changes to existing models ```prisma enum RecordSource { MANUAL IMPORT EMAIL CALENDAR } model Company { // … source RecordSource @default(MANUAL) emailThreads EmailThread[] calendarEvents CalendarEvent[] } model Contact { // … source RecordSource @default(MANUAL) emailThreads EmailThread[] calendarEvents CalendarEvent[] eventAttendance CalendarAttendee[] } model Activity { // … /// The projection link. Unique: one timeline entry per thread and per event. emailThreadId String? @unique emailThread EmailThread? @relation(fields: [emailThreadId], references: [id], onDelete: Cascade) calendarEventId String? @unique calendarEvent CalendarEvent? @relation(fields: [calendarEventId], references: [id], onDelete: Cascade) } model User { // … mailboxSyncs MailboxSync[] } ``` `Activity.createdById` stays required and is set to the user whose mailbox ingested the item — but the UI renders synced entries as "via Gmail" rather than as something that person wrote, because they didn't. Migration: `bun run --filter=@crm/db db:migrate`, named `add_google_sync`. Nothing backfills; existing rows default to `MANUAL`. ### 4.4 Why project into `Activity` rather than union three tables The timeline is one indexed range scan over `activity(companyId, createdAt)`, with cursor paging and four filter counts. A union across three tables loses the index, makes the cursor a compound mess, and rewrites code that works. Instead sync writes one `Activity` per thread and per event and keeps it current: - New message in a known thread → update the projected activity's `occurredAt` and snippet. One row per thread, not per message. - Event moved or cancelled → update the projected activity, or delete it when the event is cancelled. The row is a summary. Expanding it fetches the messages or the attendees through a separate procedure (§8), so the timeline payload does not carry email bodies. --- ## 5. Matching: from a Google payload to a CRM record One resolver, `GoogleMatchService`, used by both syncs. Given the participant addresses of a thread or an event: 1. **Drop our own people.** Any address on our own Workspace domain, and any address belonging to a `User`. What is left is the external side. 2. **Drop noise.** `domainFromEmail()` already returns `null` for free hosts. Also drop `SuppressedDomain` hits and no-reply local parts (`noreply`, `no-reply`, `notifications`, `donotreply`, `mailer-daemon`, `bounce`, `via .*` senders). 3. **Contact first, then company.** An exact `Contact.email` match wins — `Contact.email` is already unique. Otherwise resolve the domain against `Company.domain`, which is already unique and normalised. 4. **If nothing matches**, hand off to §6. If §6 declines, **the item is not stored at all.** 5. **Stamp both ids.** Per the existing timeline convention, an activity carries `companyId` even when it is really about a contact, so the company timeline stays a single scan. Multiple external domains on one thread (a customer plus their lawyer) attach to the domain with the most participants, tie-broken by whichever already exists as a `Company`. It is a heuristic; it is also visibly wrong when it is wrong, and re-attaching is a single field edit. --- ## 6. Auto-creating companies and contacts The rule is **two-way engagement**, evaluated per source: | Source | Creates when | Rationale | | --- | --- | --- | | Calendar | An event with ≥1 external attendee, status not `cancelled`, and our user is organiser or has not declined | Somebody put time in a diary. That is as strong a signal as this CRM gets. | | Gmail | The connected user has **sent** ≥1 message in the thread | A reply proves a relationship. Inbound-only is a newsletter, a recruiter, or spam. | Then, in both cases, all of: - The external domain is non-free (`domainFromEmail()` returns non-null), not our own, and not in `SuppressedDomain`. - The sender is not a no-reply address. - The thread has ≥1 message, or the event ≥1 confirmed external attendee. Creation itself reuses what exists: **`EnrichmentService.companyForEmail(email)` already does exactly this job** — normalise the domain, find or upsert the `Company`, and enqueue enrichment so the logo, industry and description fill in behind the scenes. Contacts are created alongside, with the display name from the `From` header or the attendee's `displayName` split into first/last, falling back to the local part. Everything created this way is stamped `source = EMAIL | CALENDAR`, which gives a filter on the companies and contacts tables, a bulk undo, and an honest answer to "where did all these come from". A per-user **"Auto-create records" toggle**, default **on for calendar, off for Gmail** on first connect. Meetings are unambiguous; the first week of mailbox auto-creation is where a bad rule shows up, and it should be a deliberate opt-in after the rep has seen what the sync matches. --- ## 7. Sync mechanics ### 7.1 Shape The API is a bundled Vercel function (`apps/api/api/index.ts`) — no long-lived process, so no in-memory scheduler. A **Vercel Cron** hits an internal route every 5 minutes; the route iterates connected accounts and runs one incremental pass each. ```ts // apps/api/vercel.ts (vercel.ts is the current recommendation over vercel.json) crons: [{ path: "/internal/sync/google", schedule: "*/5 * * * *" }], ``` The route sits outside tRPC — it is not a user action — and is guarded by a `CRON_SECRET` bearer check plus Vercel's cron header, alongside `/health` as one of the few non-tRPC surfaces `api.md` permits. It returns immediately after a bounded amount of work. > **Plan check:** minute-level cron needs a Pro plan; Hobby is once a day. Worth > confirming before Phase 2. ### 7.2 Incremental sync Both APIs are cursor-based, and both invalidate cursors. Verified behaviour: - **Gmail** — `users.history.list(startHistoryId)` returns changes since a `historyId`. History records are retained "typically at least one week"; an out-of-range `startHistoryId` returns **HTTP 404**, and the client must fall back to a full sync. We do not: forward-only means the cursor is re-pointed at now and the gap is dropped, because refetching it would be importing old mail. - **Calendar** — `events.list` returns `nextSyncToken` on the final page. An expired or ACL-invalidated token returns **HTTP 410 Gone**, requiring a full resync. Incremental results always include deleted entries, so cancellations arrive as events to remove — delete the projected activity when one does. ### 7.3 No backfill — forward-only Nothing from before a mailbox was first seen is ever imported. - **Gmail** — the first pass calls `users.getProfile`, stores the current `historyId` as the cursor, and imports **nothing**. Every later tick is `history.list` from that point, so only mail that arrives after connection is read. When the cursor falls out of Gmail's retention window (a 404, which takes roughly a week of the sync not running), the gap is *not* fetched: the cursor is re-pointed at now and the miss is logged. - **Calendar** — `timeMin` is `now`, never earlier. Events that *start* from now onwards are imported, including ones booked before sync was switched on, because a meeting in next Tuesday's diary is the entire point of the feature and putting it on a timeline is not back-dating anything. Past meetings are never read. What this deletes, which is most of the reason to do it: the chunk loop, the `backfilledTo` resume column, the `BACKFILLING` status, `GMAIL_BACKFILL_DAYS`, the cursor-invalidation path that re-entered a backfill, and the progress UI that had to explain all of it. A tick is now one shape — "what changed since the cursor?" — rather than two. The cost is honest and small: a company added today shows conversations from today. The history is in Gmail, where it always was. ### 7.4 Idempotency and ordering Every write is an upsert on the natural key from §4.2, so re-running a window is free. Two reps' syncs racing on the same thread is settled by the unique index on `rfcMessageId`, exactly as the `Company.domain` index already settles two contacts arriving at a new company. ### 7.5 Later: near-real-time Gmail `users.watch` + Cloud Pub/Sub push turns 5 minutes into seconds. It also adds a Pub/Sub topic, a push endpoint, and a `watch` registration that **expires every 7 days** and must be renewed on a schedule. Build it behind the same handler the cron calls, once the polling version has proven the matching rules. Not Phase 2 work. --- ## 8. API surface One module, `apps/api/src/google`, one router per `api.md` (`google.router.ts`, `@Router({ alias: "google" })`, `@UseMiddlewares(AuthMiddleware)`), thin over services. | Procedure | Kind | Notes | | --- | --- | --- | | `google.status` | query | Per-source: connected, `status`, `lastSyncedAt`, `lastError`, auto-create toggles. Drives the settings page and the poll. | | `google.purgeSyncedData` | mutation | Deletes every thread and event this mailbox contributed. Syncing continues forward from the current cursor. Not a disconnect — see §3.4. | | `google.revokeAccess` | mutation | The explicit hard revoke. Calls Google's revoke endpoint; the rep re-consents at next sign-in. Confirmed in the UI, never the default. | | `google.syncNow` | mutation | Enqueues a pass for the calling user. Rate-limited; it is a nicety, not the mechanism. | | `google.setAutoCreate` | mutation | `{ source, enabled }`. | | `google.suppressDomain` | mutation | "Not a customer" — adds to `SuppressedDomain` and optionally deletes what it matched. | | `email.thread` | query | `{ threadId }` → messages with bodies. Only called on expand. | | `calendar.event` | query | `{ eventId }` → attendees, conference link, description. | Connecting needs no procedure — the client calls `authClient.linkSocial()` directly, and `google.status` reads the result off `Account.scope`. Extensions to existing surfaces: - `activities.contracts.ts` — add `"email"` and `"meetings"` to `TimelineFilter`; `activities.service.ts` — the matching arms in `filterClause()` and the extra counts in `timelineCounts()`. - `ENTRY_SELECT` — include a *summary* of the linked thread/event (`messageCount`, `lastMessageAt`, `startsAt`, attendee count). Not bodies. - `companies.service.ts` / `contacts.service.ts` — `nextMeetingAt` and `lastEmailAt` on the detail payload, and `source` as a filterable column. Regenerate and **commit** `apps/api/src/generated/server.ts` with the router change, per `api.md`. --- ## 9. UI ### 9.1 New primitives — in `/packages/ui`, per `design.md` Nothing bespoke in `apps/app`, no `className` overrides, no new radii: - `accordion.tsx` — via `bun run --filter=@crm/ui ui:add accordion`. The thread disclosure. Not currently in the package. - `thread-message.tsx` — one message: sender, time, direction, body. Composes `avatar`, `separator`. - `attendee-list.tsx` — stacked avatars with response status, over `avatar` and `tooltip`. - `connection-card.tsx` — a connected-service row with status and actions, over `card`, `status-indicator`, `button`. Reused by any future integration. ### 9.2 Timeline `timeline-entry.tsx` gets two branches beside the existing `STAGE_CHANGE` one: - **EMAIL** — subject, `"3 messages · latest 2h ago"`, direction arrow, snippet. Expands into an accordion of `ThreadMessage`s, fetching `email.thread` on first open. Deep-links to Gmail via the stored `gmailMessageId`. - **MEETING** — title, time range, `AttendeeList`, conference link. An upcoming meeting sorts by `startsAt` and reads "in 2 days"; `relativeTimeFromIso()` already exists in `@crm/ui/lib/format`. `activity-icon.tsx` already maps both types. Filter tabs gain **Email** and **Meetings** (`timeline-search-params.ts`, nuqs, per the existing pattern). ### 9.3 Settings A new `apps/app/app/(app)/settings/page.tsx` — there is no settings route yet. **One card, not one per source.** Both scopes come from a single grant at sign-in, so "Gmail connected but Calendar not" is not a state that exists, and rendering it as two identical cards invented a distinction the system does not have. The card carries connection health, when it last checked, and the two auto-create toggles — the only decision on the page with a consequence. The destructive actions from §3.4 sit underneath as quiet text links, because neither is anybody's daily business. No connect button: the gate guarantees the scopes before anyone reaches this page. ### 9.4 Freshness Sync is a background write no client action caused, so per `api.md` it is **polled, not invalidated**. Mirror the enrichment pattern exactly — a `components/crm/sync-status.tsx` exporting `isSyncing()` and `SYNC_POLL_MS`, so the rule has one definition. Poll on the settings page *and* on the record sheet while a sync is running, remembering the lesson already recorded in `api.md`: the company sheet polled and the companies table did not, and a newly-added company stayed blank in the table until a reload. User-initiated actions still invalidate through `useCrmCache()`. Add a `cache.google()` that fans out to the activity keys plus companies and contacts lists — and note that `activities.timeline` must use `pathKey()`, not `queryKey()`, because it is read both as an infinite query and as a plain one. --- ## 10. Privacy, security, logging This is the part that makes the feature acceptable rather than merely working. - **Relevance filtering at ingest, not at read.** Non-matching mail is never written. §5. - **Never log content.** `api.md` already bans headers, query strings and bodies. Extend it in spirit: no subjects, no snippets, no addresses in log lines. Log counts, ids, and domains. `logger.log({ message: "Gmail sync", userId, threadsWritten, messagesWritten })` — one object, per the rules. - **Bodies never appear in list payloads.** `ENTRY_SELECT` carries a snippet; bodies come only from `email.thread`, one thread at a time. - **Read scopes only**, and the reconnect path revokes and re-consents rather than silently widening. - **Disconnect means it.** Deleting the account row stops sync; `{ purge: true }` removes the messages, threads, events and their projected activities. Someone will ask for this on their last day, and the answer should not be a migration. - **Every rep sees every synced thread** — that follows from "signed in is the entire authorisation model" in `api.md`, and it is a real consequence worth saying out loud before the first sync runs, not after. If that is not acceptable, it is a row-level permissions decision that belongs in this plan now rather than as a retrofit. --- ## 11. Failure modes to design for | Failure | Response | | --- | --- | | Refresh token revoked | `NEEDS_RECONNECT` on the sync row, banner in settings, stop trying. | | `historyId` too old (404) | Clear cursor, resume from now, log the gap. §7.3. | | `syncToken` gone (410) | Clear cursor, re-read from `now` forward. §7.2. | | Google rate limits (429/403) | Exponential backoff persisted on the sync row; skip that account this tick. | | Cron tick overruns | Chunked work with a wall-clock budget; the cursor makes the next tick resume, not restart. | | Two reps, one thread | Unique `rfcMessageId`. §4.2. | | Auto-create makes junk | `source` filter → bulk delete → `SuppressedDomain`. §6. | | Deploy mid-tick | The cursor only advances on a settled pass, so at worst a window is re-read — every write is an upsert. | --- ## 12. Phases | Phase | Scope | Done when | | --- | --- | --- | | **0 — Google Cloud** | Internal consent screen, both APIs enabled, redirect URIs for all envs | Consent screen shows the two scopes; no verification required. | | **1 — Connect** | Scopes on the provider, `accessType: "offline"`, `requireGoogleAccess()` gate, `/grant-access`, `GoogleTokenService`, `MailboxSync`, settings page, `google.status`. No sync. | A fresh sign-in grants both scopes and lands in the app; a user who unticks one is bounced to `/grant-access` and can re-consent; an account predating the requirement is caught by the same gate. | | **2 — Calendar** | Calendar sync, `CalendarEvent`/`CalendarAttendee`, matching, projection, `MEETING` timeline entry, Meetings filter | A meeting with an existing company appears on its timeline within 5 minutes, and a cancellation removes it. | | **3 — Gmail** | Gmail sync, `EmailThread`/`EmailMessage`, thread accordion, Email filter | A thread with an existing company appears collapsed and expands to readable messages. | | **4 — Auto-create** | §6 rules, `RecordSource`, `SuppressedDomain`, filters, bulk undo, toggles | A meeting with an unknown work domain creates an enriched company and a contact; a newsletter creates nothing. | | **5 — Real-time (optional)** | `users.watch` + Pub/Sub push, 7-day renewal | Threads land in seconds; the cron stays as the safety net. | Phases 1–3 are the user's stated ask. Phase 4 is the "would be cool", and it should not ship before Phase 3 has run on real mailboxes for a week — the matching rules need to be observed against real mail before they are allowed to write rows. --- ## 13. Environment | Variable | Where | Notes | | --- | --- | --- | | `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` | exists | Unchanged. Same client, same callback — the grant is just widened at sign-in. | | `CRON_SECRET` | new | Bearer guard on `/internal/sync/google`. The route fails closed without it, so a local checkout simply never runs the cron. | Nothing else. There is deliberately **no feature flag**: a switch earns its keep only when it gates something that can genuinely be absent, and sync has everything it needs the moment the app boots — the OAuth client is mandatory because Google is the only sign-in, and the scopes are a condition of having an account. A switch that can disable a mandatory feature, defaulting to off, is a switch that is only ever wrong. **Our own domains are derived** from the `User` table rather than configured, so they cannot go stale when the team adds a second one. All added to `EnvironmentVariables` in `apps/api/src/config/env.validation.ts` with `class-validator` decorators, and to `docs/environment.md`. --- ## 14. Testing - **Unit, no network.** Matching (§5) and the auto-create rules (§6) are pure functions over parsed payloads — table-driven `bun test` with fixtures for the awkward cases: two external domains, a no-reply sender, a free-host address, a thread the rep never replied to, a declined meeting, a moved recurrence instance. - **Idempotency.** Run a fixture window twice; assert row counts are identical. - **Cross-mailbox dedupe.** Ingest the same conversation as two different mailboxes with different `threadId`s; assert one `EmailThread`, one projected `Activity`. - **Cursor invalidation.** Stub 404 and 410 and assert the cursor is cleared and the next pass resumes from now, rather than a crash or a silent stall. --- ## 15. Open questions 1. **Whose mail syncs?** Every signed-in user who connects, or a defined sales group? Affects volume and the §10 visibility question. 2. **Visibility.** §10 — every rep seeing every synced thread follows from the current auth model. Confirm that is intended. 3. **Retention.** Do synced messages age out (say, 24 months), or live as long as the company record? 4. **Attachments.** Out of scope here. `crm-plan.md` §7 already picks Vercel Blob for CRM attachments; email attachments would reuse it, but v1 stores names and sizes only, not bytes. 5. **Cron granularity.** Confirm the Vercel plan supports `*/5`. 6. **Does hard revocation need to be per-feature?** §3.4 — with one OAuth client there is one Google grant, so revoking mailbox access also revokes sign-in. Under the mandatory-sync policy that is arguably the correct behaviour, since an account that will not grant the scopes cannot use the CRM anyway. Revisit only if someone needs "keep my login, stop reading my mail", which the policy currently says is not a state we support. --- ### Plan Mcp Catalog Research (`docs/plan/mcp-catalog-research.md`) # MCP catalog research Checked on 2026-10-20. ## Supabase - Resource: `https://mcp.supabase.com/mcp`. - Authorization server: `https://api.supabase.com`. - Dynamic registration: supported. - PKCE: `S256` supported. - Read-only transport: `https://mcp.supabase.com/mcp?read_only=true`. - Protected resource metadata: `https://mcp.supabase.com/.well-known/oauth-protected-resource/mcp`. - Authorization metadata: `https://api.supabase.com/.well-known/oauth-authorization-server`. - Official tools: `https://supabase.com/docs/guides/ai-tools/mcp`. - Discovery source: `https://integrations.sh/supabase/supabase/`. The catalog includes Supabase. The user must complete OAuth before live tool tests can run. ## Vercel - Resource: `https://mcp.vercel.com/`. - Authorization server: `https://vercel.com`. - Dynamic registration: advertised. - PKCE: `S256` supported. - Protected resource metadata: `https://mcp.vercel.com/.well-known/oauth-protected-resource`. - Authorization metadata: `https://vercel.com/.well-known/oauth-authorization-server`. - Official tools: `https://vercel.com/docs/agent-resources/vercel-mcp`. Vercel limits access to reviewed MCP clients. Comp AI CRM is not in Vercel's supported client list. The catalog must not expose a connection that Vercel refuses. Blocker: submit Vercel's client review form. Add Vercel only after Vercel approves Comp AI CRM. --- ### Plan Mcp Integrations (`docs/plan/mcp-integrations.md`) # Spec — Extensible MCP integrations Status: ready for implementation. This work adds reusable MCP integrations to the CRM. Use [comp-neo](https://github.com/trycompai/comp-neo) as the reference implementation. Use its `staging` branch because that branch contains the current integration system. Read these Neo areas before implementation: - `packages/mcp-integrations` - `apps/api/src/modules/integrations/mcp` - `apps/agents/compliance/agent/lib/agent-connections.ts` - `apps/web/app/[org-slug]/(workspace)/settings/integrations` Port the behavior into CRM architecture. Do not add comp-neo as a package dependency. ## Outcome A workspace owner connects a supported platform once. The agent builder shows the active connection as a capability. The builder grants only the tools that one agent needs. The deployment review shows every granted tool and access level. The runner calls only tools approved in that deployed version. Disconnecting a platform blocks new calls immediately. The first complete integration is Linear. Add Supabase and Vercel after the Linear flow passes every acceptance test. ## Product rules 1. A connection is a capability. 2. An agent owns the automation. 3. Settings manages connections only. 4. The builder chooses how an agent uses a connection. 5. Deployment is the approval boundary for unattended runs. 6. Every external call uses the deployed version's exact grant. 7. New or changed vendor tools stay blocked until review. ## User flow ### Connect a platform 1. The owner opens **Settings → Connections**. 2. The owner selects a platform card. 3. The owner chooses **Read only** or **Read and write**. 4. The CRM shows the requested access in plain language. 5. The owner continues to the platform's OAuth screen. 6. The platform returns the owner to the CRM. 7. The card shows **Connected** after token storage and tool discovery succeed. The page uses **Brings in** and **Sends** for capability descriptions. The page does not show raw OAuth scope names as primary copy. The management view shows scopes for technical review. Only owners and admins can connect, reconnect, or disconnect a platform. The server enforces this rule at every entry point. ### Add a connection to an agent 1. The user opens the agent builder. 2. The user tags a connected platform. 3. The builder reads the platform's reviewed tool inventory. 4. The builder selects the smallest tool set that completes the requested work. 5. The builder explains every selected tool in plain language. 6. The saved version includes the exact tool names and fingerprints. 7. The review screen shows the platform, tools, access, and possible changes. 8. The user deploys the immutable version. An unavailable platform produces a connect prompt. A disconnected platform never produces a usable runtime tool. ### Run an agent 1. The runner loads the deployed manifest. 2. The runner resolves the current connection. 3. The runner checks the connection status and access level. 4. The runner checks the tool name and fingerprint. 5. The runner writes an action ledger row before a changing call. 6. The runner calls the platform through its MCP server. 7. The runner stores the result state without storing secrets. The runner refuses every call outside the deployed grant. The runner refuses changed tools until a new version receives approval. ## Architecture ### Shared catalog Create `packages/mcp-integrations`. Use one file for each supported platform. Each catalog entry contains: - A stable slug. - The platform name. - The MCP resource URL. - Allowed authorization server origins. - Allowed OAuth endpoint origins. - Read-only scopes. - Read-and-write scopes. - Risk overrides for known tools. - Allowed documentation hosts. - Source links and the last review date. Use `integrations.sh` for discovery. Treat the vendor's live metadata and documentation as the authority. The catalog contains technical facts only. User-facing descriptions live in the app. ### Authentication Use OAuth dynamic client registration and PKCE with `S256`. Reject a platform without both features. Reject every non-HTTPS endpoint. Reject every unapproved issuer or endpoint origin. Keep OAuth state short-lived and single-use. Bind the state to the user, platform, access mode, and return path. Store access tokens and refresh tokens in encrypted form. Never return either token to the browser, model, transcript, or log. Keep configuration in the root `.env`. Document every new variable in `.env.example`. A missing encryption key disables MCP connections without breaking the CRM. `packages/auth` owns OAuth registration, exchange, refresh, and revocation. The API owns authenticated control endpoints and connection state. ### API boundary The API exposes tRPC procedures for these operations: - List catalog entries and connection states. - Start a connection. - Read one connection. - Reconnect a connection. - Disconnect a connection. - Read the cached tool inventory. The API does not call MCP tools. The API does not classify tool risk. The API does not decide which tools an agent needs. The API writes a durable agent task after a successful connection. The task tells the agent to discover and cache the tool inventory. ### Agent boundary The agent owns the MCP client and every vendor tool call. Use Eve dynamic connections for the active platform set. Read the installed Eve connection guide before writing this code. The agent parses every MCP response at the boundary. The agent treats tool descriptions and results as untrusted data. The agent never follows instructions found in a tool result. ### Data model Add `McpConnection` with these facts: - Platform slug. - Connection status. - Access mode. - Granted scopes. - Encrypted access token. - Encrypted refresh token. - Token expiry. - Connecting user. - Connected time. - Last successful call time. - Last error code. Add `McpClientRegistration` for dynamic OAuth client registrations. Reuse one valid registration for the same platform metadata. Add `McpToolSnapshot` with these facts: - Connection ID. - Tool name. - Description. - Input schema. - MCP annotations. - Stable fingerprint. - Last seen time. - Review state. Parse every JSON column with Zod at its read boundary. Keep shared schemas in `packages/validation/src`. ### Agent manifest Extend integration resource IDs with `mcp:`. Add an MCP tool action to the manifest. The action contains the platform slug and approved tools. Each approved tool contains: - Its exact name. - Its reviewed fingerprint. - Its call kind. - A plain summary. Call kinds are `read`, `change`, and `risky`. The manifest schema rejects duplicate platform and tool grants. The review screen derives its access list from this manifest. The runtime parses the manifest before it reads any field. ### Tool fingerprints Build a fingerprint from the tool name, description, schema, and annotations. Use stable JSON ordering before hashing. A new tool starts blocked. A changed fingerprint starts blocked. A removed tool makes the affected agent version unavailable. Tool review creates a new agent version. It never changes a deployed version in place. ### Call safety Classify each tool before the builder can grant it. Use MCP annotations, the tool name, and catalog risk overrides. Treat these operations as risky: - Deletes and irreversible changes. - Access, role, member, or permission changes. - Production deployments and rollbacks. - Billing and purchase operations. - Code, shell, or SQL execution. - Reads of secrets, tokens, logs, or raw customer data. An unknown tool is risky. A failed classification is risky. Read-only connections can grant only read tools. Read-and-write connections can grant all reviewed tools. Unattended runs execute only exact deployed grants. Interactive sessions ask before ungranted changes. Write every changing call to `AgentAction` before execution. Use Eve's call ID as the replay key. Record success, failure, and the external tool name. Do not record secret arguments or raw sensitive results. ### Network safety Fetch metadata through `@crm/db/safe-fetch`. Apply the same SSRF rules to MCP endpoints. Resolve and validate every redirect target. Keep timeouts, size limits, and retry limits in one integration config module. Use `as const` for all configuration values. Cap tool results before they enter the model or event archive. Return a clear truncation marker with the retained data. ## Adding a catalog integration Complete every step for each platform. 1. Find the platform on `integrations.sh`. 2. Read the platform's official MCP and OAuth documentation. 3. Read the live protected-resource metadata. 4. Read the live authorization-server metadata. 5. Confirm dynamic client registration and PKCE `S256`. 6. Confirm every issuer, endpoint origin, scope, and MCP resource URL. 7. Read the live tool list and tool documentation. 8. Add catalog risk overrides for misleading tools. 9. Add the catalog entry, display copy, and brand mark. 10. Add unit tests for metadata, scopes, and risk overrides. 11. Complete both access modes with a test account. 12. Run at least one read and one approved change. 13. Add user documentation in the same pull request. Stop the addition when the platform needs a manually registered OAuth client. Stop the addition when the platform requires a private redirect allowlist. Record the blocker in the catalog research notes. ## Delivery plan ### Phase 1 — Linear connection Build the catalog, data model, OAuth flow, and connection page. Connect Linear with read-only and read-and-write access. Discover and cache Linear tools through the agent. Done means both connection modes survive token refresh and server restart. ### Phase 2 — Builder grants Show Linear in builder resource search. Let the builder select exact Linear tools. Show the grants on the version review screen. Done means deployment refuses an unreadable or stale tool grant. ### Phase 3 — Runtime calls Build Eve dynamic connections and manifest enforcement. Add action ledger support for MCP calls. Block disconnected, changed, and ungranted tools. Done means a replay cannot execute one changing call twice. ### Phase 4 — Catalog growth Add Supabase and Vercel through the documented catalog process. Fix the shared platform when either integration exposes a general defect. Done means a new catalog entry needs no new runtime branch. ## Tests Add unit tests for these cases: - Catalog parsing and stable slugs. - Origin and resource validation. - Scope selection for both access modes. - Tool classification and risk overrides. - Stable tool fingerprints. - Manifest parsing and duplicate rejection. Add integration tests for these cases: - Owner and admin connection access. - Member connection refusal. - OAuth state expiry and replay refusal. - Token encryption and refresh. - Disconnect during an agent run. - Changed tool refusal. - Read-only mutation refusal. - Action replay protection. - Missing optional configuration. Add one end-to-end test for each catalog platform. The test connects, deploys one agent, runs one read, and disconnects. ## Release criteria The work is complete when every statement below is true. - Linear, Supabase, and Vercel connect from Settings. - The builder lists only active connections. - A deployed version names every allowed MCP tool. - A new or changed tool never runs without review. - Read-only access blocks every changing call. - Disconnecting blocks the next call. - Tokens never appear in browser data, logs, prompts, or events. - Missing MCP configuration leaves the rest of CRM operational. - Tests cover OAuth, authorization, manifests, tool drift, and replay safety. - User documentation explains connection access and agent grants. ## Later work Custom MCP server URLs are outside the first release. API-key connections are outside the first release. Per-user connections are outside the first release. Public integration REST endpoints are outside the first release. Chat-wide permission modes are outside the first release. Add custom servers after the catalog flow proves its security boundaries. --- ### Plan People Enrichment Agent (`docs/plan/people-enrichment-agent.md`) # Plan — People enrichment agent (eve + LinkedIn) An [eve](https://eve.dev/docs) agent that works out who the people in the CRM actually are, using real profile data rather than inference. **Historical.** The agent this planned is built and [`agent.md`](../agent.md) describes what it does now. One decision below did not survive: LinkedIn no longer comes from a third-party LinkedIn API of its own. Context's people enrich endpoint reads a person back from a URL, under the same key that already buys company brand data, so there is one vendor and one key rather than two. Read it with [`AGENTS.md`](../../AGENTS.md), [`api.md`](../api.md), [`design.md`](../design.md) and [`gmail-calendar-plan.md`](./gmail-calendar-plan.md), whose sync is what puts these contacts in the CRM in the first place. --- ## 0. Phase 0 is done — what the API actually does Run against the real key on 2026-07-31, using addresses from our own CRM. This section exists because it **changed the architecture**, and because half of it contradicts the vendor's own marketing page. ### People search is broken. Do not build on it. `GET /api/v1/search/people?keywords=...` returns `HTTP 200` with a well-formed payload of **entirely unrelated people**. `keywords=marchetti fernhill` returned a list of Seattle-area engineers; `keywords=Paula Marchetti` returned Italian executives — Brightwater's CEO, an HR lead at Reply. Different random set each time. The parameter appears to be ignored. This matters more than a broken endpoint usually would: search was the entry point of the original design. Email → search → candidates → match was the whole identity-resolution flow, and it does not exist. ### What does work, and works well | Endpoint | Status | Notes | | --- | --- | --- | | `GET /api/v1/profile/overview?username=` | **Excellent** | Full name, headline, current positions with company URNs and logos, follower count, connection count, join date. | | `GET /api/v1/companies/company/info?name=` | **Excellent** | Id, universal name, LinkedIn URL, tagline, full description. Takes `name`, *not* `username`. | | `GET /api/v1/profile/contact-info?username=` | Works | — | | `GET /api/v1/profile/skills` | Exists, different params | Rejects `username`; wants a URN or profile id. | | `profile/experience`, `profile/posts`, `companies/company/employees` | **Do not exist** | 404 under the documented names, despite being on the marketing page. The real names need finding before anything depends on them. | ### The architecture this forces — and it is better LinkDAPI is an **enricher, not a finder**. Given a slug it is superb; given a name it is useless. So the resolver stage moves to a real search engine: ``` email + company → Context.dev web search (site:linkedin.com/in) → slug slug → LinkDAPI profile/overview → deep structured profile ``` Proven end to end on a real CRM contact: ``` tokonkwo@northwind.com → site:linkedin.com/in tokonkwo "Northwind" → tomi-okonkwo → Tomi Okonkwo — VP, Fund Finance & Venture Lending — Northwind Bank ``` `y` + `okonkwo`. That is the "Tokonkwo" row from the contacts list, resolved to a real person with a real title at the right employer. It also means the two vendors are used for what each is actually good at. Context.dev is a search engine and cannot read LinkedIn (LinkedIn blocks scrapers); LinkDAPI can read LinkedIn and cannot search. Neither alone solves this. ### Hit rate is partial, and that is the honest headline `pmarchetti@fernhill.com` resolved to **nothing** — no candidate slug at all. An unusual local part at an 8,000-person company is the hard case, and it is the one that prompted this work. Expect a meaningful miss rate on exactly the contacts that look worst in the UI. Two consequences: - The fallback matters. When no slug resolves, the existing Context.dev extract path still runs and may find a team page. - **A miss must stay a miss.** With search returning plausible strangers, the temptation to accept a near-match is exactly how Dario Fontana ends up filed as Paula Marchetti. §5's confidence bands are not ceremony. ### A bug this spike caught in already-shipped code Context.dev's `web.search` rejects `numResults < 10`. `ContextDevClient.search` defaulted to 5, so **every call would have failed validation** — the contact enrichment shipped last turn could never have worked. Fixed; worth recording as the reason to spike against a real key rather than a type definition. --- ## 1. What this is actually for The prompt for this was a contacts list reading `Pmarchetti`, `Tokonkwo`, `Chris` — names derived from an email address because Google Calendar returned no `displayName`. Fixing those is the *entry* case, and on its own it would not justify an agent. The reason to do this properly is what a profile carries beyond a name: | Signal | Why a CRM wants it | | --- | --- | | Full name, headline, title | The contacts list stops looking broken. | | Work history and tenure | "Started 3 months ago" is a different conversation from "eight years in seat". | | **Job changes** | A champion moving to a new company is the single highest-intent event in B2B sales, and nobody sees it today. | | Seniority and function | Who actually signs, versus who you have been emailing. | | Company headcount and growth | Qualification, without asking. | | Mutual connections | A warm path in. | The agent does **both**: it fills the fields *and* it produces the output. The fields are the substrate — you cannot write "you are meeting the VP who joined four months ago" without knowing the title and the start date — and the output is the reason anyone opens the CRM. Neither half is optional, and shipping only the fields is what makes enrichment feel like admin. Job-change detection is the prize. It is also impossible without a people data source, which is why the existing Context.dev path cannot grow into it alone. ### What exists today, and what happens to it `ContactEnrichmentService` (`apps/api/src/enrichment/contact-enrichment.service.ts`) already does a sourced web lookup: search, read a page, write only what the page states. It is deliberately timid — it refuses to guess, so it returns nothing for most people. It is **not** deleted by this plan. It becomes the fallback the agent reaches for when LinkedIn has no match, which is common for junior people, non-US markets, and anyone who does not keep a profile. §5. --- ## 2. The decisions that carry the design 1. **Curated tools, not a raw OpenAPI connection.** Not to save money — one won deal at $10–15k ARR funds more enrichment than this will ever run. Because the surface is not trustworthy: several advertised endpoints 404 and the search endpoint returns confident nonsense (§0). Tools pin the agent to what is verified to work. §4. 2. **The agent proposes; the CRM decides.** Writes go through one tool that stamps provenance, and a low-confidence identity match is approval-gated rather than merged. Getting a person wrong is worse than not knowing them. §6. 3. **Identity matching is the hard part, not data fetching.** `pmarchetti@fernhill.com` → which of the four Marchettis at a 8,000-person company? This gets a skill, a confidence rubric and evals. §5. 4. **A separate Vercel app, sharing `@crm/db`.** eve is its own runtime with its own build; it does not embed in NestJS. It reads and writes the CRM through the workspace database package, not through a new internal HTTP API. §3. 5. **Rate limits are a correctness concern, not a billing one.** A 429 mid-run is a half-enriched contact. The token bucket exists so the agent never discovers the tier cap. §7. 6. **Provenance on every field.** Which endpoint, which profile URL, when, how confident. Without it nobody can tell an enriched field from a typed one, and nobody will trust either. §6. ### Choice table | Decision | Choice | Why | | --- | --- | --- | | Framework | eve, `apps/agent`, own Vercel project | Durable sessions, approval gating, schedules and evals are all things we would otherwise hand-roll. | | Model | `anthropic/claude-sonnet-5` via AI Gateway | The default. Routes over project OIDC, so no provider key to manage. Matching is not a frontier-model problem; revisit if eval scores say otherwise. | | LinkedIn access | LinkDAPI, **profile + company endpoints only** | Its people *search* is broken — §0. Profile-by-slug and company-by-name are excellent. Superseded: Context's people enrich reads the same profile under the key we already hold. | | Finding the person | Context.dev web search resolves the LinkedIn slug | §0. The one thing LinkDAPI cannot do, and the thing a real search engine is good at. | | Tool surface | Curated tools, not an OpenAPI connection | Not cost — *predictability*. Half the advertised endpoints do not exist under the documented names (§0), so the surface has to be pinned to what is verified. | | CRM access | `@crm/db` directly | Already a workspace package. A second internal API would be plumbing with no reader. | | Trigger | eve schedule (Vercel Cron) over a bounded queue | Same shape as the Google sync — cursor in Postgres, work bounded per tick. | | Write path | One `update_contact` tool, provenance stamped | §6. | | Low confidence | eve `approval: always()` on merges below the bar | §6. Cheap to add now, impossible to retrofit trust later. | | Fallback | Existing Context.dev search | Covers the long tail LinkedIn misses. | | Retention | Raw payload in `ContactEnrichment`, mirroring `CompanyEnrichment` | Re-deriving a field later costs nothing — the pattern already paid off once, for `iconTone`. | | Coverage | Every contact with a work domain, re-checked on a cadence | Billing is not a constraint: one won deal at $10–15k ARR pays for far more enrichment than this will ever run. | ### Judgment calls worth your veto - **Curated tools rather than the full API.** You lose the agent's ability to improvise across endpoints we did not anticipate. You gain a predictable bill. If a use case turns up that the four tools cannot serve, add a fifth — that is cheaper than discovering an agent spent 4,000 credits exploring. - **Job-change detection is Phase 4, not Phase 1.** It needs a baseline to diff against, so it cannot exist until enrichment has been running a while. Worth saying out loud because it is the most valuable part and it is last. - **The agent never emails anyone.** It reads, matches and writes CRM fields. Outbound stays a human action. eve makes it easy to give an agent a channel and a send tool, and that is a different product decision with a different risk profile. - **Contacts only, initially.** LinkDAPI's company endpoints are tempting, but Context.dev already fills company firmographics and two sources writing the same columns is a merge conflict nobody owns. §8. --- ## 3. Where it lives eve is a filesystem-first framework: an agent is a directory, and the slot a file lands in determines how it loads. It builds and runs as its own app, so it becomes a third workspace app beside `api` and `app`. ```text apps/agent/ ├── package.json # name = "agent"; the root agent takes its name from here ├── agent/ │ ├── agent.ts # model + runtime config │ ├── instructions.md # what the agent is for and what it must not do │ ├── lib/ # shared helpers — import-only, never reaches the sandbox │ │ ├── linkdapi.ts # typed client: auth, retry, rate limit, credit accounting │ │ └── crm.ts # @crm/db queries the tools call │ ├── tools/ # one file per tool; filename IS the model-facing name │ │ ├── find_person.ts │ │ ├── get_profile.ts │ │ ├── find_colleagues.ts │ │ └── update_contact.ts │ ├── skills/ │ │ └── identity-matching.md │ └── schedules/ │ └── enrich.ts # becomes a Vercel Cron Job on deploy └── evals/ # sibling of agent/, not inside it ``` Naming is path-derived — `agent/tools/find_person.ts` is the tool `find_person`, and you never write a `name` field. **Deployment.** `eve link` creates/links a Vercel project and pulls its env. Schedules become Vercel Cron Jobs, evaluated in UTC. A string model id routes through the AI Gateway authenticated by project OIDC, so there is no Anthropic key to rotate. Note that `eve dev` never fires schedules on their cadence — there is a dispatch route for triggering one while iterating. **Monorepo wiring.** `apps/agent` joins the bun workspace and gets a `turbo.json` declaring `passThroughEnv` for `DATABASE_URL` and the vendor keys, matching how `apps/api` does it. `check-types` and `lint` join the root pipeline; `build` is `eve build`. --- ## 4. The tools Four, deliberately. Each one runs in the app runtime with access to `process.env` and `lib/`, and each is a place to put a rate limiter rather than hope the model paces itself. | Tool | Wraps | Notes | | --- | --- | --- | | `resolve_profile` | Context.dev `web.search`, `site:linkedin.com/in` | Turns an email + company into **candidate slugs**. Never returns a match — returns candidates for §5 to judge. This replaces the LinkDAPI search that does not work. | | `get_profile` | LinkDAPI `profile/overview` (+ `contact-info`) | One slug, in full. The deep data nothing else can reach. | | `get_company` | LinkDAPI `companies/company/info?name=` | Tagline, description, LinkedIn URL — company context for a brief. | | `research_web` | Context.dev `web.extract` | The fallback when no slug resolves, and the source of non-LinkedIn context (funding, launches) for briefs. | | `update_contact` | `@crm/db` | The only write. §6. | Two things go in `lib/linkdapi.ts` rather than in each tool: - **A token bucket sized to the tier.** Testing is 7 requests/minute, Hobby 30, Developer 70. Not a cost control — a 429 in the middle of a run leaves a half-enriched contact, and the agent must not be the thing that discovers the cap. - **Call accounting.** Endpoint plus the contact it was for. Not for the bill, for the debugging: when a profile comes back wrong you want to know exactly what was asked. A note from eve's own docs that applies here: *a step interrupted mid-execution re-runs*, so anything non-idempotent must be made idempotent or gated. Our writes are upserts keyed on contact id, so a replay is free; the *credit spend* is not, which is another reason the counter is on the client and not the tool. --- ## 5. Identity matching The actual problem. `pmarchetti@fernhill.com` is one address at a company with thousands of employees, and picking the wrong Marchetti writes a real person's career history onto the wrong CRM record. This gets a **skill** — an on-demand procedure the agent loads when it needs it — rather than being buried in the system prompt: `agent/skills/identity-matching.md` sets out the evidence hierarchy: 1. **Exact email on the profile** — decisive. LinkDAPI exposes contact info on some profiles; when the address matches, the match is certain. 2. **Local-part decomposition against a real name.** `pmarchetti` is consistent with `a` + `marchetti` and with `abi` + `gham`. Checked *against candidates*, never used to generate a name — the direction matters, and it is the whole difference between this and guessing. 3. **Company match**, current employer equals the contact's company. 4. **Corroboration** — the meeting we already synced, the thread they replied on, the title they signed off with in an email footer. And the confidence bands the write path keys on: | Confidence | Evidence | Behaviour | | --- | --- | --- | | `high` | Email match, or unique local-part decomposition at the right company | Merged automatically. | | `medium` | Consistent decomposition but more than one plausible candidate | Written as a **suggestion**; a rep accepts or rejects. | | `low` | Company match only | Discarded. Not stored, not shown. | When LinkDAPI returns nothing, the agent falls back to the existing Context.dev search path, which is already sourced and already refuses to guess. **Evals live and die on this.** `evals/` gets a fixture set of real addresses with known answers — including the ones that *should* resolve to nothing — and the metric is precision first: a wrong merge costs more than a missing one. --- ## 6. Writing back, and provenance One tool, `update_contact`, and it never writes a bare field: ```prisma model ContactEnrichment { contactId String @id contact Contact @relation(fields: [contactId], references: [id], onDelete: Cascade) source String // "linkdapi" | "context.dev" sourceUrl String? // the profile the claim came from confidence String // high | medium | low raw Json // the payload, so re-deriving costs nothing fetchedAt DateTime @default(now()) @@map("contactEnrichment") } ``` Mirrors `CompanyEnrichment` deliberately: keeping the raw payload has already paid for itself once — `iconTone` and `iconDarkUrl` were backfilled for every company from stored payloads without spending a credit. `Contact` gains `enrichmentStatus`, `enrichedAt` and `enrichmentError` matching `Company`, so the UI reuses `isEnriching()` / the existing poll rather than inventing a second freshness mechanism (`api.md`: background writes the browser cannot see are polled, not invalidated). **Approval.** eve's `approval: always()` pauses a tool call durably until a person signs off. `update_contact` takes a policy rather than a flag: automatic at `high`, approval-gated at `medium`. eve's own responsible-use guidance is that agents run permissively unless you configure otherwise and that model behaviour alone should not be relied on to prevent sensitive writes — a confidence rubric in a prompt is model behaviour; the approval policy is not. **Never overwrite a human.** Same rule the rest of the codebase already follows: `brandToUpdate` fills only empty fields, and `isDerivedName()` is already the test for "this name is a placeholder nobody has improved on". --- ## 7. Scheduling and coverage `agent/schedules/enrich.ts` — a `defineSchedule` with a `run` handler rather than a markdown prompt, because the batch has to be selected in code: ```ts export default defineSchedule({ cron: "*/10 * * * *", async run({ receive }) { /* claim a bounded batch, hand each to the agent */ }, }); ``` Sizing is set by the rate limit, not the budget: - **Batch size** is the per-minute cap times the tick, with headroom. At Developer (70/min) and a 10-minute tick that is hundreds of contacts, which is more than the whole database. - **Every contact with a work domain**, not just the ones with ugly names and not just the ones on open deals. A correct title on a contact nobody is actively selling to is what makes the *next* deal's brief good. - **Re-check on a cadence, not once.** A profile read once is a fact that decays; a profile read every 30 days is a job-change feed (Phase 4). This is the single biggest behavioural difference the "billing is irrelevant" call buys, and it is worth taking deliberately rather than by omission. Pick the tier from the measured request rate, then stop thinking about it. --- ## 8. What this does *not* touch - **Company firmographics.** Context.dev owns those columns. LinkDAPI's company endpoints are for *people* context — headcount, who works there — and land in new fields or not at all. - **The Google sync's matching.** `GoogleMatchService` decides which company a thread belongs to. That is a different question from who a person is, and merging them would couple a cheap local decision to a paid network call. - **Outbound anything.** --- ## 9. The part to decide before building LinkDAPI is an **unofficial** LinkedIn API. Using it is a business decision, not a technical one, and it belongs to you rather than to this plan — but it should be made deliberately rather than discovered: - **LinkedIn's terms prohibit automated collection**, and they enforce against it. The exposure is to the data provider and to LinkedIn, not to a court's view of the CFAA, which `hiQ v. LinkedIn` largely settled in favour of public data. Practically: this is what Apollo, Clearbit, ZoomInfo and most of the sales stack do, and it is priced accordingly. - **This is personal data under UK/EU GDPR** even though it is professional and public. What that actually requires of an internal B2B CRM: a lawful basis (legitimate interest is the usual one, and wants a balancing test on file), a retention period, and the ability to answer an access or erasure request. `ContactEnrichment` with `sourceUrl` and `fetchedAt` is most of the answer — it makes "where did this come from and when" a query rather than an investigation. - **Keep it to business context.** Name, title, employer, tenure, public profile. Nothing about a person outside their work, and none of the special categories, regardless of what an endpoint returns. - **Single-provider risk.** An unofficial API breaks when LinkedIn changes. `lib/linkdapi.ts` being the only file that knows the vendor is what makes swapping it a day rather than a rewrite. None of this blocks the build. It does mean §6's provenance is not a nicety — it is the thing that makes the rest defensible. --- ## 10. Phases | Phase | Scope | Done when | | --- | --- | --- | | **0 — Spike** | ~~Prove the API resolves real CRM addresses~~ | **Done — §0.** Search is unusable; profile and company are excellent; the hybrid resolver works end to end on a real contact; hit rate is partial. | | **1 — Skeleton** | `apps/agent`, `eve link`, `update_contact` writing to `@crm/db`, schedule claiming a batch, no LinkedIn | A contact's title changes because the agent changed it, on a cron, in a deployed environment. The loop is proven before the vendor is in it. | | **2 — Enrichment** | Five tools, `identity-matching` skill, confidence bands, `ContactEnrichment` | `Tokonkwo` becomes Tomi Okonkwo, VP Fund Finance at Northwind Bank, with a `sourceUrl` against it — and an ambiguous address becomes a suggestion rather than a wrong answer. | | **3 — Trust** | Approval on `medium`, evals with a labelled fixture set, provenance in the contact sheet | Precision measured, not asserted. A rep can see where a field came from. | | **4 — Meeting prep briefs** | On a calendar event with external attendees, research everyone and write a brief to the timeline before the call | Opening a company the morning of a call shows who is attending, what they do, how long they have been there, what the company just announced, and where the deal stands. **This is the feature people will describe to other people.** It is only possible because the Calendar sync already knows the meeting is coming. | | **5 — Job changes** | Re-read on a 30-day cadence, diff against the stored payload, activity + task on a move | A champion changing employer creates an activity on their old company's timeline and a task for their owner. The highest-intent signal in B2B, and nobody else in the stack sees it. | Phase 0 already earned its keep: it deleted the original resolver design, found a shipped bug, and replaced a marketing-page assumption with a measured one. The ordering after that is deliberate. Phases 1–3 make the data trustworthy; 4 and 5 are what a rep would actually miss if you took it away. Resist doing 4 first — a brief built on unverified identities is confidently wrong in front of a customer, which is the one failure mode that costs a deal rather than a click. --- ## 11. Open questions 1. **Whose credits?** One shared app key (`principalType: "app"`, which is eve's default when `getToken` is the only auth), or per-rep? Shared is simpler and almost certainly right for an internal tool. 2. **Does a `medium` suggestion surface in the UI, or a queue?** The Gmail plan argued against approval queues because they rot. A suggestion inline on the contact — "LinkedIn thinks this is Paula Marchetti · accept" — probably beats a separate inbox, but it is more UI work. 3. **Retention.** How long does `ContactEnrichment.raw` live? It is the thing that makes re-derivation free and the thing an erasure request is about. 4. **Do we enrich every contact, or only ones on open deals?** Every contact is simpler; deal-scoped is far cheaper and probably where the value is. 5. **Is `find_colleagues` allowed to create contacts?** Listing a company's employees is a lead-generation capability, not an enrichment one. It should not create records in v1, and that line is worth holding deliberately. --- --- METRICS --- - Files Extracted: 30 - Estimated Token Budget: ~98380 tokens - Canonical Reference: https://codewiki.google/github.com/trycompai/crm