# Technical Documentation: yc-software/qm > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/yc-software/qm) ### Overview (`README.md`)

QM

**A multiplayer agent harness for work. In Slack and on the web.**
Run it in your own cloud, with your own models and keys. [](https://github.com/yc-software/qm/actions/workflows/cicd.yml) [](./LICENSE) [](./package.json) [](https://www.npmjs.com/package/@yc-software/qm) [Deploy](#deploy-it-for-your-org) · [How it works](#how-it-works) · [Security model](#security-and-secrets) · [Customize](#customize-your-instance) · [Contribute](#contributing) · [Documentation](#going-deeper)
## Overview Most agents are designed like personal assistants. You can make one work for a whole company, but it quickly gets complex. QM is designed for startups. Employees each get their own isolated workspace, and can also collaborate with the agent in channels, group messages, and projects. Each person and each room has its own scoped memory, files, keychain view, permissions, crons, web apps, and durable sandbox. It's built with open source in mind. Pick your own harness and model and switch between them. Pi and Claude Code both drive the same core, and Pi runs models from many providers, so a deployment isn't tied to any single vendor. ## Setup Tell your coding agent of choice `Let's deploy https://github.com/yc-software/qm`. From here, it should follow the deployment guide in this repo. You can also try out a 3rd-party hosted version of QM [here](https://www.agent37.com/qm). If you're an infra provider interested in offering a hosted version of QM, feel free to reach out. ## Features - **Personal and shared scopes.** People customize the agent to be _theirs_, and still work with it collaboratively in Slack channels and projects. - **Slack and web.** The same identity and configuration carries between Slack and the web app. - **Admin control.** Set org-level configuration, security and sharing postures. Choose which harnesses and models are available. - **Web apps.** Spin up custom internal apps and publish them to the right people. - **Shared skills.** Skills are scope-owned and shareable by grant, with admin-gated promotion to the whole org and skill packs imported from git repositories. - **Background work.** Crons, watches, and inbound webhooks work while you're away. ## What you can do with it - Search internal notes, email, documents, databases, and the web together - Build internal apps, publish them to the right people, and keep their data current - Learn your writing voice from past sends, then triage your inbox on a schedule — labels and reply drafts included - Work in an existing repository: run tests, open PRs, monitor CI, check system logs - Track a project in a shared channel and post updates and follow-ups ## Architecture ```mermaid flowchart LR DB[("Postgres
sessions · memory · queue")] subgraph CORE["Headless core"] API["API · identity · policy · scheduler"] LOOP["Agent loop
(Pi, Claude Code)"] API <--> LOOP end SBX["Per-scope sandbox
files · tools · logged-in services"] DB <--> API LOOP <--> SBX ``` For durability, set `DATABASE_URL` and `SESSION_STORE=postgres` — without it, sessions live in process memory and vanish on restart. To exercise a branch against a real model and real Postgres, run `npm run dev-instance:web` for web/admin or `npm run dev-instance:slack` for Slack. Use `npm run dev-instance:both` when testing both surfaces together. Bare `npm run dev-instance` defaults to web for new instances and preserves the surface on reload. Switch an existing instance with an explicit surface command. ## How it works Every turn runs through a central core, which can use a variety of models and harnesses to generate the response. A Postgres persistence layer holds user data, session history, and other durable state. The agent has a small, fixed tool surface; one of those tools is `execute`, which runs commands in the scope's own isolated sandbox — its durable computer, where installed tools stay installed. The web UI and admin panel share one service; the portal and optional built-in auth broker share another. These modules communicate with core over its HTTP API. See [combined services](docs/combined-services.md) for configuration and migration; Slack is an optional in-process plugin that core starts and supervises through a direct service client. The core runs TypeScript directly on Node and uses Fastify for HTTP. The Slack plugin uses Bolt; the web UI builds with Vite and renders with Lit. The core itself is generic. Everything specific to one company — org config, custom tools and skills, sandbox image, infrastructure — lives in a **deployment directory** that the [`qm` CLI](./cli/README.md) validates and deploys. Every substrate (harness, session store, sandbox, memory) sits behind an interface. Memory can also be routed by scope to [external providers](./docs/memory-providers.md) while retaining the built-in notebook. ## Security and secrets QM's approach follows local coding agents like OpenCode, Codex, and Claude Code: the agent acts as the person it's working for, with their credentials and permissions, and everything it does is audited. An org picks one security posture, which narrower scopes can only tighten: - **Strict** — every harness tool call pauses for human approval, except the two no-effect turn enders. - **Auto** (default) — blocks private-network access. - **Dangerous** — no tool approval gates, and content screening only observes. Content screening is a deployment setting, `securityScreen.mode`: `off` (default), `observe`, or `enforce`, with `securityScreen.classifier` choosing the built-in `model` or an external `proxy`. When on, it classifies external content under every posture. `observe` only records verdicts in the audit log. `enforce` quarantines flagged content pending release approval, except under Dangerous, which caps screening at `observe`. The predeclared command policy — approval rules and hard denials for things like recursive deletes or destructive SQL — applies in every posture, Dangerous included. Sharing posture is independent: - **Isolated** (default) — resources stay in their scope unless explicitly shared. - **Open** — on a live authenticated internal human turn, the speaker's opted-in personal files, artifacts, skills, and memory may be read in an opted-in shared room. [`SECURITY.md`](./SECURITY.md) has the threat model, the operator assumptions, and the known limitations. ## Deploy it for your org Create an organization-owned deployment repository that depends on `@yc-software/qm`: ```bash npm exec --yes --package=@yc-software/qm@latest -- \ qm init . --org --target npm install ``` Initialization materializes a deployment skill for an agent and walks through infrastructure, web sign-in, connector credentials, optional Slack access, deployment, and live verification — no source checkout required. Each deployment runs in the operator's own cloud account; initialization does not generate or enable deployment CI, and this repository has no production deployment workflow. See [`deployment.md`](./deployment.md) for the details. ## Contributing We take contributions as _human-written_ text, not code — see [`CONTRIBUTING.md`](./CONTRIBUTING.md). Describe the change you'd like informally in a `.txt` or `.md` file in [`adrs/`](./adrs/), and if we're aligned we'll handle the implementation. Report vulnerabilities privately — see [`SECURITY.md`](./SECURITY.md), not a public issue. ## Customize your instance Choose how you want to customize QM: - **Config, tools, skills, and services:** use the deployment repository above. It pins `@yc-software/qm` and uses that release's runtime images; no source copy is needed. - **Changes to QM itself:** keep your own source fork, public or private. You may modify any part of core, including the runtime, plugins, CLI, docs, and CI. Contributing those changes upstream is optional. ### Create a source fork For a private source fork, create a standalone private repository, outside GitHub's fork network. Seed only `main` and explicitly set it as the default branch: ```bash gh repo create /qm-private --private git clone --single-branch --branch main --no-tags git@github.com:yc-software/qm qm-private git -C qm-private remote rename origin upstream git -C qm-private remote add origin git@github.com:/qm-private git -C qm-private push -u origin main gh repo edit /qm-private --default-branch main ``` Do not seed with `git push --mirror`: it copies unrelated upstream branches and tags, leaves default-branch selection implicit, and can delete destination-only refs on later pushes. The `upstream` remote supplies source updates without copying those refs to your repository. For a public source fork, GitHub's Fork button is also an option. A GitHub fork of a public repository cannot be private; keep private work outside that network. Review inherited workflows before enabling Actions or adding credentials. Choose the CI checks you want, and disable or adapt upstream release and publishing workflows for your own package and image registries. Copying the source does not configure production deployment CI. ### Customize and run your source Keep deployment configuration, tools, skills, plugin images, and infrastructure in `deploy/layers//` in a private source fork, or in a separate private deployment repository when your source is public. Never commit secrets. See [`deploy/layers/README.md`](./deploy/layers/README.md) for initialization and layout. Keep deployment data separate from core code, but change core wherever your desired behavior requires it. From the source checkout, install dependencies with `npm ci` and use the in-tree CLI. After completing the provider setup in [`deployment.md`](./deployment.md), build and deploy your modified services explicitly: ```bash node cli/bin/qm.ts check --config /qm.config.jsonc node cli/bin/qm.ts plan --config /qm.config.jsonc --build-from . node cli/bin/qm.ts up --config /qm.config.jsonc --build-from . node cli/bin/qm.ts check --config /qm.config.jsonc --live ``` Use this checkout's CLI when changing the CLI itself. Without `--build-from`, the normal deployment path selects published images, so editing source alone does not change the deployed runtime. If you publish custom images instead, configure their immutable references through `imageOverrides`. Follow the provider guide for sandbox image builds; service builds do not replace that step. ### Keep it current For a source fork, `update-qm` merges upstream changes while preserving intentional local behavior. Land sync PRs with their merge ancestry intact, never squash or rebase them. Conflicts are expected maintenance work, not a requirement to discard customizations. Use `upstream-pr` only when you want to contribute a generic change; it prepares a clean upstream branch without private deployment data or history. For a package deployment, upgrade the exact `@yc-software/qm` dependency and lockfile, review contract changes and generated assets, then validate and deploy. There is no upstream source history to merge. ## Going deeper - [`docs/getting-started.md`](./docs/getting-started.md) — first run, end to end - [`cli/README.md`](./cli/README.md) — the `qm` CLI and the deployment directory contract - [`docs/deploy-directory.md`](./docs/deploy-directory.md) — the deployment directory in full - [`docs/principal-links.md`](./docs/principal-links.md) — one person, several sign-ins: linking principals - [`docs/porter.md`](./docs/porter.md) — running qm on Porter - [`docs/superserve.md`](./docs/superserve.md) — using Superserve for agent sandboxes - [`.env.example`](./.env.example) — every knob, documented in place - [`docs/swarms.md`](./docs/swarms.md) — durable agent pools, scoped messages, and blank Modal workers - [`docs/model-gateway.md`](./docs/model-gateway.md) — discover and route models through a gateway - [`plugins/`](./plugins) — the surfaces (Slack, web UI, admin, portal) ## License Except where otherwise noted, QM is available under the [MIT License](./LICENSE). --- ### Overview (`cli/README.md`) # `qm` The standalone deployment CLI for QM. The normative directory schema, security guarantees, target behavior, and lifecycle are in [`docs/deploy-directory.md`](../docs/deploy-directory.md). `qm init` materializes the agent-consumable package runbook into the deployment repository. ```bash npm exec --yes --package=@yc-software/qm@latest -- \ qm init . --org acme --target aws npm install npm exec qm -- check npm exec qm -- infra render npm exec qm -- doctor npm exec qm -- infra build-image npm exec qm -- plan npm exec qm -- up --yes npm exec qm -- check --live ``` This package is published to npm as `@yc-software/qm`, with npm provenance attesting the building workflow. A release is one dispatch of `.github/workflows/release.yml` from `main`: it signs and pushes the first-party images, publishes the package pinning their digests, and then tags `v` and creates the GitHub release with the resolved digests attached. Each release picks its own version: a patch bump past the latest released version, or `cli/package.json`'s version when a PR raised it higher (for a minor or major bump); a tag that already exists stops the release rather than moving. The checked-in image manifest is a sentinel that a deployment overrides with real digests. The packed-artifact test exercises the consumer path locally. The CLI deploys long-running QM services; it is not the runtime. Docker runs them locally, Fly runs them as Fly apps with Fly Machines for agent computers, and AWS runs digest-pinned tasks on ECS Fargate (amd64 by default for first-party services) with Lambda MicroVM agent computers. When upgrading an AWS source deployment that previously relied on the ARM64 default, set each existing workload's `architecture: "arm64"` explicitly until rebuilding its image for another platform. Secret rotation refuses a task-platform change before uploading secrets. New release candidates record their workload architectures; legacy source-built first-party candidates without that record require an explicit architecture matching the image, or regeneration with the current CLI. Candidate deployment and migration reject platform mismatches before accessing AWS. ## Deployment directory ```text qm.config.jsonc package.json package-lock.json deployment.md .codex/skills/deploy-qm/ .env.example .env slack-app-manifest.yml slack-sso-manifest.yml sandbox/ tools//tool.json tools// skills//SKILL.md Dockerfile plugins//Dockerfile infra/ ``` `qm.config.jsonc` is committed and contains no secret values. `.env` is ignored. `package.json` pins the CLI package at the exact version that scaffolded the directory — `contract: 1` is only the compatibility floor — so every checkout resolves the same interpreter; upgrade the pin deliberately. `cd` into it and the DEPLOY commands act on it; `--config` / `--env-file` / `--sandbox-dir` relocate a piece (e.g. several deployments sharing one `sandbox/`). `check` validates the config, computed secret names, tools, skills, and plugins without network access; `up`, `plan`, and `sandbox build` run the same checks first. `doctor` verifies external prerequisites read-only. `plan` renders the deployment; AWS mutation requires `up --yes`. For a single-host Docker deployment, `sandbox.backend: "local"` runs each agent computer in its own container. `qm up` builds the local runtime from the CLI's pinned sandbox base, mounts the host Docker socket into trusted core, and connects core to each sandbox's private network. An explicit `sandbox.image` uses that runnable local image instead. On AWS, `up` verifies under the deploy lease that RDS point-in-time recovery is current (its `LatestRestorableTime` must lag by at most `QM_AWS_DB_MAX_RESTORE_LAG_MS`, default 10 minutes) and records the pre-deploy timestamp in the deployment manifest it precedes. `rollback` restores code and configuration only, so it prints that timestamp as the matching data restore point (`aws rds restore-db-instance-to-point-in-time`); `aws.predeployDbSnapshot: false` opts out. On AWS, `up` waits for every rolling service to reach `COMPLETED` with a single deployment, polling until `QM_AWS_ROLLOUT_DEADLINE_MS` (default 20 minutes); a timeout rolls the changed services back. ECS stops the old tasks once they exit or their `stopTimeout` elapses (core drains for `SHUTDOWN_DRAIN_MS`, 10 s by default), and a turn cut off there resumes on the new tasks. Core images older than this release protect their task while a turn runs, so the first `qm up` that replaces them cannot finish before their longest in-flight turn ends (protection expires after 60 minutes at most); run that one deploy with a longer deadline or when no long turn is in flight. The candidate migration task and the private live-session canary also wait up to `QM_AWS_ROLLOUT_DEADLINE_MS` (default 30 minutes for them); a task still running then is left to finish, and a migration that outlasts it fails `up` before any service changes. AWS deployments can opt into durable background ownership with `aws.backgroundWorkControl: true`. The CLI allocates a unique `BACKGROUND_DEPLOYMENT_ID` for each replacement core deployment and records it in the deployment manifest. A no-op deployment and automatic ECS task replacement keep that identity; an explicit core restart allocates a new one. Pending preparation is persisted before ECS changes, and an ambiguous previous deployment must be reconciled before another identity can be allocated. Control requests require both `CORE_SIGNING_SECRET` and a distinct `DEPLOYMENT_CONTROL_SECRET` of at least 32 characters. The control secret is restricted to core. One shared record names the deployment that owns background work; a core process with a configured identity admits work only while that record names its deployment, and `BACKGROUND_WORK_ENABLED` is ignored there. The exported `awsBackgroundWorkBootState` reads the exact manifest core task to return its recorded boot flag and optional deployment identity. It returns `undefined` only without a recorded core task, and rejects missing or ambiguous boot flags or identities. The exported `awsBackgroundWorkStatus(config, configDir, candidatePath?)` proves the stack's exact healthy core task set and reads the owner record through the stack's own API, returning the responder's `ownerDeploymentId`, `setAt`, `setBy`, and whether the responding process is currently active. The exported `awsSetBackgroundWork(config, configDir, enabled, candidatePath?, expected?)` sets the owner record to the stack's identity (or clears it when `enabled` is false) without restarting ECS tasks, then polls until every core task of the stack has reported matching activity. Passing `expected: { ownerDeploymentId }` refuses the change if the owner differs from what the caller last observed. A demotion refuses to clear a different deployment's ownership. The exported `awsBackgroundWorkCapacity(config, configDir, candidatePath?)` proves that an inactive controlled stack is currently reusable. It requires another owner, stable native deployments and exact task inventories for every workload, and resolved deployment preparation. It never changes deployment state. The result binds the manifest and deployment identities, the owner, workload task definitions, native deployment IDs, and task ARNs. A release coordinator can combine both snapshots with immutable candidate provenance and compare them again under its production lock before any mutation. This is a point-in-time check, not a reservation: intervening maintenance or task replacement invalidates the proof and must block promotion. Live checks use the owning deployment's authenticated canary endpoint to verify a real model reply, session persistence, generated title, error records, session cleanup, and database catalog health. The CLI confirms the stack owns background work and is active before and after the check, and requires a final success bound to the request and responding process; heartbeats alone do not count. An uncertain result never triggers an automatic replay or fallback. Legacy deployments retain the Fargate canary, and a controlled stack that does not own background work uses that same path. Replacing or rolling back a controlled stack's core tasks requires that the stack is not the current owner: hand ownership to the other stack first. Processes that lose ownership stop claiming within their poll interval (at most their ten-second validity window) and finish admitted work under their leases; ECS task replacement then drains them through `SIGTERM`. `qm down` is not guarded: if the record still names a stack that has no tasks, set the live stack active to recover. Core secret uploads defer activation to a subsequent staged `up --restart core`. The generic upload path refuses changes to either control credential while a controlled deployment is recorded, because replacing credentials before coordinating all running processes would break ownership control. Legacy AWS deployments keep the task replacement path when ownership control is not configured. Batch operators can set `QM_DEPLOY_PROGRESS_FILE` to a new absolute file path and `QM_DEPLOY_PROGRESS_TOKEN` to a unique attempt identifier for candidate `up --yes`. After all forward service updates have been submitted, the CLI atomically creates a private JSON receipt with `phase: "monitoring"`, `token`, `orgId`, and `targets` (the selected workload-to-task-definition mapping). It then continues health checks, manifest recording, and rollback under the deployment lease. This receipt only permits the batch runner to release a submission slot; the CLI exit status still determines success. Use a fresh path and token for every attempt. Missing receipts must keep the submission slot occupied until the CLI exits. `sandbox build` is a local validation build of the sandbox layer image. At runtime sandboxes boot their platform's stock image; tools and skills arrive through the deployment-layer sync, which every ordinary `up` performs. Content screening is off by default. Set `securityScreen: { "mode": "observe" }` to record classifier verdicts without acting on them, or `"enforce"` to quarantine flagged content. The classifier is the built-in model unless `"classifier": "proxy"` names an external screener with a provider label and HTTPS endpoint; route its token through `secretEnv.core.SECURITY_SCREEN_PROXY_TOKEN`. ## Commands ```text init [dir] [--org id] [--target docker|fly|aws] check [--json] [--live] doctor infra render|build-image|delete-image|delete-task-definitions conformance [dir] [--static] plan up [--yes] [--build-from[=repo]] [--image-label label] slack render outputs [--json] admin-login [--email admin@example.com] proof scope-key secrets push [--from file] status logs [service] [-f] [--tail n] down [--purge] rollback [--to revision-or-sha] sandbox build [--from image] [--tag tag] [--dry-run] ``` ## Administrator login without email After deployment, run `qm admin-login` to print a single-use login URL valid for five minutes. Open it and confirm the displayed administrator email. The command uses the deployment's existing `PORTAL_SESSION_SECRET` and the email in `ADMIN_GRANTS`; if several admins are configured, select one with `--email`. QM checks that the selected account still has `org_admin` access when the link is redeemed. The command creates no account or role grant. Run it from the deployment directory with its `.env`, or use `--config` and `--env-file`. Inside a running deployment without a config file, supply `PORTAL_PUBLIC_URL`, `PORTAL_SESSION_SECRET`, and `ADMIN_GRANTS` through its environment. The CLI prints only the URL, which is a temporary login credential; do not publish it or put it in shared logs. `qm setup` offers email setup separately. Skip it to use administrator login without Resend or SMTP. For ordinary users before email is ready, the broker can accept passwords: hash one with `node plugins/auth/src/hash-password.ts user@example.com` and store the output with `qm secrets set AUTH_PASSWORD_USERS`. This is meant for onboarding; switch to email links or an identity provider afterwards. To enable ordinary email login later, rerun `qm setup` and configure the selected transport's complete credential set and sender, then push secrets and redeploy. Missing email credentials disable email sign-in; QM and `qm admin-login` remain available, even if a sender is still configured. Deployment commands accept `--config`, `--env-file`, and `--sandbox-dir`; `admin-login` uses only `--config` and `--env-file`. `dev` remains the contributor worktree loop and is separate from the portable deployment contract. ## Package contract The `@yc-software/qm/contract` export is the supported programmatic surface for conformance tests. It exposes the contract version, parsing/rendering functions, and provider ids without registering arbitrary runtime plugins. Incompatible directory changes increment the contract major; optional fields may be added within a major. Its one runtime dependency is Nodemailer, which `qm check` uses to verify SMTP credentials. It shells out to Docker with Buildx, Flyctl, the AWS CLI, and Git. Terraform is operator-run against the module generated by `init`. --- ### Overview (`deploy/README.md`) # Deployment templates The files under `deploy/` are service-image and Fly configuration templates used by the `qm` CLI. They are not a production deployment and contain no cloud account, workspace, or organization credentials. The one exception is [`layers/`](./layers/), which is empty in qm itself. A private fork of this repository keeps its organization's deployment there, and that material never travels back upstream. For a new hosted instance, follow the repository-root [`deployment.md`](../deployment.md) workflow. It: - asks whether to deploy into the operator's Fly.io organization or AWS account; - creates the organization's deployment directory under [`layers/`](./layers/) with its `qm.config.jsonc`; - configures verified-email web onboarding before optional Slack; - returns the exact web, admin connector, user connection, and Slack app URLs; - deploys package-selected immutable images into the operator's provider account; and - requires live health, browser chat, connector visibility, and optional Slack verification before completion. The supported command sequence is generated by `qm init` and the selected provider reference under [`.codex/skills/deploy-qm/references/`](../.codex/skills/deploy-qm/references/). Do not deploy the example names in these templates directly. ## Topology The public portal is the only Internet-facing service. It authenticates users over OIDC and proxies the private Web UI and admin surfaces. Core, Postgres, and agent computers stay private. The optional Slack surface runs inside core over outbound Socket Mode. By default the OIDC provider is the `auth` service qm deploys for itself: a sign-in broker that emails a one-time link. It is private like every other non-portal service — the portal republishes only its two browser-facing routes under `/idp/`, so the issuer and the sign-in pages live on the portal's own origin while the token, userinfo, and JWKS calls stay on the private network. The self-hoster supplies an admin address, a verified sender, and a Resend key or SMTP credentials; the CLI generates the broker's signing key and the portal's client credentials and wires both sides. Core receives the same `RESEND_API_KEY` and `AUTH_EMAIL_FROM`, which let admins email invitations to external users from the admin Users tab or by chatting with QM; both are optional on core, and without them the invitation is still created and the sign-in link is shared by hand. Dropping `"auth"` from `services` hands sign-in back to an external identity provider, which then has to register the exact `/auth/callback` redirect. Connector OAuth clients and the optional Slack bot token pair are entered at the authenticated admin connector URL returned by `qm outputs`. Secrets are encrypted in durable storage and are never committed to a deployment directory. The agent advertises only connectors whose admin configuration is enabled. `cli/templates/fly/auth.toml` declares an `[http_service]` like every other surface, but the CLI always deploys it `--flycast --no-public-ips`; on AWS the ALB assertions refuse to attach it to a load balancer. Never `fly launch` that file by hand — a broker with a public IP exposes its token endpoint and lets a caller forge the rate-limit client address. ## Fixtures [`stacks/`](./stacks/) contains account-neutral contract fixtures. The checked-in `cli/templates/fly/.toml` files are templates from which the CLI derives deployment-specific copies under the ignored `deploy/stacks/.generated/` directory. --- ### Overview (`desktop/README.md`) # QM desktop An Electron client for an existing QM web deployment. The welcome screen connects to your workspace; the app then runs the same web UI with its own persistent session. ```sh cd desktop npm ci npm start ``` Enter your QM web URL on first launch. HTTPS is required except for localhost development. Change instances with **QM → Change Instance…** (`Cmd+,` on macOS). You can also set `QM_DESKTOP_URL` when starting the app. ```sh QM_DESKTOP_URL=http://localhost:3000 npm start npm run package ``` Packaging writes a native app for the current platform into `desktop/dist/`, registering the `qm-desktop` URL scheme. Use the packaged app for browser sign-in. The macOS build is an unsigned local prototype, without automatic updates in development. It connects to a running server; it does not bundle the QM backend or provide offline agent execution. ## Browser sign-in When sign-in is needed, the app opens `/auth/desktop` in your default browser. Sign in there as usual, confirm your account, and choose **Open QM Desktop**. The browser returns a two-minute, single-use code bound to the app's proof key and random state. The app redeems it over the instance connection and stores the session in its isolated persistent cookie partition. Closing the app or starting another connection cancels unfinished sign-in requests; it does not sign out sessions whose cookie exchange already completed. The portal must include the desktop sign-in routes; older deployments need an update. The portal preserves the browser session's identity, original authentication time, and expiration. Redemption uses the existing durable core claim store to prevent replay across portal instances and deployments. The desktop proof key exists only for the disposable, in-progress sign-in attempt and is never sent to the browser. Passwords and identity-provider cookies stay in your browser. On macOS, the window controls sit inside the sidebar header. With a single session open, drag its title bar to move the window; tab dragging remains available in multi-pane layouts. Hold Command to reveal shortcuts for the first nine sessions in expanded sidebar groups, then press 1–9 to switch. Windows and Linux use Control. Collapsed groups and a hidden sidebar are excluded. Remote content is sandboxed with no Node.js access. An isolated desktop preload adds window styling and session shortcuts with one sender-validated operation for opening a same-instance setup page in the browser. Off-origin links open in your browser. Microphone, camera, and notification permissions are disabled in this prototype. The welcome screen uses _Becalmed off Halfway Rock_ (Fitz Henry Lane, 1860), sourced from [Ève Bouffard’s QM brand board](https://www.evebouffard.com/qm-brand). [National Gallery of Art collection record](https://www.nga.gov/artworks/76213-becalmed-halfway-rock). ## Connection setup and previews Slack installation and personal app authorization start in your browser before QM creates the connection attempt. Finish setup there, then return to the desktop app; it refreshes connection status on focus. Settings contains both personal Slack linking and the app picker. This keeps provider callbacks, installation POSTs, cookies, and tab-local authorization state in one browser. Same-instance links opened in a new tab use a separate sandboxed window sharing the instance session. External links open in the browser. View → Back and Forward navigate the focused window. Closing the main window closes its previews. ## Computer use On macOS the app can see and drive running Mac apps through [`desk`](desk/README.md), a Rust helper bundled at `QM.app/Contents/Resources/desk`. `npm run package` builds it with Cargo first, so install Rust (`rustup`) on the packaging Mac. Because QM launches it, grant **QM** Accessibility (and Screen Recording for screenshots) in System Settings > Privacy & Security, then relaunch QM. In development, `npm start` uses `desk/target/release/desk`; override with `QM_DESK_BIN`. The workspace preload exposes `window.qmDesktop.computer.call`. The `desktop` tool accepts JavaScript with top-level await and preserves bindings within the initiating turn. A hidden, sandboxed Electron renderer evaluates code without Node.js or network access and forwards computer API requests to `desk serve` one at a time (see `desk/PROTOCOL.md`). Each request uses the originating turn's owner and absolute call deadline. The same sender check as browser handoff applies. All installed apps are available subject to macOS permissions. The first call returns API documentation. Start with `var app = await computer.getApp("Calculator")`, then use `app.getAXState()`, `app.click(id)`, or `app.getScreenshot()` in later calls. Accessibility observations default to changes since the previous state; `{disableDiffing: true}` returns the full tree. Await every operation; background callbacks persist within the turn and should not schedule computer actions. Reset, cancellation, and timeout destroy JavaScript state. Results contain at most four resized JPEG screenshots within an 800 KB base64 budget. Before each click, scroll or drag, a transparent click-through overlay flies an agent cursor to the target point and desk waits for it to land. Set `QM_DESK_CURSOR=0` to skip the overlay. ## Mac release `npm run package` builds the unsigned local app. For a signed and notarized build, install a Developer ID Application certificate in the Mac keychain and sign in to that team in Xcode Settings → Accounts. Then run: ```sh QM_MAC_SIGN_IDENTITY="Developer ID Application: Your Organization (TEAMID)" npm run package:release ``` The release command signs the app, submits an archive using the signed-in Xcode account, waits for notarization, and verifies the stapled ticket and Gatekeeper assessment. It produces the notarized ZIP, SHA256SUMS.txt, and RELEASES.json in desktop/dist. Credentials and certificates stay outside this repository. Builds target the current machine's architecture. Packaged Apple Silicon builds check the desktop-only feed on startup and every five minutes. Updates download in the background and restart QM automatically without a confirmation dialog. Restart waits while a composer has a pending submission, file processing, staged attachments, or an open paste editor. Older servers without the update-readiness handshake defer automatic restarts until their web UI is upgraded. Open non-web previews, such as PDFs, also defer restart until closed. Interaction pauses briefly while the app saves restart state. Before restarting, QM flushes saved text drafts and pane layout, saves the current instance page, and flushes its persistent browser session. Server-side agent work continues during restart. QM → Check for Updates triggers an immediate check. Development, Intel, Windows, and Linux builds do not use this Mac feed. Versions 0.1.0 and 0.1.1 require one manual upgrade to receive the updater. Publish the verified ZIP and checksum as desktop-v, then upload RELEASES.json to the dedicated desktop-updates GitHub release using gh release upload desktop-updates desktop/dist/RELEASES.json --clobber --repo yc-software/qm. Update the feed only after the versioned download is published and verified. The feed uses Squirrel.Mac's static JSON format and points to the immutable versioned ZIP; backend and dependency releases never enter it. `npm run test:electron` exercises actual Electron preview windows, POST popups, session sharing, sandboxing, and browser handoff against a temporary local server. New chats started in desktop have a stable `desktop` tool. Send a request such as “Use Calculator to compute 17 × 23.” Only human-initiated turns submitted from the desktop app can execute it. Automatic openers, background automation, delegated turns, and retries cannot use computer access. The app must stay open and connected for that turn; switching chats is fine. Continuing the chat in a browser retains the tool definition but returns an unavailable result if it is called. Existing browser-created chats do not gain computer access: start a new desktop chat. Chats from the earlier computer-use trial retain their old tool schema and must also be replaced with a new chat. Computer requests are bound to the submitting desktop turn. Only one turn can control the computer at a time; other turns receive a busy result until it finishes. A closed window, disconnected stream, stopped run, or helper deadline cancels native execution; reconnecting does not replay actions. After an interrupted action, observe the app before retrying because it may have partially completed. Screenshots return as image content, without base64 duplicated in model-visible text. ### Guided computer access On macOS, missing computer permissions open a local setup window before the workspace. It checks Accessibility and Screen Recording automatically, opens their System Settings panes, and provides the installed app as a draggable icon with a Finder fallback. Choose **Enable computer access** to request the native prompts, then enable this app in both settings lists. **Continue to workspace** becomes available when both checks pass and restarts the native helper to pick up the grants. **Not now** skips setup; **QM → Computer Access…** reopens it. A permission failure during a task reopens setup and reports the failure without replaying the action. For local macOS builds, set `QM_MAC_SIGN_IDENTITY` to a consistent signing certificate to keep the application identity stable across updates. Ad-hoc builds can leave stale macOS permission entries: remove the old entry, add the current app, and enable it again. --- ### Overview (`fly/README.md`) # Sandbox base image `fly/Dockerfile` builds the shared sandbox **base image**: a minimal generic toolset (the coding-agent CLIs, AWS CLI v2, the baked Python venv at `/opt/agent-venv`, and the optional agentic browser engine behind the `INSTALL_BROWSER_ENGINE` build arg). The directory name is historical — it predates the retirement of the Fly Machines sandbox backend — and stays because the Dockerfile is consumed by paths that pin it by name. Where the image is used: - **Signed releases.** `release-package.yml` builds and signs it as `ghcr.io//qm-sandbox-base` alongside the service images. - **Local docker sandboxes.** `scripts/local-sandbox-build.sh` builds it as `qm-sandbox-base:dev`, then stacks `local/Dockerfile` on top to produce `qm-sandbox-local:latest` for `SANDBOX_BACKEND=local`. - **Deployment layer builds.** A deployment's `sandbox/Dockerfile` may build FROM a published base via `qm sandbox build` (a local validation build). Runtime sandbox backends (sprites, smolmachines, e2b, modal) do **not** boot this image. Sprites and smolmachines boot their platform's stock image. E2B boots the template named by `E2B_TEMPLATE_ID` (build and publish it from `deploy/e2b/e2b.Dockerfile`) or, when unset, E2B's stock `base` template. Modal boots `MODAL_IMAGE` or, when unset, a default assembled at runtime from a digest-pinned `node:24-slim` with the sandbox toolset apt-installed on top (`src/sandbox/modal-client.ts`). The deployment layer's tool descriptors and skills arrive through the deployment-layer sync. Deployment-specific tools are NOT baked here — a deployment stacks them via its sandbox layer (`qm sandbox build` over `/sandbox/`). `fly/tools/x-api` is copied into the image by the Dockerfile and its file list feeds the local sandbox image fingerprint (`src/sandbox/local-sandbox.ts`). --- ### Overview (`deploy/helm/README.md`) # QM Helm chart This chart runs core, web-ui (including `/admin`), portal (including `/idp` auth), and the optional egress proxy. `services.auth` and `services.admin` configure embedded modules, not separate Deployments or Services. ## Images There is deliberately no default image tag. Build all enabled services from the same checked-out QM revision that supplies this chart, or independently verify a release's images include the combined topology and core `/readyz` endpoint. Old standalone auth/admin images are not compatible. The chart version is **not** an application image version. For example, from the repository root, with a registry you own and are signed into: ```bash export REGISTRY=registry.example.com/your-project/qm export TAG=$(git rev-parse HEAD) for service in core web-ui portal egress-proxy; do docker buildx build --platform linux/amd64 --push \ --build-arg GIT_SHA="$TAG" \ -f "deploy/$service/Dockerfile" -t "$REGISTRY/$service:$TAG" . done cat > images.yaml <`. `imageRef` takes precedence over repository/tag composition. Otherwise `services..tag` overrides `image.tag`. Configure `imagePullSecrets` for a private registry. Neither this chart nor these examples publish an official release. ## Cluster setup Bring these before installing: - Kubernetes, Helm 3, a reachable durable Postgres database, and backups. - A supported sandbox backend and its credentials. Kubernetes hosting does not configure sandbox provisioning or app hosting automatically. - DNS and an ingress controller. For TLS, provision the referenced certificate or install cert-manager with the configured ClusterIssuer. - The signing/encryption secrets, portal session secret, first administrator grant, sign-in allowlist, auth client/signing configuration and email transport described in [the deployment reference](../../docs/porter.md) and [`src/deployment/secret-schema.ts`](../../src/deployment/secret-schema.ts). Store credentials in a Kubernetes Secret and use `envFrom`, or supply `secretEnv` through a protected values file. Helm stores `secretEnv` values in its release history. Never commit these values. With `envFrom`, explicitly provide both `AUTH_ALLOWED_EMAILS` and `OIDC_ALLOWED_EMAILS` (or both corresponding domain variables); Helm cannot inspect externally supplied Secret values. When `envFrom` is present, the chart does not derive OIDC allowlist environment entries from `env` or `services.auth.env`, since those entries would override a potentially narrower policy in an external Secret. Set OIDC explicitly in values or the external Secret, including when mixing auth values with `envFrom`. Without `envFrom`, auth allowlists in `env` or `services.auth.env` propagate unless explicit OIDC values in `env` or `secretEnv` override them. Auth allowlists in `secretEnv` are aliased inside the chart-managed Secret; later `envFrom` sources retain their precedence. Only portal should normally have public ingress. `ingress.service` accepts an enabled `portal`, `core` or `web-ui`, not embedded modules or the TCP egress proxy. Exposing core or web-ui is an operator choice, not required for the portal setup. A minimal **Kubernetes configuration fragment**, not a complete installation: ```yaml publicUrl: https://qm.example.com imagePullSecrets: - name: registry-credentials envFrom: - secretRef: name: qm-runtime ingress: enabled: true service: portal hosts: [qm.example.com] className: nginx clusterIssuer: letsencrypt-prod services: core: replicas: 2 env: SESSION_STORE: postgres SNAPSHOT_STORE: s3 TRANSFER_STORE: s3 S3_BUCKET: qm-example-data S3_REGION: us-west-2 ``` This fragment expects `DATABASE_URL`, a remote `SANDBOX_BACKEND` and S3 credentials in `qm-runtime`; see [Core data and rollout behavior](#core-data-and-rollout-behavior). After supplying runtime config and `images.yaml`, render and inspect before applying: ```bash helm lint deploy/helm -f images.yaml -f cluster.yaml helm template qm deploy/helm -f images.yaml -f cluster.yaml helm upgrade --install qm deploy/helm --namespace qm --create-namespace \ -f images.yaml -f cluster.yaml --wait --timeout 10m ``` Validate the complete installation: test your database, ingress/TLS, sign-in, real turns, sandbox provider and backup/restore in your target cluster. ## Core data and rollout behavior Core keeps durable state in two places: Postgres (`DATABASE_URL`) and an object store for file bytes. Choose one of two layouts. **Shared storage (recommended, supports multiple replicas).** Set `DATABASE_URL` with `SESSION_STORE=postgres`, `SNAPSHOT_STORE=s3`, `TRANSFER_STORE=s3`, `S3_BUCKET` and `S3_REGION` (optionally `S3_PREFIX`) on core, use a remote `SANDBOX_BACKEND`, and leave `services.core.persistence` disabled. Sessions and runs then live in Postgres, and uploaded artifacts, session shares, published-app source archives and sandbox file transfers go to the bucket, so replicas share that state and pods can be replaced with rolling updates. Production core refuses to start without `DATABASE_URL` and `SESSION_STORE=postgres`, or with `RUN_STORE=memory`. S3 credentials come from the standard AWS chain, such as a pod IAM role bound through `serviceAccount.create=true` with `serviceAccount.annotations`, or `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` in the runtime Secret. Any S3-compatible store works outside AWS: set `AWS_ENDPOINT_URL_S3` to its endpoint (for example Cloudflare R2, Tigris or Google Cloud Storage with HMAC keys) and `S3_REGION` to what it expects, such as `auto` for R2. `/data` is then pod-local, and a few things still live only on the replica that created them: agent workspace files outside `artifacts/` (including files shared from there) and pending ChatGPT device logins. The chart sets no session affinity, so these can be unavailable from another replica and are lost on pod replacement. Switching an existing volume-backed install to S3 does not migrate bytes already under `/data`; copy `/data/docstore/` to the bucket under `S3_PREFIX` and `/data/session-shares/` under `session-shares/` first, or existing uploads and shares become unreadable. **Local volume (single replica).** Without an object store, core writes file bytes under `/data`, and they are lost when the pod is replaced even with Postgres configured. `services.core.persistence.enabled=true` mounts a ReadWriteOnce PVC at `/data`, sets `DATA_DIR=/data` and uses group 1000 for the Node image's volume permissions. Storage drivers must support `fsGroup`, or an existing volume must already be writable by UID/GID 1000. Persistent core supports zero or one replica, with `Recreate` deployment strategy so upgrades do not mount the same data concurrently. Expect downtime on upgrades; shared-storage multi-replica operation over a volume is not supported. Use the object-store layout for high availability. `replicas: 0` scales down without discarding the claim. The chart-created `-core-data` claim has `helm.sh/resource-policy: keep`; it is retained on uninstall or when persistence is removed. It is not a backup. Record the actual claim name before uninstalling. Reinstall with `services.core.persistence.existingClaim=` and persistence enabled to reuse it, rather than trying to create the retained claim again. Existing claims are never created or deleted by this chart. Delete retained data manually only when you intend to discard it. A null `storageClass` selects the cluster default; `""` requests no storage class. Existing-claim size and class are managed externally. `services..port` controls the process, Service and probes; conflicting `PORT` values are rejected. The egress-proxy image's Envoy listener is fixed at 48080, so other proxy ports are rejected rather than rendered as broken Services. Core liveness uses `healthPath: /healthz`; readiness uses `readinessPath: /readyz`. Its startup probe allows five minutes for initialization before liveness begins. Override `startupProbe` fields as needed, or set it to null to disable. Helm merges probe maps, so set `startupProbe.httpGet: null` when switching to an `exec` or `tcpSocket` handler. When overriding endpoint paths, update the startup probe too. Database unavailability should remove core from endpoints, not restart it through liveness. ## Regression tests Install Helm 3 on PATH (or set `HELM_BIN`) and run: ```bash node --test test/helm-chart.test.ts ``` These tests render real manifests and parse them with Helm's YAML parser. Missing Helm is a local skip and a CI failure. They do not replace live Kubernetes tests. --- ### Overview (`deploy/layers/README.md`) # Organization layers A private source fork can keep its deployment material under `deploy/layers//`. Core code may change independently to implement the organization's desired behavior. Public source checkouts keep private deployment material in a separate private repository. Package deployments need only that deployment repository, with no source fork. See [the README](../../README.md#customize-your-instance) for both paths. Upstream qm keeps only this shared README here. Organization layers never travel upstream; `upstream-pr` checks that boundary when a contribution is requested. ## Creating a layer ```bash node cli/bin/qm.ts init deploy/layers/ --org --target ``` `qm init` writes the deployment config, the secret-name example, the sandbox and provider scaffolding, an operator runbook, and a per-directory `.gitignore` that keeps `.env` values and Terraform state out of Git. Generate the layer rather than hand-building it so that `.gitignore` comes with it; the root `.gitignore` covers the same files as a backstop. The result, described in full in [`docs/deploy-directory.md`](../../docs/deploy-directory.md): ```text deploy/layers// qm.config.jsonc the deployment config; committed, no secret values .gitignore scaffolded; keeps .env and tfstate out of Git .env.example computed secret names, never values .env local secret values; never committed sandbox/ org tools and skills for agent computers plugins// org-specific service images infra/ provider infrastructure and tfvars, on AWS targets slack-app-manifest.yml generated bot manifest deployment.md operator runbook ``` Point the CLI at a layer with `--config`: ```bash node cli/bin/qm.ts check --config deploy/layers//qm.config.jsonc ``` For a separate deployment directory, substitute its path in both commands. Deploy modified services with the explicit `--build-from` workflow in the README. Run the CLI from the tree as shown. `npm exec qm` does not work in a source checkout because the workspace symlink points at `cli/`, which is unbuilt. ## Nearby directories `deploy/stacks/` holds account-neutral contract fixtures used to test the Fly backend, and `deploy//` holds the service image and Fly templates the CLI renders from. Neither is a place for organization material. ## The rule Nothing under `deploy/layers/` may reach upstream qm: not the config, not the sandbox tools, not the infrastructure coordinates, and not the names of systems or people that appear inside them. Secrets never enter Git at all, in this directory or any other. They belong in the provider's encrypted secret store, with local values only in the gitignored `.env`. --- ### Overview (`deploy/stacks/README.md`) # Fly deployment fixtures The directories here are account-neutral contract fixtures for the `qm` Fly backend. They are not production deployments and CI does not deploy them. To self-host qm, start from the repository-root [`deployment.md`](../../deployment.md). It creates the organization's deployment directory under [`../layers/`](../layers/), asks for the operator's Fly organization or AWS account before mutation, and generates the optional bot manifest. It generates a separate Slack SSO manifest only when Slack is selected as the OIDC provider. The packaged `cli/templates/fly/.toml` files are the service templates. The CLI derives per-deployment copies under the ignored `deploy/stacks/.generated/` directory. Secrets belong in the provider's encrypted secret store and never in these fixtures. --- ### Overview (`desktop/desk/README.md`) # desk A small Rust program that lets an agent see and drive Mac apps: list running apps, read a window's accessibility tree as numbered elements (plus an optional screenshot), and run batches of clicks, typing, keys, scrolls and drags. It speaks JSON lines on stdio; see `PROTOCOL.md`. It copies the _shape_ of Codex's Computer Use: AX tree first, indexed targets, events posted to the target pid. No Codex code or prompt text is used. The QM desktop app bundles the release binary at `QM.app/Contents/Resources/desk` and spawns `desk serve` on first use, so macOS attributes Accessibility and Screen Recording to QM. `npm run package` in `desktop/` builds it with Cargo first. **Status:** exercised in the signed macOS desktop trial for permission setup, app observation, and Calculator interaction. ## Build and try it alone ```sh cargo build --release ./target/release/desk permissions --prompt open -a Calculator ./target/release/desk apps ./target/release/desk observe Calculator --png /tmp/calc.png ./target/release/desk act Calculator '[{"type":"click","target":{"index":12}}]' ``` Run alone, macOS grants permissions to the terminal that launches desk, so switch your terminal on under System Settings > Privacy & Security > Accessibility and Screen Recording, then restart it. App listing shows regular Mac apps. Explicit names, bundle IDs, and full application paths can also target accessory apps with an accessible, visible window. Background-only processes are excluded; observation and control remain subject to macOS permissions and app support. ## Platform considerations - **Background input is experimental.** Desk sends target-only synthetic activation and input without raising the app, but some apps may ignore it. `{"type":"raise"}` is an explicit foreground action. Live Mac verification is still required. - **Screenshots.** On macOS 14+ they use ScreenCaptureKit's `SCScreenshotManager`. The app's own menus, popovers and sheets in front of the target window are composited in, and the image grows to include any part outside the window, across displays if needed; `frame` says which screen area it covers. macOS 12.3–13 uses `CGWindowListCreateImage` for the main window only; menu/panel composition requires macOS 14+. desk links ScreenCaptureKit, so it needs macOS 12.3 or later. `pixels_per_point` still needs checking on a Retina screen. - **`_AXUIElementGetWindow`** (private, widely used) maps the AX window to a window id for capture and event targeting. - **Clipboard.** `paste` replaces the general pasteboard, presses Cmd+V and restores the previous contents only once the focused field visibly contains the pasted text. When it cannot confirm that (canvas or terminal views without readable text, password fields, slow apps), it leaves the pasted text on the clipboard and says so, rather than risk the app pasting the user's clipboard. Clipboard data is available before Cmd+V is sent, so the receiving app does not need a callback from desk to read it. - **Keyboard layout.** `type_text` reads the current layout with `TISCopyCurrentKeyboardLayoutInputSource` and `UCKeyTranslate` on desk's main thread (TIS calls are main-thread only on recent macOS). - **Accessibility enablement.** On first observation, desk requests `AXManualAccessibility` and uses `AXEnhancedUserInterface` only if an empty web area still needs it. Prior settings are restored on normal exit. - **Speed.** Element attributes are read in batches. Actions and settability are cached until that app posts an accessibility notification; frames, values and children are always read fresh. Offscreen pruning and a 6,000-element read cap bound each walk. - **Universal binary:** `cargo build --release --target aarch64-apple-darwin --target x86_64-apple-darwin`, then `lipo`. --- ### Overview (`plugins/admin/README.md`) # Admin plugin A minimal **admin surface** for the qm — the operator/admin plane from spec §14, delivered as an _added_ plugin (like the Slack plugin). It is a separate process that talks to the core **only** over the admin governance API; the core has zero dependency on it. Don't run it and nothing about the core changes. You reach it through the **portal** (real SSO); the surface trusts the portal-synthesized `admin=` cookie as identity and asks the **core** whether that principal is an admin (`GET /api/whoami` → core `GET /v1/admin/whoami` → `canAdminister`). It holds **no admin id list** of its own. Pick a scope, then either **edit governance** (command policy, SOUL, egress), **manage users** (the org-wide **Users** tab), or read the **observability** views — History, Files, Live, Errors, Audit, Skills, Crons, Deployments, Volumes. The **Users** tab (org-wide, org_admin-only) lists everyone who has used the agent (from session metadata — no content) with admin status joined, plus the authoritative grant list, and lets an org_admin **promote** a principal to org_admin or **revoke** — every mutation attributed and audited, the last org_admin protected. **Invite teammate** adds someone by email with Member or Org admin access. The invite form grants access without an expiration date. Teammates appear in Users before their first session. Addresses already admitted through the domain or allow-list may receive a sign-in invitation too. Re-inviting an admin never removes their admin role. Invitations are durable and listed with their access status. Revoke ends access immediately. Invitation email uses Resend when configured. Teammate invitations contain a single-use sign-in link valid for 24 hours. If delivery is unavailable or fails, Admin shows the same link with a copy control after the invitation is created. Redemption checks current membership and claims the token through durable storage; revocation or a new invitation invalidates earlier links. Company access displays the web email domain and the local Slack allow-list; these are deployment settings, not editable Admin policy. The existing external-user API still requires an expiry and rejects org members. Teammate invitations and revocations are restricted to the Admin surface. (`org_admin` is the only supported role for now; `team_admin` was removed — team-scoped admin observability is future work. See `src/admin/admin-service.ts`.) History (newest sessions, org-wide or narrowed to one scope, by origin, drilling into transcripts with per-turn model-context breakdowns), Files (workspace contents), and Live (ongoing/recent runs) are **top-down content** views: an org-scope query spans the whole org; a narrower scope is limited to that scope. Every action is authorized in the core and audited. ## Run ```bash PORT=8080 ORG_ID=acme npm start cd plugins/admin CORE_API_URL=http://localhost:8080 CORE_ORG_ID=acme PORT=8090 npm start ``` No separate build command is needed. The Node 24+ server bundles the admin Lit modules once at startup and embeds them in the existing CSP-hashed script. The backend uses `node:http` and native TypeScript, with optional Sentry error reporting. The admin tabs render through the Lit modules in `ui/`, with `ui/admin.ts` as their shared entry point. They use the existing light-DOM elements, classes, and styles without layout wrappers. Settings drafts, validation, dirty state, and save feedback render from state; list views own filtering, pagination, and editor state. Stable row keys preserve focus. Shared table, card, and list templates live in `ui/shared.ts`. Specialized safe Markdown, XML, and tool-output formatters remain shared adapters in the shell. The shared admin controller retains routing, API calls, and change-review dialogs. Successful saves commit the submitted snapshot, preserving newer edits made while a request was in flight. Scope and render generations prevent stale requests from replacing a newer page. Related settings refresh independently so changing one section cannot discard neighboring drafts. Run `npm test` and `npm run typecheck` from this directory after changing the UI. Env: `CORE_API_URL` (default `http://localhost:8080`), `CORE_ORG_ID` (default `acme`), `PORT` (default `8090`), `CORE_SIGNING_SECRET` (required outside isolated development), and `INBOX_USERS` (the same comma-separated principal allowlist used by Inbox and Calendar). The portal also supplies a short-lived `x-portal-identity` token, which this surface forwards to core. There is **no** `ADMIN_PRINCIPALS` — admin identity + role + scope live solely in the core's durable, mutable `admin_grants` store, and this surface derives admin status from it via `/api/whoami`. `ADMIN_GRANTS` (env) is now only the **one-time seed** for an empty store; after that, admins are promoted/revoked at runtime through the Users tab (a redeploy never clobbers runtime grants). ## How it stays safe - **The browser never holds an admin credential.** The portal supplies the verified identity in a short-lived signed header; the compatibility cookie alone is not accepted when auth is configured. Core verifies the token and decides admin-ness on every action. - **All authority is enforced in the core**, not here (spec §14): the core authorizes every read and write against `admin_grants` (`canAdminister`) and refuses scopes you don't administer (403). `GET /api/whoami` reports that same grant state (it grants no new power). - **Content reads are scope-authorized and audited.** Transcript, model-request, file, memory, keychain, and other sensitive bodies are served only to admins of the owning scope, and every read lands in the audit log. ## Endpoints it serves `GET /` (UI) · `GET /healthz` · `GET /api/me` + `GET /api/whoami` (identity + derived admin status) · `POST /api/logout` · `GET /api/scopes/:scopeId` + `PUT /api/scopes/:scopeId/:resource` (`command-policy|soul|egress`) · `GET /api/{errors|audit|crons|deployments|skills|sessions|runs|files}?scope=` · `GET /api/sessions/:id?scope=` (transcript) · `GET /api/sessions/:id/llm?scope=` (captured model requests) · `GET /api/files/read?id=` + `GET /api/files/download?id=` (document content) · `GET /api/retention` (org-wide) · `GET /api/users` (org-wide; roster + grants) · `POST /api/grants` (promote) · `DELETE /api/grants/:principalId?scope=&role=` (revoke) · `POST /api/external-users` (invite) · `DELETE /api/external-users/:email` (revoke) — all proxied to the core's `/v1/admin/…` with the admin actor injected. Grant and external-user mutation is **org_admin-only** (enforced in the core, not the surface). ## Slack setup Hosted Add to Slack is shown only when core reports `installAvailable`, enabled by `QM_SLACK_SERVICE_URL` and its matching `QM_SLACK_SERVICE_TOKEN`. Public deployments without that service lead with the preconfigured Slack app manifest and token form. Connected hosted apps offer Re-add to Slack; custom apps must be disconnected before switching to the hosted app. Both paths retain the custom app setup instructions. --- ### Overview (`plugins/auth/README.md`) # auth — the built-in sign-in broker An OIDC authorization server that speaks exactly the subset [`plugins/portal`](../portal/src/oidc.ts) consumes. Instead of an external identity provider, people prove who they are by opening a one-time link emailed to an allowed address. ## Endpoints | Route | Reached by | Notes | | --------------------------------------- | ------------------------------------------- | ---------------------------------------------------------------------------------------------- | | `GET /authorize` | browser, via the portal at `/idp/authorize` | validates the request and renders the email form | | `POST /authorize` | browser, via the portal | with email configured, answers with the same confirmation page, then emails a link out of band | | `GET /verify` | browser, via the portal at `/idp/verify` | consumes the link and redirects to the portal's `/auth/callback` with a code | | `POST /token` | portal, over the private network | HTTP Basic client auth, authorization-code grant, PKCE S256 | | `GET /userinfo` | portal, over the private network | Bearer access token, verified statelessly | | `GET /.well-known/jwks.json` | portal, over the private network | the ES256 public key | | `GET /.well-known/openid-configuration` | operators | discovery, for debugging | | `GET /healthz` | the platform | liveness | The broker is never published directly. The portal republishes only the three browser-facing routes under `AUTH_BROKER_PREFIX` (`/idp` by default), which is why the issuer is `https:///idp` and the sign-in pages share the portal's origin, cookies, and CSP. ## Durability Nothing about a sign-in lives in this process. The sign-in link, the authorization code, and the access token are self-contained JWTs sealed with purpose-separated keys derived from `AUTH_TOKEN_SECRET`; the id_token is signed with the P-256 key in `AUTH_SIGNING_JWK`. Single use — of both the link and the code — and the send rate limits are claimed through core's Postgres-backed `ReplayDedupe` over the chassis signed core client, so a restart, a blue-green deploy, or a second instance cannot resurrect a spent link. If core cannot record a claim the broker fails closed and refuses the sign-in. ## Configuration Values below are set by `qm` from the deployment config and the secret store. Authentication keys and an email allowlist or domain remain required. Email delivery is optional; missing or incomplete email configuration disables delivery. Fully supplied email configuration is validated at boot. | Variable | Source | | ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | `AUTH_ISSUER`, `AUTH_CLIENT_ID`, `AUTH_REDIRECT_URI` | derived from `publicUrl` | | `AUTH_CLIENT_SECRET`, `AUTH_TOKEN_SECRET`, `AUTH_SIGNING_JWK` | generated by `qm setup` | | `AUTH_ALLOWED_EMAILS`, `AUTH_ALLOWED_EMAIL_DOMAIN` | the operator's admin address or domain | | `AUTH_PASSWORD_USERS` | optional `:` entries for password sign-in (below) | | `AUTH_PASSWORD_LIMIT_PER_EMAIL`, `AUTH_PASSWORD_LIMIT_PER_IP` | optional, attempts per `AUTH_SEND_WINDOW_S` (defaults 10 and 30) | | `AUTH_EMAIL_FROM` | the operator's verified sender | | `AUTH_BRAND_NAME` | `botName` in the deployment config; the Admin page's live branding, when set, takes precedence on rendered pages and emails | | `AUTH_EMAIL_TRANSPORT` and the chosen transport's variables (below) | the operator's email provider | | `AUTH_LINK_TTL_S`, `AUTH_CODE_TTL_S`, `AUTH_ACCESS_TTL_S`, `AUTH_REQUEST_TTL_S` | optional, capped | | `AUTH_SEND_WINDOW_S`, `AUTH_SEND_LIMIT_PER_EMAIL`, `AUTH_SEND_LIMIT_PER_IP` | optional | | `CORE_API_URL`, `CORE_ORG_ID`, `CORE_SIGNING_SECRET` | the chassis core block | The signing key is single, not a set: rotating it means redeploying, and links minted by the previous key stop verifying at that moment. ## Password sign-in for getting started A fresh deployment can reach its first sign-in without Resend, SMTP, or an identity provider. Set `AUTH_PASSWORD_USERS` to comma-separated `:` entries and the sign-in page offers an email-and-password form beside (or instead of) the email-link form. A successful password sign-in creates the same remembered browser session and the same authorization code as an email link, so admin bootstrap, allowlists, and everything downstream behave identically. Each address must still be permitted by `AUTH_ALLOWED_EMAILS`, `AUTH_ALLOWED_EMAIL_DOMAIN`, or an invitation. Generate a hash from a checkout (the password is read from a hidden prompt, or from stdin when piped, and must be at least 12 characters): ```sh node plugins/auth/src/hash-password.ts admin@example.com ``` Then, from the deployment directory: ```sh qm secrets set AUTH_PASSWORD_USERS 'admin@example.com:scrypt$15$8$1$...' ``` Only scrypt hashes are stored; a plaintext password in the list refuses boot. Passwords are compared in constant time, and attempts are rate limited per account and per client address through the same durable claims as link sends, so a core outage fails closed. Password sign-in is meant for onboarding. Once the deployment is running, configure an email transport or an external identity provider and unset `AUTH_PASSWORD_USERS`; the page says as much. ## Invited external users An address an org admin has invited as an external user (Admin → Users, or by asking the agent) may sign in until its expiry even though it is on neither `AUTH_ALLOWED_EMAILS` nor `AUTH_ALLOWED_EMAIL_DOMAIN`. The env list is checked first and settles the answer on its own; only an address it does not cover is looked up in core over the signed core client (`GET /v1/auth/broker/email-allowed`), at every step — when the link is requested, when it is opened, and when the code is exchanged — so a revoked or expired invitation stops working at once. A lookup that fails or times out counts as not allowed. One of the two env variables is still required at boot. ## Email transport `AUTH_EMAIL_TRANSPORT` selects Resend (the default) or SMTP. When `AUTH_EMAIL_FROM` or any of the selected transport's credentials are absent, the broker starts without email delivery. The sign-in page and submissions return 503 with “Email delivery isn't configured”; no form is offered and no email is claimed to have been sent. Previously issued links and codes keep their normal expiration and single-use checks. To enable email sign-in, configure the selected transport's credentials and `AUTH_EMAIL_FROM`, then restart. The sender must be verified and may be written as `Name `. Supplying only part of the configuration leaves email delivery disabled. Fully supplied but invalid configuration is refused at boot. | Transport | Variables | Notes | | --------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `resend` | `RESEND_API_KEY` | A key with send access from . The sending domain must be verified under Domains, which needs DNS records; an unverified domain fails at delivery, not at boot. | | `smtp` | `SMTP_HOST`, `SMTP_USERNAME`, `SMTP_PASSWORD`, and optionally `SMTP_PORT`, `SMTP_TLS` | Any relay. `SMTP_PORT` defaults to `587`. `SMTP_TLS` defaults to `implicit` on port `465` and `starttls` otherwise; `none` is refused in production, and a relay that does not advertise STARTTLS is refused rather than sent credentials in cleartext. | `qm doctor` proves the Resend key is accepted, or that the SMTP relay is reachable and answers. Neither proves deliverability — the first real sign-in link does that. ## Known trade-offs The sign-in link carries its token in the URL **fragment**, which browsers never put on the wire, so it reaches no access log, no proxy, and no `Referer`. The confirmation page moves it from `location.hash` into the form and calls `history.replaceState`, so it does not linger in the address bar or the history entry either; the value is held in `sessionStorage` for the life of the tab so a reload still works. That last step needs JavaScript — the page says so, and the link can be re-requested if a mail gateway strips the fragment. The per-mailbox send budget is keyed on the mailbox _and_ the requesting client address, so a stranger cannot exhaust a known user's budget and lock them out; the per-address budget is what bounds a single source. Both are durable claims, so they survive restarts, and both are keyed by an HMAC under `AUTH_TOKEN_SECRET` so another plugin holding the shared core signing secret cannot compute — and pre-claim — a chosen mailbox's slots. ## Remembered browsers Verifying an email link creates a durable broker session in core's Postgres store. The browser receives an HttpOnly, Secure, SameSite=Lax cookie scoped to the issuer path (`/idp` behind the portal). Core stores only a hash of the random token; the cookie also carries a broker-only MAC, so another plugin's core signing credential cannot manufacture an email sign-in. Rotating `AUTH_TOKEN_SECRET` invalidates all remembered cookies. `AUTH_SESSION_IDLE_S` defaults to 30 days and `AUTH_SESSION_ABSOLUTE_S` defaults to 90 days. Both are whole seconds; the idle limit must not exceed the absolute limit, and the absolute limit cannot exceed 90 days. Reauthorization slides the idle expiry without moving the original authentication time or absolute expiry. The broker rechecks email eligibility before issuing and exchanging a code. `prompt=login` and `max_age=0` always require a fresh email. A positive `max_age` compares against the original email authentication time, also returned as `auth_time` in the ID token. `prompt=none` returns `login_required` when silent reauthentication is unavailable. Invalid prompt and max-age values fail closed. The portal forwards only the broker cookie to `/idp` and preserves all broker Set-Cookie headers. Ordinary `POST /auth/logout` clears both local cookies. `POST /auth/logout?everywhere=1` additionally revokes all remembered browsers for the authenticated email principal. The signed core endpoint `POST /v1/auth/broker/sessions/revoke` accepts `{ "email": "user@example.com" }` from that user's portal identity or an authorized administrator. These operations prevent future silent reauthentication; already-issued stateless portal sessions remain valid until their normal expiry. When hosted by a portal with trusted OIDC and an explicit provider label configured, the embedded broker receives its trusted provider label and shows an alternate sign-in link throughout the email flow. The link starts a fresh trusted login through the portal, without submitting an email or linking the two identities. Standalone brokers omit it. --- ### Overview (`plugins/onboarding/README.md`) # Onboarding plugin Adds a memory-first onboarding skill: it connects the user's accounts, reads a light snapshot of their real work, and proposes concrete automations — then persists the profile in memory. The core discovers plugin skills from `plugins/*/skills` by default and installs them into the org skill catalog at startup. The onboarding skill is therefore visible in a user's DM as `/onboarding`, and the core injects a pending-onboarding prompt when the personal memory notebook does not contain a completion or dismissal marker for the current version. ## Source Of Truth Onboarding state is readable memory, not a separate database table: ```md ## Onboarding - Onboarding: completed v2 on 2026-06-09. - I write in direct, straightforward, concise sentences. ``` `completed v2` suppresses the pending prompt. `dismissed v2` also suppresses it until the onboarding version changes — bumping the version re-triggers onboarding so an improved flow reaches users who finished an earlier one. SOUL updates are optional and should only hold compact first-person operating instructions. --- ### Overview (`plugins/portal/README.md`) # Portal — the public SSO front door The portal is the **one** publicly-reachable app in the stack. It replaces the per-surface `fly proxy` tunnels (one per app, each on its own local port) with a single signed-in URL that fronts every surface. It does provider-neutral **OIDC** sign-in (Google Workspace is the email-first deployment default) and reverse-proxies — over Fly's private 6PN — to the surfaces, which all stay **private** (no public `[http_service]` of their own): | Path | → upstream | Notes | | ----------- | ----------------- | ----------------------------------------------------------------------------- | | `/*` (root) | `-web-ui` | Pi web UI SPA, root-mounted (`/web-ui/*` 308-redirects to root for old links) | | `/admin/*` | `-admin` | governance — admin access derived from the core (`canAdminister`) | User deployments are never served on this authenticated origin; they use the dedicated apps domain. Set `PORTAL_APPS_DOMAIN` to a child domain of the company portal, with matching core `DEPLOY_APPS_DOMAIN`, gateway/session secrets, wildcard DNS and TLS. App hostnames can point at the same portal ingress: portal forwards them to core's deployment gateway before its own routes. Core authorizes each app request and removes gateway cookies before forwarding to the app. Each app has a separate browser origin; cross-origin browser writes are rejected. Keep the session cookie domain within the company, never a parent shared with other companies. For companies using the additional trusted OIDC entry, set core `DEPLOY_APPS_LOGIN_PATH=/auth/trusted/login` so app sign-in uses that identity provider. The default remains `/auth/login`. It is a thin `node:http` server (native TS type-stripping), like the other surfaces, and it does **not** import the core. ## How a request flows 1. **Sign in** — `GET /auth/login` starts an OIDC Authorization-Code flow with PKCE(S256) + `state` + `nonce`, all sealed into a short-lived signed `portal_oidc_tmp` cookie. 2. **Callback** — `GET /auth/callback` exchanges the code over the TLS back-channel (confidential client, `client_secret_basic` + the PKCE verifier), then binds the id_token signature and payload (`nonce`/`aud`/`iss`/`sub`/timestamps and, for Slack, the `team_id` workspace pin) against the configured HTTPS JWKS, then reads the subject from userinfo. The verified `sub` **is** the core principal id. It mints a signed `portal_session` cookie (`{sub, org, auth, exp}`, HMAC, 7-day sliding lifetime with a 30-day absolute maximum by default). 3. **Proxy** — every other path requires a valid session. The portal picks the upstream by the **exact first path segment**, strips the prefix, and proxies to the private upstream, synthesizing the surface cookie for compatibility and attaching a short-lived signed portal identity. Surfaces pass that identity to core, which verifies it before any user-scoped action. Before proxying, the portal asks core for the session subject's canonical principal (`GET /v1/principals/:id/canonical`, cached for a minute). A sign-in that an org admin has linked to another identity therefore acts as that person's canonical principal; see [`docs/principal-links.md`](../../docs/principal-links.md). ## Additional trusted entry (PoC) An optional OIDC entry route at `/auth/trusted/login` operates alongside the existing `/auth/login` provider. Configure `PORTAL_TRUSTED_OIDC` as JSON with `issuer`, `authEndpoint`, `tokenEndpoint`, `userinfoEndpoint`, `jwksUri`, and `clientId`; supply its distinct `PORTAL_TRUSTED_OIDC_CLIENT_SECRET` separately. Register the exact `${PORTAL_PUBLIC_URL}/auth/trusted/callback` URL at that provider. Endpoints must use HTTPS in production. This is an operator-configured trust relationship: the additional provider must enforce the destination company's admission policy before issuing tokens. Primary-provider email and workspace restrictions do not apply to this separate route. The route verifies OIDC signature, issuer, audience, nonce, PKCE and subject binding, and consumes login state through core's durable replay store. Principals are scoped to the issuer and subject, without automatic linking to email or Slack identities. No administrator role is assigned by default. To provision administrators on this entry path, set `PORTAL_TRUSTED_OIDC_ADMIN=1` on portal and set core's `TRUSTED_OIDC_ADMIN_ISSUER` to the exact trusted issuer. This requires a distinct `PORTAL_IDENTITY_SECRET` and durable replay storage. After verified provider login, portal submits a short-lived, single-use, purpose-bound assertion; core creates a normal durable organization-admin grant before portal issues the session. Primary login and impersonation cannot request this promotion. Deactivated users stay blocked. Existing sessions need a fresh trusted sign-in. Revoking a grant removes admin access until the next successful trusted sign-in; disabling provisioning prevents future grants but does not remove existing ones. Company login and Slack integration retain their existing configuration. Deployment CLI secret wiring, account linking, and live qualification are pending; this is not a released deployment feature. When `PORTAL_TRUSTED_OIDC_LABEL` is explicitly set, successful trusted sign-in makes the portal remember the configured issuer in a host-only, HttpOnly browser preference for one year. Ordinary `/auth/login` requests then return to the trusted route, including after logout or session expiry. Session renewal and logout also remember existing trusted sessions. A successful primary or operator sign-in clears the preference. The preference only selects a provider; it grants no identity or administrator access. A new browser or cleared cookies still uses the primary provider by default. Use `/auth/login?provider=primary` to explicitly select the primary provider; the trusted sign-in error page offers this alternative. ## Operator admin login without email Run `qm admin-login` with the deployment's configuration and secrets to generate a private admin login URL. Open it and confirm the displayed email. The URL expires after five minutes and works once. With multiple email-based `ADMIN_GRANTS` administrators, use `qm admin-login --email admin@example.com`. The command can also run with `PORTAL_PUBLIC_URL`, `PORTAL_SESSION_SECRET`, and `ADMIN_GRANTS` in its environment, without a deployment config file. `GET /auth/admin-login` only renders the confirmation page. Its script moves the token out of the URL fragment into the form and clears the fragment. Nothing signs in until the user submits that same-origin form. `POST /auth/admin-login` verifies the token's purpose, portal origin, signature and lifetime, checks the account's current admin status, and consumes its unique ID through core's durable Postgres replay store. It then issues the ordinary portal session and clears any old impersonation cookie. No account or role is created. This is an operator capability: keep deployment secrets and generated URLs private. It does not prove an email inbox, replace another user's login, or depend on a hosting provider. Leave the built-in broker's sender and selected email credentials unset to run without email; configure them later and restart to enable ordinary email sign-in. Existing external OIDC login is unchanged. Live integration checks require a running core, Postgres, admin, portal and built-in auth broker with email unset. Set `QM_ADMIN_LOGIN_ENV_FILE` to a private JSON file containing that deployment's environment and run `node --test test/integration/admin-login.test.ts` from the repository root. Use a disposable test deployment: the test redeems real links for its configured admin. ## Security model (the parts that must be right) - **Identity comes from verified OIDC or an operator-issued admin link.** The browser cannot assert an unsigned identity. The upstream request is built **from scratch** — an _allowlist_ of safe headers (`content-type`/`accept`/`user-agent`/…) plus the one synthesized cookie. The client's headers and cookies are **never** forwarded, so a browser can't smuggle a forged `admin=`/`x-as-principal` into a 6PN-trusting upstream, and there's no CL/TE desync surface (`node:http` frames the piped body itself). - **Routing decides once.** The router rejects pathologically-encoded targets (`%2f`, `%5c`, `%2e%2e`, `\`, `//`, NUL → 400), selects the upstream by exact segment, and derives the `/admin` gate from that _selected_ key — never a second `startsWith` that could disagree (tier-escape guard). - **Admin is derived from ONE source — the core's `admin_grants`.** The portal asks the admin surface (`GET /api/whoami` over 6PN, **no signing secret**), which asks the core (`GET /v1/admin/whoami` → `canAdminister`). The portal caches the boolean ~60s and **fails closed** (admin surface down ⇒ no Admin link on the landing, `/admin` 403). `canAdminister` is re-checked **per action** at the core — the live boundary. There is **no portal/admin id list** anymore; the only place an admin is named is the core's `ADMIN_GRANTS`. - **Non-admin users need no per-id config.** `PORTAL_EXPECTED_TEAM_ID` pins the workspace, so any verified member is a valid user (`WEB_UI_PRINCIPALS` empty = any verified principal). - **CSRF / open-redirect.** Every non-GET requires a same-origin `Origin`; `returnTo` is reduced to a same-origin path (rejects `//evil`, `/\evil`, `https:/evil`, `%2f%2f`/`%5c`). The one exception is an origin listed in `PORTAL_API_ALLOWED_ORIGINS`, and only on the chat API (below). - **Embedding the chat from another app.** `PORTAL_API_ALLOWED_ORIGINS` lists the exact origins (comma-separated, bare `https://host[:port]`, no wildcards) that may call the web surface's chat API from their own pages with the user's portal session: `/me`, sending a turn, following, cancelling and withdrawing runs, listing and reading sessions and their approvals, deciding an approval, uploading attachment blobs, and downloading files. Those routes answer CORS preflights for listed origins and grant them credentialed reads and writes. Every other route keeps the same-origin rule, and unlisted origins get nothing. The session cookie is `SameSite=Lax`, so a browser only sends it on these calls when the embedding app is on the same site as the portal (for example `internal.example.com` beside `qm.example.com`). Listing an origin trusts every page on it to act as any signed-in user who visits it, so list only first-party apps. - **Secret hygiene.** In production the portal refuses to boot unless `PORTAL_SESSION_SECRET`, `PORTAL_IDENTITY_SECRET`, and `OIDC_CLIENT_SECRET` are set and distinct from core ingress auth. Public and OIDC endpoints must use HTTPS. Session and temporary cookies use domain-separated keys. ### Documented trade-offs & residual risks (v1) - **Surface isolation.** The private surface hop carries a signed portal identity; core verifies it independently, so a synthesized cookie alone confers no user authority. User deployments stay on a dedicated apps hostname and are never proxied through the portal or admin origin. - **Local logout.** `POST /auth/logout` clears the portal cookies and returns `redirectTo`; browsers follow it to `/auth/signed-out`, a terminal page that never redirects to the provider. Sign-out is local to this portal: it does not contact the OIDC provider or end the provider session, so choosing **Sign in** may complete silently through an existing provider session. Sign out of the provider directly to end it everywhere. Built-in broker and anonymous logout retain their existing flows. Local-development logout also stops on the signed-out page until the user chooses **Sign in**. - **Stateless session limits.** Clearing browser cookies does not revoke a copied portal session before `exp`; the core's `canAdminister` (re-read per request) remains the live admin revocation path. ## Playground mode `PORTAL_PLAYGROUND=1` turns the deployment into a public try-it instance: an unauthenticated browser navigation (a `GET` that accepts HTML) mints an anonymous principal (`playground-`), seals it into the ordinary `portal_session` cookie, and continues — so each visitor's sessions, files, memory, and sandbox are pinned to their browser through the same scoping that isolates real teammates. Non-HTML requests without a session still get `401`, so the SPA's API calls ride the cookie from the first page load and bare `curl` never mints. What playground mode does **not** change: `/auth/login` still runs the full OIDC flow (that's how the one admin signs in — production still demands the usual OIDC config), `/admin` refuses anonymous sessions outright, and admin identity remains the core's `ADMIN_GRANTS`. Signing out of an anonymous session just clears the cookie; the next visit starts a fresh playground identity. Minting is rate-limited per client address through the core's Postgres-backed single-use claim store (the same one the sign-in broker uses), so restarts and blue-green deploys can't reset it; if the core can't record the claim the portal fails closed and answers 429. `PORTAL_PLAYGROUND_MINTS_PER_IP` (default 30, at most 64 — the core grants at most 64 claim slots per request) per `PORTAL_PLAYGROUND_MINT_WINDOW_S` (default 3600, at most 86400 — the core's claim horizon); the portal refuses to boot outside those ranges rather than silently serving 429 to everyone. IPv6 clients are bucketed per /64, not per address, so a visitor with a routed prefix can't rotate through fresh budgets. The client address comes from `clientIpOf` — on Fly that's `fly-client-ip`; elsewhere set `PORTAL_XFF_TRUSTED_HOPS` when a reverse proxy fronts the portal, or every visitor (and every crawler that accepts HTML) shares the socket address's one bucket. Because playground authority must never leave this origin, the portal refuses to boot with `PORTAL_PLAYGROUND` alongside `PORTAL_COOKIE_DOMAIN`, an apps domain (`PORTAL_APPS_DOMAIN` / `DEPLOY_APPS_DOMAIN`), or an explicit `PORTAL_DEPLOYMENTS_ENABLED=1` — a domain-wide cookie or the deployment proxy would hand anonymous sessions to surfaces that never see the `anon` flag. The deployment proxy (`/d//`), on by default for signed-in portals, turns itself off under the playground, and anonymous sessions are refused it at request time too. Outside the playground, `PORTAL_APPS_DOMAIN` defaults to `DEPLOY_APPS_DOMAIN`, and `PORTAL_COOKIE_DOMAIN` to the portal host itself when the apps domain sits directly under it (`apps.`), so one core-side variable configures both processes. Any other layout needs an explicit `PORTAL_COOKIE_DOMAIN` — deriving a shared parent by guesswork risks landing on a public suffix browsers refuse. Anonymous sessions are also refused the `/connect/*` and `/drop/*` flows, so a visitor can't attach real OAuth tokens or dropped secrets to a throwaway principal that a cleared cookie orphans. The `anon` flag lives only in the portal's session cookie — it does not cross the portal identity boundary. To the core, a playground visitor is an ordinary **internal** principal of the deployment's org: they can run turns, use their sandbox, create crons, and reach anything granted or published at `org:` scope, including org-granted credentials. That is the design — visitors are members of the playground org — so a playground must be its own deployment with nothing sensitive at org scope: no org-wide credential grants, no real connector credentials, no company data. A cleared cookie mints a fresh principal, so pair this with the core's real brakes: `BUDGET_USD_PER_WINDOW`, `ORG_BUDGET_USD_PER_WINDOW`, `RATE_LIMIT_PER_WINDOW`, and a single pinned model via the admin `base-model` / `webui-models` resources. Nothing garbage-collects an abandoned visitor's scope yet. ## Env Non-secret (`[env]`): `PORT` (8097 local / 8080 image), `PORTAL_PUBLIC_URL`, `CORE_API_URL`, `CORE_ORG_ID`, `WEB_UI_UPSTREAM`, `ADMIN_UPSTREAM`, `OIDC_AUTH_ENDPOINT` / `OIDC_TOKEN_ENDPOINT` / `OIDC_USERINFO_ENDPOINT` / `OIDC_ISSUER` / `OIDC_JWKS_URI` / `OIDC_SCOPES` / `OIDC_CLIENT_ID`, `PORTAL_EXPECTED_TEAM_ID`, `PORTAL_SESSION_TTL_S`, `PORTAL_SESSION_MAX_TTL_S`. `PORTAL_SESSION_MAX_TTL_S` caps a session's total life from authentication; it defaults to the larger of 30 days and `PORTAL_SESSION_TTL_S`, and boot fails if it is set below the TTL. There is no `PORTAL_ADMIN_PRINCIPALS` — admin access is derived from the core (see the security model above). `PORTAL_API_ALLOWED_ORIGINS` (default empty) opens the chat API to other first-party origins; see the security model. Boot fails on an entry that is not a bare https origin (plain http only for localhost). For local development only, `PORTAL_LOCAL_AUTH_BYPASS=1` mints a local session as `PORTAL_DEV_PRINCIPAL` without contacting OIDC. The portal refuses this in production and only accepts it when `PORTAL_PUBLIC_URL` is loopback. Identity: `OIDC_PRINCIPAL_CLAIM` — `email` (default; the org-canonical id: the verified work email, lowercased; sign-in fails unless the IdP marks the email verified) or `sub` (the IdP's opaque subject, e.g. the Slack U… id — only for deployments still keyed on Slack ids). `OIDC_ALLOWED_EMAIL_DOMAIN` — with `email`, additionally reject any account outside this domain (checked against the email suffix and Google's `hd` claim). An address those rules reject still signs in when an org admin has invited it as an external user: the callback asks core over the signed core client (`GET /v1/auth/broker/email-allowed`) and accepts an active invitation, `hd` notwithstanding. An address the env rules permit never triggers the lookup; core unreachable means not allowed. Deployments keyed on `sub` never consult it, so external users cannot sign in there. ### Google Workspace SSO with the email principal The OIDC client is generic, so Google is pure config. One-time setup: 1. In a Google Cloud project under the org's Workspace: **APIs & Services → Credentials → Create OAuth client ID** (type "Web application"), authorized redirect URI `https:///auth/callback`. On the consent screen choose **Internal** (members of the Workspace only). 2. Point the portal at Google: ``` OIDC_AUTH_ENDPOINT=https://accounts.google.com/o/oauth2/v2/auth OIDC_TOKEN_ENDPOINT=https://oauth2.googleapis.com/token OIDC_USERINFO_ENDPOINT=https://openidconnect.googleapis.com/v1/userinfo OIDC_ISSUER=https://accounts.google.com OIDC_JWKS_URI=https://www.googleapis.com/oauth2/v3/certs OIDC_SCOPES="openid email profile" OIDC_CLIENT_ID= OIDC_ALLOWED_EMAIL_DOMAIN= ``` (unset `PORTAL_EXPECTED_TEAM_ID` — it's a Slack-OIDC concept). 3. The Slack plugin keys on emails by default too (bot scope `users:read.email`, already in the repo manifest — reinstall the app if it predates the scope), so Slack turns and web logins resolve to the same principal. 4. A deployment whose state predates email keying (Slack U… ids in the DB) must be re-keyed **once**: `scripts/migrate-principals-to-email.mjs` (dry-run by default; see its header). Update the core's `ADMIN_GRANTS` to emails in the same change. Secrets: `OIDC_CLIENT_SECRET`, `PORTAL_SESSION_SECRET`, and `PORTAL_IDENTITY_SECRET`; each must be distinct from `CORE_SIGNING_SECRET`. ## Run / test ```bash npm start npm run typecheck npm test ``` See **`deploy/README.md` → Portal** for the public bring-up (IPs + cert, DNS, the Slack OIDC app, and secrets) and `cli/templates/fly/portal.toml`. With embedded auth, trusted OIDC, and `PORTAL_TRUSTED_OIDC_LABEL` explicitly set, all email sign-in pages offer a link to the trusted provider, including the email-unavailable and link-sent pages. The label is the provider display name. Without it, the email page and primary login routing retain their existing behavior. Switching keeps the destination from the signed email login transaction. The link is available without a remembered browser preference. The proxy also signs the original authenticated subject as `authenticatedAs`. Core verifies that it still belongs to the canonical person, rejecting stale claims after unlinking. Trusted-entry failures offer retry of that provider without an alternate email sign-in link; invitation authentication remains unchanged. The portal always issues `portal_session_x`, a `SameSite=None` twin of the session cookie that lets opted-in apps be framed by named origins. The cores serving the portal must include `f75cfd1c` (#1503), which strips `portal_session_x` before forwarding requests to deployed apps. Once this portal is deployed, do not roll core back below `f75cfd1c` on its own: an older core forwards that cookie's session bearer to app code, and cookies already issued stay in browsers until they expire (`PORTAL_SESSION_TTL_S`, 7 days by default). --- ### Overview (`plugins/web-ui/README.md`) # Web UI plugin An end-user web surface with a custom chat shell, connected to the platform core. It still uses Pi's `Agent` state machine and selected Pi web utilities for markdown, attachment loading, and model metadata, but the visible conversation UI is owned by this plugin. Two processes: - **A zero-dep `node:http` server** (`server/index.ts`) — holds the signed-in principal in an `HttpOnly` cookie, injects it as the turn's actor, and proxies a small set of `/api/*` routes to the core (chat turns, unified history, and **webhook management** — see below). It never imports the core and never sends a model/API key to the browser. - **A Vite-bundled front-end** (`src/`) — a custom Lit shell + transcript + composer. Pi's `Agent` LLM-call boundary (`streamFn`) is swapped for a bridge to the core. It `POST`s the turn (`POST /v1/turns?async=1`), then watches the in-flight reply over **SSE** (`GET /api/runs/:id/events`, relayed from the core's `partial`) so tokens are pushed as they arrive instead of discovered on the next poll tick. It falls back to polling `GET /api/runs/:id` where SSE can't be established (old/proxy-hostile environments). The real agent loop (model + the three primitives + memory + audit) runs server-side in the core's sandbox; the browser is a thin chat client. **Behind the portal.** When fronted by `plugins/portal` (the public SSO front door), this SPA is served at the portal root — the default `WEB_UI_BASE=/` build is the one the portal fronts (the old `/web-ui/` prefix is gone; the portal 308-redirects `/web-ui/*` to root for stale links). The front-end joins its base via `import.meta.env.BASE_URL` (the `withBase()` helper in `core-bridge.ts`), so direct/standalone access and `npm run dev` behave identically. Core binds run reads and signals to the portal-verified actor; no run bearer is exposed to browser code or placed in a URL. The active-run resume index (`/api/runs/active`) is per-process best-effort with a durable core fallback for personal threads. ``` npm install --prefix ../admin npm install npm run build npm run serve npm start ``` Dev (HMR): run `npm run serve` in one terminal and `npm run dev` in another — Vite serves the front-end on :5173 and proxies `/signin`, `/me`, `/api/*` to the node server. Env (see `.env.example`): `CORE_API_URL` (default `http://localhost:8080`), `CORE_ORG_ID` (default `acme`), `PORT` (default 8096), `WEB_UI_PUBLIC_URL`, `WEB_UI_PRINCIPALS` (csv allowlist; empty = any id, **dev only**), and `CORE_SIGNING_SECRET` (same value as the core when source-auth is enabled). ## Suggested activities Suggested activity generation is **on by default** when the configured harness supports it. Set `SUGGESTED_ACTIVITIES_ENABLED=false` on core to disable generation. Optionally set `WEB_UI_SUGGESTED_ACTIVITIES` on web to a JSON array for fixed fallback starters; unset it as well to hide suggestions entirely. No deployment-specific activity content is bundled into the public application. ```json [ { "id": "weekly-brief", "title": "Wake up to a fresh briefing", "prompt": "Let's set up a recurring briefing on the topics I follow.", "icon": "schedule" }, { "id": "project-app", "title": "Build a home for my projects", "prompt": "Let's build a private app to keep track of my projects.", "icon": "app" } ] ``` The first three entries appear above the empty personal-chat composer with colored icons, without a heading or expansion link. Selecting an entry fills and focuses an editable draft; it never submits a turn. Suggestions fade and collapse while a draft or attachment is present and do not appear in existing chats, shared contexts, or compact pane views. Collapsed suggestions are inert and hidden from assistive technology; reduced-motion preferences disable the transition. Dark mode uses subdued blue-gray suggestion text. Drafts use the normal persistence path. Each entry requires a unique lowercase alphanumeric/hyphen `id` (up to 64 characters), `title` (up to 65 characters), `prompt` (up to 1,200 characters), and `icon` (one emoji or `yc` for the orange YC mark; legacy `schedule`, `app`, `deck`, `people`, `calendar`, and `book` values also render as emoji). Configuration accepts up to 12 entries and 20,000 characters. Invalid configuration fails startup without printing its contents. Restart the web service after changing it. The authenticated `/me` response supplies the configured fallback and whether generation is enabled. Fixed starters are organization-wide; keep them free of personal activity or credentials. When enabled, opening a new personal chat enrolls the user in an ordinary personal cron named **Refresh my suggested activities**. Its first run starts immediately; subsequent runs happen around 2am in the browser's timezone, with the minute staggered by user. The cron runner uses the normal owner-scoped session, runtime selection, memory, history, tools, and authorized data access. It has no delivery destination. The standing task in `src/suggestions/activities.ts` asks it to research relevant context, avoid mutations or notifications, and return five validated activity objects: three are shown and two wait in reserve. Draft prompts use a natural, collaborative voice, such as "Let's...", with relevant facts and uncertainties as neutral context, without attributing knowledge, feelings, or beliefs to the user. They preserve explicit user preferences and scope while leaving the approach to the responding agent. It does not create a separate reduced-context model call. The UI reads the latest valid result from the cron's completed personal session, including responses larger than the truncated fire-log preview. It displays the previous result while a refresh runs and briefly polls for the first/new result; opening another chat does not normally invoke a model. Failed initial generations can retry after five minutes. Existing cron queueing, run persistence, authorization, fire history, and failure handling apply. When generation is enabled, each suggestion has a low-contrast dismiss button that strengthens on row hover or keyboard focus. Dismissing collapses that row while the next reserve suggestion expands into the list with the same height-and-fade motion the list uses when it collapses and reappears around a draft. The core stores dismissals on the user's suggestion profile (the 30 most recent IDs and titles), filters them out of every later response on every device, and adds their titles to the managed cron task so the next run avoids them; user-edited cron tasks are left alone. Three seconds after the last dismissal in a burst, the browser asks for a refresh; opening a new chat does the same if that request never arrived. The core starts one immediate run of the user's suggestions cron only when a dismissal is newer than the latest run and no run is in flight. Until it lands, fewer rows show; new rows then expand in. Cadence is reevaluated hourly. Ten or more user messages in sampled recent private conversations within 24 hours increases refreshes to every four hours; otherwise it returns to nightly. Accounts with no observed conversation activity or suggestion visits for 30 days are paused until activity returns. The owner can pause, delete, or edit the cron; custom task text and schedules are preserved. The global disable flag pauses managed jobs. Background work must also be enabled. Set `SUGGESTED_ACTIVITIES_CONTEXT` on core for rollout guidance (up to 8,000 characters). Describe the organization’s workflows and available tools there; optionally provide fallback starters on web. Guidance updates propagate to unmodified managed tasks. Public QM has no organization-specific suggestion context by default. Suggestions are private to their owner; generated sessions use the same scoped access controls as other personal work. ## On a phone Below 860px the same build behaves like an app rather than a shrunken desktop: - **Drawer, not rail.** The sidebar slides over the content from a floating menu button (or an edge swipe); a leftward swipe or a tap on the scrim closes it. Every top bar reserves the button's column so nothing renders under it. - **Bottom sheets.** Popover menus — composer settings, a session's ⋯, the user menu, the per-session tools — render as sheets with a backdrop; tap outside or swipe down to dismiss. - **Compact composer.** Attach · input · settings · send on one row; model, harness, effort, and Fast live in the settings sheet. Inputs are 16px so iOS never zooms on focus, and the layout tracks the visual viewport so the composer stays above the on-screen keyboard. - **Touch targets.** Message actions, file chips, selects, drawer rows, approvals, and back links are ≥44px; hover tooltips are suppressed on hoverless devices. - **Split canvas stays on the desk.** A phone never mounts the split layout and leaves the persisted desktop layout untouched. - **Installable.** `manifest.webmanifest` (named after the org's brand label), home-screen icons, Apple meta, and theme colors — "Add to Home Screen" opens standalone at `/`. The phone-class breakpoint is one constant (`src/viewport.ts`, `PHONE_MAX_WIDTH`), shared by the CSS media queries, the composer, and the split canvas. ## What you get - **Custom chat UI** — a first-party conversation surface with a left history rail, centered transcript, bottom composer, inline **model selector** (the models core reports as serviceable for the approved harnesses), explicit **effort selector** (`low|medium|high|xhigh|max|ultracode|auto`), **Fast mode** toggle, attachments, streaming partials, and a theme picker. Settings → Theme takes light, dark, or system, or an imported palette: drop in an iTerm2 `.itermcolors` preset or a VS Code color theme `.json` and the app repaints from it (background, text, sidebar, buttons, links, text selection, status badges, and code highlighting, mapped from the terminal's ANSI colours or the theme's token scopes). The palette is kept per browser alongside the light/dark choice. The UI drives Pi's `Agent` with a custom `streamFn` (`src/core-bridge.ts`) instead of mounting Pi's stock `AgentInterface`. - **Slash-command skill picker** — type `/` at the start of the composer to browse the **skills** available to you (B6): icon · name · description · scope, with the typed letters emboldened. Arrow/Tab/Enter to choose (it inserts `/ `), Esc/click-out to dismiss. The list is the signed-in principal's _visible_ skills — the same set the agent gets materialized into a DM turn — fetched once per session via the server's `/api/skills` proxy (→ core `GET /v1/skills?principalId=`, resolved by `app.listVisibleSkills`). The skill _body_ never reaches the browser; only name/description/scope do. - **Functional selectors** — the chosen model + effort + Fast mode ride along on each turn (`POST /v1/turns` `model`/`thinkingLevel`/`fastMode`); the Pi harness applies them server-side, bypassing Pi's stale thinking-level clamp for newer effort values and falling back to its default on any unknown value. Attachments are forwarded as `IncomingAttachment`s and materialized into the agent's sandbox inbox (same path as Slack file shares). - **Unified history** — every session you participated in (DMs, channels, web), via `GET /v1/sessions?principalId=`. The same person sees the same history here and in Slack (surface-independence, spec B6). - **Continue web DMs** — threads originated here (`threadRef` = `web:…`) are continuable. - **Contexts** (the **Contexts** sidebar view) — one card per scope the signed-in person can talk to the agent in: their `personal:` scope plus every channel/group context (each context is a separate workspace — own files, own memory). `GET /api/contexts` → core `GET /v1/contexts?principalId=` (personal + channels the pre-pushed directory membership places the principal in + any shared scope their own sessions place them in — assembled entirely from core-local data, never a live Slack call). Opening a context shows its conversations and a **New chat** that starts a _web_ conversation inside that shared workspace: the turn carries `scopeId` (+ `channelName` for the session label), the server maps it onto `conversation {kind: channel|group, channelRef}`, and **core re-authorizes membership** (pre-pushed channel membership, or prior participation in the scope's sessions — deliberately stricter than the cron-create "any internal may post to a public channel" rule: a post is visible to the channel, mounting its workspace from the web is not). The web-ui never vouches for membership — it only shapes the claim. Such chats are continuable like any web thread and show a floating pill naming the shared context; replies stay on the web (nothing is posted to Slack). - **Deep links** — the address bar always identifies the open conversation (`?session=`) or view (`?view=…`), kept in sync via `replaceState`; each conversation's ⋯ menu has a **Copy link**. Opening a link while signed out keeps the query string through sign-in, so shared links land in the right conversation (subject to the recipient's own access). - **Read Slack & group/channel sessions** — they render **read-only** (transcript only, no composer). One-way projection: a web reply would be invisible to Slack participants, so contributing to those Slack threads from the web isn't allowed yet (spec B6) — start a fresh web chat in the same context from the **Contexts** page instead. In the sidebar these read-only rows sit visibly recessed (a shade darker/dimmer) and carry the Slack mark plus the channel they live in (for example, `#engineering`) — the channel name is captured from the surface onto the session record (`Session.channelName`, plumbed `conversation.channelName` → `getOrCreateByThread`) and surfaced via `GET /v1/sessions`, so you recognize _where_ a conversation happened at a glance. - **Webhook management** (the **Webhooks** sidebar view) — register, list, and disable your own incoming webhooks (spec §7). Each registration is created with `owner = createdBy = you` in your `personal:` scope (identity comes from the cookie, **never** the request body — the same trust model as `/api/turn`). The server proxies three routes: - `POST /api/webhooks` → core `POST /v1/webhooks`; relays core's response — the webhook, the **absolute** public ingress URL (core builds it from its public base — the portal in prod), and the signing secret **once** (auto-generated if you leave it blank; never shown again). - `GET /api/webhooks` → core `GET /v1/webhooks`, then **filtered to `owner === you`** (core's source-auth list is operator-wide; secrets are already elided by the core). - `POST /api/webhooks/:id/disable` → ownership is **verified here first** (core's operator disable has no ownership check), mirroring the run-ownership gate, then proxied. The inbound ingress (`POST /v1/webhooks/incoming/:id`) is served by the **core** receiver, reached in prod through the **portal**'s one unauthenticated passthrough (the core is not publicly exposed); senders sign with their own per-webhook secret, which is the auth on that path. Dev single-host posts to the core directly. - **Cron management** (the **Crons** sidebar view) — create, list, run-now, enable/disable, and delete your own scheduled tasks (spec §7), same trust model as webhooks: created with `owner = createdBy = you` in your `personal:` scope (identity from the cookie, never the body), list filtered to `owner === you`, and every per-cron route ownership-gated here first (core's source-auth routes are operator-wide). A cron is either a **task** (a prompt the agent re-runs at each fire) or a **message** (literal text relayed as-is — requires a destination, since a relay with nowhere to deliver is a no-op), on an `everyMs` interval and/or a one-time `firstFireAt` (which may not be in the past). The server enforces a 1-minute interval floor; the scheduler itself runs in the core. - **Files / Connectors / Deploys** (sidebar views — management lives here in one place): - **Files** — the doc store (spec §19 `artifacts(kind=file)`; §3 "files & sharing = Google Docs"): one `GET /api/files` call (→ core `GET /v1/files?viewer=`) lists files you **created/uploaded** (owned) + files **shared with you**, recency-sorted, with Open/Download (`GET /api/files/:id/content`, streamed binary). Image files (`image/*`) show an inline thumbnail rendered straight from that same `/content` stream. Backed by a durable, owner-scoped registry — NOT a transcript scan. Delivered/uploaded files are owned at the initiator's personal scope so they surface here, and auto-shared with the conversation (ADR-0003 D2/D4): a public-channel file gets an `org:` read grant so every member sees it under "Shared with you"; private-channel per-member grants are sequencing-gated (off until enabled), so those stay owner-only until then. - **Connectors** — per-provider OAuth status with Connect / Reconnect / Disconnect. The server proxies `GET /api/connectors` → core `/v1/connectors/oauth/status`, `POST /api/connectors/:p/start` → core `/v1/connectors/oauth/:p/start` (redirect URI = this surface's `/connectors/oauth/:p/callback`), and `POST /api/connectors/revoke` → core `/v1/connectors/oauth/revoke`. The callback exchanges the code server-side (tokens never reach the browser) and bounces back into the SPA via a base-relative redirect. - **Deploys** — `GET /api/deployments` → core `/v1/deployments`, grouped into manageable, shared, and archived views. Detail and restore routes expose authorized metadata and bring an archived version back online; running apps open through the surface's signed deployment proxy. ## Notes - **No browser-side model keys.** The real agent loop (model + the three primitives + memory + audit) runs server-side in the core's sandbox. The model picker only expresses a _preference_ the core honors within its policy floor — keys, egress, and tools stay server-side. - Pi's client-side artifacts / JavaScript REPL are not enabled (tools run server-side). - The custom transcript renders from Pi `Agent` lifecycle events and `waitForIdle()`, so in-place streaming mutations are reflected without depending on Pi's stock chat renderer. This is a **surface plugin**: it carries its own front-end deps (Vite, lit, pi-web-ui) and runs as a separate process. The zero-runtime-dep core is untouched. ## Chat connection chips Chat authorization links share the connector service logos. Composio links use recognized service names in their Markdown labels; unknown or ambiguous names keep a generic icon. The destination URL and authorization behavior do not depend on the inferred logo. Gmail, Google Calendar, Google Drive, and Google Sheets artwork comes from [Simple Icons v16.0.0](https://github.com/simple-icons/simple-icons/tree/16.0.0) (CC0) and is bundled locally with the existing connector SVG artwork. ## Cohort welcome and app picker Set `WEB_UI_WELCOME_COHORT=F26` on the web surface to show the cohort welcome in a new user's empty chat. It replaces the automatic first agent turn for that deployment; ordinary chat starts when the user sends a message. The greeting uses the signed-in display name. The welcome remains above the messages in the earliest personal web conversation, including when reopened. It is selected from persisted session creation times. The champagne and soft flutter sequence replays on refresh only before the first message and respects reduced motion. The picker reads Composio's live catalog in usage order, omits apps that need no authorization, and searches the complete paginated catalog. Known services use local logos; remaining catalog logos use Composio's logo host. Selecting an app submits to the authenticated web surface and opens the provider's authorization link directly. Consent remains on the provider page. Slack is excluded from this picker. A dedicated Connect Slack card in onboarding and Settings authorizes the signed-in person’s Slack tools through Composio and links their verified Slack workspace identity to their existing web account. Installing the company bot remains a separate administrator action. The core bridge accepts a verified portal identity and uses either that person's own `COMPOSIO_API_KEY` keychain entry or an enabled org service credential granted to them. Secrets never enter the browser. The agent skill reads `/v1/composio/identity` to use the same organization/person identity as the picker. A company project key retains Composio's existing project-wide access boundary; the identity selects accounts and does not isolate them from other holders of that key. ### Local connection-return preview Open `http://localhost:8138/?connectionDemo=1` on the local dev instance. This loopback-only UI mode replaces app authorization with a provider simulation offering approval, cancellation, and failure. It makes a full navigation round trip with a callback URL, attempt nonce, status, and connected-account ID. The simulated verifier checks its own record rather than trusting `status=success` in the URL. Session storage retains the account-scoped attempt for twenty minutes, picker query, expanded state, scroll position, and simulated connections. Returning skips the welcome animation, verifies the simulated result, clears callback parameters, and restores the picker. Reset clears the preview's simulated connections. No provider authorization, tokens, or actual connected accounts are changed by this mode. Real authorization supplies a callback URL on the configured public origin and returns to the same conversation. A twenty-minute, user-bound session-storage attempt retains the account ID, originating widget, search, expanded state, and scroll position. The server lists only the authenticated actor’s active connected accounts; the browser verifies the expected account before showing success and removes callback parameters. Connected apps are refreshed on page load and window focus. Returning skips the welcome animation. Reply widgets become available after the reply is persisted. The loopback preview remains a separate simulation and does not connect real accounts. The welcome uses the organization's configured branding `orgName`, falling back to “your company” when it is unavailable. ### Setup widgets in agent replies In web chat, an assistant reply can include `::connect-apps{}` as a standalone paragraph to render the reusable app picker. The separate `::add-to-slack{}` directive renders the Slack setup action for administrators; include both to show both. It omits the welcome and animation and uses the signed-in viewer’s authorization routes. The Composio skill teaches this response for requests to connect apps or reopen setup. Code blocks, quotations, and inline examples remain ordinary text. The directive persists in the transcript and renders again when reopened. Connected-account status retains the same limitations as the onboarding picker and local return-flow preview. ## Optional product analytics Set `POSTHOG_API_KEY` to a PostHog project ingestion token to enable browser analytics. `POSTHOG_HOST` defaults to `https://us.i.posthog.com` and must be an HTTPS origin. The authenticated `/me` response supplies this public configuration; the portal serves the same web application, so it needs no separate SDK. Events are explicit pageviews by navigation view, accepted `message_sent` events, and `session_started` for the first user message in a chat. Company grouping uses `CORE_ORG_ID`; user identities combine company and authenticated principal. Browser analytics is disabled during impersonation. Autocapture, replay, exception capture, performance capture and feature flags are disabled. Event properties exclude chat content, URLs, query strings, titles and referrers. Delivery is best effort. Set the same variables on core to capture `app_published` after a successful new application or version deployment. Publications use the application's creator and company, matching browser identity. No key means no analytics requests. Core also captures `response_completed` and `response_failed` when a human turn reaches its final run state, including when a separate worker executes it. `completion_boundary=run` means processing finished, not confirmed delivery to the user. Successful silent or reaction-only results count as completed processing. `result_status` distinguishes those results; `surface` identifies web or Slack. Automation, automatic openers, impersonated turns, stopped turns, refusals, queued results and pending approvals do not emit outcomes. Retries emit only at the final run state. Stable insert IDs support deduplication. These best-effort events are not a complete reliability ledger and contain no response text or raw errors. ## Optional browser error reporting Set `SENTRY_BROWSER_DSN` on the web server to enable browser error reporting. Use a public HTTPS DSN without a secret key, such as `https://public@sentry.example.com/1`. Backend `SENTRY_DSN` is never exposed or used as a browser fallback. The authenticated `/me` response supplies the public DSN and an optional `SENTRY_RELEASE` (or `GIT_SHA`). The server adds only the DSN origin to the web application's connection policy. Reporting starts after authentication and stops on sign-out or an authentication failure. It is disabled during impersonation and when the browser DSN is unset. It collects uncaught errors, unhandled promise rejections, and errors the client catches and handles (tagged with an `error_code` naming the catch site and grouped by it). Events carry the full error message, cause chain, stack, release, org and signed-in user. Delivery is best effort. ### Browser performance timing Set `SENTRY_BROWSER_TRACES_SAMPLE_RATE` (0 to 1, default 0) alongside `SENTRY_BROWSER_DSN` to sample browser timings; `0.1` is a reasonable start. Each timing is sampled independently at that rate and carries its own random trace id. A page reports one `pageload` transaction (time to first byte, DOM content loaded, load, first and largest contentful paint, and a `page` attribute for the application view) and one `http.client` transaction per same-origin request made through the web client's shared fetch helper, measured to response headers and named by a fixed `/api/` allowlist (`GET /api/sessions/*`) with the HTTP status. At most 200 timings are sent per page. The full page/request URL, including query strings, is retained as a span attribute. Transactions, child spans, descriptions, attributes, and any attached request or user context are sent without scrubbing. Timings stop with error reporting on sign-out, authentication failure, and impersonation. ## Personal AI accounts Open **Settings → AI access**, or use the account label beside the model picker. Choose Company, Claude, or ChatGPT / Codex. Sign in with your subscription or use the secondary API-key option; connecting automatically selects that account. **Company access** switches back without disconnecting personal credentials. The choice is durable per person and applies to their human chat turns on the web and in Slack; background tasks retain company access. A submitted turn keeps its account choice, so switching affects new turns. Personal access failures do not retry on company credentials. Messages using a different account queue separately instead of steering an existing run; an explicit steer across accounts is refused. Organizations that already require individual accounts continue to require them. The standalone `::link-slack-account{}` directive offers personal Slack account linking in a web reply. It shows the account card or linked status, requires the company bot to be installed first, and does not include the app picker. `::add-to-slack{}` remains the company installation trigger. --- ### Overview (`src/slack/README.md`) # Slack surface (Socket Mode) The Slack surface, run **in-process with the agent core**: core boots it when Slack tokens are present in its env and hands it a direct client into core's services. **Socket Mode** means no public URL, ingress, domain, or TLS — it opens an outbound WebSocket to Slack, so you can run it from a laptop or any box with internet. ``` Slack ⇄ (WebSocket) slack surface (in core) ── direct calls ──▶ core services (Pi) ``` ## 1. Create the Slack app (one paste) 1. Go to → **Create New App** → **From a manifest**. 2. Pick your workspace, paste [`manifest.json`](./manifest.json), create. 3. **Install to Workspace** (OAuth & Permissions → Install). Copy the **Bot User OAuth Token** → `xoxb-…` → `SLACK_BOT_TOKEN`. 4. **Basic Information → App-Level Tokens → Generate Token and Scopes**: add the `connections:write` scope, generate. Copy `xapp-…` → `SLACK_APP_TOKEN`. (Socket Mode is already enabled by the manifest. No request URLs to configure.) ### Slack Agents (top-bar pin) The manifest enables Slack's [Agents & AI Apps](https://docs.slack.dev/ai/developing-agents) feature (`agent_view`) so users can pin the app to the Slack top bar and open it anywhere. Conversations in the resulting split pane are ordinary DM thread messages and flow through the normal DM machinery unchanged. Thread titles, context passing and suggested prompts are not implemented. Activity cards are experimental and off by default. Two caveats: - **Paid plan required.** Slack only serves the AI-apps experience on paid plans (the Developer Program sandbox also works). - **Existing installs must update the manifest and reinstall** to pick up the `assistant:write` scope and the `assistant_thread_started` and `assistant_thread_context_changed` events (Slack requires both to enable the feature; the plugin acknowledges them as no-ops). Installs that don't update keep working exactly as before. ### Optional activity cards In Admin → Customize → Feature flags, enable **Slack loading indicator (experimental)** for selected people or conversation scopes. It is off by default. The Slack app must support [`task_card`](https://docs.slack.dev/reference/block-kit/blocks/task-card-block) blocks and have `chat:write`. After QM commits to answering, one card is posted in the existing reply thread and updated in place: **Working**, **Waiting to resume**, **Background work**, **Monitoring**, or **No active work**. This describes recorded activity, not success, failure or delivery. Quiet ambient turns and unthreaded DMs do not create cards. Cards are visible to everyone in their thread; own status cards and edits are excluded from conversation history. Waiting cards survive turns and restarts. Only this conversation's registered background jobs and enabled, unexpired process monitors count, not recurring crons or agent prose. Unwatched exits become visible when checked, reconciled during sandbox setup, or expired. Commands and monitor instructions are never rendered. After more than five minutes, including waits, a subtle **Follow via QM Web** link opens the exact web conversation with normal authorization. Without a configured web URL, the link is omitted. A minute sweep refreshes activity and retries transient failures. Durable receipts and per-thread leases preserve card identity across workers; expired run leases are shown as waiting, not working. Slack has no conditional writes: late updates are repaired, and an ambiguous initial post can duplicate if Slack does not deduplicate `client_msg_id`. An outage can leave a stale card until recovery. Status calls have a separate five-second timeout and never delay reply delivery. Once no activity remains, the card settles without a spinner and is no longer tracked. Later work creates a fresh card. Disabling the flag settles existing cards without stopping work or deleting messages. Native session status and Stop events are not used. > **Running locally alongside another developer? Each dev needs their OWN app.** > See [Local dev with multiple developers](#local-dev-with-multiple-developers) > below — set the manifest `name`/`display_name` to `agent-` before > creating, so you don't share a bot identity. ## 2. Run One process — core boots the Slack surface itself when the tokens are in its env: ```bash cd ~/Programming/qm nvm use HARNESS=pi ORG_ID=acme ANTHROPIC_API_KEY=… \ SLACK_BOT_TOKEN=xoxb-… SLACK_APP_TOKEN=xapp-… \ npm start ``` (`OPENAI_API_KEY` or `OPENROUTER_API_KEY` works in place of `ANTHROPIC_API_KEY`; the base model follows whichever provider you configure.) (Or put the tokens in the repo-root `.env` — `npm start` loads it via `node --env-file-if-exists`.) It logs `connected as @agent …` when live; without the tokens, core simply runs without Slack. ## 3. Use it - **DM the bot** anything → it replies in the DM (one continuous session per DM). While it's working on your reply you'll see a **⚙ Working…** status message; once the agent starts producing text, the reply streams in place by editing that same message — so the wait is never blank and there's never a duplicate post. - **@mention it in a channel** → it replies in a thread (one session per thread). After a real Slack mention in that thread, **just keep talking in the thread — no need to re-@mention.** The bot then _listens_ to every reply and decides whether to chime in **like a thoughtful coworker** (a "turn detection" step in the core): it answers a question or a request aimed at it, but stays out of a back-and-forth between other people. When it decides to stay quiet it posts nothing at all (no spinner, no "I'll stay out of it"). An explicit @mention always gets a reply, and a _new_ top-level message still needs an @mention to start a fresh thread. - **Share a file** with it (DM or @mention with an attachment) → it downloads the file and the agent can read/process it; **ask it to send a file** and it uploads the produced file back into the thread. Bytes travel **out-of-band**: the plugin streams them to the core's blob endpoint (`POST /v1/blobs`) and the turn carries only a lightweight `blobId`, so files up to **~1 GB** (Slack's own per-file limit), up to 10 per message, ride the same path — no more squeezing base64 into the 1 MB turn body. Outbound files are fetched back from the core (`GET /v1/blobs/:id`) and uploaded via `files.uploadV2`. **Only Slack-hosted files are ingested** — externally-hosted ("remote") files are refused, since fetching an off-Slack URL from the plugin host would be an SSRF / token-leak vector. - **Replies render natively.** The agent emits standard Markdown; the plugin translates it to Slack "mrkdwn" before posting (`toSlackMrkdwn` in [`lib.ts`](./lib.ts)) so `**bold**`, `[links](url)`, `# headers`, and `- bullets` show as Slack styling instead of raw markup. - **The agent can react with emoji — a Slack-only feature with two paths.** For explicit reaction requests inside a normal assistant turn, the plugin teaches the agent the reply convention via a surface instruction in the turn's `gatewayContext` (`REACTION_INSTRUCTION`): include a directive like `[[react: eyes white_check_mark]]` anywhere in the reply. The reply can be **just the directive with no other text** — the agent then reacts silently, the way a human drops a 👍 without typing anything (no "(no response)" placeholder, and the streamed status is torn down cleanly rather than flashing an empty bubble). The plugin parses the directive out of the reply (`extractReactions` in [`lib.ts`](./lib.ts)), removes it from the text, and applies the emoji with `reactions.add` to the message that triggered the turn — or, when the directive carries an `@ ` target (`[[react: tada @ ~abc123]]`), to that _specific_ earlier message instead. So the agent can resolve a phrase like "react to Alice's last message," each channel turn reconstructs the conversation as structured roled turns (`renderConversationView`, fed by a best-effort `conversations.history`/`replies` fetch that reuses the `channels:history` / `groups:history` scopes); every reaction-eligible message carries a **short, stable id** in brackets like `[~abc123]` — the message's Slack `ts` packed into base36 (`encodeTs`), written **once** on that message's turn. The agent copies that `~id` into the directive and the plugin **decodes it straight back to the `ts`** (`decodeTs` — no lookup map), validated against the set of message ts it actually showed this turn so a stale/forged id is dropped rather than reacting to the wrong message. The id is short (~10 chars vs a 17-char raw timestamp) and STABLE (it always points at the same message, so persisting it never drifts, unlike the old positional `[N]` handle). A raw `@ ` still works for back-compat. In-thread replies are tagged `(reply)` so the agent sees thread structure, and the parent of any listed reply is pulled in alongside it (`recentWindow`/`arrangeForDisplay`). The **bot token never leaves the plugin**. Names are normalized against the **complete Slack emoji set** — `:eyes:`, `eyes`, or a literal char all work, including the ~1900 standard emoji, ZWJ sequences (`🧑‍💻` → `technologist`), the `❤️`/`❤` variation-selector forms, and skin tones (`👍🏽` → `+1::skin-tone-4`). That table (`src/emoji-map.ts`) is built at module load from the [`emoji-datasource`](https://www.npmjs.com/package/emoji-datasource) package (iamcal/emoji-data, the dataset Slack itself uses) — bump the dependency to refresh it. Names are deduped and capped; a per-emoji failure (or a missing scope) is logged, never breaking the reply. The directive is also stripped from streamed partials so it never flashes while the reply types out. Separately, unprompted channel-thread messages can be acknowledged before the assistant turn runs: Slack passes `gatewayContext.reactionGuidance`, the core detector may return `status: "react"` with emoji names, and the plugin applies those to the triggering message without posting a reply. - **The agent also HEARS reactions — if someone reacts to a message, it can see it.** With the `reactions:read` scope and the `reaction_added` / `reaction_removed` events, a reaction the bot has a **stake** in is surfaced to the agent: a reaction on its **own** message, a reaction in a **DM** with it, or one on a **thread root it's following** (`shouldSurfaceReaction` in [`lib.ts`](./lib.ts)). Everything else — a stray reaction in a channel it merely sits in — is ignored, so a busy channel never floods the agent. A relevant reaction is delivered as an **unprompted turn** (exactly like a thread-follow message): the core runs **turn detection**, so a passing 👍 stays silent while a ✅ on the bot's "shall I deploy?" gets acted on. The reacted message is read once via `reactions.get` so the agent sees its text **and its full current emoji set** — e.g. _"Alice reacted with :white_check_mark: on your message: …(it now has :white_check_mark: :eyes:)"_ (`buildReactionTurnText`). The bot's own reactions never loop back, and the same emoji the agent can read on recent messages are appended to the reaction-targeting list (`buildRecentMessages`), so it can read the slackmojis there too. The reactor is classified and the channel audience is computed exactly as for a message — so a reaction in a Slack-Connect / guest-containing context never makes the bot respond — and that full audience is resolved (stake → dedup → coalesce → membership) **before** any message content is read, so a non-internal channel's content is never fetched. A burst of reactions on one message is **collapsed** to a single turn while one is in flight (`reactionsInFlight` guard), so an emoji-pile can't wake the agent once per emoji. _(Bounded by design: a reaction on someone else's non-root reply inside a followed thread isn't surfaced — the message thread-follow path owns that conversation.)_ - **Dangerous commands ask for approval in Slack.** When a tool call hits a `require_approval` command-policy rule, the bot posts buttons for **Allow once**, **Allow session**, **Allow always**, and **Deny**. Session/always approvals are exact-command grants; deny rules still cannot be bypassed. This works whether the turn _paused_ awaiting approval (`pending_approval`) or _finished with a reply but skipped_ the gated command (collect-mode `ok` + `pendingApprovals`, what the real Pi harness does) — in the latter case the buttons are posted alongside the reply. Clicking re-runs the turn with the command approved. - **Hand-offs are plain DMs.** When channel work needs another member's own logins, files or memory, the agent follows the `hand-off` skill: in Open conversations it uses the requester's own access; otherwise it sends that person a DM through `POST /v1/reach` with the full request, and says so only after reach confirms who it matched. - In an **externally-shared (Slack Connect) channel**, or any channel/mpim whose audience includes a **guest or external member**, it stays silent in-channel and replies privately (ephemeral) — and uploads no file there — internal-only (spec §9). Group DMs (mpim) aren't handled yet (it says so out-of-band rather than mis-scoping them). > **Re-install after upgrading to file sharing.** The `files:read` / `files:write` scopes > were added to `manifest.json`; update the app's scopes (or re-create from the manifest) > and **reinstall to the workspace** so Slack grants them, then restart the plugin. > **Re-install after upgrading to emoji reactions.** The `reactions:write` scope was added to > `manifest.json`. Update the app from the manifest and **reinstall to the workspace** so Slack > grants it, then restart the plugin — otherwise the agent's reactions are silently dropped > (logged as a missing-scope hint). > **Re-install after upgrading to inbound reactions.** The `reactions:read` scope and the > `reaction_added` / `reaction_removed` events were added to `manifest.json`. Update the app from > the manifest and **reinstall to the workspace** (re-subscribing the events), then restart the > plugin — otherwise the bot won't receive reactions others add, and `reactions.get` reads fail. > **Re-install after upgrading to workspace emoji.** The `emoji:read` scope lets the > acknowledgment picker include custom workspace emoji. Update the app from the manifest and > **reinstall to the workspace**, then restart the plugin. > **Re-install after upgrading to thread-follow.** The `channels:history` / `groups:history` > scopes and the `message.channels` / `message.groups` events were added to `manifest.json`. > Update the app from the manifest and **reinstall to the workspace** (and re-subscribe the > events), then restart the plugin — otherwise the bot won't see non-mention thread replies. > **Re-install after upgrading to approval buttons.** The manifest now enables Slack > interactivity so Block Kit button clicks are delivered as Socket Mode `block_actions` > payloads. No public request URL is needed, but the app must be updated from the manifest > and reinstalled. > **Re-install after upgrading to group-DM thread-follow.** The `message.mpim` event and the > `mpim:write` / `mpim:history` scopes were added to `manifest.json`. Update the app from the > manifest and **reinstall to the workspace** (re-subscribing the events), then restart the > plugin — otherwise the bot won't see non-mention messages in **group DMs (mpims)**, so it only > ever responds there when explicitly @-mentioned. ## Local dev with multiple developers Slack **Socket Mode load-balances events across every client connected to the same app**. If two developers both run this plugin against one shared app, Slack delivers each incoming message to only _one_ of them (round-robin) — so you each lose ~half your messages, and a message can land on a teammate whose core is down. More config won't fix it; the collision is the shared _app identity_. **Fix: one Slack app per developer.** Separate apps = separate bot identities, and Slack delivers each app's events only to that app's own connection — zero contention. Each developer, once: 1. **Create your own app** from [`manifest.json`](./manifest.json) (step 1 above), but first change the manifest's `name` to `Agent (yourname)` and `bot_user.display_name` to `agent-yourname`. Install it; grab your own `xoxb-…` + `xapp-…`. 2. Put your tokens in the repo-root `.env` (gitignored — see [`.env.example`](../../.env.example)). Pick a distinct core `PORT` per dev if you share a machine. 3. `npm start` at the repo root. In Slack, talk to **your** bot: `@agent-yourname` / DM it. Nothing shared, nothing to coordinate. The repo's `manifest.json` is the template; your per-dev name + tokens live only in your local app and your gitignored `.env`. ## How it maps to the core | Slack | Core | | ---------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `app_mention` / `message.im` | `POST /v1/turns` | | `message.channels` / `message.groups` in a thread where the bot has a stake (it posted or was really @mentioned) | `POST /v1/turns` with `unprompted: true` (+ serialized conversation context) → core runs turn detection; `status: "silent"` ⇒ post nothing; `status: "react"` ⇒ add reactions and post nothing | | `reaction_added` / `reaction_removed` on a message the bot has a stake in (DM / own message / followed root) | `reactions.get` to read the message + its emoji → `POST /v1/turns` with `unprompted: true` → core runs turn detection; `status: "silent"` ⇒ post nothing; `status: "react"` ⇒ add reactions and post nothing | | user id + `users.info` (guest/restricted/other-team → guest) | `actor` + `isExternalGuest` | | `conversations.info.is_ext_shared` | audience contains a non-internal member → core refuses | | DM channel | `threadRef` = `dm:` (one continuous session per DM — channel-scoped; the assistant `thread_ts` only routes the reply + native typing status, never session identity) | | channel `thread_ts` (root) | `threadRef` = `ch::` (one session per thread) | | `TurnResult.reply` | posted in-thread; unprompted `refused` (`security_quarantine` included) → silent, addressed `refused` → requester-only (ephemeral in channels) except addressed `security_quarantine` → fixed in-thread acknowledgement; `pending_approval` (or `ok` + `pendingApprovals`, collect-mode) → Block Kit approval buttons | | Block Kit approval button (`block_actions`) | `POST /v1/turns` with `approval` (`once` / `session` / `always` / deny) | | Pending `/v1/surface-context` request (hanging long-poll, source-authed) | The agent's mid-turn context pull: core resolved the target (current conversation via the capability's opaque destination, or a directory-resolved public channel) and parked the request; the plugin enforces visibility (public, bot-member, not externally shared — or the current conversation itself, even private), fetches history with `before` paging + `match` filtering, and POSTs the messages back (`buildContextWindow`). The default injected window is small (`MAX_RECENT_MESSAGES`, env `SLACK_RECENT_MESSAGES`); context beyond it is pull, not push | ## Notes / next - The failed-turn admin link needs **no Slack config**: when a turn comes back `refused` with a real failure, the core supplies the admin transcript deep link (`TurnResult.adminUrl`, built from the session org's portal) and the plugin appends it as `Full error:`. Absent when there's nothing to link (boundary refusals, no portal configured). - Channel audience is enumerated per-member: `computeChannelAudience` (lib.ts) resolves the full member list (with a Slack-Connect / guest external marker, and an actor-only fallback when membership is unreadable) so the core's audience-floor is fine-grained. ## History page size `SLACK_HISTORY_LIMIT` optionally sets the live history/replies page size and default `read_thread` result count for workspace-owned apps (integer 1–200). Without it, owned apps request 200-message pages and return up to 100 messages by default. Shared-app installations use 15 for both defaults regardless of this setting. An explicit tool count overrides the result count, but not the live page size. Stored mirror reads keep their existing window; shadow comparisons match the live page size. ## Mirror qualification `SLACK_CONTEXT_SOURCE` controls conversation context and default Slack tool reads: - `live` (default) keeps Slack history as the source, including automatic channel thread expansion. - `shadow` returns the same live context while comparing stored events in the background. It logs `slack_mirror_shadow` counts for missing messages, text differences, and missing thread parents. It does not log message text or channel identifiers, backfill history, or wait for comparison before returning context. The live page is snapshotted before callers add trigger context, and shadow uses the same stored-message selection as mirror mode. Thread selection matches Slack’s first page, including its limitation that newer replies beyond that page may be absent. Channel roots use the newest page, and thread expansion truncation is tracked separately from root-page truncation. An extra stored row distinguishes exactly full pages from truncated pages; it does not prove complete Slack coverage. Counts cover raw message fields, not complete rendered context or author/subtype fidelity. At most one comparison runs per Slack client; concurrent reads are not all sampled. - `mirror` explicitly enables stored context and mirror search, with the documented partial coverage and bounded live fallback. Keep this opt-in until shadow evidence and development qualification are satisfactory. Ingestion continues in every mode. Shadow mismatch counts describe the sampled live window, not complete channel coverage. Compare channel and thread samples, edits/deletes, and normal agent turns before enabling mirror reads. No automatic mode promotion occurs. Slack mirror records preserve provider subtypes so stored context uses the same system-message filtering as live context. A missing stored subtype means legacy or unobserved metadata; an empty subtype means a verified ordinary message. Partial event updates preserve omitted subtype and thread metadata. Mirror-mode history fallback shares event content normalization and prefers fresh Slack duplicates; live and shadow reads never backfill their comparison source. --- ### Overview (`superserve/templates/README.md`) # Agent template `qm-agent.ts` builds the `qm-agent-` template used by the Superserve sandbox backend. ## Contents The template uses `ubuntu:24.04` on linux/amd64 and includes: - Shell and file utilities, Git, curl, jq, and SSH. - Node and npm, Python with a virtual environment at `/opt/agent-venv`. - Coding-agent CLIs, `gh`, `aws`, and `x-api`. Tool versions are pinned in the build script. Downloaded Node and CLI archives are checksum-verified. Python and pip wrappers select the virtual environment regardless of the exec environment's `PATH`. Commands run as root; QM sets `HOME` to `SUPERSERVE_HOME_DIR` (default `/root`) and uses `/workspace`. Deployment tools and skills are installed during provisioning. The optional browser engine is not included. The default Superserve sandbox has 8 vCPUs, 16384 MiB (16 GiB) memory, and 32768 MiB (32 GiB) disk. The disk includes the OS and toolchain as well as workspace files, dependencies, package caches, and build output. Override these with `--vcpu`, `--memory-mib`, and `--disk-mib` when building. CPU, memory, and disk capacity are set on the template, not at sandbox creation. ## Build ```sh export SUPERSERVE_API_KEY=your-superserve-api-key node superserve/templates/qm-agent.ts --release 0.1.0 --wait ``` `--release` names the template; use the release tag your deployment runs. `--wait` streams logs until the build completes. Without it, the script returns after queuing the build. Use `--base-url` or `SUPERSERVE_BASE_URL` to override the API endpoint. A ready template with the same name is reused. Failed builds are replaced; `--force` also replaces a ready template. Names are unique within a team. Set `SUPERSERVE_TEMPLATE=qm-agent-` on core after the build succeeds. The backend requires a ready template with this toolchain. Rebuilding under the same name does not update existing sandboxes. Changing `SUPERSERVE_TEMPLATE` to a different name replaces them and deletes their resident files. Export needed files first; see [updates and retention](../../docs/superserve.md#updates-and-retention). ## Increase sandbox capacity Existing templates and sandboxes keep their original capacity when these defaults change. Passing `--vcpu`, `--memory-mib`, or `--disk-mib` with an already-ready template name also reuses that template without resizing it. To roll out larger sandboxes: 1. Pause ongoing jobs, commit and push source changes to Git, and export other needed resident files before replacing sandboxes. 2. Build under a new template name, for example: `node superserve/templates/qm-agent.ts --release 0.1.0-8cpu-16ram-32disk --vcpu 8 --memory-mib 16384 --disk-mib 32768 --wait`. 3. Verify it with `node superserve/templates/verify-qm-agent.ts --template qm-agent-0.1.0-8cpu-16ram-32disk`. 4. Set `SUPERSERVE_TEMPLATE=qm-agent-0.1.0-8cpu-16ram-32disk` on core and deploy. This replaces existing scope sandboxes on use and deletes their old resident files; it does not resize them in place. Restore exported work into the new sandboxes. 5. Check `nproc`, `free -h`, and `df -h /root/workspace` in a new sandbox, then rerun representative builds and full typechecking before resuming heavy jobs. Use your deployment's release tag in place of `0.1.0`. A distinct suffix keeps the old template available during verification. `--force` rebuilds a same-name template but does not upgrade existing sandboxes. Confirm the requested shape is supported and allow for the increased CPU, memory, and storage in provider quotas and costs before rollout. ## Verify ```sh node superserve/templates/verify-qm-agent.ts --release 0.1.0 ``` The verifier creates a sandbox, checks its tools, workspace, Python environment, and command timeout behavior, then deletes it. It exits nonzero if verification or cleanup fails. `--template` selects a template directly instead of by release. `--keep` retains the sandbox for inspection and schedules deletion one hour after it pauses. Otherwise, automatic deletion on pause backs up explicit cleanup, including when the verifier is interrupted. --- ### Overview (`test/live-slack/README.md`) # Live Slack E2E (example) Post-merge live scenarios against example.slack.com. The workflow (`.github/workflows/live-e2e.yml`) boots the merged commit as a real instance on the GHA runner — core (`SESSION_STORE=memory`) + Slack plugin in Socket Mode on a **CI-reserved pool app** (`qm dev --ci`) — then `run.ts` drives the catalog in `scenarios.ts` posting as a real example **user** (the "QA Human" account), so every message takes the exact path a teammate's would: classification, lazy directory push, threading, acknowledgments, tool execution, and file delivery. ## Layout - `run.ts` — runner: lane scheduling (parallel lane fans out, the DM lane is serial because a DM is one session — `dm:` in src/slack), one retry (retried pass = _flaky_, still green), results + failing-session transcript dumps to `out/`. - `scenarios.ts` — the catalog. Prompts embed `ctx.marker()` (an `ci--` token) so assertions grep for unforgeable strings and teardown can sweep crons/channels by marker. - `harness.ts` — per-scenario `Ctx` (fresh channels, final-reply detection, LLM judge). - `slack.ts` / `core.ts` — fetch-based Slack Web API + source-auth-signed core admin clients (transcripts via `/v1/admin/sessions`, crons via `/v1/admin/crons`). A bot reply is accepted after it has stopped changing for 5s and satisfies the scenario's content or attachment assertion. Legacy status text is ignored. The long-task scenario requires a bot reaction or an acknowledgment before the substantive answer. The audience scenario accepts an answer at the channel top level or in the original thread; the threading scenario still requires a thread reply. The performance scenario runs alone and requires successful command execution before evaluating its timing budgets. ## Run locally Bring up a dev instance (`/dev-instance up`) or `node cli/bin/qm.ts dev --ci up`, then: ```sh SLACK_QA_USER_TOKEN=xoxp-… SLACK_BOT_TOKEN=xoxb-… \ CORE_API_URL=http://localhost:8181 CORE_SIGNING_SECRET=… ANTHROPIC_API_KEY=… \ LIVE_E2E_FILTER=@smoke node test/live-slack/run.ts ``` `LIVE_E2E_FILTER` takes a name substring, `@tag`, or `all` (`@smoke` = the two-scenario subset; `sandbox`-tagged scenarios are auto-skipped unless `SPRITES_TOKEN` is set). `LIVE_E2E_SHARD=n/m` splits the selected catalog deterministically for a CI matrix. Set `LIVE_E2E_OBSERVATIONAL=1` to report completed catalog failures (and alert when configured) without a nonzero exit; runner, setup, event-pump, and required-alert failures remain blocking. ## Arga twin backend (no shared test workspace, no pool app, fully parallel) With an Arga API key the same catalog runs against an **isolated Slack digital-twin workspace** ([docs.argalabs.com](https://docs.argalabs.com)) instead of a shared test workspace: its own bot, QA user, and minted `alice`/`bob`/`carol` actors per run — no shared workspace, no slot serialization, no hand-provisioned accounts, and no Slack rate budget shared across shards. The `twin-e2e` matrix job in the workflow provisions one twin per shard. Mechanics: `arga-provision.ts up` provisions the twin, seeds users/tokens (`arga.ts:seedTwinUsers`), pre-creates the reach target channel, and emits the boot env — `SLACK_API_URL` (the plugin's Web API seam), `SLACK_EVENTS_MODE=http` + `SLACK_SIGNING_SECRET` (the plugin listens for signed Events-API POSTs instead of Socket Mode, which the twin doesn't speak), and the minted QA/actor tokens. `run.ts` then starts an **event pump** (`arga.ts:startEventPump`) that polls the twin's `/admin/events` log and delivers each recorded event — signed, per-channel FIFO — to the instance's receiver. The pump also papers over two twin fidelity gaps: it stamps the missing `channel_type` and synthesizes the `app_mention` twin of a bot-mentioning `message` event (real Slack sends both; the plugin only dispatches channel mentions from `app_mention`). Run locally (Free-tier keys cap the twin at 10 minutes — enough for `@smoke`): ```sh export ARGA_API_KEY=arga_sk_… eval "$(node test/live-slack/arga-provision.ts up | grep '^export ')" node cli/bin/qm.ts dev --ci up LIVE_E2E_FILTER=@smoke node test/live-slack/run.ts node cli/bin/qm.ts dev --ci down; node test/live-slack/arga-provision.ts down ``` Twin-only scenarios (`scenarios-twin.ts`, tag `@twin`) exercise what a live workspace can't: event redelivery (dedupe/idempotency), mid-session scope revocation, and per-method rate limiting. Scenarios that mutate workspace-global twin config use the `exclusive` lane (run alone, after the other lanes) and restore what they touched. ### App access requests (`@deploy-access`) `scenarios-deploy-access.ts` drives the published-app access flow end to end on a twin: a stranger hits an app's subdomain, clicks _Request access_, the owner gets a DM card with Approve/Decline, a click (a signed `block_actions` POST to the receiver, as Slack would send) grants view access and DMs the requester, and the requester then reaches the app through the gateway. They carry the `apps-gateway` capability tag and skip unless the instance has the gateway configured: ```sh export DEPLOY_APPS_DOMAIN=apps.e2e.test AWS_DEPLOY_GATE_SECRET=$(openssl rand -hex 16) \ DEPLOY_APPS_SESSION_SECRET=$(openssl rand -hex 16) DEPLOY_APPS_LOGIN_URL=http://localhost:8181 ``` The default `docker` deploy provider needs a `docker` on PATH; a shim that runs each published entrypoint as a host process is enough (the scenarios publish a one-file `node server.mjs`). ## Tiers, triggers, quarantine - **Tiers via tags.** `@core` is the fast high-signal subset run on every push to main; the full catalog (`all`) runs on PR-label + on-demand. `@multiuser` = the multi-user channel scenarios; `@sandbox` = needs Fly; `@smoke` = the tiny two-scenario subset. Capability tags gate _where_ a scenario can run: `twin` needs a twin backend, and `no-twin` is its inverse — a capability the twin cannot serve, skipped on twin runs (with the reason) so it can't produce a red that says nothing about the product. - **Triggers.** push→main runs `@core`; add the **`live-e2e` label** to a PR to run the full catalog against that branch; `workflow_dispatch` takes a `filter` + a `slot` (1–3) so an on-demand feature-push run doesn't queue behind main's. Slots 2/3 need `LIVE_E2E_SLACK_{BOT,APP}_TOKEN_{2,3}` secrets (extra CI apps); slot 1 uses the existing pair. - **Quarantine.** tag a known-flaky scenario `quarantine`: it still runs and shows in the gallery/summary (with a chip), but never counts toward the pass/fail signal (exit code + `#ci-alerts`) while it's being stabilized. "Don't accept flaky" — fix or quarantine, don't let a red scenario erode trust in the suite. **Rollout state:** CI is required. Catalog failures alert `#ci-alerts` and fail the job; twin failures alert through the real CI Slack app. Observational mode is available only for explicit local runner invocations; the GitHub workflow does not enable it. **Growing the catalog:** `scripts/mine-slack-scenarios.ts` reads a core Postgres, clusters recent _multi-user channel_ sessions by shape, and emits a digest to hand-author new scenarios from (curation over an auto-compiler, on purpose). Run it in the core box (see the top-of-file usage). The multi-user set lives in `scenarios-multiuser.ts`. ## One-time example setup 1. **CI pool app** — create a Slack app from the same manifest as the local pool apps (`src/slack/README.md`), named "Agent (ci1)", Socket Mode on. Do **not** reuse a local `poolN.env` app: a CI run stealing a slot mid-debug would be miserable. → secrets `LIVE_E2E_SLACK_BOT_TOKEN` (xoxb), `LIVE_E2E_SLACK_APP_TOKEN` (xapp). 2. **QA Human** — a real example member account (full member, not guest — guests classify differently). Create a second tiny app "QA Driver" with **user** token scopes `chat:write, channels:read, channels:write, channels:history, groups:history, im:history, im:write, reactions:write, files:read, files:write, users:read` and install it **as that account** → secret `LIVE_E2E_SLACK_QA_USER_TOKEN` (xoxp). (`files:write` lets the QA user upload the selfie in the `photo-identity` scenario; reinstall the app if you added it after first install.) 3. **Channels** — create `#ci-target` (public; invite the ci1 bot _and_ QA Human; repo var `LIVE_E2E_TARGET_CHANNEL` = its channel id) and `#ci-alerts` (invite the ci1 bot; repo var `LIVE_E2E_ALERT_CHANNEL` = its id). Failures on main post there. 4. **Remaining secrets** — `LIVE_E2E_ANTHROPIC_API_KEY`, `LIVE_E2E_CORE_SIGNING_SECRET` (any random string; the runner instance is loopback-only), and the existing `SPRITES_TOKEN` (sandbox turns) and `FLY_API_TOKEN` (the cloudflared self-API tunnel). Runs are serialized per Slack app by the workflow's concurrency group — that _is_ the lease; there is no poolN.env machinery in CI. ## Fake-human actors (multi-user scenarios) Multi-user channel scenarios need several real example member accounts, each with its own user token, so a scenario reads like a play: `ch.as(bob).mention(...)`, `ch.as(carol).threadReply(root, ...)`. A scenario declares the actors it needs (`actors: ["bob","carol"]`); the runner **skips** it unless every declared actor has a token, and **invites only** those actors to its channel. At boot each actor posts one warm-up message in a scratch channel (the real lazy Slack→core directory push) and the runner polls the admin directory until it classifies — an actor that never resolves is dropped and its scenarios skip, so you never get a mysterious empty-directory miss mid-run. **Provisioning (~1 hr for five, one-time; the install step is dashboard-manual):** 1. **Five member accounts** via Gmail plus-addressing — `alice+slackqa1@example.com` … `+slackqa5`, all landing in your inbox. Invite each to example as a **full member** (not guest — guests classify differently). Name them alice … eve. 2. **User tokens** — reuse the "QA Driver" app manifest (same user scopes as the QA Human above) and **install it once per account** while signed in as that account; each install yields that member's `xoxp` token. 3. **Configure** — either `LIVE_E2E_ACTOR_TOKENS='{"alice":"xoxp-…","bob":"xoxp-…",…}'` or one var per actor: `LIVE_E2E_ACTOR_TOKEN_ALICE=xoxp-…`. Names are lowercased. 4. **Verify** before a run: `node test/live-slack/actors-verify.ts` prints each actor's handle + user id, or the auth error. ## Screenshots (pixel-truth, local, best-effort) The gallery reconstructs message content faithfully and links to the real thread; for actual Slack-rendered pixels run the **decoupled** screenshot sweep after a run. It drives a _dedicated_ Chrome profile (never your main Chrome), so it's non-disruptive, and gates each capture on the conversation's text appearing in the DOM — a flaky capture is skipped, never fatal. ```sh npm run gallery:shots:login # one-time: sign into example in the window that opens npm run gallery:shots # headless sweep → out/shots/*.png, folds them into gallery.html ``` `LIVE_E2E_SHOTS_PROFILE` overrides the profile dir (default `~/.cache/live-slack-shots-chrome`), `LIVE_E2E_SHOTS_PORT` the debug port (default 9333). ## Reading a run **Start with `out/gallery.html`** — one self-contained page (opens offline) that reconstructs what happened in Slack for _every_ scenario, pass or fail, so you can verify by eye instead of trusting the LLM judge. Each card shows the driver's messages, the bot's reply, its **full streaming edit history** (the `⚙ Working…` ticker → `▌` cursor → final text, collapsed under "N streaming edits"), and a **↗ slack** permalink into the real thread for pixel-true inspection. Failures sort to the top with their error and core turn errors inline. This is the primary artifact for the on-demand / feature-push use case. For deeper debugging, the artifact also has `out/summary.md` (also in the step summary), `out/results.json` (includes each scenario's timeline), `out/transcripts/*.json` — the failing scenario's full session entries **and LLM requests** (ground truth for "what did the agent actually see") — and `.ci-instance/{core,slack}.log`. Don't trust the agent's in-channel self-explanations; read the transcript. ## Sandbox-provider release qualification Set `LIVE_E2E_SANDBOX_PROVIDERS=all` with `LIVE_E2E_GATE=1` to require a real agent `execute` on every implemented provider: Sprites, AWS, local Docker, Smolmachines, E2B, Modal, Porter, and Agent37. These eight scenarios run concurrently in their own lane alongside the selected catalog. The source type requires a coverage entry when a provider is added. Missing providers, skips, retries, execution errors, and cleanup errors block release. Provider checks do not retry; a failed first execution blocks qualification. Release qualification cannot shard away a required provider check. Alternatively, set `LIVE_E2E_SANDBOX_PROVIDERS` to an explicit comma-separated provider list. Every listed provider remains mandatory and runs concurrently. Providers omitted from that list are outside the gate, not failed or skipped required tests. Empty, unknown, and duplicate provider names are rejected. Each check creates an isolated sandbox, makes it the default only for its fresh test channel, and asks the agent to execute a nonce command on its explicit ID. The assertion requires a correlated tool call and result with exit code zero, no timeout, and exact stdout. Agent reply text cannot satisfy it. Cleanup clears the test channel default and retires its named test resources, including failed provisioning records. Core requests are bounded, cleanup uses a separate deadline, and the runner waits for provider cleanup to settle before recording an attempt as timed out. Slack requests time out after 30 seconds without SDK retries or automatic rate-limit waits; the scenario runner owns retry policy. A cleanup deadline failure remains a release blocker and requires operator reconciliation of retained resource records. The target instance must enable sandbox resources and configure every required provider with real credentials and infrastructure. Local Docker additionally requires a Docker-capable host and a built sandbox image; a Fargate service alone cannot supply it. Unconfigured providers are failures, never optional coverage. When staging cannot host Docker, run `scripts/qualify-local-provider.sh /path/to/release-candidate.json` as a separate required job on an ephemeral Docker-capable ARM64 runner. It pulls the candidate's immutable core image from ECR and checks its embedded source SHA against the checkout. The runner builds the local sandbox from that source, then invokes the candidate's real application, Pi harness, model, and sandbox tools. Only the probe and transcript assertion are mounted into the core container; application code comes from the candidate image. Provide AWS ECR access and `ANTHROPIC_API_KEY`. Require this job alongside cloud qualification before promotion. If capturing output through a pipe, use a shell with `pipefail` so a failed probe cannot become a successful job. --- ### Overview (`vendor/pi-ai/README.md`) # Vendored pi-ai 0.82.0 (qm.1) qm installs `@earendil-works/pi-ai` from the GitHub release [`vendored-pi-ai-0.82.0-qm.1`](https://github.com/yc-software/qm/releases/tag/vendored-pi-ai-0.82.0-qm.1), built from stock npm 0.82.0 plus `provider-error.patch` (dist only; version unchanged; not sent upstream). Why: stock pi-ai folds a provider HTTP failure into the display string `errorMessage`, so qm had to parse text to tell a budget cap from a rate limit. The patch keeps the structured data next to it: - `AssistantMessage.providerError?: { status?, type?, code?, body? }`, from the SDK error, in the error path of openai-completions, openai-responses, azure-openai-responses, openai-codex-responses, anthropic-messages, google-generative-ai, google-vertex, mistral-conversations and bedrock-converse-stream (`extractProviderError` in `dist/utils/error-body.js`). - `AssistantMessage.rawStopReason?: string` when a stop/finish reason maps to `stopReason: "error"` (anthropic-messages, openai-completions, bedrock-converse-stream), e.g. Anthropic `refusal`. `errorMessage` is byte-for-byte what stock 0.82.0 produces. The vendored pi-coding-agent (`vendored-pi-coding-agent-0.82.0-security.6`) is security.5 with only its `npm-shrinkwrap.json` pi-ai entry repointed at this tarball, so there is one pi-ai in the tree; the root `overrides` entry enforces the same. Rebuild (both are byte-reproducible and verify their inputs' SHA-256): vendor/pi-ai/build.sh # earendil-works-pi-ai-0.82.0-qm.1.tgz vendor/pi-ai/repack-coding-agent.sh # earendil-works-pi-coding-agent-0.82.0-qm-security.6.tgz Drop this directory when qm upgrades past 0.82.0 to a pi-ai that exposes equivalent fields. --- ### Overview (`cli/test/e2e/README.md`) # `qm` CLI end-to-end tests These drive the **real** `qm` binary as a subprocess and verify each command's _objective_ — files actually scaffolded, containers actually running, the right `flyctl` actually issued, the deployment actually torn down — not just that something was printed. ```bash cd cli npm run test:e2e npm run test:all npm test ``` They load the repo's `./.env` into the child environment (so `ANTHROPIC_API_KEY` / `CORE_SIGNING_SECRET` / `SKILL_SIGNING_SECRET` are present, as an operator's shell would have them). ## What each file covers | file | command(s) | how it's real | | ------------------ | --------------------------------------------------------------------------------------- | ---------------------------------------------------------- | | `meta` | `help` · `version` · unknown cmd | exit codes + stream routing through the shipped bin | | `init` | `init` (+ `--org`/`--target`, clobber refusal) | scaffolds, then **round-trips through `check`** | | `check` | `check` (+ `--sandbox-dir`) | pass on a valid layer; exit 1 naming each break | | `plan` | `plan` = `up --dry-run` (docker) (+ `--build-from`/`--config`/`--sandbox-dir`/basePort) | asserts the resolved plan; verifies it starts nothing | | `sandbox-build` | `sandbox build --dry-run` (+ `--from`/`--app`/`--tag`) | asserts the generated Dockerfile + push command | | `docker-lifecycle` | `up`→`status`→`logs`→`down`/`--purge` | **real Docker daemon**, asserts against docker's own state | | `identity` | `init`→`check`→`slack render`→`outputs` with a custom `botName` | brands the real manifest; outputs rejects a stale one | | `fly` | `plan`/`status`/`logs`/`down`/`--target`/`--only`/`-f` (fly) | a **fake flyctl** records the exact commands issued | | `dev` | `dev status`/`down`/`logs`/`up` precondition | isolated pool store; never touches the real pool | ## Design notes - **Docker lifecycle uses stand-in images.** The CLI's job is to _provision and supervise_ services, so the test builds tiny `alpine` stand-ins (they echo every service's readiness line then idle) via `up --build-from `. That exercises the full build → run → wire → wait-readiness → status → logs → teardown path without depending on the heavyweight (and not-yet-published) service images. Skips when no daemon is reachable. - **Fly is never really deployed.** A fake `flyctl` (`FLY_BIN`) answers the read/list verbs and logs every invocation; real Fly deploys are outward-facing and create apps. - **Isolation.** Every deployment uses a unique org `qm-e2e-