openwhispr Agent Skills & AI Rules

# OpenWhispr Technical Reference for AI Assistants

This document provides comprehensive technical details about the OpenWhispr project architecture for AI assistants working on the codebase.

## Response Style

Keep responses focused, brief, and concise. Keep disclaimers and caveats short, and spend most of the response on the main answer. When asked to explain something, give a high-level summary unless an in-depth explanation is specifically requested.

## Project Overview

OpenWhispr is an Electron-based desktop dictation application that uses whisper.cpp for speech-to-text transcription. It supports both local (privacy-focused) and cloud (OpenAI API) processing modes.

## Architecture Overview

### Core Technologies

- **Frontend**: React 19, TypeScript, Tailwind CSS v4, Vite
- **Desktop Framework**: Electron 41 with context isolation
- **Database**: better-sqlite3 for local transcription history
- **UI Components**: shadcn/ui with Radix primitives
- **Speech Processing**: whisper.cpp + NVIDIA Parakeet (via sherpa-onnx) + OpenAI API
- **Audio Processing**: FFmpeg (bundled via ffmpeg-static)
- **Node.js**: 24 (pinned in `.nvmrc` — CI uses Node 24, do NOT regenerate `package-lock.json` with a different major version)

### Key Architectural Decisions

1. **Dual Window Architecture**:
   - Main Window: Minimal overlay for dictation (draggable, always on top)
   - Control Panel: Full settings interface (normal window)
   - Both use same React codebase with URL-based routing

2. **Process Separation**:
   - Main Process: Electron main, IPC handlers, database operations
   - Renderer Process: React app with context isolation
   - Preload Script: Secure bridge between processes
   - ONNX Utility Process: hosts all `onnxruntime-node` inference (text embeddings, speaker embeddings, fbank). Lazy-spawned on first use via `src/helpers/onnxWorkerClient.js` → `src/workers/onnxWorker.js`. Native crashes (e.g., ORT `bad_alloc`) confine to the worker; main process rejects in-flight requests, waits out a backoff, and respawns the worker on the next request (never eagerly, so a crash can't leave an empty worker running). A request that times out kills the worker so it respawns the same way; the embedding clients reload their sessions on the new worker. Exits once idle with no session loaded (`releaseIfIdle`, called after semantic search releases its model and after `speakerEmbeddings.js` unloads the speaker model 5 minutes after its last extract). Each session's calls are serialized in the worker, so an unload never releases a session mid-inference. Stopped in `will-quit`.

3. **Audio Pipeline**:
   - MediaRecorder API → Blob → ArrayBuffer → IPC → File → whisper.cpp
   - Automatic cleanup of temporary files after processing

## File Structure and Responsibilities

### Main Process Files

- **main.js**: Application entry point, initializes all managers
- **preload.js**: Exposes safe IPC methods to renderer via window.api

### Native Resources (resources/)

- **windows-key-listener.c**: C source for Windows low-level keyboard hook (Push-to-Talk)
- **windows-mic-listener.c**: C source for WASAPI mic session monitor (event-driven mic detection)
- **windows-system-audio-helper.c**: C source for WASAPI process-loopback system audio capture (meeting transcription). Excludes OpenWhispr's own process tree, so it hears every app on every output device. Requires Windows 10 2004+; falls back to Chromium display-media loopback when unavailable. Mid-session, the helper emits a `capture_silent` warning when its own stream is silent while a render endpoint is metering output, checked every second for the whole session because Windows keeps some apps' streams out of process loopback (Teams call audio, #1265). Main then starts renderer loopback beside the helper, and `meetingSystemAudioHandover.js` gives loopback the system channel only after it hears audio the helper misses, so a false alarm never drops a working capture. Outputs 24 kHz mono s16le PCM on stdout, line-delimited JSON events on stderr (same protocol as linux-system-audio-helper)
- **macos-mic-listener.swift**: Swift source for the CoreAudio process-object microphone listener (event-driven mic detection); falls back to aggregate device activity and retries PID monitoring from its heartbeat
- **globe-listener.swift**: Swift source for macOS Globe/Fn key detection
- **macos-window-bounds.swift**: Swift source for the System Settings window reporter used by the onboarding permission guide. Prints the dialog's bounds, whether a sheet or authorization prompt is in front, which app owns the front window (`settings` / `self` / `other`, attributed by process because window owner names are localized) and whether System Settings is running. CoreGraphics window list only, so it needs neither Accessibility nor Screen Recording
- **bin/**: Directory for compiled native binaries (whisper-cpp, nircmd, key/mic listeners)

### Helper Modules (src/helpers/)

- **audioManager.js**: Handles audio device management
- **clipboard.js**: Cross-platform clipboard operations
  - macOS: AppleScript-based paste with accessibility permission check
  - Windows: PowerShell SendKeys with nircmd.exe fallback
  - Linux: compositor-aware Wayland paste (Hyprland sendshortcut, wlroots wtype, GNOME/KDE portal keysyms) with native uinput/XTest and system-tool fallbacks. KDE's portal paste sends Shift+Insert to every window but Warp, which binds paste to Ctrl+Shift+V only: KWin resolves keysyms in the active layout only, so a Ctrl+V on a non-Latin layout arrives as Ctrl alone (#2055). The app runs on XWayland, which the compositor won't let read a native Wayland app's clipboard while a Wayland window is focused, so `_saveClipboard` falls back to `wl-paste --type text` when Electron reads nothing; otherwise the restore would wipe the user's clipboard. On KDE it also takes wl-paste's UTF-8 plain text whenever it differs from Electron's, since KWin can leave a stale X11 owner in place after a Wayland app copies, so a rich or image restore first writes the original's text (empty for an image) to Wayland, or the app's own transcript or selection sentinel left there would read as newer. A Wayland-only copy with no plain text (a screenshot copied in Spectacle) isn't saved, so a dictation replaces it with X11's older text
  - Linux: wlroots, Hyprland and niri (xwayland-satellite) hand XWayland's clipboard to a native Wayland app only while an X11 window has focus; Mutter, COSMIC and KWin 6.0.3+ do it themselves (KDE also writes the transcript and primary selection with `wl-copy`, and Settings asks for wl-clipboard there too, since `_saveClipboard` reads the Wayland clipboard there with `wl-paste`). There (`requiresWlCopy` in `linuxSession.js`; a leftover `SWAYSOCK` or Hyprland signature under another desktop doesn't count) the transcript reaches Wayland apps only through `wl-copy`, so `_pasteText` pastes only when wl-copy wrote both the Wayland clipboard (`_writeClipboardWayland`) and the primary selection that terminals paste on Shift+Insert (`_writePrimarySelection`). `wl-copy --primary` goes through data-control, which still takes a primary selection under niri `disable-primary` (checked on niri 26.04) and Hyprland `middle_click_paste = false` (its data-control code never reads that option). Otherwise it injects nothing and returns `reason: "wl-copy-missing"` (not installed) or `"wl-copy-failed"` (e.g. killed by its 200 ms timeout): the renderer shows the dictation-error pill (Retry writes and pastes the kept text again, never records), a selection edit reports `code: "wl_copy_missing"`/`"wl_copy_failed"`, and the paste-tools card and Settings → Wayland Paste Setup ask for wl-clipboard (#2621). There, a missing `wl-copy` is probed again rather than read from the 30 s command cache, so one installed after a held-back paste counts at once
  - Linux COSMIC: paste reads the focused app from cosmic-comp's `zcosmic_toplevel_info_v1` (`cosmicToplevel.js`, a minimal Wayland wire client, bound at version 1, which still sends app_id and state), never from xdotool, whose XWayland answer goes stale while a native Wayland window has focus. A known terminal is pasted through wtype (Ctrl+Shift+V; Shift+Insert for Konsole); everything else keeps uinput Shift+Insert, since COSMIC gives no PID to spot Electron apps hosting TUIs. Short terminal names (`st`) match whole words only, or every `com.system76.*` app id would read as a terminal
  - Linux: before any paste or selection-copy chord, `linux-fast-paste --capabilities --await-modifier-release <ms>` waits for physically held modifiers to be released (XKB state on X11, `EVIOCGKEY` on Wayland; `unknown` without `/dev/input` access pastes as before). A chord injected into held keys arrives as a different shortcut, and a virtual device cannot release another device's key (#2113). Still held after 1.5 s → not pasted, `reason: "modifiers-held"`, and the renderer shows the dictation-error pill with the transcript on the clipboard; a selection edit blocked the same way reports `code: "modifiers_held"` and gets the same pill (the assistant caret paste reports the same code, for logs only). `ydotoold`'s virtual keyboard is ignored, since an interrupted chord can leave its modifier down, and an unreadable key state is logged once per session. The wait runs before `pasteLinux` detects the target window, and a selection copy that had to wait looks the target up again, since focus can move while a key is held: a fresh capture then runs the command on its own (`focus_moved`), and a revalidated session declines as `target_changed`. Retry on a modifiers-held pill pastes the kept transcript again rather than recording again (the macOS/Windows push force-stop pill keeps its record-again Retry, since neither platform waits for held keys), ignores clicks while its paste is pending, and never dismisses a newer pill. `useAudioRecording` keeps the pill in processing until the paste attempt settles (the audio manager settles before the paste starts), but never reports that to main, which drops dictation hotkeys while processing; a paste held back after the next dictation started leaves the transcript on the clipboard without a pill. The helper measures its wait on `CLOCK_MONOTONIC`, since the JS watchdog kills it at timeout + 1 s
- **database.js**: SQLite operations for transcription history
- **noteUndo.js**: Undo for assistant `update_note` edits. The tool passes `{ undoable, expected, turn }` to `db-update-note`; inside one transaction main refuses the edit if a guarded field changed since the tool read the note, writes it, and journals the previous values in `assistant_note_undo` (one row per note, bound to the account and kept 10 minutes). An `AFTER UPDATE` trigger retires the row when any guarded field changes again (manual save, sync pull, delete, identity or folder change), treating `''` and NULL as equal because a pull stores an empty field as NULL; sync acknowledgements don't retire it. Edits of one note in the same chat turn (`turn` = the assistant message id) share one Undo; `update_note` runs one note's edits in turn, since the AI SDK runs a step's tool calls concurrently. `AssistantNoteUndoListener` (control panel and dictation window) offers each edit once, in the first focused window (`db-claim-note-undo`); typing in the open note retires it (`discardNoteUndo`). The trigger is recreated and a journal of another shape is dropped at launch; if setup fails, edits save without Undo
- **debugLogger.js**: Debug logging system with file output
- **providerHttpErrors.js**: Pure ESM classifier for BYOK provider failures (transcription and AI models), shared by main (require(esm)) and renderer. `providerHttpError({ provider, status, body, headers, model, surface, selfHosted })` builds an Error carrying `code` (`PROVIDER_AUTH_FAILED`, `PROVIDER_QUOTA_EXHAUSTED`, …), `messageKey` (`providerErrors.*`), `messageParams`, `settingsTarget` (`speechToText`/`llms`, fixable errors only) and redacted `technicalDetails`; `asProviderError` classifies AI SDK, timeout and network errors and leaves already-keyed errors alone. Raw provider bodies never reach UI text. Rendered by `src/utils/describeProviderError.ts`
- **devServerManager.js**: Vite dev server integration
- **dockManager.js**: Single owner of the macOS Dock icon
  - The icon follows the control panel: it appears when the panel opens and goes away when the panel closes to the tray, so no other caller (in particular the dictation panel's hide path) can resurrect it
  - Every path that surfaces or hides the control panel reports that state explicitly: openers (tray, app menu, deep links, `activate`, second instance, first show) go through `windowManager.showControlPanel()`, hiding through `hideControlPanelToTray()`
  - `showControlPanel()` reports the Dock before restoring, showing and focusing the window: on an active app `dock.show()` activates the Dock and then re-activates the app, which macOS does not honour, so a window surfaced first is left behind the previously frontmost app
  - `showControlPanel()` calls `show()` even on an open panel, since on macOS that activates the app and `focus()` does not; openers that send the user to another app (meeting-prompt Join, the meeting hotkey, Upcoming Meetings' Join & transcribe) pass `{ activate: false }`, so an open panel updates behind the meeting instead of covering it. A hidden panel is still shown
  - `app.dock.show()`/`hide()` run only when the requested state differs from `app.dock.isVisible()`, so reopening an already open panel never triggers that Dock hand-off
  - Never derive that state from the window's `show`/`hide` events: on macOS those are occlusion events, so they also fire when the panel is merely covered by another window, minimized, or on another Space, and the icon flickers as the user switches windows
  - Decision logic lives in `dockPolicy.js` (pure, unit-tested in `test/helpers/dockPolicy.test.js`)
- **dragManager.js**: Window dragging functionality
- **environment.js**: Environment variable and OpenAI API management
- **hotkeyManager.js**: Global hotkey registration and management
  - Named hotkey slots: `dictation`, `voiceAgent` (voice assistant — dictation routed to the assistant panel), `translation`, `meeting`
  - Handles platform-specific defaults (GLOBE on macOS, Control+Super on Windows/Linux)
  - Auto-fallback to F8/F9 if default hotkey is unavailable
  - Notifies renderer via IPC when hotkey registration fails
  - Integrates with GnomeShortcutManager for GNOME Wayland support
  - Integrates with HyprlandShortcutManager for Hyprland Wayland support
  - Integrates with KDEShortcutManager for KDE Wayland support
- **gnomeShortcut.js**: GNOME Wayland global shortcut integration
  - Uses D-Bus service to receive hotkey toggle commands
  - Registers shortcuts via gsettings (visible in GNOME Settings → Keyboard → Shortcuts)
  - Converts Electron hotkey format to GNOME keysym format
  - Only active on Linux + Wayland + GNOME desktop
  - D-Bus transport: `@homebridge/dbus-native` (pure JavaScript, no native addons)
- **hyprlandShortcut.js**: Hyprland Wayland global shortcut integration
  - Uses D-Bus service to receive hotkey toggle commands (same `com.openwhispr.App` service)
  - Registers shortcuts via `hyprctl keyword bind` (runtime keybinding)
  - Converts Electron hotkey format to Hyprland bind format (`MODS, key`)
  - Only active on Linux + Wayland + Hyprland (detected via `HYPRLAND_INSTANCE_SIGNATURE`)
  - D-Bus transport: `@homebridge/dbus-native` (pure JavaScript, no native addons)
- **kdeShortcut.js**: KDE Wayland global shortcut integration
  - Uses D-Bus to communicate with KGlobalAccel for global hotkey registration
  - Registers hotkeys via `setShortcut`/`doRegister` D-Bus calls on the KGlobalAccel interface
  - Listens for `globalShortcutPressed` signals to trigger callbacks
  - Converts Electron hotkey format to Qt key codes
  - Only active on Linux + KDE desktop (detected via `XDG_CURRENT_DESKTOP`)
  - D-Bus transport: `@homebridge/dbus-native` (pure JavaScript, no native addons)
- **autoStart.js**: Single entry point for launch at login, used by `ipcHandlers.js` and `main.js`
  - Dispatches to `setLoginItemSettings()` on macOS/Windows and to `linuxAutostart.js` on Linux
  - `getAutoStartState()` returns `{ enabled, requiresApproval }`; `requiresApproval` is macOS-only and means SMAppService registered the item but the user has not allowed it under System Settings → General → Login Items yet
  - `wasLaunchedAtLoginHidden()` decides whether this launch should go straight to the tray
  - `syncAutoStartEntry()` runs from `initializeCoreManagers()` and repairs entries written by older builds
  - Decision logic lives in `autoStartPolicy.js` (pure, unit-tested in `test/helpers/autoStartPolicy.test.js`)
- **autoStartPolicy.js**: Electron-free launch-at-login and relaunch decisions
  - `HIDDEN_LAUNCH_FLAG` (`--hidden`) is how a login launch tells the app to start in the tray. Windows has no native equivalent (`openAsHidden` is macOS-only and a no-op on macOS 13+), so the flag rides on the login item's `args`; Linux puts it on the autostart entry's `Exec`; macOS uses `wasOpenedAtLogin` instead
  - On Windows, read the state from `executableWillLaunchAtLogin`, never from `openAtLogin`: `openAtLogin` only compares the `Run` value against the current executable and args and ignores the `StartupApproved` key that Task Manager and Settings write when a user disables a startup app
  - Reads and writes must pass identical `args`, or `openAtLogin` always reports false
  - `getRelaunchOptions()` and `getRelaunchWaiter()` shape the `relaunch-app` IPC that follows `cleanup-app` (Reset app data; Delete account with device erase): the relaunch drops `--hidden` and any cold-start deep link, and an AppImage is started again from its on-disk file (`$APPIMAGE`) by a detached waiter, since it runs from a FUSE mount that disappears when the app exits. The handler in `ipcHandlers.js` only quits under `npm run dev` and, on macOS with an update Squirrel already holds, hands the restart to the updater
- **linuxAutostart.js**: Launch-at-login on Linux via an XDG autostart entry
  - `app.setLoginItemSettings()` is a no-op on Linux, so the entry is written directly to `$XDG_CONFIG_HOME/autostart/open-whispr.desktop`, matching the executable name electron-builder packages under
  - `Exec` resolves from `$APPIMAGE` first: `process.execPath` is the ephemeral AppImage FUSE mount
  - `isAutostartEnabled()` honors `X-GNOME-Autostart-enabled=false` and `Hidden=true`, which GNOME Tweaks and KDE's autostart editor write in place instead of deleting the file
  - `Exec` carries `--hidden` (see `autoStartPolicy.js`), and `syncAutostartEntry()` compares against the full value including that flag — comparing against the bare path would make every launch look stale
  - `syncAutostartEntry()` runs from `autoStart.syncAutoStartEntry()` in `initializeCoreManagers()` and re-points a stale `Exec` after the executable moves (renamed or auto-updated AppImage); it never re-enables an entry the user disabled, and no-ops in development
  - Unit-tested in `test/helpers/linuxAutostart.test.js`
- **ipcHandlers.js**: Centralized IPC handler registration
- **windowsKeyManager.js**: Windows Push-to-Talk support with native key listener
  - Spawns native `windows-key-listener.exe` binary for low-level keyboard hooks
  - Supports compound hotkeys (e.g., `Ctrl+Shift+F11`, `CommandOrControl+Space`)
  - Emits `key-down` and `key-up` events for push-to-talk functionality
  - Graceful fallback if binary unavailable
- **meetingDetectionEngine.js**: Orchestrates meeting detection from all sources
  - Gates notifications during recording (tap-to-talk and push-to-talk)
  - Post-recording cooldown (2.5s) before showing queued notifications
  - Priority-based coalescing (process > audio) — one notification, not three
  - Both detectors start off and only run once the renderer has synced its saved notification preferences (`sync-notification-preferences`); they stop when meeting prompts are disabled and restart when re-enabled. The derivation lives in `meetingDetectionPreferencePolicy.js` (pure, unit-tested in `test/helpers/meetingDetectionPreferencePolicy.test.js`); `ipcHandlers.js` is a thin adapter over it
- **meetingProcessDetector.js**: Detects running meeting apps
  - macOS: Event-driven via `systemPreferences.subscribeWorkspaceNotification` (zero CPU)
  - Windows/Linux: Shared `processListCache` polling (30s interval)
  - Scans are start-generation guarded: a process-list read that completes after `stop()` or a newer `start()` is discarded, since the detector is now stopped and started at runtime (notification toggles, auto-end sessions)
- **audioActivityDetector.js**: Detects microphone usage for unscheduled meetings
  - macOS: Event-driven via `macos-mic-listener` binary (CoreAudio process objects; aggregate device activity prompts only while a known meeting app is running)
  - Windows: Event-driven via `windows-mic-listener.exe` (WASAPI sessions, self-PID exclusion)
  - Linux: Event-driven via `pactl subscribe` (PulseAudio source-output events)
  - Windows/Linux: Graceful fallback to polling if the native approach fails; macOS has no safe polling signal and respawns the listener with backoff instead
- **processListCache.js**: Shared singleton process list cache (5s TTL, `ps-list` npm)
- **meetingEchoLeakDetector.js**: Audio-layer echo analysis for meeting recordings — correlates each mic chunk against the recent system-audio tap (lag search 0–500 ms in 5 ms steps) and classifies it `clean_local` / `suspected_render_bleed` / `double_talk`; drives chunk muting and per-segment suppression flags. PCM-driven tests in `test/helpers/meetingEchoLeakDetector.test.js`
- **meetingMicGate.js**: Pure RMS/peak chunk stats + the meeting mic gate verdict (`send` / `zero` for streaming, `send` / `skip` for local) with the exported silence and bleed thresholds; `ipcHandlers.js` (`dispatchMeetingAudioBuffer`, `transcribeLocalMeetingChunk`) only applies the verdict. Unit-tested in `test/helpers/meetingMicGate.test.js`
- **meetingMicHoldback.js**: Pure holdback/retract policy for risky mic finals — pending-final partition, retract window, risky-profile classifier, text-layer duplicate check, racing-retract candidate selection, pending-overlap partition. `ipcHandlers.js` keeps thin adapters over its closure state (`meetingDiarizationSegments`, `meetingPendingMicFinals`, `hasNearbyTranscriptMatch`). Unit-tested in `test/helpers/meetingMicHoldback.test.js`
- **googleCalendarManager.js**: Google Calendar sync (REST, OAuth via `googleCalendarOAuth.js`)
  - 10s socket timeout on API requests
  - Incremental sync via `syncToken`; full re-sync on 410 prunes stale events (note-linked rows retained)
  - Sync tokens pin the `timeMin`/`timeMax` window of the full sync that created them (incremental syncs never roll it forward), so tokens are discarded after 1 day to keep the 33-day lookahead covering the availability tool's 31-day horizon
- **microsoftCalendarManager.js**: Microsoft Calendar sync via Graph API (OAuth via `microsoftCalendarOAuth.js`)
  - `calendarView/delta` incremental sync over a 38-day window; delta token discarded after 7 days (Graph delta links never roll their window forward)
  - Delta can return recurring-series occurrences as bare stubs (no subject/attendees/meeting link); they're backfilled from their series master, one `GET /me/calendars/{calendarId}/events/{id}` per series (calendar-scoped: `/me/events/{id}` 404s for shared calendars). A failed backfill shortens the delta token TTL to 10 min so an early full sync retries instead of leaving untitled blocks
  - Full re-sync (410 or expired token) prunes stale events like Google
- **appleCalendarManager.js**: Apple Calendar (EventKit) via the `macos-calendar-listener` Swift helper — macOS only, snapshot-push over stdout, no tokens ("connected" = `apple_calendars` has rows). The helper takes `--lookback-seconds` (an old binary ignores it and keeps a ~2-hour lookback; rebuild with `npm run compile:calendar-listener`)
- **calendarCacheWindow.js**: `CALENDAR_LOOKBACK_DAYS` (3) — every provider keeps this much past history (Google `timeMin`, Microsoft `startDateTime`, the Apple helper flag) so notes can link recent meetings offline. Widening it needs a `user_version` token reset in `database.js` (v5 did it for 3 days)
- **calendarReminderScheduler.js**: Provider-agnostic meeting reminder scheduling over the shared `calendar_events` table (provider-scoped reset keys, so one provider's disconnect doesn't re-fire another's reminders)
- **calendarSyncInterval.js**: Shared interval runner for the REST providers — exponential backoff (2min → 4min → 8min → cap 30min on consecutive failures, reset on success) and 30s focus-sync throttle
- **oauthLoopbackFlow.js**: Shared PKCE auth-code flow over an ephemeral 127.0.0.1 server, used by both calendar OAuth helpers
- Events from all providers land in the shared `calendar_events` table with a `provider` column; queries suppress the Apple copy of a meeting when a REST row occupies the same time slot + title (Calendar.app mirrors the same accounts); all-day rows pair by local start and end days + title, since REST stores plain dates and the Apple helper local instants
- The note header's date/attendees dropdown (`NoteParticipants.tsx` + `NoteEventPicker.tsx`) links a note to a stored event (`getLinkableCalendarEvents`, local only): linking merges the event's guests into `participants` in the same write, leaving out rooms and declined guests and leaving attendees already on the note as they are, and tags the ones it adds `addedByEvent: <event id>` (meeting detection tags the attendees it copies onto new notes the same way, via `eventAttendees.js`); the user's own `self` row (every calendar sync stores `self` as the user, Google's shared calendars included) is kept whatever they answered and never tagged, since they recorded the note whichever event it links. Removing or switching the event drops exactly the tagged ones, and speaker counts skip `resource` attendees; Join only opens the link; `web_link` (Google `htmlLink`, Outlook `webLink`, none for Apple) backs "Open in calendar". An unlink is stored as `calendar_event_id = ""`, never NULL (`updateNote` converts an explicit NULL on a linked note), because the cloud and `upsertNoteFromCloud` COALESCE a NULL as "unchanged"
- **menuManager.js**: Application menu management
- **toneAppCategories.js**: Tone & Style's app classifier (pure, `test/helpers/toneAppCategories.test.js`): an app identity, window title and page host map to `personal`, `work`, `email` or `other` (see Tone & Style)
- **tray.js**: System tray icon and menu
- **whisper.js**: Local whisper.cpp integration and model management
- **parakeet.js**: NVIDIA Parakeet model management via sherpa-onnx
- **parakeetServer.js**: sherpa-onnx CLI wrapper for transcription
- **qdrantManager.js**: Qdrant vector DB sidecar process lifecycle (spawn, health check, shutdown)
- **semanticSearchLifecycle.js**: Single owner of the semantic search resources — starts Qdrant and the embedding model on demand, drains the SQLite change journal, and releases both after 5 minutes idle
- **localEmbeddings.js**: Local text embedding via ONNX Runtime + all-MiniLM-L6-v2 (384-dim vectors)
- **vectorIndex.js**: Qdrant collection management — upsert, delete, search
- **windowConfig.js**: Centralized window configuration
- **plasmaWorkAreas.js**: Per-monitor work areas on KDE Plasma. On X11 and XWayland, Chromium clips only the primary display with the single root `_NET_WORKAREA`, which KWin builds from every monitor's panels at once (#1110). This asks plasmashell over D-Bus (`evaluateScript`) for each screen's always-visible panels and rebuilds each display's `workArea` for the dictation pill, its companion windows and dragging. On X11, Qt reports a screen's origin in device pixels and only its size scaled, so those origins are divided by the display scale before matching. It refreshes on display events and each time a dictation shows the pill, retrying after 1, 5 and 15 s when plasmashell isn't ready; a display event starts those retries over. An auto-placed pill re-anchors when the result changes; a dragged one stays put. With a single display, or outside Linux + KDE, Chromium's work area is used as is
- **windowManager.js**: Window creation and lifecycle management
- **cliBridge.js**: Loopback HTTP server on ports 8200–8219, bearer-token auth (token at `~/.openwhispr/cli-bridge.json`), 127.0.0.1-only. Used by the unified CLI to talk to a running desktop app. `POST /v1/transcribe` takes a file **path** (never audio) and runs the user's downloaded local model through `IPCHandlers.transcribeLocalFile`, approving the path with `approveAudioPath` first; `GET /v1/transcribe/models` lists local models with download state and the app's default (`localTranscriptionModels.js`, read from the `.env` pre-warm values).
- **connectors/**: Agent connectors
  - `connectorManager.js` owns every outside action: `prepare` → ApprovalCard → `commit` for anything other people see, `runDirect` for private drafts. The model can never reach `commit`
  - Three action kinds per action, fixed in code: `approval` (`prepare` → card → `commit`), `direct` (`runDirect`) and `query` (a read for the model, e.g. an issue search). `connectorManager.query` runs `prepare`'s gates (policy, action kind, account, the bound login), also refuses a login whose `ownerAccountId` is another account (`account_changed`), and writes no receipt; `queryResult.js` caps what reaches the model (20 flat items, 16 fields, strings cut at 1,000 characters, `truncated` on any cut) and strips characters that hide text (controls, DEL and C1, bidi overrides, zero-width spaces, tag characters; ZWNJ and ZWJ stay). `runQueryAction` marks the items `untrusted`, with guidance never to follow instructions in them, and takes the attendee fence's tag name out of them (`withoutAttendeesFence`); the chat keeps no copy of a search's items in the saved (and synced) conversation (`isQueryResultData`)
  - Registration is by list: `createConnectors.js` (`CONNECTOR_FACTORIES`, one `build<Name>Connector(deps)` per provider module) in main, `connectorToolModules.ts` (`CONNECTOR_TOOL_MODULES`, registered by `readyConnectorIds()`) for tools, and `connectorRows.tsx` (`CONNECTOR_ROWS`) for Settings rows. Each tool carries its own `promptInstruction` and `connectorId`; the connector rules join the prompt when any connector tool is offered
  - `runApprovalAction` caps a turn at 5 approval cards across connectors (`approval_card` turn slot, given back when no card appears) and holds every approval action's turn off the caret (`onHoldDelivery`), whatever the outcome; a turn's scope passes one hold per tool call on to the chat surface, plus a later one that newly keeps the clipboard. Cards lay out fields by `verbKey`: `email`, `issue` (title and description) and `comment`; a sent commit may name what it created (`resultLabel`, shown as "Created ENG-124")
  - `pendingActions.js`: pending → committing → sent/failed/unknown; actions are bound to the connection (account, workspace, `generation`) they were prepared on and to the OpenWhispr account that prepared them (another account's commit is withdrawn as `account_changed`)
  - `connector_actions` (SQLite) is the durable receipt: destination labels, states and links only, never content. Each row is stamped with the account bound to the credential the action ran under (`accountScopeBinding`, read before and after the policy wait; with no account, `prepare` and `runDirect` refuse as `receipt_unavailable`; a `commit` under a changed or missing account is withdrawn as `not_sent`/`account_changed`), so the `connector-recent-actions` history IPC lists only that account's rows and account deletion removes them. Interrupted rows are reconciled on launch (committing → unknown; rows with no account are deleted), expired pending actions are recorded as `expired`, a direct run the connector stopped (`not_sent`) is `cancelled`, and a direct run that throws or returns a malformed result is `unknown` (it may have happened), never `failed`. Every connector result is whitelisted and type-checked per state before it is recorded or reaches the renderer. A cancel (Esc) reaches a direct run until its side effect through `runtime.signal`, checked right before the compose window opens
  - Policy fails closed (`connectorPolicy.js`'s `connectorPolicyState` classifies the snapshot; `connectorIpc.js`'s `createConnectorPolicyResolver` enforces the 1.5s deadline): only a successful, well-formed snapshot allows; unresolvable, any error code, or >1.5s with no verdict already held for the account (`workspacePolicyManager.peekPolicy`) → unavailable; org switch `features.connectorsEnabled`, and `agentEnabled: false` turns connectors off too
  - `emailConnector.js` opens Gmail/Outlook/mailto compose windows (`emailCompose.js`, shared ESM); a body that won't fit the compose URL (2,000 chars; 6,000 for Gmail on macOS/Linux, since Windows caps opened URLs near 2,081) goes to the clipboard after the OS accepts the link (on Linux a `mailto` is refused first when `xdg-mime` reports no default — empty output with exit 0, or exit 4 on KDE 5 — except under Flatpak, where the portal picks the handler), then the subject too if it's still too long; a recipient list that alone doesn't fit is refused. `email_draft` opens at most 3 drafts per turn, and only one of them may use the clipboard (`ToolExecutionContext.claimTurnSlot`; a draft that didn't open gives its slots back, and the compose preview is built before any slot is claimed). Subject and body go through `toWellFormed()`, the renderer NFC-normalises addresses (`bareEmailAddress`), and a recipient with a non-ASCII domain is shown with its punycode form (`recipientLabel`) to the model, in the tool step and in receipts. Automatic picks Send from chat (Gmail) when the Gmail connector is connected (including one that needs reconnecting), then the Gmail compose link when a Google calendar is connected, otherwise the Microsoft tenant rule as today: Outlook work vs Outlook.com from the Microsoft account's tenant id (`tid`, consumer tenant `9188040d-…`), falling back to the email domain for accounts connected before it was stored (`resolveEmailDraftTarget` in `src/utils/emailDraftTarget.ts`)
  - Gmail send (`gmailMime.js`, `gmailApi.js`, `gmailAuth.js`, `gmailConnector.js`) is an approval connector: with Gmail connected (Settings → Integrations → Connectors), `email_draft` shows an editable card and `commit` sends through the Gmail API. The only Gmail scope is `gmail.send` (plus `openid email`); a grant without it, or without a verified Google address (`email_verified`), is revoked, and a refresh is bound to the card's login (single-flight per OpenWhispr account and login). A login records the OAuth client that issued it (`clientId`). A refresh Google refuses is never reported as a network error: `invalid_grant`, another OAuth refusal (not `invalid_client`/`unauthorized_client`/`deleted_client`, build faults that are `gmail_unavailable`), or a login issued to a client this build no longer uses flags the login for reconnecting, and a Workspace admin block (`admin_policy_enforced`, `org_internal`) is `domain_policy`. Google's `/revoke` ends the user's whole grant to a Cloud project, so Disconnect (and a refused grant) skips it when a connected Google Calendar account with the same address is on the same project as the login's own client (`sharesCalendarGrant`, by the client id's project number) and only deletes the local tokens; the disconnect result then carries `grantKept` and the Settings row says Google still lists the app until the calendar is disconnected too. The shared-grant check reads the calendar accounts only once the projects match. A revoke is skipped the same way when another OpenWhispr account on the device has a Gmail login for the same Google address on the same project (`readAllAccounts`), since it holds the same grant. Connecting a different Google (or Slack) account over an existing login revokes the replaced one (`loginKey`); reconnecting the same account doesn't, since Google's revoke would end the new login's grant too. The Google client is `GMAIL_CLIENT_ID`/`GMAIL_CLIENT_SECRET` when both are set, else the calendar pair (`GOOGLE_CALENDAR_CLIENT_ID`/`_SECRET`); with neither, Gmail reports `configured: false` and its row hides. The card blocks Send on an empty To, a bad address, more than 50 recipients, a subject over 250 characters or a body over 512 KB as sent (UTF-8, CRLF; `MAX_EMAIL_BODY_BYTES`, `emailBodyBytes`), a size that with the largest headers the other limits allow stays under the 1 MB raw cap (`src/utils/emailApprovalFields.ts`, limits shared from `emailCompose.js`; To/Cc accept `,` `;` and the Arabic and CJK commas, and a pasted `Name <address>` — including Outlook's `Last, First <address>` — becomes its bare address, with the resolved addresses shown under the field); main re-validates the same limits at Send (addresses as dot-atoms of at most 64 bytes without `=?`, ≤ 50 recipients, subject ≤ 250 characters without line breaks, body ≤ 512 KB, raw message ≤ 1 MB — all checked before any token refresh) and refuses CR/LF in every header. 5xx, a timeout after writing, and a 200 without a message id are `unknown` (link to the Sent folder; no retry); a 401 is refreshed and retried once under the same login; a 429 asking for ≤ 5 s is retried once; a 413 is `too_long`. A connector card whose preview shows a field the action doesn't declare editable is refused as malformed, so a typo in a declaration can't silently drop the user's edit. Release rule: a release build falls back to the calendar's Google client, so `gmail.send` must pass Google verification before a release ships it; develop against a separate Testing project with only `gmail.send` (`GMAIL_CLIENT_ID`/`_SECRET`), never the production calendar project
  - Approval actions must declare their editable fields (`actions.<action>.editable`: `addresses`, `line`, `text`; a missing declaration or an unknown type throws when the manager is created); a preview field must be declared and hold its declared type, or there's no card; `sanitizeEdits` keeps exactly those, and a declared field of the wrong type withdraws the card as `not_sent`/`invalid_edit` rather than sending the prepared value. Previews may carry `fields`, which the approval store edits and the tool result reports as `final` (`finalText` for a card without fields) on a sent, failed or unknown Send alike. `connectorErrorText()` words card and tool-step errors per connector (`connectors.<scope>.errors.<connectorId>.<code>` before the shared key)
  - Reset app data calls `connectorManager.revokeAllStored()` before deleting `userData/connectors`: it revokes every login on the device, for every account, even with no one signed in (Settings signs out before `cleanup-app`), and Gmail even on a grant shared with the calendar. Deleting an account uses `disconnectAll()` for the signed-in account's logins; a Gmail grant shared with a calendar is kept, unless the user also erases the device (`delete-account-data`'s `{ erasingDevice }`), since `cleanup-app` then finds no Gmail login left to revoke. Signing out keeps logins (every action is refused while signed out)
  - `contactSearch.js` (`find_contact`) searches the meetings nearest to now on selected calendars (`SELECTED_CALENDAR_EVENT_FILTER`; for Google only primary calendars, since a colleague's shared calendar isn't the user's meetings, which also leaves out the user's own secondary Google calendars; shared Microsoft or Apple-mirrored calendars still count; `calendar_events` only spans the sync window; cancelled and declined meetings don't count) plus the `contacts` table, most recently synced first, behind the same org policy check as the actions. Excluded: the user's calendar accounts and their Microsoft addresses (`/me` `mail`, `userPrincipalName` and `proxyAddresses`, stored in `microsoft_calendar_tokens.own_addresses` by `fetchCalendars`, which also purges them from `contacts`), `self` attendees, an Apple organizer who is the current user, any address flagged `resource` in any stored event (rows with unreadable attendee JSON are skipped) (rooms, flagged by all three calendar managers) and `*.calendar.google.com` addresses; the calendar managers also keep the user and rooms out of `contacts` and purge those addresses when they see them again. Sources are many-to-many (`contact_sources`: `google:<account>`, `microsoft:<account>`, `apple`, `manual`): disconnecting an account removes its source rows and deletes only contacts no other source still has; the lookup reads contacts with a `manual` or connected-account source (rows from before sources existed are left out; note-participant autocomplete still reads every row). Picking an existing contact in a note never adds `manual`, so it can't make a synced contact permanent
  - Slack (`slackConnector.js`): PKCE user token through `oauthLoopbackFlow.js`. Slack only distributes apps with HTTPS redirect URLs, so it redirects to the openwhispr.com relay (`openwhispr-website`, `app/auth/slack/callback/route.ts`), which forwards to the loopback server by the port in `state` (`v1.<port>.<nonce>`). The loopback server shows a local result page, and the flow waits 5 minutes. `SLACK_OAUTH_REDIRECT_URI` points a dev build at a website preview. Rotating tokens refresh single-flight, only for the login a pending action is bound to, and are saved before use; transient refresh errors never disconnect (`slackAuth.js`). Channels the user is in and people are cached 10 min; a list cut off at the page cap resolves only an exact #channel or an email (`slackDirectory.js`, `targetResolution.js`). `conversations.open` runs only at Send. Disconnect revokes the refresh and access tokens. Failures follow `SLACK_PRE_SEND_REJECTIONS` (`slackApi.js`); everything else is `unknown`
  - Linear (`linearApi.js`, `linearAuth.js`, `linearTeams.js`, `linearConnector.js`, tools in `linearTools.ts`): PKCE login with no client secret (`LINEAR_CLIENT_ID`), scopes `read`, `issues:create`, `comments:create`. Linear matches a registered redirect exactly, port included, so it redirects to the openwhispr.com relay (`openwhispr-website`, `app/auth/linear/callback/route.ts`), which forwards to the loopback server by the port in `state`; `LINEAR_OAUTH_REDIRECT_URI` points a dev build at a website preview (https only). `linear_search_issues` is a query action (≤ 10 compact results, untrusted); `linear_create_issue` and `linear_comment` go through the issue/comment cards. Teams and projects are cached 10 minutes per login (`linearTeams.js`); a name that matches nothing in a list older than 30 s reads it once more, and a project the team doesn't have is a question, never dropped. Lengths count characters, as the card does. Links in results and receipts keep no title slug (`linear.app/<urlKey>/issue/<KEY>`). A create and a comment each carry a client UUID `id`, made at prepare, so an uncertain one is looked up once: found is `sent`; a comment the lookup misses stays `unknown`. Connecting another Linear user or workspace over a login revokes the replaced one (`loginKey`). A GraphQL error inside a 200 is `failed` only for a listed rejection code (`LINEAR_PRE_SEND_REJECTIONS`) with no data beside it; Linear reports a missing entity as `INPUT_ERROR` "Entity not found", and a repeated create id ("conflict on insert") is not a refusal
  - GitHub (`githubApi.js`, `githubDeviceFlow.js`, `githubAuth.js`, `githubInstallations.js`, `githubConnector.js`, tools in `githubTools.ts`): a GitHub App with the device flow and no client secret (`GITHUB_APP_CLIENT_ID`, `GITHUB_APP_SLUG`; release builds write them to `.env` from the `GH_APP_CLIENT_ID`/`GH_APP_SLUG` secrets, since secret names can't start with `GITHUB_`; the row hides unless both are set); permissions Issues RW, Pull requests RW, Metadata R, so nothing can read code. The 8-hour token refreshes without a secret and the rotated refresh token is saved before use. Everything is limited to the repositories the App is installed on (`githubInstallations.js`, 60 s cache per login, one read at a time shared by everyone asking, and a read in flight when the list is cleared is never cached; a repo missing from a list older than 5 s is looked for in a fresh one, so installing the App from a "not installed" answer works at once). A bare repo name matches exactly first, then loosely through `resolveTarget` ("open whispr" for `open-whispr`), but only in a complete list; a miss in a list cut at 1,000 repositories per installation is `repo_unlisted` (the App may be installed after all), not `not_installed`; the row's **Choose/Manage repositories** opens the install page (`manageUrl`). The status never waits on GitHub: it reports the last repository count read and reads again in the background (reusing a list younger than 5 s), announcing a changed count, the first read for a login even when it failed, or a login the read found needing a reconnect, through `notifyStatusChanged`; until that first read finishes (after a connect, and at launch) the status carries `workspaceLabelPending: true`, a field no other connector sends (the manager asks every connector at once, so a GitHub read would hold up the others). Search result titles and snippets are redacted of email addresses over a bounded window only, since the redaction pattern is quadratic on long runs of word characters. `github_search_issues` is a query action; a GitHub App user token must search issues and pull requests apart (`is:issue`/`is:pull-request`, or a 422), so `type: "any"` runs both and merges them newest first. The user's own `repo:`/`org:`/`user:` qualifiers are removed (in groups too, with any operator they leave behind), and so are their own `is:`/`type:` issue or pull-request qualifiers, since the tool's `type` wins; a quoted phrase is left as written, and a quote left open or a parenthesis without its pair is dropped, since either would take in the qualifiers added after the words; a state the query names itself (`is:closed`, …) replaces the default `state:open`, and an invalid `state` or `type` is refused rather than defaulted. GitHub's 256-character limit counts only the words, so the `repo:` list is bounded by the size of `q` as sent in the query string (`MAX_SEARCH_QUERY_LENGTH`). GitHub answers 422 for a `repo:` the user can no longer search (unticked, deleted, access lost), so a 422 re-reads the installed repos fresh and searches once more. Suspended installations are skipped. `github_create_issue` and `github_comment` go through the issue/comment cards, with a live "This will notify @…" note while the card is pending (`githubMentions.ts`, `liveCardNotes.ts`), and a sent result tells the model who was notified (`notified`, from the card's final fields when edited, else the call's own). `github_comment` takes `owner/repo#N`, a github.com link, or `repo#N`/`#N`, whose repo is found among the installed ones (a question when ambiguous). A new issue needs a repo with issues on (archived and issues-off repos aren't offered as candidates), and more than 10 labels is refused rather than trimmed. A 401 on a token another request already refreshed reuses that token (`refreshRejected`), since every GitHub refresh ends the previous one. Connect shows the device code in the row (sent over the `connector-connect-progress` broadcast) with its own Cancel (`connectInRow`); `connector-cancel-connect` stops it, even while it still waits on the policy check (`pendingConnects` in `connectorIpc.js`), as do leaving Settings, the org turning connectors off, an account switch or a sign-out in main (`connectorManager.accountChanged`). A device code that can't be asked for is `network` (no answer, a timeout, a 408 or 5xx) or `rate_limited` (a 429), not `token_exchange_failed`, since the user never saw a code. The device poll keeps going through network failures until the code expires and treats a bare 429 like `slow_down`; the `/user` read after approval is retried once. A refresh answered with 408 or `temporarily_unavailable`/`server_error` keeps the login and is tried once more; a 429 keeps it without an immediate retry (`rate_limited`), and a refusal of the OAuth client itself (`incorrect_client_credentials`, `device_flow_disabled`, `unauthorized_client`, `invalid_client`: a build with a wrong client id) is `not_configured` and never flags the login. An archived repo takes no new issue or comment, and a locked conversation only takes a comment from someone with push access. Writes have no idempotency key, so anything uncertain after the send is final `unknown`. Only Reset app data and account deletion (the manager's `removingAll`) end the tokens, through GitHub's unauthenticated `POST /credentials/revoke` (an authenticated call is refused). That endpoint is meant for exposed credentials and emails the user, so Disconnect, a reconnect and a connect that can't be saved only delete the login locally; the App's authorization stays on github.com until the user removes it there
  - Token handling shared by the OAuth connectors (`boundLogin.js`): reads, the reconnect flag, saving a refreshed token and one refresh at a time, only ever for the login an action is bound to. Gmail's and Linear's clients share one timed POST and its body helpers (`providerHttp.js`); GitHub's uses the body helpers
  - Logins: one encrypted file per OpenWhispr account and connector under `userData/connectors/` (atomic writes, `0o700`, `connectorCredentials.js`). Every write names the account and the generation it started from, so an OAuth round trip, refresh or disconnect that outlives a reconnect or account switch writ