# Technical Documentation: clidey/whodb > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/clidey/whodb) ### Overview (`README.md`)

WhoDB [](https://mcptoplist.com/server/glama%2Fclidey%2Fwhodb) ### A lightweight, self-hosted workspace for your databases [](https://github.com/clidey/whodb/actions/workflows/release-ce.yml) [](https://github.com/clidey/whodb/releases) [](https://github.com/clidey/whodb/blob/main/LICENSE) [](https://github.com/clidey/whodb/stargazers) ## Available on [](https://hub.docker.com/r/clidey/whodb) [](https://apps.microsoft.com/detail/9pftx5bv4ds6) [](https://apps.apple.com/app/whodb/id6754566536) [](https://snapcraft.io/whodb) [](https://github.com/clidey/whodb/blob/main/cli/README.md) [Quick start](https://github.com/clidey/whodb/blob/main/README.md#quick-start) · [Documentation](https://docs.whodb.com/) · [Live demo](https://demo.whodb.com/) · [Community](https://github.com/clidey/whodb/discussions)

WhoDB showing a database table

WhoDB gives you one place to explore your databases, edit data, run queries, and understand how a schema fits together. It runs in the browser, is easy to self-host, and is available as a desktop app or terminal CLI too. Use it when you want to inspect a local database, help a teammate explore unfamiliar data, or work without installing a heavyweight database client. AI features are optional: connect Ollama, OpenAI, Anthropic, LM Studio, or another OpenAI-compatible provider if you want to ask questions in plain English. ## Quick start Run WhoDB with Docker: ```bash docker run --rm -it -p 8080:8080 clidey/whodb ``` Then open [http://localhost:8080](http://localhost:8080) and enter your database connection details. Want to look around first? Try the [live demo](https://demo.whodb.com/) or watch the [video walkthrough](https://youtu.be/hnAQcYYzcLo). ## What you can do - **Browse and edit data** in a spreadsheet-style grid with sorting, filtering, pagination, and inline editing. - **Understand a schema visually** with an interactive graph of tables and relationships. - **Work through queries in a scratchpad** with multiple cells, autocomplete, history, and results kept alongside each query. - **Move data in and out** with imports, exports, and mock-data generation for development and testing. - **Ask questions in plain English** using a local or hosted AI provider that you choose. - **Work from your terminal** through the WhoDB CLI and its MCP server.


_{Browse, filter, and edit data}	_{Follow relationships through the schema graph}

### Supported databases WhoDB Community supports: - PostgreSQL, CockroachDB, YugabyteDB, and QuestDB - MySQL, MariaDB, and TiDB - SQLite and DuckDB - MongoDB and FerretDB - Redis, Valkey, and Dragonfly - Elasticsearch and OpenSearch - ClickHouse and Memcached Support varies by database because not every system has the same concepts or capabilities. The connection screen shows the options available for each source. ## Installation options ### Docker with persistent sessions The one-line Docker command is ideal for trying WhoDB. To keep encrypted login sessions when the container is replaced, first generate a key and save it somewhere secure: ```bash openssl rand -hex 32 ``` Then mount `/data` and reuse that key whenever you start the container: ```bash docker run -it -p 8080:8080 \ -v whodb-data:/data \ -e WHODB_ENCRYPTION_KEY=your_saved_64_character_hex_key \ clidey/whodb ``` Keep that key somewhere safe. Changing it invalidates existing sessions. If WhoDB is served through an HTTPS reverse proxy, also set `WHODB_SECURE=true` so browsers only send the session cookie over HTTPS. See the [documentation](https://docs.whodb.com/) for Docker Compose, connection profiles, SSL, AI providers, and other configuration options. ### Desktop - [macOS](https://apps.apple.com/app/whodb/id6754566536) - [Windows](https://apps.microsoft.com/detail/9pftx5bv4ds6) - [Snap](https://snapcraft.io/whodb) ### CLI and MCP server The CLI includes an interactive terminal UI and an MCP server for AI tools: ```bash # macOS and Linux curl -fsSL https://raw.githubusercontent.com/clidey/whodb/main/cli/install/install.sh | bash # or install with npm npm install @clidey/whodb ``` ```bash whodb # open the terminal UI whodb mcp serve # start the MCP server ``` See the [CLI guide](https://github.com/clidey/whodb/blob/main/cli/README.md) for connection examples and the full command reference. ## AI providers You can add a hosted AI provider directly from WhoDB—no backend configuration or restart is required. Open Chat, choose **Add Provider** from the provider menu, then select OpenAI, Anthropic, or Gemini and enter your API key. WhoDB will fetch the available models from that provider. Ollama and LM Studio are available as local options in WhoDB Community. By default, the backend looks for Ollama at `localhost:11434` and LM Studio at `localhost:1234/v1`, with local addresses adjusted automatically for Docker and WSL. Use `WHODB_OLLAMA_HOST`, `WHODB_OLLAMA_PORT`, or `WHODB_LMSTUDIO_BASE_URL` only when those defaults are not suitable for your setup. ### Optional backend provider configuration Environment variables let deployment administrators declare providers when the server starts. This is useful for preconfiguring OpenAI or Anthropic, changing provider endpoints, or adding an OpenAI-compatible service for everyone using that deployment: - `WHODB_OPENAI_*` for OpenAI - `WHODB_ANTHROPIC_*` for Anthropic - `WHODB_OLLAMA_*` for Ollama connection settings - `WHODB_LMSTUDIO_*` for LM Studio connection settings - `WHODB_AI_GENERIC__*` for OpenAI-compatible providers See the [installation guide](https://github.com/clidey/whodb/blob/main/docs/installation.mdx#ai-providers) for the complete environment variable list and the [AI provider guide](https://github.com/clidey/whodb/blob/main/docs/ai/setup-providers.mdx) for setup examples. ## WhoDB Community and WhoDB Platform This repository contains **WhoDB Community**, the Apache-2.0-licensed database workspace described above. It is free to self-host and is the best place to start if you want to explore and work with databases. **WhoDB Platform** is the commercial, self-hosted edition for organizations that need shared projects, more data sources, SSO, fine-grained access controls, audit logs, pipelines, reporting, and internal apps. You can read the [WhoDB overview](https://whodb.com/) or compare plans on the [pricing page](https://whodb.com/pricing/). ## Development WhoDB has a Go backend and a React/TypeScript frontend. For local development, run them in separate terminals. Requirements: - Go - Node.js and pnpm Start the backend: ```bash cd core go run ./cmd/whodb ``` Start the frontend: ```bash cd frontend pnpm install pnpm start ``` The frontend opens at [http://localhost:3000](http://localhost:3000) and talks to the backend on port `8080`. See the [CE build and run guide](https://github.com/clidey/whodb/blob/main/.agents/docs/build-and-run.md) for generation and build commands. ## Contributing Bug reports, feature ideas, documentation improvements, and code contributions are all welcome. - [Open an issue](https://github.com/clidey/whodb/issues) for a bug or concrete request. - [Start a discussion](https://github.com/clidey/whodb/discussions) for questions and ideas. - Read [CONTRIBUTING.md](https://github.com/clidey/whodb/blob/main/CONTRIBUTING.md) before sending a pull request. ## More screenshots

See WhoDB in action

### Query scratchpad Writing queries in the WhoDB scratchpad

### Add and edit records Adding a row in WhoDB

### Build filters visually Building WHERE conditions in WhoDB

### Export data Exporting data from WhoDB

## Support - [Documentation](https://docs.whodb.com/) - [GitHub Issues](https://github.com/clidey/whodb/issues) - [support@clidey.com](mailto:support@clidey.com) WhoDB is licensed under the [Apache License 2.0](https://github.com/clidey/whodb/blob/main/LICENSE).

If WhoDB saves you time, consider [giving the project a star](https://github.com/clidey/whodb/stargazers).

--- ### Overview (`.agents/README.md`) # Agent Instructions Index This directory contains shared agent guidance for WhoDB. It is intentionally tool-neutral so Codex, Claude Code, opencode, Pi, and other coding agents can use the same source of truth. Start with `../AGENTS.md`. Use this index only to find the one or two deeper references that match the task. ## Directory Layout | Path | Purpose | |------|---------| | `.agents/docs/` | Detailed runbooks and reference material for specific systems or policies. | | `.agents/rules/` | Domain-specific rules for backend, frontend, GraphQL, localization, and E2E work. | | `.agents/workflows/` | Step-by-step procedures for recurring implementation, review, handoff, and research tasks. | If `ee/` is present, also read `ee/AGENTS.md` for Enterprise Edition boundaries and use `ee/.agents/` for EE-specific rules, workflows, and docs. ## Common Workflows | Task | Read | |------|------| | Add a database plugin | `.agents/workflows/new-plugin.md` | | Add a GraphQL field end-to-end | `.agents/workflows/new-graphql-field.md` | | Add a frontend page | `.agents/workflows/new-frontend-page.md` | | Add translation keys | `.agents/workflows/add-translation.md` | | Add CLI behavior | `.agents/workflows/cli-feature.md` | | Prepare or consume a handoff | `.agents/workflows/task-handoff.md` | | Prove claims about external behavior | `.agents/workflows/research-proof.md` | | Verify before finishing | `.agents/workflows/review-checklist.md` | ## Usage Rules - Do not read the whole `.agents` tree by default. Start with `AGENTS.md`, then open only the relevant rule, workflow, or doc. - Keep tool-specific instructions in adapter files such as `CLAUDE.md`; keep shared behavior here. - Do not move user-facing product documentation into this directory. Public docs belong under `docs/`. - When adding new agent guidance, prefer a focused workflow or rule over expanding always-loaded instructions. - After changing agent instructions, run `dev/check-agent-instructions.sh`. --- ### Overview (`cli/README.md`) # WhoDB CLI An interactive, production-ready command-line interface for WhoDB with a Claude Code-like experience. ## Features - **Interactive TUI** - Terminal UI built with Bubble Tea - **Multi-Database Support** - PostgreSQL, MySQL/MariaDB, SQLite, MongoDB, Redis, ClickHouse, ElasticSearch - **Table Browser** - Navigate schemas and tables with visual grid layout - **WHERE Builder** - Build AND-based filters for table browsing - **SQL Editor** - Multi-line editor with schema-aware autocomplete - **AI Chat** - Optional AI-assisted querying with consent gate (requires configured provider) - **Responsive Data Viewer** - Paginated results with horizontal column scrolling - **Column Selection** - Choose which columns are visible in results - **Export Capabilities** - Export to CSV and Excel formats - **Schema Diff** - Compare schema metadata across environments in the CLI and TUI - **Cloud Discovery** - Inspect configured cloud providers and discovered cloud-managed resources from the CLI - **Discovered Connect** - Prefill connect/save flows directly from discovered cloud resources - **ERD Graph Output** - Inspect backend graph metadata from the CLI or TUI - **Explain Plans** - Run database-native `EXPLAIN` from the CLI or TUI - **Backend Query Suggestions** - Shared onboarding suggestions in the CLI and TUI editor - **Bookmarks and Profiles** - Shared saved queries and connection profiles across CLI and TUI - **Workspace Restore** - Resume your last reconnectable TUI session on startup - **Query History** - Persistent history with re-execution - **Shell Completion** - Bash/Zsh/Fish install plus PowerShell script generation - **Programmatic Mode** - JSON/NDJSON/CSV/plain output plus streamed query/export paths for scripting and automation - **Hosted Platform Commands** - Browser login, workspace selection, manifest inspection, and hosted source management - **Agent Manifest** - Machine-readable command, source, MCP, workflow, and safety metadata - **Database Doctor** - Redacted connection and metadata diagnostics for support and AI agents - **Built-in Runbooks** - Repeatable workflows for connection checks, schema audits, and schema diffs - **Assistant Integration Installer** - Install bundled WhoDB skills, agents, MCP configs, and rule files into local assistant directories - **MCP Server** - Model Context Protocol server for AI assistants (Claude, Cursor, etc.) ## Installation ### Native Install (Recommended) **macOS / Linux:** ```bash curl -fsSL https://raw.githubusercontent.com/clidey/whodb/main/cli/install/install.sh | bash ``` **Windows (PowerShell):** ```powershell irm https://raw.githubusercontent.com/clidey/whodb/main/cli/install/install.ps1 | iex ``` The native installer: - Detects your OS and architecture - Downloads the correct binary from GitHub releases - Installs to `~/.local/bin` (macOS/Linux) or `%LOCALAPPDATA%\WhoDB\bin` (Windows) - Adds to PATH if needed > **Linux note:** if the WhoDB desktop snap is installed, it also provides a `whodb` command at `/snap/bin/whodb`. Which one runs depends on your PATH order. > **Renamed from `whodb-cli`:** the command used to be `whodb-cli`. The installer, Homebrew formula, npm package, and Docker image all still provide a deprecated `whodb-cli` alias that prints a warning and forwards to `whodb`. The alias will be removed in a future release — update any scripts or configs to use `whodb`. To install a specific version: ```bash # macOS/Linux curl -fsSL https://raw.githubusercontent.com/clidey/whodb/main/cli/install/install.sh | bash -s v0.62.0 # Windows $env:WHODB_VERSION = "v0.62.0"; irm https://raw.githubusercontent.com/clidey/whodb/main/cli/install/install.ps1 | iex ``` ### Homebrew (macOS/Linux) ```bash brew install whodb-cli ``` The formula is named `whodb-cli` (the `whodb` name is used by the desktop app cask), but it installs the `whodb` command. ### npm ```bash npm install @clidey/whodb ``` Or with npx (no install): ```bash npx @clidey/whodb ``` ### From Source Requires Go 1.21+: ```bash git clone https://github.com/clidey/whodb.git cd whodb/cli go build -o whodb . ``` Or using the Makefile: ```bash cd cli make build make install # installs to /usr/local/bin ``` ### Using Docker ```bash # Build the Docker image (from repo root) docker build -t whodb-cli:latest -f cli/Dockerfile . # Or pull pre-built docker pull clidey/whodb-cli:latest ``` ### Verify Installation ```bash whodb --version whodb --help ``` ## Quick Start For hosted WhoDB platform login and source management, see [the hosted platform CLI guide](../.agents/docs/hosted-platform-cli.md). For task-oriented MCP catalogs, unattended service-account jobs and security ledger access, see [platform operations](docs/platform-operations.md). ### 1. Connect to a Database If you omit required flags, the interactive connection form opens: ```bash whodb connect ``` #### PostgreSQL ```bash whodb connect \ --type postgres \ --host localhost \ --port 5432 \ --user postgres \ --database mydb \ --name my-postgres ``` #### PostgreSQL (non-interactive password) ```bash printf "%s\n" "$PGPASSWORD" | whodb connect \ --type postgres \ --host localhost \ --port 5432 \ --user postgres \ --database mydb \ --name my-postgres \ --password ``` #### PostgreSQL with SSL ```bash whodb connect \ --type postgres \ --host localhost \ --port 5432 \ --user postgres \ --database mydb \ --ssl-mode verify-ca \ --ssl-ca ./ca.pem ``` #### Discovered Cloud Resource ```bash # Open the TUI form prefilled from discovery whodb connect --discovered aws-prod-us-west-2/prod-db # One-shot connect when you already know the missing credentials whodb connect \ --discovered aws-prod-us-west-2/prod-db \ --user postgres \ --database app ``` #### MySQL ```bash whodb connect \ --type mysql \ --host localhost \ --port 3306 \ --user root \ --database mydb \ --name my-mysql ``` #### SQLite ```bash whodb connect \ --type sqlite \ --user sqlite \ --database /path/to/database.db \ --name my-sqlite ``` #### MongoDB ```bash whodb connect \ --type mongodb \ --host localhost \ --port 27017 \ --user admin \ --database mydb \ --name my-mongo ``` ### 1b. Use Environment Profiles Commands that accept `--connection` can also use environment profiles, for example `WHODB_POSTGRES='[{"alias":"prod","host":"localhost","user":"user","password":"pass","database":"mydb","port":"5432"}]'` or `WHODB_MYSQL_1='{"alias":"dev","host":"localhost","user":"user","password":"pass","database":"devdb","port":"3306"}'`. Each object supports `alias` (connection name), `host`, `user`, `password`, `database`, `port`, and optional `config` for advanced settings. `port` stays at the root level; the CLI also forwards it as the `Port` advanced key when building plugin credentials, so you do not need to include `Port` in `config`. Advanced `config` keys are plugin-specific; see `core/src/plugins/*/db.go` for the keys that are read. ```bash # Array format (multiple profiles for a database type) export WHODB_POSTGRES='[{"alias":"prod","host":"localhost","user":"user","password":"pass","database":"mydb","port":"5432"}]' # Numbered format (one profile per variable) export WHODB_MYSQL_1='{"alias":"dev","host":"localhost","user":"user","password":"pass","database":"devdb","port":"3306"}' ``` ### 2. Start Interactive Mode ```bash # Start the database TUI whodb --tui ``` ### 3. Execute a Quick Query ```bash whodb query "SELECT * FROM users LIMIT 10" --connection my-postgres ``` ## Commands ### Root Command (Interactive Mode) Running `whodb` without arguments starts the interactive TUI. ```bash whodb [flags] ``` Flags: - `--debug`: Enable debug mode - `--no-color`: Disable colored output ### connect Connect to a database and optionally save the connection. If required flags are missing, the interactive connection form opens. ```bash whodb connect [flags] ``` Flags: - `--type`: Database type (postgres, mysql, sqlite, mongodb, redis, clickhouse, elasticsearch, mariadb) - `--host`: Database host (default: localhost) - `--port`: Database port (default varies by type) - `--user`: Username - `--database`: Database name - `--schema`: Preferred schema (optional) - `--name`: Connection name (saves for later use) - `--password`: Read password from stdin when not using a TTY (pipe a single line) - `--ssl-mode`: SSL mode for the selected database type - `--ssl-ca`: Path to a CA certificate PEM file - `--ssl-cert`: Path to a client certificate PEM file - `--ssl-key`: Path to a client private key PEM file - `--ssl-server-name`: Override server name used for SSL hostname verification On a TTY, you will be prompted for the password with input hidden. ### query Execute a SQL query directly. Use `-` to read SQL from stdin. ```bash whodb query "SQL" [flags] ``` Flags: - `--connection, -c`: Connection name to use (optional; if omitted, the first available connection is used) - `--format, -f`: Output format: `auto`, `table`, `plain`, `json`, `ndjson`, `csv` - `--stream`: Stream result rows incrementally (supported for `plain`, `json`, `ndjson`, and `csv`) - `--quiet, -q`: Suppress informational messages `auto` uses table output for terminals and plain output for pipes. `ndjson` writes one JSON object per result row. ### suggestions Show backend-generated query suggestions for a connection. ```bash whodb suggestions --connection my-postgres whodb suggestions --connection my-postgres --format json ``` Flags: - `--connection, -c`: Connection name to use - `--schema, -s`: Schema to use for suggestion generation - `--format, -f`: Output format: `table`, `plain`, `json`, `ndjson`, `csv` - `--quiet, -q`: Suppress informational messages ### completion Generate or install shell completion scripts. ```bash # Show help whodb completion # Print completion script to stdout whodb completion bash whodb completion zsh whodb completion fish whodb completion powershell # Install completion (auto-detects shell) whodb completion install # Install for specific shell whodb completion install bash # Uninstall completion whodb completion uninstall ``` Install paths (bash/zsh rc files updated automatically): - Bash: `~/.local/share/bash-completion/completions/whodb` - Zsh: `~/.zsh/completions/_whodb` - Fish: `~/.config/fish/completions/whodb.fish` - PowerShell: Manual install (see `whodb completion powershell`) ## Programmatic Commands These commands output structured data for scripting, automation, and AI integration. - Query and list commands such as `query`, `schemas`, `tables`, `columns`, `connections list`, and `history list/search` keep their existing raw JSON array output. - Action and analysis commands such as `connections add/remove/test`, `history clear`, `audit`, `mock-data`, `diff`, `erd`, `bookmarks save/delete`, and `profiles save/delete` return a JSON envelope with `command`, `success`, and `data` when you pass `--format json`. - `query --stream` supports `plain`, `json`, `ndjson`, and `csv`. `export --stream` supports CSV only. ### agent schema Emit a machine-readable manifest of WhoDB's agent-facing surface. The manifest includes source types, connection fields, programmatic commands, MCP tools, platform MCP prompts/resources, safety modes, and built-in workflows. ```bash whodb agent schema --format json ``` Flags: - `--format, -f`: Output format: `json` ### doctor Run redacted connection, schema, and metadata diagnostics for one connection. ```bash whodb doctor --connection my-postgres whodb doctor --connection my-postgres --schema public --format json ``` Flags: - `--connection, -c`: Connection name to inspect - `--schema, -s`: Schema override for metadata checks - `--format, -f`: Output format: `table` or `json` - `--quiet, -q`: Suppress informational messages ### runbooks List, describe, and run built-in database workflows. Built-in runbooks are intentionally limited to WhoDB operations. ```bash # List available workflows whodb runbooks list # Inspect a workflow whodb runbooks describe schema-audit # Show planned steps without executing whodb runbooks run schema-audit --connection my-postgres --dry-run # Run a schema audit whodb runbooks run schema-audit --connection my-postgres --schema public --format json # Compare two environments whodb runbooks run schema-diff --from staging --to prod --format json ``` Built-in runbooks: - `connection-doctor`: Runs the same diagnostics as `doctor` - `schema-audit`: Loads storage units and runs data-quality checks - `schema-diff`: Compares schema metadata between two connections Flags: - `--format, -f`: Output format: `table` or `json` - `--quiet, -q`: Suppress informational messages - `run --connection, -c`: Connection name for `connection-doctor` and `schema-audit` - `run --schema, -s`: Schema override - `run --from`: Source connection for `schema-diff` - `run --to`: Target connection for `schema-diff` - `run --from-schema`: Source schema override for `schema-diff` - `run --to-schema`: Target schema override for `schema-diff` - `run --dry-run`: Show planned steps without executing ### skills List bundled WhoDB assistant skills and install native assistant integrations. ```bash # List bundled skills and agents whodb skills list whodb skills list --format json # Install all skills into an explicit skills directory whodb skills install --target-dir ~/.codex/skills # Install one skill whodb skills install query-builder --target-dir ~/.codex/skills # Install skills and bundled agents for Claude Code whodb skills install --target claude-code --include-agents # Install native MCP configuration for an assistant whodb skills install --target cursor whodb skills install --target vscode whodb skills install --target gemini-cli # Preview files without modifying disk whodb skills install --target cursor --dry-run ``` Supported targets: | Target | Files installed | |--------|-----------------| | `codex` | Skills under `~/.codex/skills` | | `claude-code` | Skills under `~/.claude/skills`, plus Markdown agents under `~/.claude/agents` with `--include-agents` | | `cursor` | `~/.cursor/mcp.json` | | `vscode` | VS Code user `mcp.json` | | `github-copilot` | GitHub Copilot CLI `~/.copilot/mcp-config.json` | | `gemini-cli` | `~/.gemini/extensions/whodb/gemini-extension.json` and `GEMINI.md` | | `windsurf` | `~/.codeium/mcp_config.json` | | `opencode` | `~/.config/opencode/opencode.json` with `mcp.whodb` | | `cline` | Cline MCP settings plus `~/Documents/Cline/Rules/whodb.md` | | `zed` | `~/.config/zed/settings.json` with `context_servers.whodb` | | `continue` | `~/.continue/config.yaml` | | `aider` | `~/.aider.conf.yml` plus `~/.aider/whodb-conventions.md` | Existing JSON and JSONC configuration files are merged in place and rewritten as formatted JSON. Before an existing JSON or YAML assistant config is rewritten, the original file is saved beside it as `.whodb.bak`. Use `--dry-run` to preview created or updated files and any backup paths without writing changes. Flags: - `--format, -f`: Output format: `table` or `json` - `--quiet, -q`: Suppress informational messages - `install --target`: Assistant target to install. Supported values are listed above - `install --target-dir`: Directory where skills should be installed - `install --agents-dir`: Directory where agents should be installed - `install --include-agents`: Install bundled Markdown agents as well as skills. With `--target`, this is supported for `claude-code`; use `--agents-dir` for any custom agent destination - `install --force`: Overwrite existing installed files - `install --dry-run`: Show files that would be written without modifying disk ### explain Run `EXPLAIN` using the current database plugin's native explain prefix. ```bash whodb explain --connection my-postgres "SELECT * FROM users" whodb explain --connection my-postgres --format json "SELECT * FROM users" ``` Flags: - `--connection, -c`: Connection name to use - `--format, -f`: Output format: `auto`, `table`, `plain`, `json`, `ndjson`, `csv` - `--quiet, -q`: Suppress informational messages ### schemas List database schemas. ```bash whodb schemas --connection my-postgres --format json ``` Flags: - `--connection, -c`: Connection name (optional; if omitted, the first available connection is used) - `--format, -f`: Output format: `auto`, `table`, `plain`, `json`, `csv` - `--quiet, -q`: Suppress informational messages ### tables List tables in a schema. ```bash whodb tables --connection my-postgres --schema public --format json ``` Flags: - `--connection, -c`: Connection name (optional; if omitted, the first available connection is used) - `--schema, -s`: Schema name (default varies by database) - `--format, -f`: Output format: `auto`, `table`, `plain`, `json`, `csv` - `--quiet, -q`: Suppress informational messages ### columns Describe table columns. ```bash whodb columns --connection my-postgres --table users --format json ``` Flags: - `--connection, -c`: Connection name (optional; if omitted, the first available connection is used) - `--table, -t`: Table name (required) - `--schema, -s`: Schema name - `--format, -f`: Output format: `auto`, `table`, `plain`, `json`, `csv` - `--quiet, -q`: Suppress informational messages ### connections Manage saved connections. ```bash # List connections whodb connections list --format json # Test a connection whodb connections test my-postgres --format json # Add a connection whodb connections add --name prod --type postgres --host db.example.com --port 5432 --user app --database mydb --format json # Remove a connection whodb connections remove prod --format json ``` Flags (applies to all subcommands): - `--format, -f`: Output format: `auto`, `table`, `plain`, `json`, `csv` - `--quiet, -q`: Suppress informational messages ### cloud Inspect configured cloud providers and discovered resources. Cloud provider support follows the shared provider flags: - `WHODB_ENABLE_AWS_PROVIDER=true` - `WHODB_ENABLE_AZURE_PROVIDER=true` - `WHODB_ENABLE_GCP_PROVIDER=true` ```bash # List configured providers whodb cloud providers list # Test or refresh providers whodb cloud providers test aws-prod-us-west-2 whodb cloud providers refresh --all # List discovered resources whodb cloud connections list whodb cloud connections list --provider aws-prod-us-west-2 # Use a discovered resource in the normal connect/save flows whodb connect --discovered aws-prod-us-west-2/prod-db whodb connections add --from-discovered aws-prod-us-west-2/prod-db --user alice --database app ``` ### diff Compare schema metadata between two connections. By default, `diff` uses each connection's configured schema when one exists. For database-scoped connections such as MySQL and MariaDB, it uses the connection's configured database when no schema flag is provided. ```bash # Compare two connections using their default schemas whodb diff --from staging --to prod # Compare the same schema on both sides whodb diff --from staging --to prod --schema public # Compare Postgres to MySQL using each connection's configured namespace whodb diff --from dev-e2e_postgres-1 --to dev-e2e_mysql-1 # Emit machine-readable JSON whodb diff --from staging --to prod --format json ``` Flags: - `--from`: Source connection name (required) - `--to`: Target connection name (required) - `--schema`: Schema name to compare on both sides - `--from-schema`: Source schema name - `--to-schema`: Target schema name - `--format, -f`: Output format: `table` or `json` - `--quiet, -q`: Suppress informational messages ### erd Render the same backend graph metadata used by the TUI ER diagram view. ```bash whodb erd --connection my-postgres whodb erd --connection my-postgres --schema public --format json ``` Flags: - `--connection, -c`: Connection name to use - `--schema, -s`: Schema name - `--format, -f`: Output format: `text` or `json` - `--quiet, -q`: Suppress informational messages ### export Export table data or query results to file. ```bash # Export to CSV whodb export --connection my-postgres --table users --format csv --output users.csv # Export to Excel whodb export --connection my-postgres --table orders --format excel --output orders.xlsx # Export query results whodb export --connection my-postgres --query "SELECT * FROM users" --output users.csv ``` Flags: - `--connection, -c`: Connection name (optional; if omitted, the first available connection is used) - `--table, -t`: Table name (required unless using `--query`) - `--query, -Q`: SQL query to export results from (use instead of `--table`) - `--schema, -s`: Schema name - `--format, -f`: Export format: `csv` or `excel` (auto-detected from filename if omitted) - `--output, -o`: Output file path (required) - `--delimiter, -d`: CSV delimiter (default: comma) - `--stream`: Stream CSV exports incrementally to the output file - `--quiet, -q`: Suppress informational messages ### history Access query history. ```bash # List recent queries whodb history list --limit 20 --format json # Search history whodb history search "SELECT.*users" # Clear history whodb history clear --format json ``` Flags: - `--limit, -l`: Limit number of results (0 = no limit) - `--format, -f`: Output format: `auto`, `table`, `plain`, `json`, `csv` - `--quiet, -q`: Suppress informational messages ### bookmarks Manage the same saved query bookmarks used by the TUI editor. ```bash whodb bookmarks list whodb bookmarks save recent-users "SELECT * FROM users ORDER BY id DESC" whodb bookmarks load recent-users whodb bookmarks delete recent-users --format json ``` ### profiles Manage the same saved connection profiles used by the TUI. ```bash whodb profiles list whodb profiles save production --connection prod --theme Dracula --page-size 100 --timeout 30 whodb profiles show production --format json whodb profiles delete production --format json whodb --profile production ``` ## Startup modes Bare `whodb` shows saved platform login/workspace metadata and next commands; `whodb status` checks the live login. Use `whodb --tui` for the existing database terminal UI. Explicit `connect`, `query`, and `--profile` behavior is unchanged. `whodb setup` saves the modules and access settings used by `whodb mcp serve`. Platform tools are enabled by default. Enable database-only tools or both modules in setup; every configuration uses the same MCP command. `--database` remains an advanced override for a database-only process, and `--platform` remains a compatibility override for platform-only tools. Both modules support stdio and HTTP transports; database-only does not imply that the database is local. ## Guided MCP setup Run `whodb setup` once. The wizard lets you choose modules, write access, hosts, database connections, and optional restrictions. It previews the settings before saving, offers browser sign-in for platform hosts, and can verify connections. Existing settings are the defaults when you run the wizard again. You can also choose an optional default platform workspace using canonical organization and project IDs. Password entry is hidden and passwords are stored through the existing CLI credential store. Keep one entry in your MCP client: ```json { "mcpServers": { "whodb": {"command": "whodb", "args": ["mcp", "serve"]} } } ``` ### Agent-guided setup **MCP is self-sufficient; skills are optional supplements.** The server supplies startup instructions, tool descriptions, schemas, workflow guidance, and recovery steps. If a WhoDB skill is installed, agents may use it as additional guidance; no essential setup or usage instructions require it. An agent with MCP access alone can call `whodb_mcp_setup`: 1. `action: "inspect"` returns saved settings, allowed target metadata, choices, schema, active process settings, and usage guidance. 2. `action: "preview", patch: {...}` validates changes without saving and returns the proposed settings and a confirmation token. 3. After showing the exact preview and obtaining user approval, `action: "apply", confirmation_token: "...", approved: true` saves that preview. Do not include a patch in apply. Tokens expire after five minutes, are consumed by apply, and are replaced by a newer preview. Changed saved settings invalidate the preview. 4. Restart the MCP connection after changes, then `action: "verify"` checks active host logins, workspaces, and database connections. Until restart, verification uses the original process settings. The setup tool manages local MCP configuration, including access restrictions; it always requires preview approval to save, even in allow-write mode. It follows normal tool filters: include `whodb_mcp_setup` when using an explicit enabled-tool list, or disable it to keep configuration management outside the agent. Changes never relax the current process's controls. Startup flags still override saved settings after restart. Browser sign-in and credential entry remain user actions; agents can relay the required commands without needing shell access. Agents with shell access can also use the non-interactive JSON commands: ```bash whodb setup inspect whodb setup validate --file setup.json # After the user approves the proposed settings: whodb setup apply --file setup.json --yes whodb setup verify ``` `inspect` returns current settings, available target metadata, tool choices and a settings schema. `validate` previews the merged settings without saving. `apply` requires `--yes` and reports whether a restart is needed. `verify` checks the configured host logins and database connections, reporting each failure and next command. Authentication is separate: use `whodb login --host ` for browser sign-in. Agents must not request passwords or tokens in chat or setup JSON. Example `setup.json` enabling both modules with production read-only: ```json { "modules": ["platform", "database"], "write_mode": "confirm", "platform_policy": { "allowed_hosts": ["https://uat.whodb.com", "https://app.whodb.com"], "read_only_hosts": ["https://app.whodb.com"] }, "allowed_connections": ["development"] } ``` Settings patches preserve omitted fields; arrays replace existing arrays. Use `[]` to clear a tool/connection allowlist and `"platform_policy": null` to clear platform restrictions. The empty allowlists mean unrestricted access. Unknown fields are rejected. Settings are saved in the `mcp` section of WhoDB's existing configuration file; applying a settings patch does not rewrite the credential or connection section. Use `--file -` to supply a patch on stdin. Restart an existing MCP connection after changing modules or access settings. An optional `default_workspace` uses `{host, org, project}` with canonical IDs. Explicit process flags or session environment variables replace that saved default target; calls with `workspace` still override it. Platform policies apply to platform tools; database tools use `allowed_connections` and the shared `write_mode`. Host/project targeting remains per-call: platform tools accept `workspace`, database-only tools accept `connection`. You do not need another MCP entry or a restart to target a different saved host/project. Read `whodb://mcp/configuration` for the running server's active modules and restrictions. Explicit startup flags override saved settings for that process without saving those overrides. To keep one chat on one host (for example UAT in one Claude Code or Codex window and production in another) without separate MCP entries, the agent calls `whodb_platform_session_use` with `workspace {host, org, project}` once. That default applies to later calls in that MCP session only, is never written to disk, and is removed with `whodb_platform_session_clear`. `whodb_platform_status` shows it as `session_scope`. Resolution order for a call without `workspace`: session default, then process flags/environment, then the saved CLI default. `whodb use` changes the saved default for every session, so it is the wrong tool for switching a single chat. ## MCP Server WhoDB can run as an MCP (Model Context Protocol) server, enabling AI assistants like Claude, Cursor, and others to query your databases. ### Access modes The default supports reads and writes, with confirmation required before writes execute. Platform tools always enforce the signed-in user's platform permissions. For direct database tools, access is limited by the configured database account's privileges; platform permissions do not apply to those connections. - Default (`whodb mcp serve`): reads and confirmed writes. - `--read-only`: restrict operations to reads. - `--safe-mode`: read-only with strict security checks. - `--allow-write`: execute writes without the MCP confirmation step, for trusted automation. Database and platform permissions still apply. Saved setup settings can select a different default. Inspect them with `whodb setup inspect`; restart the MCP connection after changing access modes. ### Start the MCP Server ```bash # Default: stdio transport (for Claude Desktop, Claude Code, etc.) whodb mcp serve # HTTP transport (for cloud deployments, Docker, Kubernetes) whodb mcp serve --transport=http --port=3000 ``` The platform module exposes `whodb_platform_*` tools. Enable the database module in `whodb setup` to expose the following tools in the same server: | Tool | Description | |------|-------------| | `whodb_connections` | List available database connections | | `whodb_schemas` | List schemas in a database (set `include_tables` for tables too) | | `whodb_tables` | List tables in a schema (set `include_columns` for column details too) | | `whodb_columns` | Describe table columns | | `whodb_query` | Execute SQL queries (results include `column_types`) | | `whodb_confirm` | Confirm pending write operations (only when confirm-writes is enabled) | | `whodb_pending` | List pending write confirmations (only when confirm-writes is enabled) | | `whodb_explain` | Run database-native `EXPLAIN` for a SQL query | | `whodb_diff` | Compare schema metadata between two connections | | `whodb_erd` | Inspect backend graph/ERD metadata | | `whodb_audit` | Run data quality audits for a schema or table | | `whodb_suggestions` | Get backend-generated starter queries | Without saved settings, only the platform module is enabled. Setup can enable either module or both together. `--tools` and `--disable-tools` work in both modes. Platform mode uses full `whodb_platform_*` names; database-only mode uses short names such as `query`. ### Platform access policy and tool selection Save `platform_policy` through setup to restrict access independently of saved logins. The advanced `--platform-policy policy.json` flag overrides that policy for one process. For example, allow two hosts but keep production read-only: ```json { "allowed_hosts": ["https://uat.whodb.com", "https://app.whodb.com"], "read_only_hosts": ["https://app.whodb.com"] } ``` ```bash whodb mcp serve --platform-policy policy.json whodb mcp serve --tools whodb_platform_hosts,whodb_platform_workspace_resolve,whodb_platform_sources whodb mcp serve --disable-tools whodb_platform_source_delete,whodb_platform_delete ``` For project restrictions, add `allowed_workspaces` or `read_only_workspaces`: ```json { "allowed_workspaces": [ {"host": "https://uat.whodb.com", "org": "", "project": ""}, {"host": "https://app.whodb.com", "org": "", "project": ""} ], "read_only_workspaces": [ {"host": "https://app.whodb.com", "org": "", "project": ""} ] } ``` Policy workspace entries require canonical organization and project IDs, not names or slugs. Tool calls can still resolve names through `workspace`. When a policy is configured, include matching `workspace.org` and `workspace.project` when using tools with top-level `org` or `project` arguments. Host and workspace allowlists intersect; omitted or empty allowlists impose no restriction. Read-only rules override `--allow-write` and also apply when confirming an earlier preview. Global `--read-only` remains stricter than any per-target setting. Discovery lists omit disallowed targets. The policy is loaded at startup; restart MCP after changing it. Tool disabling takes precedence over enabling. Unknown platform tool names are rejected at startup. Include `whodb_platform_confirm` and `whodb_platform_pending` when selecting tools that need write confirmation. Tool filtering controls tool availability; generic write tools can support overlapping operations. Use read-only policies to block all writes to a target. Platform authorization still applies to every request. `whodb://platform/schema` reports the active access policy and enabled tools. Failed operations set MCP `isError` while preserving structured error details, request IDs, scope, and recovery guidance when supplied by the handler. ### Per-call workspace targeting `whodb mcp serve` exposes hosted platform tools. Sign in to each required host with `whodb login --host `. Start with `whodb_platform_hosts` to discover saved hosts and accounts, then `whodb_platform_orgs` with `workspace: {host}` and `whodb_platform_projects` with `workspace: {host, org}`. Resolve the intended target with `whodb_platform_workspace_resolve`. Pass an explicit `workspace: {host, org, project}` on subsequent tools when working across UAT, production, or projects. Targets apply to one call, including parallel calls, and never change saved defaults. Check the returned `scope`. Do not run `whodb use` to switch another session's defaults. Omitted targets use process overrides, then saved defaults. Read `whodb://platform/schema` for operations and payloads. Writes return a preview bound to its host, account, organization, and project. Confirm only after approval of that exact preview. Changing defaults does not redirect a pending confirmation. Persisted workflow plans from before account binding must be recreated before applying them. Ask for `whodb login --host ` when a host needs authentication; no separate login is needed per project. Example arguments for `whodb_platform_sources`: ```json {"workspace":{"host":"https://uat.whodb.com","org":"acme","project":"analytics"}} ``` An explicit host uses that host's saved defaults. An explicit organization clears any inherited project; project-scoped operations then require a project. Invalid or ambiguous selectors fail instead of using another workspace. Host discovery reads local metadata; it does not claim the saved login is still valid. Hosted setup: ```bash # Sign in to app.whodb.com whodb login # Optional: save a default for calls without an explicit workspace whodb use --org --project # Start stdio MCP for your MCP client whodb mcp serve # Read-only hosted platform MCP whodb mcp serve --read-only # Hosted writes without confirmation; use only for trusted automation whodb mcp serve --allow-write ``` To capture an actual rendered app with its current data, use the app commands or the `whodb_platform_app_views` and `whodb_platform_app_screenshot` MCP tools. The MCP screenshot tool returns WebP image content directly to the model. Both paths use the selected hosted workspace and are available in read-only mode. ```bash whodb apps views "Clidey Sales" --org clidey-erp --project erp-demo --env dev whodb apps screenshot "Clidey Sales" --org clidey-erp --project erp-demo \ --env dev --tab Catalog --output sales-catalog.webp whodb apps setup-capture # Or install the optional browser on demand when taking a screenshot: whodb apps screenshot "Clidey Sales" --install --org clidey-erp --project erp-demo ``` For a specific page or interactive state, list pages first, then pass `--page`. Use `--click` for a visible app control, or `--actions` for an ordered sequence. Selectors are Playwright selectors inside the app frame. Each action selector must match one element; scope repeated text to a node or table row. `--js` evaluates a JavaScript expression or IIFE in that frame after the actions; `--script-file` reads the expression from a file. JSON output includes the screenshot path, visible text, browser console errors, page errors, failed requests, HTTP error responses, and the JavaScript result. For content loaded after mount, include a `wait_for` action on an expected row or control so the screenshot captures the loaded state. An assertion in `--js` can throw an error. The command still saves the screenshot and JSON diagnostics, sets `checks_passed` to `false`, and exits nonzero. `checks_passed` confirms that actions ran and explicit `wait_for` or JavaScript assertions passed; a click alone does not prove the app reached an intended view. ```bash whodb apps views "Clidey Sales" --org clidey-erp --project erp-demo --env dev whodb apps screenshot "Clidey Sales" --org clidey-erp --project erp-demo \ --env dev --page main --click 'text=Orders' \ --js '(() => ({ rows: document.querySelectorAll("tr").length }))()' \ --output sales-orders.webp --format json # Capture the same app with mobile viewport and touch behavior. whodb apps screenshot "Clidey Sales" --mobile --org clidey-erp --project erp-demo \ --output sales-orders-mobile.webp # Capture a tablet in landscape orientation. whodb apps screenshot "Clidey Sales" --tablet --orientation landscape \ --org clidey-erp --project erp-demo --output sales-orders-tablet.webp ``` The MCP screenshot tool accepts the same `page`, `tab`, `actions`, `script`, `mobile`, `tablet`, and `orientation` inputs, so an agent can click, fill, press, wait for a selector, or inspect DOM state without creating a Playwright script. For example, use an action `{"kind":"click","selector":"text=Orders"}` followed by a script `"(() => document.querySelectorAll('tr').length)()"`. Set `trace: true` to request an interaction trace from a compatible hosted runtime. App capture needs Node.js and npm. `setup-capture` installs Playwright and Chromium in the user's cache; `--install` does the same before capturing. Without them, the command and MCP tool return setup instructions. Existing Playwright installations in the WhoDB source checkout work without setup. `apps screenshot` and the MCP screenshot tool only allow hosted GraphQL reads by default. Use `--allow-writes` for a function call or another mutation. `apps dev` and `apps verify` have their own write setting, described below. Captures never upload or publish the image. Inspect sample data and loading states before using a capture as product media. ### Build an app from a local folder An app folder contains `app.json`, `pages//tree.json`, page scripts and state, and optional `shared/` files. `app.json` declares `name`, `defaultPage`, and optional `ontologies`, `readOnlyOntologies`, and `functions` arrays. Resource entries use the ontology API name or function name; the CLI resolves them to IDs in the selected project. App scripts use the normal `ctx.whodb` browser API for ontology and function calls. ```bash whodb apps init ./orders --name Orders whodb apps check ./orders --format json # Or start from an existing hosted app's draft files and bindings. whodb apps init ./existing-orders --from --org --project # Create the hosted app identity and bind its resources without promoting it. whodb apps publish ./orders --draft --org --project # Files are read from disk on each reload; edits are not uploaded. whodb apps dev ./orders --org --project # Replay the generated scenario, save a WebP, and return a JSON flow trace. whodb apps verify ./orders --scenario ./orders/scenarios/smoke.json \ --output /tmp/orders.webp --format json --org --project # Replay the same flow at a phone viewport. whodb apps verify ./orders --mobile --scenario ./orders/scenarios/smoke.json \ --output /tmp/orders-mobile.webp --format json --org --project # Replay at the default tablet viewport. whodb apps verify ./orders --tablet --scenario ./orders/scenarios/smoke.json \ --output /tmp/orders-tablet.webp --format json --org --project # Sync changed and deleted files, then promote the resulting app version. whodb apps publish ./orders --message "Orders v1" --org --project ``` `apps check` validates file structure and handler references; it does not render the app or verify scroll behavior. `apps dev` and `apps verify` use the hosted app runner and its existing resource permissions while replacing app files with the local folder in the browser. For a desktop app with a sidebar that should stay visible, make the main pane the scroll container: set the root `Row` to viewport height with hidden overflow, the sidebar to full height with auto overflow, and the main `Col` to full height with `minH: 0` and auto overflow. Test a page longer than the viewport by scrolling the main pane; a full-height root with overflowing content makes a sticky sidebar move with the outer app viewport. `apps init --from` creates a new local folder from the selected app's readable draft files, preserving existing `app.json` settings and writing resource names for version control. It refuses to overwrite an existing folder. The hosted app is unchanged; pass its ID with `--app` to `apps dev`, `apps verify`, or `apps publish` if you rename the local app. `apps dev` and `apps verify` require an app in the selected project with the same resource bindings as `app.json`. Use `--app ` when the hosted name differs. Both commands allow app actions to write to the selected project's data by default; pass `--allow-writes=false` only when the preview needs GraphQL queries and no function calls. Every `ctx.whodb.call()` uses a GraphQL mutation, including reporting functions, and is blocked by that flag. Local app files are overlaid in the browser, while ontology and function calls still use the selected hosted project. Select an isolated development project for workflow tests, and keep `--host`, `--org`, and `--project` aligned with the MCP workspace you inspected. MCP read-only mode does not change the CLI preview's write setting. Both CLI commands require Node.js and the optional capture browser (`apps setup-capture` or `--install`). `--mobile` on `apps screenshot` or `apps verify` uses a 390x844 viewport with mobile and touch emulation. Pass `--width` and `--height` with it to test another phone size. `--tablet` uses an 820x1180 touch viewport. Add `--orientation landscape` to rotate either device preset; explicit `--width` and `--height` can set another size. Default filenames include `-landscape` when applicable. When testing a locally built app runtime, pass the directory containing `sandbox.html`, `sandbox-kernel.js`, and `sandbox-kernel.css` with `--kernel-dir` or `WHODB_APP_KERNEL_DIR`. Flow tracing requires a compatible runtime; `apps dev` can use the hosted runtime for visual preview. A scenario is JSON with an `actions` array. It accepts `click`, `fill`, `press`, `wait_for`, `wait`, `expect_text`, and `expect_not_text` actions. Selectors run inside the app frame; stable node IDs can be selected with `[data-node-id='save-button']`. For text repeated in multiple tables, scope the selector to a table, such as `[data-node-id='attention-table'] tbody tr:first-child`. An empty `value` on `fill` clears an input. `apps verify` waits for handlers and bridge calls to settle, then outputs visible text, browser diagnostics, and a trace of handlers, state writes, navigation, and WhoDB calls. It exits nonzero on a failed assertion or runtime error. `apps publish --dry-run` reports file changes without writing them. File writes update the draft one at a time; an interrupted sync can be completed by rerunning the command. Published file content changes only after promotion; changes to the app's resource bindings take effect when the draft metadata is updated. MCP clients can read `whodb://platform/app-workflow` or request the `whodb_platform_build_app_workflow` prompt for the app build sequence. The prompt guides the agent through a brief-specific visual plan, real data and SDK contracts, working interactions, and screenshot critique across viewports. MCP prompts are client-requested; the server instructions point app-building agents to this prompt. The `whodb_platform_workflow_recipe` tool also returns an app-specific plan. Hosted MCP tools can inspect and edit hosted draft files and capture hosted views. `whodb_platform_app_screenshot` accepts `trace: true` on runtimes that support interaction tracing. `whodb_platform_app_components`, `whodb_platform_app_component`, `whodb_platform_app_sdks`, and `whodb_platform_app_sdk` read the same reference material as the in-app builder when the hosted platform advertises that capability. `whodb_platform_app_logic_graph` runs the builder's interaction and function contract audit on an authorized hosted draft. A client needs local filesystem and CLI access to use `apps init`, `apps dev`, `apps verify`, and `apps publish` against a version-controlled folder. Local or staging setup: ```bash whodb login --host http://localhost:8080 whodb use --host http://localhost:8080 --org --project whodb mcp serve ``` Recommended MCP client behavior: 1. Read `whodb://platform/tool-guide` to understand the active platform MCP mode, available tool categories, field projection guidance, and write behavior. 2. Call `whodb_platform_status` before project-scoped reads or writes. 3. Use `fields` on hosted read tools whenever supported. Start narrow, for example `["id", "name", "type"]`, and request heavier fields only when needed. 4. For writes in default mode, explain `confirmation_preview` to the user and call `whodb_platform_confirm` only after the user approves that exact preview. 5. In `--allow-write`, hosted writes execute immediately, so ask the user before calling the mutating tool itself. Most hosted read tools accept a `fields` array. Agents should request only the fields needed for the current answer, for example `["id", "name"]`, then call again with additional fields only when needed. Backend permissions still decide what the signed-in user can see or change. Platform mode uses the same permission modes as local MCP. By default, hosted create, update, delete, and action tools return confirmation tokens and do not execute until approved with `whodb_platform_confirm`. `--read-only` and `--safe-mode` hide hosted write tools entirely. `--allow-write` executes hosted writes immediately without confirmation. | Tool | Description | |------|-------------| | `whodb_platform_status` | Show hosted login and selected workspace | | `whodb_platform_orgs` | List hosted organizations visible to the signed-in user | | `whodb_platform_projects` | List hosted projects for an organization | | `whodb_platform_sources` | List hosted sources in the selected project | | `whodb_platform_source_types` | List hosted source types available for creation | | `whodb_platform_source_fields` | List connection fields for one hosted source type | | `whodb_platform_source_objects` | Browse hosted source objects | | `whodb_platform_source_columns` | Inspect hosted source object columns | | `whodb_platform_source_rows` | Preview hosted source object rows | | `whodb_platform_source_constraints` | Inspect editable source field constraints | | `whodb_platform_source_content` | Read hosted source object content when supported | | `whodb_platform_source_config` | Inspect redacted hosted source config | | `whodb_platform_source_test` | Test saved or draft hosted source connections | | `whodb_platform_secrets` | List secret metadata without secret values | | `whodb_platform_ai_providers` | List hosted AI provider metadata without API keys | | `whodb_platform_ai_provider_models` | List models for one hosted AI provider | | `whodb_platform_ontologies` | List hosted ontology object types | | `whodb_platform_ontology` | Inspect one hosted ontology object type | | `whodb_platform_ontology_fast_lookups` | List saved fast lookups for one ontology | | `whodb_platform_ontology_fast_lookup_suggestions` | List suggested fast lookups | | `whodb_platform_ontology_rows` | Preview rows for one ontology | | `whodb_platform_ontology_follow_link` | Follow one ontology link from a row | | `whodb_platform_ontology_describe` | Describe ontologies, fields and relationships in text and structure | | `whodb_platform_ontology_query` | Filter, search, join, group and compute metrics over ontology rows (bounded scan) | | `whodb_platform_ontology_aggregate` | Exact SQL COUNT/SUM/AVG/MIN/MAX over one ontology, grouped and filtered | | `whodb_platform_ontology_stats` | Min, max, total and non-null counts for one property | | `whodb_platform_ontology_similar` | Similarity search over an ontology with embeddings enabled | | `whodb_platform_ontology_embedding_jobs` | Embedding backfill job status for one ontology | | `whodb_platform_ontology_record_capabilities` | Readable/writable properties and allowed behavior actions for a record | | `whodb_platform_ontology_action_preview` | Preview a behavior action without executing it | | `whodb_platform_ontology_action_executions` | Behavior action execution history for a record | | `whodb_platform_shared_ontologies` | Ontologies the project shares with other projects | | `whodb_platform_discover_shared_ontologies` | Shared ontologies the project may reference | | `whodb_platform_ontology_references` | Shared ontologies referenced by the project | | `whodb_platform_ontology_reference_impact` | Dependents of one ontology reference | | `whodb_platform_transform_preview` | Run a transform graph without saving it | | `whodb_platform_service_accounts` | Service accounts and key metadata (no secrets) | | `whodb_platform_lineage_providers` | OpenLineage providers for the project | | `whodb_platform_user_owned_resources` | Resources owned by one member | | `whodb_platform_user_resource_access` | Resources one member can access and why | | `whodb_platform_share_dependents` | Resources that must be shared alongside a resource | | `whodb_platform_org_package_installations` | Package installations across the organization | | `whodb_platform_app_page_files` | Generated files for one app page | | `whodb_platform_datasets` | List hosted datasets | | `whodb_platform_dataset` | Inspect one hosted dataset | | `whodb_platform_dataset_rows` | Preview hosted dataset rows | | `whodb_platform_lineage` | Inspect lineage around one root node | | `whodb_platform_lineage_neighbors` | Inspect immediate lineage neighbors | | `whodb_platform_project_lineage` | Inspect project-level lineage | | `whodb_platform_transforms` | List hosted transforms | | `whodb_platform_transform_runs` | List recent runs for one transform | | `whodb_platform_functions` | List hosted ontology functions | | `whodb_platform_function` | Inspect one hosted ontology function | | `whodb_platform_files` | List hosted project folders and files | | `whodb_platform_file_preview` | Preview one hosted project file | | `whodb_platform_file_search` | Search hosted project files | | `whodb_platform_tabular_files` | List tabular hosted project files | | `whodb_platform_storage_usage` | Show project storage usage in bytes | | `whodb_platform_source_create` | Prepare hosted source creation for `whodb_platform_confirm` | | `whodb_platform_source_update` | Prepare hosted source updates for `whodb_platform_confirm` | | `whodb_platform_source_delete` | Prepare hosted source deletion for `whodb_platform_confirm` | | `whodb_platform_create` | Prepare generic hosted resource creation | | `whodb_platform_update` | Prepare generic hosted resource updates | | `whodb_platform_delete` | Prepare generic hosted resource deletion | | `whodb_platform_add_ontology_records` | Add many ontology rows at once with an idempotency key | | `whodb_platform_action` | Prepare generic hosted actions such as upload, move, run, cancel_run, deploy, restore, purge, share, reembed, service account and package actions | | `whodb_platform_pending` | List pending hosted platform confirmations | | `whodb_platform_confirm` | Confirm pending hosted platform writes after user approval | Hosted create, update, delete, and action tools return confirmation tokens by default and do not execute until approved with `whodb_platform_confirm`. Use `whodb_platform_pending` to recover active confirmation tokens. Agents should explain the returned `confirmation_preview` to the user before confirming, especially for delete, deploy, move, and source/object changes. #### Answering data questions Agents answer questions about ontology data without paging through rows: 1. `whodb_platform_ontology_describe` to learn entity and property names. 2. `whodb_platform_ontology_stats` to profile the properties involved. 3. `whodb_platform_ontology_aggregate` for exact totals and group-bys, or `whodb_platform_ontology_query` for filtered rows, free-text search, joins and in-memory metrics. The query scans at most `scan_limit` rows (default 1000, max 5000) before filtering, so prefer the aggregate for large entities. 4. `whodb_platform_ontology_rows` or `whodb_platform_ontology_follow_link` to drill into the rows behind a number. Every ontology read tool accepts the same `where` object: `{prop: value}`, `{prop: {op: value}}` and `_and` / `_or` lists with operators `eq`, `neq`, `gt`, `gte`, `lt`, `lte`, `contains`, `icontains`, `startswith`, `in`, `nin`, `between` and `isnull`. The aggregate, stats and similarity tools translate it to the platform `WhereCondition` for you; a raw `WhereCondition` object is still accepted. Organization-scoped actions default `orgId` to the selected organization, so it only needs to be passed when targeting another organization. #### Bulk inserts, run control, restore and purge - `whodb_platform_add_ontology_records` inserts many rows in one confirmation and accepts an `idempotency_key` so a retried call never duplicates rows. - `whodb_platform_action` with resource `function_run` and action `cancel`, resource `transform` and action `cancel_run`, or resource `app` and action `cancel_generation` stops work that is still running. - `whodb_platform_restore` now covers `project`, `file` and `folder`; the matching `purge` actions permanently delete soft-deleted resources, and `purge_data` on an `ontology` or `dataset` empties it while keeping the definition. - Service accounts are managed through `whodb_platform_action` on the `organization` resource: `create_service_account`, `create_service_account_key` (the `rawKey` is returned once), `revoke_service_account_key` and `delete_service_account`. - Ontologies are shared with `share` / `unshare` actions and consumed from another project through the `ontology_reference` resource. The same capabilities are available from the CLI: `whodb ontologies query|aggregate|stats|explain|similar`, `whodb ontologies records add-many`, `whodb functions runs cancel`, `whodb transforms cancel-run`, `whodb projects restore|purge`, `whodb service-accounts ...`, and generically `whodb resources action ` or `whodb resources query `. Security model: - The hosted platform remains the authorization boundary. The CLI sends the signed-in user's token and selected workspace; the backend still decides what that user may read or mutate. - Workspace IDs are selectors, not proof of access. Pending confirmations store the host, organization, and project and re-check them before execution. - Source config reads return redacted values, secret listing tools return metadata only, and confirmation previews include changed field names rather than credential values. For source creation, agents should call `whodb_platform_source_types` and `whodb_platform_source_fields` first so they use backend-published source type ids and field names. If no workspace is selected yet, agents should discover saved hosts with `whodb_platform_hosts`, then call `whodb_platform_orgs` with `workspace: {host}` and `whodb_platform_projects` with `workspace: {host, org}`. Resolve the chosen target with `whodb_platform_workspace_resolve` and pass its canonical IDs in `workspace` on subsequent calls. `whodb use` is optional for saving a default. For single-workspace accounts, hosted `login` or `status` can select the only organization/project automatically and report what was selected. Example hosted platform MCP config: ```json { "mcpServers": { "whodb-platform": { "command": "whodb", "args": ["mcp", "serve"] } } } ``` For read-only access, save `{"write_mode":"read-only"}` through setup. The MCP client entry remains the same: ```json { "mcpServers": { "whodb-platform-readonly": { "command": "whodb", "args": ["mcp", "serve"] } } } ``` Example local or staging platform MCP config: ```json { "mcpServers": { "whodb-platform-local": { "command": "whodb", "args": ["mcp", "serve"] } } } ``` One MCP server can use multiple saved host logins. Sign in to each host with `whodb login --host `, then pass `workspace: {host, org, project}` per tool call to target local, UAT, or production workspaces without restarting the MCP server or changing saved defaults. Without an explicit target, the server uses its process overrides and then saved defaults. Database-only MCP exposes these resources: | Resource | Description | |----------|-------------| | `whodb://connections` | Available connection names | | `whodb://agent/schema` | Machine-readable WhoDB agent capability manifest | Platform MCP exposes these resources instead: | Resource | Description | |----------|-------------| | `whodb://platform/schema` | Machine-readable platform MCP contract, enabled tools, platform prompts, and platform resources | | `whodb://platform/workspace` | Current hosted login and selected workspace metadata | | `whodb://platform/tool-guide` | Platform tool categories, recommended usage, field projection guidance, and write behavior | The same metadata is available from `whodb agent schema --format json`. Its `platform_mcp` section describes the default platform mode, default host, login/workspace requirements, field projection support, platform-only tool prefix, platform prompts/resources, and write behavior for confirm, read-only, safe, and allow-write modes. Write operations require confirmation by default. Use `--allow-write` to disable confirmations, or `--read-only` to block writes entirely. #### Upload review and confirmation limitations For file uploads, the write plan, confirmation preview, and pending-action list show the full absolute source path, destination host and project, and folder ID (`null` means the project root). The summary quotes the path and escapes control characters. Relative paths are resolved against the MCP server's working directory when the action is prepared; confirmation uses that stored absolute path. A literal `~` is not expanded to the user's home directory. The CLI reads file contents only when the upload executes. The preview approves a path, not an immutable file snapshot: replacing the file or changing a symlink before execution can change the uploaded contents. This flow does not sandbox local file access; the CLI can read files permitted by its operating-system identity. **A confirmation token does not independently prove human approval.** The model receives the token and can call `whodb_platform_confirm`. The assistant is instructed to ask the user first, but an independent approval guarantee requires the MCP host or another trusted interaction outside the model's control to enforce that approval. Clear previews support informed review; they do not prevent a malicious or prompt-injected model from calling available tools. Use `--read-only` or `--safe-mode` to hide hosted write tools when writes are not needed. `--allow-write` executes uploads without the CLI confirmation step. Upload source paths are intentionally visible to the MCP client for review. They are excluded from WhoDB upload telemetry; client-side logging and retention are controlled by the MCP host. Other sensitive fields remain redacted, and persisted workflow payloads still reject local file paths. ### Transport Modes **stdio (default)** - For local CLI integration with Claude Desktop, Claude Code, etc. ```bash whodb mcp serve ``` **HTTP** - For cloud deployments, Docker, Kubernetes, or shared access. ```bash whodb mcp serve --transport=http --host=0.0.0.0 --port=8080 ``` HTTP mode exposes: - `/mcp` - MCP endpoint (streaming HTTP) - `/health` - Health check endpoint ### Security Modes | Mode | Flag | Description | |------|------|-------------| | Confirm-writes | *(default)* | Write operations require user approval | | Safe mode | `--safe-mode` | Read-only + strict security (for demos/playgrounds) | | Read-only | `--read-only` | Blocks all write operations | | Allow-write | `--allow-write` | Full write access without confirmation | ### MCP Flags **Security:** - `--safe-mode`: Read-only + strict security (for demos/playgrounds) - `--read-only`: Block all write operations - `--allow-write`: Allow writes without confirmation (use with caution) - `--allow-drop`: Allow DROP/TRUNCATE when running with `--allow-write` - `--security`: Validation level (`strict`, `standard`, `minimal`) **Query Limits:** - `--timeout`: Query timeout (default 30s) - `--max-rows`: Limit rows returned per query (0 = unlimited) - `--allow-multi-statement`: Allow multiple SQL statements in one query **Transport:** - `--transport`: `stdio` (default) or `http` - `--host`: Bind address (default: localhost) - `--port`: Listen port (default: 3000) **Connection Scoping:** - `--allowed-connections`: Comma-separated list of connections to allow (restricts access) - `--default-connection`: Default connection when not specified (does not restrict access) **Hosted Platform:** - `--database`: Expose database-only MCP tools instead of platform tools. - `--platform`: Compatibility alias for the default; prints a notice to stderr. ```bash # Restrict AI to specific connections only whodb mcp serve --database --allowed-connections prod,staging # Set default without restricting access whodb mcp serve --database --default-connection prod # Combine: restrict to prod/staging, default to staging whodb mcp serve --database --allowed-connections prod,staging --default-connection staging ``` When `--allowed-connections` is set: - `whodb_connections` only shows allowed connections - Queries to other connections are rejected - First allowed connection becomes the default (unless `--default-connection` is set) ### Configure Connections The MCP server uses the same connection sources as the CLI: **Option 1: Environment Profiles** (recommended for production) Use env profiles like `WHODB_POSTGRES='[{"alias":"prod","host":"host","user":"user","password":"pass","database":"dbname","port":"5432"}]'` or `WHODB_MYSQL_1='{"alias":"staging","host":"host","user":"user","password":"pass","database":"dbname","port":"3306"}'`. Each object supports `alias` (connection name), `host`, `user`, `password`, `database`, `port`, and optional `config`. Use the JSON formats shown above. `alias` sets the connection name used in MCP tools. ```bash # Array format export WHODB_POSTGRES='[{"alias":"prod","host":"host","user":"user","password":"pass","database":"dbname","port":"5432"}]' # Numbered format (one profile per variable) export WHODB_MYSQL_1='{"alias":"staging","host":"host","user":"user","password":"pass","database":"dbname","port":"3306"}' ``` If `alias` is omitted, the CLI assigns a name like `postgres-1`. Saved connections take precedence if names collide. **Option 2: Saved Connections** Use `whodb connect --name mydb ...` to save connections that the MCP server can access. If a tool call omits `connection`, the MCP server uses the only available connection or returns an error if multiple are available. ### MCP Client Configuration (Example) Example configuration (from `whodb mcp serve --help`): ```json { "mcpServers": { "whodb": { "command": "whodb", "args": ["mcp", "serve"], "env": { "WHODB_POSTGRES_1": "{\"alias\":\"prod\",\"host\":\"localhost\",\"user\":\"user\",\"password\":\"pass\",\"database\":\"db\"}" } } } } ``` ### Docker MCP Server ```bash docker run -i --rm \ -e WHODB_POSTGRES_1='{"alias":"prod","host":"host","user":"user","password":"pass","database":"db"}' \ --network host \ whodb-cli:latest mcp serve --database ``` ## Interactive Mode Views ### 1. Connection View Select and manage database connections. | Key | Action | |---------|------------------------------| | `↑/k/Shift+Tab` | Move up | | `↓/j/Tab` | Move down | | `Enter` | Connect to selected database | | `n` | New connection | | `d` | Delete connection | | `Esc` | Back (form) / press twice to quit (list) | | `Ctrl+C` | Force quit | Form mode: `Tab`/`Shift+Tab` or `↑/↓` to move fields, `←/→` to change database type, `Enter` to connect. ### 2. Browser View Navigate schemas and tables in a visual grid layout. | Key | Action | |-------------------------|--------------------| | `↑/k` `↓/j` `←/h` `→/l` | Navigate grid | | `/` or `f` | Filter tables | | `Ctrl+S` | Select schema | | `Enter` | View table data | | `Ctrl+E` | Open SQL editor | | `Ctrl+H` | View query history | | `Ctrl+A` | Open AI chat | | `Ctrl+R` | Refresh table list | | `Tab` | Next view | | `Esc` | Disconnect | | `Ctrl+C` | Quit | ### 3. Editor View Write and execute SQL queries with schema-aware autocomplete. | Key | Action | |----------------------------------|----------------------| | `Alt+Enter` (`Option+Enter` Mac) | Execute query | | `Ctrl+Space` (`Ctrl+@`) | Trigger autocomplete | | `↑/↓` or `Ctrl+P/N` | Navigate suggestions | | `Enter` | Accept suggestion | | `Alt+L` | Clear editor | | `Tab` | Next view | | `Esc` | Back to browser | Features: - Schema-aware autocomplete (tables, columns, keywords, snippets) - Multi-line editing - Error display ### 4. Results View View query results in a responsive, paginated table. | Key | Action | |-------------|---------------------| | `↑/k` `↓/j` | Navigate rows | | `←/h` `→/l` | Scroll columns | | `n` | Next page | | `p` | Previous page | | `s` | Cycle page size | | `Shift+S` | Custom page size | | `e` | Export data | | `w` | Add WHERE condition | | `c` | Select columns | | `Esc` | Back | Features: - Pagination (configurable, default 50 rows) - Column selection - Data export (CSV, Excel) ### 5. History View Browse and re-execute past queries. | Key | Action | |-------------|------------------------| | `↑/k` `↓/j` | Navigate history | | `/` | Filter history | | `Enter` | Load query into editor | | `r` | Re-run query | | `Shift+D` | Clear history | | `y`/`n` | Confirm clear | | `Tab` | Next view | | `Esc` | Back | ### 6. AI Chat View AI-assisted database chat (requires a configured provider and consent). | Key | Action | |---------------|------------------------| | `↑/↓` | Cycle fields | | `←/→` | Change selection | | `Ctrl+L` | Load models | | `Ctrl+I` | Focus message input | | `Ctrl+P/N` | Select message | | `Enter` | Confirm/send/view | | `Ctrl+R` | Revoke consent | | `Esc` | Back to browser | Consent gate: press `a` to accept or `Esc` to exit. ### 7. Export View Export data to CSV or Excel format. | Key | Action | |-------------|--------------------------------| | `Tab`/`↑/↓` | Move between fields | | `←/→` | Change format/delimiter/toggle | | `Enter` | Confirm export | | `Esc` | Cancel | ## Configuration ### Config File Location WhoDB CLI stores data in the unified WhoDB config: - macOS: `~/Library/Application Support/whodb/config.json` - Linux: `$XDG_DATA_HOME/whodb/config.json` (default: `~/.local/share/whodb/config.json`) - Windows: `%APPDATA%\\whodb\\config.json` Development builds append `-dev` to the data directory name, and EE builds append `-ee`. Query history is stored alongside the config as `history.json`. ### Config Structure ```json { "cli": { "connections": [ { "name": "local-postgres", "type": "Postgres", "host": "localhost", "port": 5432, "username": "postgres", "database": "mydb", "schema": "public" } ], "history": { "max_entries": 1000, "persist": true }, "display": { "theme": "dark", "page_size": 50 }, "ai": { "consent_given": false }, "query": { "timeout_seconds": 30 } } } ``` Database and SSH passwords are stored in the OS keyring. If it is unavailable, saving passwords requires explicitly setting `WHODB_CLI_ALLOW_PLAINTEXT_CREDENTIALS=true`; they are then written to `config.json` (new files are created with `0600` permissions). A failed keyring write aborts saving. Hosted login refresh tokens always require secure storage. ### Environment Variables ```bash export WHODB_CLI_DEBUG=true export WHODB_CLI_NO_COLOR=true export NO_COLOR=1 ``` ## Docker Usage ### Run Interactively ```bash docker run -it --rm whodb-cli:latest ``` ### Connect to Host Database ```bash docker run -it --rm --network host whodb-cli:latest connect \ --type postgres \ --host localhost \ --user postgres \ --database mydb ``` ### Execute Query ```bash docker run -it --rm --network host whodb-cli:latest query "SELECT version()" ``` ### Persist Configuration ```bash docker run -it --rm \ -v ~/.local/share/whodb:/root/.local/share/whodb \ --network host \ whodb-cli:latest ``` ### Environment Variables - `TERM=xterm-256color` - Proper terminal colors (set by default) - `WHODB_CLI_DEBUG` / `WHODB_CLI_NO_COLOR` / `NO_COLOR` - Control CLI output ## Keyboard Reference Card ### Global | Key | Action | |----------|-----------| | `Ctrl+C` | Quit | | `Esc` | Go back | | `?` | Show help | ### Navigation (Vim-style) | Key | Action | |------------|---------------| | `↑` or `k` | Up | | `↓` or `j` | Down | | `←` or `h` | Left | | `→` or `l` | Right | | `/` | Filter/Search | ### Common Actions | Key | Action | |---------|----------------| | `Enter` | Select/Execute | | `r` | Refresh/Re-run | | `e` | Edit/Export | | `n` | New/Next | | `p` | Previous | | `d` | Delete | ## Troubleshooting ### Connection Issues **"Cannot connect to database"** ```bash # Verify database is running pg_isready -h localhost -p 5432 # PostgreSQL mysql -h localhost -u root -p # MySQL ``` **"Plugin not found"** Supported database types: `postgres`, `mysql`, `sqlite`, `mongodb`, `redis`, `clickhouse`, `elasticsearch`, `mariadb` ### Display Issues **"Garbled text / incorrect colors"** ```bash # Set terminal type export TERM=xterm-256color ``` Recommended terminals: iTerm2, Alacritty, Windows Terminal, Kitty ### Configuration Issues **"Config not found"** ```bash mkdir -p ~/.local/share/whodb whodb connect --type postgres --host localhost --name test ``` Adjust the path for your OS (see Configuration). **"Permissions error"** ```bash chmod 700 ~/.local/share/whodb chmod 600 ~/.local/share/whodb/config.json ~/.local/share/whodb/history.json ``` ### Debug Mode ```bash whodb --debug ``` ## Architecture ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Development ```bash # Run in development mode go run . # Run tests go test ./... # Build with race detector go build -race -o whodb . # Lint golangci-lint run ./... ``` ## License Apache License 2.0 - See LICENSE file for details. ### HTTP MCP authentication and credential storage HTTP MCP listeners bound outside loopback require `--auth-token` or `WHODB_MCP_AUTH_TOKEN`. Clients send that value as a bearer token on `/mcp`. Loopback HTTP and stdio retain their local defaults. Saving database or SSH passwords requires the OS keyring. If it is unavailable, plaintext storage requires the explicit environment setting `WHODB_CLI_ALLOW_PLAINTEXT_CREDENTIALS=true`; the config file has mode 0600. A failed keyring write aborts saving instead of copying passwords to disk. Hosted login refresh tokens always require secure storage. Platform collection tools use server-side metadata pages when the host advertises `PlatformResources`: 50 items by default, at most 100, with `offset` and `truncated` indicating continuation. Filters run before pagination and access checks run before rows are selected. Use the corresponding detail tool for full source, schema, bindings, or usage. Explicit function, ontology, dataset, and app fields can fetch details for the selected page. Older hosts retain their existing list contract. Platform wait tools return the latest observed running status with a warning when their wait budget expires; repeat with the same run ID. Write idempotency keys are limited to 200 bytes and scoped to the account, workspace, and operation. Matching completed requests replay bounded identifiers/status for 30 days; secret values and job output are never retained for replay. Conflicting payloads are rejected. --- ### Overview (`desktop-common/README.md`) # WhoDB Desktop Application This is the WhoDB desktop application built with [Wails](https://wails.io/), providing a native window experience while maintaining full compatibility with the web version. Available in both Community Edition (CE) and Enterprise Edition (EE). ## Prerequisites 1. **Install Wails CLI** (version is pinned in `desktop-common/go.mod`): ```bash make install-wails ``` 2. **Check dependencies:** ```bash wails doctor ``` 3. **Platform requirements:** - **Windows:** WebView2 (pre-installed on Windows 10/11) - **macOS:** Xcode Command Line Tools (`xcode-select --install`) - **Linux:** `gcc`, `libgtk-3`, `libwebkit2gtk-4.1` ## Quick Start ### Running the Desktop Application Navigate to the edition directory and use make commands: ```bash # Community Edition cd desktop-ce make dev # Development mode with hot reload make build # Build for current platform make help # See all available commands # Enterprise Edition cd ee/desktop make dev # Development mode with EE features make build # Build for current platform make help # See all available commands ``` ## Build Commands ### Development & Basic Builds ```bash make dev # Run in development mode with hot reload make build # Build for current platform make build-all # Build for all platforms make build-windows # Windows AMD64 & ARM64 make build-mac # macOS Universal Binary make build-mac-arm64 # macOS ARM64 only (Apple Silicon) make build-mac-amd64 # macOS AMD64 only (Intel) make build-linux # Linux AMD64 & ARM64 ``` ### Production Builds ```bash make build-prod-windows # Windows NSIS installer with UPX make build-prod-mac # macOS Universal Binary make build-prod-mac-arm64 # macOS ARM64 only make build-prod-mac-amd64 # macOS AMD64 only make build-prod-linux # Linux binary with UPX compression ``` ### macOS Packaging & Signing ```bash # Create unsigned .pkg make package-mac # Sign app and .pkg make package-mac-signed \ CODESIGN_ID="Developer ID Application: Your Name (TEAMID)" \ INSTALLER_ID="Developer ID Installer: Your Name (TEAMID)" # Notarize for distribution make notarize-mac \ NOTARY_PROFILE="WhoDBNotary" # or make notarize-mac \ APPLE_ID="appleid@example.com" \ TEAM_ID="TEAMID" \ APPLE_APP_PASSWORD="app-specific-password" # One-shot GitHub Release (build + sign + notarize) make release-mac \ CODESIGN_ID="..." \ INSTALLER_ID="..." \ NOTARY_PROFILE="WhoDBNotary" # Create DMG for drag-and-drop install make dmg-mac make release-dmg # DMG with signing and notarization ``` ### Mac App Store Build ```bash make macstore-mac \ MAS_CODESIGN_ID="Apple Distribution: Your Name (TEAMID)" \ MAS_INSTALLER_ID="3rd Party Mac Developer Installer: Your Name (TEAMID)" \ MAS_PROFILE=/path/to/embedded.provisionprofile \ MAS_ENTITLEMENTS=/path/to/entitlements.plist ``` ## Output Files Binaries are generated in `build/` organized by platform: ### Community Edition (desktop-ce) - Windows: `build/windows/[arch]/whodb-ce.exe` - macOS Universal: `build/darwin/universal/WhoDB.app` - macOS ARM64: `build/darwin/arm64/WhoDB.app` - macOS AMD64: `build/darwin/amd64/WhoDB.app` - Linux: `build/linux/[arch]/whodb-ce` ### Enterprise Edition (ee/desktop) - Windows: `build/windows/[arch]/whodb-ee.exe` - macOS Universal: `build/darwin/universal/WhoDB - Enterprise.app` - macOS ARM64: `build/darwin/arm64/WhoDB - Enterprise.app` - macOS AMD64: `build/darwin/amd64/WhoDB - Enterprise.app` - Linux: `build/linux/[arch]/whodb-ee` **Note:** Architecture-specific macOS builds (`arm64`/`amd64`) are optimized single-architecture builds that will be smaller than the universal binary (~50% size reduction) since they only contain code for one architecture. ## Manual Build (Advanced) If you need to build manually without Make: ```bash # Community Edition cd desktop-ce GOWORK=$PWD/../go.work.desktop-ce wails build -o whodb-ce # Enterprise Edition cd ee/desktop wails build -o whodb-ee ``` ## Architecture The desktop application: 1. Uses the same Go backend as the web version 2. Embeds the React frontend build 3. Serves through a native window instead of browser 4. Routes all API calls through the same Chi router/GraphQL endpoint 5. Maintains full compatibility with all database plugins 6. EE includes all enterprise features and additional plugins ### Workspace Structure - **CE workspace** (`go.work.desktop-ce`): - `./core` - Core WhoDB functionality - `./desktop-ce` - CE desktop module - `./desktop-common` - Shared desktop code - **EE workspace** (`ee/go.work.desktop`): - `../core` - Core WhoDB functionality - `.` - Enterprise Edition modules (ee) - `./desktop` - EE desktop module - `../desktop-common` - Shared desktop code ## Development Tips - Use `make dev` for hot reload during development - The desktop app shares the same codebase as the web version - BrowserRouter works seamlessly with Wails - GraphQL API calls work identically to the web version - All database plugins work without modification ## Troubleshooting ### Module Not Found Errors ```bash # For CE cd desktop-ce GOWORK=$PWD/../go.work.desktop-ce go mod tidy # For EE cd ee/desktop && go mod tidy ``` ### Platform-Specific Issues Run `wails doctor` to check dependencies and ensure platform tools are installed. ## Useful Commands ```bash make clean # Remove all build artifacts make doctor # Check Wails dependencies make install-wails # Install/update Wails CLI make help # Show all available commands ``` ## Versioning Set version for builds and packages: ```bash make build-mac VERSION=1.2.3 make package-mac VERSION=1.2.3 ``` If VERSION is not specified, a timestamp is used. ## Code Signing Options ### Using Sigstore (Recommended for Open Source) ```bash # Install cosign go install github.com/sigstore/cosign/v2/cmd/cosign@latest # Sign binary cosign sign-blob build/bin/whodb-ce-windows-amd64.exe \ --output-signature whodb-ce-windows-amd64.exe.sig \ --output-certificate whodb-ce-windows-amd64.exe.crt ``` ### Platform-Specific Signing See the macOS packaging section above for Apple Developer ID signing, or use Windows EV certificates with Wails build flags. ## Notes - The Makefile system automatically handles frontend building and asset preparation - Both editions share common code via the `desktop-common` module - The `-obfuscated` flag is currently disabled due to hanging issues - Linux builds use `webkit2_41` tag for better compatibility --- ### Overview (`sdk/README.md`) # WhoDB Platform SDKs Customer-facing SDKs for the WhoDB hosted platform: ontology, datasets, sources, transforms, functions, versioning and promotion, service accounts, lineage providers, and ontology sharing as in-code function APIs. All five languages expose the same domains and methods; the shared fixtures in `spec/fixtures/` keep them behaviorally identical. | Package | Registry | Path | |---|---|---| | `@clidey/whodb-sdk` | npm | `packages/typescript/` | | `whodb-sdk` | PyPI | `packages/python/` | | `github.com/clidey/whodb/sdk/packages/go` | Go modules (git tag) | `packages/go/` | | `whodb-sdk` | crates.io | `packages/rust/` | | `com.clidey:whodb-sdk` | Maven Central | `packages/java/` | Design: `SDK_DESIGN.md` in the EE repository. Architecture in one line: a **generated wire core** (rendered from the committed spec snapshot in `spec/`) under a **handwritten facade** per language, kept behaviorally identical across languages by shared conformance fixtures. ## Layout ``` spec/ # committed sync boundary with the platform backend platform-schema.graphql # merged SDL (generated — ee/dev/export-sdk-spec.sh) platform-manifest.json # curated public operation list (generated) surface.yaml # hand-curated operation → facade mapping fixtures/ # cross-language behavior fixtures tools/ generate-core.mjs # spec → per-language wire cores (--check = drift gate) render/ # per-language renderers conformance-runner.mjs # runs fixtures against a language (--lang ts|python|go|rust|java) sync-versions.mjs # lockstep version stamping (--set X.Y.Z / --check) smoke.mjs # pre-release staging round-trip (real API key) packages/ # the SDKs ``` ## Workflows Backend API changed? (schema or platform manifest) ```bash bash ee/dev/export-sdk-spec.sh # re-export the spec snapshot node sdk/tools/generate-core.mjs # regenerate wire cores # commit both — EE and CE CI both gate on drift ``` Verify everything: ```bash cd sdk pnpm install pnpm run check # generate --check + version lockstep pnpm -r --if-present run build pnpm -r test node tools/conformance-runner.mjs --lang ts # also: python, go, rust, java ``` Releases ride `release-ce.yml` (the `deploy-sdk` toggle) and share the repo release version across all five targets: npm (`_deploy-sdk-npm.yml`) plus the inlined `deploy-sdk-pypi`, `deploy-sdk-cargo`, and `deploy-sdk-maven` jobs (inlined because PyPI attestations and crates.io trusted publishing validate the top-level workflow), and `deploy-sdk-go-tag`, which tags `sdk/packages/go/v` only after all four registry publishes succeed. Adding a language: `docs/porting.md`. Adding a domain: extend `spec/surface.yaml`, add fixtures, then add the facade to every language and its conformance runner in the same change. --- ### Overview (`.agents/hooks/README.md`) # Agent Hooks Hooks run after tool actions (edit, write, apply_patch) and help keep code consistent. They receive a JSON payload on stdin containing the tool input. ## Hook Scripts | Script | Trigger | What It Does | |--------|---------|--------------| | `auto-format-go.sh` | `.go` files edited | Runs `gofmt -w` then `golangci-lint run --fix` | | `auto-lint-ts.sh` | `.ts`/`.tsx` files edited | Runs `oxlint --fix` | | `auto-graphql-codegen.sh` | `.graphqls` files edited | Runs `go generate` in `core/` and `pnpm generate` in `frontend/` | | `verify-build.sh` | Agent stop (turn end) | Compiles/type-checks only the areas changed this turn (CE/EE Go via `go build`, CE/EE frontend via `pnpm typecheck`) and blocks the turn (exit 2) if any fail. Debounces repeated identical errors to avoid loops. | | `session-context.sh` | Session start | Injects branch name and uncommitted file count | Translation is intentionally not an automatic hook. Agents should add or update `en_US` strings only; run `dev/translate` tooling manually when translations are needed. ## Supporting Files - `changed-files.py` — parses the JSON payload to extract file paths from edit/write/apply_patch hooks ## Platform Compatibility These hooks use the Codex/Claude Code hook format (JSON on stdin). Pi uses an extension-based hook system instead — wrap these scripts in a Pi extension for equivalent functionality. ## Adding a New Hook 1. Write the script in this directory (root `.agents/hooks/`). 2. Add a forwarding shim at `ee/.agents/hooks/` so the hook still resolves when the agent's cwd is inside the `ee/` submodule (its own git repo, so `git rev-parse --show-toplevel` resolves to `ee/` rather than the monorepo root). Match the existing shim pattern exactly: ```bash #!/usr/bin/env bash exec "$(cd "$(dirname "$0")/../../.." && pwd)/.agents/hooks/" "$@" ``` Then `chmod +x` it. 3. Register the command in both `.claude/settings.json` and `.codex/hooks.json` using `bash "$(git rev-parse --show-toplevel)/.agents/hooks/"` under the appropriate event (`SessionStart`, `PostToolUse`, `Stop`, etc.). Both files must be updated — neither tool reads the other's config. 4. Update the table above with the script's trigger and behavior. --- ### Overview (`cli/external-plugin/whodb/README.md`) # WhoDB Plugin for Claude Code WhoDB platform and database-only MCP tools for Claude Code. Work across platform hosts and projects, or query databases directly. ## Installation Methods This plugin supports multiple installation methods. Choose the one that works best for you: ### Method 1: npm (Recommended - No pre-install needed) Run `whodb setup` to enable platform tools, database-only tools, or both. Keep one MCP entry at `whodb mcp serve`; agents can guide setup with `whodb setup inspect`, `validate`, `apply`, and `verify`. Sign in to platform hosts through `whodb login --host ` and target workspaces per call. Docker examples below use an explicit database-only override; saved setup applies to the configuration directory of the running CLI. The default configuration uses npx to auto-download and run the MCP server: ```json { "whodb": { "command": "npx", "args": ["-y", "whodb", "mcp", "serve"] } } ``` ### Method 2: Docker (Database-only MCP) For database-only MCP access through Docker, update your Claude settings (`.claude/settings.local.json`): ```json { "mcpServers": { "whodb": { "command": "docker", "args": ["run", "-i", "--rm", "--network", "host", "clidey/whodb-cli", "mcp", "serve", "--database"] } } } ``` ### Method 3: Local Binary (Best performance) Install the CLI once, then it runs instantly: 1. Download from [GitHub Releases](https://github.com/clidey/whodb/releases) 2. Add to your PATH 3. Update Claude settings: ```json { "mcpServers": { "whodb": { "command": "whodb", "args": ["mcp", "serve"] } } } ``` ### Method 4: Homebrew (macOS/Linux) ```bash brew install whodb-cli ``` The formula is named `whodb-cli` (the `whodb` name is used by the desktop app cask), but it installs the `whodb` command. ### Method 5: Go Install (Requires Go) ```bash go install github.com/clidey/whodb/cli@latest ``` ## Supported Databases - PostgreSQL - MySQL / MariaDB - SQLite - MongoDB - Redis - ClickHouse - Elasticsearch - And more via WhoDB plugins ## Manual CLI installation For the local binary method, install the CLI using one of these options: ### Option 1: Download Binary Download the latest release from [GitHub Releases](https://github.com/clidey/whodb/releases) and add it to your PATH. ### Option 2: Build from Source ```bash git clone https://github.com/clidey/whodb.git cd whodb/cli go build -o whodb . sudo mv whodb /usr/local/bin/ ``` ### Option 3: Docker If you prefer Docker, update your Claude Code settings to use: ```json { "whodb": { "command": "docker", "args": ["run", "-i", "--rm", "--network", "host", "clidey/whodb-cli", "mcp", "serve", "--database"] } } ``` ## Configuration ### Database-only MCP connections These connection settings apply to `whodb mcp serve --database`. Platform mode uses the sources configured in the selected platform workspace. Configure database connections using environment variables: ```bash # Environment profiles (examples below) export WHODB_POSTGRES='[{"alias":"prod","host":"localhost","user":"user","password":"pass","database":"mydb","port":"5432"}]' export WHODB_MYSQL_1='{"alias":"dev","host":"localhost","user":"user","password":"pass","database":"devdb","port":"3306"}' ``` Or use the CLI to save connections: ```bash whodb connect --type postgres --host localhost --port 5432 --user myuser --database mydb --name prod ``` ## MCP Tools This plugin provides the following tools: | Tool | Description | |------|-------------| | `whodb_connections` | List available database connections | | `whodb_schemas` | List database schemas/namespaces | | `whodb_tables` | List tables in a schema (supports `include_columns` for inline column details) | | `whodb_columns` | Describe table columns and types | | `whodb_query` | Execute SQL queries (security-validated, supports parameterized queries) | | `whodb_explain` | Run EXPLAIN for a SQL query | | `whodb_diff` | Compare schema metadata between two connections | | `whodb_erd` | Load graph/relationship metadata for a schema | | `whodb_audit` | Run data quality checks on one schema or table | | `whodb_suggestions` | Load backend-generated starter queries | | `whodb_confirm` | Confirm pending write operations (only with confirm-writes mode) | | `whodb_pending` | List pending write confirmations awaiting approval | ## Security Modes By default, write operations require confirmation before executing (confirm-writes mode). | Flag | Description | |------|-------------| | *(default)* | Confirm-writes: all writes require user confirmation | | `--safe-mode` | Read-only + strict security (for demos and playgrounds) | | `--read-only` | Read-only: SELECT, SHOW, DESCRIBE, EXPLAIN only | | `--allow-write` | Full write access without confirmation (use with caution) | | `--security=strict` | Blocks dangerous functions (pg_read_file, COPY, LOAD_FILE, etc.) | | `--security=standard` | Basic validation (default) | | `--security=minimal` | Only blocks DELETE without WHERE (when writes allowed) | Additional options: | Flag | Description | |------|-------------| | `--timeout=30s` | Query timeout duration | | `--max-rows=100` | Limit rows returned per query (0 = unlimited) | | `--allow-drop` | Allow DROP/TRUNCATE even with `--allow-write` | | `--allow-multi-statement` | Allow multiple SQL statements in one query | | `--tools=schemas,tables` | Comma-separated list of tools to enable (default: all) | | `--disable-tools=query` | Comma-separated list of tools to disable | | `--default-connection=prod` | Set default connection (no access restriction) | | `--allowed-connections=prod,staging` | Restrict access to listed connections only | | `--transport=http` | Run as HTTP service instead of stdio | | `--host=localhost` | Bind address (HTTP mode only) | | `--port=3000` | Listen port (HTTP mode only) | | `--no-analytics` | Disable anonymous usage analytics | ## Skills ### whodb Main database skill - activates for any database-related task. ### query-builder Natural language to SQL conversion. Activates when you ask questions like: - "Show me users who signed up last week" - "Find orders over $100" - "Get the top 10 customers" ### schema-designer Database schema design assistance. Activates when you ask to: - Create tables - Design schemas - Model data relationships ## Agents ### database-analyst Deep database analysis expert. Use for: - Schema analysis and documentation - Data quality assessment - Multi-step data exploration **Invoke:** "Use the database-analyst agent to analyze my schema" ### query-optimizer Query performance specialist. Use for: - Slow query analysis - Index recommendations - Query rewriting **Invoke:** "Use the query-optimizer agent to optimize this query" ### report-generator Data reporting specialist. Use for: - Formatted reports from queries - Data summaries and trends - Export preparation **Invoke:** "Use the report-generator agent to create a sales report" ## Examples ``` # Explore a database Show me the tables in my prod database # Query data Find all users created in the last 7 days # Analyze performance Use the query-optimizer agent to analyze: SELECT * FROM orders WHERE status = 'pending' # Design schema Help me design a schema for a blog with posts, comments, and tags ``` ## Links - [WhoDB Repository](https://github.com/clidey/whodb) - [WhoDB Documentation](https://github.com/clidey/whodb#readme) - [CLI Documentation](https://github.com/clidey/whodb/tree/main/cli#readme) ## License Apache License 2.0 ### MCP without skills Skills are optional supplements. MCP initialization instructions, tool schemas, and `whodb://mcp/configuration` provide essential guidance directly. Use `whodb_mcp_setup` actions `inspect`, `preview` (with a settings patch), `apply` (with the exact approved preview token and `approved: true`), and `verify`. No shell access is required for these actions. Browser login and credential entry remain user actions. Restart after changing settings; verification checks the active process. The setup tool follows tool filters and can be disabled when configuration should remain outside the agent. --- ### Overview (`sdk/packages/go/README.md`) # whodb (Go SDK) Official Go SDK for the [WhoDB](https://whodb.com) hosted platform — your ontology, datasets, sources, transforms, functions, and org administration as in-code function APIs. ## Install ```bash go get github.com/clidey/whodb/sdk/packages/go ``` Requires Go ≥ 1.27.3. ## Quickstart ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Authentication Credential precedence: `Config` fields (`APIKey` / `Token` / `TokenFunc` / `Credentials`) → `WHODB_API_KEY` env var → the `whodb` CLI's stored login. Workspace (`Org` / `Project`) is optional with an API key; set `Project` when the key has access to more than one project. Inside a WhoDB Function, `whodb.New(whodb.Config{})` auto-detects the runtime and needs no configuration at all. ## Errors Sentinel errors for `errors.Is`: `ErrAuth`, `ErrNotFound`, `ErrValidation`, `ErrVersion` (SDK outdated for the platform API — upgrade the module), `ErrCliCredentials`, `ErrTransportCapability`; plus `*PlatformError` (via `errors.As`) carrying the platform's error code. ## License Apache-2.0 --- ### Overview (`sdk/packages/java/README.md`) # whodb-sdk (Java) Official Java SDK for the [WhoDB](https://whodb.com) hosted platform — your ontology, datasets, sources, transforms, functions, and org administration as in-code function APIs. ## Install ```xml com.clidey whodb-sdk LATEST ``` Requires Java ≥ 17. One dependency (`jackson-databind`); HTTP via the JDK's built-in `java.net.http.HttpClient`. ## Quickstart ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Rows are `Map` with native-typed values; timestamps hydrate to `java.time.OffsetDateTime`. ## Authentication Credential precedence: `Config` fields (`apiKey` / `token` / `credentials`) → `WHODB_API_KEY` env var → the `whodb` CLI's stored login. Workspace (`org` / `project`) is optional with an API key; set `project` when the key has access to more than one project. Inside a WhoDB Function, `new WhoDB()` auto-detects the runtime and switches to the in-container IPC transport (unix socket in Docker, TCP in K8s). ## Errors All failures throw `WhoDBException`; `kind()` identifies the class: `AUTH`, `NOT_FOUND`, `VALIDATION`, `VERSION` (SDK outdated for the platform API — upgrade the artifact), `CLI_CREDENTIALS`, `TRANSPORT_CAPABILITY`, `PLATFORM` (with `code()`), `TRANSPORT`. ## License Apache-2.0 --- ### Overview (`sdk/packages/python/README.md`) # whodb-sdk Official Python SDK for the [WhoDB](https://whodb.com) hosted platform — your ontology, datasets, sources, transforms, functions, and org administration as in-code function APIs. ## Install ```bash pip install whodb-sdk ``` Requires Python ≥ 3.10. ## Quickstart ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Authentication Credential precedence: constructor args (`api_key=` / `token=` / `credentials=`) → `WHODB_API_KEY` env var → the `whodb` CLI's stored login. Workspace (`org=` / `project=`) is optional with an API key; pass `project=` when the key has access to more than one project. Inside a WhoDB Function, `WhoDB()` auto-detects the runtime and needs no configuration at all. ## Typed clients ```bash whodb sdk generate --language python --out whodb_gen/ ``` ## License Apache-2.0 --- ### Overview (`sdk/packages/rust/README.md`) # whodb-sdk (Rust) Official Rust SDK for the [WhoDB](https://whodb.com) hosted platform — your ontology, datasets, sources, transforms, functions, and org administration as in-code function APIs. ## Install ```bash cargo add whodb-sdk ``` Requires Rust ≥ 1.75. Synchronous API; two dependencies (`serde_json`, `ureq`). ## Quickstart ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Rows are `serde_json::Map` with native-typed values. Hydrated timestamps carry an `@date:` string marker (Rust has no std datetime type) — parse with your preferred time crate. ## Authentication Credential precedence: `Config` fields (`api_key` / `token` / `credentials`) → `WHODB_API_KEY` env var → the `whodb` CLI's stored login. Workspace (`org` / `project`) is optional with an API key; set `project` when the key has access to more than one project. Inside a WhoDB Function (K8s/TCP runtime), `Client::new(Config::default())` auto-detects the runtime. The Docker runtime's unix-socket IPC is not yet supported by this crate. ## Errors `whodb_sdk::Error` variants: `Auth`, `NotFound`, `Validation`, `Version` (SDK outdated for the platform API — upgrade the crate), `CliCredentials`, `TransportCapability`, `Platform { message, code }`, `Transport`. ## License Apache-2.0 --- ### Overview (`sdk/packages/typescript/README.md`) # @clidey/whodb-sdk Official TypeScript/JavaScript SDK for the [WhoDB](https://whodb.com) hosted platform — your ontology, datasets, sources, transforms, functions, and org administration as in-code function APIs. ## Install ```bash npm install @clidey/whodb-sdk ``` Requires Node.js ≥ 20. ## Quickstart ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Authentication Credential precedence: 1. `new WhoDB({ apiKey })` / `{ token }` / `{ credentials }` constructor options 2. `WHODB_API_KEY` environment variable 3. The `whodb` CLI's stored login (`whodb login`), gcloud-ADC-style — ideal for local development Workspace (`org` / `project`) is optional with an API key; pass `{ project }` or set `WHODB_PROJECT` when the key has access to more than one project. ## Typed clients Generate typed entity classes from your project's ontology: ```bash whodb sdk generate --language ts --out src/whodb-gen/ ``` ```ts import { createClient } from './whodb-gen/whodb.generated.js'; const { ontology } = createClient({ apiKey: process.env.WHODB_API_KEY }); const user = await ontology.User.get('u_123'); // fully typed ``` ## Versioning Each SDK release is generated against one WhoDB platform release. Deprecated operations warn once per process with their removal date; operations removed from the platform raise `WhoDBVersionError` — upgrade the package to resolve. ## License Apache-2.0 --- ### Overview (`cli/npm-package/packages/whodb-cli/README.md`) # @clidey/whodb-cli WhoDB CLI - a powerful database management tool with interactive TUI, programmatic commands, and MCP server for AI assistants. Installing this package provides the `whodb` command. If you'd rather use a namespaced package name, install [`@clidey/whodb`](https://www.npmjs.com/package/@clidey/whodb) instead — both provide the same command. ## Installation ```bash # Run directly (no install required) npx @clidey/whodb-cli --help # Or install globally npm install @clidey/whodb-cli ``` Both install methods provide the `whodb` command. ## Features - **Interactive TUI**: Full terminal UI for database management - **Programmatic Commands**: Query, export, and manage databases from scripts - **MCP Server**: Integrate with Claude Desktop, Claude Code, and other MCP clients - **Assistant Integrations**: Install ready-to-use MCP configs for Codex, Claude Code, Cursor, VS Code, GitHub Copilot CLI, Gemini CLI, Windsurf, OpenCode, Cline, Zed, Continue, and aider ## Supported Databases - PostgreSQL - MySQL / MariaDB - SQLite - MongoDB - Redis - Elasticsearch - ClickHouse ## Usage ### Interactive Mode (TUI) ```bash npx @clidey/whodb --tui ``` ### Programmatic Commands ```bash # Query a database npx @clidey/whodb query "SELECT * FROM users LIMIT 10" --connection mydb # List schemas npx @clidey/whodb schemas --connection mydb --format json # List tables npx @clidey/whodb tables --connection mydb --schema public # Describe columns npx @clidey/whodb columns --connection mydb --table users # Export data npx @clidey/whodb export --connection mydb --table users --format csv --output users.csv ``` ### MCP Server Mode Platform MCP is the default. Sign in to each host, then start the server: ```bash npx @clidey/whodb login --host npx @clidey/whodb mcp serve ``` Use `workspace: {host, org, project}` on platform tool calls to target a specific workspace without changing saved defaults. For database-only tools, enable the database module using `npx @clidey/whodb setup`. The server command stays the same: ```bash npx @clidey/whodb mcp serve ``` Write operations require confirmation by default. Use `--allow-write` to disable confirmations or `--read-only` to block writes. Restrict AI access to specific connections with `--allowed-connections`: ```bash npx @clidey/whodb mcp serve --database --allowed-connections prod,staging ``` ## Database-only MCP Client Configuration Example configuration (from `whodb mcp serve --database --help`): ```json { "mcpServers": { "whodb": { "command": "whodb", "args": ["mcp", "serve"], "env": { "WHODB_POSTGRES_1": "{\"alias\":\"prod\",\"host\":\"localhost\",\"user\":\"user\",\"password\":\"pass\",\"database\":\"db\"}" } } } } ``` ## Assistant Integration Installer Install native assistant configuration with the bundled CLI: ```bash npx @clidey/whodb skills install --target cursor npx @clidey/whodb skills install --target vscode npx @clidey/whodb skills install --target gemini-cli npx @clidey/whodb skills install --target claude-code --include-agents npx @clidey/whodb skills install --target cursor --dry-run ``` Supported targets: `codex`, `claude-code`, `cursor`, `vscode`, `github-copilot`, `gemini-cli`, `windsurf`, `opencode`, `cline`, `zed`, `continue`, and `aider`. Existing JSON and JSONC configuration files are merged in place and rewritten as formatted JSON. Before an existing JSON or YAML assistant config is rewritten, the original file is saved beside it as `.whodb.bak`. Use `--dry-run` to preview created or updated files and any backup paths without writing changes. ## Environment Variables Configure database connections via environment profiles, for example `WHODB_POSTGRES='[{"alias":"prod","host":"localhost","user":"user","password":"pass","database":"db","port":"5432"}]'` or `WHODB_MYSQL_1='{"alias":"dev","host":"localhost","user":"user","password":"pass","database":"devdb","port":"3306"}'`. Each object supports `alias` (connection name), `host`, `user`, `password`, `database`, `port`, and optional `config`. | Variable | Description | |----------|-------------| | `WHODB_` | JSON array of credential objects for a database type | | `WHODB__N` | JSON object for a single credential (numbered profiles) | Use the `alias` field to set the connection name (e.g., `prod`). If omitted, the CLI assigns a name like `postgres-1`. ## Available MCP Tools | Tool | Description | |------|-------------| | `whodb_connections` | List available database connections | | `whodb_schemas` | List schemas in a database | | `whodb_tables` | List tables in a schema | | `whodb_columns` | Get column details for a table | | `whodb_query` | Execute SQL queries | | `whodb_confirm` | Confirm pending write operations (only when confirm-writes is enabled) | ## Platform Support This package automatically installs the correct binary for your platform: - macOS (Intel and Apple Silicon) - Linux (x64, ARM64, ARMv7) - Windows (x64, ARM64) ## Links - [GitHub Repository](https://github.com/clidey/whodb) - [Documentation](https://github.com/clidey/whodb#readme) ## License Apache-2.0 --- ### Overview (`cli/npm-package/packages/whodb/README.md`) # whodb The [WhoDB](https://whodb.com) CLI — an interactive, production-ready command-line interface for navigating SQL and NoSQL databases, with a TUI, programmatic commands, and an MCP server for AI assistants. ## Use `@clidey/whodb` instead This unscoped package is a thin wrapper kept published so the name isn't squatted by someone else. The primary, namespaced package is [`@clidey/whodb`](https://www.npmjs.com/package/@clidey/whodb): ```bash npm install @clidey/whodb whodb --help ``` Installing this package also works and provides the same `whodb` command — it depends on and forwards to [`@clidey/whodb-cli`](https://www.npmjs.com/package/@clidey/whodb-cli), same as `@clidey/whodb` does. Prefer the scoped name in new setups. ## Documentation Full CLI documentation, command reference, and MCP setup: https://github.com/clidey/whodb/tree/main/cli#readme --- ### Overview (`cli/npm-package/packages/@clidey/whodb/README.md`) # @clidey/whodb The [WhoDB](https://whodb.com) CLI — an interactive, production-ready command-line interface for navigating SQL and NoSQL databases, with a TUI, programmatic commands, and an MCP server for AI assistants. This is the primary namespaced package for installing WhoDB's CLI: ```bash npm install @clidey/whodb whodb --help ``` ## What this package is This is a thin wrapper: all the real work — the launcher, the platform-specific binaries, and the BAML runtime — lives in [`@clidey/whodb-cli`](https://www.npmjs.com/package/@clidey/whodb-cli), which this package depends on and forwards to. Installing this package gives you the `whodb` command. > Renamed from `whodb-cli`: the command used to be `whodb-cli`. If you're > upgrading, `@clidey/whodb-cli` still ships a deprecated `whodb-cli` binary > that prints a warning and forwards to `whodb` — update scripts and configs > to use `whodb` going forward. ## Features - **Interactive TUI** — terminal UI with split-pane layouts, themes, and an SQL editor with schema-aware autocomplete - **Multi-database support** — PostgreSQL, MySQL/MariaDB, SQLite, MongoDB, Redis, ClickHouse, Elasticsearch - **Programmatic mode** — JSON/NDJSON/CSV/plain output for scripting and CI - **Import/export** — CSV and Excel, plus FK-aware mock data generation - **Schema tools** — ERD graph output, EXPLAIN plans, schema diff between connections - **MCP server** — expose database access to AI assistants (Claude, Cursor, and others) with read-only and confirm-writes safety modes - **Assistant integration installer** — bundled skills, agents, and MCP configs for popular coding assistants ## Usage ```bash # Interactive TUI whodb --tui # Run a query whodb query "SELECT * FROM users LIMIT 10" --connection mydb # Configure platform, database-only, or both modules once npx @clidey/whodb setup # Start the single MCP server whodb mcp serve ``` Or without installing: ```bash npx @clidey/whodb ``` ## Documentation Full CLI documentation, command reference, and MCP setup: https://github.com/clidey/whodb/tree/main/cli#readme --- ### Query Scratchpad Intro (`docs/query/scratchpad-intro.mdx`) --- title: "Scratchpad Introduction" description: "WhoDB's Scratchpad is a notebook-style query workspace with pages, cells, and history for SQL, MongoDB, and Redis queries" --- # Scratchpad Introduction Scratchpad is WhoDB's query workspace for connections that support raw queries — SQL databases, plus shell-style commands for MongoDB and commands for Redis. It gives you page-based query organization, one or more query cells per page, query execution, result grids, action-query confirmations, and per-cell query history. Use Scratchpad for custom SQL. Use the Storage Unit data page when you want the guided grid, search, WHERE condition builder, row editing, export, and mock data actions. ## Open Scratchpad Open **Scratchpad** from the left sidebar. The route is `/scratchpad`. The current layout is vertical: - The SQL editor is at the top of each cell. - Query results or action status render below the editor. - Multiple cells stack vertically within the active page. - Scratchpad pages appear as tabs above the cells. ## Run Queries Type SQL into the editor and run it with any of: - The run button at the bottom-right of the cell. It appears on hover (or when the page has a single cell) and is hidden while the editor is empty. - The per-statement play buttons in the editor's left gutter, which run one statement at a time. - `Cmd/Ctrl + Enter`. `SELECT` queries show a results grid below the editor. Action queries such as `INSERT`, `UPDATE`, and `DELETE` show an action result after confirmation. WhoDB prompts before executing destructive SQL detected in Scratchpad. Review the SQL before confirming. ## Errors Database errors are shown below the editor for the cell that failed. Start with a small query such as `SELECT 1;`, then add complexity back until the error is isolated. ## Pages And Cells Scratchpad has two levels of organization: - **Pages** are tabs across the top. Use the plus tab to add a page. Double-click a page name to rename it. Use the close button on a page tab to delete it. - **Cells** are query editors inside a page. Hover over a cell to reveal the add, clear, delete, and run controls. The cell options menu currently contains **Clear**, **Copy Code**, and **Query History**. Use the visible page tabs and cell controls to organize Scratchpad work. ## Query History Open query history from the cell options menu or the history button on the cell. History is stored per cell. Entries show the query text, success/error status, and timestamp. From a history entry you can copy the SQL, load it back into the current editor, or run it again. Loading a history entry puts that SQL into the current editor. ## Editor Shortcuts | Shortcut | Action | |----------|--------| | `Cmd/Ctrl + Enter` | Run the current query or selected SQL | | `Cmd/Ctrl + U` | Clear the current editor | ## Related Pages Learn the editor behavior and SQL execution flow. See what history stores and how to reuse entries. --- ### Resources Comparisons Overview (`docs/resources/comparisons/overview.mdx`) --- title: "Database Tool Comparison Overview" description: "Compare WhoDB with popular database management tools. See which solution best fits your needs with our comprehensive comparison guide." --- # Database Tool Comparison Overview Choosing the right database management tool can significantly impact your productivity, workflow efficiency, and team collaboration. This comprehensive comparison guide helps you understand how WhoDB stacks up against popular alternatives and make an informed decision. ## Quick Comparison Table This comparison reflects the current CE build in this repository. | Feature | WhoDB | phpMyAdmin | pgAdmin | DBeaver | TablePlus | |---------|-------|------------|---------|---------|-----------| | **Price** | Free (Open Source) | Free | Free | Free / $199 | $89-$129 | | **Resource Usage** | Lightweight server | Moderate web stack | Heavier admin app | Heavy desktop IDE | Native desktop app | | **Startup Model** | Run server, open browser | Web/PHP app | Desktop or server app | Java desktop app | Native desktop app | | **Web-Based** | Yes | Yes | Desktop/Web | Desktop | Desktop | | **Multi-Database** | Yes (broad CE catalog) | MySQL only | PostgreSQL only | Yes (50+) | Yes (15+) | | **Schema Visualization** | Interactive Graph | No | Basic ERD | Basic ERD | Basic ERD | | **Modern UI** | Yes | No | No | Mixed | Yes | | **Query History** | Built-in | Basic | Basic | Advanced | Advanced | | **Mock Data Generation** | Yes | No | No | No | No | | **Search / Filtering** | Loaded-row search + WHERE filters | Limited | Limited | Yes | Yes | | **Multi-cell Queries** | Yes | No | No | Yes | No | | **Docker Support** | Excellent | Good | Good | N/A | N/A | | **Self-Hosted** | Yes | Yes | Yes | N/A | N/A | | **Learning Curve** | Very Low | Low | Medium | High | Low | ## Detailed Feature Comparison ### Performance and Resource Usage **Strengths:** - Lightweight server process - Browser-based workflow - Browser-based, no installation required - Lightweight Go backend - React frontend with efficient virtualization **Best For:** Teams prioritizing speed and efficiency, resource-constrained environments, Docker deployments **Strengths:** - Moderate resource usage - Fast page loads - Widely deployed and tested **Limitations:** - PHP runtime overhead - Limited to MySQL/MariaDB - Older architecture **Best For:** Legacy MySQL deployments, shared hosting environments **Strengths:** - Comprehensive PostgreSQL features - Active development - Good for complex operations **Limitations:** - Heavy resource usage (300MB+) - Slow startup time - Desktop application overhead - PostgreSQL-only **Best For:** PostgreSQL DBAs needing advanced features **Strengths:** - Supports 50+ database types - Advanced query tools - Plugin ecosystem **Limitations:** - Heavier desktop runtime - More startup and workspace overhead than lightweight web tools - Complex interface - Steep learning curve **Best For:** Power users managing diverse database ecosystems **Strengths:** - Modern, native interface - Good performance - Multi-database support **Limitations:** - Commercial license required - Desktop-only - MacOS/Windows/Linux versions vary - No web interface **Best For:** Individual developers who prefer native apps ### User Interface and Experience **Modern, Clean Design** Intuitive spreadsheet-like interface with minimal learning curve. Interactive graph visualization for schema exploration. Keyboard shortcuts for power users. **UI Score: 9/10** **Functional but Dated** Legacy interface with numerous menus. Gets the job done but feels outdated. Limited visualization capabilities. **UI Score: 5/10** **Complex and Dense** Feature-rich but overwhelming interface. Steep learning curve for new users. Tree-based navigation can be slow. **UI Score: 6/10** **Feature-Heavy** Powerful but cluttered interface. Many hidden features require exploration. Customizable but complex. **UI Score: 6/10** **Modern and Polished** Native interface with good design. Intuitive for common tasks. Limited customization. **UI Score: 8/10** **Choose Based On Priority** Speed & Simplicity → WhoDB Advanced PostgreSQL → pgAdmin Maximum Databases → DBeaver Native Experience → TablePlus MySQL Legacy → phpMyAdmin ### Database Support The current CE catalog includes PostgreSQL, MySQL, MariaDB, CockroachDB, SQLite3, MongoDB, Redis, ElasticSearch, ClickHouse, DuckDB, Memcached, TiDB, Valkey, Dragonfly, OpenSearch, YugabyteDB, QuestDB, and FerretDB, plus compatible managed variants surfaced through provider discovery. - MySQL 5.5+ - MariaDB 10.1+ That's it. No other database support. - PostgreSQL 9.6+ - EDB Postgres Advanced Server PostgreSQL-only tool with excellent PostgreSQL-specific features. **50+ databases including:** - All major SQL databases - Many NoSQL databases - Cloud databases - Data warehouses - Custom JDBC connections Most comprehensive database support. **15+ databases including:** - PostgreSQL - MySQL / MariaDB - SQL Server - SQLite - MongoDB - Redis - Cassandra - CockroachDB Good coverage of popular databases. ### Key Features Comparison **WhoDB**: Interactive graph view with relationship visualization, pan/zoom, node cards, and Data buttons **phpMyAdmin**: Designer view (basic, limited) **pgAdmin**: ERD tool (functional but outdated) **DBeaver**: ERD diagrams (comprehensive but complex) **TablePlus**: Basic structure viewer, no graph visualization Winner: WhoDB (most intuitive and interactive) **WhoDB**: Multi-cell scratchpad with syntax highlighting, query history, and one-click reuse **phpMyAdmin**: Basic SQL tab, limited history **pgAdmin**: Query tool with explain plans, advanced PostgreSQL features **DBeaver**: Powerful editor with auto-completion, multiple tabs, visual query builder **TablePlus**: Good editor with auto-completion, saved queries Winner: Tie between DBeaver (power) and WhoDB (simplicity) **WhoDB**: Add row, edit row, delete row, and selected-row actions through table controls and sheets **phpMyAdmin**: Row-by-row editing, basic interface **pgAdmin**: Grid editing, limited inline editing **DBeaver**: Spreadsheet-like grid, advanced editing features **TablePlus**: Inline editing with good UX Winner: Tie between WhoDB and TablePlus (both excellent) **WhoDB**: CSV and Excel for tables, plus JSON Lines for MongoDB-style records when supported **phpMyAdmin**: Multiple formats, good options **pgAdmin**: CSV, binary, text formats **DBeaver**: Extensive export options, templates, scheduling **TablePlus**: CSV, JSON, SQL, good options Winner: DBeaver (most comprehensive) **WhoDB**: Built-in mock data generation when the connected source and object support it **phpMyAdmin**: No built-in support **pgAdmin**: No built-in support **DBeaver**: Mock data plugin available **TablePlus**: No built-in support Winner: WhoDB (only tool with native support) ## Decision Matrix Use this matrix to determine which tool best fits your needs: ### Choose WhoDB If You: Need instant startup and minimal resource usage Prefer intuitive interfaces over feature overload Work with multiple database types regularly Appreciate clean, modern design Deploy in containerized environments Need interactive schema exploration ### Choose phpMyAdmin If You: Exclusively use MySQL/MariaDB and need basic management on shared hosting ### Choose pgAdmin If You: Need advanced PostgreSQL-specific features and are comfortable with complex interfaces ### Choose DBeaver If You: Manage many database types, need advanced features, and have resources to spare ### Choose TablePlus If You: Prefer native desktop apps and don't mind paying for software ## Use Case Recommendations **Recommended: WhoDB** Startups need to move fast with limited resources. WhoDB provides: - Zero configuration deployment - Minimal infrastructure requirements - Multi-database support as you scale - Easy onboarding for new team members - No licensing costs **Alternative:** TablePlus for Mac-centric teams who prefer native apps **Recommended: WhoDB CE or DBeaver paid editions, depending on the workflow** Large organizations need: - **WhoDB CE**: Best for standardized, web-based access when the current CE source catalog fits - **DBeaver paid editions**: Best for power users needing maximum database coverage Consider WhoDB CE if you want: - Central web-based access - Easier IT management - Lower training costs - Security handled through your existing network, auth, and database controls Consider DBeaver paid editions if you need: - Maximum database type support - Advanced query capabilities - Desktop-first workflow **Recommended: pgAdmin or WhoDB** For PostgreSQL-only environments: - **pgAdmin**: Maximum PostgreSQL-specific features - **WhoDB**: Better UX and performance for common tasks Choose pgAdmin for advanced PostgreSQL administration features. Choose WhoDB for daily development work with better speed and UX. Many teams use both: pgAdmin for DBA tasks, WhoDB for development. **Recommended: WhoDB (migrating from phpMyAdmin)** If you're currently using phpMyAdmin: - WhoDB provides everything phpMyAdmin does, plus much more - Migration is painless (just connect to the same databases) - Modern interface improves productivity - Opens door to multi-database support See our [phpMyAdmin migration guide](/resources/comparisons/vs-phpmyadmin) for details. **Recommended: WhoDB or TablePlus** Individual developers should choose based on preference: - **WhoDB**: Free, web-based, lightweight, multi-database - **TablePlus**: Native app, polished UX, one-time payment Both are excellent choices. Try WhoDB first (it's free), then evaluate TablePlus if you prefer native apps. **Recommended: WhoDB or DBeaver** Data analysts need good export, filtering, and query capabilities: - **WhoDB**: Better for quick analysis, easier to learn - **DBeaver**: Better for complex analysis, visual query builder WhoDB excels at: - Quick ad-hoc queries - Fast data export - Easy filtering and search - Lower learning curve DBeaver excels at: - Complex query building - Extensive export options - Multiple result sets - Advanced data transformations ## Pricing Comparison **WhoDB Community Edition** - Completely free - Open source (can audit code) - Self-hosted - No user limits - Community support **phpMyAdmin** - Completely free - Open source - Self-hosted - Community support **pgAdmin** - Completely free - Open source - Community support **DBeaver** - Community Edition: Free - Paid editions add broader drivers, team features, and vendor support **TablePlus** - Commercial desktop app - Free trial available - Native desktop experience ## Migration Difficulty Switching database tools is easier than you might think. Here's how difficult it is to migrate: **Difficulty: Very Easy (30 minutes)** 1. Install WhoDB 2. Connect to same MySQL databases 3. Start using immediately [Complete migration guide](/resources/comparisons/vs-phpmyadmin) **Difficulty: Easy (1 hour)** 1. Install WhoDB 2. Connect to PostgreSQL databases 3. Learn new interface (minimal learning curve) [Complete migration guide](/resources/comparisons/vs-pgadmin) **Difficulty: Moderate** 1. Evaluate database coverage (DBeaver supports more types) 2. Copy important SQL into Scratchpad pages, CLI bookmarks, or your repository 3. Recreate commonly-used workflows in WhoDB 4. Adjust to simpler interface Consider keeping DBeaver for specialized databases while using WhoDB for common ones. **Difficulty: Easy (1 hour)** 1. Install WhoDB 2. Connect to databases (similar process) 3. Adjust to web-based interface Main difference is web vs. native app experience. ## Common Questions Yes, and many teams do. Common combinations: - **WhoDB + pgAdmin**: WhoDB for daily work, pgAdmin for advanced PostgreSQL tasks - **WhoDB + DBeaver**: WhoDB for common databases, DBeaver for specialized ones - **WhoDB + CLI tools**: WhoDB for visual work, CLI for scripting WhoDB doesn't lock you in - use it alongside other tools as needed. **WhoDB** has the lowest learning curve. New users become productive in minutes. **TablePlus** is also beginner-friendly with its clean, native interface. **Avoid** pgAdmin and DBeaver for beginners - these tools have steep learning curves and can be overwhelming. **WhoDB** is ideal for teams because: - Web-based access (no installation required) - Consistent experience across platforms - Easy to deploy centrally - No per-user licensing costs - Quick onboarding Paid DBeaver editions are a better fit for large teams with diverse database needs and dedicated training. **For most users, no.** Free versions of WhoDB, phpMyAdmin, pgAdmin, and DBeaver cover 90%+ of use cases. Consider paid versions if you need: - **DBeaver paid editions**: Broader driver support, team features, or vendor support - **TablePlus**: You prefer native apps and like their design Try free versions first. You'll know when you need paid features. All tools can be secured properly with: - SSL/TLS connections - Strong authentication - Network restrictions - Regular updates **Self-hosted tools (WhoDB, phpMyAdmin, pgAdmin)** give you complete control over security. **Desktop tools (DBeaver, TablePlus)** store credentials locally on your machine. For stricter security needs, run WhoDB behind your existing auth layer and use database-native auditing plus infrastructure logs. ## Final Recommendation For most users and teams, **WhoDB** offers the best balance of speed, simplicity, and features. Here's our honest recommendation by scenario: **WhoDB** Fast, modern, multi-database support, excellent UX, free and open source **pgAdmin (advanced) or WhoDB (daily use)** Maximum PostgreSQL features vs. better UX **DBeaver** Maximum databases and advanced features **TablePlus or WhoDB** Native app vs. web-based **WhoDB** Easy deployment, no licensing costs, low learning curve **WhoDB (upgrade from phpMyAdmin)** Modern replacement for phpMyAdmin ## Try WhoDB Today Ready to see how WhoDB compares in practice? Install the current CE build Try WhoDB without installing ## Detailed Comparisons Want to dive deeper into specific tool comparisons? Detailed comparison with phpMyAdmin Detailed comparison with pgAdmin Detailed comparison with DBeaver Detailed comparison with TablePlus Still not sure? Join our [community discussions](https://github.com/clidey/whodb/discussions) to ask questions and hear from other users who have made the switch. --- ### Dory.Json (`docs/dory.json`) { "name": "WhoDB", "url": "https://docs.whodb.com", "description": "WhoDB documentation — a lightweight database management tool for exploring, querying, and visualizing your data", "seo": { "titleTemplate": "%s — WhoDB Docs", "keywords": ["whodb", "database management", "database gui", "sql client", "postgresql", "mysql", "mongodb", "redis", "database tool"], "author": "Clidey", "locale": "en_US" }, "colors": { "primary": "#ca6f1e", "light": "#ca6f1e", "dark": "#F18D3B" }, "fonts": { "family": "Montserrat", "source": "https://fonts.gstatic.com/s/googlesansflex/v5/t5sJIQcYNIWbFgDgAAzZ34auoVyXkJCOvp3SFWJbN5hF8Ju1x6sKCyp0l9sI40swNJwGpVd4AZzz0v6lJ4qFXNZhGjLvDSkV4USKMo6qQzwliVdHAy9jxTDHg_ugnAakpwGc-cxgUqDCIUI.woff2", "format": "woff2" }, "navigation": { "tabs": [ { "tab": "Documentation", "groups": [ { "group": "Getting Started", "pages": [ "introduction", "why-whodb", "installation", "first-login", "quick-start" ] }, { "group": "Core Features", "pages": [ "features/database-connectivity", "features/cloud-providers", "features/schema-explorer", "features/sidebar-navigation", "features/storage-units", "features/settings" ] }, { "group": "AI Chat Assistant", "pages": [ "ai/introduction", "ai/setup-providers", "ai/querying-data", "ai/modifying-data", "ai/conversation-features" ] }, { "group": "Data Management", "pages": [ "data/viewing-data", "data/adding-records", "data/editing-records", "data/importing", "data/deleting-records", "data/filtering-searching", "data/sorting-pagination" ] }, { "group": "Query Interface", "pages": [ "query/scratchpad-intro", "query/writing-queries", "query/query-results", "query/query-history", "query/multiple-cells" ] }, { "group": "Visualization", "pages": [ "visualization/graph-view", "visualization/schema-topology", "visualization/relationships" ] }, { "group": "Advanced Features", "pages": [ "advanced/mock-data", "advanced/export-options", "advanced/where-conditions", "advanced/batch-operations" ] }, { "group": "Platform & Tooling", "pages": [ "features/desktop-app", "features/cli-and-mcp" ] } ] }, { "tab": "Guides", "groups": [ { "group": "Getting Started Guides", "pages": [ "guides/complete-beginner", "guides/migrating-from-phpmyadmin", "guides/migrating-from-pgadmin", "guides/team-setup" ] }, { "group": "Step-by-Step Tutorials", "pages": [ "guides/tutorials/first-database-connection", "guides/tutorials/data-exploration-workflow", "guides/tutorials/building-complex-queries", "guides/tutorials/data-export-analysis", "guides/tutorials/schema-visualization", "guides/tutorials/ai-first-query" ] }, { "group": "Role-Based Guides", "pages": [ "guides/roles/developers", "guides/roles/data-analysts", "guides/roles/database-administrators", "guides/roles/qa-testers" ] }, { "group": "Use Case Guides", "pages": [ "use-cases/database-exploration", "use-cases/data-analysis", "use-cases/testing-development", "use-cases/data-migration", "use-cases/debugging-production", "use-cases/api-development", "use-cases/ai-data-exploration" ] } ] }, { "tab": "Best Practices", "groups": [ { "group": "General Best Practices", "pages": [ "best-practices/security", "best-practices/performance", "best-practices/data-management", "best-practices/query-optimization", "best-practices/ai-usage" ] }, { "group": "Database-Specific", "pages": [ "best-practices/postgresql", "best-practices/mysql", "best-practices/mongodb" ] }, { "group": "Team Workflows", "pages": [ "best-practices/collaboration", "best-practices/access-control", "best-practices/documentation" ] } ] }, { "tab": "Resources", "groups": [ { "group": "Help & Support", "pages": [ "resources/faq", "resources/troubleshooting", "resources/common-errors", "resources/performance-tuning" ] }, { "group": "Comparisons", "pages": [ "resources/comparisons/overview", "resources/comparisons/vs-phpmyadmin", "resources/comparisons/vs-pgadmin", "resources/comparisons/vs-dbeaver", "resources/comparisons/vs-tableplus" ] }, { "group": "Integrations", "pages": [ "resources/integrations/docker", "resources/integrations/aws-deployment", "resources/integrations/ci-cd", "resources/integrations/monitoring", "resources/integrations/backup-tools" ] }, { "group": "Reference", "pages": [ "resources/keyboard-shortcuts", "resources/glossary", "resources/supported-databases", "resources/enterprise-edition", "resources/changelog" ] } ] } ] }, "logo": { "light": "./logo/logo.svg", "dark": "./logo/logo.svg" }, "favicon": "./favicon.svg", "analytics": { "posthog": { "token": "phc_hxwfcMiIgSXYKkbWl9zpQE0HaPPmlRaVXCWODricYu1", "apiHost": "https://us.i.posthog.com" } } } --- ### First Login (`docs/first-login.mdx`) --- title: "First Login" description: "Connect to your first database with the current WhoDB login flow" --- # First Login The login page is the entry point for the web app and desktop app. It adapts to the selected database type and can show saved or environment-defined profiles. In the web app, a successful login is kept server-side: WhoDB stores your database credentials encrypted in a local session database and gives your browser only an opaque, `HttpOnly` session cookie — the credentials themselves never reach browser storage. Non-secret display details (connection name, host, database) are kept in the browser so the profile switcher and login state survive a page reload. The desktop app, which cannot rely on cookies, stores saved credentials in the OS keyring instead (for example, macOS Keychain); if no keyring is available, it warns and continues without persisting credentials. For shared or sensitive machines, prefer managed database accounts and environment-defined profiles either way. ## Ways To Start A Connection Pick a database type and fill in the fields shown for that type. Choose from previously saved local profiles or profiles injected through environment variables. When enabled, AWS, Azure, and GCP pickers can prefill the login form from discovered resources. First-time users can be offered a built-in sample database flow and guided tour. ## Step-By-Step Open the database type dropdown and select the system you want to connect to. The available catalog in the current CE codebase includes PostgreSQL, MySQL, MariaDB, CockroachDB, SQLite3, MongoDB, Redis, ElasticSearch, ClickHouse, DuckDB, Memcached, TiDB, Valkey, Dragonfly, OpenSearch, YugabyteDB, QuestDB, and FerretDB. The main form changes by database type: - Most network databases use host, username, password, and database - PostgreSQL-family databases can also expose search path / schema behavior - SQLite and DuckDB use a database file path instead of host and port - Some document and cache systems only require a host and optional credentials The web app advanced panel is for SSL/TLS and backend-defined database-specific fields. Shared SSL fields include: - `SSL Mode` - `SSL CA Content` - `SSL Client Cert Content` - `SSL Client Key Content` - `SSL Server Name` If you need SSH tunneling, set up an external tunnel first or use the [CLI & MCP](/features/cli-and-mcp) tooling. SSH tunneling is not a web login form control in this codebase. Click the main connect button for manual credentials, or choose a saved profile and log in through that profile flow. ## Saved Profiles Profiles can appear from two places: - Local profiles saved by the app - Environment-defined profiles such as `WHODB_POSTGRES_1`, `WHODB_MYSQL_1`, or `WHODB_REDIS_1` WhoDB Community Edition treats access to the instance as the trust boundary for profiles. Anyone who can reach the instance can list and use its available profiles with the database permissions configured for those profiles. The database password is used by the server, so users do not need to know or enter it. Only make a profile available on instances restricted to trusted users. When the manual credential form is disabled with `WHODB_DISABLE_CREDENTIAL_FORM=true`, the login page can still work through managed profiles. ## File-Based Databases SQLite3 and DuckDB behave differently from host-based systems: - The required field is the database file path - The desktop app can open a native file picker for these databases - DuckDB is only available on supported builds ## Next Steps See the current CE connection model and advanced option reference. Review the current database catalog grouped by compatibility family. Use local CLI discovery for AWS, Azure, and GCP. Move from login to exploring data and writing queries. --- ### Installation (`docs/installation.mdx`) --- title: "Installation" description: "Install, run, and configure WhoDB using the current repository layout" --- # Installation Guide WhoDB can run as a container, a standalone binary, a desktop app, or a local source checkout. ## Installation Methods Run the published image and expose the web UI on port 8080. Download a release binary and run it directly. Build the frontend and backend from this repository. Use the Wails desktop shell around the same core app. ## Docker (Recommended) ```bash docker pull clidey/whodb:latest ``` ```bash docker run -d \ --name whodb \ -p 8080:8080 \ clidey/whodb:latest ``` Visit `http://localhost:8080`. ### Docker Compose ```yaml services: whodb: image: clidey/whodb:latest ports: - "8080:8080" environment: - PORT=8080 - WHODB_LOG_LEVEL=info # Keeps encrypted login sessions valid across container recreation. # Generate once with: openssl rand -hex 32 - WHODB_ENCRYPTION_KEY=replace_with_openssl_rand_hex_32 restart: unless-stopped volumes: - whodb-data:/data volumes: whodb-data: ``` ### Session Storage WhoDB keeps login sessions server-side. When you log in from a browser, WhoDB stores your database credentials encrypted (AES-256-GCM) in a local SQLite file under its data directory, and the browser only holds an opaque, `HttpOnly` session cookie — never the credentials themselves. Without any configuration, this works out of the box: a plain `docker run` generates an encryption key on first start and keeps you logged in until the container is recreated (at which point you simply log in again). To keep sessions valid across upgrades and restarts: - Mount `/data` as a volume so the encrypted session database and generated key persist. - Set `WHODB_ENCRYPTION_KEY` to a stable 64-character hex string (`openssl rand -hex 32`) so the key does not depend on the container's filesystem. Keep `WHODB_ENCRYPTION_KEY` safe and stable. Changing or losing it invalidates every existing session — sessions encrypted with the old key can no longer be decrypted, so users are simply logged out and need to sign in again. If WhoDB sits behind an HTTPS-terminating reverse proxy (nginx, Traefik, Caddy, a cloud load balancer), also set `WHODB_SECURE=true` so the session cookie is marked `Secure`. Leave it unset for plain HTTP, including local development — WhoDB does not infer HTTPS from proxy headers, and setting it on an HTTP deployment causes the browser to drop the cookie. ## Binary Installation Download the appropriate release artifact from GitHub Releases, make it executable if needed, and run it: ```bash chmod +x whodb ./whodb ``` The default port is `8080`. Override it with `PORT=3000 ./whodb` if you want a different listener port. ## Build From Source ### Prerequisites - Go `1.26.1` or newer - Node.js and `pnpm` ### Build Steps ```bash git clone https://github.com/clidey/whodb.git cd whodb ``` ```bash cd frontend pnpm install pnpm run build:ce cd .. ``` ```bash cd core go build ./cmd/whodb ``` ```bash ./whodb ``` ### Development Mode Use two terminals: ```bash Backend cd core go run ./cmd/whodb ``` ```bash Frontend cd frontend pnpm start ``` The frontend dev server runs on `http://localhost:3000` and proxies its backend requests to the WhoDB server. ## Desktop App The desktop app lives in `desktop-ce/` and uses the same frontend and backend codepaths. See [Desktop App](/features/desktop-app) for the current behavior. ## Common Environment Variables ### Server | Variable | Description | Default | |----------|-------------|---------| | `PORT` | HTTP port WhoDB listens on | `8080` | | `WHODB_LOG_LEVEL` | Logging level: `debug`, `info`, `warn`, `error`, `none` | `info` | | `WHODB_LOG_FORMAT` | Set to `json` for JSON logs; unset for the default text format | `text` | | `WHODB_LOG_FILE` | Redirect non-HTTP logs to a file. `default` means `/var/log/whodb/whodb.log` | unset | | `WHODB_ACCESS_LOG_FILE` | Redirect HTTP access logs to a file. `default` means `/var/log/whodb/whodb.access.log` | unset | | `WHODB_AUDIT_LOG_FILE` | Append-only NDJSON log of audited actions such as logins, data reads and changes. Request telemetry is excluded. `default` means `/var/log/whodb/whodb.audit.log` | unset | | `WHODB_ALLOWED_ORIGINS` | Comma-separated CORS allowlist | unset | | `WHODB_BASE_PATH` | URL path prefix for bundled web deployments, such as `/whodb` | unset | | `WHODB_DISABLE_CREDENTIAL_FORM` | Hide the manual credential form and require managed profiles/providers | `false` | | `WHODB_MAX_PAGE_SIZE` | Maximum rows allowed per page request | `10000` | | `WHODB_DISABLE_MOCK_DATA_GENERATION` | Disable mock data globally or for named tables | unset | | `WHODB_DISABLE_UPDATE_CHECK` | Disable release availability checks | `false` | | `WHODB_ENCRYPTION_KEY` | 64-char hex key used to encrypt stored login sessions. Auto-generated and persisted to `WHODB_DATA_DIR` if unset | unset | | `WHODB_DATA_DIR` | Directory for the encrypted session database and generated key | platform default data directory | | `WHODB_SESSION_TTL` | Sliding idle timeout for login sessions, as a Go duration (for example `168h`) | `168h` (7 days) | | `WHODB_SECURE` | Mark the session cookie `Secure`. Set to `true` only when served over HTTPS (including behind a TLS-terminating proxy) | `false` | The `default` log paths write to `/var/log/whodb/`. That works naturally in Docker but usually requires a custom path for local development. `WHODB_BASE_PATH` only applies to production versions, such as the Docker image or a production binary build. It is intended for reverse-proxy subpath hosting like `/whodb/`, not split frontend/backend dev mode. ### AI Providers | Variable | Description | Default | |----------|-------------|---------| | `WHODB_OLLAMA_HOST` | Ollama host | `localhost` (resolved for Docker/WSL when needed) | | `WHODB_OLLAMA_PORT` | Ollama port | `11434` | | `WHODB_OLLAMA_NAME` | Display name for Ollama | unset | | `WHODB_OPENAI_API_KEY` | OpenAI API key | unset | | `WHODB_OPENAI_ENDPOINT` | OpenAI API base URL | `https://api.openai.com/v1` | | `WHODB_OPENAI_NAME` | Display name for OpenAI | unset | | `WHODB_ANTHROPIC_API_KEY` | Anthropic API key | unset | | `WHODB_ANTHROPIC_ENDPOINT` | Anthropic API base URL | `https://api.anthropic.com/v1` | | `WHODB_ANTHROPIC_NAME` | Display name for Anthropic | unset | | `WHODB_LMSTUDIO_BASE_URL` | LM Studio base URL | `http://localhost:1234/v1` | | `WHODB_LMSTUDIO_API_KEY` | LM Studio API key | unset | | `WHODB_LMSTUDIO_NAME` | Display name for LM Studio | unset | Generic OpenAI-compatible providers can be added with the `WHODB_AI_GENERIC__*` pattern. See [Setting Up AI Providers](/ai/setup-providers). ### CLI Cloud Providers | Variable | Description | Default | |----------|-------------|---------| | `WHODB_ENABLE_AWS_PROVIDER` | Enable AWS provider support in the CLI | `false` | | `WHODB_AWS_PROVIDER` | JSON array of AWS provider definitions | unset | | `WHODB_ENABLE_AZURE_PROVIDER` | Enable Azure provider support in the CLI | `false` | | `WHODB_AZURE_PROVIDER` | JSON array of Azure provider definitions | unset | | `WHODB_ENABLE_GCP_PROVIDER` | Enable GCP provider support in the CLI | `false` | | `WHODB_GCP_PROVIDER` | JSON array of GCP provider definitions | unset | These variables do not enable provider operations in the CE web server. See [Cloud Providers](/features/cloud-providers) for the local CLI workflow. ## Database Connection Profiles WhoDB can preload connection profiles from environment variables so they appear on the login page. In Community Edition, connection profiles are shared with everyone who can reach the WhoDB instance. A user can select a profile and connect with its server-configured credentials without entering the database password. Profile names and IDs are not authentication factors. When profiles grant access to private or sensitive data, expose WhoDB only to trusted users and use a least-privileged database account. Profiles work for every database type in the catalog: the prefix is `WHODB_` followed by the uppercased type name. Examples for the core types include: - `WHODB_POSTGRES` - `WHODB_COCKROACHDB` - `WHODB_MYSQL` - `WHODB_MARIADB` - `WHODB_TIDB` - `WHODB_SQLITE3` - `WHODB_MONGODB` - `WHODB_REDIS` - `WHODB_ELASTICSEARCH` - `WHODB_CLICKHOUSE` - `WHODB_DUCKDB` - `WHODB_MEMCACHED` The same pattern applies to the rest of the catalog, such as `WHODB_VALKEY`, `WHODB_DRAGONFLY`, `WHODB_OPENSEARCH`, `WHODB_YUGABYTEDB`, `WHODB_QUESTDB`, and `WHODB_FERRETDB`. Two formats are supported: ```bash Array Format export WHODB_POSTGRES='[ {"alias":"prod","host":"db.example.com","user":"postgres","password":"secret","database":"app","port":"5432"}, {"alias":"staging","host":"staging.example.com","user":"postgres","password":"secret","database":"app","port":"5432"} ]' ``` ```bash Numbered Format export WHODB_POSTGRES_1='{"alias":"prod","host":"db.example.com","user":"postgres","password":"secret","database":"app","port":"5432"}' export WHODB_POSTGRES_2='{"alias":"staging","host":"staging.example.com","user":"postgres","password":"secret","database":"app","port":"5432"}' ``` Each profile object supports these common fields: | Field | Description | |-------|-------------| | `alias` | Label shown on the login page | | `host` | Database hostname or IP | | `user` | Username | | `password` | Password | | `database` | Database name or file path | | `port` | Port number as a string | | `advanced` | Key/value map of advanced options | Setting `WHODB_DISABLE_CREDENTIAL_FORM=true` hides manual credential entry, but it does not add authentication in front of configured profiles. Users who can reach the instance can still select and use them. ### SSL Example For profile-based connections, the backend accepts both certificate content keys and server-side file-path keys: ```bash export WHODB_POSTGRES_1='{ "alias": "prod", "host": "db.example.com", "user": "postgres", "password": "secret", "database": "app", "port": "5432", "advanced": { "SSL Mode": "verify-ca", "SSL CA Path": "/etc/certs/ca.pem", "SSL Client Cert Path": "/etc/certs/client.pem", "SSL Client Key Path": "/etc/certs/client-key.pem" } }' ``` Cloud-managed variants such as `DocumentDB` and `ElastiCache` accept `WHODB_` profile prefixes like any other catalog type (`WHODB_DOCUMENTDB`, `WHODB_ELASTICACHE`), though provider discovery is the more common way to introduce them. Those provider-discovery paths are still not production-ready in the current CE build, which is why the provider integrations stay disabled unless you explicitly turn them on. ## Custom Port Example ```bash PORT=3000 ./whodb ``` To publish the default container listener on host port `3000`: ```bash docker run -d \ -p 3000:8080 \ clidey/whodb:latest ``` If you intentionally change the internal listener with `PORT`, map the same internal port: ```bash docker run -d \ -e PORT=3000 \ -p 3000:3000 \ clidey/whodb:latest ``` ## Verify The Install Visit `http://localhost:8080` or your configured port. You should see the login page with the current database catalog. Connect to a known local or development database to confirm network reachability and credentials. ## Troubleshooting Use a different external port mapping in Docker or run the binary with `PORT=3000`. ```bash chmod +x whodb ``` Verify: - the database is reachable from the WhoDB process - firewall rules allow the connection - credentials are correct - Docker deployments use the correct host naming (`host.docker.internal` for host services when appropriate) ## Related Pages Walk through the connection flow after WhoDB is running. Use local CLI discovery for AWS, Azure, and GCP. --- ### Introduction (`docs/introduction.mdx`) --- title: "Welcome to WhoDB" description: "A lightweight, modern database management tool for exploring, querying, and visualizing your data" --- # Welcome to WhoDB WhoDB is a powerful, intuitive database management tool designed to simplify how you interact with your databases. Whether you're a developer, data analyst, or database administrator, WhoDB provides a streamlined interface for exploring schemas, managing data, and visualizing relationships. ## What is WhoDB? WhoDB is an open-source database management solution that combines the power of traditional database clients with modern visualization and query capabilities. It provides a unified interface for working with multiple database types while maintaining a focus on simplicity and user experience. Connect to PostgreSQL, MySQL, SQLite, MongoDB, Redis, and more with a single interface Explore your database structure with interactive graph visualizations showing table relationships Write and execute SQL queries with syntax highlighting, history tracking, and result visualization View, add, edit, and delete records with an intuitive spreadsheet-like interface ## Key Features WhoDB supports multiple database engines out of the box. Connect with standard host/database credentials or environment-defined profiles, and use advanced options such as SSL/TLS certificates when your database requires them. Navigate source objects through Storage Units. Use **Describe** for metadata and columns, and **Data** for rows. View and edit data in a familiar grid. Sort columns, filter rows with WHERE conditions, search loaded values, and use selected-row actions where the source supports them. Write SQL queries in a syntax-highlighted editor with page and cell organization. Execute queries, view results, and reuse entries from cell history. Visualize your database schema as an interactive graph. See table relationships and foreign keys at a glance, pan and zoom around the canvas, and open a node's Data action when you want to inspect rows. Generate realistic test data for development and testing. Configure the number of rows, choose between append or overwrite modes, and let WhoDB populate your tables with sample data. Export grid and query-result data as CSV or Excel, plus JSON Lines for records from sources like MongoDB when supported. Apply filters before export and select CSV delimiters. ## Why Choose WhoDB? WhoDB is fast, lightweight, and fully open source, with a clean interface that puts your data front and center instead of burying it under menus. One tool covers all your databases, so there's less context switching and nothing to install per user. See how WhoDB compares to traditional database tools and what makes it different ## Use Cases WhoDB is perfect for a wide range of database management tasks **Development & Testing** - Quickly inspect database state during development - Generate test data for your applications - Debug SQL queries and view results instantly - Explore schema changes and relationships **Data Analysis** - Export data for further analysis in Excel or other tools - Run ad-hoc queries to answer business questions - Filter and sort large datasets to find specific records - Visualize table relationships to understand data flow **Database Administration** - Monitor table structures and indexes - Browse and edit data across multiple databases - Run ad-hoc queries against any connected database - Maintain data quality with filtering and validation **Learning & Education** - Explore database structures to learn SQL - Visualize relationships between tables - Practice queries in a safe environment - Understand database design patterns ## Getting Started Learn how to install WhoDB on your system Connect to your first database Get up and running in minutes WhoDB is under active development. New features and improvements are added regularly. Check our GitHub repository for the latest updates and release notes. ## Community & Support Join our growing community of users and contributors: - **GitHub**: Report issues, request features, and contribute code - **Documentation**: Comprehensive guides and API references - **GitHub Discussions & Issues**: Ask questions, share tips, and report problems Ready to start exploring your databases? Continue to the **Installation** page to get WhoDB running on your system. --- ### Quick Start (`docs/quick-start.mdx`) --- title: "Quick Start Guide" description: "Use the current WhoDB interface to browse data, run queries, visualize schema, and export results" --- # Quick Start Guide This guide starts after WhoDB is running and you have connected to a database. WhoDB uses source-first terminology. Most database tables, collections, buckets, or equivalent objects are shown as **Storage Units**. ## Current Interface After login, the left sidebar is the primary navigation surface. Depending on the connected source, it can show: - **Chat** when AI chat is supported and a provider/model is configured - **Storage Units** for browsing tables, collections, views, or similar objects - **Graph** when relationship visualization is supported - **Scratchpad** when custom query execution is supported - **Settings** when the settings page is enabled The Storage Units page is the default browse view. It can show cards or a list. Use the search field at the top of the page to filter object names. ## First Five Minutes Open **Storage Units** from the sidebar. Use the profile, database, and schema selectors in the sidebar when they are available for the current source. In card view, click **Describe** on a storage unit. The card expands and shows metadata attributes. For tabular objects, WhoDB also loads column names and types. Click **Data** on a storage unit. WhoDB opens the object data page with row data, total count, search, page size, Where conditions, and query controls. Open **Scratchpad** from the sidebar or use the Scratchpad button on a data page. Type SQL and click the run button, the gutter play button, or press `Mod + Enter`. Open **Graph**. Select the storage units you want to show from the graph sidebar. The canvas renders selected nodes and foreign-key relationships when the source exposes them. ## Viewing And Filtering Data On a storage unit data page: - **Search** highlights and scrolls to matching values in currently loaded rows. Use WHERE conditions when you want to filter the result set. - **Where condition** opens the condition builder. Add field/operator/value conditions, then click **Query** on the page to reload data with those conditions. - **Page Size** controls how many rows WhoDB asks the backend for per page. Built-in options include `10`, `25`, `50`, `100`, `250`, `500`, `1000`, and `Custom`, subject to the configured maximum. - **Pagination** appears at the bottom when more pages are available. - **Column headers** can be clicked to sort. Sorting is sent back through the data request. ## Managing Rows Row write actions are only shown when the connected source and object allow them. - **Add Row** is below the table, beside pagination. It opens a right-side sheet with one input per column, or a JSON editor for records from sources like MongoDB. - **Edit Row** is available from a row context menu or by focusing a row and pressing `Enter`/`Mod + E`. It opens a right-side edit sheet. - **Delete Row** is available from a row context menu or `Mod + Delete`/`Mod + Backspace` on a focused row. Current CE code shows a confirmation dialog before deleting. - **Select rows** with checkboxes. The header checkbox toggles all currently loaded rows. Writes are sent to the connected database. Test changes on a development database and keep backups for important data. ## Scratchpad Basics Scratchpad is organized as **pages**. Each page contains one or more query cells. - Click the `+` tab to add a new page. - Double-click a page name to rename it. - Use the `+` button that appears on a cell to add another cell below it. - Use the cell three-dot menu for **Clear**, **Copy Code**, and **Query History**. - Query results render below the editor. Action statements show an action-executed result after confirmation. - Query history is stored per cell and contains status, timestamp, copy, clone-to-editor, rerun, and clear-history actions. ## Graph Basics The Graph page has two parts: - A left graph sidebar with a search field and checkboxes for selecting storage units. - A React Flow canvas showing selected nodes and relationship edges. Graph controls are on the canvas: - React Flow zoom, fit, and lock/pan controls are at the lower-left. - WhoDB download and layout buttons are at the lower-right. - Holding `Space` enables drag-pan behavior; scroll/pinch gestures zoom or pan according to browser and React Flow behavior. - Clicking a node loads that node's columns when needed. Node cards show metadata and columns directly. - Use the **Data** button inside a node to open the storage unit data page. ## Export Basics Use the table **Export All** button, or selected-row export from the table context menu. Current CE formats are: - **CSV** for tabular exports, with delimiter options `,`, `;`, and `|` - **Excel** (`.xlsx`) for tabular exports - **JSON Lines (NDJSON)** for records from sources like MongoDB when supported For full SQL backups or migration dumps, use your database's native tools such as `pg_dump`, `mysqldump`, or `mongodump`. ## Useful Shortcuts `Mod` means `Cmd` on macOS and `Ctrl` on Windows/Linux. | Shortcut | Action | |----------|--------| | `?` | Open shortcut help | | `Mod + K` | Open command palette | | `Mod + B` | Toggle sidebar | | `Mod + Enter` | Run the current Scratchpad editor | | `Mod + U` | Clear the current editor | | `Mod + R` | Refresh table data | | `Mod + Shift + E` | Open export | | `Mod + I` | Open import when supported | | `Mod + Shift + G` | Open mock data generation when supported | See [Keyboard Shortcuts](/resources/keyboard-shortcuts) for the full current list. ## Next Steps Learn the storage unit data page. Run and organize SQL queries. Understand the graph canvas and controls. Export current table or selected rows. --- ### Read Only Execution (`docs/read-only-execution.md`) # Protected SQL execution Automatic SQL execution uses the same classifier in CLI/MCP and chat. A query must fit the supported read grammar **and** have a connector with enforced read-only execution. Unknown syntax requires approval; read-only mode rejects it. Unsupported connectors return an error instead of executing normally. Explicitly writable query/editor flows keep their existing permissions. ## Supported grammar and compatibility The shared classifier accepts a deliberately bounded SQL subset: SELECT, read-only CTEs and subqueries, explicit `AS` aliases, joins, ordinary comparisons, grouping, ordering, limits, set operations, VALUES, TABLE, simple EXPLAIN forms, a few standalone SHOW forms, and SQLite metadata PRAGMAs. Standard unqualified COUNT/SUM/AVG/MIN/MAX aggregates are supported. Every statement in a batch must pass. The parser is not intended to validate every dialect's complete grammar. The database can still reject a syntactically unsupported read. Dialect extensions, implicit aliases, casts, most function calls, qualified or quoted routine calls, table functions, backslash-dependent quoting, executable comments and unrecognized syntax require approval. So do SELECT INTO, OUTFILE/DUMPFILE, sequence NEXTVAL, explicit locking, configuration commands, procedural execution, DDL/DML, and native commands outside the SQL grammar. Adding syntax support requires positive and adversarial regression tests; adding a leading verb or keyword exception is insufficient. The contract assumes trusted database schema objects, built-in routine resolution, and appropriate database credentials. It does not sandbox a hostile database server, administrator-installed routines/operators/views, or external storage hidden behind a table. Use least-privilege database accounts and trusted schema/search-path configuration. It does not promise zero engine-internal writes (such as caches or temporary query processing), resource isolation, or that MCP confirmation proves a human clicked a button. ## Enforcement PostgreSQL-family and MySQL-family connectors use native read-only transactions. SQLite uses a read-only file connection plus query_only. ClickHouse uses readonly=1. Protected GORM requests own their connections, close them after the request, and cannot join an existing transaction; ordinary connection pooling is unchanged. This costs an extra connection establishment for protected requests. DuckDB uses native `BEGIN TRANSACTION READ ONLY` on a pinned connection because duckdb-go rejects `sql.TxOptions.ReadOnly`. The query and rollback use that same connection; the request owns and closes its connection pool. This coexists with ordinary writable connections without changing the database's file-access mode. DuckDB's own [transaction regression test](https://github.com/duckdb/duckdb/blob/v1.5.5/test/sql/transactions/test_read_only_transactions.test) exercises native write refusal. Native transaction mode still permits `COPY TO`, so the shared SQL classifier remains necessary to block exports and other unsupported statements before execution. DuckDB's existing multi-statement restriction remains in place. Sources without an enforcement implementation reject protected execution. A source catalog capability is a requirement for execution, not permission to silently drop the read-only flag. MCP default confirmation mode uses protected execution even for queries that do not need approval. Approval retains SQL, parameters and a fingerprint of the resolved connection. Changed targets require new approval. Read-only and current security restrictions still apply. Approval is consumed before dispatch: after an error or timeout the caller must inspect the outcome and obtain new approval before another attempt. This provides at-most-one dispatch per token, not exactly-once database effects. ## Local security verification These commands start **disposable localhost databases**, with dedicated names and ports. Do not substitute production connections. They do not use the shared development database stack. ```sh docker run -d --name whodb-readguard-pg -p 127.0.0.1:55439:5432 -e POSTGRES_USER=guard -e POSTGRES_PASSWORD=guard-local -e POSTGRES_DB=guard postgres:18 docker run -d --name whodb-readguard-mysql -p 127.0.0.1:53319:3306 -e MYSQL_ROOT_PASSWORD=guard-local -e MYSQL_DATABASE=guard mysql:8.4 --secure-file-priv=/tmp docker run -d --name whodb-readguard-maria -p 127.0.0.1:53329:3306 -e MARIADB_ROOT_PASSWORD=guard-local -e MARIADB_DATABASE=guard mariadb:latest --secure-file-priv=/tmp docker run -d --name whodb-readguard-clickhouse -p 127.0.0.1:59009:9000 -e CLICKHOUSE_USER=guard -e CLICKHOUSE_PASSWORD=guard-local -e CLICKHOUSE_DB=guard -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 clickhouse/clickhouse-server:25.5 ``` Wait for the database health checks to succeed, then run sequentially: ```sh # From core/ go test ./src/sqlguard ./src/plugins/... ./src/source/adapters ./src/bamlconfig ./graph WHODB_READGUARD_LOCAL=1 go test -tags integration ./src/plugins/clickhouse -run '^TestClickHouseProtectedExecution$' -count=1 go test ./src/sqlguard -run '^$' -fuzz '^FuzzProtectedRead$' -fuzztime=15s -parallel=2 # From cli/ go test ./pkg/mcp ./internal/database go build -o /tmp/whodb-readguard-cli . WHODB_READGUARD_LOCAL=1 WHODB_READGUARD_BINARY=/tmp/whodb-readguard-cli go test -tags integration ./pkg/mcp -run '^TestReadGuardTransport$' -count=1 ``` Without WHODB_READGUARD_BINARY the transport suite runs the same MCP server inside the test process over real HTTP. With it, it launches the specified compiled CLI. Database observers verify no table/file creation before approval, positive effects after approval, parameter preservation and replay rejection. Native transaction tests bypass the classifier to check database write refusal. SQLite tests independently disable query_only on a protected connection and verify that file access mode still rejects writes. ClickHouse tests independently exercise its native readonly setting. These tests fail when their required local database is missing. The ClickHouse test also runs in CI: `dev/run-backend-tests.sh ce-integration` starts the `readguard_clickhouse` service from `dev/docker-compose.yml` (compose profile `readguard`, same credentials and port as above) and sets `WHODB_READGUARD_LOCAL=1`. The CLI transport suite remains local-only. ```sh docker stop whodb-readguard-pg whodb-readguard-mysql whodb-readguard-maria whodb-readguard-clickhouse docker rm -v whodb-readguard-pg whodb-readguard-mysql whodb-readguard-maria whodb-readguard-clickhouse rm /tmp/whodb-readguard-cli ``` The live matrix above does not certify every PostgreSQL/MySQL-compatible product or provider version. Those need independent connector integration evidence. ## Verification snapshot (2026-09-28) The live tests used PostgreSQL 18.6, MySQL 8.4.10, MariaDB 13.0.2 and ClickHouse 25.5.11.15. The transport suite passed with the CE binary against all three transactional engines in read-only, default-confirmation and strict-confirmation modes. Both MySQL file-output forms were exercised with root/FILE privileges so the negative result was not merely a missing-permission error. Their approved positive controls created actual files in the disposable containers. The focused Go suites, backend/CLI builds, and lint checks passed. The SQL fuzz run completed 1,817,694 executions in 16 seconds without a failing input. These are local results, not a patched release or production/hosted verification. ## DuckDB verification (2026-09-29) The embedded engine reports DuckDB v1.5.5. Connector tests cover parameterized reads, CTEs, simultaneous protected reads and ordinary writes, native DML/DDL refusal independently of the classifier, error/cancellation cleanup, and classifier rejection of file exports and transaction-escape batches. A native `COPY TO` positive control proves that export prevention requires the classifier. The isolated MCP transport suite uses temporary DuckDB files and needs no Docker services. It covers read-only, default-confirmation and strict-confirmation modes, including actual file creation only after approval, parameter binding, replay rejection, and protected reads after approved writes: ```sh # From core/ go test ./src/plugins/duckdb ./src/sourcecatalog ./src/source/adapters # From cli/ go test -tags integration ./pkg/mcp -run '^TestDuckDBReadGuardTransport$' -count=1 go build -o /tmp/whodb-duckdb-cli . WHODB_READGUARD_BINARY=/tmp/whodb-duckdb-cli go test -tags integration ./pkg/mcp -run '^TestDuckDBReadGuardTransport$' -count=1 rm /tmp/whodb-duckdb-cli ``` The in-process HTTP transport and compiled CLI runs passed all three modes. The connector, source catalog, source adapter and classifier tests passed under the Go race detector; the backend/CLI builds and scoped lint checks passed. ### Browser regression The CE browser regression `protected-chat.ce.spec.mjs` uses a local model stub and the real browser SSE endpoint, BAML parser, source session and DuckDB file. It verifies a successful automatic read, confirmation for a `CREATE TABLE AS SELECT` query incorrectly labeled `get` by the model, absence of the table before approval, creation after clicking Yes, and a protected read of the new table. The test removes its isolated table afterward. It does not mock query results or the application's chat response. ```sh # From the repository root; the runner creates local test fixtures and services. VITE_E2E_TEST=true bash dev/run-e2e.sh true duckdb protected-chat.ce # Full DuckDB browser regression suite: VITE_E2E_TEST=true bash dev/run-e2e.sh true duckdb ``` Local CE browser verification completed with 168 passed (including auth setup) and 72 existing skips across the DuckDB suite and the new security regression. One cross-database login test initially failed because the local PostgreSQL fixture was absent; it passed after that fixture was started. The remaining mutating tests ran sequentially and passed. The CE frontend build and lint also passed. This verifies the local CE browser path with a stubbed model provider; it does not establish EE browser or live-provider coverage. --- ### Why Whodb (`docs/why-whodb.mdx`) --- title: "Why Choose WhoDB?" description: "Discover why WhoDB is the best lightweight database management tool for developers, data analysts, and teams. Fast, intuitive, and powerful." --- # Why Choose WhoDB? **WhoDB is the modern database management tool that developers actually want to use.** Unlike bloated legacy tools, WhoDB delivers a lightning-fast, intuitive experience that makes working with databases enjoyable instead of frustrating. ## The Problem with Traditional Database Tools Traditional database management tools like phpMyAdmin, pgAdmin, and others were built for a different era. They're: - **Slow and Resource-Intensive**: Heavy desktop applications that consume gigabytes of RAM - **Cluttered Interfaces**: Dozens of menus, toolbars, and options that hide what you actually need - **Difficult to Learn**: Steep learning curves that waste hours of productive time - **Poor Visualization**: Limited or non-existent schema visualization capabilities - **Inconsistent Experience**: Different tools for different databases, each with unique quirks Context switching between tools and wrestling with clunky interfaces costs real development time every week. ## The WhoDB Difference WhoDB was built from the ground up to solve these problems with a focus on **speed, simplicity, and developer experience**. ### 1. Lightning Fast Performance Lightweight Go backend and browser UI, without a heavy desktop client Query views stay focused on the current result set, page size, and filters Small server process compared with full desktop database IDEs No installation required - access from any device with a browser ### 2. Beautifully Simple Interface WhoDB's clean, modern interface eliminates clutter and puts your data front and center. New users become productive in minutes, not hours. No training required. Everything you need is exactly where you expect it. Sidebar navigation, context menus, and keyboard shortcuts make navigating databases effortless. - **Clear browse flow**: Connect, open Storage Units, then use Describe or Data - **Quick search**: Find source objects and visible row values quickly - **Simple navigation**: Sidebar routes, Storage Unit cards, and Scratchpad pages keep work organized Work with database data in a familiar grid. Click column headers to sort, right-click rows for actions, search loaded rows, and use WHERE conditions for server-side filtering. - **Familiar interface**: If you can use Excel, you can use WhoDB - **Edit sheets**: Open row values in a side sheet before saving changes - **Selected-row actions**: Select rows for supported export and delete workflows Write SQL with syntax highlighting, query history, and page/cell organization in the web Scratchpad. Use the CLI TUI when you want terminal autocomplete, formatting, bookmarks, and external editor support. - **Syntax highlighting**: Color-coded SQL for easy reading - **Query history**: Reuse previous cell queries - **Multi-cell support**: Organize queries into logical sections - **CLI editor features**: Autocomplete and SQL formatting in the terminal TUI ### 3. Visual Schema Explorer Understanding database relationships has never been easier. WhoDB's interactive graph view shows your entire schema at a glance. See all tables and relationships in a visual diagram - no need to read through pages of schema definitions Click any table to see its columns, indexes, and foreign keys. Follow relationships to explore data flow Identify hub tables, isolated tables, and complex relationship patterns instantly Export schema diagrams for team documentation and onboarding ### 4. Multi-Database Support **One tool for all your databases.** Why juggle multiple database tools when WhoDB supports them all? Full support for PostgreSQL Complete MySQL and MariaDB compatibility Local and embedded databases NoSQL document databases Key-value data stores Additional support for ClickHouse, DuckDB, OpenSearch, Memcached, and compatible managed variants ### 5. Developer-Friendly Features WhoDB includes features that developers actually use every day: Generate realistic test data in seconds. Perfect for development and testing environments. - Configure row counts (1 to 10,000+) - Smart data type handling - Append or overwrite modes - Realistic fake data for supported column types Export data in the format you need: - **CSV**: For Excel and spreadsheet tools - **Excel (.xlsx)**: Native Excel format with formatting - **JSON Lines**: For records from sources like MongoDB when supported Configure CSV delimiters and apply filters before export. Use database-native tools for full backup and migration exports. Build complex WHERE conditions with a visual interface: - Multiple conditions with AND/OR logic - All standard operators (`=`, `!=`, `>`, `<`, `>=`, `<=`) - Date range filtering - Text pattern matching - NULL value handling Scratchpad keeps history for each web cell so you can reload SQL you already ran. The CLI also has searchable history and bookmarks for terminal workflows. - **Cell history**: Reload or rerun SQL from the current cell - **Query reuse**: Copy previous SQL into the editor and adjust it - **Execution details**: Review result status and timing with the query output - **CLI bookmarks**: Save named queries when working in the terminal ## Who Uses WhoDB? ### Developers **"Finally, a database tool that doesn't get in my way."** Developers love WhoDB's speed and keyboard-driven workflow. Connect to any database, run queries, inspect data, and move on - all in seconds. **Perfect for:** - Local development workflows - Debugging production issues - API development and testing - Schema exploration for new projects ### Data Analysts **"I can explore data and export results faster than ever before."** Analysts appreciate WhoDB's filtering, sorting, and export capabilities. Build complex queries, filter results, and export to Excel - all without leaving the browser. **Perfect for:** - Ad-hoc data analysis - Report generation - Data quality checks - Business intelligence queries ### Database Administrators **"Managing multiple databases has never been easier."** DBAs use WhoDB to monitor schemas, manage data, and perform administrative tasks across multiple database servers from a single interface. **Perfect for:** - Schema management - Data maintenance - Performance monitoring - Multi-database administration ### QA Engineers **"Mock data generation saves me hours every sprint."** QA teams use WhoDB to generate test data, verify database states, and debug test failures quickly. **Perfect for:** - Test data generation - Database state verification - Test debugging - Test automation support ## Comparison with Alternatives See how WhoDB compares to phpMyAdmin See how WhoDB compares to pgAdmin See how WhoDB compares to DBeaver See how WhoDB compares to TablePlus ## Real-World Benefits **Spend less time fighting tools, more time building features.** WhoDB's speed and intuitive interface cut down the time spent wrestling with database tools compared to traditional alternatives. **Visual schema exploration prevents relationship mistakes.** Understanding database relationships visually reduces data integrity errors by catching relationship issues before they become problems. **Get more done with fewer context switches.** One unified tool for all databases means less context switching and faster workflow. **New team members become productive immediately.** WhoDB's intuitive interface requires no training. New developers can start exploring databases on day one. ## Open Source & Community WhoDB is **fully open source** under a permissive license, giving you: Audit the code and ensure it meets your security requirements Customize and extend WhoDB for your specific needs Self-host without dependencies on external services Use GitHub issues and discussions to follow the project and report problems ## Team Usage Teams usually run WhoDB behind their normal network controls and connect with database accounts that match each environment's access policy. Use read-only database credentials where possible, native database audit logging for compliance trails, and the CLI for repeatable checks. Set up a shared CE deployment using Docker, database credentials, and operational guardrails. ## Getting Started Ready to experience the fastest, most intuitive database tool you've ever used? Install the current CE build See WhoDB in action Complete documentation ## Frequently Asked Questions Yes! WhoDB Community Edition is completely free and open source. You can use it for personal projects, commercial projects, and everything in between. No hidden costs, no usage limits. If you are evaluating a separate commercial package, use that product channel for exact packaging and support terms. This documentation focuses on the CE code in this repository. We've created detailed comparison guides: - [WhoDB vs phpMyAdmin](/resources/comparisons/vs-phpmyadmin) - [WhoDB vs pgAdmin](/resources/comparisons/vs-pgadmin) - [WhoDB vs DBeaver](/resources/comparisons/vs-dbeaver) - [WhoDB vs TablePlus](/resources/comparisons/vs-tableplus) The short answer: WhoDB is faster, simpler, and more modern while supporting multiple database types. Yes. For production access, use read-only database accounts where possible, enable SSL/TLS, and put the app behind your normal network and authentication controls. Credentials are kept server-side: after login, WhoDB stores them encrypted (AES-256-GCM) in a local SQLite session database, and your browser only holds an opaque, `HttpOnly` session cookie. Secrets are stripped from anything the frontend persists — only non-secret display details (connection name, host, database) remain in the browser so the profile switcher survives a reload. Environment-defined profiles are another option when you want connection details supplied by the server environment instead of typed manually. Yes! WhoDB is designed for self-hosting. You can run it on your own servers, in Docker containers, on Kubernetes, or even on your local machine. You have complete control over where your data goes. Check our [Installation Guide](/installation) for deployment options. WhoDB Community Edition supports: - PostgreSQL - MySQL / MariaDB - SQLite 3 - MongoDB - Redis The current CE catalog also covers engines and compatible variants such as CockroachDB, TiDB, DuckDB, OpenSearch, Memcached, Valkey, Dragonfly, YugabyteDB, QuestDB, and FerretDB. See the [full list](/resources/supported-databases). Migration is simple because WhoDB works with your existing databases - no data migration required! Just: 1. Install WhoDB 2. Connect to your existing databases 3. Start using WhoDB alongside your current tool 4. Switch completely when you're comfortable We have migration guides for users coming from [phpMyAdmin](/guides/migrating-from-phpmyadmin) and [pgAdmin](/guides/migrating-from-pgadmin). ## What Developers Say "WhoDB has become my go-to tool for database work. It's fast, it's beautiful, and it just works. I recommend it to every developer on my team." **— Sarah Chen, Senior Backend Developer** "The graph visualization alone makes WhoDB worth it. Understanding complex schemas used to take hours. Now it takes minutes." **— Michael Rodriguez, Database Architect** "Finally, a database tool that doesn't make me want to pull my hair out. WhoDB is what database management should have been all along." **— Alex Thompson, Full-Stack Developer** ## Ready to Try WhoDB? Start with the current CE build and validate it against your databases and workflows. Install the current CE build Star us on GitHub ⭐ **No credit card required. No account needed. Start using WhoDB immediately.** --- ### .Agents Docs Analytics (`.agents/docs/analytics.md`) # Analytics Contract (PostHog) All senders (browser frontend, Go backend, CLI, desktop) share one PostHog project. This contract keeps their events consistent and queryable. Follow it whenever adding or changing analytics events or properties. ## Required properties on every event Stamped automatically at the two choke points — do not set these manually at call sites, and do not bypass the choke points: | Property | Values | Stamped by | |---|---|---| | `source` | `web`, `backend`, `cli`, `desktop` | Go: `buildProperties()` from `Config.Source`; TS: `sanitizeAnalyticsProperties()` | | `build_edition` | `ce`, `ee` | Go: `Config.Edition` (required — never leave unset); TS: runtime context | | `build_environment` | e.g. `production`, `cli` | Go: `Config.Environment`; TS: runtime context | | `app_version` | semver | Go config; TS not applicable | | `platform` | `browser`, `wails` | Web events only (TS runtime context) | | `$host` | request/browser host | Go: from request metadata; TS: auto-captured by posthog-js | Choke points: - Go: `buildProperties()` in `core/src/analytics/posthog.go` — every capture from every Go sender flows through it. - TS: `sanitizeAnalyticsProperties()` in `frontend/src/config/analytics-sanitize.ts`. Naming: use the `build_`-prefixed names above. Do not introduce `environment`, `edition`, or a second `platform` meaning. Desktop OS goes in `os` (GOOS), not `platform`. ## Event ownership: one owner per concept - **Backend owns "it happened" events** — resource created/updated/deleted, runs, auth outcomes, billing. Server-side capture cannot be ad-blocked and is the source of truth for counts and funnels. - **Frontend owns UX-only events** — screen views, form opened/submitted/abandoned, option toggles, client-side failures (`ui.*` and `*_opened`/`*_abandoned`-style funnel events). Never emit the same concept from both sides: that double-counts every action. If the backend already records it, the frontend must not. ## Identity - Backend distinct IDs resolve in order: registered resolver (`analytics.SetDistinctIDResolver`, set at boot by edition entry points) → `X-WhoDB-Analytics-Id` header from the frontend → `anonymous`. - Never invent synthetic distinct IDs (random, per-process, hashed timestamps). Unattributable events belong to `anonymous`. ## Privacy - Never send raw error messages. Map errors through `analytics.ErrorCode` (Go) or the frontend error-code helpers to a fixed taxonomy. - Frontend exception captures may include `error_frame` (bundle file name with line and column of the first same-origin stack frame) and `error_component` (innermost React component name). Both are build identifiers; never add the message, full stack, or URL. - New properties must pass the sanitizer allowlists (`SAFE_ANALYTICS_PROPERTY_KEYS` in `frontend/src/config/analytics-events.ts`; backend detail allowlists in the edition analytics wrappers). No hostnames, emails, SQL, credentials, file paths, or free text. - High-cardinality values (counts, sizes, durations, text lengths) go through the bucket helpers, not raw. ## Consent - Single source of truth: `frontend/src/config/posthog.tsx` — use its exports (`CONSENT_STORAGE_KEY`, `remoteAnalyticsAllowed`, `grantStoredConsentIfUnset`, `getStoredConsentState`). Never read or write the consent localStorage key directly, and never duplicate the host-gating logic. - Consent-unknown users are not captured (`opt_out_capturing_by_default`). ## Install heartbeat (separate from all of the above) `telemetry.heartbeat` (`core/src/analytics/heartbeat.go`) is an anonymous install counter, sent once at startup and every 24h from Go backends and desktop. It deliberately bypasses `Enabled()` and `buildProperties()`: - Distinct ID is a **stable random install UUID** persisted in the unified config file — this makes unique installs, retention, and version adoption measurable. It has no relation to the user or machine, and deleting the config file resets it. - `$process_person_profile: false` (no person created) and `$ip: ""` (IP discarded at ingestion). - Payload is fixed: `build_edition`, `app_version`, `dev_build`, `os`, `arch`, `source`. Never add request context, usage data, or new properties to it — the install ID must never become linkable to behavioral analytics or a person. - Source builds (no ldflags-stamped version) are tagged `dev_build: true` and `app_version: "dev"` so real installs are filterable, not skipped. CI runs (`CI=true`) are skipped entirely. - Governed solely by `WHODB_HEARTBEAT_DISABLED=true`, independent of the consent system. It is publicly documented in the README; keep code, README, and this doc in sync if the payload ever changes. ## Verifying changes - A PostHog "data quality" insight counts events with null `build_edition` or `source`, broken down by `$lib`. If your change makes it nonzero, you broke the contract. - Locally, frontend events echo to the debug sink (`window.__WHODB_ANALYTICS_DEBUG__` and console) when remote analytics is disallowed — check required properties there before shipping. --- ### .Agents Docs Aws Integration (`.agents/docs/aws-integration.md`) # AWS Integration This document describes the AWS-specific implementation details for WhoDB. For the generic cloud provider architecture and how to add new providers (GCP, Azure), see [Cloud Providers](./cloud-providers.md). The AWS integration allows WhoDB to auto-discover source resources from AWS accounts and connect to them using the existing source catalog and plugin layer. ## Architecture Overview AWS is implemented as a **Connection Provider**, not a separate source type. This means: 1. **AWS discovers source resources** (RDS, ElastiCache, DocumentDB, plus registered extensions such as S3) 2. **AWS generates credentials** that existing connectors/plugins understand 3. **The existing source/session flow connects** using those credentials The MySQL plugin doesn't know or care if it's connecting to AWS RDS MySQL or a self-hosted MySQL - it just receives standard credentials. ``` ┌─────────────────────────────────────────────────────────────────┐ │ Connection Sources │ ├─────────────────────────────────────────────────────────────────┤ │ Manual Entry (current) │ AWS Provider (new) │ │ - User enters hostname │ - Auto-discovers resources │ │ - User enters credentials │ - Builds credentials for: │ │ │ • RDS → MySQL/Postgres/MariaDB│ │ │ • ElastiCache → Redis │ │ │ • DocumentDB → MongoDB │ │ │ • Extensions → source types │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ Existing Database Plugins │ │ MySQL │ PostgreSQL │ MariaDB │ Redis │ MongoDB │ ... │ │ (unchanged - they receive standard engine.Credentials) │ └─────────────────────────────────────────────────────────────────┘ ``` ## Package Structure ``` core/src/ ├── aws/ # AWS infrastructure layer │ ├── config.go # AWS SDK configuration │ ├── credentials.go # Credential parsing and provider building │ ├── errors.go # AWS error handling │ ├── profiles.go # Local AWS profile discovery │ ├── regions.go # AWS region definitions │ └── *_test.go # Tests │ └── providers/ # Connection provider abstraction ├── provider.go # ConnectionProvider interface ├── registry.go # Provider registry └── aws/ # AWS provider implementation ├── provider.go # Main AWS provider ├── rds.go # RDS instance discovery ├── elasticache.go # ElastiCache cluster discovery ├── documentdb.go # DocumentDB cluster discovery └── *_test.go # Tests ``` ## Key Interfaces ### ConnectionProvider All connection sources implement this interface: ```go type ConnectionProvider interface { Type() ProviderType ID() string Name() string DiscoverConnections(ctx context.Context) ([]DiscoveredConnection, error) TestConnection(ctx context.Context) error RefreshConnection(ctx context.Context, connectionID string) (bool, error) Close(ctx context.Context) error } ``` ### DiscoveredConnection Represents a source discovered by a provider. Internally the provider layer still stores an `engine.DatabaseType`; the GraphQL layer translates that into the public `SourceType` string. ```go type DiscoveredConnection struct { ID string // "providerID/resourceID" ProviderType ProviderType // "aws", "manual" ProviderID string // Provider instance ID Name string // Display name DatabaseType engine.DatabaseType // MySQL, Postgres, Redis, etc. Region string // Geographic location Status ConnectionStatus // available, starting, stopped, etc. Metadata map[string]string // Provider-specific details } ``` ## AWS Provider Configuration ```go config := &aws.Config{ ID: "aws-us-west-2", Name: "Production AWS", Region: "us-west-2", AuthMethod: awsinfra.AuthMethodDefault, // or profile DiscoverRDS: true, DiscoverElastiCache: true, DiscoverDocumentDB: true, DiscoverS3: true, // used when an S3 discovery extension is registered } provider, err := aws.New(config) ``` ### Authentication Methods | Method | Description | Use Case | |--------|-------------|----------| | `default` | AWS SDK credential chain | Recommended for most cases | | `profile` | Named AWS profile | Multiple AWS accounts | No credentials are stored in config.json. For explicit access keys, set `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` env vars (picked up by the `default` chain). Discovery flags default to enabled for newly configured providers. Persisted provider configs that predate a newer discovery flag should be treated as if the new flag is enabled unless the user explicitly disables it later. ## Usage Example ```go // Create provider provider, _ := aws.New(&aws.Config{ ID: "aws-prod", Name: "Production", Region: "us-west-2", }) // Register with the global registry registry := providers.GetDefaultRegistry() registry.Register(provider) // Discover all connections ctx := context.Background() connections, _ := registry.DiscoverAll(ctx) // Connection metadata (endpoint, port, TLS) is exposed to frontend. // Users select a connection in the UI, which prefills the login form. // Users can then modify values (e.g., hostname to localhost for tunneling) // before submitting via the standard LoginSource mutation. // The source login resolver maps source types to connectors/plugins: // - ElastiCache → Redis plugin // - DocumentDB → MongoDB plugin // - S3 extension → S3 source connector ``` ## Supported AWS Services ### RDS (Relational Database Service) Discovers MySQL, PostgreSQL, and MariaDB instances including Aurora variants. **Mapped to WhoDB types:** - `mysql` → `MySQL` - `mariadb` → `MariaDB` - `postgres` → `Postgres` - `aurora-mysql` → `MySQL` - `aurora-postgresql` → `Postgres` **IAM Authentication:** Supported when enabled on the RDS instance. The provider generates short-lived auth tokens (15-min validity). ### ElastiCache Discovers Redis clusters (both standalone and replication groups). **Note:** Memcached is not supported because WhoDB doesn't have a Memcached plugin. **Supports:** - Cluster mode enabled/disabled - AUTH token authentication - TLS encryption ### DocumentDB Discovers MongoDB-compatible DocumentDB clusters. **Note:** DocumentDB requires TLS and doesn't support all MongoDB features (e.g., no retryWrites). ## Error Handling The `aws` package maps AWS SDK errors to user-friendly messages: ```go var ( ErrAccessDenied // IAM permission issues ErrInvalidCredentials // Bad access key/secret ErrExpiredCredentials // Session token expired ErrResourceNotFound // Resource doesn't exist ErrThrottling // Rate limited ErrServiceUnavailable // AWS service issues ErrConnectionFailed // Network issues ) // Usage if aws.IsRetryable(err) { // Safe to retry } ``` ## Testing Tests are designed to run without AWS credentials: ```bash # Run AWS infrastructure tests go test ./src/aws/... # Run provider tests go test ./src/providers/... ``` ## Security Considerations 1. **No credentials stored** - config.json contains only region, profile name, and discovery flags 2. **Use IAM roles** in production - Avoid static credentials on AWS infrastructure 3. **TLS required** - DocumentDB and optionally ElastiCache require TLS ## Environment Variables Configure AWS providers via environment: ```bash # Single provider with default auth WHODB_AWS_PROVIDER='[{ "name": "Production", "region": "us-west-2" }]' # Multiple regions with a named profile WHODB_AWS_PROVIDER='[ {"name": "US", "region": "us-west-2"}, {"name": "EU", "region": "eu-west-1", "profileName": "eu-account"} ]' ``` ## Future Extensions These are planned but not yet implemented: 1. **Secrets Manager integration** - Retrieve database passwords from AWS Secrets Manager 2. **S3 export** - Export query results to S3 3. **CloudWatch metrics** - Show database performance metrics --- ### .Agents Docs Build And Run (`.agents/docs/build-and-run.md`) # CE Build and Run Guide Use this guide for the Community Edition development workflow. For the broader command reference, see [commands.md](commands.md). ## Quick Start ```bash cd core && go run ./cmd/whodb ``` ## GraphQL Generation ### Backend ```bash cd core && go generate ./... ``` ### Frontend ```bash # Start backend first cd core && go run ./cmd/whodb # Generate types cd frontend && pnpm run generate ``` ## Dependency Management ```bash cd core && go mod tidy ``` ## Development ### Backend ```bash cd core && go run ./cmd/whodb ``` ### Frontend ```bash cd frontend pnpm install pnpm start ``` --- ### .Agents Docs Ci Cd (`.agents/docs/ci-cd.md`) # CI/CD and GitHub Actions WhoDB uses GitHub Actions for automated builds, testing, and multi-platform deployment. ## Workflow Structure ``` .github/workflows/ lint-check.yml # PR lint + typecheck gate (Go + Frontend) e2e-ce.yml # CE end-to-end tests security.yml # CE security checks release-ce.yml # Main CE release orchestrator _build-*.yml # Reusable build workflows (called by release) _deploy-*.yml # Reusable deploy workflows _sign-validate.yml # Code signing validation ee/.github/workflows/ lint-check.yml # EE lint and typecheck gate e2e-ee.yml # EE end-to-end tests go-deps.yml # EE dependency checks security.yml # EE security checks release-ee.yml # EE release orchestrator _build-docker-ee.yml _deploy-docker-ee.yml _build-docker-bridge.yml _deploy-docker-bridge.yml .github/actions/ calculate-version/ # Semantic version calculation deployment-summary/ # Release summary generator .github/scripts/ appstore-connect.sh # Apple App Store Connect API build-appimage.sh # AppImage packaging generate-homebrew-cask.sh # Homebrew formula generation .github/templates/ RELEASE_TEMPLATE.md # GitHub release notes template AppxManifest.xml.template # Windows Store manifest homebrew-cask.rb.template # Homebrew cask template ``` ## Release Workflow (release-ce.yml) The main release workflow orchestrates all builds and deployments. ### Triggers - **Push to `release` branch** - Automatic deployment - **Manual dispatch** - Fine-grained control ### Deployment Modes | Mode | Description | Channels | |------|-------------|----------| | `stage-only` | Testing builds | Docker (tag), Snap (edge), MS Store (draft), Apple (TestFlight) | | `production` | User releases | Docker (latest), Snap (stable), MS Store (prod), Apple (App Store) | ### Version Bumping ``` major # 1.0.0 → 2.0.0 minor # 0.61.0 → 0.62.0 patch # 0.61.0 → 0.61.1 current # Rebuild without version increment ``` ### Store Selection Each deployment target can be enabled individually: - Docker Hub - Snap Store (Linux) - Microsoft Store (Windows) - Apple App Store (macOS) - Linux Terminal binaries (amd64, arm64, riscv64, armv6, armv7) ## Reusable Build Workflows Prefixed with `_` to indicate they're called by other workflows. ### _build-docker.yml - Builds Docker images for linux/amd64 and linux/arm64 - Uses native ARM runners for true ARM64 builds - Outputs artifacts for multi-arch manifest ### _build-apple.yml - Builds macOS DMG (direct download) and MAS (App Store) variants - Universal binary (Intel + Apple Silicon) - Handles code signing with Apple certificates ### _build-windows.yml - Builds MSIX package for Microsoft Store - Code signing with certificate - Supports Windows AMD64 ### _build-snap.yml - Builds Snap package for Snapcraft/Ubuntu Store - Supports multiple architectures ### _build-linux-terminal.yml - Builds standalone terminal binaries (no GUI) - Cross-compiles for: amd64, arm64, riscv64, armv6, armv7 ## Deployment Workflows ### _deploy-docker.yml - Pushes multi-arch manifest to Docker Hub - Tags: `latest`, `vX.Y.Z`, `edge` (for staging) ### _deploy-apple.yml - Uploads to App Store Connect - Handles TestFlight (staging) vs App Store (production) - Notarization for DMG downloads ### _deploy-microsoft.yml - Submits to Microsoft Partner Center - Draft submission (staging) vs production release ### _deploy-snap.yml - Publishes to Snapcraft - Edge channel (staging) vs stable (production) ### _deploy-homebrew.yml - Updates Homebrew cask formula - Generates SHA256 checksums ## Custom Actions ### calculate-version Calculates semantic version based on: - Git tags (latest tag as base) - Bump type input (major/minor/patch/current) - Outputs version for all downstream jobs ### deployment-summary Generates deployment summary with: - Build artifacts and sizes - Store deployment status - Download links ## Environment Secrets Required See `ee/.agents/docs/github-actions-secrets.md` for the inventory derived from the checked-in CE and EE workflows. The workflow files remain the executable source of truth whenever a secret reference changes. ## Running Releases ### Stage Release (Testing) 1. Go to Actions → Release CE 2. Select `stage-only` mode 3. Choose stores to deploy 4. Run workflow ### Production Release 1. Merge to `release` branch, OR 2. Manual dispatch with `production` mode 3. Enable `publish-github-release` to make release public ## Platform Build Tags (ARM / RISC-V64) Some CE-owned features depend on CGO libraries or platform-specific bindings that are unavailable on ARM and RISC-V64. These are excluded using Go build tags. **Pattern:** every file with `//go:build !arm && !riscv64` MUST have a matching `_unsupported.go` stub file with `//go:build arm || riscv64` that provides the same exported function signatures (returning HTTP 501 or no-op behavior). ``` core/graph/ http_ai_stream_handler.go → http_ai_stream_unsupported.go ``` Edition or add-on HTTP routes should not add CE route names and CE unsupported stubs. Register those routes from the add-on package with `graph.RegisterHTTPRoutes`. **When adding a new HTTP handler with a build-tag constraint:** 1. Create the main handler file with `//go:build !arm && !riscv64` 2. Create a corresponding `_unsupported.go` with `//go:build arm || riscv64` 3. The stub must define every function referenced from `SetupHTTPServer` or other unconditionally-compiled code 4. Verify with: `GOOS=linux GOARCH=riscv64 go build ./graph/` If this is missed, CI will fail on the "Build Linux Binaries" job for RISC-V64. ## Security All workflows use: - `step-security/harden-runner` for egress control - Pinned action versions with SHA hashes - Environment-based secret access - Minimal permissions per job --- ### .Agents Docs Cli (`.agents/docs/cli.md`) # WhoDB CLI The CLI is an interactive terminal interface for WhoDB with split-pane TUI support. It lives in `cli/` and is a separate Go module. ## Quick Reference ```bash # Build cd cli && go build -o whodb . # Run interactive mode ./whodb --tui # Connect to database ./whodb connect --type postgres --host localhost --user postgres --database mydb ./whodb connect --type sqlite3 --database ./app.db ./whodb connect --docker # Execute query ./whodb query "SELECT * FROM users" --connection my-postgres ./whodb query --stream --format ndjson "SELECT * FROM audit_log" # Cloud discovery (when provider support is enabled) ./whodb cloud providers list ./whodb cloud connections list ./whodb connect --discovered aws-prod-us-west-2/prod-db # Schema diff ./whodb diff --from staging --to prod # Mock data ./whodb mock-data --connection mydb --table orders --rows 50 --analyze # Configure modules once, then start the single MCP server whodb setup ./whodb mcp serve ``` ## Current Architecture ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Runtime Split The CLI now has a dedicated shared runtime layer: - `cli/app/app.go` is the shared process runner - `cli/internal/bootstrap/bootstrap.go` owns CE startup/bootstrap - `cli/pkg/identity/identity.go` owns runtime identity such as: - command name - display name - local config/cache directory naming - keyring service name - update-check URLs Keep CE defaults in `cli/pkg/identity`. Do not add non-CE product-specific values there. Shared code should consume generic identity fields instead of branching on edition names. ## Key Design Decisions 1. **Direct Plugin Access** — The CLI uses core plugins directly, not GraphQL, for command/TUI execution 2. **Shared Runtime** — `main.go` should stay thin and call the shared runner in `cli/app` 3. **Identity-Driven Text and Paths** — Command text, completion install paths, BAML library storage, keyring naming, and update messaging should come from `pkg/identity` 4. **Bubble Tea + Pane Model** — All panes implement the same interface and render into shared layouts 5. **Backend-Driven Database Catalog** — Database picker/completions should come from `core/src/dbcatalog`, not a CLI-local hardcoded list 6. **Shared Feature Surfaces** — Import, mock data, ERD graph, suggestions, and diff should reuse backend/shared logic rather than reimplementing locally 7. **Structured Automation Output** — Machine-readable commands should keep stable JSON/NDJSON contracts and avoid spinner noise 8. **Workspace Restore** — Persist only lightweight restorable UI state, not heavy query payloads 9. **Hosted Platform Manifest** — Hosted-platform commands use EE `PlatformManifest` as the source of truth for supported GraphQL operations and fields. When EE hosted schema changes affect public CLI capabilities, update `ee/core/graph/platform_manifest.go` and run its contract test. ## Current Capabilities - Split-pane TUI layouts with workspace restore - Query execution with plain/json/csv/ndjson output - Additive streaming mode for query/export - Backend-generated suggestions - Shared backend import pipeline - FK-aware mock data generation - Add/edit/delete row flows in the TUI - Schema diff command and TUI diff view - ERD based on backend graph data - SSL configuration and SSL status visibility - Expanded MCP read tool surface - Cloud provider discovery commands backed by the shared provider runtime - Discovered-resource connect/save flows built on the shared cloud prefill path ## Configuration CLI state is stored in the unified `config.json` under the `cli` section through `core/src/common/config`. The CLI also keeps edition-local files under the identity-specific home dir. Shared code should use `cli/pkg/identity.HomePath(...)` instead of hardcoding `~/.whodb-cli`. Passwords are stored in the OS keyring when available. The keyring service name is also identity-driven. ## Rules for Future CLI Changes ### 1. Keep entrypoints thin `main.go` should only: - set up identity/bootstrap - call the shared runtime Do not move command/TUI logic back into the binary entrypoint. ### 2. Do not hardcode command identity Avoid new hardcoded uses of: - `whodb` - `WhoDB CLI` - `.whodb-cli` - `WhoDB-CLI` - `WHODB_CLI` Use the shared identity helpers instead. ### 3. Prefer backend/shared sources over CLI-local copies Use shared backend logic for: - database catalog - import behavior - mock data planning/generation - graph/ERD data - query suggestions ### 4. Do not add edition checks to shared CLI features unless there is no better extension point If a future edition needs different behavior, prefer: - bootstrap/import differences - identity config - extension hooks Avoid baking product-specific branching into normal command/TUI paths. ## Testing ```bash cd cli # All CLI tests go test ./... # TUI tests go test ./internal/tui/... # Command tests go test ./cmd # Shared output/runtime tests go test ./pkg/output ./pkg/version ./pkg/identity # Full dev script bash ../dev/run-cli-tests.sh ``` ## Related Files - `cli/README.md` — user-facing usage guide - `.agents/docs/hosted-platform-cli.md` — hosted platform login, manifest, source command, automation, and security contract - `dev/run-cli-tests.sh` — CLI verification script ### MCP without skills Skills are optional supplements. MCP initialization instructions, tool schemas, and `whodb://mcp/configuration` provide essential guidance directly. Use `whodb_mcp_setup` actions `inspect`, `preview` (with a settings patch), `apply` (with the exact approved preview token and `approved: true`), and `verify`. No shell access is required for these actions. Browser login and credential entry remain user actions. Restart after changing settings; verification checks the active process. The setup tool follows tool filters and can be disabled when configuration should remain outside the agent. --- ### .Agents Docs Cloud Providers (`.agents/docs/cloud-providers.md`) # Cloud Provider Architecture This document describes the generic cloud provider architecture for WhoDB. Cloud providers auto-discover source resources from cloud accounts (AWS, GCP, Azure) and connect to them using the existing source catalog and connector stack. ## Architecture Pattern: Interface + Union The cloud provider system uses the **Interface + Union** pattern in GraphQL, which provides: 1. **Generic operations** that work across all providers 2. **Type-safe provider-specific inputs** for add/update operations 3. **O(1) complexity** when adding new providers (no existing code changes) ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Key Types ### CloudProviderType Enum ```graphql enum CloudProviderType { AWS Azure GCP } ``` ### CloudProviderStatus Enum ```graphql enum CloudProviderStatus { Connected Discovering Error Disconnected CredentialsRequired } ``` ### CloudProvider Interface Common fields that all providers implement: ```graphql interface CloudProvider { Id: ID! ProviderType: CloudProviderType! Name: String! Region: String! Status: CloudProviderStatus! LastDiscoveryAt: String DiscoveredCount: Int! Error: String } ``` ### DiscoveredConnection Type Represents a source found by any provider: ```graphql type DiscoveredConnection { Id: ID! ProviderType: CloudProviderType! ProviderID: String! Name: String! SourceType: String! Region: String! Status: String! Metadata: [Record!]! } ``` ## Generic vs Provider-Specific Operations ### Generic Operations (work with all providers) | Operation | Description | |-----------|-------------| | `CloudProviders` | List all configured providers | | `CloudProvider(id)` | Get a specific provider | | `RemoveCloudProvider(id)` | Remove any provider | | `TestCloudProvider(id)` | Test any provider connection | | `RefreshCloudProvider(id)` | Refresh discovery for any provider | | `DiscoveredConnections` | List all discovered connections | ### Provider-Specific Operations (type-safe inputs) | Operation | Description | |-----------|-------------| | `AddAWSProvider(input)` | Add AWS provider with AWS-specific config | | `UpdateAWSProvider(id, input)` | Update AWS provider | | `AddGCPProvider(input)` | Add GCP provider with GCP-specific config | | `UpdateGCPProvider(id, input)` | Update GCP provider | | `AddAzureProvider(input)` | Add Azure provider with Azure-specific config | | `UpdateAzureProvider(id, input)` | Update Azure provider | ## Adding a New Cloud Provider Follow these steps to add support for a new cloud provider: ### 1. Update GraphQL Schema (`core/graph/schema.graphqls`) ```graphql # Add to CloudProviderType enum enum CloudProviderType { AWS GCP # Add new provider } # Create provider-specific type implementing CloudProvider type GCPProvider implements CloudProvider { # Interface fields (required) Id: ID! ProviderType: CloudProviderType! Name: String! Region: String! Status: CloudProviderStatus! LastDiscoveryAt: String DiscoveredCount: Int! Error: String # GCP-specific fields ProjectID: String! ServiceAccountEmail: String DiscoverCloudSQL: Boolean! DiscoverMemorystore: Boolean! DiscoverBigtable: Boolean! } # Create provider-specific input input GCPProviderInput { Name: String! Region: String! ProjectID: String! ServiceAccountKey: String # JSON key file content DiscoverCloudSQL: Boolean DiscoverMemorystore: Boolean DiscoverBigtable: Boolean } # Add mutations (in Mutation type) AddGCPProvider(input: GCPProviderInput!): GCPProvider UpdateGCPProvider(id: ID!, input: GCPProviderInput!): GCPProvider ``` ### 2. Regenerate Go Models ```bash cd core && go generate ./... ``` ### 3. Create Provider Package (`core/src/providers/gcp/`) ``` core/src/providers/gcp/ ├── provider.go # Main GCP provider implementation ├── cloudsql.go # Cloud SQL discovery ├── memorystore.go # Memorystore (Redis) discovery └── provider_test.go # Tests ``` **provider.go:** ```go package gcp import ( "context" "github.com/clidey/whodb/core/src/providers" ) type Config struct { ID string Name string Region string ProjectID string ServiceAccountKey string DiscoverCloudSQL bool DiscoverMemorystore bool } type Provider struct { config *Config // GCP clients } func New(config *Config) (*Provider, error) { // Initialize GCP clients return &Provider{config: config}, nil } func (p *Provider) Type() providers.ProviderType { return providers.ProviderTypeGCP } func (p *Provider) ID() string { return p.config.ID } func (p *Provider) DiscoverConnections(ctx context.Context) ([]providers.DiscoveredConnection, error) { var connections []providers.DiscoveredConnection if p.config.DiscoverCloudSQL { sqlConns, err := p.discoverCloudSQL(ctx) if err != nil { return nil, err } connections = append(connections, sqlConns...) } // Add other discovery methods... return connections, nil } // Implement remaining ConnectionProvider interface methods... ``` ### 4. Update Resolvers (`core/graph/schema.resolvers.go`) ```go // Add GCP-specific mutations func (r *mutationResolver) AddGCPProvider(ctx context.Context, input model.GCPProviderInput) (*model.GCPProvider, error) { // Create and register GCP provider } func (r *mutationResolver) UpdateGCPProvider(ctx context.Context, id string, input model.GCPProviderInput) (*model.GCPProvider, error) { // Update GCP provider } // Update generic resolvers to handle GCP func (r *queryResolver) CloudProviders(ctx context.Context) ([]model.CloudProvider, error) { // Return both AWS and GCP providers } ``` ### 5. Update Settings State (`core/src/settings/providers.go`) Add GCP to the settings state management if needed. ### 6. Update Frontend **Add GraphQL operations** (`frontend/src/pages/settings/gcp-providers.graphql`): ```graphql mutation AddGCPProvider($input: GCPProviderInput!) { AddGCPProvider(input: $input) { Id ProviderType Name Region ProjectID # ... other fields } } ``` **Create UI components** (`frontend/src/components/gcp/`): ``` frontend/src/components/gcp/ ├── gcp-provider-modal.tsx # Add/edit GCP provider ├── gcp-connection-picker.tsx └── index.ts ``` **Update Redux store** - the generic `cloudProviders` array already supports multiple provider types via the `ProviderType` discriminator. ### 7. Add Localization Create YAML files for GCP-specific translations: ```yaml # frontend/src/locales/components/gcp-provider-modal.yaml en: projectId: "GCP Project ID" serviceAccountKey: "Service Account Key" discoverCloudSQL: "Discover Cloud SQL" ``` ### 8. Add Tests - Unit tests for GCP provider package - Integration tests for discovery - E2E tests for UI flows ## File Locations | Component | Location | |-----------|----------| | GraphQL Schema | `core/graph/schema.graphqls` | | Go Models | `core/graph/model/models_gen.go` (generated) | | Resolvers | `core/graph/schema.resolvers.go` | | Provider Interface | `core/src/providers/provider.go` | | Provider Registry | `core/src/providers/registry.go` | | AWS Provider | `core/src/providers/aws/` | | GCP Provider | `core/src/providers/gcp/` | | Azure Provider | `core/src/providers/azure/` | | Settings State | `core/src/settings/providers.go` | | Frontend Types | `frontend/src/generated/graphql.tsx` (generated) | | Redux Store | `frontend/src/store/providers.ts` | | AWS Components | `frontend/src/components/aws/` | ## Design Principles 1. **Generic where possible** - Use `CloudProvider` interface for shared operations 2. **Specific where needed** - Use provider-specific types for inputs requiring validation 3. **No switch statements** - Provider logic stays in provider packages, not shared code 4. **Discriminated unions** - Use `ProviderType` to identify provider at runtime 5. **Backwards compatible** - Adding providers doesn't change existing GraphQL queries ## Connection Prefill Rules When a user selects a discovered cloud connection, the frontend prefills connection fields (hostname, database/bucket, port, and advanced settings). The login flow and the EE New Data Source form both use the shared prefill helper. ### Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ cloud-connection-prefill.ts │ ├─────────────────────────────────────────────────────────────────┤ │ basePrefillRules (CE) │ │ ┌─────────────────────────────────────────────────────────────┐ │ │ │ SourceTypeA → { "Setting": "value" } │ │ │ │ SourceTypeB → { "TLS": "true" } (conditional on meta) │ │ │ └─────────────────────────────────────────────────────────────┘ │ │ ▼ │ │ merge at runtime (if extensions loaded) │ │ ▼ │ │ extensionPrefillRules │ │ ┌─────────────────────────────────────────────────────────────┐ │ │ │ ExtensionSourceTypeA → { "Custom Setting": "value" } │ │ │ │ CustomSourceTypeB → { ... } │ │ │ └─────────────────────────────────────────────────────────────┘ │ │ ▼ │ │ prefillRules (combined) │ │ ▼ │ │ buildConnectionPrefill(conn) │ └─────────────────────────────────────────────────────────────────┘ ``` ### File Locations | Component | Location | |-----------|----------| | CE Prefill Logic | `frontend/src/utils/cloud-connection-prefill.ts` | | Picker Component | `frontend/src/components/aws/aws-connection-picker.tsx` | ### Adding a New Prefill Rule (CE) Edit `frontend/src/utils/cloud-connection-prefill.ts`: ```typescript const basePrefillRules: Record = { // Existing rules... // Simple rule - always apply these settings NewSourceType: () => ({ "Setting Name": "value" }), // Conditional rule - check metadata from discovery AnotherSourceType: (_, meta) => meta("someMetadataKey") === "true" ? { "Conditional Setting": "value" } : {}, }; ``` ### Adding a New Prefill Rule (Extension) ```typescript export const eePrefillRules: Record = { CustomSourceType: () => ({ "Custom Setting": "value" }), // Conditional based on metadata AnotherEEType: (_, meta) => { const customValue = meta("customMetadataKey"); return customValue ? { "Custom Setting": customValue } : {}; }, }; ``` ### How It Works 1. **Backend Discovery** - Provider discovers connections and populates `Metadata` map 2. **GraphQL Response** - `DiscoveredConnection.Metadata` contains allowed keys (endpoint, port, etc.) 3. **Frontend Prefill** - `buildConnectionPrefill(conn)` applies rules based on `SourceType` 4. **Connection Form** - Receives prefill data and populates hostname, database/bucket, port, and advanced settings ### PrefillRule Function Signature ```typescript type PrefillRule = ( conn: LocalDiscoveredConnection, // Full connection object meta: (key: string) => string | undefined // Helper to read metadata ) => Record; // Advanced settings to apply ``` ### Available Metadata Keys The backend exposes these metadata keys for prefill decisions: | Key | Description | |-----|-------------| | `endpoint` | Database hostname/endpoint | | `port` | Database port | | `databaseName` | Default database/bucket value | | `bucket` | S3 bucket name | | `authMethod` | Cloud-provider auth method | | `profileName` | Cloud-provider profile name | | `transitEncryption` | Whether TLS is enabled (for cache services) | | `serverless` | Whether instance is serverless | | `iamAuthEnabled` | Whether IAM auth is available | | `authTokenEnabled` | Whether auth token is enabled | ### Design Principles 1. **CE defines base rules** - Common source types in `basePrefillRules` 2. **Extensions merge with base rules, don't replace** 3. **Rules are per source type** - Each source type has explicit rules 4. **Metadata-driven** - Rules can inspect discovery metadata for conditional logic 5. **No provider knowledge in rules** - Rules don't know about AWS/GCP, only SourceType ## Related Documentation - [AWS Integration](./aws-integration.md) - AWS-specific implementation details - [Plugin Architecture](./plugin-architecture.md) - Database plugin pattern (similar concept) --- ### .Agents Docs Commands (`.agents/docs/commands.md`) # Development Commands Reference ## Backend ```bash # Run backend cd core && go run ./cmd/whodb # Build CE binary cd core && go build -o whodb ./cmd/whodb # Lint core backend cd core && ./lint.sh # Run tests (see testing.md for full guide) cd core && go test ./... bash dev/run-backend-tests.sh all # Unit + integration ``` ## Frontend ```bash # Run frontend (separate terminal) cd frontend && pnpm start # Lint + type check cd frontend && pnpm lint # oxlint cd frontend && pnpm lint:fix # oxlint --fix cd frontend && pnpm typecheck # tsc --noEmit # E2E tests (Playwright) - see testing.md for full guide cd frontend && pnpm e2e:ce # Interactive, all databases cd frontend && pnpm e2e:ce:headless # Headless, all databases cd frontend && pnpm e2e:db postgres # Single database # Available: postgres, mysql, mysql8, mariadb, sqlite, mongodb, redis, elasticsearch, clickhouse ``` ## CLI ```bash # Build CLI cd cli && go build -o whodb . # Lint CLI only cd cli && make lint # Run interactive mode cd cli && go run . # Run CLI tests (see testing.md for full guide) bash dev/run-cli-tests.sh # All CLI tests cd cli && go test ./... # Unit tests only ``` ## GraphQL Workflow When modifying GraphQL: 1. Edit schema: `core/graph/schema.graphqls` 2. Run backend code generation: `cd core && go generate ./...` 3. Start backend server 4. Run frontend code generation: `cd frontend && pnpm run generate` 5. Import generated hooks from `@graphql` alias ```typescript // Always use generated hooks, never inline gql strings import { useMyQuery } from '@graphql'; ``` --- ### .Agents Docs Data Sources (`.agents/docs/data-sources.md`) # The Ultimate WhoDB Data Source Implementation Guide This document is the **strict, authoritative protocol** for adding new data sources to WhoDB. The entire project relies on the architectural consistency defined here. **Do not deviate from these patterns.** Whether you are an AI assistant (Claude, Codex, Gemini) or a human developer, you must follow this guide exhaustively. --- ## How to Use This Guide WhoDB supports **two implementation paths** for new data sources: | Path | When to Use | Base Layer | |------|-------------|------------| | **A: Database Plugin** | SQL/NoSQL databases that speak a wire protocol | `engine.PluginFunctions` + `DatabaseConnector` adapter | | **B: Source-First Connector** | Non-database sources (S3, APIs, queues, graph DBs, anything else) | `source.SourceConnector` + `source.SourceSession` interfaces directly | Both paths share the same catalog registration, frontend integration, and GraphQL exposure. The only difference is how the backend session is implemented. Read the **Core Architecture** section first, then follow the path that matches your source. --- ## Core Architecture ### Mental Model ``` ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ ┌────────────┐ │ Frontend │────▶│ GraphQL API │────▶│ Source Registry │────▶│ Driver │ │ (catalog │ │ (generic, │ │ (TypeSpec + │ │ (opens │ │ driven) │ │ source- │ │ SourceConnector│ │ sessions)│ │ │ │ agnostic) │ │ per DriverID) │ │ │ └─────────────┘ └──────────────┘ └─────────────────┘ └────────────┘ │ ┌────────────────────────┤ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ │ Path A: │ │ Path B: │ │ DatabaseConnector │ │ Custom Connector │ │ (DriverID = │ │ (DriverID = │ │ "database") │ │ your custom ID) │ │ adapts legacy │ │ implements source │ │ PluginFunctions │ │ interfaces directly│ └──────────────────┘ └──────────────────┘ ``` ### Key Principles 1. **Source-First GraphQL API** — You do NOT add custom GraphQL queries or mutations for new data sources. The public API (`SourceTypes`, `SourceObjects`, `SourceRows`, `RunSourceQuery`, `SourceGraph`, etc.) is generic and works for all source types automatically. 2. **Source Contract Authority** — Declare capabilities once in the source catalog. `SourceContract.Surfaces`, `RootActions`, `BrowsePath`, `ObjectTypes.Actions`, and `ObjectTypes.Views` are the single source of truth for what the backend allows and what the frontend shows. 3. **Plugin Self-Registration** — Plugins register themselves via `init()` functions. The frontend and API automatically adapt based on the plugin's declared catalog entries. 4. **CE vs. EE Strict Boundary** — CE (Community Edition) knows *nothing* about EE (Enterprise Edition). CE code must never contain `ee/` imports, references, or `if isEE` logic. EE extends CE purely through registries at boot time. Edition is controlled by which entry point is compiled (`core/cmd/whodb/main.go` for CE, `ee/cmd/whodb/main.go` for EE), not build tags. 5. **No Defensive Code** — Do not write fallback logic unless explicitly requested. 6. **No SQL Injection** — Parameterize values and render every raw schema/table/column component through `core/src/sqlident`. Metadata validation does not replace quoting at the SQL sink. See `.agents/docs/sql-security.md`. 7. **Localization** — All user-facing strings must use `t()` with YAML keys. No hardcoded UI text. --- ## Reference: Source-First Type System Before implementing anything, understand these types. Every value listed below is the **complete** set — do not invent new values. ### `source.Category` — Broad source classification | Value | Use For | |-------|---------| | `CategoryDatabase` | SQL/NoSQL databases | | `CategoryCache` | Redis, Memcached | | `CategorySearch` | ElasticSearch, OpenSearch | | `CategoryObjectStore` | S3, GCS, Azure Blob | | `CategoryFileStore` | Filesystem-like sources | ### `source.Model` — Data organization model | Value | Use For | |-------|---------| | `ModelRelational` | Tables with rows and columns | | `ModelDocument` | JSON documents / collections | | `ModelKeyValue` | Key-value stores | | `ModelSearch` | Full-text search indexes | | `ModelGraph` | Graph databases (nodes + edges) | | `ModelObject` | Object/blob storage | ### `source.Surface` — UI experiences enabled | Value | Description | |-------|-------------| | `SurfaceBrowser` | Object tree navigation | | `SurfaceQuery` | SQL/query editor | | `SurfaceChat` | AI chat interface | | `SurfaceGraph` | Relationship visualization | ### `source.ObjectKind` — Browseable entity types | Value | Use For | |-------|---------| | `ObjectKindDatabase` | Top-level container | | `ObjectKindSchema` | Namespace (Postgres schemas, Oracle schemas) | | `ObjectKindTable` | Tabular data | | `ObjectKindView` | Read-only table/view | | `ObjectKindCollection` | Document collection (MongoDB) | | `ObjectKindIndex` | Search index (Elasticsearch) | | `ObjectKindKey` | Cache key (Redis) | | `ObjectKindItem` | Item (Memcached) | | `ObjectKindFunction` | Function | | `ObjectKindProcedure` | Stored procedure | | `ObjectKindTrigger` | Trigger | | `ObjectKindSequence` | Sequence | ### `source.Action` — Operations per object kind | Value | Description | |-------|-------------| | `ActionBrowse` | Navigate into / list children | | `ActionInspect` | View metadata | | `ActionViewRows` | View tabular data | | `ActionViewContent` | View blob/text content | | `ActionViewDefinition` | View DDL/source definition | | `ActionCreateChild` | Create child objects | | `ActionDelete` | Delete object | | `ActionInsertData` | Insert rows/documents | | `ActionUpdateData` | Update rows/documents | | `ActionDeleteData` | Delete rows/documents | | `ActionImportData` | Bulk import | | `ActionGenerateMockData` | Generate synthetic data | | `ActionExecute` | Execute an action | | `ActionViewGraph` | Visualize as graph | ### `source.DataShape` — How data is rendered | Value | Use For | |-------|---------| | `DataShapeTabular` | Row/column grid | | `DataShapeDocument` | JSON/document viewer | | `DataShapeContent` | Blob/text content | | `DataShapeGraph` | Graph visualization | | `DataShapeMetadata` | Metadata-only display | ### `source.View` — Available rendering views `ViewGrid`, `ViewJSON`, `ViewText`, `ViewSQL`, `ViewBinary`, `ViewMetadata`, `ViewGraph` ### `source.ConnectionTransport` — How the source is reached | Value | Use For | |-------|---------| | `ConnectionTransportNetwork` | TCP/HTTP to remote host | | `ConnectionTransportFile` | Local file (SQLite, DuckDB) | | `ConnectionTransportBridge` | Bridge/sidecar transport (EE) | ### `ConnectionTraits.SupportsCustomCAContent` — SSL CA content upload A `bool` on `ConnectionTraits`. When `true` (the default for most network sources), the frontend SSL config allows the user to paste/upload a custom CA certificate. Set to `false` for sources where the system CA bundle is always used (e.g., MSSQL, Oracle with system-CA-only mode). See the EE `relationalDatabaseSchemaSystemCAFamily()` and `relationalSchemaSystemCAFamily()` helpers. ### `source.HostInputMode` — Hostname field behavior `HostInputModeNone`, `HostInputModeHostname`, `HostInputModeHostnameOrURL` ### `source.HostInputURLParser` — URL auto-parsing `HostInputURLParserNone`, `HostInputURLParserPostgres`, `HostInputURLParserMongoSRV` ### `source.ProfileLabelStrategy` — How saved profiles are labeled `ProfileLabelStrategyDefault`, `ProfileLabelStrategyHostname`, `ProfileLabelStrategyDatabase` ### `source.SchemaFidelity` — Metadata precision | Value | Use For | |-------|---------| | `SchemaFidelityExact` | Schema from system tables (SQL databases) | | `SchemaFidelitySampled` | Schema inferred from data samples (MongoDB, Elasticsearch) | ### `source.MetadataFidelity` — Column, constraint, graph, and filtering precision | Value | Use For | |-------|---------| | `MetadataFidelityExact` | Authoritative source metadata | | `MetadataFidelityDriver` | Metadata reported by a driver or bridge metadata API | | `MetadataFidelitySampled` | Metadata inferred from sampled documents or records | | `MetadataFidelityInferred` | Metadata inferred from conventions, such as ID-like names | | `MetadataFidelitySynthetic` | WhoDB-created metadata for synthetic shapes | | `MetadataFidelityUnsupported` | The metadata surface is not available | | `MetadataFidelityUnknown` | The source did not declare fidelity | `TypeTraits.Metadata` declares fidelity for `Columns`, `Constraints`, `Graph`, and `SystemObjectFiltering`. Database-backed sources should also declare internal object names or prefixes in the source catalog rather than filtering system schemas, collections, indices, or tables in shared code. ### `source.QueryExplainMode` — Query plan inspection `QueryExplainModeNone`, `QueryExplainModeExplain`, `QueryExplainModeExplainAnalyze`, `QueryExplainModeExplainPipeline` --- ## Reference: Source Session Interfaces GraphQL resolvers type-assert your session to check what it can do. Implement only the interfaces that apply to your source. At minimum, implement `SourceSession`. ### Type Ownership The `source` package owns all shared data types (`Column`, `Record`, `RowsResult`, `GraphUnit`, `ChatMessage`, `SSLStatus`, `TypeDefinition`, etc.). The `engine` package re-exports them as type aliases (in `core/src/engine/aliases.go`) so that existing database plugins can continue using `engine.Column` etc. without changes. When writing new code, prefer importing from `source` directly. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Database-backed sources must declare execution behavior in `SourceType.Traits.Query`: `SupportsScripts`, `SupportsStreaming`, and `SupportsMultiStatement`. Plain English: WhoDB now asks each source what kinds of execution it supports, then both the backend and frontend use that same answer instead of guessing from the database type or failing at runtime. --- ## Reference: ObjectType Builder Helpers When defining `ObjectTypes` in your `FamilySpec`, use these helpers from `sourcecatalog`. Do NOT construct `source.ObjectType` structs manually. ```go // Container objects (Database, Schema, Keyspace, etc.) // createChild=true adds ActionCreateChild metadataObjectType(kind, singularLabel, pluralLabel, createChild bool) // Fully mutable tabular objects (Tables) // Actions: Inspect, ViewRows, InsertData, UpdateData, DeleteData, ImportData, GenerateMockData tabularObjectType(kind, singularLabel, pluralLabel) // Read-only tabular objects (Views) // Actions: Inspect, ViewRows, ViewDefinition tabularReadOnlyObjectType(kind, singularLabel, pluralLabel) // Document objects (Collections, Indexes) // Actions: Inspect, ViewRows, InsertData, UpdateData, DeleteData, [extraActions...] documentObjectType(kind, singularLabel, pluralLabel, extraActions ...source.Action) // Fully mutable key-value objects (Redis keys) // Actions: Inspect, ViewRows, InsertData, UpdateData, DeleteData keyValueObjectType(kind, singularLabel, pluralLabel) // Existing-only mutable objects (Memcached items — no insert, only update/delete) // Actions: Inspect, ViewRows, UpdateData, DeleteData keyValueExistingMutableObjectType(kind, singularLabel, pluralLabel) ``` ### Trait Builder Helpers ```go // Network source with exact schema networkTraits(hostInputMode, urlParser) source.TypeTraits // Network source with sampled schema (MongoDB, Elasticsearch) sampledNetworkTraits(hostInputMode, urlParser, profileLabelStrategy) source.TypeTraits // File-based source (SQLite, DuckDB) fileTraits(profileLabelStrategy) source.TypeTraits // Bridge/sidecar source (EE only) bridgeTraits() source.TypeTraits // EE only, in ee/core/src/sourcecatalog/register.go ``` When a source has internal schemas, collections, indices, tables, or keys, declare them on `TypeTraits.Metadata.HiddenObjectNames` or `HiddenObjectPrefixes` in the catalog family. The database adapter applies those rules consistently to browse and graph metadata. --- ## Path A: Adding a Database Plugin Use this path for SQL/NoSQL databases. This is the most common path. ### Phase 1: Define the Engine Constant Every data source needs a unique `DatabaseType` identifier. - **CE sources**: Add to `core/src/engine/engine.go` - **EE sources**: Add to `ee/core/src/engine/types.go` ```go const DatabaseType_MyNewDB DatabaseType = "MyNewDB" ``` The string value must be globally unique and becomes the source's ID everywhere. ### Phase 2: Implement the Plugin Create your plugin package in `core/src/plugins//` (CE) or `ee/core/src/plugins//` (EE). #### For SQL databases: Embed `GormPlugin` ```go package mynewdb import ( "github.com/clidey/whodb/core/src/engine" gorm_plugin "github.com/clidey/whodb/core/src/plugins/gorm" ) type MyNewDBPlugin struct { gorm_plugin.GormPlugin } func init() { engine.RegisterPlugin(NewMyNewDBPlugin()) } func NewMyNewDBPlugin() *engine.Plugin { plugin := &MyNewDBPlugin{} plugin.Type = engine.DatabaseType_MyNewDB plugin.PluginFunctions = plugin plugin.GormPluginFunctions = plugin return &plugin.Plugin } ``` **You MUST implement these `GormPluginFunctions` methods** (the base provides defaults for most `PluginFunctions` methods, but these have no default): | Method | Purpose | |--------|---------| | `DB(config *engine.PluginConfig) (*gorm.DB, error)` | Open a GORM connection | | `GetPlaceholder(index int) string` | SQL parameter placeholder (e.g., `$1` for Postgres, `?` for MySQL) | | `GetTableInfoQuery() string` | SQL query listing tables with metadata for a schema | | `GetStorageUnitExistsQuery() string` | SQL query checking if a table exists | | `GetPrimaryKeyColQuery() string` | SQL query finding primary key columns | | `GetAllSchemasQuery() string` | SQL query listing all schemas | | `GetCreateTableQuery(db, schema, storageUnit, columns) string` | DDL for CREATE TABLE | | `GetSupportedOperators() map[string]string` | Map of display→SQL operators | | `GetGraphQueryDB(db, schema) *gorm.DB` | Query builder for graph/FK relationships | | `GetTableNameAndAttributes(rows) (string, []Record)` | Parse table info query results | | `ParseConnectionConfig(config) (*ConnectionInput, error)` | Parse credentials into connection params | **Size attribute conventions** — `GetTableNameAndAttributes` (and any custom `GetStorageUnits` implementation) must follow these rules for size-related attributes: | Rule | Detail | |------|--------| | **Emit raw bytes** | Size values MUST be numeric strings representing bytes (e.g., `"1048576"` not `"1 MB"`). The frontend owns formatting via `formatBytes()`. | | **Standard key names** | Use `"Total Size"` (data + indexes) and/or `"Data Size"` (data only). Do NOT invent custom names like `"Size (KB)"` or `"Storage Size"`. | | **Use `sql.NullInt64`** | Scan size columns as `sql.NullInt64`. Only append the attribute when `.Valid` is true — this gracefully handles tables without allocation data. | | **No backend formatting** | Never use `pg_size_pretty()`, `formatReadableSize()`, `ROUND(... / 1024)`, or `fmt.Sprintf("%.2f MB", ...)` in size queries. | | **Non-byte counts** | For counts that are NOT bytes (e.g., Redis element count), use a distinct key like `"Entries"` or `"Item Count"` — these must NOT be named `"Size"`. | Example (`GetTableNameAndAttributes`): ```go func (p *MyPlugin) GetTableNameAndAttributes(rows *sql.Rows) (string, []engine.Record) { var tableName, tableType string var totalSize, dataSize sql.NullInt64 if err := rows.Scan(&tableName, &tableType, &totalSize, &dataSize); err != nil { return "", nil } attributes := []engine.Record{{Key: "Type", Value: tableType}} if totalSize.Valid { attributes = append(attributes, engine.Record{Key: "Total Size", Value: fmt.Sprintf("%d", totalSize.Int64)}) } if dataSize.Valid { attributes = append(attributes, engine.Record{Key: "Data Size", Value: fmt.Sprintf("%d", dataSize.Int64)}) } return tableName, attributes } ``` **You SHOULD override these for correctness** (GormPlugin provides defaults): | Method | Default | Override When | |--------|---------|---------------| | `NormalizeType(typeName) string` | Strips length, uppercases | Your DB has type aliases (e.g., `INT4`→`INTEGER`) | | `ConvertStringValue(value, columnType, isNullable) (any, error)` | Generic conversion | Your DB has custom types | | `GetColumnConstraints(config, schema, storageUnit) (map, error)` | Empty map | Your DB has column constraints | | `GetForeignKeyRelationships(config, schema, storageUnit) (map, error)` | Empty map | Your DB has foreign keys | | `GetRowsOrderBy(db, schema, storageUnit) string` | Empty string | Your DB needs default ordering | | `MarkGeneratedColumns(config, schema, storageUnit, columns) error` | No-op | Your DB has auto-increment/computed columns | | `GetColumnCodec(columnType) ColumnCodec` | nil (use default) | Your DB has custom scan/format for columns | | `IsGeometryType(columnType) bool` | false | Your DB has spatial types | | `IsArrayType(columnType) bool` | false | Your DB has array types | | `ShouldCheckRowsAffected() bool` | true | Your DB driver doesn't report affected rows | | `GetMaxBulkInsertParameters() int` | 65535 | Your DB has a lower param limit | | `GetLastInsertID(db) (int64, error)` | 0, nil | Your DB supports returning last insert ID | **The base `GormPlugin` provides these `PluginFunctions` implementations for free**: `GetStorageUnits`, `StorageUnitExists`, `GetAllSchemas`, `GetRows`, `GetRowCount`, `GetColumnsForTable`, `WithTransaction`, `MarkGeneratedColumns`, `GetForeignKeyRelationships` **You MUST implement these `PluginFunctions` methods yourself** (no GormPlugin default): | Method | Purpose | |--------|---------| | `GetDatabases(config) ([]string, error)` | List available databases | | `IsAvailable(ctx, config) bool` | Verify connectivity | | `AddStorageUnit(config, schema, storageUnit, fields) (bool, error)` | CREATE TABLE | | `UpdateStorageUnit(config, schema, storageUnit, values, updatedColumns) (bool, error)` | UPDATE row | | `AddRow(config, schema, storageUnit, values) (bool, error)` | INSERT row | | `AddRowReturningID(config, schema, storageUnit, values) (int64, error)` | INSERT returning ID | | `BulkAddRows(config, schema, storageUnit, rows) (bool, error)` | Bulk INSERT | | `DeleteRow(config, schema, storageUnit, values) (bool, error)` | DELETE row | | `GetGraph(config, schema) ([]GraphUnit, error)` | FK relationship graph | | `RawExecute(config, query, params...) (*GetRowsResult, error)` | Execute raw SQL | | `Chat(config, schema, previousConversation, query) ([]*ChatMessage, error)` | AI chat | | `ExportData(config, schema, storageUnit, writer, selectedRows) error` | Export rows | | `FormatValue(val any) string` | Format a value for display | | `GetSSLStatus(config) (*SSLStatus, error)` | SSL/TLS connection status | | `ClearTableData(config, schema, storageUnit) (bool, error)` | Truncate table | | `NullifyFKColumn(config, schema, storageUnit, column) error` | Nullify FK column | **Connection lifecycle**: Always use `plugins.WithConnection(config, p.DB, func(db *gorm.DB) ...)` for all database operations. This handles connection pooling and lifecycle. **Identifier security**: Every SQL plugin must configure its identifier quote style with `ConfigureIdentifierQuoting`. Do not add database-specific `BuildFullTableName`/`BuildQuotedTableName` variants; shared SQL paths and raw database-specific paths must use `core/src/sqlident`. If the source contract exposes `ImportData`, add a runtime overwrite test with a delimiter-bearing table reference and an unrelated guard table, then classify the connector in `core/src/sourcecatalog/identifier_coverage_test.go` or the EE equivalent. The coverage test must fail when an import-capable connector has no declared proof. **Error handling**: Use `ErrorHandler` from `core/src/plugins/gorm/errors.go` for user-friendly error messages. Call `p.InitPlugin()` in your constructor or first method to initialize it. #### For non-SQL databases: Embed `BasePlugin` For databases that don't use SQL (MongoDB, Redis, Elasticsearch, etc.), embed `engine.BasePlugin` and override only the methods you support. `BasePlugin` (`core/src/engine/base_plugin.go`) provides default implementations for all `PluginFunctions` methods — user-facing operations return `errors.ErrUnsupported`, internal operations return empty results. ```go type MyNoSQLPlugin struct { engine.BasePlugin // Provides defaults for all 27 PluginFunctions methods } // Override only what your source supports func (p *MyNoSQLPlugin) GetDatabases(config *engine.PluginConfig) ([]string, error) { ... } func (p *MyNoSQLPlugin) IsAvailable(ctx context.Context, config *engine.PluginConfig) bool { ... } func (p *MyNoSQLPlugin) GetStorageUnits(config *engine.PluginConfig, schema string) ([]engine.StorageUnit, error) { ... } // ... etc ``` See existing examples: - MongoDB: `core/src/plugins/mongodb/` - Redis: `core/src/plugins/redis/` - Elasticsearch: `core/src/plugins/elasticsearch/` - Memcached: `core/src/plugins/memcached/` - DynamoDB (EE): `ee/core/src/plugins/dynamodb/` - Cassandra (EE): `ee/core/src/plugins/cassandra/` ### Phase 3: Register the Plugin Entry Point Add a blank import so `init()` runs: - **CE**: `core/cmd/whodb/main.go` - **EE**: `ee/cmd/whodb/main.go` (import BOTH CE and EE plugins) ```go _ "github.com/clidey/whodb/core/src/plugins/mynewdb" ``` ### Phase 4: Register in the Database Catalog This controls the connection UI form and which fields are shown. - **CE**: Add to the `catalog` slice in `core/src/dbcatalog/catalog.go` - **EE**: Call `corecatalog.Register()` in `ee/core/src/dbcatalog/register.go` ```go { ID: engine.DatabaseType_MyNewDB, Label: "My New DB", PluginType: engine.DatabaseType_MyNewDB, Extra: map[string]source.ConnectionExtraField{ "Port": {DefaultValue: "5432"}, }, Fields: FieldVisibility{ Hostname: true, Username: true, Password: true, Database: true, }, RequiredFields: FieldRequirements{ Hostname: true, Username: true, Password: true, Database: true, }, SSLModes: sslModesFor(engine.DatabaseType_MyNewDB), // Or omit if no SSL } ``` **`ConnectableDatabase` fields reference:** | Field | Type | Purpose | |-------|------|---------| | `ID` | `engine.DatabaseType` | Unique identifier — becomes the source type ID | | `Label` | `string` | UI display name | | `PluginType` | `engine.DatabaseType` | Which plugin handles this. Can differ from ID for aliases (e.g., Valkey→Redis) | | `Extra` | `map[string]source.ConnectionExtraField` | Advanced field definitions and defaults (Port, Region, etc.) | | `Fields` | `FieldVisibility` | Which standard fields are shown: `Hostname`, `Username`, `Password`, `Database`, `SearchPath` | | `RequiredFields` | `FieldRequirements` | Which fields are mandatory: `Hostname`, `Username`, `Password`, `Database` | | `IsAWSManaged` | `bool` | True for AWS managed services (ElastiCache, DocumentDB) | | `SSLModes` | `[]source.SSLModeInfo` | SSL mode options. Use `sslModesFor()` helper or define custom | `ConnectionExtraField` keeps advanced-field metadata in one place: - `DefaultValue` sets the default UI value. - `Kind` is optional and controls how the field renders (`Boolean`, `Password`, `FilePath`, etc.). - `Required` is optional and marks an advanced field as mandatory in the shared connection form. - `LabelKey` and `PlaceholderKey` are optional overrides for localization keys. **Wire-compatible alias pattern**: If your source is wire-compatible with an existing plugin (e.g., Valkey is Redis-compatible), set `ID` to the new name but `PluginType` to the existing plugin: ```go { ID: engine.DatabaseType_Valkey, // New unique ID Label: "Valkey", PluginType: engine.DatabaseType_Redis, // Reuses Redis plugin Extra: map[string]source.ConnectionExtraField{ "Port": {DefaultValue: "6379"}, }, // ... } ``` ### Phase 5: Register the Source Family Spec This defines browsing hierarchy, capabilities, and actions. The source family spec is the authority for source behavior: the backend adapter enforces it, and the frontend uses it to decide which screens, buttons, and object actions are available. - **CE**: Add to the `familySpecs` map in `core/src/sourcecatalog/catalog.go` - **EE**: Call `coresourcecatalog.RegisterFamilySpec()` in `ee/core/src/sourcecatalog/register.go` ```go "MyNewDB": { Category: source.CategoryDatabase, Traits: networkTraits(source.HostInputModeHostname, source.HostInputURLParserNone), Model: source.ModelRelational, Surfaces: []source.Surface{source.SurfaceBrowser, source.SurfaceQuery, source.SurfaceChat, source.SurfaceGraph}, BrowsePath: []source.ObjectKind{source.ObjectKindDatabase, source.ObjectKindSchema, source.ObjectKindTable}, DefaultObject: source.ObjectKindTable, GraphScopeKind: ptr(source.ObjectKindSchema), ObjectTypes: []source.ObjectType{ metadataObjectType(source.ObjectKindDatabase, "Database", "Databases", true), metadataObjectType(source.ObjectKindSchema, "Schema", "Schemas", true), tabularObjectType(source.ObjectKindTable, "Table", "Tables"), tabularReadOnlyObjectType(source.ObjectKindView, "View", "Views"), }, }, ``` **Choosing your `BrowsePath`** — this controls the object drill-down hierarchy: | Pattern | Example | BrowsePath | |---------|---------|-----------| | Database → Schema → Table | Postgres, MSSQL | `[Database, Schema, Table]` | | Database → Table | MySQL, ClickHouse | `[Database, Table]` | | Schema → Table | Oracle | `[Schema, Table]` | | Table only | SQLite, Athena | `[Table]` (add `RootActions: [Browse, CreateChild]`) | | Database → Collection | MongoDB | `[Database, Collection]` | | Database → Key | Redis | `[Database, Key]` | | Keyspace → Table | Cassandra | Use ObjectKindSchema with label "Keyspace" | | Index only | Elasticsearch | `[Index]` | **`GraphScopeKind`**: Set to the ObjectKind that scopes relationship visualization. Use `ptr(source.ObjectKindSchema)` for schema-scoped, `ptr(source.ObjectKindDatabase)` for database-scoped, or `nil` for no graph support. **Contract enforcement**: Declare every operation in `ObjectTypes.Actions` or `RootActions`. `ListObjects` requires `Browse`, object creation requires `CreateChild` on the root or parent, row reads and exports require `ViewRows`, content reads and downloads require `ViewContent`, column/constraint inspection requires `Inspect`, mutations require their matching data action, imports require `ImportData`, mock data requires `GenerateMockData`, and graph reads require `SurfaceGraph` plus `ViewGraph` on the graph scope object (or root graph support for flat sources). Query and chat surfaces are controlled by `Surfaces`. Plain English: the source contract is the operation policy; WhoDB asks the source what this object can do, then both backend resolvers and frontend controls use that same answer. **Metadata fidelity**: Confirm the family declares appropriate `TypeTraits.Metadata` values. Use exact metadata for system catalogs, driver metadata for bridge-backed families, sampled metadata for document/search sources, inferred metadata for heuristic graph edges, and unsupported for surfaces the source does not expose. Plain English: object metadata behavior is declared once in the source catalog, and the adapter normalizes columns, constraints, graph relationships, and hidden objects from that declaration. Hidden object names and prefixes must reference declared object kinds. ### Phase 6: Register Session Metadata This provides type definitions, operators, and aliases for the query builder UI. Plain English: query/editor behavior is declared once in the source catalog, and the frontend renders editor tools from that declaration instead of checking source names. - **CE**: Add to `registerSessionMetadata()` in `core/src/sourcecatalog/metadata.go` - **EE**: Add to `Register()` in `ee/core/src/sourcecatalog/register.go` ```go RegisterSessionMetadata( string(engine.DatabaseType_MyNewDB), SessionMetadataFromOperatorMap(specs.MyNewDBTypeDefinitions, specs.MyNewDBSupportedOperators, specs.MyNewDBAliasMap), ) ``` **You must define these in the specs package**: - **CE**: `core/src/sourcecatalog/specs/sql.go` (or a new file for non-SQL) - **EE**: `ee/core/src/sourcecatalog/specs/sql.go` ```go // Operators: map of display name → SQL operator var MyNewDBSupportedOperators = map[string]string{ "=": "=", "!=": "!=", ">": ">", "<": "<", ">=": ">=", "<=": "<=", "LIKE": "LIKE", "IN": "IN", "NOT IN": "NOT IN", "IS NULL": "IS NULL", "IS NOT NULL": "IS NOT NULL", "BETWEEN": "BETWEEN", "NOT BETWEEN": "NOT BETWEEN", } // Type aliases: map alias → canonical name var MyNewDBAliasMap = map[string]string{ "INT4": "INTEGER", "BOOL": "BOOLEAN", } // Type definitions for the column type selector UI var MyNewDBTypeDefinitions = []engine.TypeDefinition{ {ID: "INTEGER", Label: "integer", Category: engine.TypeCategoryNumeric}, {ID: "VARCHAR", Label: "varchar", HasLength: true, DefaultLength: engine.IntPtr(255), Category: engine.TypeCategoryText}, {ID: "BOOLEAN", Label: "boolean", Category: engine.TypeCategoryBoolean}, {ID: "TIMESTAMP", Label: "timestamp", Category: engine.TypeCategoryDatetime}, {ID: "JSON", Label: "json", Category: engine.TypeCategoryJSON}, } ``` Every query-capable source must resolve `SourceSessionMetadata`. Operators must be non-empty, type IDs must be unique, and every alias target must match a declared type definition. Sources that expose create-child actions must also resolve supported object creation metadata when the create-object UI needs it. The source catalog tests call `ValidateSessionMetadataContract` and `ValidateObjectCreationMetadataContract` to catch drift across these declarations. They also call `ValidateObjectMetadataContract` so column, constraint, graph, and internal-object fidelity stays aligned with the source contract. ### Phase 7: Frontend Integration Because of the catalog system, the UI is mostly auto-generated. You only need icons and locale strings. #### 7a. Frontend Source Type Discovery The frontend discovers all source types dynamically from the backend via the `SourceTypes` GraphQL query. There is a `DatabaseType` constant in `frontend/src/config/source-types.ts` for convenience references to well-known CE types, but **it is not required for new sources to appear**. The frontend renders any source type returned by the backend catalog. If you want to add a convenience constant (optional): ```typescript export const DatabaseType = { // ... existing types MyNewDb: "MyNewDB", // Must exactly match the backend Go constant string } as const; ``` #### 7b. SVG Icon Registration - **CE**: Add to the `ceLogos` object in `frontend/src/components/icons.tsx` - **EE**: Add to `eeLogos` in `ee/frontend/src/icons.tsx` ```tsx MyNewDB: , ``` The key MUST exactly match your `DatabaseType` string (the `ID` field from the catalog). Icon resolution order: `Icons.Logos[sourceId]` → `Icons.Logos[connector]` → empty span. #### 7c. Localization Strings Add to `frontend/src/locales/common.yaml` (CE) or `ee/frontend/src/locales/` (EE): ```yaml en_US: myNewDBDesc: "Description for cloud provider integration" ``` #### 7d. Custom Connection Form (Rarely Needed) Only needed if your source requires a heavily customized connection form that doesn't fit the standard field-based layout (e.g., DynamoDB's AWS region picker). For normal database sources, do not build a bespoke form first. Prefer the backend-declared `connectionFields` and `sslModes` contract and let the shared frontend connection form render it. CE login and EE platform create/edit now share the same primary-field, advanced-field, and SSL rendering path, so generic field-based sources should work in both places without extra frontend code. In plain English: declare the connection inputs once in the backend source catalog, and both editions render and validate the same form from that declaration. - **CE**: Register via `registerSourceTypeOverrides()` in `frontend/src/config/source-registry.ts` - **EE**: Register in `ee/frontend/src/config.tsx` via `eeSourceTypeOverrides` array, which is passed to `registerSourceTypeOverrides()` in `ee/frontend/src/register.ts` ```typescript const override: SourceTypeOverride = { id: "MyNewDB", customFormRenderer: MyNewDBLoginForm, }; ``` Use `customFormRenderer` only when the standard field-based flow is genuinely insufficient. Connection prefills must target declared `connectionFields` or reserved SSL fields such as `SSL Mode`; `ValidateConnectionContract` tests this for the CE and EE catalogs. The frontend source-contract helpers auto-derive capability flags (`supportsChat`, `supportsGraph`, `supportsScratchpad`, `supportsSchema`, `supportsDatabaseSwitching`, `supportsMockData`, `supportsScripts`, `supportsStreaming`, `supportsMultiStatement`, etc.) from the backend contract and traits. Do not set these manually or add source-name checks for capabilities — surface/object capabilities come from `source.Contract.Surfaces`, `BrowsePath`, `RootActions`, and `ObjectTypes.Actions`; execution capabilities come from `source.TypeTraits.Query`. --- ## Path B: Adding a Source-First Connector Use this path for non-database sources (S3, APIs, queues, graph databases) that don't fit the `PluginFunctions` interface. ### Phase 1: Implement SourceConnector and SourceSession Create your connector package in `core/src/sources//` (CE) or `ee/core/src/sources//` (EE). ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### Phase 2: Register the TypeSpec In the same `init()` or a separate registration file, register the full `TypeSpec`: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### Phase 3: Entry Point Import Add the blank import: ```go _ "github.com/clidey/whodb/core/src/sources/s3" ``` ### Phase 4: Frontend Integration Same as Path A Phase 7 — add icon, locale strings, and optionally a custom form. ### Key Differences from Path A | Aspect | Path A (Database Plugin) | Path B (Source-First) | |--------|--------------------------|----------------------| | DriverID | `"database"` (shared adapter) | Custom (e.g., `"s3"`, `"kafka"`) | | Session impl | `DatabaseSession` adapter wraps `PluginFunctions` | You implement `SourceSession` + interfaces directly | | Catalog | Use `dbcatalog` + `sourcecatalog.FamilySpec` | Register `TypeSpec` directly via `source.RegisterType()` | | Connection fields | Derived from `FieldVisibility` + `Extra` map | Defined explicitly in `TypeSpec.ConnectionFields` | | Object hierarchy | Adapter maps `GetDatabases`/`GetAllSchemas`/`GetStorageUnits` to `BrowsePath` | You control `ListObjects` directly | --- ## Phase: SSL Mode Registration (Both Paths) If your source supports SSL/TLS, define SSL modes. ### Using existing SSL mode helpers In `core/src/dbcatalog/ssl_modes.go`, add your database to `sslModesFor()`, or define modes inline: ```go SSLModes: []source.SSLModeInfo{ {Value: "disable", Label: "Disabled", Description: "No SSL"}, {Value: "require", Label: "Required", Description: "SSL required, no verification"}, {Value: "verify-full", Label: "Verify Full", Description: "SSL with full certificate verification"}, }, ``` ### For Path B sources Include `SSLModes` directly in your `TypeSpec`: ```go source.RegisterType(source.TypeSpec{ // ... SSLModes: []source.SSLModeInfo{ {Value: "disable", Label: "Disabled"}, {Value: "enable", Label: "Enabled"}, }, }) ``` --- ## EE-Only Sources For EE-only data sources, see `ee/.agents/docs/data-sources.md`. It covers EE engine types, catalog registration, source family specs, session metadata, and frontend icons. --- ## Verification & Testing A task is **NOT** complete until all of these pass. ### 1. Type Generation ```bash cd core && go generate . cd frontend && pnpm run generate # If EE: cd ee && go generate . cd ee/frontend && pnpm run generate ``` ### 2. Build Checks ```bash # Backend cd core && go build ./cmd/whodb # CE cd ee && go build ./cmd/whodb # EE # Frontend cd frontend && pnpm run build:ce # CE ``` ### 3. Automated Tests Add tests in your plugin package (`plugin_test.go`). At minimum, test: - Connection parsing - Raw query execution - Schema introspection ```bash bash dev/run-backend-tests.sh all ``` ### 4. Linting Fix all linter errors before considering the job done. Do not suppress warnings. --- ## Registration Pipeline Summary Understanding the full registration flow helps debug issues: ### Path A (Database Plugin): ``` 1. Plugin init() → engine.RegisterPlugin(plugin) [plugin available in engine registry] 2. dbcatalog entry → ConnectableDatabase{ID, PluginType, Fields} [connection form defined] 3. sourcecatalog FamilySpec → familySpecs["MyNewDB"] [capabilities defined] 4. sourcecatalog metadata → RegisterSessionMetadata(...) [query builder metadata] sourcecatalog metadata → RegisterObjectCreationMetadata(...) [create-object metadata] 5. core/src/sources/database/register.go init(): - Iterates dbcatalog.All() - Calls sourcecatalog.BuildTypeSpec(entry) for each - Calls source.RegisterType(spec) for each [TypeSpec in global registry] 6. source/adapters/database_connector.go init(): - Registers DatabaseConnector as driver "database" [driver ready] 7. GraphQL resolver: - getSourceSessionForContext(ctx) - source.Open(spec, credentials) → DatabaseConnector.Open() - Returns DatabaseSession wrapping the engine.Plugin ``` ### Path B (Source-First Connector): ``` 1. Connector init(): - source.RegisterDriver(driverID, connector) [driver registered] - source.RegisterType(TypeSpec{DriverID: driverID, ...}) [TypeSpec in global registry] 2. GraphQL resolver: - getSourceSessionForContext(ctx) - source.Open(spec, credentials) → YourConnector.Open() - Returns your SourceSession implementation ``` --- ## Final Checklist ### For Path A (Database Plugin): - [ ] Engine constant defined (`engine.go` or `ee/core/src/engine/types.go`) - [ ] Plugin package created with `init()` calling `engine.RegisterPlugin()` - [ ] All required `PluginFunctions` / `GormPluginFunctions` methods implemented - [ ] Blank import added to entry point (`main.go`) - [ ] `ConnectableDatabase` entry in dbcatalog (with SSL modes if applicable) - [ ] `FamilySpec` in sourcecatalog - [ ] Source contract declares the correct surfaces, browse path, root actions, object actions, and views - [ ] Session metadata registered (operators, type definitions, alias map) - [ ] Frontend icon added (keyed by exact source ID string) - [ ] Localization strings added - [ ] `go generate`, `pnpm run generate` — both pass - [ ] `go build`, `pnpm run build:ce` — zero errors - [ ] Tests added and passing ### For Path B (Source-First Connector): - [ ] Connector package created with `init()` calling `source.RegisterDriver()` and `source.RegisterType()` - [ ] `SourceSession` implemented + relevant capability interfaces - [ ] Blank import added to entry point (`main.go`) - [ ] `TypeSpec` has correct `DriverID`, `ConnectionFields`, `Contract`, `Traits` - [ ] Frontend icon added - [ ] Localization strings added - [ ] `go generate`, `pnpm run generate` — both pass - [ ] `go build`, `pnpm run build:ce` — zero errors - [ ] Tests added and passing ### For EE sources (additional): See `ee/.agents/docs/data-sources.md` for the full EE checklist. --- ### .Agents Docs Desktop (`.agents/docs/desktop.md`) # Desktop App Development WhoDB has native desktop apps built with [Wails](https://wails.io/). The desktop apps embed the frontend and run the backend locally. ## Directory Structure ``` desktop-ce/ # CE desktop app main.go # Entry point - calls common.RunApp("ce", ...) Makefile # CE-specific config, includes Makefile.common wails.json # Wails config frontend/dist/ # Built frontend (copied during build) desktop-common/ # Shared desktop code app.go # Wails App struct, window management, menus run.go # RunApp() - starts Wails with embedded server Makefile.common # Shared build targets ``` ## Go Workspace Files Desktop builds use separate workspace files to resolve dependencies correctly: - `go.work.desktop-ce` - CE desktop workspace ## Development Commands ```bash # CE Desktop cd desktop-ce && make dev # Development mode with hot reload cd desktop-ce && make build # Build for current platform # Cross-platform builds (from desktop-ce) make build-windows # Windows AMD64 make build-mac # macOS (universal binary) make build-linux # Linux AMD64 make build-all # All platforms ``` ## Build Process 1. `make prepare` - Cleans artifacts, builds frontend via `pnpm run build` 2. Copies `frontend/build/*` to `desktop-*/frontend/dist/` 3. Wails embeds `frontend/dist` via `//go:embed all:frontend/dist/*` 4. Wails builds native app with embedded assets ## Key Environment Variables ```bash ENVIRONMENT=dev # Enables dev mode (set automatically by make dev) WHODB_DESKTOP=true # Set automatically by desktop apps at runtime ``` ## Desktop-Specific Code Patterns Desktop mode is detected at runtime: ```go import "github.com/clidey/whodb/core/src/env" if env.IsDesktopApp() { // Desktop-specific behavior } ``` ## Makefile Variables Each app's Makefile sets these before including `Makefile.common`: ```makefile PKG_ID := com.clidey.whodb.ce # Bundle ID GOWORK := go.work.desktop-ce # Workspace file BUILD_TAGS := # Empty (controlled by entry points, not tags) FRONTEND_BUILD_CMD := build:ce # Frontend build script APP_NAME := WhoDB.app # macOS app name ``` ## Signing & Notarization (macOS) For release builds, set these environment variables: ```bash NOTARY_PROFILE=your-profile APPLE_ID=your@apple.id TEAM_ID=XXXXXXXXXX APPLE_APP_PASSWORD=xxxx-xxxx-xxxx-xxxx ``` Then run: ```bash make build-mac-signed # Build + sign make notarize # Notarize with Apple make build-mac-dmg # Create DMG installer ``` --- ### .Agents Docs Documentation (`.agents/docs/documentation.md`) # Documentation Standards This document defines the documentation requirements for the WhoDB codebase. ## Go Documentation ### Doc Comments for Exported Items Every exported function, type, constant, and variable MUST have a doc comment. The comment starts with the name of the item. **Format:** ```go // FunctionName does X for Y reason. // It returns Z when condition is met. func FunctionName(param Type) ReturnType { ``` **Example - Good:** ```go // Initialize prepares the global PostHog client. It is safe to invoke multiple times. // Only the first call performs actual initialization. func Initialize() { ``` **Example - Bad (missing doc comment):** ```go func Initialize() { // NO - exported function without doc comment ``` ### Comment Quality **Good comments explain WHY:** ```go // Fall back to empty constraints if Migrator fails. // This maintains backward compatibility with older database versions. return make(map[string]map[string]any), nil ``` **Bad comments restate WHAT:** ```go // Build full table name var fullTableName string // NO - obvious from code ``` ### What to Document - Explain non-obvious behavior or edge cases - Document database-specific quirks (e.g., "ClickHouse driver doesn't report affected rows") - Explain security or performance trade-offs - Cross-reference related functions when helpful ### What NOT to Document - Self-explanatory variable names - Standard library function calls - Obvious control flow - Index positions (e.g., "// index 0") - Per-item comments in data lists (e.g., `dbtypes.go` type sets should NOT have comments like "// PostgreSQL canonical" next to each type) ## TypeScript/React Documentation ### JSDoc for Exported Items Every exported function, component, type, and constant MUST have a JSDoc comment. **Format:** ```typescript /** * Brief description of what the function does. * @param paramName - Description of parameter * @returns Description of return value */ export function functionName(param: Type): ReturnType { ``` **Example - Good:** ```typescript /** * Migrates AI-related data from the old database store to the new aiModels store. * This handles the transition from the legacy storage format. */ export function migrateAIModelsFromDatabase(): void { ``` **Example - Bad (missing JSDoc):** ```typescript export function migrateAIModelsFromDatabase(): void { // NO - exported without JSDoc ``` ### React Components ```typescript /** * Displays a table of storage unit data with editing, sorting, and export capabilities. * @param columns - Column definitions for the table * @param rows - Data rows to display * @param onUpdate - Callback when a row is updated */ export const StorageUnitTable: FC = ({ columns, rows, onUpdate }) => { ``` ### Interface/Type Documentation ```typescript /** * Configuration for a database connection profile. */ export interface ConnectionProfile { /** Unique identifier for this profile */ id: string; /** Human-readable name for the connection */ name: string; /** Database host address */ host: string; } ``` ## Inline Comments ### When to Use - Explain WHY something is done a certain way - Document workarounds or hacks with context - Note cross-component dependencies - Flag database-specific behavior **Good:** ```typescript // Add autocomplete for SQL, but allow disabling it during Cypress tests to prevent flakiness. // It is disabled by default in the test environment. ``` **Good:** ```go // TODO: BIG EDGE CASE - ClickHouse driver doesn't report affected rows properly for DELETE ``` ### When NOT to Use - Restating what the code does - Labeling obvious sections - Commenting variable declarations with self-explanatory names **Bad:** ```typescript // SQL validation function const isValidSQLQuery = (text: string): boolean => { // NO - function name is clear ``` **Bad:** ```go // Initialize form inputs m.nameInput = textinput.New() // NO - obvious from code ``` ## Reference Implementation For Go: See `core/src/analytics/posthog.go` - every exported function has a doc comment. For TypeScript: See `frontend/src/utils/database-operators.ts` - proper JSDoc with params and returns. --- ### .Agents Docs Environment Variables (`.agents/docs/environment-variables.md`) # Environment Variables ## Source Connection Profiles Connection profiles are now surfaced through the source-first login flow as `SourceProfiles`, but the environment-variable format is still keyed by source type IDs such as `WHODB_POSTGRES` and `WHODB_MONGODB`. ### Array format A single variable holds a JSON array of connection profiles: ```bash export WHODB_POSTGRES='[ {"alias":"prod","host":"db.example.com","user":"postgres","password":"secret","database":"mydb","port":"5432"}, {"alias":"staging","host":"staging-db.example.com","user":"postgres","password":"secret","database":"mydb","port":"5432"} ]' ``` ### Numbered format Each variable holds a single JSON object: ```bash export WHODB_POSTGRES_1='{"alias":"prod","host":"db.example.com","user":"postgres","password":"secret","database":"mydb","port":"5432"}' export WHODB_POSTGRES_2='{"alias":"staging","host":"staging-db.example.com","user":"postgres","password":"secret","database":"mydb","port":"5432"}' ``` ### Supported prefixes `WHODB_POSTGRES`, `WHODB_MYSQL`, `WHODB_MARIADB`, `WHODB_MONGODB`, `WHODB_REDIS`, `WHODB_CLICKHOUSE`, `WHODB_ELASTICSEARCH`, `WHODB_SQLITE3` ### JSON fields | Field | Type | Description | |---|---|---| | `alias` | string | Display name for the connection | | `host` | string | Hostname or IP address | | `user` | string | Username | | `password` | string | Password | | `database` | string | Database name | | `port` | string | Port number | | `advanced` | object | Key-value map of advanced options (see below) | The `advanced` field replaces the legacy `config` key. Both are accepted for backwards compatibility. ### Parsing logic Defined in `core/src/envconfig/envconfig.go`. The array format (`WHODB_`) is checked first. If empty, numbered variables (`WHODB__1`, `WHODB__2`, ...) are read sequentially until a gap is found. These values are parsed into internal profile records and then exposed to the source-first auth layer as `SourceProfiles`. ## Advanced Connection Options These key-value pairs go in the `advanced` field of a connection profile. At runtime they are carried into the credential record passed to the active source connector during login. ### All source types with host/port connections | Key | Default | Description | |---|---|---| | `Port` | varies per DB | Overrides the default port | ### SQL databases (Postgres, MySQL, MariaDB, ClickHouse) Parsed in `core/src/plugins/gorm/db.go`. | Key | Default | Description | |---|---|---| | `Port` | DB-specific | Connection port | | `Parse Time` | `True` | Parse time values (MySQL/MariaDB) | | `Loc` | `UTC` | Time zone location (MySQL/MariaDB) | | `Allow clear text passwords` | `0` | Allow cleartext auth (MySQL/MariaDB) | | `HTTP Protocol` | `disable` | Use HTTP protocol (ClickHouse) | | `Readonly` | `disable` | Read-only mode (ClickHouse) | | `Debug` | `disable` | Debug mode (ClickHouse) | | `Connection Timeout` | `90` | PostgreSQL, MySQL, MariaDB, ClickHouse. Connection timeout in seconds | ### SSL/TLS options Available for all source types that use the shared SSL/TLS plugin support. Parsed in `core/src/plugins/ssl/ssl.go`. See `ssl.md` for full SSL documentation. | Key | Default | Description | |---|---|---| | `SSL Mode` | `disabled` | SSL mode: `disabled`, `require`, `verify-ca`, `verify-full` | | `SSL CA Content` | `""` | CA certificate content (inline PEM) | | `SSL CA Path` | `""` | Path to CA certificate file | | `SSL Client Cert Content` | `""` | Client certificate content (inline PEM) | | `SSL Client Cert Path` | `""` | Path to client certificate file | | `SSL Client Key Content` | `""` | Client key content (inline PEM) | | `SSL Client Key Path` | `""` | Path to client key file | | `SSL Server Name` | hostname | Server name for certificate verification | For each certificate, use either the `Content` or `Path` variant, not both. The `Path` variants are only available for preconfigured connection profiles (environment variables); the login form uses the `Content` variants. ### MongoDB Parsed in `core/src/plugins/mongodb/db.go`. | Key | Default | Description | |---|---|---| | `Port` | `27017` | Connection port | | `URL Params` | `""` | Additional URL query parameters | | `DNS Enabled` | `false` | Use `mongodb+srv://` scheme | ### Redis Parsed in `core/src/plugins/redis/db.go`. | Key | Default | Description | |---|---|---| | `Port` | `6379` | Connection port | ### Elasticsearch Parsed in `core/src/plugins/elasticsearch/db.go`. | Key | Default | Description | |---|---|---| | `Port` | `9200` | Connection port | ### Example with advanced options ```bash export WHODB_POSTGRES_1='{ "alias": "prod", "host": "db.example.com", "user": "postgres", "password": "secret", "database": "mydb", "port": "5433", "advanced": { "SSL Mode": "verify-ca", "SSL CA Path": "/path/to/ca.pem", "SSL Client Cert Path": "/path/to/client-cert.pem", "SSL Client Key Path": "/path/to/client-key.pem", "Connection Timeout": "30" } }' ``` ## AI Provider Variables | Variable | Default | Description | |---|---|---| | `WHODB_OLLAMA_HOST` | `localhost` (`host.docker.internal` in Docker) | Ollama server hostname | | `WHODB_OLLAMA_PORT` | `11434` | Ollama server port | | `WHODB_OLLAMA_NAME` | unset | Display name for Ollama in the provider dropdown | | `WHODB_OPENAI_API_KEY` | unset | OpenAI API key | | `WHODB_OPENAI_ENDPOINT` | `https://api.openai.com/v1` | OpenAI API endpoint | | `WHODB_OPENAI_NAME` | unset | Display name for OpenAI in the provider dropdown | | `WHODB_ANTHROPIC_API_KEY` | unset | Anthropic API key | | `WHODB_ANTHROPIC_ENDPOINT` | `https://api.anthropic.com/v1` | Anthropic API endpoint | | `WHODB_ANTHROPIC_NAME` | unset | Display name for Anthropic in the provider dropdown | | `WHODB_BLOCK_INTERNAL_AI_ENDPOINTS` | `false` | Set `true` to reject AI endpoints that resolve to loopback, link-local, private, metadata, or standardized IPv6 transition ranges | | `WHODB_AI_ENDPOINT_BLOCKED_CIDRS` | unset | Comma-separated deployment-specific CIDRs that AI endpoints must not reach, including network-specific NAT64 prefixes | ### Generic AI providers Connect any OpenAI-compatible provider. Configured via multiple variables per provider with a unique ``. See `core/src/envconfig/envconfig.go:ParseGenericProviders()`. | Variable | Required | Default | Description | |---|---|---|---| | `WHODB_AI_GENERIC__NAME` | No | `` | Display name in provider dropdown | | `WHODB_AI_GENERIC__TYPE` | No | `openai-generic` | Client type | | `WHODB_AI_GENERIC__BASE_URL` | Yes | | API base URL | | `WHODB_AI_GENERIC__API_KEY` | No | | API key | | `WHODB_AI_GENERIC__MODELS` | Yes | | Comma-separated list of model names | | `WHODB_AI_GENERIC__ICON` | No | | Custom icon URL for the provider dropdown | ## Server Variables | Variable | Default | Description | |---|---|---| | `PORT` | `8080` | TCP port WhoDB listens on | | `WHODB_LOG_LEVEL` | `info` | Log level: `debug`, `info`, `warn`, `error`, `none` | | `WHODB_LOG_FORMAT` | `text` | Log format: `text` or `json` | | `WHODB_LOG_FILE` | unset | Redirect all non-HTTP logs to a file. `default` uses `/var/log/whodb/whodb.log` | | `WHODB_ACCESS_LOG_FILE` | unset | Redirect HTTP access logs to a file. `default` uses `/var/log/whodb/whodb.access.log` | | `WHODB_AUDIT_LOG_FILE` | unset | Community Edition only. Append-only NDJSON log of audited actions such as logins, data reads and changes. Request telemetry is excluded. `default` uses `/var/log/whodb/whodb.audit.log` | | `WHODB_ALLOWED_ORIGINS` | unset | Comma-separated CORS origins (defaults to all) | | `WHODB_BASE_PATH` | unset | URL path prefix for bundled WhoDB web builds, e.g. `/whodb`. Applies only when WhoDB serves embedded frontend assets; split frontend/backend dev mode is unchanged. Must be a slash-prefixed path with segments containing only letters, numbers, `.`, `_`, or `-` | | `WHODB_DISABLE_CREDENTIAL_FORM` | `false` | Set `true` to hide the source credential form on the login page | | `WHODB_MAX_PAGE_SIZE` | `10000` | Maximum number of rows returned per page | | `WHODB_DISABLE_MOCK_DATA_GENERATION` | unset | Disable mock data generation. `*` disables for all tables, or a comma-separated list of table names to disable selectively (e.g., `logs, metrics`) | | `WHODB_HEARTBEAT_DISABLED` | unset | Set `true` to disable the anonymous daily install heartbeat (random install ID plus edition, version, OS — no IP, no usage data). Does not affect the consent-gated product analytics | ### Serving WhoDB under a subpath Use `WHODB_BASE_PATH` when WhoDB serves its bundled frontend and you want the app mounted below `/`, for example behind Nginx at `/whodb/`. ```bash WHODB_BASE_PATH=/whodb ``` This applies to bundled deployments such as the Docker image or a binary built with embedded frontend assets. It does not apply to split dev mode (`pnpm start` for the frontend and `go run` for the backend). When `WHODB_BASE_PATH=/whodb`: - Browse to `/whodb/` - Reverse proxy `/whodb/` to the WhoDB server - Keep container health checks on `/health` ## CLI Cloud Provider Variables These variables configure local CLI cloud discovery. They do not enable cloud-provider operations in the CE web server. | Variable | Default | Description | |---|---|---| | `WHODB_ENABLE_AWS_PROVIDER` | `false` | Set `true` to enable the AWS provider in the CLI | | `WHODB_AWS_PROVIDER` | unset | JSON array of AWS provider configs (see `aws-integration.md`) | --- ### .Agents Docs Hosted Platform Cli (`.agents/docs/hosted-platform-cli.md`) # Hosted WhoDB Platform CLI The hosted platform commands let an existing WhoDB account use the public CLI against a hosted or self-hosted WhoDB platform host. The default host is `https://app.whodb.com`. Use `--host` for local, staging, or self-hosted environments. ```bash whodb login whodb login --host http://localhost:8080 whodb status whodb manifest ``` ## Authentication `login` opens the browser and uses the hosted platform auth flow. The CLI stores the refresh token in the OS keyring and stores only non-secret account metadata in the CLI config. The CLI retains one hosted login per host. Signing in to a different host does not revoke existing sessions. Signing in again to the same host asks before revoking and replacing that host's prior session. ```bash whodb login --host https://app.whodb.com whodb whoami whodb logout ``` If the account has exactly one organization, the CLI selects it automatically. If that selected organization has exactly one project, the CLI selects that project automatically and prints what it selected. If a host is no longer reachable and you only need to remove local credentials: ```bash whodb logout --host http://localhost:8080 --local ``` ## Workspace Selection Hosted source commands need an organization and project. When there is only one possible organization or project, the CLI can select it automatically. Otherwise, select defaults once: ```bash whodb orgs list whodb projects list --org whodb use --org --project ``` Power users can pass `--org` and `--project` directly on `sources` commands. For independent terminal sessions, set a process-local workspace scope. These values are resolved as ids, slugs, or names and do not overwrite the workspace saved by `whodb use`: ```bash # Terminal 1 export WHODB_PLATFORM_SESSION_ORG=acme export WHODB_PLATFORM_SESSION_PROJECT=analysis whodb ontologies list # Terminal 2 export WHODB_PLATFORM_SESSION_ORG=acme export WHODB_PLATFORM_SESSION_PROJECT=operations whodb ontologies list ``` Set `WHODB_PLATFORM_SESSION_HOST` as well when the session should use a non-default host. `WHODB_PLATFORM_SESSION_ORG` and `WHODB_PLATFORM_SESSION_PROJECT` must be set together. `status` shows the current login, workspace selection, manifest, and source management capability state. ```bash whodb status whodb status --format json ``` ## Platform Manifest The hosted platform publishes a small authenticated manifest for CLI compatibility. The manifest tells the CLI which hosted operations and fields are available. It is not a permission system; the backend still enforces access for every request. The CLI caches the manifest in the local config with the platform version. It confirms the version against the server at most once every ten minutes rather than on every command. When the platform version changes, or a GraphQL validation error indicates schema drift, the CLI refreshes the manifest and retries the failed request once. Commands that take `--org` and `--project` use the saved default workspace directly when neither flag is given, without listing organizations and projects first. Naming an organization or project by id, slug or name still resolves it against the server. A default that no longer exists surfaces as the error from the command's own request. ```bash whodb manifest whodb manifest --refresh --format json ``` ## Source Management Discover available source types and required fields: ```bash whodb sources types whodb sources fields Postgres ``` Create a source: ```bash printf "%s\n" "$PGPASSWORD" | whodb sources create Postgres \ --name local-postgres \ --hostname localhost \ --port 5432 \ --username postgres \ --database postgres \ --password-stdin ``` List, inspect, and update sources: ```bash whodb sources list whodb sources get local-postgres whodb sources config local-postgres whodb sources update local-postgres --database analytics ``` Test connections: ```bash whodb sources test local-postgres printf "%s\n" "$PGPASSWORD" | whodb sources test \ --type Postgres \ --hostname localhost \ --port 5432 \ --username postgres \ --database postgres \ --password-stdin ``` Browse source metadata and preview rows: ```bash whodb sources objects local-postgres whodb sources columns local-postgres --ref table:public.users whodb sources rows local-postgres --ref table:public.users --limit 25 ``` Delete is destructive and prompts by default: ```bash whodb sources delete local-postgres whodb sources delete local-postgres --yes ``` ## Platform MCP (default) `whodb mcp serve` exposes hosted platform tools. Sign in to each required host with `whodb login --host `. Start with `whodb_platform_hosts` to discover saved hosts and accounts, then `whodb_platform_orgs` with `workspace: {host}` and `whodb_platform_projects` with `workspace: {host, org}`. When the user wants the whole conversation on one host (for example UAT or production), call `whodb_platform_session_use` once with `workspace: {host, org, project}`. It verifies the login, resolves canonical IDs, and becomes the default for later calls in this MCP session only. It is never saved to disk, so other chats and terminals keep their own defaults. Confirm with `whodb_platform_status`, which reports `session_scope`. Use `whodb_platform_session_clear` to drop it. For one-off calls to another host or project, pass an explicit `workspace: {host, org, project}` on that call. Resolve it first with `whodb_platform_workspace_resolve` if unsure. Per-call targets, including parallel calls, never change the session default or saved defaults. Check the returned `scope`. Resolution order for an omitted `workspace`: session default, then process overrides, then saved defaults. Do not run `whodb use` to switch hosts for one chat; it rewrites the saved default for every session. Read `whodb://platform/schema` for operations and payloads. Writes return a preview bound to its host, account, organization, and project. Confirm only after approval of that exact preview. Changing defaults does not redirect a pending confirmation. Ask for `whodb login --host ` when a host needs authentication; no separate login is needed per project. ## MCP Platform Tools Saved setup settings select modules. Platform is the default; `whodb setup` can enable database-only tools or both modules in the same server: ```bash whodb mcp serve ``` Each MCP process can select its own host and workspace without changing the global default: ```bash whodb mcp serve \ --platform-host http://localhost:8080 \ --platform-org acme \ --platform-project analysis ``` The same `WHODB_PLATFORM_SESSION_HOST`, `WHODB_PLATFORM_SESSION_ORG`, and `WHODB_PLATFORM_SESSION_PROJECT` environment variables work for MCP processes, which is useful for defining several independent MCP server entries. Flags and environment variables pin a default for the whole process, so every chat that launches the same MCP entry gets the same host. To give each chat its own host without separate MCP entries, let the agent call `whodb_platform_session_use` with `workspace {host, org, project}` at the start of the conversation. That default lives in memory for that MCP session only: nothing is written to the CLI config, other sessions are unaffected, and `whodb_platform_session_clear` removes it. `whodb_platform_status` and `whodb_platform_setup_status` report the active `session_scope`. Resolution order when a call omits `workspace`: session default, then process flags/environment, then the saved CLI default. An explicit `workspace` on a call always wins for that call. `--platform-policy` still limits which hosts and workspaces any of these may select. Resolving a workspace needs the host's manifest and the account's organization and project lists. The MCP server caches those per process: the manifest per host for five minutes, the lists per host and signed-in account for one minute. Entries are keyed by account, so hosts, accounts and sessions never share them. When a cached list has no match the server fetches it once more before reporting "not found", and every write drops the account's cached lists, so a workspace created a moment ago resolves immediately. `whodb_platform_orgs` and `whodb_platform_projects` always list live and refresh the cache. The tool's own query is never cached. Every platform request carries an `X-WhoDB-Client` header (`whodb-cli/` from commands, `whodb-cli-mcp/` from the MCP server) so the hosted audit ledger and analytics can tell CLI traffic from browser and hosted-MCP traffic. In database mode, set `WHODB_AUDIT_LOG_FILE` before `whodb mcp serve` to keep a local NDJSON record of each statement the MCP server ran (connection, database type, statement type and row count; never the SQL text). Platform tools use saved host logins. Saving a default workspace is optional when tools pass an explicit `workspace`: ```bash whodb login # Optional default for calls without an explicit workspace whodb use --org --project ``` For single-workspace accounts, `login` or `status` can select the workspace automatically and report what was selected. Platform mode also exposes MCP resources and prompts that agents should read before choosing tools: - `whodb://platform/schema` — exact enabled tool list, resource metadata, prompt metadata, generic write specs, and payload shapes for the current server mode. - `whodb://platform/workspace` — current host, signed-in user, selected organization, selected project, and readiness state. - `whodb://platform/tool-guide` — tool categories, read/write behavior, row limits, field projection guidance, and confirmation behavior. Platform prompts: - `whodb_platform_overview` - `whodb_platform_read_workflow` - `whodb_platform_write_safety` - `whodb_platform_source_workflow` Hosted MCP tool groups: - Workspace and readiness: `whodb_platform_status`, `whodb_platform_orgs`, `whodb_platform_projects` - Sources: `whodb_platform_sources`, `whodb_platform_source_types`, `whodb_platform_source_fields`, `whodb_platform_source_objects`, `whodb_platform_source_columns`, `whodb_platform_source_rows`, `whodb_platform_source_constraints`, `whodb_platform_source_content`, `whodb_platform_source_config`, `whodb_platform_source_test`, `whodb_platform_source_create`, `whodb_platform_source_update`, `whodb_platform_source_delete` - Project resources: `whodb_platform_secrets`, `whodb_platform_ai_providers`, `whodb_platform_ai_provider_models`, `whodb_platform_datasets`, `whodb_platform_dataset`, `whodb_platform_dataset_rows`, `whodb_platform_ontologies`, `whodb_platform_ontology`, `whodb_platform_ontology_fast_lookups`, `whodb_platform_ontology_fast_lookup_suggestions`, `whodb_platform_ontology_rows`, `whodb_platform_ontology_follow_link` - Ontology analytics: `whodb_platform_ontology_describe`, `whodb_platform_ontology_query`, `whodb_platform_ontology_aggregate`, `whodb_platform_ontology_stats`, `whodb_platform_ontology_similar`, `whodb_platform_ontology_embedding_jobs`, `whodb_platform_ontology_record_capabilities`, `whodb_platform_ontology_action_preview`, `whodb_platform_ontology_action_executions`, `whodb_platform_add_ontology_records` - Ontology sharing: `whodb_platform_shared_ontologies`, `whodb_platform_discover_shared_ontologies`, `whodb_platform_ontology_references`, `whodb_platform_ontology_reference_impact` For a cross-project ontology link, create the consumer's ontology reference first. Set `OntologyLinkInput.targetKind` to `reference`, `targetReferenceId` to the returned reference ID, and `targetOntologyApiName` to the owner's ontology `apiName`. The local alias is for resolving the reference in the consumer project; it is not the link target name. Verify traversal with `whodb_platform_ontology_follow_link` after saving. - Service accounts, lineage providers and administration: `whodb_platform_service_accounts`, `whodb_platform_lineage_providers`, `whodb_platform_user_owned_resources`, `whodb_platform_user_resource_access`, `whodb_platform_share_dependents`, `whodb_platform_org_package_installations`, `whodb_platform_app_page_files`, `whodb_platform_transform_preview` - Lineage and transforms: `whodb_platform_project_lineage`, `whodb_platform_lineage`, `whodb_platform_lineage_neighbors`, `whodb_platform_transforms`, `whodb_platform_transform_runs` - Functions and files: `whodb_platform_functions`, `whodb_platform_function`, `whodb_platform_files`, `whodb_platform_file_preview`, `whodb_platform_file_search`, `whodb_platform_tabular_files`, `whodb_platform_storage_usage` - Generic writes and confirmations: `whodb_platform_create`, `whodb_platform_update`, `whodb_platform_delete`, `whodb_platform_action`, `whodb_platform_pending`, `whodb_platform_confirm` Read `whodb://platform/schema` at runtime for the authoritative list. Some write and confirmation tools are hidden in read-only or safe modes. Answering a data question: describe first (`whodb_platform_ontology_describe`), profile with `whodb_platform_ontology_stats`, then use `whodb_platform_ontology_aggregate` for exact totals or `whodb_platform_ontology_query` for filtered rows, search and joins. All of them share one `where` grammar (`{prop: value}`, `{prop: {op: value}}`, `_and`, `_or`). The query is a bounded in-memory scan (`scan_limit` default 1000, max 5000); when the matched total reaches the scan limit the tool adds a warning so the agent switches to the aggregate for exact numbers. Policy-protected ontologies only allow `whodb_platform_ontology_rows`. `whodb_platform_workflow_recipe` returns the same steps for goals that mention counts, totals, reports or questions, and a `bulk_import` recipe built around `whodb_platform_add_ontology_records` with an `idempotency_key`. Writes added for these capabilities go through the generic tools and `whodb resources action`: `function_run cancel`, `transform cancel_run`, `app cancel_generation`, `project restore|purge`, `file|folder restore|purge`, `ontology add_records|share|unshare|reembed|purge_data`, `dataset purge_data`, `ontology_reference create|delete`, `lineage_provider create|update|delete`, `organization create_service_account|create_service_account_key|revoke_service_account_key|delete_service_account|rename_team`, `function generate_test_input`, and `package add_changelog|add_media|review`. Payload shapes for all of them are published in `whodb://platform/schema`. The isolated CLI e2e stack (`ee/dev/run-cli-platform-e2e.sh`) now includes a `function-docker` dind sidecar sharing the backend's `TMPDIR`, so function test, execute, start_run, wait and cancel run for real in e2e. Embeddings remain opt-in through the dev stack's `embeddings` compose profile. Operations that older hosted deployments do not advertise in their platform manifest (project restore and purge, ontology sharing and references, team rename, member ownership and access views) return "this WhoDB host does not support X yet" until the host is upgraded. `whodb_platform_source_config` returns redacted configuration only. Secret-looking values such as passwords, tokens, client secrets, and private keys are masked. Hosted create, update, delete, and action tools do not execute immediately in the default mode. They return a confirmation token, and the write runs only after approval through `whodb_platform_confirm`. Use `whodb_platform_pending` to recover active confirmation tokens. Upload previews must show the stored absolute source path and destination across write plans, confirmations, and pending-action retrieval. Keep this display exception specific to uploads; do not relax secret detection for persisted workflows or send paths to telemetry. Confirmation tokens are available to the model and do not independently prove human approval. File contents and symlink targets are not snapshotted at preview time. See the CLI guide's [upload review and confirmation limitations](../../cli/README.md#upload-review-and-confirmation-limitations) for the user-facing security contract. Generic write tools are capability-backed. Before using `whodb_platform_create`, `whodb_platform_update`, `whodb_platform_delete`, or `whodb_platform_action`, agents should read `whodb://platform/schema` and use the `write_specs` and `payload_shapes` entries instead of guessing GraphQL mutation names or payload fields. For hosted source creation, agents should call `whodb_platform_source_types` and `whodb_platform_source_fields` first instead of guessing source type ids or connection field names. For read tools that accept a `fields` parameter, agents should request only the fields needed for the current answer, then call the tool again with more fields only if needed. Avoid broad details such as source content, file previews, function files, row previews, and large lineage graphs unless the user asks for them or they are required for the task. Agents can discover and pass explicit workspaces without changing defaults, or set a per-session default with `whodb_platform_session_use`. Users can optionally save a default for every session with: ```bash whodb use --org --project ``` With only the platform module enabled, database-only tools are not registered. Enable the database module in setup to expose `whodb_query`, `whodb_connections`, and the other database tools alongside platform tools. Example hosted platform MCP config: ```json { "mcpServers": { "whodb-platform": { "command": "whodb", "args": ["mcp", "serve"] } } } ``` Local smoke test with the MCP inspector: ```bash whodb login --host http://localhost:8080 whodb use --host http://localhost:8080 --org --project npx @modelcontextprotocol/inspector whodb mcp serve ``` In the inspector, call: 1. `whodb_platform_status` 2. Read `whodb://platform/workspace` 3. Read `whodb://platform/schema` 4. Read `whodb://platform/tool-guide` 5. `whodb_platform_orgs` 6. `whodb_platform_projects` 7. `whodb_platform_sources` 8. `whodb_platform_source_types` 9. `whodb_platform_source_fields` 10. `whodb_platform_datasets` with narrow `fields` 11. `whodb_platform_files` with narrow `fields` 12. `whodb_platform_create`, then `whodb_platform_pending`, then `whodb_platform_confirm` 13. `whodb_platform_update`, then `whodb_platform_pending`, then `whodb_platform_confirm` 14. `whodb_platform_delete`, then `whodb_platform_pending`, then `whodb_platform_confirm` ## Automation Output All hosted commands accept `--format json`. Lifecycle and mutating commands use an automation envelope: ```json { "command": "sources.create", "success": true, "data": {} } ``` Read commands emit the requested resource directly: ```bash whodb status --format json whodb manifest --format json whodb sources types --format json whodb sources list --format json ``` Use `--quiet` to suppress informational text in human-readable output. ## Security Model - The CLI acts as the logged-in user. - Source permissions are enforced by the hosted backend. - The manifest only gates compatibility and command UX. - Refresh tokens are stored in the OS keyring. - Source secrets are never written to the CLI config. - `sources config` redacts secret-looking values in human and JSON output. ### MCP process restrictions `whodb mcp serve --platform-policy policy.json` supports `allowed_hosts`, `allowed_workspaces`, `read_only_hosts`, and `read_only_workspaces`. Workspace rules use `{host, org, project}` with canonical IDs. Empty allowlists are unrestricted; populated host and workspace allowlists intersect. Read-only rules also apply to pinned confirmations and override `--allow-write`. Use `workspace` arguments when a policy is configured; top-level `org` and `project` selectors require matching explicit workspace selectors. Inspect `whodb://platform/schema` for the active policy and enabled tools. Platform `--tools` and `--disable-tools` accept full `whodb_platform_*` names; disabling wins. Include confirmation and pending tools when selecting write tools. MCP failures set `isError` and retain the handler's structured recovery details. See `cli/README.md` for policy examples. Restart MCP after editing the policy file. ### Single-entry guided setup Use `whodb setup` for the human wizard. Agents use `whodb setup inspect`, then `whodb setup validate --file ` to preview a settings patch, followed by `whodb setup apply --file --yes` after user approval. `whodb setup verify` checks configured logins and database connections. Never request credentials in chat or include them in setup JSON; use browser login and the CLI credential store. Saved `modules` can contain `platform`, `database`, or both. Policy and tool selection are saved alongside them. All use one MCP entry: `whodb mcp serve`. Restart MCP after settings changes; per-call host/project switching needs no restart. `whodb://mcp/configuration` describes the active modules and restrictions. ### MCP without skills Skills are optional supplements. MCP initialization instructions, tool schemas, and `whodb://mcp/configuration` provide essential guidance directly. Use `whodb_mcp_setup` actions `inspect`, `preview` (with a settings patch), `apply` (with the exact approved preview token and `approved: true`), and `verify`. No shell access is required for these actions. Browser login and credential entry remain user actions. Restart after changing settings; verification checks the active process. The setup tool follows tool filters and can be disabled when configuration should remain outside the agent. --- ### .Agents Docs Localization (`.agents/docs/localization.md`) # Localization Standards This document defines the localization requirements for the WhoDB frontend. ## Overview WhoDB uses a YAML-based localization system with the `useTranslation` hook. All user-facing strings MUST be localized. At build time `frontend/plugins/vite-plugin-locale-packs.mjs` splits the YAML files into one pack per language: `en_US` is bundled with the app (it is the fallback), and every other language is downloaded on demand. ## How It Works 1. Translation files are YAML files in `frontend/src/locales/` (CE) and `ee/frontend/src/locales/` (EE) 2. Components load translations via `useTranslation('component-path')` 3. The `t()` function retrieves strings by key, with automatic pluralization and interpolation 4. Translation merge order: **common → component → extension (EE)**. Every component automatically inherits all keys from `common.yaml`, then its own YAML file overrides/extends, then EE translations override on top 5. The app waits for the saved language's pack before its first render. Change the language only with `changeLanguage()` from `frontend/src/utils/change-language.ts`: it downloads the pack (showing "Switching language…" for at least one loader animation cycle) and then switches, so text never falls back to English mid-switch. `pnpm lint` fails on direct `SettingsActions.setLanguage` calls ## File Structure ``` frontend/src/locales/ # CE translations ├── common.yaml # Shared keys available to ALL components ├── components/ │ ├── table.yaml │ ├── editor.yaml │ └── ... ├── pages/ │ ├── chat.yaml │ ├── login.yaml │ └── ... ├── config/ │ └── graphql-client.yaml └── hooks/ └── use-page-size.yaml ee/frontend/src/locales/ # EE-only translations (overrides CE) ├── components/ │ └── dynamodb-login-form.yaml └── pages/ ├── login.yaml └── sql-agent.yaml ``` ## Supported Languages Languages are defined in `frontend/src/utils/languages.ts`. Each YAML file contains all languages as top-level keys. Agents should add or update `en_US` entries only unless the user explicitly asks for non-English translations. Do not machine-translate other locale entries as part of normal feature work. ## common.yaml — Shared Translations `frontend/src/locales/common.yaml` contains translation keys shared across multiple components. Every component automatically inherits these keys — no extra import or `useTranslation` call needed. **When to put a key in common.yaml:** - The same key+value is needed by 2+ components (e.g., `cancel`, `save`, `close`) - Generic UI terms: action buttons, status labels, error messages - Page/feature names used in both navigation and page headers (e.g., `chat`, `settings`, `graph`) **When NOT to use common.yaml:** - Keys specific to one component (even if the value seems generic like "Title") - Keys where the same English word has different meanings in context (use component-specific keys instead — they may need different translations in other languages) **Component-specific keys override common keys.** If `common.yaml` has `title: "Default"` and `components/export.yaml` has `title: "Export Data"`, the export component gets "Export Data". ## YAML File Format Each file has one top-level key per language, with `en_US` as the source of truth: ```yaml en_US: keyName: English text here anotherKey: Another string parameterized: "Hello {name}, you have {count} messages" fr_FR: keyName: Texte en français ici anotherKey: Une autre chaîne parameterized: "Bonjour {name}, vous avez {count} messages" ``` ## Usage in Components ### Basic Usage ```typescript import { useTranslation } from '@/hooks/use-translation'; export const MyComponent: FC = () => { const { t } = useTranslation('components/my-component'); return ; }; ``` ### Supported `t()` Patterns All interpolation uses `{placeholder}` syntax in YAML. The `t()` function's return type adapts based on the param values. #### Plain string (no interpolation) ```yaml en_US: submit: Submit ``` ```typescript t('submit') // → "Submit" (string) ``` #### String interpolation ```yaml en_US: greeting: "Hello {name}!" itemCount: "{count} items selected" ``` ```typescript t('greeting', { name: userName }) // → "Hello Alice!" (string) t('itemCount', { count: 5 }) // → "5 items selected" (string) ``` #### Multiple placeholders ```yaml en_US: searchMatch: "Match {current} of {total}" ``` ```typescript t('searchMatch', { current: 3, total: 10 }) // → "Match 3 of 10" (string) ``` #### JSX element interpolation When any param value is a React element, `t()` returns `ReactNode` instead of `string`. This allows translators to reorder the element freely within the sentence. ```yaml en_US: details: "Please refer to our {link} for more info." ``` ```typescript t('details', { link: {t('privacyPolicy')} }) // → ReactNode: ["Please refer to our ", , " for more info."] ``` #### Mixed string and JSX params ```yaml en_US: welcome: "Hello {name}, see {link} for details." ``` ```typescript t('welcome', { name: 'Alice', link: the docs }) // → ReactNode (JSX detected, so all params go through the ReactNode path) ``` ### 5. No Sentence Fragments Never split a sentence across multiple translation keys or concatenate translated text with runtime values outside the string. Languages like Japanese, Chinese, and Korean have different word orders, so fragments can't be reordered by translators. **Wrong:** ```yaml en_US: createPrefix: Create a ``` ```typescript // Concatenating outside the translation — translators can't reorder {t('createPrefix')} {storageUnitLabel} ``` **Correct:** ```yaml en_US: createTitle: "Create a {storageUnit}" ``` ```typescript t('createTitle', { storageUnit: storageUnitLabel }) ``` ### JSX Inside Translated Strings When a translated string must contain a React element (link, bold text, etc.), pass the element as a param value. The `t()` function detects JSX automatically and returns `ReactNode` instead of `string`. ```yaml en_US: details: "For more information, see our {link}." ``` ```typescript // t() detects the JSX element and returns ReactNode t('details', { link: {t('privacyPolicy')} }) ``` This works because `t()` has overloaded return types: - `Record` params → returns `string` - `Record` params (any JSX element) → returns `ReactNode` No special function needed — just pass JSX as a param value. ## Critical Rules ### 1. No Fallback Strings The `t()` function does NOT accept a fallback string parameter. This is enforced at the TypeScript level — `t('key', 'fallback')` is a compile error. Keys must exist in YAML. ```typescript t('buttonLabel') // YES - key must exist in YAML ``` ### 2. No Hardcoded UI Text **Wrong:** ```typescript ``` **Correct:** ```typescript ``` ### 3. All User-Facing Strings Must Be Localized This includes: - Button labels - Form placeholders - Error messages - Toast notifications - Tooltips and aria-labels - Dialog titles and content - Table headers - Status messages ### 4. Key Naming Conventions Use camelCase keys that describe the content: ```yaml en_US: submitButton: Submit cancelAction: Cancel loadingMessage: Loading... errorNotFound: Item not found confirmDelete: Are you sure you want to delete this? ``` ### 5. No Duplicate Keys Before adding a new key, check if it already exists in `common.yaml` or in another component's YAML file with the same value. **Wrong:** Adding `cancel: Cancel` to a new component's YAML when `common.yaml` already has it. **Correct:** Just use `t('cancel')` — it comes from the common layer automatically. If the same key+value is needed by 2+ components, it belongs in `common.yaml`, not duplicated in each component file. When adding a new string that could be reusable (action buttons, status messages, generic labels), add it to `common.yaml` directly. ## Pluralization Uses CLDR plural rules via `Intl.PluralRules`. Add suffix keys for plural categories (`_one`, `_other`, `_two`, `_few`, `_many`, `_zero`): ```yaml en_US: rowsSelected: "{count} rows selected" rowsSelected_one: "{count} row selected" deleteRowConfirmTitle: Delete Rows? deleteRowConfirmTitle_one: Delete Row? ``` ```typescript t('rowsSelected', { count: 1 }) // → "1 row selected" (picks _one) t('rowsSelected', { count: 5 }) // → "5 rows selected" (picks base key) ``` The `count` param triggers plural resolution automatically. The base key (without suffix) is used as fallback when no matching plural form exists. ## Adding New Strings 1. **Check common.yaml first.** If the string already exists there (e.g., `cancel`, `save`, `close`, `delete`), just use `t('keyName')` — it's available to all components automatically 2. **Decide where it belongs:** - Generic UI terms used by multiple components → `common.yaml` - Component-specific strings → the component's own YAML file (e.g., `components/table.yaml`) 3. Add the `en_US:` key to the chosen YAML file 4. Use `t('keyName')` in your component 5. Leave other language entries untouched unless the user explicitly requested translation ## What NOT to Localize - Technical identifiers (CSS classes, IDs) - Console log messages (developer-only) - Test fixtures - Code comments ## Common Patterns ### Arrays of Strings **Wrong:** ```typescript const items = ["Option 1", "Option 2", "Option 3"]; ``` **Correct:** ```yaml en_US: option1: Option 1 option2: Option 2 option3: Option 3 ``` ```typescript const items = [t('option1'), t('option2'), t('option3')]; ``` ### Conditional Text ```typescript {isLoading ? t('loading') : t('ready')} ``` ## Translation Tooling Translation tooling is manual maintenance, not part of the automatic agent hook flow. Run it only when explicitly requested. ### Full workflow: detect drift, then translate ```bash cd dev/translate python3 detect.py # step 1: find missing/stale/orphaned keys node translate.mjs # step 2: machine-translate and apply node translate.mjs fr_FR de_DE # step 2: specific languages only ``` ### Drift Detection (`dev/translate/detect.py`) Scans all YAML files (CE and EE) to find: - **Missing** keys: in `en_US` but absent from a translation - **Stale** keys: English text changed since last translation (SHA-256 checksum mismatch) - **Orphaned** keys: in a translation but removed from `en_US` Outputs `drift.json` for `translate.mjs` to consume. Uses `checksums.json` to track what was translated when. ### Machine Translation (`dev/translate/translate.mjs`) Translates missing/stale keys using Google Translate (via `google-translate-api-x`). - Reads `drift.json` produced by `detect.py` - Protects `{placeholder}` tokens during translation - Protects brand/technical terms (`GLOSSARY_TERMS`, e.g. `Redis`, `AlloyDB`, `PostgreSQL`) from mistranslation - Copies literal command values (`NO_TRANSLATE_KEYS`, e.g. `ollamaRunCommand`) verbatim into every locale instead of translating them - Handles `en_GB` via US→UK spelling substitutions - Groups duplicate Google Translate codes to avoid redundant API calls - Removes orphaned keys automatically ## Verifying Translations Before committing, ensure: 1. All `t()` calls have corresponding YAML keys (in the component's file or `common.yaml`) 2. No hardcoded user-facing text remains 3. No duplicate keys — if a key+value exists in `common.yaml`, don't add it to a component file 4. TypeScript compiles cleanly (`tsc --noEmit`) — the `t()` overloads will catch invalid usage ## Reference Files - Languages: `frontend/src/utils/languages.ts` - Hook: `frontend/src/hooks/use-translation.ts` - i18n engine: `frontend/src/utils/i18n.ts` - Shared translations: `frontend/src/locales/common.yaml` - Translation script: `dev/translate/translate.mjs` - Drift detection: `dev/translate/detect.py` --- ### .Agents Docs Plugin Architecture (`.agents/docs/plugin-architecture.md`) # Plugin Architecture Guide The public WhoDB API is now source-first, but the execution layer is still plugin-driven. `core/src/source/` and `core/src/sourcecatalog/` define the public `Source*` contract, while database-specific execution logic still lives in plugins under `core/src/plugins/`. ## Core Principle: No Switch Statements ```go // WRONG - Don't do this: func GetConstraints(sourceType string, ...) { switch sourceType { case "Postgres": // PostgreSQL logic case "MySQL": // MySQL logic } } // CORRECT - Add to PluginFunctions interface: GetColumnConstraints(config *PluginConfig, schema string, storageUnit string) (map[string]map[string]any, error) // Then implement in each plugin: func (p *PostgresPlugin) GetColumnConstraints(...) { /* PostgreSQL-specific */ } func (p *MySQLPlugin) GetColumnConstraints(...) { /* MySQL-specific */ } ``` ## Adding New Functionality If the change is public/source-facing, update the source layer as well as the plugin layer: 1. Add or adapt the execution method on `PluginFunctions` in `core/src/engine/plugin.go` if the capability needs new plugin behavior 2. Update the source adapter in `core/src/source/adapters/` if the public `Source*` API needs to expose that behavior 3. Update `core/src/sourcecatalog/catalog.go` if the source contract, surfaces, or object actions/views change 4. Provide default behavior in shared plugin code (`GormPlugin` in `core/src/plugins/gorm/plugin.go`) where appropriate 5. Override in specific plugins as needed If the change is purely plugin-internal, you may only need steps 1, 4, and 5. ## Alias Databases vs Wrapper Plugins Database catalog aliases are only for cases where the runtime behavior is genuinely identical and only the product metadata changes (label, default port, TLS defaults, managed-service flags, source traits, etc.). Connection defaults such as ports belong in the shared database/source catalog metadata (`dbcatalog` `Extra["Port"]`, which flows into `SourceType.ConnectionFields`), not in per-plugin registries. Connection forms should also flow from `SourceType.ConnectionFields` and `SSLModes`: the shared frontend renderer now handles the standard primary, advanced, and SSL fields for CE login and EE platform create/edit. Discovery prefills should reference declared fields or reserved SSL keys such as `SSL Mode`, and sourcecatalog tests enforce that with `ValidateConnectionContract`. If an alias needs even one runtime override, promote it to a thin first-class plugin wrapper instead of adding alias-specific branches in shared code. Common promotion triggers: - different introspection SQL or system catalog support - different namespace or schema handling - different auth/DSN behavior - different mutation semantics - different graph or metadata queries Examples in the repo: - `CockroachDB` is a PostgreSQL-derived wrapper plugin because it needs catalog query overrides - `MariaDB` and `TiDB` are MySQL-derived wrapper plugin types - `QuestDB` is treated as a PostgreSQL-derived wrapper because it is schema-less in our product model and cannot use the default PostgreSQL table-info query Do not solve alias incompatibilities with `if dbType == ...` checks in shared code. ## Request Context and Cancellation Every request-scoped plugin operation must use the context carried by `*engine.PluginConfig`. Do not use `context.Background()` for query execution, metadata fetches, SDK calls, or health checks that are part of a user request. - GORM-based SQL plugins should inherit cancellation and timeout behavior through `plugins.WithConnection()`, `connection_pool.go`, `connection_cache.go`, and `GormPlugin` - Direct-driver plugins must use `config.OperationContext()` for request-scoped SDK calls - Use `config.OperationContextWithTimeout(...)` when the plugin needs an explicit upper bound for a long-running operation - Reserve `context.Background()` for non-request cleanup work such as cache eviction or best-effort disconnects - If a plugin talks to a driver directly, add a small local helper so future methods inherit the same context behavior instead of repeating it by hand ```go ctx, cancel := config.OperationContextWithTimeout(30 * time.Second) defer cancel() _, err := client.ListTables(ctx, input) ``` ```go func queryWithContext(session *gocql.Session, config *engine.PluginConfig, stmt string, values ...any) *gocql.Query { return session.Query(stmt, values...).WithContext(config.OperationContext()) } ``` ## Plugin File Organization SQL-based plugins follow this structure (see `core/src/plugins/postgres/` as reference): - `db.go` - Connection creation (implements DB method) - `postgres.go` (or `mysql.go`, etc.) - Plugin struct, NewXxxPlugin(), database-specific queries - `types.go` - Type definitions and alias map wrappers used by the plugin runtime - `constraints.go` - Column constraint detection (optional override) Source-owned editor/query metadata now lives in side-effect-free specs under `core/src/sourcecatalog/specs/`. Plugins may read those specs, but they should not register session metadata themselves. GormPlugin base class (`core/src/plugins/gorm/`) provides: - `plugin.go` - 40+ default method implementations - `sqlbuilder.go` - SQL query building - `errors.go` - ErrorHandler for user-friendly error messages - `add.go`, `update.go`, `delete.go` - CRUD operations ## Adding a New Database 1. Create plugin directory in `core/src/plugins/` 2. Implement `PluginFunctions` interface (extend GormPlugin for SQL databases) 3. Add `init()` function calling `engine.RegisterPlugin(NewYourPlugin())` — the plugin self-registers when imported 4. Add a blank import in the entry point (`core/cmd/whodb/main.go`): `_ "github.com/clidey/whodb/core/src/plugins/yourplugin"` ## Key Methods to Override for SQL Plugins ```go // Most SQL plugins override these: GetAllSchemasQuery() string // information_schema query for schemas GetSchemaTableQuery() string // Query for columns in a table FormTableName(schema, table) string // Default: "schema.table" (override for different behavior, e.g. SQLite ignores schema) GetPlaceholder(index int) string // $1 for Postgres, ? for MySQL DB(config) (*gorm.DB, error) // Connection with driver-specific config GetLastInsertID(db *gorm.DB) (int64, error) // Default: returns 0 (override for MySQL, Postgres, SQLite) ``` ## Session Metadata (types.go) Each SQL plugin family must provide session metadata for editor/query-builder UI through the source-owned specs in `core/src/sourcecatalog/specs/`. The shared `core/src/sourcecatalog/metadata.go` file registers those specs centrally. This metadata is the source of truth for: - Valid operators (=, >=, LIKE, etc.) - Type definitions (VARCHAR, INTEGER, etc.) with UI hints (hasLength, hasPrecision) - Alias maps (INT → INTEGER, BOOL → BOOLEAN) Plain English: query/editor behavior is declared once in the source catalog, and the frontend renders editor tools from that declaration instead of checking source names. This metadata is exposed through the source-first GraphQL `SourceSessionMetadata` query after login. **No fallbacks** - if the backend doesn't provide it, the UI type selectors and query helpers will be broken. Every alias target must match a declared type definition, and every query-capable source must resolve non-empty operator metadata. The source catalog tests call `ValidateSessionMetadataContract` and `ValidateObjectCreationMetadataContract` so metadata drift fails in tests. Do not call `sourcecatalog.RegisterSessionMetadata(...)` from plugin `init()` functions anymore. Keep plugin `init()` limited to runtime plugin registration (`engine.RegisterPlugin(...)`). If a plugin needs to reuse the same alias map or type definitions for runtime normalization, import them from `core/src/sourcecatalog/specs/`. Feature gating is not owned by session metadata. Public behavior such as chat/query/graph surfaces and source object actions/views comes from the source catalog contract in `core/src/sourcecatalog/catalog.go`. The source contract is authoritative: `SourceContract.Surfaces`, `RootActions`, `BrowsePath`, and `ObjectTypes.Actions` are used by the backend source adapter to block unsupported operations and by the frontend to show or hide source surfaces and object controls. Do not add source-name conditionals for behavior that can be represented in the contract. Reading or changing source data must be declared with the matching object action: `ViewRows` for row reads and exports, `ViewContent` for content reads and downloads, `CreateChild` for child creation, and `InsertData`, `UpdateData`, `DeleteData`, `ImportData`, or `GenerateMockData` for writes. Plain English: the source contract is the operation policy; WhoDB asks the source what this object can do, then both backend resolvers and frontend controls use that same answer. Frontend and CLI connection/presentation behavior also comes from the source model now. Use `SourceType.Traits` for things like file-vs-network transport, host input parsing, profile labeling, schema fidelity, and query UI options. Execution behavior belongs there too: `TypeTraits.Query` declares whether a source supports scripts, streaming queries, and multi-statement scripts. Plain English: WhoDB asks each source what kinds of execution it supports, then both the backend and frontend use that same answer instead of guessing from the database type or failing at runtime. Do not reintroduce `DatabaseType` branches for those decisions. Source metadata reliability also belongs in `SourceType.Traits`. Use `TypeTraits.Metadata` to declare column, constraint, graph, and internal-object filtering fidelity. Declare system schemas, internal collections, hidden indices, or synthetic keys with `HiddenObjectNames` or `HiddenObjectPrefixes` in the source catalog; the database adapter applies those rules consistently to browse and graph metadata. Plain English: object metadata behavior is declared once in the source catalog, and the adapter normalizes columns, constraints, graph relationships, and hidden objects from that declaration. The source catalog tests call `ValidateObjectMetadataContract`, while the database adapter validates normalized `Column`, `FieldConstraints`, and `GraphUnit` results before returning them. ### types.go Structure ```go package postgres import ( "github.com/clidey/whodb/core/src/common" sourcecatalogspecs "github.com/clidey/whodb/core/src/sourcecatalog/specs" ) // AliasMap maps type aliases to canonical names (UPPERCASE keys and values) var AliasMap = sourcecatalogspecs.PostgresAliasMap // TypeDefinitions - canonical types shown in UI type selector var TypeDefinitions = sourcecatalogspecs.PostgresTypeDefinitions func NormalizeType(typeName string) string { return common.NormalizeTypeWithMap(typeName, AliasMap) } ``` ### Type Validation Column type validation uses `engine.ValidateColumnType()` which checks against TypeDefinitions. Types not in TypeDefinitions will be rejected when adding columns. ## Quirks to Know - SQLite doesn't use schemas - `FormTableName()` returns just table name - PostgreSQL array types display with underscore prefix (`_text`) - MySQL `GetDatabases()` returns `ErrUnsupported` - Redis iterates through 16 database slots to discover databases - Plugin architecture ensures clean code separation --- ### .Agents Docs Sql Security (`.agents/docs/sql-security.md`) # SQL Security Guidelines **CRITICAL**: Preventing SQL injection is non-negotiable. Never use string formatting for SQL queries with user input. ## Parameterized Queries ```go // WRONG - SQL Injection vulnerability: query := fmt.Sprintf("SELECT * FROM %s WHERE id = '%s'", table, userInput) db.Raw(query).Scan(&result) // CORRECT - Use prepared statements: db.Raw("SELECT * FROM users WHERE id = ?", userInput).Scan(&result) // CORRECT - Use GORM builder: db.Table("users").Where("id = ?", userInput).Find(&result) ``` ## Identifier Quoting SQL parameters protect values only. They cannot represent database object names. Every schema, table, and column name that reaches SQL must be rendered as an identifier with `core/src/sqlident`; validation through metadata APIs such as `StorageUnitExists` is not a substitute for quoting at the SQL sink. GORM-backed plugins must configure their database's quote style in the plugin constructor: ```go p.ConfigureIdentifierQuoting(sqlident.DoubleQuote) // PostgreSQL, SQLite, Snowflake p.ConfigureIdentifierQuoting(sqlident.Backtick) // MySQL-family databases p.ConfigureIdentifierQuoting(sqlident.Bracket) // SQL Server ``` Use `sqlident.New` when database-specific code must construct a raw qualified name: ```go name, err := sqlident.New(schemaName, tableName) if err != nil { return false, err } qualifiedTable, err := name.SQL(p.IdentifierQuoteStyle()) if err != nil { return false, err } if err := db.Exec("TRUNCATE TABLE " + qualifiedTable).Error; err != nil { return false, err } ``` `sqlident.New` receives raw components and quotes each component separately. Do not pass a pre-joined `schema.table` string, add connector-specific full-name builders, or concatenate raw identifiers. Use a strict quote style only when the database rejects embedded delimiter characters; strict styles must return the validation error instead of emitting SQL. JDBC bridge dialects declare `IdentifierQuote` with `bridge.QuoteDouble`, `bridge.QuoteBacktick`, or `bridge.QuoteBracket`. The shared bridge plugin then uses `sqlident` for qualified names and applies the same delimiter rules to columns. ## WithConnection Pattern Always use `plugins.WithConnection()` for database operations: ```go _, err := plugins.WithConnection(config, p.DB, func(db *gorm.DB) (bool, error) { rows, err := db.Raw("SELECT * FROM users WHERE status = ?", "active").Rows() // ... process rows return true, nil }) ``` ## GORM Query Patterns - Use `db.Raw()` with placeholders for values in complex queries - Use GORM's query builder for simple operations - Always close rows when done: ```go rows, err := db.Raw("SELECT * FROM table WHERE col = ?", value).Rows() defer rows.Close() ``` --- ### .Agents Docs Ssl (`.agents/docs/ssl.md`) # SSL/TLS Configuration This document covers SSL/TLS support in WhoDB, including architecture, configuration, and how to add SSL support to new database plugins. ## Architecture Overview ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Key Files | File | Purpose | |------|---------| | `core/src/common/ssl/ssl.go` | Mode constants plus source-backed validation and alias normalization | | `core/src/plugins/ssl/tls.go` | `BuildTLSConfig()`, certificate loading | | `core/src/dbcatalog/catalog.go` | CE source-type SSL mode declarations | | `ee/core/src/dbcatalog/register.go` | EE source-type SSL mode declarations | | `core/src/plugins/gorm/db.go` | `parseSSLConfig()` for Gorm-based plugins | | `core/src/engine/plugin.go` | `SSLStatus` type, `GetSSLStatus()` interface | | `frontend/src/graphql/source-types.graphql` | GraphQL query for source-declared SSL modes | | `frontend/src/config/source-types.ts` | Typed frontend source catalog including SSL modes | | `frontend/src/components/ssl-config.tsx` | SSL configuration UI component | | `cli/internal/sourcetypes/sourcetypes.go` | CLI access to source-declared SSL modes | ## SSL Modes ### Unified Mode Names WhoDB uses unified SSL mode names across all databases: | Mode | Constant | Description | |------|----------|-------------| | `disabled` | `SSLModeDisabled` | No SSL/TLS encryption | | `preferred` | `SSLModePreferred` | Use TLS if server supports it (MySQL only) | | `required` | `SSLModeRequired` | Require TLS, skip certificate verification | | `verify-ca` | `SSLModeVerifyCA` | Verify server certificate against CA | | `verify-identity` | `SSLModeVerifyIdentity` | Verify CA + server hostname | | `enabled` | `SSLModeEnabled` | Simple TLS toggle with verification | | `insecure` | `SSLModeInsecure` | TLS enabled, skip all verification | ### Database-Specific Mode Support | Database | Supported Modes | |----------|-----------------| | PostgreSQL | `disabled`, `required`, `verify-ca`, `verify-identity` | | MySQL/MariaDB | `disabled`, `preferred`, `required`, `verify-ca`, `verify-identity` | | ClickHouse | `disabled`, `enabled`, `insecure` | | MongoDB | `disabled`, `enabled`, `insecure` | | Redis | `disabled`, `enabled`, `insecure` | | Elasticsearch | `disabled`, `enabled`, `insecure` | ## Mode Aliasing Different databases use different native terminology. WhoDB normalizes these: ```go // PostgreSQL native → WhoDB unified "disable" → "disabled" "require" → "required" "verify-full" → "verify-identity" // MySQL native → WhoDB unified "DISABLED" → "disabled" "REQUIRED" → "required" "VERIFY_IDENTITY" → "verify-identity" ``` Aliases are declared on the source type itself via `source.SSLModeInfo.Aliases`. ## Certificate Loading Certificates can be provided two ways: 1. **Content** (frontend/API): Certificate PEM content sent directly 2. **Path** (profiles only): Server reads file from path ```go type CertificateInput struct { Content string // PEM content (frontend sends this) Path string // File path (profile-based only, admin-controlled) } ``` **Security**: Path-based loading is restricted to profile connections to prevent path traversal attacks from untrusted frontend input. ## Adding SSL to a New Database Plugin ### Step 1: Declare SSL Modes on the Source Entry Add `SSLModes` to the source entry in `core/src/dbcatalog/catalog.go` or `ee/core/src/dbcatalog/register.go`: ```go SSLModes: []source.SSLModeInfo{ {Value: "disabled", Label: "Disabled", Description: "No SSL/TLS encryption", Aliases: []string{"disable"}}, {Value: "enabled", Label: "Enabled", Description: "Enable TLS with certificate verification"}, {Value: "insecure", Label: "Insecure", Description: "Enable TLS, skip certificate verification"}, } ``` If the source is an alias, prefer inheriting the base source's SSL modes unless the wire protocol really differs. ### Step 2: Parse SSL Config **Option A: Gorm-based plugin** (extends `GormPlugin`) The base class handles everything. Just use `connectionInput.SSLConfig`: ```go func (p *NewDBPlugin) DB(config *engine.PluginConfig) (*gorm.DB, error) { connectionInput, err := p.ParseConnectionConfig(config) if err != nil { return nil, err } // connectionInput.SSLConfig is already parsed if connectionInput.SSLConfig != nil && connectionInput.SSLConfig.IsEnabled() { // Apply SSL... } } ``` **Option B: Non-Gorm plugin** - Create a local `parseSSLConfig`: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ### Step 3: Build and Apply TLS Config In your `DB()` function: ```go func (p *NewDBPlugin) DB(config *engine.PluginConfig) (*gorm.DB, error) { // ... parse connection config ... sslConfig := parseSSLConfig(config.Credentials.Advanced, config.Credentials.Hostname, config.Credentials.IsProfile) if sslConfig != nil && sslConfig.IsEnabled() { tlsConfig, err := ssl.BuildTLSConfig(sslConfig, config.Credentials.Hostname) if err != nil { log.WithError(err).Error("Failed to build TLS configuration") return nil, err } // Apply to your driver (varies by database): // pgx: pgxConfig.TLSConfig = tlsConfig // mysql: mysqldriver.RegisterTLSConfig(name, tlsConfig); cfg.TLSConfig = name // mongodb: clientOptions.SetTLSConfig(tlsConfig) // redis: opts.TLSConfig = tlsConfig // url-based: query.Set("encrypt", "true") } } ``` ### Step 4: Implement GetSSLStatus Create `plugins/newdb/ssl_status.go`: ```go package newdb import ( "github.com/clidey/whodb/core/src/engine" "github.com/clidey/whodb/core/src/log" "github.com/clidey/whodb/core/src/plugins/ssl" ) func (p *NewDBPlugin) GetSSLStatus(config *engine.PluginConfig) (*engine.SSLStatus, error) { log.Debug("[SSL] NewDBPlugin.GetSSLStatus: checking SSL mode") sslConfig := parseSSLConfig(config.Credentials.Advanced, config.Credentials.Hostname, config.Credentials.IsProfile) if sslConfig == nil || !sslConfig.IsEnabled() { return &engine.SSLStatus{ IsEnabled: false, Mode: string(ssl.SSLModeDisabled), }, nil } return &engine.SSLStatus{ IsEnabled: true, Mode: string(sslConfig.Mode), }, nil } ``` **Advanced**: For databases that can report actual SSL status (like PostgreSQL), query the connection: ```go func (p *PostgresPlugin) GetSSLStatus(config *engine.PluginConfig) (*engine.SSLStatus, error) { return plugins.WithConnection(config, p.DB, func(db *gorm.DB) (*engine.SSLStatus, error) { var sslInUse bool err := db.Raw("SELECT ssl FROM pg_stat_ssl WHERE pid = pg_backend_pid()").Scan(&sslInUse).Error // ... }) } ``` ## Testing SSL ### Local Development Use the SSL-enabled containers in `dev/docker-compose.yml`: ```bash cd dev docker-compose --profile ssl up -d e2e_postgres_ssl ``` SSL containers run on different ports: - PostgreSQL SSL: 5433 - MySQL SSL: 3309 - MariaDB SSL: 3310 - MongoDB SSL: 27018 - Redis SSL: 6380 - ClickHouse SSL: 8443/9440 - Elasticsearch SSL: 9201 ### Test Certificates Development certificates are in `dev/certs/`: - `ca//ca.pem` - CA certificate - `server//server-cert.pem` - Server certificate - `server//server-key.pem` - Server private key - `client//client-cert.pem` - Client certificate (for mTLS) - `client//client-key.pem` - Client private key (for mTLS) ### Verify SSL Connection After connecting, check the SSL status indicator in the sidebar or query: ```sql -- PostgreSQL SELECT ssl, version FROM pg_stat_ssl WHERE pid = pg_backend_pid(); -- MySQL/MariaDB SHOW SESSION STATUS LIKE 'Ssl_cipher'; ``` ## Checklist for New Database SSL Support - [ ] SSL modes declared on the source/catalog entry (`dbcatalog` or EE register) - [ ] Aliases declared on `source.SSLModeInfo.Aliases` when the wire protocol has native names - [ ] `parseSSLConfig()` implemented or using GormPlugin base - [ ] `ssl.BuildTLSConfig()` called in DB connection - [ ] TLS config applied to database driver correctly - [ ] `GetSSLStatus()` implemented in `ssl_status.go` - [ ] Logging added for SSL operations - [ ] Backend builds: `cd core && go build .` - [ ] Frontend type checks: `cd frontend && ./node_modules/.bin/tsc --noEmit` - [ ] Extension builds pass ## Frontend Integration The `SSLConfig` component (`frontend/src/components/ssl-config.tsx`) automatically: - Receives SSL modes via props from the source catalog (`SourceTypes`) - Handles mode aliasing (shows correct selection for aliased values) - Shows/hides certificate inputs based on mode requirements - Supports file picker and paste modes for certificates ### Source-Owned Contract SSL modes are no longer duplicated in a frontend-only registry. The source catalog is the single authority: - backend validation reads source-declared SSL modes through `NormalizeSSLMode`, `ValidateSSLMode`, and `ParseSSLConfig` - frontend reads the same modes from `SourceTypes` - CLI reads the same modes through `cli/internal/sourcetypes` --- ### .Agents Docs Testing (`.agents/docs/testing.md`) # Testing Guide This document covers all testing infrastructure for WhoDB: frontend Playwright E2E tests, Go backend tests, and CLI tests. ## Quick Reference ```bash # Frontend Playwright E2E (CE) cd frontend && pnpm e2e:ce:headless # All CE databases, headless cd frontend && pnpm e2e:ce # Interactive mode (headed) # Backend Go tests bash dev/run-backend-tests.sh all # CE + EE unit and integration bash dev/run-backend-tests.sh unit # CE + EE unit tests only bash dev/run-backend-tests.sh integration # CE + EE integration tests only bash dev/run-backend-tests.sh ce-integration # CE integration tests only bash dev/run-backend-tests.sh ee-integration # EE integration tests only # CLI tests bash dev/run-cli-tests.sh # All CLI tests bash dev/run-cli-tests.sh --skip-postgres # Skip PostgreSQL E2E ``` --- ## Frontend Playwright E2E Tests ### Architecture ``` frontend/e2e/ ├── tests/ │ ├── features/ # 29 feature test files (.spec.mjs) │ └── postgres-screenshots.spec.mjs # Screenshot generation (run separately) ├── fixtures/databases/ # Database configuration JSON files ├── support/ │ ├── test-fixture.mjs # Test fixture (whodb helper, forEachDatabase, coverage) │ ├── whodb.mjs # WhoDB helper class (all page commands) │ ├── database-config.mjs # Database config loader │ ├── global-setup.mjs # Global setup (health checks) │ ├── helpers/ │ │ ├── animation.mjs # Browser state helpers │ │ └── fixture-validator.mjs # Feature validation │ └── categories/ # Database category helpers │ ├── sql.mjs # SQL database helpers │ ├── document.mjs # MongoDB/Elasticsearch helpers │ ├── keyvalue.mjs # Redis helpers │ └── index.mjs ├── reports/ │ ├── blobs/ # Blob reports per database (merged after all runs) │ ├── html/ # Merged HTML report (all databases combined) │ └── test-results/ # Test artifacts (screenshots, traces, videos) └── logs/ # Per-database execution logs ``` ### Test Categories Tests are organized by database category: | Category | Databases | |----------|-----------| | `sql` | PostgreSQL, MySQL, MySQL8, MariaDB, SQLite, ClickHouse, CockroachDB, DuckDB, TiDB, YugabyteDB, QuestDB | | `document` | MongoDB, Elasticsearch, OpenSearch, FerretDB | | `keyvalue` | Redis, Valkey, Dragonfly, Memcached | ### Database Fixtures Located in `frontend/e2e/fixtures/databases/`: - `postgres.json`, `mysql.json`, `mysql8.json`, `mariadb.json`, `sqlite.json` - `mongodb.json`, `elasticsearch.json`, `clickhouse.json`, `redis.json` - `cockroachdb.json`, `duckdb.json`, `tidb.json`, `yugabytedb.json`, `questdb.json` - `opensearch.json`, `ferretdb.json`, `valkey.json`, `dragonfly.json`, `memcached.json` Each fixture defines: connection details, expected schemas, test data, and feature flags. ### NPM Scripts #### E2E Tests ```bash # Interactive mode (opens browser UI) pnpm e2e:ce # All CE databases pnpm e2e:db # Single database (e.g., postgres, mysql) pnpm e2e:feature # Single feature file # Headless mode (CI/automated) pnpm e2e:ce:headless # All CE databases pnpm e2e:db:headless # Single database pnpm e2e:feature:headless # Single feature # Debug mode pnpm e2e:ce:headless:debug # With Playwright debug logging ``` ### Feature Test Files Located in `frontend/e2e/tests/features/`: | File | Tests | |------|-------| | `accessibility.spec.mjs` | Accessibility compliance | | `chat.spec.mjs` | AI chat functionality | | `cloud-providers.spec.mjs` | Cloud provider integration | | `crud.spec.mjs` | Create, Read, Update, Delete operations | | `data-types.spec.mjs` | Data type handling | | `data-view.spec.mjs` | Data view UI | | `error-handling.spec.mjs` | Error scenarios | | `explore.spec.mjs` | Database exploration | | `export.spec.mjs` | Data export functionality | | `graph.spec.mjs` | Entity relationship graphs | | `health-overlays.spec.mjs` | Health check overlays | | `import.spec.mjs` | Data import functionality | | `key-types.spec.mjs` | Redis key types | | `keyboard-shortcuts.spec.mjs` | Keyboard navigation | | `loading-states.spec.mjs` | Loading state display | | `login.spec.mjs` | Authentication flow | | `mock-data.spec.mjs` | Mock data generation | | `pagination.spec.mjs` | Pagination controls | | `profiles.spec.mjs` | User profiles | | `query-history.spec.mjs` | Query history tracking | | `schema-management.spec.mjs` | Schema operations | | `scratchpad.spec.mjs` | SQL/query editor | | `search.spec.mjs` | Search functionality | | `settings.spec.mjs` | User settings | | `sidebar.spec.mjs` | Sidebar navigation | | `sorting.spec.mjs` | Column sorting | | `ssl-config.spec.mjs` | SSL configuration | | `ssl-modes.spec.mjs` | SSL modes | | `storage.spec.mjs` | Browser storage/persistence | | `tables-list.spec.mjs` | Table listing | | `tour.spec.mjs` | User onboarding tour | | `type-casting.spec.mjs` | Data type conversion | | `where-conditions.spec.mjs` | SQL WHERE clause building | ### Screenshot Tests `postgres-screenshots.spec.mjs` is excluded from the normal test suite (`testIgnore` in config). Run it separately: ```bash cd frontend && DATABASE=postgres pnpm exec playwright test \ --config=e2e/playwright.config.mjs \ --project=standalone \ tests/postgres-screenshots.spec.mjs ``` ### GraphQL Authorization Regression The CE live-server regression is `frontend/e2e/tests/features/graphql-authorization.ce.spec.mjs`. It sends an anonymous `UpdateSettings` mutation under the public-looking operation name `SettingsConfig` and verifies the setting is unchanged. ```bash cd frontend VITE_E2E_TEST=true bash ../dev/run-e2e.sh true sqlite graphql-authorization.ce ``` EE platform coverage lives in `ee/frontend/e2e/tests/platform/graphql-authorization.spec.mjs` and verifies that an operation mixing public `Health` with a protected platform field is rejected as a whole. Run it through the platform E2E stack described in `ee/.agents/docs/ee-testing.md`. ### Playwright Configuration Key settings in `frontend/e2e/playwright.config.mjs`: - **Viewport**: 1920×1080 - **Base URL**: `http://localhost:3000` - **Workers**: 1 (sequential — databases share backend state) - **Retries**: 1 locally, 2 in CI - **Reporter**: Blob (merged into HTML after all database runs) - **Traces/Videos**: Retained on failure ### Test Execution Model The run script (`dev/run-e2e.sh`) executes databases **sequentially**: 1. Each database gets its own `playwright test` invocation with `DATABASE=` 2. Each invocation writes a blob report to `e2e/reports/blobs/` 3. After all databases finish, blobs are merged into a single HTML report 4. Output is piped through `tee` to both terminal and `e2e/logs/.log` ### Coverage Frontend code coverage is collected via Istanbul instrumentation: 1. `vite-plugin-istanbul` instruments source code when `NODE_ENV=test` 2. The test fixture collects `window.__coverage__` after each test 3. Coverage data is written to `.nyc_output/coverage-.json` 4. `nyc report` merges and generates reports ```bash cd frontend && pnpm view:coverage # Text summary cd frontend && pnpm view:coverage:frontend # HTML report cd frontend && pnpm coverage:clean # Clear coverage data ``` --- ## Docker Test Infrastructure ### Docker Compose Location: `dev/docker-compose.yml` #### CE Database Services | Service | Port | Description | |---------|------|-------------| | `e2e_postgres` | 5432 | PostgreSQL | | `e2e_mysql` | 3306 | MySQL | | `e2e_mysql_842` | 3308 | MySQL 8.4.2 | | `e2e_mariadb` | 3307 | MariaDB | | `e2e_mongo` | 27017 | MongoDB | | `e2e_redis` | 6379 | Redis | | `e2e_elasticsearch` | 9200 | Elasticsearch | | `e2e_clickhouse` | 8123 | ClickHouse (server limits from `dev/conf/clickhouse_e2e.xml`; the default RAM-ratio cap starved CI runs) | SQLite uses a local file (`core/tmp/e2e_test.db`), no container needed. ### Sample Data Scripts Located in `dev/sample-data/`: - `postgres/data.sql` - PostgreSQL initialization - `mysql/data.sql` - MySQL initialization - `mongo/data.js` - MongoDB initialization - `elasticsearch/upload.sh` - Elasticsearch data upload - `redis/init.sh` - Redis initialization ### E2E Orchestration Scripts #### Main Runner: `dev/run-e2e.sh` ```bash ./dev/run-e2e.sh [headless] [database] [spec] # Examples: ./dev/run-e2e.sh # Interactive, all databases ./dev/run-e2e.sh true # Headless, all databases ./dev/run-e2e.sh true postgres # Headless, PostgreSQL only ./dev/run-e2e.sh true postgres crud # Specific spec file ``` #### Environment Variables | Variable | Description | |----------|-------------| | `WHODB_DATABASES` | Space-separated list of databases to test | | `WHODB_DB_CATEGORIES` | Colon-separated db:category pairs | | `WHODB_VITE_EDITION | Build edition | | `WHODB_SETUP_MODE | Setup mode | | `WHODB_LOG_LEVEL` | Log level (default: 'error') | | `DATABASE` | Target database (set by run script per invocation) | | `CATEGORY` | Target category (set by run script per invocation) | #### Setup/Cleanup Scripts ```bash # Setup E2E environment bash dev/setup-e2e.sh # Cleanup after tests bash dev/cleanup-e2e.sh ``` Features: - Smart test binary caching (detects source changes) - Parallel port waiting for services - Docker volume pruning on cleanup --- ## Go Backend Tests ### Architecture ``` core/ ├── server_test.go # Server startup tests ├── graph/ │ ├── graphql_queries_test.go # GraphQL query tests │ ├── resolver_test.go # Resolver tests │ ├── graphql_integration_test.go # Integration tests │ ├── resolver_mutation_test.go # Mutation tests │ ├── http_resolvers_test.go # HTTP resolver tests │ ├── import_helpers_test.go # Import helper tests │ └── mockdata_resolver_test.go # Mock data tests ├── src/ │ ├── settings/*_test.go # Settings tests │ ├── mockdata/generator_test.go # Generator tests │ ├── llm/llm_test.go # LLM tests │ ├── auth/*_test.go # Auth tests │ └── plugins/ │ ├── common_test.go # Common plugin tests │ └── gorm/constraint_helpers_test.go └── test/integration/ ├── main_test.go # Integration setup ├── roundtrip_test.go # Basic roundtrip ├── full_roundtrip_test.go # Full roundtrip └── ssl_test.go # SSL integration tests ``` ### Running Backend Tests ```bash # Main test runner bash dev/run-backend-tests.sh all # CE + EE unit and integration (default) bash dev/run-backend-tests.sh unit # CE + EE unit tests only bash dev/run-backend-tests.sh integration # CE + EE integration tests only bash dev/run-backend-tests.sh ce-integration # CE integration tests only bash dev/run-backend-tests.sh ee-integration # EE integration tests only # Direct Go commands cd core && go test ./... # All unit tests cd core && go test ./graph/... # GraphQL tests only cd core && go test ./src/auth/... # Auth tests only ``` ### Integration Tests Integration tests require Docker services running: ```bash # Start CE services manually docker compose -f dev/docker-compose.yml up -d # Start EE services manually docker compose -f ee/dev/docker-compose.yml --profile ee up -d # Run integration tests directly cd core && go test ./test/integration/... cd ee && go test ./test/integration/... # Or use the script (manages Docker automatically) bash dev/run-backend-tests.sh integration ``` ### Preserving `data-testid` Attributes During Refactoring E2E tests locate elements via `data-testid` selectors. When extracting inline JSX into shared components, **always carry over `data-testid` attributes** to the new component's rendered elements. Dropping them silently breaks every E2E test that depends on those selectors. Key patterns to preserve: - Login form advanced fields use `data-testid={`${field.Key}-input`}` (e.g., `Port-input`, `Search Path-input`) - Login form standard fields: `hostname`, `username`, `password`, `database` - Buttons: `login-button`, `advanced-button`, `database-type-select` - SSL config: `ssl-mode-select`, `ssl-ca-certificate-content` When refactoring UI code, search for `data-testid` in the original code and ensure every attribute is present in the replacement. Run `grep -r "data-testid" frontend/e2e/` to see what selectors the tests actually use. ### Debug Mode ```bash # Playwright debug logging pnpm e2e:ce:headless:debug # Go test verbose go test -v ./... # CLI test verbose bash dev/run-cli-tests.sh -v ``` --- ### .Agents Docs Verification (`.agents/docs/verification.md`) # Verification Checklist Before marking any task as complete, run through this verification checklist. ## Frontend (TypeScript/React) ### Type Checking ```bash cd frontend && pnpm typecheck ``` ### Lint ```bash cd frontend && pnpm lint ``` ### Build Verification ```bash cd frontend && pnpm run build:ce ``` ### Dead Code Check After adding new code, verify it's actually used: - Search for function/component names to confirm they're imported and called - Check that new exports are imported somewhere - Remove any unused imports, variables, or functions ## Backend (Go) ### Build Verification ```bash cd core && go build ./cmd/whodb ``` ### Lint ```bash cd core && ./lint.sh ``` ### Dead Code Check - Verify exported functions are called from somewhere - Check that new types are actually used - Remove unused imports (Go compiler will catch these) ## Common Issues to Catch 1. **Unused imports** - Both Go and TypeScript will flag these 2. **Unused variables** - Especially after refactoring 3. **Orphaned helper functions** - Functions added but never called 4. **Stale utility code** - Code added for a purpose that changed 5. **Commented-out code** - Remove instead of leaving commented ## Quick Verification Commands ```bash # Frontend full check cd frontend && pnpm lint && pnpm typecheck && pnpm run build:ce # Backend full check cd core && ./lint.sh && go build ./cmd/whodb # Search for unused exports (manual) # Use grep to search for function/type names and verify usage ``` --- ### Cli Docs Platform Operations (`cli/docs/platform-operations.md`) # Focused MCP and platform operations ## Task catalogs ```sh whodb mcp serve --tool-profile data whodb mcp serve --tool-profile functions whodb mcp serve --tool-profile workspace ``` The default `all` catalog is unchanged. Profiles narrow discovery; permissions, write confirmation, read-only mode and explicit disabled tools still apply. `--tools` intersects the profile. Use `all` for advanced operations. Persist a profile through `whodb setup` or `whodb_mcp_setup` with `{"tool_profile":"functions"}`. Setup's preview/approval and restart rules apply. The `whodb://mcp/configuration` resource lists the active and available profiles. `whodb_platform_runtime_readiness` now reports observed configuration separately from runtime reachability and verification. Metadata alone leaves runtime states `unknown`. Supply `function_id` to inspect the latest persisted run; its completion timestamp is historical evidence, not a probe of current capacity. Unavailable inventories produce warnings and unknown state rather than a zero-resource claim. Workspace maps include apps and package installations. ## Unattended CLI jobs Supply a service-account API key through your secret runner's environment: ```sh export WHODB_PLATFORM_SESSION_HOST=https://your-whodb.example export WHODB_PLATFORM_SESSION_ORG="organization-id" export WHODB_PLATFORM_SESSION_PROJECT="project-id" # Inject WHODB_PLATFORM_API_KEY from your job's secret store. whodb sources list --format json --quiet ``` Use canonical workspace IDs. The key is bound to the explicit session host; a conflicting `--host` is rejected. HTTP redirects are not followed. An empty, invalid, expired or revoked key never falls back to a personal login or refresh token. The key is not written to configuration or the keyring. Backend grants still govern reads and writes. This credential path is for CLI platform commands; hosted MCP continues to use its existing OAuth flow. ## Security ledger Organization managers can use: ```sh whodb security-audit --limit 50 --request-id whodb security-audit --run-id whodb security-audit --outcome DENIED whodb security-audit --cursor whodb security-audit --mode exports whodb security-audit --mode verify ``` Output is JSON. Optional event filters include project, actor, action, outcome, and RFC3339 `--since`/`--until`. MCP exposes the same operations through `whodb_platform_security_audit`. This is the platform security ledger; the database `audit` command remains a data-quality inspection. Exports report organization-scoped receipt backlog and oldest pending event. Exporter attempt timestamps and failure stage describe the responding process; missing observations are unknown. Verification checks retained chained rows against the stored tip and reports excluded unchained events. It is not an independent external integrity anchor. Disabled auditing is reported explicitly. --- ### Advanced Batch Operations (`docs/advanced/batch-operations.mdx`) --- title: "Batch Operations" description: "Use current table grid selection and batch actions" --- # Batch Operations The table grid supports row selection and selection-based actions for sources that expose those capabilities. ## Select Rows Use any of these current selection methods: - Click a row checkbox. - Use the header checkbox to select or deselect the visible page. - Focus a row and press `Space`. - Use `Shift + Arrow Up/Down` to extend selection. - Use `Cmd/Ctrl + A` to select all rows in the current view. - Use the row context menu action **Select Row**. Use the header checkbox, row checkboxes, shortcuts, and context menus to select or clear rows. ## Export Selected Rows After selecting rows, use the selected-row export action. The export starts from the selected rows you chose in the grid. ## Edit And Delete If the connected source allows writes: - `Enter` or `Cmd/Ctrl + E` opens edit for the focused row. - `Cmd/Ctrl + Delete` or `Cmd/Ctrl + Backspace` starts delete for the focused row. - The row context menu also exposes **Edit Row** and **Delete Row** when supported. Delete actions show a confirmation dialog before the mutation is submitted. ## Context Menus Right-click a grid row to open the row context menu. Header context menus expose table-level actions such as refresh, export, select/deselect all visible rows, and mock data when supported. ## Related Pages Modify rows from the grid. Delete rows with confirmation. --- ### Advanced Export Options (`docs/advanced/export-options.mdx`) --- title: "Export Options" description: "Export table data and query results from WhoDB as CSV or Excel, with delimiter options and full-table or selected-row exports" --- # Export Options WhoDB exports data from the table grid and Scratchpad result grids. The export mode is determined by where you start the export, not by a mode selector inside the dialog. ## Entry Points Current export entry points include: - **Export All** from the table footer/action area. - **Export Selected** after selecting rows in the grid. - Header or row context-menu export actions. - Export from Scratchpad result grids. ## Formats CE currently supports: - **CSV** for tabular rows. - **Excel `.xlsx`** for tabular rows. - **JSON Lines (NDJSON)** for records from sources like MongoDB when supported. For SQL backups or migration dumps, use your database's native tools such as `pg_dump`, `mysqldump`, or `mongodump`. ## CSV Delimiters CSV export supports delimiter choices: - Comma - Semicolon - Pipe ## Excel Export Excel export creates an `.xlsx` file from tabular data. Choose Excel to download an `.xlsx` file from the current grid or result set. ## Filtering Before Export To export filtered data: 1. Add WHERE conditions or adjust the grid query. 2. Click **Query** so the grid reloads. 3. Use **Export All** from the filtered grid. Use the grid and the downloaded file to verify row counts, selected columns, and file size. ## Selected Rows To export selected rows: 1. Select one or more rows with the row checkboxes or table selection shortcuts. 2. Use the export action for selected rows. 3. Choose a supported format. ## Related Pages Open the grid that provides export actions. Select rows and apply grid-level actions. --- ### Advanced Mock Data (`docs/advanced/mock-data.mdx`) --- title: "Mock Data" description: "Generate mock rows from the current table grid actions" --- # Mock Data Mock data generation is available from the table grid when the connected source, object, and deployment configuration allow it. ## Open Mock Data Use one of the current entry points: - Right-click the table header and choose the mock data action. - Use the row/header context menu where the mock data action appears. - Press `Cmd/Ctrl + Shift + G` while the table has focus. Use the grid context menu for mock data actions. ## Configure Generation The mock data sheet lets you choose: - Row count. - Generation method. - Data handling mode. - Foreign-key variety when the source supports it. ## Append Or Overwrite Use **Append** to add generated rows to existing data. Use **Overwrite** only when you intentionally want generated data to replace existing rows. WhoDB shows an overwrite confirmation before proceeding. ## Row Counts The row-count field accepts a positive number up to the current backend limit of 200 rows per generation. ## Disable Mock Data Administrators can disable mock data with the `WHODB_DISABLE_MOCK_DATA_GENERATION` environment variable. It has three modes: - Unset or empty: mock data is allowed for all objects. - `*`: mock data is disabled everywhere. - A comma-separated list of object names: mock data is disabled only for those objects. To disable globally: ```bash WHODB_DISABLE_MOCK_DATA_GENERATION="*" ``` or a comma-separated object list: ```bash WHODB_DISABLE_MOCK_DATA_GENERATION=users,payments ``` ## Related Pages Open the table grid that exposes mock data actions. Use row selection and context-menu actions. --- ### Advanced Where Conditions (`docs/advanced/where-conditions.mdx`) --- title: "WHERE Conditions" description: "Filter Storage Unit data with the current WHERE condition builder" --- # WHERE Conditions WHERE conditions add server-side filters to the Storage Unit data page. They are separate from the Search field, which only searches currently loaded grid rows. ## Add A Condition Open a table or collection from Storage Units. Click **Add** beside the **Where condition** label. Select a field from the field dropdown. Pick an operator and enter the value. Click **Add Condition** or the add control in the condition UI. Click the page-level **Query** button to reload the grid with the condition applied. ## Operators The operator dropdown lists the database's own operator tokens, provided by the backend for the connected source. For a typical SQL database such as PostgreSQL, the set includes: - `=`, `!=`, `<>` - `>`, `>=`, `<`, `<=` - `BETWEEN`, `NOT BETWEEN` - `LIKE`, `NOT LIKE`, `ILIKE`, `NOT ILIKE` - `IN`, `NOT IN` - `IS NULL`, `IS NOT NULL` The exact list varies by database. ## Multiple Conditions Add multiple conditions when you need a narrower result set. Conditions are combined by the current builder logic and applied when you click **Query**. ## Edit Or Remove Conditions Use the condition controls in the WHERE condition UI to edit or remove an existing condition, then click **Query** again to refresh the grid. ## Search Versus WHERE | Feature | Behavior | |---------|----------| | Search field | Searches/highlights values in the currently loaded rows | | WHERE condition | Sends a server-side filter and changes the result set after **Query** | Use WHERE conditions for real filtering. Use Search to quickly find a value in the rows already visible. ## Related Pages Compare quick search and server-side filters. Filter before exporting. --- ### Ai Conversation Features (`docs/ai/conversation-features.mdx`) --- title: "Conversation Management & Advanced Features" description: "Master multi-turn conversations, chat history navigation, scratchpad integration, and provider management in WhoDB's AI assistant" --- # Conversation Management & Advanced Features WhoDB's AI Chat Assistant is designed for interactive, contextual conversations that build on previous exchanges. This guide covers advanced features that help you work more efficiently with the AI, including conversation history, chat navigation, scratchpad integration, and provider management. The AI assistant maintains full context throughout your conversation, allowing natural follow-up questions without repeating information Conversation features require a selected AI model. If the model dropdown is empty, configure a provider before testing chat history, generated SQL, or Move to Scratchpad. ## Understanding Conversational Context Unlike traditional SQL editors where each query is independent, WhoDB's AI assistant maintains conversational context throughout your session. This enables a natural, iterative approach to data exploration and analysis. ### How Context Works When you ask a question, the AI assistant considers: 1. Your current question 2. All previous messages in the conversation 3. Your database schema 4. Previous query results (implicitly) This context awareness allows the assistant to: - Understand pronouns and references ("show me that data", "what about the previous month?") - Refine previous queries based on feedback - Build complex analyses through multiple steps - Maintain topic continuity across questions ### Context Window Limitations Each AI provider and model has limits on how much conversation history can be maintained. Context window sizes vary by model — check your provider's documentation for the models you use. One token is approximately 4 characters. Longer schemas and verbose queries consume more tokens When the context window approaches its limit, consider starting a new conversation to maintain optimal performance. ## Multi-Turn Conversations Multi-turn conversations allow you to build complex analyses through natural dialogue, refining and iterating on results. ### Follow-Up Questions and Refinement After receiving initial results, ask follow-up questions that reference previous context: ```text You: Show me recent orders AI: [Returns orders from the last 30 days] You: Actually, just from the last week AI: [Adjusts the query to show 7 days instead] You: Sort by total amount descending AI: [Adds ORDER BY clause to the refined query] ``` The assistant understands that "them", "those", or "that query" refer to results from earlier in the conversation. ### Building Complex Analysis Break down complex questions into multiple steps: **You**: "What tables contain customer information?" **You**: "Show me the structure of the customers table" **You**: "Get all customers from California" **You**: "What's the average order value for these customers?" **You**: "Show me monthly trends" Each step builds on previous context, so you never repeat information the assistant already knows. ### Conversation Patterns Discover database contents through iterative questions: 1. "What tables exist in this database?" 2. "Tell me about the products table" 3. "Show me a sample of products" 4. "How many products are out of stock?" Start general and narrow down results: 1. "Show me all sales data" 2. "Just from Q4 2024" 3. "Only sales over $10,000" 4. "Show only the top 10 performers" Compare different data sets: 1. "Show me revenue for January" 2. "Now show me February" 3. "What's the percentage difference?" 4. "Which products drove the increase?" Drill down into specific issues: 1. "Are there any duplicate email addresses?" 2. "Show me the duplicates" 3. "Which accounts were created most recently?" 4. "Generate a query to merge the duplicates" ## Chat History Navigation WhoDB provides keyboard shortcuts to quickly access your previous questions, making it easy to rerun or modify earlier queries. ### Keyboard Shortcuts | Shortcut | Action | Description | |----------|--------|-------------| | ↑ (Arrow Up) | Previous message | Navigate backward through your message history | | ↓ (Arrow Down) | Next message | Navigate forward through your message history | | Enter | Send message | Submit your current message to the AI | ### Using Arrow Keys Click in the chat input field at the bottom of the screen. Press ↑ to load your most recent message, and continue pressing to move backward through your history. Press ↓ to move forward. Edit the message if needed, then press Enter to send. Or press Enter without changes to rerun the exact same query. This is useful for rerunning queries after data modifications, adjusting date ranges or filters in a previous question, and recalling phrasing that worked well. Use ↑ to quickly access and modify your last query instead of retyping similar questions ## Starting New Conversations When you need to start fresh or when your current conversation becomes too long, use the New Chat feature to clear context and begin a new session. ### When to Start a New Chat You're switching to a completely different topic or database area Your conversation has 30+ messages and performance is slowing The AI seems confused by mixed context from different topics You want to approach a problem with a clean slate ### Creating a New Chat Find the **New Chat** (+) button in the header of the Chat History sidebar. Click the button to create a new session. It is added to the Chat History sidebar, and your other sessions remain available there. The new session opens in the initial state with example prompts. Your provider and model selections remain unchanged. To clear the messages in the current session instead of starting a new one, click **Clear Chat** next to the AI provider dropdowns. ### Chat Sessions WhoDB supports multiple chat sessions, shown in a chat history sidebar: - **Create**: Click the add-session button in the sidebar to start a new session - **Switch**: Click any session in the sidebar to resume that conversation with its context intact - **Rename**: Edit a session's title directly; titles can also be auto-generated by the AI based on the conversation Use separate sessions for separate topics instead of mixing everything into one long conversation. ### Preserving Important Queries Before clearing a session, consider saving important queries to Scratchpad: 1. Find the query result you want to keep 2. Click **Move to Scratchpad** below the result 3. Choose or create a page for the query 4. Start your new chat ## Moving Queries to Scratchpad The Scratchpad integration allows you to save generated SQL queries for later use, modification, or documentation. This bridges the gap between conversational exploration and traditional SQL editing. ### Moving a Query to Scratchpad Ask the AI assistant a question that generates SQL. Wait for the query to execute and display results. Only SQL queries can be moved to Scratchpad. Text responses and errors cannot be transferred Locate the query result you want to save. Its actions appear in the row below the result. Click **Move to Scratchpad**. This opens the "Move to Scratchpad" dialog. Select where to add the query: **Option A: Existing Page** — select a page from the dropdown; the query is added as a new cell at the bottom. **Option B: New Page** — select "Create new page" and enter a descriptive name (e.g., "User Analysis Queries"). Click **Move to Scratchpad** to complete the transfer. WhoDB saves the query to the selected page, navigates you to the Scratchpad, and highlights the newly added query. ### Database Support The **Move to Scratchpad** option appears for SQL sources whose connection supports raw query execution — for example PostgreSQL, MySQL, MariaDB, SQLite, ClickHouse, and DuckDB. It is not shown for sources without a SQL query surface, such as Redis and MongoDB. For non-SQL sources, copy the generated query manually if needed for documentation ### Organizing Scratchpad Queries Develop a structure for organizing Scratchpad pages — by functionality (User Queries, Order Analysis, Data Cleanup), by frequency (Daily Reports, Monthly Summaries), or by complexity. Create descriptive page names that make it easy to find queries later, add comments documenting what each query accomplishes, and copy important SQL into your normal shared docs, runbooks, or repository. ## Managing Conversation History As you work with the AI assistant, managing your conversation history becomes important for maintaining clarity and performance. ### Conversation Lifecycle A conversation starts with minimal context and quick responses. As you build on previous questions, context accumulates — enabling sophisticated follow-ups but gradually increasing processing time. After 20-30 messages, performance may slow slightly; as you approach the model's context limit, consider starting a new chat or being more concise. ### Signs You Should Start a New Chat - Responses become noticeably slower - The AI references incorrect previous context - You're switching to a completely different task - Error messages about context length - You've had 50+ messages in one conversation ### Strategies for Long Sessions If you need extended conversations: **Use Models with Large Context Windows**: Some models support much larger context windows than others — check your provider's documentation for context sizes. **Be Concise**: Use shorthand in follow-ups ("show top 10" vs. "can you please show me the top 10 results") and reference previous results by position ("those top 3" instead of repeating criteria). **Periodically Summarize**: ```text Summarize what we've learned about the users table so far ``` **Save to Scratchpad**: Move important queries to Scratchpad, then start a new chat with a fresh context. ## Provider Management Managing your AI providers allows you to switch between different AI services, update credentials, and remove unused providers. ### Viewing Current Provider Your active provider and model are displayed in two dropdowns at the top of the Chat interface: - **Left dropdown**: AI Provider (OpenAI, Anthropic, Gemini, Ollama, LM Studio, etc.) - **Right dropdown**: Specific model, from the list fetched from that provider ### Switching Providers Click the AI Provider dropdown on the left. All configured providers appear in the list. Click on another provider to switch immediately. The model dropdown will update with available models for that provider. Choose the desired model from the updated model dropdown. You can now chat using the new provider and model. Switching providers does NOT clear your conversation history. The new provider will have access to previous messages ### Adding New Providers Click the AI Provider dropdown and select the green **"Add a provider"** option at the bottom. Choose the provider type, enter the required credentials, and click Submit. See [Setting Up AI Providers](/ai/setup-providers) for detailed configuration instructions, provider comparisons, and troubleshooting. ### Deleting Providers When you no longer need a provider (e.g., expired API key, switching to a different service), you can remove it: Use the provider dropdown to select the provider you want to remove. Click the **Delete Provider** button at the right end of the chat toolbar. This button is available for manually added providers. Click **Delete** in the confirmation dialog. This action cannot be undone. You'll need to re-enter credentials to use this provider again Environment-defined providers are managed by deployment configuration. ### Managing Multiple Providers You can configure multiple providers simultaneously, useful for: - **Flexibility**: Use a smaller, cheaper model for simple queries and a more capable model for complex analysis - **Redundancy**: Have backup providers if the primary service is down or rate-limited - **Privacy**: Use local models (Ollama, LM Studio) for sensitive queries and cloud providers for faster responses on non-sensitive data For provider selection strategy and cost considerations, see [AI Chat Assistant Best Practices](/best-practices/ai-usage); for setup details, see [Setting Up AI Providers](/ai/setup-providers). Keep at least two providers configured for reliability and flexibility ## Advanced Conversation Patterns ### Context Anchoring Reference specific previous results: ```text Take that query and add a filter for... Apply the same logic to the orders table instead ``` ### Explaining AI Decisions Ask the AI to explain its query choices: ```text Why did you use a LEFT JOIN instead of INNER JOIN? Is there a more efficient way to write this? ``` This helps you learn SQL while using the assistant. ### Multi-Step Workflows Combine conversation with Scratchpad for complex workflows: 1. **Explore with AI**: Ask questions to understand data structure 2. **Generate Base Query**: Get AI to create the initial query 3. **Move to Scratchpad**: Save the query for refinement 4. **Refine Manually**: Add complexity or optimization 5. **Iterate**: Return to the AI with follow-up questions as needed This hybrid approach combines AI speed with manual control. ## Troubleshooting Conversation Issues **Symptoms**: Incorrect query generated, wrong table used, unexpected results **Solutions**: - Be more specific with table and column names - Break complex questions into smaller parts - Use full names instead of pronouns in early messages **Example Fix**: - ❌ "Show me the data" (too vague) - ✅ "Show me all rows from the users table" **Solutions**: - Start a new chat to clear context - Switch to a faster model from your provider's lineup - Save important queries to Scratchpad and start fresh - Use models with larger context windows for long sessions **Symptoms**: Follow-up questions use incorrect table or data from earlier in the conversation **Solutions**: - Be explicit in your question (mention the table name) - Start a new chat if context is too mixed **Example Fix**: - ❌ "Now show me those records" (ambiguous) - ✅ "Show me users from the last query who have orders" **Reasons**: - Current database doesn't support SQL (Redis, MongoDB) - Message is a text response or an error, not a SQL query **Solutions**: Only SQL query results can be moved to Scratchpad. For NoSQL databases, manually copy the query text. **Solutions**: - Ensure the input field is focused (click in the input box) - Verify messages were actually sent (look for user messages in chat) **Note**: Arrow keys only navigate through your sent messages, not AI responses For provider connection issues (invalid API keys, rate limits, Ollama not running), see [Setting Up AI Providers](/ai/setup-providers). ## Next Steps Now that you understand conversation management and advanced features: Learn techniques for effective data retrieval with natural language Safely update, insert, and delete data with AI assistance Master the Scratchpad for advanced SQL editing and query management Discover tips for optimal AI assistant usage Mastering conversation features transforms the AI assistant from a query generator into a powerful collaborative partner for database work --- ### Ai Introduction (`docs/ai/introduction.mdx`) --- title: "AI Chat Assistant - Introduction" description: "Transform how you interact with databases using WhoDB's natural language AI assistant" --- # AI Chat Assistant - Introduction WhoDB's AI Chat Assistant revolutionizes database interaction by letting you use natural language instead of SQL. Ask questions in plain English, and the assistant generates accurate queries, executes them, and presents results in an intuitive format. The AI Chat Assistant is like having an expert database administrator and SQL developer available 24/7 ## What is the AI Chat Assistant? The AI Chat Assistant bridges the gap between human language and database queries. Instead of remembering SQL syntax, table names, or join conditions, you simply describe what you want to know. ### Key Capabilities Ask questions in plain English and get structured results The assistant writes optimized SQL based on your database schema Works with PostgreSQL, MySQL, MongoDB, SQLite, Redis, and more Choose from OpenAI, Anthropic, Gemini, Ollama, LM Studio, or custom providers Maintains conversation history for follow-up questions Requires explicit confirmation for data modifications and schema changes (INSERT, UPDATE, DELETE, CREATE, ALTER, DROP) ## How It Works The AI Chat Assistant follows a simple workflow: Type your question in natural language: - "Show me all users" - "What's the total revenue this month?" - "Find products that are out of stock" The assistant analyzes your question and your database schema to create the appropriate SQL query. The assistant has full awareness of your tables, columns, relationships, and database type WhoDB runs the generated SQL against your database. The assistant's text responses stream token-by-token as they are generated. Results appear in an interactive table with sorting, filtering, and export capabilities. ## Who Should Use the AI Assistant? ### Complete Beginners Never written SQL before? The AI assistant is perfect for you: - No SQL knowledge required - Learn by seeing how questions translate to queries - Explore databases without fear of making mistakes - Get immediate results without syntax errors ### Data Analysts Speed up your analysis workflow: - Generate complex aggregations in seconds - Explore data relationships naturally - Create ad-hoc reports without writing SQL - Focus on insights, not query syntax ### Developers Accelerate development tasks: - Quickly understand unfamiliar database schemas - Test queries before implementing in code - Debug data issues in development and production - Prototype features without writing boilerplate SQL ### Database Administrators Simplify routine tasks: - Run diagnostics with natural language commands - Generate reports for stakeholders - Validate data integrity - Perform maintenance queries efficiently ## Example Interactions ### Simple Data Retrieval **You**: "Show me all users" **Assistant**: "Here are all the users in the database." Result: Interactive table with all user records --- **You**: "Find users created in the last week" **Assistant**: [Generates and executes appropriate date filter query] Result: Filtered user list with recent signups ### Complex Analysis **You**: "Count users by email domain" **Assistant**: "Here's the user count by email domain." Result: Aggregated data showing domain distribution --- **You**: "Show average order value by month for the last year" **Assistant**: [Generates query with date grouping and aggregation] Result: Monthly revenue analysis table ### Data Modification **You**: "Delete user with id 5" **Assistant**: [Generates the DELETE statement and shows an inline confirmation alert in the chat. You can view the SQL behind a toggle, then click Confirm or Cancel.] **You**: [Click Confirm] Result: "Action Executed - DELETE" The same inline confirmation applies to all data modifications and schema changes (INSERT, UPDATE, DELETE, CREATE, ALTER, DROP). ## Supported AI Providers WhoDB supports multiple AI providers, giving you flexibility based on your needs: Model lists are fetched live from each provider when you add it, so the available models always reflect your provider's current lineup. **Best for**: Most users, general-purpose queries **Models Available**: Fetched from your OpenAI account when you add the provider **Requirements**: OpenAI API key **Pros**: - Highly accurate SQL generation - Fast response times - Excellent natural language understanding **Cons**: - Requires paid API key - Data sent to external servers **Best for**: Complex reasoning, large context windows **Models Available**: Fetched from your Anthropic account when you add the provider **Requirements**: Anthropic API key **Pros**: - Excellent handling of complex queries - Very large context windows - Strong reasoning capabilities **Cons**: - Requires paid API key - Data sent to external servers **Best for**: Google Cloud users, general-purpose queries **Models Available**: Fetched from the Gemini API when you add the provider **Requirements**: Gemini API key **Pros**: - Strong SQL generation and reasoning - Competitive pricing **Cons**: - Requires API key - Data sent to external servers **Best for**: Privacy-focused deployments, no API costs **Models Available**: Whatever models you have downloaded locally with Ollama **Requirements**: Ollama installed locally **Pros**: - Complete data privacy (runs locally) - No API costs - No internet required - Full control over model selection **Cons**: - Requires local installation - Slower than cloud providers - Requires sufficient local hardware **Best for**: Running local models with a desktop GUI **Models Available**: Whatever models you have loaded in LM Studio **Requirements**: LM Studio installed locally with its local server running **Pros**: - Complete data privacy (runs locally) - No API costs - Easy model management via the LM Studio app **Cons**: - Requires local installation - Requires sufficient local hardware **Best for**: Enterprise deployments, specific requirements Configure custom AI endpoints for: - Self-hosted models - Enterprise AI platforms - Specialized database assistants Contact your administrator for configuration details. ## Privacy and Security ### Data Privacy Considerations **What's Sent**: Your database schema and query text **Not Sent**: Actual database data or query results **Consideration**: Review your organization's policies on external AI services **What's Sent**: Nothing—all processing is local **Privacy**: Complete data isolation **Consideration**: Ideal for sensitive or regulated environments For highly sensitive data, use local models (Ollama) or consult your security team before enabling external AI providers ### Security Features - **Confirmation Required**: All data modifications and schema changes (INSERT, UPDATE, DELETE, CREATE, ALTER, DROP) require explicit confirmation - **Read-Only Mode**: Can be configured with read-only database users - **Permission Aware**: Respects your database user permissions ## Getting Started Ready to use the AI Chat Assistant? Follow these steps: Decide between external providers (OpenAI, Anthropic, Gemini) or local models (Ollama, LM Studio). - **Use External Providers** if: You want the best accuracy and speed - **Use Local Models** if: Privacy is critical or you want zero API costs Configure your chosen AI provider with necessary credentials. [Learn how to set up providers →](/ai/setup-providers) Navigate to the Chat page and begin exploring your database. [See querying guide →](/ai/querying-data) ## AI Assistant Features Overview ### Core Features - **Natural Language Understanding**: Ask questions conversationally - **Schema Awareness**: Knows your tables, columns, and relationships - **SQL Generation**: Creates optimized, database-specific queries - **Result Display**: Interactive tables with full functionality - **Error Handling**: Clear explanations when queries fail ### Advanced Features - **Conversation Context**: Follow-up questions build on previous queries - **Multi-Turn Dialogues**: Complex analysis through iterative refinement - **SQL Code View**: Toggle between results and generated SQL - **Scratchpad Integration**: Move queries to Scratchpad for refinement - **Query History**: Navigate through previous questions with keyboard shortcuts - **Chat Sessions**: Keep multiple conversations in a chat history sidebar and switch between them ### Data Operations - **SELECT Queries**: Retrieve and analyze data - **Aggregations**: COUNT, SUM, AVG, MIN, MAX automatically applied - **Filtering**: WHERE conditions from natural language - **Sorting**: ORDER BY inferred from context - **Joins**: Multi-table queries generated correctly - **Data and Schema Modifications**: INSERT, UPDATE, DELETE, CREATE, ALTER, and DROP with confirmation ## Use Case Examples Quickly understand unfamiliar databases Generate reports without writing SQL See how natural language translates to SQL Find data quality issues efficiently ## Comparison: Traditional SQL vs. AI Assistant | Task | Traditional Approach | With AI Assistant | |------|---------------------|-------------------| | Find all users | Write `SELECT * FROM users` | Ask "Show me all users" | | Complex aggregation | Write GROUP BY with multiple joins | Ask "Count orders by product category" | | Filter by date | Remember date syntax for your database | Ask "Show orders from last month" | | Debug query | Read error message, fix syntax | Rephrase question naturally | | Learn schema | Query information_schema tables | Ask "What tables exist?" | | Data modification | Write UPDATE/DELETE carefully | Describe change, confirm when prompted | The AI assistant eliminates syntax memorization and reduces the time from question to insight. ## Limitations While powerful, the AI assistant has some limitations: Extremely complex multi-step operations may require: - Breaking into multiple queries - Manual refinement in Scratchpad - Traditional SQL for edge cases The assistant respects your database user permissions: - Cannot perform operations beyond granted privileges - No elevation of user permissions - Same access as manual SQL queries Query generation depends on: - AI provider speed - Network latency (for external providers) - Database query execution time - Schema complexity While highly accurate, the assistant may occasionally: - Misinterpret ambiguous questions - Generate suboptimal queries for very specific needs - Require clarification for context-dependent requests Always review generated SQL for critical operations. ## Best Practices Start specific, then generalize. Instead of "show me the data," ask "show me all users created this month" Key practices for effective AI assistant use: 1. **Be Specific**: Clear questions get better results 2. **Provide Context**: Mention table names when ambiguous 3. **Verify SQL**: Review generated queries for important operations 4. **Use Confirmation**: Take advantage of modification confirmations 5. **Leverage Context**: Build on previous questions in conversation 6. **Move to Scratchpad**: Save useful queries for reuse [Learn more best practices →](/best-practices/ai-usage) ## Next Steps Configure OpenAI, Anthropic, Gemini, Ollama, or LM Studio for your needs Learn how to ask questions and retrieve data Step-by-step tutorial for beginners Safely update, insert, and delete records The AI Chat Assistant makes database interaction accessible to everyone, from complete beginners to experienced professionals --- ### Ai Modifying Data (`docs/ai/modifying-data.mdx`) --- title: "Modifying Data with AI Chat Assistant" description: "Learn how to safely insert, update, and delete data using WhoDB's AI Chat Assistant with built-in confirmation safeguards" --- # Modifying Data with AI Chat Assistant WhoDB's AI Chat Assistant enables you to modify database records using natural language while maintaining strict safety controls. Every data modification operation requires explicit confirmation before execution, preventing accidental changes to your database. Data modification operations are permanent and affect your actual database. Always review confirmation prompts carefully before proceeding. This workflow requires an active AI provider and selected model. If the Chat input is disabled or no model is available, configure the provider before testing data modifications. ## Understanding AI-Powered Data Modification The AI Chat Assistant translates your natural language instructions into SQL statements for data modifications and schema changes (INSERT, UPDATE, DELETE, CREATE, ALTER, DROP). Unlike read-only queries, these operations follow a confirmation workflow to ensure safety. ### Key Safety Features All data modifications and schema changes (INSERT, UPDATE, DELETE, CREATE, ALTER, DROP) require manual confirmation before execution See exactly what SQL will execute before confirming Clear confirmation when operations complete successfully Detailed error messages if modifications fail ### How the Confirmation Workflow Works Ask the AI assistant to insert, update, or delete data (or change the schema) using natural language. The assistant analyzes your request and generates the appropriate SQL statement. Nothing executes yet. An inline alert appears in the chat warning that the operation will modify your database, with a **Show Query** toggle to view the SQL and Confirm/Cancel buttons. Click **Show Query** to review the SQL statement carefully. Confirm to proceed or cancel to abort. After confirmation, the SQL executes against your database. An "Action Executed" message appears confirming the operation completed. The confirmation workflow ensures you always know exactly what changes will be made before they occur. The same workflow applies to schema changes (DDL): asking the assistant to CREATE, ALTER, or DROP a table produces the same inline confirmation before anything executes. ## INSERT Operations - Adding Records Use natural language to add new records to your tables. The AI assistant generates appropriate INSERT statements based on your table schema. ```text User Request Create a new product named Laptop with price 999.99, category Electronics, and stock 50 ``` The AI generates: ```sql Generated SQL INSERT INTO products (name, price, category, stock) VALUES ('Laptop', 999.99, 'Electronics', 50) ``` The inline confirmation appears; click **Show Query** to verify the table, columns, and values before confirming. ### Best Practices for INSERT Operations Mention all columns that don't have default values or aren't auto-generated. **Good**: "Add a user with name, email, and status" **Avoid**: "Add a user" (missing required fields) In databases with multiple schemas or similar table names, specify the full table path, e.g. "Add to test_schema.users". Provide values in the expected format for the column type. **Good**: "Add order with date 2025-01-15" **Avoid**: "Add order with date next Monday" (ambiguous) Always check the generated INSERT statement in the confirmation prompt (via **Show Query**) to ensure the correct table is targeted, all required columns are included, and data types match expectations. ## UPDATE Operations - Modifying Records Update existing records by describing what should change and which records to affect. ```text User Request Update the email for user with id 5 to newemail@example.com ``` The AI generates: ```sql Generated SQL UPDATE users SET email = 'newemail@example.com' WHERE id = 5 ``` Conditional and bulk updates work the same way: "Set all products in Electronics category to have discount 10" generates an UPDATE with the matching WHERE clause. When the confirmation appears, pay special attention to the WHERE clause. Missing WHERE clauses will update ALL records in the table. Always verify the WHERE condition before confirming. After confirming, verify the "Action Executed" message, then run a follow-up SELECT to confirm the intended records changed. ### Best Practices for UPDATE Operations Be explicit about which records to update to avoid accidentally modifying all records. **Good**: "Update user with id 5" **Dangerous**: "Update all users" (only if you mean all records) Reference records by primary keys or unique columns when possible. **Good**: "Update user with id 123" **Risky**: "Update user named John" (may match multiple records) For complex updates, first run a SELECT with the same criteria to see which records will be affected. **Step 1**: "Show me all inactive users" **Step 2** (after reviewing): "Set all inactive users to status deleted" Generally avoid updating primary key columns, as this can break foreign key relationships. ## DELETE Operations - Removing Records Delete operations require the most caution as they permanently remove data. Like all modifications, DELETE requests go through the inline confirmation before anything executes. ```text User Request Delete user with id 5 ``` The AI generates the statement and shows the inline confirmation: ```sql Generated SQL DELETE FROM users WHERE id = 5 ``` Conditional deletions ("Delete all inactive users", "Remove all log entries older than 90 days") follow the same pattern. Click **Show Query** to review the DELETE statement and its WHERE clause, then Confirm to execute or Cancel to abort. Afterwards, run a follow-up SELECT to confirm the records are gone. DELETE operations are permanent and cannot be undone through WhoDB. Only database backups can restore deleted data. ### Best Practices for DELETE Operations Never delete without specifying which records to remove. **Good**: "Delete user with id 5" **Extremely Dangerous**: "Delete all users" (removes all records) Run a SELECT query first to verify which records will be deleted. **Step 1**: "Show me all users with status inactive" **Step 2** (after reviewing): "Delete all users with status inactive" If your schema has CASCADE DELETE rules, deleting one record might remove related records in other tables. Conversely, foreign key constraints may block the deletion until referencing records are handled. Verify your schema's constraints before deleting. For recoverable deletions, use UPDATE to set a deleted flag instead of DELETE: "Update user with id 5 to set deleted true". Before deleting thousands of records, verify your database backup is current and tested. ## Understanding Confirmation Prompts The inline confirmation is your final checkpoint before data modification. ### What the Confirmation Shows An inline alert warning that the operation will modify your database. Click **Show Query** to see the exact SQL that will execute, including the target table, columns being affected, WHERE conditions, and values being set. - **Confirm**: Proceeds with the operation - **Cancel**: Aborts the operation ### Checklist Before Confirming - [ ] The SQL targets the intended table and schema - [ ] For UPDATE and DELETE, the WHERE clause targets the correct records - [ ] For INSERT and UPDATE, values are correct and properly formatted - [ ] The number of rows likely affected matches your expectation - [ ] A WHERE clause exists unless you truly intend to affect all records If anything in the confirmation looks unexpected, click Cancel and rephrase your request to the AI assistant. ## Verifying Modifications After executing a modification, verify the changes were applied correctly. Look for the "Action Executed" confirmation in the chat. The success message shows which operation ran (for example, "DELETE" or "UPDATE"). Run a SELECT query to verify the changes: **After INSERT**: "Show me the user with email newemail@example.com" **After UPDATE**: "Show user with id 5" **After DELETE**: "Count how many log entries are older than 90 days" (expect 0) ## Rollback Strategies Plan recovery before making data modifications. ### Prevention is Best Practice modifications in a development database first Run critical operations in transaction blocks (via Scratchpad) Maintain frequent database backups for recovery Use status flags instead of permanent deletion where possible ### If You Make a Mistake Don't make additional changes. Note exactly what operation was executed, how many records were affected, and which tables. For small mistakes (wrong value in one record), correct it with another UPDATE: ```text Update product_id 123 to set price 10 ``` For significant data loss, restore the database from the most recent backup before the mistake. Restoring from backup will lose any changes made after the backup was created. ### Using Transactions for Safety For critical modifications, use the Scratchpad to wrap operations in transactions: ```sql Transaction Example BEGIN; UPDATE users SET status = 'inactive' WHERE last_login < '2023-01-01'; -- Review the affected rows SELECT * FROM users WHERE status = 'inactive' AND last_login < '2023-01-01'; -- If everything looks correct: COMMIT; -- If something is wrong: ROLLBACK; ``` The AI Chat Assistant executes operations immediately after confirmation. For transaction control, use the Scratchpad query interface where you can manually manage BEGIN, COMMIT, and ROLLBACK. ## Safety Practices for Production Data The essentials when modifying production databases: - **Verify your connection** — confirm you're on the correct database (production vs. development) - **SELECT before you modify** — understand the current state and estimate the affected row count - **Check backups** — ensure recent backups exist and have been tested - **Start small** — for bulk operations, test with a single record first - **Verify each step** — check results after each modification before proceeding - **Schedule bulk operations** — perform large modifications during maintenance windows For the full set of production guidelines, change management checklists, and audit considerations, see [AI Chat Assistant Best Practices](/best-practices/ai-usage). ## Common Modification Patterns The core pattern for every safe modification is **preview → modify → verify**: ```text Show all orders with status pending older than 30 days ``` Review (and optionally count) the records that will be affected. ```text Update all orders with status pending older than 30 days to status cancelled ``` Review the confirmation and confirm. ```text Show all orders with status cancelled from the last 30 days ``` Confirm the change was applied correctly. The same three-step shape applies to inserts (check the record doesn't exist, insert, confirm it was created), deletions (preview, delete, confirm records are gone), and cleanups. For records with dependencies — such as deleting a user who has orders — handle the related records first, then the parent record. ## Troubleshooting **Symptom**: You request a modification, but no confirmation prompt shows. **Possible Causes**: The AI generated a read-only query instead, or the request wasn't recognized as a modification. **Solutions**: Rephrase your request more explicitly ("Delete user with id 5"), check the AI response for error messages, or try a simpler, more direct modification request. **Symptom**: "Foreign key constraint violation" error when deleting or updating. **Cause**: Other tables have records referencing the record you're trying to modify. **Solutions**: Delete or update referencing records first, check foreign key relationships, and consider CASCADE rules in your schema. **Symptom**: AI warns "This will affect all records in the table". **Solutions**: Cancel the operation and rephrase with specific conditions ("Delete user with id 5" not "Delete user"). If you truly want to affect all records, explicitly confirm. **Symptom**: The operation modified different records than intended. **Solutions**: Check what was actually modified with a SELECT query. If correctable, run a compensating UPDATE; if serious, restore from backup. In future, use SELECT first to verify target records. **Symptom**: "Permission denied" or "Insufficient privileges" error. **Solutions**: Verify your database user has INSERT/UPDATE/DELETE permissions, contact your database administrator, or check if the connection is read-only. **Symptom**: "Data type mismatch" or "Invalid input syntax" error. **Solutions**: Check the table schema for column types. Format dates as 'YYYY-MM-DD', use numbers without quotes for numeric columns, and proper boolean values (true/false). **Solutions**: Break the operation into smaller batches, use the Scratchpad to run batched statements with transaction control, and schedule large modifications during low-traffic periods. ## Comparing AI Chat vs. Traditional Methods | Scenario | AI Chat Assistant | Traditional UI | Scratchpad SQL | |----------|-------------------|----------------|----------------| | **Quick single record change** | Fast and convenient | Multiple clicks required | Overkill for simple changes | | **Bulk updates with conditions** | Natural language, easy | Must use SQL | Most control and visibility | | **Complex multi-table operations** | May require multiple steps | Not supported | Best option | | **Production-critical changes** | Good with careful review | Limited capabilities | Recommended for control | Use AI Chat for quick modifications and exploration. Use Scratchpad for complex operations requiring transaction control or multiple related statements. ## Security Considerations Your modification requests and table/column names may be sent to external AI providers. However, actual data values are not sent. **What's Sent to AI Providers**: your natural language request, database schema (table and column names), and database type. **What's NOT Sent**: actual data values, query results, or existing record contents. For maximum privacy, use local models (Ollama, LM Studio) for complete data isolation, avoid mentioning sensitive values in your requests, and use Scratchpad for modifications involving sensitive data. **Audit and Compliance**: Modifications are associated with the database user credentials used, so database-level logs can attribute changes. Enable your database's audit or query logging for comprehensive tracking. **Permission Management**: Follow the principle of least privilege — use database users with only necessary permissions, read-only users for exploration, and separate credentials for production vs. development. See [AI Chat Assistant Best Practices](/best-practices/ai-usage) for read-only user setup. ## Next Steps Learn how to retrieve and analyze data without modifications Master multi-turn conversations for complex operations Use SQL directly for complex modifications requiring transactions The canonical guide to safe and effective AI assistant usage Every data modification and schema change (INSERT, UPDATE, DELETE, CREATE, ALTER, DROP) requires explicit confirmation, with full visibility into the SQL before it executes. Review carefully, verify with follow-up queries, and maintain backups for recovery. --- ### Ai Querying Data (`docs/ai/querying-data.mdx`) --- title: "Querying Data with AI Chat" description: "Master the art of retrieving and analyzing database data using natural language queries" --- # Querying Data with AI Chat WhoDB's AI Chat Assistant transforms database querying from a technical task into a natural conversation. Ask questions in plain English and get instant results with interactive tables, generated SQL, and intelligent error handling. The AI assistant understands your database schema and generates optimized queries specific to your database type AI querying requires an active provider and selected model. If the Chat page shows no available model, configure a provider first; prompts cannot be sent without a model. ## Getting Started with Simple Queries Traditional database querying requires knowledge of SQL syntax, table structures, and join relationships. The AI Chat Assistant eliminates these barriers by translating natural language into accurate SQL queries. ### Asking Your First Question Click the Chat icon in the sidebar to access the AI assistant. In the input field at the bottom, type a natural language question: ```text Example Questions Show me all users List all products Display recent orders ``` Send your message. The AI's response streams in token-by-token as it is generated. The assistant will display the results in an interactive table. ### Simple Text Responses For general questions or informational queries, the assistant provides text responses: **Example interactions:** - "Hello" - Gets a greeting and overview of capabilities - "What tables are available?" - Lists your database tables - "Explain what the users table contains" - Provides schema information Text responses are perfect for understanding your database structure before diving into queries ## Retrieving Data with SELECT Queries The AI assistant excels at generating SELECT queries. Describe what you want — including columns, filters, sorting, and limits — and it produces the SQL: ```text Natural Language Show me usernames and emails from users created after 2024-01-01, sorted by most recent, limit 10 ``` **Generated SQL**: ```sql SELECT username, email FROM users WHERE created_at > '2024-01-01' ORDER BY created_at DESC LIMIT 10 ``` You can phrase filtering, sorting, and limiting naturally: "the first 10 users", "products that cost more than $100", "orders with status 'completed'", "sorted by price descending". Specify column names in your questions for more focused results ## Viewing SQL Code One of the most powerful features is the ability to toggle between table results and the generated SQL code. ### SQL Code Toggle When the assistant returns query results: Look at the row of actions below the results table. Click **SQL** to switch from table view to SQL code view. Examine the generated SQL to understand how your question was translated. Click **Show as table** in the same row to return to the table view. ### Why View SQL Code? See how natural language maps to SQL syntax. Asking "Count users by email domain" and viewing the generated SQL teaches you string functions, aggregation, and grouping all at once. For important queries, review the SQL to ensure it matches your intent: check JOIN conditions, WHERE clauses, GROUP BY logic, and ORDER BY direction. Copy generated SQL to use in application code, save in the Scratchpad for refinement, or share with teammates. If results aren't what you expected, viewing the SQL helps identify wrong table references, incorrect column names, missing filters, or unexpected joins. ## Complex Queries The AI assistant handles sophisticated queries including aggregations, joins, and multi-step logic. ### Aggregation Queries Count, sum, average, and other statistical operations: ```text Natural Language Count users by email domain ``` **Generated SQL**: ```sql SELECT SUBSTRING(email FROM POSITION('@' IN email) + 1) as domain, COUNT(*) as user_count FROM users GROUP BY domain ORDER BY user_count DESC ``` All the standard aggregation functions are available through natural phrasing: "How many users do we have?" (COUNT), "What's the total revenue?" (SUM), "Average order value" (AVG), "Highest and lowest prices" (MIN/MAX). ### Multi-Table Queries (Joins) Ask questions that span multiple tables: ```text Natural Language Show me users and their order counts ``` **Generated SQL**: ```sql SELECT u.id, u.username, COUNT(o.id) as order_count, SUM(o.total) as total_spent FROM users u LEFT JOIN orders o ON u.id = o.user_id GROUP BY u.id, u.username ORDER BY total_spent DESC ``` The AI assistant automatically determines the correct JOIN type and conditions based on your schema ### Dates, Patterns, and Multiple Conditions The same natural phrasing works for time filters, pattern matching, and combined criteria: ```text Natural Language Examples Show revenue grouped by month for 2024 Find users with gmail email addresses Show me active users who joined after 2024-01-01 and have made at least 5 orders ``` The AI translates these into `DATE_TRUNC`/date-range filters, `LIKE` patterns, and combined `WHERE` conditions or subqueries as appropriate for your database type. ## Understanding Query Results When the AI assistant returns data, you get an interactive table with powerful features. ### Interactive Results Table **Table Features**: - **Column Headers**: Show field names from your query - **Scrollable**: Navigate large result sets - **Readable Formatting**: Data types displayed appropriately The assistant also provides context about your results, such as the number of rows returned and the columns included. NULL values are clearly distinguishable from empty strings. ### Large Result Sets Chat results are shown in a fixed-height scrollable table containing all returned rows. For queries returning many rows: - Scroll within the table to view all data - Consider adding LIMIT in your question Very large result sets may take longer to load. Ask the assistant to limit results if you only need a sample ## Handling Errors The AI assistant provides clear, helpful error messages when queries fail. ### Common Error Types **Error Example**: `ERROR: relation "test_schema.nonexistent_table" does not exist` The table or column name doesn't exist in your schema. Check your names, ask "What tables are available?", and verify spelling and schema name. **Error Example**: `ERROR: permission denied for table users` Your database user lacks permissions for this operation. Contact your database administrator or use a connection with appropriate permissions. **Error Example**: `ERROR: syntax error at or near "FROM"` The generated SQL has invalid syntax (rare but possible). Rephrase your question more clearly or break complex questions into simpler steps. **Error Example**: `ERROR: column "price" is of type numeric but expression is of type text` Data type incompatibility in the query. Ask about the schema first, then rephrase with correct type expectations. ### Recovering from Errors The error message contains clues about what went wrong. Try asking the same thing in a different way, or break a complex question into steps: ```text Complex (may fail) Show me average order value by month for premium users who joined in 2024 Simpler (more reliable) 1. Show me premium users 2. [After reviewing] Show me order values for these users 3. [After reviewing] Calculate average by month ``` The assistant can explain what went wrong: ```text Why did my last query fail? Can you explain the error message? ``` ## Multi-Turn Conversations One of the AI assistant's most powerful features is maintaining conversation context for follow-up questions. ### Building on Previous Queries The assistant remembers your conversation, so each question can build on the last: ```text Iterative Refinement You: Show me all users AI: [Returns 1000 users] You: Just the first 10, sorted by most recent AI: [Returns 10 users sorted by created_at DESC] You: Only active ones AI: [Adds the status filter, keeping earlier refinements] ``` Use pronouns like "them," "those," "these" naturally—the assistant understands what you're referring to ### Chat History Navigation Navigate through your previously sent messages using the arrow keys: press Up in the input field to load your previous message, keep pressing to go further back, and press Down to move forward. Edit the loaded message and press Enter to resend. ## Example Workflow: Data Exploration Explore an unfamiliar database systematically: ```text What tables are available? ``` ```text Describe the users table ``` ```text Show me 5 sample users ``` ```text How are users related to orders? ``` ```text Count users by status ``` For a complete walkthrough of this approach, see [AI-Powered Data Exploration](/use-cases/ai-data-exploration). ## Moving Queries to Scratchpad When you generate a useful query, save it for future use: Locate a query result with SQL you want to save. Click **Move to Scratchpad** below the result. **New Page**: Creates a new Scratchpad page with this query **Existing Page**: Adds to an existing Scratchpad page Provide a page name if creating new. Click Confirm to move the query. Navigate to Scratchpad to find your saved query, where you can edit and refine it, add comments, execute it multiple times, or copy important SQL into shared docs, runbooks, or your repository. Queries moved to Scratchpad retain their original SQL exactly as generated by the AI For guidance on formulating effective questions, safety, and performance, see [AI Chat Assistant Best Practices](/best-practices/ai-usage). ## Keyboard Shortcuts | Shortcut | Action | |----------|--------| | `Up Arrow` | Previous message in history | | `Down Arrow` | Next message in history | | `Enter` | Send message | Arrow keys navigate through your previously sent messages, letting you quickly reload, edit, and resend earlier questions. ## Troubleshooting **Symptom**: The generated query doesn't match what you wanted. **Solutions**: 1. Rephrase using different words 2. Be more specific about table and column names 3. Break complex questions into simpler parts Try: "Show me users who signed up recently" instead of "Get the new people" **Symptom**: Query executes but returns unexpected data. **Solutions**: 1. View the generated SQL to understand the logic 2. Check if filters are applied correctly and you're querying the right table 3. Add more specific conditions to your question **Symptom**: The assistant or query execution is very slow. **Solutions**: 1. Add LIMIT to your question: "Show me the first 100 users" 2. Add specific filters or time ranges to reduce data volume 3. Break large operations into smaller queries **Symptom**: You want to re-run a query but can't find it. **Solutions**: 1. Use Up arrow to navigate through message history 2. Scroll up in the chat to find the previous response 3. Check Scratchpad if you moved the query there Next time: Move important queries to Scratchpad for permanent storage **Symptom**: You expected a table but got a text explanation. **Solutions**: Make your question action-oriented using verbs like "show," "list," "display," "find". Try: "Show me all users" instead of "Tell me about users" **Symptom**: The result table doesn't include columns you need. **Solutions**: Explicitly mention the columns you want, or ask a follow-up: "Show me the same data but include [column name]". ## Limitations Understanding limitations helps set appropriate expectations: Some extremely complex operations — recursive CTEs with deep nesting, dynamic pivots, complex window functions with multiple partitions — may require manual SQL in Scratchpad or breaking into multiple simpler queries. The AI adapts to your database type, but some advanced, database-specific functions or proprietary extensions may need clarification. Move to Scratchpad for advanced database-specific features. The AI generates correct queries but may not always produce the most optimized version for very large tables or complex join scenarios. Review and optimize generated SQL in Scratchpad for performance-critical queries. If your question could be interpreted multiple ways, the AI will make its best guess or ask clarifying questions. Be as specific as possible, especially with column names and conditions. ## Privacy and Security For sensitive data, consider using local AI models (Ollama, LM Studio) instead of cloud providers **Sent to AI**: - Your question text - Database schema (table and column names) - Database type (PostgreSQL, MySQL, etc.) **Never Sent**: - Actual data from your tables - Query results - Connection credentials - Passwords or sensitive configuration For sensitive data: use local models for complete privacy, review your organization's policies on external AI services, avoid sensitive schema names with cloud providers, and use read-only connections for exploratory querying. See [AI Chat Assistant Best Practices](/best-practices/ai-usage) for full guidance. ## Next Steps Learn how to safely update, insert, and delete data using the AI assistant Master advanced conversation features and context management Move generated queries to Scratchpad for refinement and reuse Discover optimal patterns for AI-assisted database work You now have the skills to query databases naturally using AI, view and understand generated SQL, and handle complex data retrieval scenarios --- ### Ai Setup Providers (`docs/ai/setup-providers.mdx`) --- title: "Setting Up AI Providers" description: "Configure OpenAI, Anthropic, Gemini, Ollama, LM Studio, or custom AI providers for WhoDB's chat assistant" --- # Setting Up AI Providers Before you can use WhoDB's AI Chat Assistant, you need to configure at least one AI provider. This guide walks you through setting up each supported provider and choosing the right model for your needs. ## Accessing the AI Configuration Navigate to the Chat page in WhoDB to access AI provider settings: The provider configuration is located at the top of the Chat interface with two dropdowns: 1. **AI Provider**: Select your provider (OpenAI, Anthropic, Gemini, Ollama, LM Studio, etc.) 2. **AI Model**: Choose the specific model to use Pick a provider and model before starting the query tutorials. The chat box becomes active once both are selected. ## Choosing Your AI Provider Different providers offer different trade-offs between accuracy, speed, cost, and privacy: **Best for**: Most users, general-purpose queries - Industry-leading accuracy - Fast response times - Pay-per-use pricing **Best for**: Complex reasoning, large contexts - Excellent with sophisticated queries - Very large context windows - Strong safety features **Best for**: Google Cloud users, general-purpose queries - Strong reasoning and SQL generation - Competitive pricing - Pay-per-use pricing **Best for**: Privacy, zero API costs - Complete data privacy - No internet required - Free to use **Best for**: Enterprise deployments - Self-hosted models - Organization-specific configurations - Complete control ## Setting Up OpenAI OpenAI provides GPT models that offer excellent SQL generation capabilities with fast response times. ### Prerequisites Before configuring OpenAI: 1. Create an OpenAI account at https://platform.openai.com/ 2. Add payment method to your account 3. Generate an API key from https://platform.openai.com/api-keys Keep your OpenAI API key secure. Never share it or commit it to version control ### Configuration Steps On the Chat page, click the AI Provider dropdown. Click **"Add a provider"** at the bottom of the dropdown menu. A sheet will slide in from the right side of the screen. In the Model Type dropdown, select **OpenAI**. The dropdown shows icons for each provider to help identify them quickly. Paste your OpenAI API key in the Token field. ```text Example API Key Format sk-proj-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ``` API keys starting with `sk-proj-` are project-specific keys recommended by OpenAI Click the **Submit** button to test the connection and save your configuration. WhoDB will: - Verify the API key is valid - Fetch available models - Save the configuration for future use Once configured, the AI Model dropdown populates with the models fetched live from your OpenAI account. Pick a model based on your accuracy, speed, and cost needs. ### Cost Considerations OpenAI charges based on tokens used. Costs vary by model, schema size, and conversation length. See [OpenAI's pricing page](https://openai.com/api/pricing/) for current rates and monitor usage in your OpenAI dashboard. ## Setting Up Anthropic (Claude) Anthropic's Claude models excel at complex reasoning and handle large database schemas exceptionally well. ### Prerequisites 1. Create an Anthropic account at https://console.anthropic.com/ 2. Add payment method 3. Generate an API key from the console ### Configuration Steps Click the AI Provider dropdown and select **"Add a provider"**. Choose **Anthropic** from the Model Type dropdown. Paste your Anthropic API key in the Token field. ```text Example API Key Format sk-ant-api03-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx ``` Click Submit to verify and save the configuration. Available Claude models are fetched live from the Anthropic API and appear in the Model dropdown. See [Anthropic's pricing page](https://www.anthropic.com/pricing) for current models and rates. ### When to Use Claude Claude excels in these scenarios: - **Large Schemas**: Handles databases with hundreds of tables - **Complex Joins**: Better at understanding multi-table relationships - **Contextual Queries**: Superior at maintaining long conversation contexts - **Ambiguous Requests**: Better at asking clarifying questions ## Setting Up Google Gemini Gemini models offer strong SQL generation with competitive pricing. ### Prerequisites 1. Create a Google AI Studio account at https://aistudio.google.com/ 2. Generate an API key from the AI Studio console ### Configuration Steps Click the AI Provider dropdown and select **"Add a provider"**. Choose **Gemini** from the Model Type dropdown. Paste your Gemini API key in the Token field. Click Submit to verify and save the configuration. Available Gemini models are fetched live from the Gemini API and appear in the Model dropdown. See [Google's Gemini pricing page](https://ai.google.dev/pricing) for current models and rates. To route Gemini requests through a custom endpoint (e.g., a proxy), set the `WHODB_GEMINI_ENDPOINT` environment variable. ## Setting Up Ollama (Local Models) Ollama allows you to run AI models locally on your machine, providing complete privacy with zero API costs. ### Prerequisites Download and install Ollama from https://ollama.com **Supported Platforms**: - macOS (Apple Silicon and Intel) - Linux (x86_64, ARM64) - Windows (via WSL2) Open a terminal and download your preferred model: ```bash Popular Models # Recommended: Llama 3.1 (8B) ollama run llama3.1 # Alternative: CodeLlama (optimized for code) ollama run codellama # Alternative: Mistral (fast and efficient) ollama run mistral # Alternative: Llama 3.1 (70B - requires significant RAM) ollama run llama3.1:70b ``` The first `ollama run` command downloads the model (several GB). Subsequent runs use the cached model Check that Ollama is accessible: ```bash curl http://localhost:11434/api/tags ``` You should see a JSON response with available models. ### Configuring Ollama in WhoDB Navigate to the Chat page. Ollama is always listed in the AI Provider dropdown, whether or not it is currently running. Models only load when the Ollama service is reachable. Choose **Ollama** from the provider dropdown. No API key required—WhoDB connects to Ollama at localhost:11434 by default. If Ollama runs on a different host or port, set the `WHODB_OLLAMA_HOST` and `WHODB_OLLAMA_PORT` environment variables. Select your downloaded model from the AI Model dropdown. Available models are those you've downloaded via `ollama run`. ### Recommended Ollama Models for SQL | Model | Size | RAM Required | Best For | Download Command | |-------|------|-------------|----------|------------------| | Llama 3.1 (8B) | 4.7 GB | 8 GB | General use, good balance | `ollama run llama3.1` | | CodeLlama (7B) | 3.8 GB | 8 GB | Code/SQL generation | `ollama run codellama` | | Mistral (7B) | 4.1 GB | 8 GB | Fast responses | `ollama run mistral` | | Llama 3.1 (70B) | 40 GB | 64 GB | Maximum accuracy | `ollama run llama3.1:70b` | Start with Llama 3.1 (8B) for the best balance of performance and resource usage ### Ollama Performance Optimization **Minimum**: - CPU: 4 cores - RAM: 8 GB - Disk: 10 GB free **Recommended**: - CPU: 8+ cores - RAM: 16 GB+ - GPU: NVIDIA GPU with 8GB+ VRAM (optional but faster) - Disk: 50 GB free for multiple models Ollama automatically uses GPU if available: **NVIDIA GPU**: - Requires CUDA drivers - Dramatically faster inference - Supports larger models with less RAM **Apple Silicon (M1/M2/M3)**: - Native Metal acceleration - Excellent performance - No additional configuration needed Check GPU usage: ```bash ollama ps ``` Adjust model size based on your hardware: **If queries are slow**: - Use smaller models (7B instead of 70B) - Close other applications - Enable GPU acceleration - Consider using cloud providers for complex queries **If accuracy is poor**: - Upgrade to larger models (70B) - Provide more specific queries - Use cloud providers (OpenAI/Anthropic) for critical tasks ### Ollama Privacy Benefits Your database schema never leaves your machine Works in air-gapped or offline environments No API charges regardless of usage Choose models, control updates, customize behavior Ollama is ideal for regulated industries, sensitive data, or organizations requiring complete data sovereignty ## Setting Up LM Studio (Local Models) LM Studio is another local option with a desktop app for downloading and managing models. Download LM Studio from https://lmstudio.ai and load a model. Enable LM Studio's local server (default: `http://localhost:1234`). Like Ollama, LM Studio is always listed in the AI Provider dropdown. Select it and choose one of your loaded models. If LM Studio runs on a non-default endpoint, set `WHODB_LMSTUDIO_BASE_URL`. Use `WHODB_LMSTUDIO_API_KEY` if your server requires a key, and `WHODB_LMSTUDIO_NAME` to change the display name. ## Advanced Configuration ### Multiple Providers You can configure multiple AI providers and switch between them: Add OpenAI, Anthropic, and Ollama providers using the "Add a provider" option. Use the AI Provider dropdown to switch between configured providers at any time. **Strategy**: - Use a local model (Ollama, LM Studio) for exploration and learning - Use a fast, low-cost cloud model for quick production queries - Use your provider's most capable model for complex analytics This optimizes both cost and performance. ### Removing Providers On the Chat page, select a manually added provider and click the **Delete Provider** button. A confirmation dialog appears. Click **Delete** to remove the current provider configuration. This removes the API key and disconnects the provider. You'll need to reconfigure to use it again Providers configured by environment variables are managed through deployment configuration. Remove or change those providers by updating the deployment environment. As a security measure, a server-configured provider API key is never combined with a client-supplied endpoint — if a request overrides the endpoint, WhoDB uses the server-configured endpoint with the server key instead. ### Built-In Providers via Environment Variables You can preconfigure the built-in providers on the server so users don't need to enter API keys in the UI: | Variable | Description | |----------|-------------| | `WHODB_OPENAI_API_KEY` | OpenAI API key | | `WHODB_OPENAI_ENDPOINT` | Custom OpenAI-compatible endpoint (optional) | | `WHODB_OPENAI_NAME` | Display name override (optional) | | `WHODB_ANTHROPIC_API_KEY` | Anthropic API key | | `WHODB_ANTHROPIC_ENDPOINT` | Custom Anthropic endpoint (optional) | | `WHODB_ANTHROPIC_NAME` | Display name override (optional) | | `WHODB_GEMINI_ENDPOINT` | Custom Gemini endpoint (optional) | | `WHODB_OLLAMA_HOST` / `WHODB_OLLAMA_PORT` | Ollama host and port (default: localhost:11434) | | `WHODB_OLLAMA_NAME` | Display name override (optional) | | `WHODB_LMSTUDIO_BASE_URL` | LM Studio server URL (default: http://localhost:1234) | | `WHODB_LMSTUDIO_API_KEY` | LM Studio API key, if required (optional) | | `WHODB_LMSTUDIO_NAME` | Display name override (optional) | | `WHODB_BLOCK_INTERNAL_AI_ENDPOINTS` | Set to `true` to block AI requests to internal/private network endpoints (SSRF protection for hosted deployments) | ### Custom Providers via Environment Variables You can connect any OpenAI-compatible AI provider (LM Studio, OpenRouter, vLLM, etc.) using `WHODB_AI_GENERIC__*` environment variables. This is useful for Docker deployments, self-hosted models, or providers not built into the UI. Each provider needs a unique `` (e.g., `LMSTUDIO`, `OPENROUTER`) and up to six variables: | Variable | Required | Default | Description | |----------|----------|---------|-------------| | `WHODB_AI_GENERIC__NAME` | No | `` | Display name shown in the provider dropdown | | `WHODB_AI_GENERIC__TYPE` | No | `openai-generic` | Client type (leave default for OpenAI-compatible APIs) | | `WHODB_AI_GENERIC__BASE_URL` | Yes | | API base URL (e.g., `http://localhost:1234/v1`) | | `WHODB_AI_GENERIC__API_KEY` | No | | API key if required by the provider | | `WHODB_AI_GENERIC__MODELS` | Yes | | Comma-separated list of model names | | `WHODB_AI_GENERIC__ICON` | No | | URL of an icon to show in the provider dropdown | ```bash LM Studio (local) export WHODB_AI_GENERIC_LMSTUDIO_NAME="LM Studio" export WHODB_AI_GENERIC_LMSTUDIO_BASE_URL="http://localhost:1234/v1" export WHODB_AI_GENERIC_LMSTUDIO_MODELS="mistral-7b,llama-3-8b" ``` ```bash OpenRouter (cloud) export WHODB_AI_GENERIC_OPENROUTER_NAME="OpenRouter" export WHODB_AI_GENERIC_OPENROUTER_BASE_URL="https://openrouter.ai/api/v1" export WHODB_AI_GENERIC_OPENROUTER_API_KEY="your_key_here" export WHODB_AI_GENERIC_OPENROUTER_MODELS="google/gemini-2.0-flash-001,anthropic/claude-3.5-sonnet" ``` ```bash Requesty (cloud) export WHODB_AI_GENERIC_REQUESTY_NAME="Requesty" export WHODB_AI_GENERIC_REQUESTY_BASE_URL="https://router.requesty.ai/v1" export WHODB_AI_GENERIC_REQUESTY_API_KEY="your_key_here" export WHODB_AI_GENERIC_REQUESTY_MODELS="openai/gpt-4o-mini,anthropic/claude-3.5-sonnet" ``` ```yaml Docker Compose services: whodb: image: clidey/whodb:latest ports: - "8080:8080" environment: - WHODB_AI_GENERIC_LMSTUDIO_NAME=LM Studio - WHODB_AI_GENERIC_LMSTUDIO_BASE_URL=http://host.docker.internal:1234/v1 - WHODB_AI_GENERIC_LMSTUDIO_MODELS=mistral-7b,llama-3-8b ``` You can configure multiple generic providers at the same time — each `` creates a separate entry in the provider dropdown. The `BASE_URL` and `MODELS` variables are required; all others are optional. ## Provider Comparison Choose the right provider for your needs: | Feature | OpenAI | Anthropic | Ollama | |---------|--------|-----------|--------| | **Setup Complexity** | Easy | Easy | Moderate | | **Cost** | Pay per use | Pay per use | Free | | **Privacy** | External | External | Complete | | **Speed** | Fast | Medium | Varies | | **Accuracy** | Excellent | Excellent | Good | | **Internet Required** | Yes | Yes | No | | **Best For** | General use | Complex queries | Privacy/Cost | ## Troubleshooting **Symptom**: "Invalid API key" error when adding OpenAI provider **Solutions**: 1. Verify key is copied correctly (no extra spaces) 2. Check key hasn't been revoked in OpenAI dashboard 3. Ensure billing is set up on your OpenAI account 4. Try generating a new API key 5. Verify you're using a valid key format (starts with `sk-`) **Symptom**: Cannot connect to Anthropic after entering API key **Solutions**: 1. Verify API key from https://console.anthropic.com/ 2. Check your account has available credits 3. Ensure no network/firewall blocking claude.ai 4. Try a new API key 5. Check Anthropic service status **Symptom**: Ollama is selected but no models load or queries fail **Solutions**: 1. Verify Ollama is running: `curl http://localhost:11434/api/tags` 2. Restart Ollama service 3. Check Ollama is accessible on port 11434 (or set `WHODB_OLLAMA_HOST`/`WHODB_OLLAMA_PORT` for a custom endpoint) 4. Ensure no firewall blocking localhost connections 5. Verify at least one model is downloaded Test Ollama: ```bash ollama list # Shows downloaded models ollama ps # Shows running models ``` **Symptom**: Model dropdown is empty after adding provider **Solutions**: 1. Wait a few seconds for models to load 2. Refresh the page 3. Verify API key has proper permissions 4. Check provider dashboard for account status 5. Try removing and re-adding the provider For Ollama: ```bash ollama list # Verify models are downloaded ``` **Symptom**: Ollama queries take very long to respond **Solutions**: 1. Close resource-intensive applications 2. Use smaller models (7B instead of 70B) 3. Verify GPU acceleration is working: `ollama ps` 4. Increase system RAM allocation 5. Consider cloud providers for time-sensitive queries Check resource usage: ```bash # Monitor Ollama resource usage ollama ps top -p $(pgrep ollama) ``` **Symptom**: "Rate limit exceeded" from OpenAI or Anthropic **Solutions**: 1. Wait before retrying (limits reset after time window) 2. Upgrade your API plan for higher limits 3. Reduce query frequency 4. Check usage in provider dashboard 5. Consider switching to Ollama for unlimited queries ## Security Best Practices - Never share API keys publicly - Don't commit keys to git repositories - Use environment variables in production - Rotate keys regularly (every 90 days) - Monitor usage for unauthorized access **Highly Sensitive Data**: - Use Ollama exclusively - Never send to external providers **Moderately Sensitive Data**: - Review provider terms of service - Verify data handling policies - Consider data residency requirements **Public or Non-Sensitive Data**: - Any provider acceptable **OpenAI**: https://platform.openai.com/usage **Anthropic**: https://console.anthropic.com/settings/usage Set up billing alerts to prevent unexpected charges. When creating API keys: - Limit permissions to only what's needed - Create separate keys for development/production - Set spending limits where available - Enable key restrictions (IP allowlists, etc.) ## Next Steps Learn how to ask questions and retrieve data using your configured provider Step-by-step tutorial to get started with the AI assistant Master multi-turn conversations and context management Learn optimal patterns for using the AI assistant With your AI provider configured, you're ready to start querying your database using natural language --- ### Best Practices Access Control (`docs/best-practices/access-control.mdx`) --- title: Database Access Control description: Best practices for database access control, role-based access, read-only users, and security policies with WhoDB --- # Database Access Control Proper access control is fundamental to database security. This guide covers implementing role-based access, managing read-only users, establishing connection profiles, and enforcing security policies to protect your databases while enabling productive work. ## Access Control Fundamentals ### Principle of Least Privilege Grant users only the minimum permissions necessary to perform their job functions. **Least Privilege Benefits:** - Reduces blast radius of compromised credentials - Minimizes accidental data damage - Simplifies security audits - Improves compliance posture - Enables precise accountability tracking **Implementation Process:** 1. Document each user's actual job requirements 2. Grant read-only access initially 3. Expand permissions only when justified 4. Review periodically and remove unused access 5. Audit changes regularly **Dangerous Anti-Patterns:** - Sharing credentials between team members - Using administrative accounts for routine work - Granting broad permissions "just in case" - Not reviewing access after role changes - Maintaining access after team transitions ### Database User Categories Design your access control strategy around user categories. **Purpose**: Data exploration, reporting, analysis **Permissions**: - SELECT on specific schemas - View creation (read-only views) - Query execution only **Restrictions**: - No INSERT, UPDATE, DELETE - No DDL (CREATE, ALTER, DROP) - No administrative functions - Limited to development/reporting data **Example (PostgreSQL)**: ```sql CREATE ROLE analyst_readonly; GRANT CONNECT ON DATABASE analytics TO analyst_readonly; GRANT USAGE ON SCHEMA public TO analyst_readonly; GRANT SELECT ON ALL TABLES IN SCHEMA public TO analyst_readonly; ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO analyst_readonly; CREATE USER analyst_user WITH PASSWORD 'secure_password'; GRANT analyst_readonly TO analyst_user; ``` **Purpose**: Read and write application data in development/staging **Permissions**: - SELECT, INSERT, UPDATE on specific tables - View creation - Stored procedure execution - Limited schema modifications in dev only **Restrictions**: - No production write access - No administrative operations - No user management - No backup/restore operations **Example (PostgreSQL)**: ```sql CREATE ROLE developer_app; GRANT CONNECT ON DATABASE dev_db TO developer_app; GRANT USAGE ON SCHEMA public TO developer_app; GRANT SELECT, INSERT, UPDATE ON ALL TABLES IN SCHEMA public TO developer_app; GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO developer_app; ``` **Purpose**: ETL operations, data transformations, pipeline management **Permissions**: - Full access to staging schemas - SELECT on source tables - Full access to transformation tables - Scheduled job execution - Read-only access to production source data **Restrictions**: - No direct production data modifications - No user management - No security configuration changes - No backup operations **Example (PostgreSQL)**: ```sql CREATE ROLE data_engineer; GRANT CONNECT ON DATABASE analytics TO data_engineer; GRANT USAGE ON SCHEMA public, staging TO data_engineer; GRANT SELECT ON ALL TABLES IN SCHEMA public TO data_engineer; GRANT ALL ON ALL TABLES IN SCHEMA staging TO data_engineer; GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA staging TO data_engineer; ``` **Purpose**: System administration, maintenance, emergency operations **Permissions**: - Full database access - User and role management - Backup and recovery operations - Configuration management - Performance tuning and monitoring **Restrictions**: - MFA required - Limited to administrative tasks only - Comprehensive audit logging mandatory - Change approval process required - Time-limited elevated sessions **Purpose**: Automated processes, scheduled jobs, application runtime **Permissions**: - Only necessary for application function - Environment-specific (dev, staging, prod) - Limited to required tables and operations **Restrictions**: - No interactive use - No shared with developers - Rotated regularly - Monitored for suspicious activity - No elevated privileges ## Role-Based Access Control (RBAC) ### Designing Your RBAC Schema Establish a systematic approach to managing roles and permissions. **RBAC Structure Example:** ``` Database: production ├── Organization Role │ ├── department_sales_read │ ├── department_marketing_read │ └── department_finance_read ├── Function Role │ ├── analyst_read │ ├── developer_app │ ├── engineer_etl │ └── admin_full └── Data Role ├── can_access_customer_pii ├── can_access_financial ├── can_access_internal_only └── can_modify_production ``` **Naming Conventions:** ``` Format: [environment]_[function]_[permission] Examples: - dev_analyst_read (development, analytics, read-only) - prod_app_readwrite (production, application, read+write) - staging_engineer_full (staging, data engineering, full access) - prod_admin_emergency (production, admin, emergency access) ``` ### Role Hierarchy Organize roles hierarchically to simplify administration. **Hierarchy Example:** ``` admin_full ├── developer_app │ ├── analyst_read │ └── user_basic ├── engineer_etl │ ├── analyst_read │ └── user_basic └── support_tier1 └── user_basic ``` **Benefits:** - Inheriting permissions reduces duplication - Changes to parent roles propagate automatically - Clear permission hierarchy understood by teams - Easier onboarding and offboarding ### Implementing RBAC **Step 1: Create roles** ```sql CREATE ROLE analyst_read; CREATE ROLE developer_app; CREATE ROLE engineer_etl; CREATE ROLE admin_full; ``` **Step 2: Define permissions** ```sql -- Read-only analyst role GRANT CONNECT ON DATABASE myapp TO analyst_read; GRANT USAGE ON SCHEMA public TO analyst_read; GRANT SELECT ON ALL TABLES IN SCHEMA public TO analyst_read; -- Developer role (inherits analyst permissions + write) GRANT analyst_read TO developer_app; GRANT INSERT, UPDATE ON ALL TABLES IN SCHEMA public TO developer_app; ``` **Step 3: Create users and assign roles** ```sql CREATE USER alice WITH PASSWORD 'secure_password'; GRANT analyst_read TO alice; CREATE USER bob WITH PASSWORD 'secure_password'; GRANT developer_app TO bob; ``` **Step 4: Set default permissions** ```sql ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO analyst_read; ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE ON TABLES TO developer_app; ``` **Step 1: Create users with roles** ```sql CREATE USER 'analyst'@'%' IDENTIFIED BY 'secure_password'; CREATE USER 'developer'@'%' IDENTIFIED BY 'secure_password'; CREATE USER 'admin'@'%' IDENTIFIED BY 'secure_password'; ``` **Step 2: Grant role-based permissions** ```sql -- Analyst: read-only GRANT SELECT ON myapp.* TO 'analyst'@'%'; -- Developer: read and write GRANT SELECT, INSERT, UPDATE, DELETE ON myapp.* TO 'developer'@'%'; -- Admin: full access GRANT ALL PRIVILEGES ON myapp.* TO 'admin'@'%' WITH GRANT OPTION; ``` **Step 3: Apply changes** ```sql FLUSH PRIVILEGES; ``` **Step 4: Set default role** ```sql ALTER USER 'developer'@'%' DEFAULT ROLE 'developer_app'; ``` **Step 1: Create roles with specific privileges** ```javascript db.createRole({ role: "analyst_readonly", privileges: [ { resource: { db: "analytics", collection: "" }, actions: ["find", "listCollections"] } ], roles: [] }) ``` **Step 2: Create users with role assignment** ```javascript db.createUser({ user: "analyst_user", pwd: "secure_password", roles: [ { role: "analyst_readonly", db: "analytics" } ] }) ``` **Step 3: Verify role permissions** ```javascript db.getRole("analyst_readonly", { showPrivileges: true }) ``` ## Read-Only Access Management ### Creating Safe Read-Only Users Read-only users are essential for non-destructive database access. **Read-Only User Best Practices:** - Use for analysts, consultants, auditors - Default to read-only, expand only when justified - Combine with IP restrictions and time-based limits - Monitor for suspicious activity patterns - Rotate credentials regularly ### Read-Only Verification Verify that read-only access truly prevents modifications. Some database configurations can accidentally grant write access through views or functions. **Verification Queries:** ```sql -- PostgreSQL: Verify user has no write permissions SELECT grantee, privilege_type FROM information_schema.role_table_grants WHERE table_name='your_table' AND grantee='analyst_user'; -- Should return: SELECT only, not INSERT, UPDATE, DELETE ``` ```sql -- MySQL: Check user privileges SHOW GRANTS FOR 'analyst_user'@'%'; -- Verify only SELECT is granted ``` ```javascript -- MongoDB: List user roles and privileges db.getUser("analyst_user") // Verify role contains only read actions ``` ### View-Based Read-Only Access Use views to provide controlled, read-only access to specific data. **Sensitive Data Masking View:** ```sql -- Hide sensitive columns for read-only users CREATE VIEW customers_redacted AS SELECT id, name, city, country, '***' as email, -- Mask email '***' as phone, -- Mask phone FALSE as is_premium -- Hide business logic FROM customers; -- Grant read-only access to view only GRANT SELECT ON customers_redacted TO analyst_readonly; ``` **Time-Series Data View:** ```sql -- Provide read-only access to recent data only CREATE VIEW recent_transactions AS SELECT * FROM transactions WHERE created_at >= NOW() - INTERVAL '90 days'; -- Grant access to view, not underlying table GRANT SELECT ON recent_transactions TO analyst_readonly; REVOKE SELECT ON transactions FROM analyst_readonly; ``` ## Connection Profile Management ### Establishing Connection Profiles WhoDB connection profiles securely store database credentials and settings. **Connection Profile Best Practices:** - Use environment-specific profiles (dev, staging, prod) - Never hardcode credentials in configuration files - Rotate credentials regularly - Use least privilege database users - Enable SSL/TLS for all connections - Document connection purpose and usage ### Organizing Connection Profiles Use consistent naming for easy identification: Format: `[environment]-[database-type]-[region]-[purpose]` Examples: - `dev-postgres-us-east-analytics` - `prod-mysql-us-west-app` - `staging-postgres-eu-reporting` - `dev-mongodb-local-testing` Keep profiles organized by environment: ``` Development ├── dev-postgres-local ├── dev-mysql-docker └── dev-mongodb-local Staging ├── staging-postgres-rds ├── staging-mysql-rds └── staging-mongodb-atlas Production ├── prod-postgres-primary ├── prod-postgres-replica ├── prod-mysql-primary └── prod-mongodb-cluster ``` Use unambiguous profile names to prevent accidental production operations: - Prefix names with the environment: `prod-`, `staging-`, `dev-` - Include the access level where relevant: `prod-postgres-readonly` - Always verify the profile name before executing modifications - Train the team on the naming convention Configure security parameters in each profile: ``` Host: db.example.com Port: 5432 Database: myapp Username: app_user SSL Mode: require (or verify-full) Timeout: 30 seconds Pool Size: 10 Notes: - Uses IAM authentication with assumed role - Read replica: only for reporting queries - Backup location: us-east-1a - VPN required: Yes ``` ### Credential Rotation Regular credential rotation is essential for security. Credentials should be rotated at least quarterly, immediately after employee departure, and after any suspected breach. **Credential Rotation Process:** ``` Step 1: Prepare new credentials ├── Generate new password (16+ chars, mixed case, symbols) ├── Test in development environment first └── Verify old credentials still work (for rollback) Step 2: Update database ├── Create new user with same permissions ├── Verify new user can connect └── Keep old user active temporarily Step 3: Update applications ├── Update connection string in WhoDB ├── Verify all services connect successfully ├── Monitor logs for connection errors Step 4: Verify and cleanup ├── Confirm all services use new credentials ├── Run security audit ├── Remove old credentials └── Document rotation date and approver ``` **Rotation Schedule:** ``` Production Databases: Quarterly + immediate after separation Staging Databases: Semi-annually + after breach suspicion Development Databases: Annually + after onboarding Service Accounts: Quarterly + after vulnerability scan ``` ## Security Policies ### Data Classification Classify data by sensitivity to guide access control decisions. **Data Classification Levels:** | Level | Examples | Access | Encryption | Auditing | |-------|----------|--------|------------|----------| | **Public** | Product catalog, public documentation | Everyone | Optional | Optional | | **Internal** | Sales reports, team info | Employees | Recommended | Recommended | | **Sensitive** | Customer data, health records | Department specific | Required | Required | | **Restricted** | Passwords, API keys, PII | Minimal, need-based | Required | Required | **Classification Process:** 1. Audit all tables and columns 2. Document sensitivity level 3. Define access restrictions 4. Implement technical controls 5. Review and update annually **Example Classification:** ``` Table: users ├── id: PUBLIC ├── name: INTERNAL ├── email: SENSITIVE (PII) ├── password_hash: RESTRICTED └── ssn: RESTRICTED (highly sensitive) ``` ### Implementing Access Policies Restrict access to sensitive columns: ```sql -- PostgreSQL: Create role without access to sensitive columns CREATE ROLE analyst_no_pii; GRANT SELECT (id, name, city, country) ON customers TO analyst_no_pii; -- Not SELECT *, only specified columns -- Excludes email, phone, ssn ``` PostgreSQL Row-Level Security (RLS): ```sql -- Enable RLS on table ALTER TABLE customers ENABLE ROW LEVEL SECURITY; -- Analysts can only see customers in their region CREATE POLICY analyst_access_by_region ON customers FOR SELECT USING (region = current_setting('app.user_region')); -- Set user's region SET app.user_region = 'NORTH_AMERICA'; ``` Restrict access by time: ```sql -- PostgreSQL: Create role valid only during business hours CREATE ROLE developer_business_hours; -- Check access time before granting connection CREATE OR REPLACE FUNCTION check_business_hours() RETURNS BOOLEAN AS $$ BEGIN RETURN EXTRACT(HOUR FROM CURRENT_TIME) BETWEEN 9 AND 18 AND EXTRACT(DOW FROM CURRENT_DATE) BETWEEN 1 AND 5; END; $$ LANGUAGE plpgsql; ``` Restrict connections by IP: ``` Connection Profile: prod-postgres-primary ├── Host: db.prod.example.com ├── Allowed IPs: │ ├── 10.0.1.0/24 (office network) │ ├── 10.0.2.0/24 (VPN network) │ └── 203.0.113.45/32 (admin home) └── Blocked IPs: 0.0.0.0/0 (deny all by default) ``` ### Audit Logging Enable comprehensive audit logging for all database access. Audit logs are critical for security investigations, compliance audits, and incident response. **What to Log:** - User login/logout events - Query execution (SELECT, INSERT, UPDATE, DELETE) - Schema modifications - Security policy changes - Failed authentication attempts - Administrative operations **PostgreSQL Audit Configuration:** ```sql -- Install pgAudit extension CREATE EXTENSION pgaudit; -- Log all write operations ALTER SYSTEM SET pgaudit.log = 'write, ddl'; -- Log which tables accessed ALTER SYSTEM SET pgaudit.log_relation = on; -- Log statement details ALTER SYSTEM SET pgaudit.log_statement = off; ALTER SYSTEM SET pgaudit.log_statement_once = off; -- Reload configuration SELECT pg_reload_conf(); ``` **Audit Log Review Process:** ``` Daily: - Check for failed authentication attempts - Review administrative operations - Monitor unusual access patterns Weekly: - Analyze access by user and role - Identify overprivileged accounts - Review data exports Monthly: - Comprehensive access review - Compliance verification - Detection of suspicious patterns Quarterly: - Formal access audit - Recertification of access - Policy effectiveness review ``` ## Managing Access Lifecycle ### User Onboarding - [ ] Database account created - [ ] Appropriate role assigned - [ ] Temporary password issued securely - [ ] Connection profile provided - [ ] Security policy overview provided - [ ] WhoDB access granted - [ ] User confirmed connection works - [ ] Permissions verified correct - [ ] Training completed on security policies - [ ] First queries documented - [ ] Questions answered - [ ] Permissions still appropriate for role - [ ] Access usage review - [ ] Feedback on onboarding process - [ ] Additional training needs identified - [ ] Permanent password set ### Access Reviews **Process:** 1. Generate user list with assigned roles 2. Send to department managers for verification 3. Collect feedback on access appropriateness 4. Identify and remove unnecessary access 5. Document review and approvals 6. Archive for compliance **Template:** ``` User: alice@example.com Current Role: analyst_readonly Department: Sales Manager: Bob Johnson Questions: - Does this user still need database access? YES / NO - Is the role appropriate? YES / NO / NEEDS_UPGRADE / NEEDS_DOWNGRADE - Any concerns? ___________________ Recommendation: APPROVE / REVOKE / MODIFY ``` When user changes roles: - [ ] Remove old role permissions - [ ] Document date of change - [ ] Assign new role permissions - [ ] Verify appropriate access - [ ] Document business justification - [ ] Notify security team Inactive accounts should be disabled: ``` Audit Query (PostgreSQL): SELECT usename, valuntil, last_login FROM pg_user LEFT JOIN pg_stat_user_tables ON 1=1 WHERE last_login < NOW() - INTERVAL '90 days' OR last_login IS NULL; Action: - Disable account after 90 days inactivity - Archive after 1 year - Delete after 2 years (per retention policy) ``` ### User Offboarding Prompt offboarding is critical when employees leave. Delayed credential removal represents a significant security risk. **Offboarding Checklist:** ``` Effective Date: [departure date] Employee: [name] Immediate (Day 0): [ ] Disable database user account [ ] Revoke all role memberships [ ] Disable SSH keys if applicable [ ] Remove VPN access [ ] Notify security team Within 24 Hours: [ ] Confirm account is disabled [ ] Check for running queries/sessions [ ] Audit recent query history [ ] Document access used over final period Within 7 Days: [ ] Archive credentials securely [ ] Document any outstanding work [ ] Transfer owned queries to team [ ] Update access documentation [ ] File security incident if needed ``` ## Access Control Checklist **Initial Setup:** - [ ] Database roles defined for each user type - [ ] Least privilege principle implemented - [ ] Read-only users created for analysts - [ ] Connection profiles configured securely - [ ] SSL/TLS enabled for all connections **Role Management:** - [ ] RBAC hierarchy established - [ ] Default permissions set - [ ] New table permissions automated - [ ] Role documentation complete - [ ] Service accounts use least privilege **Security Policies:** - [ ] Data classified by sensitivity - [ ] Column-level security implemented - [ ] Row-level security configured - [ ] IP restrictions enforced - [ ] Audit logging enabled **Ongoing Maintenance:** - [ ] Quarterly access reviews scheduled - [ ] Credential rotation calendar maintained - [ ] Dormant accounts identified monthly - [ ] Audit logs reviewed regularly - [ ] Policy violations investigated **Incident Response:** - [ ] Breach response procedures documented - [ ] Escalation path defined - [ ] Audit log preservation process established - [ ] Forensic analysis capabilities in place - [ ] Communication template prepared ## Summary Robust access control requires careful planning, consistent implementation, and ongoing maintenance. Use database roles, dedicated credentials, network controls, and database-native audit logs to protect your data while keeping WhoDB useful for daily inspection and query workflows. Access control is not a one-time setup; review it as team members, environments, and threats change. --- ### Best Practices Ai Usage (`docs/best-practices/ai-usage.mdx`) --- title: AI Chat Assistant Best Practices description: Comprehensive guidance for optimal and safe use of WhoDB's AI Chat Assistant --- # AI Chat Assistant Best Practices The AI Chat Assistant transforms database interaction from technical SQL writing to natural conversation. This guide is the canonical reference for using the AI assistant effectively, safely, and efficiently in production environments. Effective AI assistant usage combines clear communication, security awareness, and strategic provider selection ## Understanding AI-Powered Database Interaction The AI Chat Assistant is fundamentally different from traditional database tools. Rather than writing SQL directly, you describe what you want in natural language, and the AI generates appropriate queries based on your database schema. ### How AI Assistants Work The AI assistant analyzes your complete database schema, including tables, columns, data types, and relationships. Your question is processed to understand intent, entities, conditions, and desired operations. Based on schema and intent, the AI generates database-specific SQL optimized for your database type. WhoDB executes the query and presents results, which become part of conversation context. ### Key Differences from Traditional SQL | Aspect | Traditional SQL | AI Assistant | |--------|----------------|-------------| | **Input Method** | Write exact syntax | Describe desired outcome | | **Schema Knowledge** | Must memorize or reference | Automatically aware | | **Error Handling** | Syntax errors require fixes | Rephrase in natural language | | **Learning Curve** | Steep for beginners | Accessible immediately | | **Precision** | Exact control | Interpretation required | | **Speed** | Fast for experts | Fast for everyone | ## Query Formulation Best Practices Effective communication with the AI assistant follows specific patterns that produce accurate, efficient results. ### Be Specific and Explicit Vague questions produce unreliable results. Specificity ensures the AI understands your exact intent. **Good Examples:** ```text Show me all records from the users table Count orders in the orders table Display products from the inventory.products table ``` **Avoid:** ```text Show me the data Get everything ``` When table names might be ambiguous, include schema names: `test_schema.users` **Good Examples:** ```text Show user_id, email, and created_at from users Display product names and prices ``` **Avoid:** ```text Show some user information Get order stuff ``` Explicit column names help the AI generate precise SELECT statements. **Good Examples:** ```text Show orders from the last 7 days Display users created after 2024-01-01 ``` **Avoid:** ```text Show recent orders Find old logs ``` Use specific dates or clear relative ranges (last 7 days, this month, last year). **Good Examples:** ```text Show users where status is active and email_verified is true Find products where price is greater than 100 and stock is less than 10 ``` **Avoid:** ```text Show active users Find expensive products ``` Explicitly state field names, comparison operators, and values. ### Provide Context Context helps the AI understand your intent and generate more accurate queries. **Include Business Context:** ```text Show revenue by product category for the last quarter (for quarterly report) Find users who haven't logged in for 90 days (for cleanup campaign) ``` **Mention Expected Results:** ```text Show all orders (expecting about 1000 records) Count active subscriptions (should be around 500) ``` Expected results help you quickly identify when queries return unexpected data. ### Use Proper Database Terminology Use terminology appropriate to your database type: tables, rows, columns, JOINs, and WHERE clauses for SQL databases; collections, documents, fields, and aggregation pipelines for MongoDB; keys, values, sets, and hashes for Redis. **SQL Example:** ```text Join the orders table with customers table on customer_id and show customer names with their order totals ``` **MongoDB Example:** ```text Aggregate users collection grouped by email domain with count ``` ### Start Simple, Then Refine Build complex queries through iterative refinement rather than trying to get everything perfect in one question: ```text 1. Show me all orders 2. Just orders from the last 30 days 3. Group those by customer 4. Show total order value for each customer 5. Sort by total value descending ``` The AI understands each question refines the previous one. This iterative approach is faster and more reliable than trying to construct complex queries in a single request. ## Safety and Security Best Practices Using AI assistants safely requires understanding what data is shared, potential risks, and protective measures. ### Understand Data Sharing Different AI providers have different data handling policies. Your database schema structure and query text are sent to AI providers. However, actual data values and query results are not transmitted. **What Gets Sent to AI Providers:** - Your natural language questions - Database table names and schemas - Column names and data types - Database type (PostgreSQL, MySQL, etc.) - Previous conversation context **What Does NOT Get Sent:** - Actual row data from your database - Query result contents - Stored data values - Connection credentials **For Maximum Privacy:** - Use local models (Ollama, LM Studio) for complete data isolation - Avoid mentioning sensitive values in questions - Use generic terms instead of revealing schema names ### Verify Before Modifying Data Always review and verify before confirming data modification operations. Before clicking Confirm on any INSERT, UPDATE, DELETE, CREATE, ALTER, or DROP operation, click **Show Query** and check: - [ ] Correct table is targeted - [ ] WHERE clause is present and accurate - [ ] Values are correct and properly formatted - [ ] No unintended side effects Always verify which records will be affected before modifying them. **Step 1 - Verify:** ```text Show me all users where last_login is before 2020-01-01 ``` Review the results carefully. Count the records. Verify these are the records you want to modify. **Step 2 - Modify:** ```text Delete all users where last_login is before 2020-01-01 ``` This two-step approach prevents accidental data loss. For critical operations, test the query in a development environment before running in production: 1. Connect to development database 2. Ask the AI assistant to generate the query 3. Review the generated SQL, execute, and verify results 4. Copy the verified SQL to production (via Scratchpad) 5. Execute in production during an appropriate window ### Use Read-Only Users When Possible For data exploration and analysis tasks, connect with read-only database credentials. **PostgreSQL Read-Only User:** ```sql CREATE ROLE readonly_user WITH LOGIN PASSWORD 'secure_password'; GRANT CONNECT ON DATABASE mydb TO readonly_user; GRANT USAGE ON SCHEMA public TO readonly_user; GRANT SELECT ON ALL TABLES IN SCHEMA public TO readonly_user; ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO readonly_user; ``` **MySQL Read-Only User:** ```sql CREATE USER 'readonly_user'@'%' IDENTIFIED BY 'secure_password'; GRANT SELECT ON mydb.* TO 'readonly_user'@'%'; FLUSH PRIVILEGES; ``` Confirmation prompts are triggered by the SQL operation type, not by database permissions. With a read-only user you still see the confirmation for a modification request; the database rejects the statement if you confirm it. Use read-only credentials for most of your database work. Only use write credentials when actually modifying data. ### Backup Before Bulk Operations Before executing bulk modifications, ensure a recent, tested backup exists that includes all affected tables, and that the restoration procedure is documented. **Quick Backup Commands:** PostgreSQL: ```bash pg_dump -h localhost -U username -d database -t table_name > backup_$(date +%Y%m%d_%H%M%S).sql ``` MySQL: ```bash mysqldump -h localhost -u username -p database table_name > backup_$(date +%Y%m%d_%H%M%S).sql ``` ### Review Generated SQL The AI assistant shows generated SQL before execution. Use this visibility to verify correctness: - **SELECT**: correct tables, appropriate JOIN conditions, WHERE filters match intent, no expensive functions on large tables - **UPDATE**: WHERE clause present and correct, SET values appropriate - **DELETE**: WHERE clause present (unless intentionally deleting all), correct table, backup exists - **INSERT**: all required columns included, values match column data types ## Provider Selection Strategy Choosing the right AI provider for each situation optimizes cost, performance, privacy, and accuracy. ### When to Use Each Provider Model lists are fetched live from each provider when you add it, so pick from your provider's current lineup. For current model options and rates, check your provider's pricing page. **Best For:** - General-purpose queries across all database types - Fast response requirements - Users without local AI infrastructure **Cost Optimization:** - Use a smaller, cheaper model for simple queries - Reserve the most capable model for complex multi-table operations - See [OpenAI's pricing page](https://openai.com/api/pricing/) for current rates **Best For:** - Complex analytical queries - Large database schemas (100+ tables) - Long conversation contexts - Sophisticated reasoning requirements **Model Selection:** Choose from the models fetched from the Anthropic API — smaller models for speed and cost, larger models for the most complex scenarios. See [Anthropic's pricing page](https://www.anthropic.com/pricing) for current models and rates. **Best For:** - Google Cloud users - General-purpose queries with competitive pricing **Model Selection:** Choose from the models fetched from the Gemini API. See [Google's Gemini pricing page](https://ai.google.dev/pricing) for current models and rates. **Best For:** - Privacy-sensitive environments and regulated industries - Air-gapped or offline deployments - Unlimited usage without API costs **Model Selection:** Use whatever models you have downloaded locally. Larger models are more accurate but need more RAM; smaller models respond faster on modest hardware. **Trade-offs:** - Typically slower than cloud providers - Smaller local models may be less accurate than the top cloud models - Complete privacy and zero ongoing costs ### Cost Optimization Strategies Your provider's smaller models handle simple queries well at a fraction of the cost Long conversations consume more tokens. Start fresh when switching topics Shorter questions and responses reduce token usage and costs Local models (Ollama, LM Studio) eliminate per-query costs for high-volume usage Per-token pricing changes frequently — compare current rates on your providers' pricing pages rather than relying on fixed estimates. ### Privacy Considerations Choose providers based on data sensitivity and organizational policies. **Highly Sensitive** (PHI, financial records, trade secrets, PII): use local models exclusively. **Moderately Sensitive** (internal business data, non-PII customer information): review provider terms of service, consider local models or ensure provider compliance. **Public or Non-Sensitive** (public datasets, demonstration databases): any provider acceptable. Consult your information security, data governance, legal/compliance, and privacy teams. Verify alignment with data residency requirements, third-party data processing policies, and regulations such as GDPR, HIPAA, and PCI DSS. For external providers: document approved use cases, train users on privacy boundaries, and establish escalation procedures. For local models: document installation and configuration, maintain model versions, and plan capacity for user growth. Schema names and table names are sent to external AI providers. Avoid using sensitive or revealing names if privacy is critical. ## Performance Optimization Optimize AI assistant performance through efficient query patterns and conversation management. ### Efficient Query Patterns Request aggregated data rather than retrieving all rows. **Efficient:** ```text Count users by email domain Show average order value by month ``` **Inefficient:** ```text Show me all users (then manually count by domain) ``` Let the database perform aggregations rather than retrieving large datasets. Request only necessary data for large tables. **Efficient:** ```text Show top 100 users by registration date Display 50 most recent orders ``` **Inefficient:** ```text Show me all users (millions of rows) ``` Use LIMIT clauses for exploratory queries on large tables. Structure queries to leverage existing indexes. **Index-Friendly:** ```text Find users where user_id equals 12345 Show orders where order_date is 2024-01-15 ``` **Index-Inefficient:** ```text Find users where email contains gmail Show orders where YEAR(order_date) equals 2024 ``` Ask about indexed columns: "What indexes exist on the orders table?" Ensure JOINs have proper conditions. **Efficient:** ```text Join orders with customers on customer_id and show customer names with order totals ``` The AI generally avoids this, but verify JOIN conditions in the generated SQL. ### Managing Conversation Context Long conversations accumulate context that slows response times and increases costs. Short conversations (1-10 messages) are fastest and cheapest; conversations beyond 30-50 messages become noticeably slower and more expensive. **When to Start New Conversations:** When switching to a completely different database area or analysis focus, click "New Chat" and start fresh. When queries take noticeably longer to respond, save important queries to Scratchpad, then start a new chat. When the AI references incorrect previous context or misunderstands follow-up questions, start a new conversation with clear, explicit questions. Every model has a context limit, and quality degrades as you approach it. Limits vary widely by model — check your provider's documentation, and start fresh proactively. Move important queries to Scratchpad before starting a new chat "Show top 10" instead of "Can you please show me the top 10 results" ## Collaboration and Documentation Preserve valuable queries and build shared knowledge with a lightweight process. When a query is worth reusing — regular reports, complex analysis you'll repeat, or queries that revealed useful insights — click **Move to Scratchpad** below the result. Choose or create an appropriately named page, then add a comment explaining what the query does, when to use it, and any caveats. Organize Scratchpad pages by purpose (reporting, analysis templates, data quality checks, maintenance) so queries are easy to find later. For team sharing, copy important SQL into your normal shared docs, runbooks, or repository. Finally, well-documented schemas help the AI generate more accurate queries. Add database-level comments to tables and non-obvious columns: ```sql COMMENT ON TABLE users IS 'Customer user accounts with authentication'; COMMENT ON COLUMN users.last_login IS 'UTC timestamp of most recent successful login'; ``` ## Error Handling and Recovery Understand common mistakes and recovery strategies. ### Common Mistakes to Avoid **Mistake:** Executing a bulk DELETE directly in production without verification. **Prevention:** 1. Test in development database first 2. Use SELECT to verify before DELETE 3. Ensure backup exists **Recovery:** If executed accidentally, restore from the most recent backup. **Mistake:** "Update all users to set role to admin" — no WHERE clause, so all records are affected. **Prevention:** - Always specify which records to modify - Review the confirmation prompt carefully before confirming - Use SELECT first to verify target records **Recovery:** If a backup exists, restore the affected table. Otherwise, manually identify and correct affected records. **Mistake:** "Show me the data" — too vague, the AI must guess intent. **Better:** ```text Show user_id, email, and created_at from users table where created_at is in the last 7 days ``` **Mistake:** Clicking Confirm without reviewing the SQL in the confirmation prompt. **Prevention:** Always read the generated SQL before confirming. Verify table names, WHERE clauses, and values. Cancel and rephrase if anything looks wrong. **Mistake:** Continuing conversations for 50+ messages, assuming the AI remembers early context accurately. **Prevention:** Start new conversations when switching topics, be explicit in follow-up questions, and watch for signs of context confusion. ### Troubleshooting Approach When queries don't work as expected, follow a systematic troubleshooting process. What did you ask for? What did you actually receive? What error message appeared? Does the generated SQL match your intent? Click **SQL** below the result to see the generated SQL. Check table and column names, WHERE conditions, JOIN conditions, and aggregations against your intent. If the query is complex, break it into smaller parts: ```text 1. Show me all orders from the last quarter 2. Now filter those to only California customers 3. Join with products to get categories 4. Calculate total revenue by category and month ``` Incremental refinement is more reliable than complex single questions. Ensure the data you're querying actually exists and names match your schema: ```text Count records in the users table Show me a sample of 5 records from orders What columns exist in the products table ``` Empty result sets might indicate an empty table, overly restrictive filters, a wrong table, or data in a different schema. If the AI misunderstood, rephrase with more explicit details: **Vague:** "Show user data" **Explicit:** "Show user_id, email, first_name, last_name, and created_at from the users table for all active users" ### Recovery Strategies For incorrect UPDATE operations, run a compensating query to restore values For significant data loss, restore affected tables from a recent backup If using Scratchpad with transactions, ROLLBACK before COMMIT For small-scale errors, manually correct affected records ## Production Environment Guidelines Using AI assistants in production requires additional discipline and procedures. ### Testing Queries Never execute untested queries directly in production. Connect to a development or staging database first. Generate and test queries in the non-production environment. Execute the generated query and verify it returns expected data, performance is acceptable, and there are no unintended side effects. Assess how many rows will be affected, whether this locks tables, and what the rollback plan is. Copy the verified SQL from Scratchpad, execute during an appropriate window, monitor execution, and verify results immediately. ### Change Management Follow established change management procedures for data modifications. **Pre-Change Checklist:** - [ ] Change request documented and approved - [ ] Testing completed in non-production - [ ] Backup verified and accessible - [ ] Rollback procedure documented - [ ] Team members notified **During and After:** - [ ] Execute during scheduled window and monitor progress - [ ] Verify results match expectations - [ ] Update documentation and notify stakeholders ### Audit Requirements Maintain audit trails for compliance and troubleshooting. **Audit Logging in WhoDB Community Edition:** Use database-level audit logging to record executed queries. Per-query audit trails with user attribution are available in the Enterprise Edition. **Additional Audit Measures:** - Enable database-level query logging - Review audit logs regularly and retain them per compliance requirements - Consider regulatory obligations: GDPR (personal data access), HIPAA (PHI access), SOX (financial data), PCI DSS (cardholder data) ## Learning and Improvement The AI assistant is also an effective SQL tutor. Ask it to explain generated queries ("Why did you use LEFT JOIN instead of INNER JOIN?"), request alternative approaches ("Is there a more efficient way to write this query?"), and explore features in context ("How do window functions work in PostgreSQL?"). Build skills progressively: start with simple SELECT queries, then add WHERE conditions, aggregations, JOINs, and eventually subqueries and window functions. Study the SQL the AI generates — its JOIN structure, aliasing, and date-filtering patterns — rather than using queries blindly. Comparing AI-generated SQL against queries you write manually in Scratchpad is a fast way to discover techniques you might not have considered. ## Next Steps Review AI Chat Assistant capabilities and features Configure OpenAI, Anthropic, Gemini, Ollama, or LM Studio for your needs Learn effective techniques for data retrieval Understand safe data modification with AI assistance Master multi-turn conversations and context management Comprehensive security practices for database management Combine AI assistance with human judgment for optimal database management—the AI generates queries efficiently, and you verify they're correct before execution --- --- METRICS --- - Files Extracted: 54 - Estimated Token Budget: ~130373 tokens - Canonical Reference: https://codewiki.google/github.com/clidey/whodb