{"owner":"tobi","repo":"qmd","hasSkills":true,"totalSkillsCount":6,"totalTokensCount":9617,"categories":["marketplace","anthropic-skill","claude-rule"],"hasMcp":true,"mcpConfig":{"mcpServers":{"qmd":{"command":"npx","args":["-y","@modelcontextprotocol/server-qmd"]}}},"found":[".claude-plugin/marketplace.json","skills/release/SKILL.md","CLAUDE.md","skills/qmd/SKILL.md","skills/qmd/references/mcp-setup.md","finetune/CLAUDE.md"],"skills":{".claude-plugin/marketplace.json":"{\n\t\"name\": \"qmd\",\n\t\"owner\": {\n\t\t\"name\": \"tobi\",\n\t\t\"email\": \"tobi@lutke.com\"\n\t},\n\t\"plugins\": [\n\t\t{\n\t\t\t\"name\": \"qmd\",\n\t\t\t\"source\": \"./skills\",\n\t\t\t\"description\": \"Search and retrieve documents from local markdown files.\",\n\t\t\t\"version\": \"2.8.3\",\n\t\t\t\"author\": {\n\t\t\t\t\"name\": \"tobi\",\n\t\t\t\t\"email\": \"tobi@lutke.com\"\n\t\t\t},\n\t\t\t\"repository\": \"https://github.com/tobi/qmd\",\n\t\t\t\"license\": \"MIT\",\n\t\t\t\"keywords\": [\"markdown\", \"search\", \"qmd\"],\n\t\t\t\"skills\": [\"./qmd\", \"./release\"],\n\t\t\t\"mcpServers\": {\n\t\t\t\t\"qmd\": {\n\t\t\t\t\t\"command\": \"qmd\",\n\t\t\t\t\t\"args\": [\"mcp\"]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t]\n}\n","skills/release/SKILL.md":"---\nname: release\ndescription: Manage releases for this project. Validates changelog, installs git hooks, and cuts releases. Use when user says \"/release\", \"release 1.0.5\", \"cut a release\", or asks about the release process. NOT auto-invoked by the model.\ndisable-model-invocation: true\n---\n\n# Release\n\nCut a release, validate the changelog, and ensure git hooks are installed.\n\n## Usage\n\n`/release 1.0.5` or `/release patch` (bumps patch from current version).\n\n## Process\n\nWhen the user triggers `/release <version>`:\n\n1. **Gather context** — run `skills/release/scripts/release-context.sh <version>`.\n   This silently installs git hooks and prints everything needed: version info,\n   working directory status, commits since last release, files changed, current\n   `[Unreleased]` content, and the previous release entry for style reference.\n\n2. **Commit outstanding work** — if the context shows staged, modified, or\n   untracked files that belong in this release, commit them first. Use the\n   /commit skill or make well-formed commits directly.\n\n3. **Write the changelog** — if `[Unreleased]` is empty, write it now using\n   the commits and file changes from the context output. Follow the changelog\n   standard below. Re-run the context script after committing if needed.\n\n4. **Check dependency updates** — before cutting the release, check for\n   updates to `sqlite-vec` (and platform packages), `node-llama-cpp`,\n   and `better-sqlite3`. Run `pnpm outdated` and report any available\n   updates for these packages. If updates exist, bump them (pinned, no\n   `^` ranges) and re-run tests before proceeding.\n\n5. **Cut the release** — run `scripts/release.sh <version>`. This renames\n   `[Unreleased]` → `[X.Y.Z] - date`, inserts a fresh `[Unreleased]`,\n   bumps `package.json` and the plugin version in\n   `.claude-plugin/marketplace.json` (so installed plugins see the update),\n   commits, and tags.\n\n6. **Show the final changelog** — print the full `[Unreleased]` +\n   minor series rollup via `scripts/extract-changelog.sh <version>`.\n   Ask the user to confirm before pushing.\n\n7. **Push** — after explicit confirmation, run `git push origin main --tags`.\n\n8. **Watch CI** — after the push, start a background dispatch to watch the\n   publish workflow. Use `interactive_shell` in dispatch mode with:\n   ```\n   gh run watch $(gh run list --workflow=publish.yml --limit=1 --json databaseId --jq '.[0].databaseId') --exit-status\n   ```\n   The agent will be notified when CI completes and should report the result.\n\nIf any step fails, stop and explain. Never force-push or skip validation.\n\n## Dependency Policy\n\nAll dependencies must be pinned to exact versions (no `^` or `~` ranges).\nThe lockfile ensures reproducible installs. When adding or updating any\ndependency, always use the exact version string (e.g. `\"3.18.1\"` not\n`\"^3.18.1\"`).\n\n## Changelog Standard\n\nThe changelog lives in `CHANGELOG.md` and follows [Keep a Changelog](https://keepachangelog.com/) conventions.\n\n### Heading format\n\n- `## [Unreleased]` — accumulates entries between releases\n- `## [X.Y.Z] - YYYY-MM-DD` — released versions\n\n### Structure of a release entry\n\nEach version entry has two parts:\n\n**1. Highlights (optional, 1-4 sentences of prose)**\n\nImmediately after the version heading, before any `###` section. The elevator\npitch — what would you tell someone in 30 seconds? Only for significant\nreleases; skip for small patches.\n\n```markdown\n## [1.1.0] - 2026-03-01\n\nQMD now runs on both Node.js and Bun, with up to 2.7x faster reranking\nthrough parallel contexts. GPU auto-detection replaces the unreliable\n`gpu: \"auto\"` with explicit CUDA/Metal/Vulkan probing.\n```\n\n**2. Detailed changelog (`### Changes` and `### Fixes`)**\n\n```markdown\n### Changes\n\n- Runtime: support Node.js (>=22) alongside Bun. The `qmd` wrapper\n  auto-detects a suitable install via PATH. #149 (thanks @igrigorik)\n- Performance: parallel embedding & reranking — up to 2.7x faster on\n  multi-core machines.\n\n### Fixes\n\n- Prevent VRAM waste from duplicate context creation during concurrent\n  `embedBatch` calls. #152 (thanks @jkrems)\n```\n\n### Writing guidelines\n\n- **Explain the why, not just the what.** The changelog is for users.\n- **Include numbers.** \"2.7x faster\", \"17x less memory\".\n- **Group by theme, not by file.** \"Performance\" not \"Changes to llm.ts\".\n- **Don't list every commit.** Aggregate related changes.\n- **Credit contributors:** end bullets with `#NNN (thanks @username)` for\n  external PRs. No need to credit the repo owner.\n\n### What not to include\n\n- Internal refactors with no user-visible effect\n- Dependency bumps (unless fixing a user-facing bug)\n- CI/tooling changes (unless affecting the release artifact)\n- Test additions (unless validating a fix worth mentioning)\n\n## GitHub Release Notes\n\nEach GitHub release includes the full changelog for the **minor series** back\nto x.x.0. The `scripts/extract-changelog.sh` script handles this, and the\npublish workflow (`publish.yml`) calls it to populate the GitHub release.\n\n## Git Hooks\n\nThe pre-push hook (`scripts/pre-push`) blocks `v*` tag pushes unless:\n\n1. `package.json` version matches the tag\n2. `CHANGELOG.md` has a `## [X.Y.Z] - date` entry for the version\n3. CI passed on GitHub (warns in non-interactive shells, blocks in terminals)\n\nHooks are installed silently by the context script. They can also be installed\nmanually via `skills/release/scripts/install-hooks.sh` or automatically via\n`bun install` (prepare script).\n","CLAUDE.md":"# QMD - Query Markup Documents\n\nUse Bun instead of Node.js (`bun` not `node`, `bun install` not `npm install`).\n\n## Commands\n\n```sh\nqmd collection add . --name <n>   # Create/index collection\nqmd collection list               # List all collections with details\nqmd collection remove <name>      # Remove a collection by name\nqmd collection rename <old> <new> # Rename a collection\nqmd collection metadata [name...] # Discover metadata keys, types, and value counts (--match, --filter, --key-limit, --value-limit)\nqmd init                          # Create a project-local .qmd index\nqmd ls [collection[/path]]        # List collections or files in a collection\nqmd context add [path] \"text\"     # Add context for path (defaults to current dir)\nqmd context list                  # List all contexts\nqmd context check                 # Check for collections/paths missing context\nqmd context rm <path>             # Remove context\nqmd get <file>[:from[:count]]     # Get by path or docid (#abc123); optional line range\nqmd multi-get <pattern>           # Get multiple docs by glob or comma-separated list\nqmd status                        # Show index status and collections\nqmd doctor                        # Diagnose config, index, model, and device issues\nqmd update                        # Re-index collections; configured update hooks run first\nqmd trust [list|revoke]           # Approve a checked-in .qmd config's hooks/paths/models\nqmd embed                         # Generate vector embeddings (uses node-llama-cpp)\nqmd query <query>                 # Search with query expansion + reranking (recommended)\nqmd search <query>                # Full-text keyword search (BM25, no LLM)\nqmd vsearch <query>               # Vector similarity search (no reranking)\nqmd bench <fixture.json>          # Run search-quality benchmarks\nqmd mcp                           # Start MCP server (stdio transport)\nqmd mcp --http [--port N]         # Start MCP server (HTTP, default port 8181)\nqmd mcp --http --daemon           # Start as background daemon\nqmd mcp stop                      # Stop background MCP daemon\n```\n\n## Collection Management\n\n```sh\n# List all collections\nqmd collection list\n\n# Create a collection with explicit name\nqmd collection add ~/Documents/notes --name mynotes --mask '**/*.md'\n\n# Remove a collection\nqmd collection remove mynotes\n\n# Rename a collection\nqmd collection rename mynotes my-notes\n\n# Show collection details, including the top metadata keys\nqmd collection show mynotes\n\n# Discover metadata keys and values to filter on\nqmd collection metadata mynotes\nqmd collection metadata mynotes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}'\nqmd collection metadata mynotes --match '{\"field\":\"value\",\"operator\":\"eq\",\"value\":\"docs-team\"}'\nqmd collection metadata mynotes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}' --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\nqmd collection metadata mynotes --key-limit 50 --key-offset 50   # next page of keys\n\n# Set or clear the pre-update hook (runs before re-indexing on `qmd update`)\nqmd collection update-cmd mynotes 'git pull --ff-only'\nqmd collection update-cmd mynotes            # clear\n\n# Include or exclude from default (unscoped) queries\nqmd collection exclude mynotes\nqmd collection include mynotes\n\n# List all files in a collection\nqmd ls mynotes\n\n# List files with a path prefix\nqmd ls journals/2025\nqmd ls qmd://journals/2025\n```\n\n## Context Management\n\n```sh\n# Add context to current directory (auto-detects collection)\nqmd context add \"Description of these files\"\n\n# Add context to a specific path\nqmd context add /subfolder \"Description for subfolder\"\n\n# Add global context to all collections (system message)\nqmd context add / \"Always include this context\"\n\n# Add context using virtual paths\nqmd context add qmd://journals/ \"Context for entire journals collection\"\nqmd context add qmd://journals/2024 \"Journal entries from 2024\"\n\n# List all contexts\nqmd context list\n\n# Check for collections or paths without context\nqmd context check\n\n# Remove context\nqmd context rm qmd://journals/2024\nqmd context rm /  # Remove global context\n```\n\n## Document IDs (docid)\n\nEach document has a unique short ID (docid) - the first 6 characters of its content hash.\nDocids are shown in search results as `#abc123` and can be used with `get` and `multi-get`:\n\n```sh\n# Search returns docid in results\nqmd search \"query\" --json\n# Output: [{\"docid\": \"#abc123\", \"score\": 0.85, \"file\": \"docs/readme.md\", ...}]\n\n# Get document by docid\nqmd get \"#abc123\"\nqmd get abc123              # Leading # is optional\n\n# Docids also work in multi-get comma-separated lists\nqmd multi-get \"#abc123, #def456\"\n```\n\n## Options\n\n```sh\n# Search & retrieval\n-c, --collection <name>  # Restrict search to collection(s) (repeatable)\n-n <num>                 # Number of results\n--all                    # Return all matches\n--min-score <num>        # Minimum score threshold\n--full                   # Show full document content\n--intent <text>          # Describe what you're after to sharpen ranking (query)\n--no-rerank              # Skip LLM reranking (faster, lower quality)\n--full-path              # Show on-disk paths instead of qmd:// URIs\n\n# Get / multi-get\n-l <num>                 # Maximum lines per file\n--max-bytes <num>        # Skip files larger than this (default 10KB)\n--no-line-numbers        # Disable line numbers (on by default for get/multi-get)\n\n# Output format (search, query, multi-get)\n--format <kind>          # cli (default) | json | csv | md | xml | files\n                         # legacy --json/--csv/--md/--xml/--files still work as aliases\n```\n\n## Development\n\n```sh\nbun src/cli/qmd.ts <command>   # Run from source\nbun link               # Install globally as 'qmd'\n```\n\n## Tests\n\nAll tests live in `test/`. Run everything:\n\n```sh\nnpx vitest run --reporter=verbose test/\nbun test --preload ./src/test-preload.ts test/\n```\n\n## Architecture\n\n- SQLite FTS5 for full-text search (BM25)\n- sqlite-vec for vector similarity search\n- node-llama-cpp for embeddings (embeddinggemma), reranking (qwen3-reranker), and query expansion (Qwen3)\n- Reciprocal Rank Fusion (RRF) for combining results\n- Smart chunking: 900 tokens/chunk with 15% overlap, prefers markdown headings as boundaries\n- AST-aware chunking: use `--chunk-strategy auto` to chunk code files (.ts/.js/.py/.go/.rs) at function/class/import boundaries via tree-sitter. Default is `regex` (existing behavior). Markdown and unknown file types always use regex chunking.\n\n## Important: Do NOT run automatically\n\n- Never run `qmd collection add`, `qmd embed`, or `qmd update` automatically\n- Never modify the SQLite database directly\n- Write out example commands for the user to run manually\n- Index is stored at `~/.cache/qmd/index.sqlite`\n\n## Do NOT compile\n\n- Never run `bun build --compile` - it overwrites the shell wrapper and breaks sqlite-vec\n- The `qmd` file is a shell script that runs compiled JS from `dist/` - do not replace it\n- `npm run build` compiles TypeScript to `dist/` via `tsc -p tsconfig.build.json`\n\n## Releasing\n\nUse `/release <version>` to cut a release. Full changelog standards,\nrelease workflow, and git hook setup are documented in the\n[release skill](skills/release/SKILL.md).\n\nKey points:\n- Add changelog entries under `## [Unreleased]` **as you make changes**\n- The release script renames `[Unreleased]` → `[X.Y.Z] - date` at release time\n- Credit external PRs with `#NNN (thanks @username)`\n- GitHub releases roll up the full minor series (e.g. 1.2.0 through 1.2.3)\n","skills/qmd/SKILL.md":"---\nname: qmd\ndescription: Search local markdown knowledge bases, notes, docs, and wikis with QMD. Use when users ask to find notes, retrieve documents, inspect a wiki, answer from indexed markdown, or set up QMD access.\nlicense: MIT\ncompatibility: Requires qmd CLI or MCP server. Install via `npm install -g @tobilu/qmd`.\nmetadata:\n  author: tobi\n  version: \"2.3.0\"\nallowed-tools: Bash(qmd:*), mcp__qmd__*\n---\n\n# QMD - Query Markdown Documents\n\n## How search works\n\nQMD searches local markdown collections: notes, docs, wikis, transcripts, and\nproject knowledge bases. Use it before web search when the answer may already be\nin indexed local files.\n\nThe workflow is always:\n\n1. Search for candidate documents.\n2. Retrieve the full source with `qmd get` or `qmd multi-get`.\n3. Answer from retrieved text, citing paths or docids.\n\nDo not answer from snippets alone when the user needs facts, decisions, quotes,\nor nuance. Snippets are only leads.\n\nTypical loop:\n\n```bash\nqmd search \"merchant reality support interviews\" -n 5\n# leads: #abc123 concepts/customer-proximity.md; #def432 sources/merchant-call.md\nqmd multi-get \"#abc123,#def432\" --format md\n```\n\n**Default to structured `qmd query` with `intent:`, `lex:`, `vec:`, and `hyde:`\nfields that you write yourself.** You are a better query expander than the\nbuilt-in model: you know the user's actual goal, the domain vocabulary, and the\nnearby-but-wrong concepts to avoid. Do not just paste the user's words into\n`qmd query \"...\"` and hope the expansion model guesses right — supply the\n`intent:` and craft the lexical and semantic terms deliberately (see\n[Pick the right search mode](#pick-the-right-search-mode)).\n\nWhen reporting what you retrieved, a compact note is enough; do not paste whole\nfiles unless needed:\n\n```text\nRetrieved:\n- #abc123 concepts/customer-proximity.md\n- #def432 sources/merchant-call.md\n```\n\n## Pick the right search mode\n\nUse **BM25 lexical search** when you know exact words, titles, names, code\nsymbols, or rare phrases:\n\n```bash\nqmd search \"cockpit OKR Goodhart\" -n 10\nqmd search '\"AI Before Headcount\"' -c concepts -n 5\n```\n\nUse **`qmd query` with structured fields** when the user describes an idea\nindirectly, uses different wording than the source, or needs conceptual recall.\n**This is the default mode — write the fields yourself rather than leaning on\nquery expansion.** Combine exact anchors with semantic recall:\n\n```bash\nqmd query $'intent: Find the concept note about metrics as instruments without letting OKRs replace judgment.\\nlex: cockpit instruments OKR Goodhart metrics judgment\\nvec: data informed not metric driven product judgment\\nhyde: A concept note says metrics are useful like cockpit instruments, but leaders should remain data-informed rather than metric-driven because OKRs and dashboards can Goodhart product judgment.'\n```\n\nStructured query fields (you author each one — do not delegate this to the\nexpansion model):\n\n- `intent:` states what you are trying to find **and what to avoid**. Always\n  supply this. It steers ranking away from nearby-but-wrong concepts.\n- `lex:` exact terms, aliases, titles, code symbols, and rare words you expect\n  in the source. This is your own keyword expansion.\n- `vec:` paraphrases the idea in natural language, in source-like wording.\n- `hyde:` describes the document or answer that would satisfy the request.\n\nYou do not need all four every time, but you should almost always write at least\n`intent:` plus one of `lex:`/`vec:`. A bare `qmd query \"the user's sentence\"`\nthrows away the context only you have and relies on the built-in expander to\nreconstruct it — prefer the structured form.\n\nIf you genuinely have nothing to expand (a single rare token, a verbatim phrase),\nthat is a job for `qmd search`, not bare `qmd query`:\n\n```bash\nqmd query --format json --explain $'intent: ...\\nlex: ...\\nvec: ...'  # inspect ranking\n```\n\nIf `qmd query` is slow or model/GPU setup fails, fall back to `qmd search` with\nbetter lexical terms.\n\n## Retrieve sources\n\nSearch results include docids like `#abc123` and `qmd://...` paths. Fetch them:\n\n```bash\nqmd get \"#abc123\"\nqmd get qmd://concepts/ai-before-headcount.md\nqmd multi-get \"#abc123,#def432\" --format md\nqmd multi-get 'concepts/{ai-before-headcount.md,data-informed-not-metric-driven.md}' --format md\nqmd multi-get 'sources/podcast-2025-*.md' -l 80\n```\n\nUse `multi-get` when comparing several hits or gathering context across pages.\n\n### Output is line-numbered and carries the docid — cite both\n\n`get` and `multi-get` are **line-numbered by default** and always print the\ndocument's `#docid` and `qmd://` path. So `get` output looks like:\n\n```text\nqmd://concepts/note.md  #abc123\n---\n\n1: # Metrics as instruments\n2:\n3: Treat dashboards like cockpit instruments...\n```\n\nCite the docid and exact line numbers in your answer, and use the numbers to ask\nfor the next slice. Pass `--no-line-numbers` only when you need raw content to\ncopy verbatim (e.g. reproducing a code block).\n\nWhen you need to open or edit the underlying file (e.g. hand a path to `Read`,\n`Edit`, or an editor), add `--full-path`. It replaces the `qmd://` URL + docid\nheader with the document's on-disk path, falling back to the canonical header if\nthe file no longer exists on disk:\n\n```text\n$ qmd get \"#abc123\" --full-path\n/Users/you/notes/concepts/note.md\n---\n\n1: # Metrics as instruments\n```\n\n`--full-path` works the same way on `qmd search` and `qmd query`: result paths\nbecome the file's on-disk path — `./`-prefixed relative path when the file is\ninside `$PWD`, absolute realpath otherwise — and the per-result `#docid` is\ndropped because the path is the identifier. The leading `./` is intentional so\nthe output is unambiguously a filesystem path and cannot be mistaken for a bare\ncollection-relative string. Default search/query output still uses `qmd://`\nURIs; only opt into `--full-path` when you specifically need a path you can hand\nto a non-QMD tool.\n\n### Read line ranges with the `:from:count` suffix — never pipe through `sed`/`head`/`tail`\n\n`qmd get` slices files itself. Use the suffix or flags; do **not** shell out to\n`sed -n`, `head`, `tail`, or `awk` to pull a line range. Piping defeats docid\nresolution, virtual-path lookups, line numbering, and the header, and it is\nslower and more error-prone.\n\nThe most compact form is a `:from:count` suffix right on the path or docid —\nprefer it:\n\n```bash\nqmd get \"#abc123:120:40\"                  # 40 lines starting at line 120\nqmd get qmd://concepts/note.md:200:60     # lines 200–259\nqmd get \"#abc123:120\"                      # from line 120 to end of file\nqmd get \"#abc123\" --from 120 -l 40         # equivalent, using flags\n```\n\nSuffix and flags:\n\n- `<path>:<from>:<count>` — start at line `<from>`, read `<count>` lines. **Best\n  for reading around a search hit.**\n- `<path>:<from>` — start at `<from>`, read to end of file.\n- `--from <line>` / `-l <lines>` — flag equivalents. Explicit flags override the\n  suffix, so `... :5:2 -l 1` reads 1 line.\n- `--no-line-numbers` — drop the `N:` prefixes (line numbers are on by default).\n\nWrong: `qmd get \"#abc123\" | sed -n '120,160p'`\nRight: `qmd get \"#abc123:120:40\"`\n\nSearch results include a `:line` anchor on each hit — feed it straight into\n`qmd get path:line:<n>` to read a window around the match (line numbers in the\noutput will start at `line`).\n\n## Discover what is indexed\n\n```bash\nqmd collection list\nqmd ls\nqmd status\n```\n\nAdd collection filters when broad searches drift into the wrong corpus:\n\n```bash\nqmd search \"headcount autonomous agents\" -c concepts -n 10\nqmd query \"merchant support product reality\" -c concepts -c sources -n 10\n```\n\nOmit `-c` to search everything.\n\n## Filter by metadata\n\nDocuments can carry typed metadata in a `qmd.metadata` frontmatter block\n(strings, numbers, booleans, or flat arrays). `search`, `vsearch`, and `query`\naccept `--filter` with a recursive JSON AST, and every returned result satisfies\nit. Add a metadata filter when it expresses a constraint the user intends, not\nby default, and check coverage first: a condition on a key's value only sees\ndocuments that declare that key, and discovery (below) shows how many do.\n\n```bash\nqmd search \"authentication\" --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\nqmd query \"dependency injection\" --filter '{\"operator\":\"and\",\"operands\":[{\"field\":\"topics\",\"operator\":\"all\",\"value\":[\"typescript\"]},{\"field\":\"status\",\"operator\":\"nin\",\"value\":[\"draft\",\"archived\"]}]}'\n```\n\nNodes are discriminated by `operator`: groups `and`/`or` take `operands`, `not`\ntakes one `operand`, and conditions take `field` + `value` with operators\n`eq`/`ne`/`gt`/`gte`/`lt`/`lte` (comparison), `in`/`nin`/`all` (membership),\n`contains`/`prefix`/`suffix` (text), `type` (the value is a `string`, `number`,\nor `boolean`), or `exists` (presence). Matching is typed and exact; missing keys\ndo not match `ne`/`nin` (add an `exists: false` branch in an `or` group to\ninclude them). Conditions with a string value may add `\"caseInsensitive\": true`,\nwhich folds ASCII letters. The MCP `query` tool accepts the same AST as a\n`filter` object. JSON output includes each result's `metadata`.\n\n## Discover metadata before filtering\n\nDo not guess keys or values. `qmd collection show <name>` lists the top keys\nwith types and a value preview, and `qmd collection metadata` drills in. Same\nmetadata, different unit: `--filter` narrows documents (which are counted),\n`--match` narrows the metadata itself (which entries are reported). Both take\nthe predicate AST above. In a match, a condition's `field` is `\"key\"` (the\nentry's key name) or `\"value\"` (its value), and every operator applies except\n`exists` and `all`:\n\n```bash\nqmd collection metadata notes                                                                # top keys, top values\nqmd collection metadata notes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}'       # one key in depth\nqmd collection metadata notes --match '{\"field\":\"value\",\"operator\":\"eq\",\"value\":\"docs-team\"}'  # which keys hold this value\nqmd collection metadata notes --match '{\"operator\":\"and\",\"operands\":[{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"priority\"},{\"field\":\"value\",\"operator\":\"gte\",\"value\":3}]}'\nqmd collection metadata notes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}' --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\n```\n\nThe last form shows what a filter leaves behind before you commit to it in a\nquery. Read the output like this:\n\n- **The header is the decision.** `topics  string[]  388 of 480 documents\n  1,204 distinct` gives coverage and cardinality. With `--filter`, a\n  `filter: 312 of 480 documents` line opens the output and every count is out\n  of those 312. Counts are documents, not values.\n- **Quotes mark a string that reads like something else.** `\"42\"` is a string,\n  `42` a number, `\"\"` is empty. Paste a quoted value into a filter as the JSON\n  string it is.\n- **Footers mean windowed.** `N more values` or `N more keys` means the list\n  was cut. Page with `--value-offset`/`--key-offset` or raise\n  `--value-limit`/`--key-limit` rather than assuming the rest.\n- **Numbers print `min`, `median`, `max`**, enough to write a `gt`/`lt`\n  threshold in one call.\n- **`number | string` means documents disagree on type.** Each type reports its\n  own document count. Filter by the type that covers the documents you want.\n\nEvery value shown can be matched with `eq` under the same collection scope.\n\nOver MCP, call the `metadata` tool (same options, `collections` as an array)\nand read `totalKeys`, `remainingKeys`, `remainingValues`, and `range` from the\nstructured result. Page keys with `keyOffset` and values with `valueOffset`.\nThe `status` tool lists each collection's most covered keys, so check it first.\n\n## MCP Tool: `query`\n\nWhen using the MCP server, prefer structured searches:\n\n```json\n{\n  \"searches\": [\n    { \"type\": \"lex\", \"query\": \"cockpit OKR Goodhart\" },\n    { \"type\": \"vec\", \"query\": \"data informed not metric driven product judgment\" },\n    { \"type\": \"hyde\", \"query\": \"A concept note explains that metrics are useful as instruments, but leaders should not let OKRs or dashboards replace judgment.\" }\n  ],\n  \"intent\": \"Find the concept note about using metrics as instruments without becoming metric-driven.\",\n  \"collections\": [\"concepts\"],\n  \"limit\": 10\n}\n```\n\nQuery types:\n\n- `lex` — BM25 keyword search. Best for exact terms, names, titles, and code.\n- `vec` — vector semantic search. Best for natural-language concepts.\n- `hyde` — vector search using a hypothetical answer/document passage.\n\n## Query craft\n\nGood QMD searches mix three things:\n\n1. **Title/alias anchors:** exact page titles, named entities, phrases.\n2. **Semantic paraphrase:** how a human would describe the idea.\n3. **Negative space:** enough intent to avoid nearby-but-wrong concepts.\n\nExamples:\n\n```bash\n# Exact-ish title lookup\nqmd search '\"arm the rebels\" merchants tools big companies' -c concepts\n\n# Semantic concept lookup\nqmd query $'intent: Find the customer proximity concept, not generic customer delight.\\nlex: support pseudonymous merchant customer interviews\\nvec: founder stays close to merchant reality through support and product use'\n\n# Source lookup\nqmd search \"six-week cadence WhatsApp merchant relationships Shawn Ryan\" -c sources -n 10\n```\n\n## Setup and maintenance\n\nOnly mutate indexes when the user asked for setup or maintenance. Searching and\nretrieving are safe; collection/index mutation is not a casual first step.\n\n```bash\nnpm install -g @tobilu/qmd\nqmd collection add ~/notes --name notes\nqmd update\nqmd embed\n```\n\nHealth and diagnostics:\n\n```bash\nqmd doctor\nqmd status\nqmd pull\n```\n\n`qmd doctor` checks config, model cache, device/GPU setup, vector fingerprints,\nand common environment overrides. If a model-backed command fails, run it before\nchanging configuration.\n\n## MCP setup\n\nSee `references/mcp-setup.md` for Claude Code, Claude Desktop, OpenClaw, and HTTP\nserver configuration.\n\n## Pitfalls\n\n- **Do not stop at snippets.** Fetch documents before making claims.\n- **Do not slice files with `sed`/`head`/`tail`.** Use the `path:from:count`\n  suffix (e.g. `qmd get \"#abc123:120:40\"`) or `--from`/`-l`. Output is already\n  line-numbered; piping breaks docid resolution, the header, and virtual paths.\n- **Do not lean on query expansion.** Write `intent:`/`lex:`/`vec:`/`hyde:`\n  yourself. A bare `qmd query \"user sentence\"` discards the context only you\n  have. You expand the query; the model just ranks.\n- **Do not overuse semantic search.** If you know exact titles or terms, BM25 is\n  faster and often better.\n- **Do not filter blind.** Run `qmd collection metadata` first and read the\n  coverage. A key declared by 388 of 480 documents leaves 92 that no value\n  condition on it can reach; only `exists: false` selects them.\n- **Do not mutate indexes casually.** `qmd collection add`, `qmd update`, and\n  `qmd embed` change local state and can be expensive.\n- **Model-backed commands can be environment-sensitive.** If `qmd query`,\n  `qmd vsearch`, or reranking fails because local models/GPU are unavailable,\n  use `qmd search` and stronger lexical/structured terms.\n- **Ambiguous user wording needs intent.** Add `intent:` rather than hoping query\n  expansion guesses the right domain.\n- **Collection names matter.** Search `concepts` for synthesized wiki pages,\n  `sources` for transcripts/raw source pages, and docs collections for code or\n  project documentation.\n","skills/qmd/references/mcp-setup.md":"# QMD MCP Server Setup\n\n## Install\n\n```bash\nnpm install -g @tobilu/qmd\nqmd collection add ~/path/to/markdown --name myknowledge\nqmd embed\n```\n\n## Configure MCP Client\n\n**Claude Code** (`~/.claude/settings.json`):\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n  }\n}\n```\n\n**Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n  }\n}\n```\n\n**OpenClaw** (`~/.openclaw/openclaw.json`):\n```json\n{\n  \"mcp\": {\n    \"servers\": {\n      \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n    }\n  }\n}\n```\n\n## Protocol\n\nQMD speaks MCP **2026-07-28** (SDK 2.x) and dual-speaks 2025-era stdio clients.\n\n- **stdio** (`qmd mcp`): hosts still launch this as a subprocess. Opening\n  `initialize` (2025) or a `_meta`-enveloped request / `server/discover`\n  (2026) pins the connection's era.\n- **HTTP** (`qmd mcp --http`): sessionless Streamable HTTP. No\n  `Mcp-Session-Id`, no handshake. Each POST is independent. 2026 clients\n  MUST send `MCP-Protocol-Version`, `Mcp-Method`, and (for `tools/call`)\n  `Mcp-Name`. Version/caps travel in `_meta`. Call `server/discover` to\n  learn supported versions and capabilities. `tools/list` is cacheable\n  (`ttlMs` / `cacheScope`) and returns tools in a stable order.\n\n## HTTP Mode\n\n```bash\nqmd mcp --http              # Port 8181\nqmd mcp --http --daemon     # Background\nqmd mcp stop                # Stop daemon\n```\n\n`POST /mcp` is the MCP endpoint (JSON). `GET /health` is a liveness check.\nThere is no session GET stream and no idle-session TTL.\n\n## Tools\n\n### query\n\nSearch with pre-expanded queries.\n\n```json\n{\n  \"searches\": [\n    { \"type\": \"lex\", \"query\": \"keyword phrases\" },\n    { \"type\": \"vec\", \"query\": \"natural language question\" },\n    { \"type\": \"hyde\", \"query\": \"hypothetical answer passage...\" }\n  ],\n  \"limit\": 10,\n  \"collection\": \"optional\",\n  \"minScore\": 0.0\n}\n```\n\n| Type | Method | Input |\n|------|--------|-------|\n| `lex` | BM25 | Keywords (2-5 terms) |\n| `vec` | Vector | Question |\n| `hyde` | Vector | Answer passage (50-100 words) |\n\n### get\n\nRetrieve document by path or `#docid`.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `path` | string | File path or `#docid` |\n| `full` | bool? | Return full content |\n| `lineNumbers` | bool? | Add line numbers |\n\n### multi_get\n\nRetrieve multiple documents.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `pattern` | string | Glob or comma-separated list |\n| `maxBytes` | number? | Skip large files (default 64KB) |\n\n### status\n\nIndex health and collections. No params.\n\n## Troubleshooting\n\n- **Not starting**: `which qmd`, `qmd mcp` manually\n- **No results**: `qmd collection list`, `qmd embed`\n- **Slow first search**: Normal, models loading (~3GB)\n","finetune/CLAUDE.md":"# QMD Query Expansion Fine-Tuning\n\n## Overview\n\nTrain Qwen3-1.7B to expand search queries into structured `hyde:/lex:/vec:` output for QMD's hybrid retrieval pipeline.\n\n## Output Format\n\n```\nhyde: A hypothetical document passage that would answer the query.\nlex: keyword1\nlex: keyword2\nvec: semantic query reformulation\nvec: another semantic variation\n```\n\n- `hyde:` always comes FIRST (one line max)\n- `lex:` lines for BM25 keyword search (1-3 lines, short keywords)\n- `vec:` lines for vector similarity search (1-3 lines, natural language)\n\n## Training Data Format\n\n**There is exactly one JSONL format.** Every file in `data/*.jsonl` must match the strict Pydantic schema in `dataset/schema.py`:\n\n```json\n{\"query\": \"auth config\", \"output\": [[\"hyde\", \"...\"], [\"lex\", \"...\"], [\"vec\", \"...\"]]}\n```\n\n- `query`: non-empty string\n- `output`: list of `[type, text]` pairs where type is `\"lex\"`, `\"vec\"`, or `\"hyde\"`\n- Extra metadata fields (`category`, `intent`, `is_short`) are allowed but ignored\n\nThe schema is enforced by `dataset/schema.py:TrainingExample` (Pydantic model). All data loading goes through `load_examples()` which fails loudly on invalid data. No format alternatives, no legacy fallbacks.\n\n**All `.jsonl` files in `data/` are concatenated and deduplicated for training runs.** The prepared train/val files in `data/train/` are ephemeral build artifacts.\n\n## HuggingFace Repositories\n\n| Repository | Purpose |\n|------------|---------|\n| `tobil/qmd-query-expansion-1.7B` | Final merged model (SFT baseline) |\n| `tobil/qmd-query-expansion-1.7B-gguf` | GGUF quantized versions for deployment |\n| `tobil/qmd-query-expansion-1.7B-sft` | SFT adapter checkpoint (intermediate) |\n| `tobil/qmd-query-expansion-train` | Prepared training dataset |\n| `tobil/qmd-query-expansion-1.7B-grpo` | Experimental GRPO adapter (optional) |\n\n**Rules:**\n- No versioned repos (`-v1`, `-v2`, `-v4`, etc.) - update in place\n- Only push when eval scores improve over current deployed model\n- Always include eval results in model card when pushing\n\n## Dataset Tools\n\n| Script | Purpose |\n|--------|---------|\n| `dataset/schema.py` | Pydantic `TrainingExample` model + `load_examples()` |\n| `dataset/prepare_data.py` | Load via schema, apply Qwen3 chat template, dedup, split |\n| `dataset/validate_schema.py` | Validate all JSONL files against schema |\n| `dataset/score_data.py` | Score all examples using reward.py |\n| `dataset/analyze_data.py` | Analyze distribution and quality |\n\n## Training Pipeline\n\nAlways use **Qwen3-1.7B** as the base model unless explicitly stated otherwise.\n\n### Stage 0: Prepare Data\n\n```bash\nuv run dataset/prepare_data.py\n# Creates: data/train/train.jsonl, data/train/val.jsonl (ephemeral)\n```\n\n### Stage 1: SFT\n\n```bash\n# Local (requires CUDA)\nuv run train.py sft --config configs/sft.yaml\n\n# Cloud (HuggingFace Jobs)\nhf jobs uv run --flavor a10g-large --secrets HF_TOKEN --timeout 2h jobs/sft.py\n```\n\n### Stage 2: (Experimental) GRPO\n\n```bash\n# Experimental script\ncd finetune && HF_TOKEN=${HF_TOKEN} uv run python experiments/grpo/grpo.py\n```\n\n### HuggingFace Jobs\n\n```bash\nhf jobs ps                    # List running jobs\nhf jobs logs <job-id>         # Stream logs\nhf jobs inspect <job-id>      # Check status\nhf jobs cancel <job-id>       # Cancel a job\n```\n\n### Evaluation\n\n```bash\nuv run eval.py ./outputs/sft\nuv run eval.py tobil/qmd-query-expansion-1.7B\nuv run eval.py ./outputs/sft -o eval_results.json\n```\n\n## Quality Scoring\n\n`reward.py` is the single source of truth for scoring:\n\n```bash\nuv run reward.py   # Self-test\n```\n\nSee `SCORING.md` for the full rubric.\n\n## Experiments\n\nExperimental training configurations live in `experiments/`:\n\n```\nexperiments/\n├── lfm2/          # LiquidAI LFM2-1.2B (hybrid architecture, faster inference)\n│   ├── sft_lfm2.yaml\n│   └── sft_lfm2.py\n├── grpo/          # Experimental GRPO recipe and config\n│   ├── grpo.py\n│   └── grpo.yaml\n└── gepa/          # DSPy-based prompt optimization (GEPA)\n    ├── dspy_gepa.py\n    └── ...\n```\n\nThese are not part of the main training pipeline.\n\n## Key Files\n\n```\nfinetune/\n├── reward.py          # Scoring function (single source of truth)\n├── train.py           # SFT training entrypoint\n├── eval.py            # Generate and score expansions\n├── convert_gguf.py    # GGUF conversion\n├── SCORING.md         # Detailed scoring rubric\n├── CLAUDE.md          # This file\n├── Justfile           # Common commands\n├── data/              # All training JSONL files (strict schema)\n├── dataset/           # Schema + data tools (Pydantic-based)\n├── jobs/              # Self-contained HuggingFace Jobs scripts\n├── configs/           # Training configs (sft.yaml)\n├── evals/             # Test queries\n├── experiments/       # Experimental configs (LFM2, GEPA, GRPO)\n└── outputs/           # Local training outputs (gitignored)\n```\n"},"files":{".claude-plugin/marketplace.json":"{\n\t\"name\": \"qmd\",\n\t\"owner\": {\n\t\t\"name\": \"tobi\",\n\t\t\"email\": \"tobi@lutke.com\"\n\t},\n\t\"plugins\": [\n\t\t{\n\t\t\t\"name\": \"qmd\",\n\t\t\t\"source\": \"./skills\",\n\t\t\t\"description\": \"Search and retrieve documents from local markdown files.\",\n\t\t\t\"version\": \"2.8.3\",\n\t\t\t\"author\": {\n\t\t\t\t\"name\": \"tobi\",\n\t\t\t\t\"email\": \"tobi@lutke.com\"\n\t\t\t},\n\t\t\t\"repository\": \"https://github.com/tobi/qmd\",\n\t\t\t\"license\": \"MIT\",\n\t\t\t\"keywords\": [\"markdown\", \"search\", \"qmd\"],\n\t\t\t\"skills\": [\"./qmd\", \"./release\"],\n\t\t\t\"mcpServers\": {\n\t\t\t\t\"qmd\": {\n\t\t\t\t\t\"command\": \"qmd\",\n\t\t\t\t\t\"args\": [\"mcp\"]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t]\n}\n","skills/release/SKILL.md":"---\nname: release\ndescription: Manage releases for this project. Validates changelog, installs git hooks, and cuts releases. Use when user says \"/release\", \"release 1.0.5\", \"cut a release\", or asks about the release process. NOT auto-invoked by the model.\ndisable-model-invocation: true\n---\n\n# Release\n\nCut a release, validate the changelog, and ensure git hooks are installed.\n\n## Usage\n\n`/release 1.0.5` or `/release patch` (bumps patch from current version).\n\n## Process\n\nWhen the user triggers `/release <version>`:\n\n1. **Gather context** — run `skills/release/scripts/release-context.sh <version>`.\n   This silently installs git hooks and prints everything needed: version info,\n   working directory status, commits since last release, files changed, current\n   `[Unreleased]` content, and the previous release entry for style reference.\n\n2. **Commit outstanding work** — if the context shows staged, modified, or\n   untracked files that belong in this release, commit them first. Use the\n   /commit skill or make well-formed commits directly.\n\n3. **Write the changelog** — if `[Unreleased]` is empty, write it now using\n   the commits and file changes from the context output. Follow the changelog\n   standard below. Re-run the context script after committing if needed.\n\n4. **Check dependency updates** — before cutting the release, check for\n   updates to `sqlite-vec` (and platform packages), `node-llama-cpp`,\n   and `better-sqlite3`. Run `pnpm outdated` and report any available\n   updates for these packages. If updates exist, bump them (pinned, no\n   `^` ranges) and re-run tests before proceeding.\n\n5. **Cut the release** — run `scripts/release.sh <version>`. This renames\n   `[Unreleased]` → `[X.Y.Z] - date`, inserts a fresh `[Unreleased]`,\n   bumps `package.json` and the plugin version in\n   `.claude-plugin/marketplace.json` (so installed plugins see the update),\n   commits, and tags.\n\n6. **Show the final changelog** — print the full `[Unreleased]` +\n   minor series rollup via `scripts/extract-changelog.sh <version>`.\n   Ask the user to confirm before pushing.\n\n7. **Push** — after explicit confirmation, run `git push origin main --tags`.\n\n8. **Watch CI** — after the push, start a background dispatch to watch the\n   publish workflow. Use `interactive_shell` in dispatch mode with:\n   ```\n   gh run watch $(gh run list --workflow=publish.yml --limit=1 --json databaseId --jq '.[0].databaseId') --exit-status\n   ```\n   The agent will be notified when CI completes and should report the result.\n\nIf any step fails, stop and explain. Never force-push or skip validation.\n\n## Dependency Policy\n\nAll dependencies must be pinned to exact versions (no `^` or `~` ranges).\nThe lockfile ensures reproducible installs. When adding or updating any\ndependency, always use the exact version string (e.g. `\"3.18.1\"` not\n`\"^3.18.1\"`).\n\n## Changelog Standard\n\nThe changelog lives in `CHANGELOG.md` and follows [Keep a Changelog](https://keepachangelog.com/) conventions.\n\n### Heading format\n\n- `## [Unreleased]` — accumulates entries between releases\n- `## [X.Y.Z] - YYYY-MM-DD` — released versions\n\n### Structure of a release entry\n\nEach version entry has two parts:\n\n**1. Highlights (optional, 1-4 sentences of prose)**\n\nImmediately after the version heading, before any `###` section. The elevator\npitch — what would you tell someone in 30 seconds? Only for significant\nreleases; skip for small patches.\n\n```markdown\n## [1.1.0] - 2026-03-01\n\nQMD now runs on both Node.js and Bun, with up to 2.7x faster reranking\nthrough parallel contexts. GPU auto-detection replaces the unreliable\n`gpu: \"auto\"` with explicit CUDA/Metal/Vulkan probing.\n```\n\n**2. Detailed changelog (`### Changes` and `### Fixes`)**\n\n```markdown\n### Changes\n\n- Runtime: support Node.js (>=22) alongside Bun. The `qmd` wrapper\n  auto-detects a suitable install via PATH. #149 (thanks @igrigorik)\n- Performance: parallel embedding & reranking — up to 2.7x faster on\n  multi-core machines.\n\n### Fixes\n\n- Prevent VRAM waste from duplicate context creation during concurrent\n  `embedBatch` calls. #152 (thanks @jkrems)\n```\n\n### Writing guidelines\n\n- **Explain the why, not just the what.** The changelog is for users.\n- **Include numbers.** \"2.7x faster\", \"17x less memory\".\n- **Group by theme, not by file.** \"Performance\" not \"Changes to llm.ts\".\n- **Don't list every commit.** Aggregate related changes.\n- **Credit contributors:** end bullets with `#NNN (thanks @username)` for\n  external PRs. No need to credit the repo owner.\n\n### What not to include\n\n- Internal refactors with no user-visible effect\n- Dependency bumps (unless fixing a user-facing bug)\n- CI/tooling changes (unless affecting the release artifact)\n- Test additions (unless validating a fix worth mentioning)\n\n## GitHub Release Notes\n\nEach GitHub release includes the full changelog for the **minor series** back\nto x.x.0. The `scripts/extract-changelog.sh` script handles this, and the\npublish workflow (`publish.yml`) calls it to populate the GitHub release.\n\n## Git Hooks\n\nThe pre-push hook (`scripts/pre-push`) blocks `v*` tag pushes unless:\n\n1. `package.json` version matches the tag\n2. `CHANGELOG.md` has a `## [X.Y.Z] - date` entry for the version\n3. CI passed on GitHub (warns in non-interactive shells, blocks in terminals)\n\nHooks are installed silently by the context script. They can also be installed\nmanually via `skills/release/scripts/install-hooks.sh` or automatically via\n`bun install` (prepare script).\n","CLAUDE.md":"# QMD - Query Markup Documents\n\nUse Bun instead of Node.js (`bun` not `node`, `bun install` not `npm install`).\n\n## Commands\n\n```sh\nqmd collection add . --name <n>   # Create/index collection\nqmd collection list               # List all collections with details\nqmd collection remove <name>      # Remove a collection by name\nqmd collection rename <old> <new> # Rename a collection\nqmd collection metadata [name...] # Discover metadata keys, types, and value counts (--match, --filter, --key-limit, --value-limit)\nqmd init                          # Create a project-local .qmd index\nqmd ls [collection[/path]]        # List collections or files in a collection\nqmd context add [path] \"text\"     # Add context for path (defaults to current dir)\nqmd context list                  # List all contexts\nqmd context check                 # Check for collections/paths missing context\nqmd context rm <path>             # Remove context\nqmd get <file>[:from[:count]]     # Get by path or docid (#abc123); optional line range\nqmd multi-get <pattern>           # Get multiple docs by glob or comma-separated list\nqmd status                        # Show index status and collections\nqmd doctor                        # Diagnose config, index, model, and device issues\nqmd update                        # Re-index collections; configured update hooks run first\nqmd trust [list|revoke]           # Approve a checked-in .qmd config's hooks/paths/models\nqmd embed                         # Generate vector embeddings (uses node-llama-cpp)\nqmd query <query>                 # Search with query expansion + reranking (recommended)\nqmd search <query>                # Full-text keyword search (BM25, no LLM)\nqmd vsearch <query>               # Vector similarity search (no reranking)\nqmd bench <fixture.json>          # Run search-quality benchmarks\nqmd mcp                           # Start MCP server (stdio transport)\nqmd mcp --http [--port N]         # Start MCP server (HTTP, default port 8181)\nqmd mcp --http --daemon           # Start as background daemon\nqmd mcp stop                      # Stop background MCP daemon\n```\n\n## Collection Management\n\n```sh\n# List all collections\nqmd collection list\n\n# Create a collection with explicit name\nqmd collection add ~/Documents/notes --name mynotes --mask '**/*.md'\n\n# Remove a collection\nqmd collection remove mynotes\n\n# Rename a collection\nqmd collection rename mynotes my-notes\n\n# Show collection details, including the top metadata keys\nqmd collection show mynotes\n\n# Discover metadata keys and values to filter on\nqmd collection metadata mynotes\nqmd collection metadata mynotes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}'\nqmd collection metadata mynotes --match '{\"field\":\"value\",\"operator\":\"eq\",\"value\":\"docs-team\"}'\nqmd collection metadata mynotes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}' --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\nqmd collection metadata mynotes --key-limit 50 --key-offset 50   # next page of keys\n\n# Set or clear the pre-update hook (runs before re-indexing on `qmd update`)\nqmd collection update-cmd mynotes 'git pull --ff-only'\nqmd collection update-cmd mynotes            # clear\n\n# Include or exclude from default (unscoped) queries\nqmd collection exclude mynotes\nqmd collection include mynotes\n\n# List all files in a collection\nqmd ls mynotes\n\n# List files with a path prefix\nqmd ls journals/2025\nqmd ls qmd://journals/2025\n```\n\n## Context Management\n\n```sh\n# Add context to current directory (auto-detects collection)\nqmd context add \"Description of these files\"\n\n# Add context to a specific path\nqmd context add /subfolder \"Description for subfolder\"\n\n# Add global context to all collections (system message)\nqmd context add / \"Always include this context\"\n\n# Add context using virtual paths\nqmd context add qmd://journals/ \"Context for entire journals collection\"\nqmd context add qmd://journals/2024 \"Journal entries from 2024\"\n\n# List all contexts\nqmd context list\n\n# Check for collections or paths without context\nqmd context check\n\n# Remove context\nqmd context rm qmd://journals/2024\nqmd context rm /  # Remove global context\n```\n\n## Document IDs (docid)\n\nEach document has a unique short ID (docid) - the first 6 characters of its content hash.\nDocids are shown in search results as `#abc123` and can be used with `get` and `multi-get`:\n\n```sh\n# Search returns docid in results\nqmd search \"query\" --json\n# Output: [{\"docid\": \"#abc123\", \"score\": 0.85, \"file\": \"docs/readme.md\", ...}]\n\n# Get document by docid\nqmd get \"#abc123\"\nqmd get abc123              # Leading # is optional\n\n# Docids also work in multi-get comma-separated lists\nqmd multi-get \"#abc123, #def456\"\n```\n\n## Options\n\n```sh\n# Search & retrieval\n-c, --collection <name>  # Restrict search to collection(s) (repeatable)\n-n <num>                 # Number of results\n--all                    # Return all matches\n--min-score <num>        # Minimum score threshold\n--full                   # Show full document content\n--intent <text>          # Describe what you're after to sharpen ranking (query)\n--no-rerank              # Skip LLM reranking (faster, lower quality)\n--full-path              # Show on-disk paths instead of qmd:// URIs\n\n# Get / multi-get\n-l <num>                 # Maximum lines per file\n--max-bytes <num>        # Skip files larger than this (default 10KB)\n--no-line-numbers        # Disable line numbers (on by default for get/multi-get)\n\n# Output format (search, query, multi-get)\n--format <kind>          # cli (default) | json | csv | md | xml | files\n                         # legacy --json/--csv/--md/--xml/--files still work as aliases\n```\n\n## Development\n\n```sh\nbun src/cli/qmd.ts <command>   # Run from source\nbun link               # Install globally as 'qmd'\n```\n\n## Tests\n\nAll tests live in `test/`. Run everything:\n\n```sh\nnpx vitest run --reporter=verbose test/\nbun test --preload ./src/test-preload.ts test/\n```\n\n## Architecture\n\n- SQLite FTS5 for full-text search (BM25)\n- sqlite-vec for vector similarity search\n- node-llama-cpp for embeddings (embeddinggemma), reranking (qwen3-reranker), and query expansion (Qwen3)\n- Reciprocal Rank Fusion (RRF) for combining results\n- Smart chunking: 900 tokens/chunk with 15% overlap, prefers markdown headings as boundaries\n- AST-aware chunking: use `--chunk-strategy auto` to chunk code files (.ts/.js/.py/.go/.rs) at function/class/import boundaries via tree-sitter. Default is `regex` (existing behavior). Markdown and unknown file types always use regex chunking.\n\n## Important: Do NOT run automatically\n\n- Never run `qmd collection add`, `qmd embed`, or `qmd update` automatically\n- Never modify the SQLite database directly\n- Write out example commands for the user to run manually\n- Index is stored at `~/.cache/qmd/index.sqlite`\n\n## Do NOT compile\n\n- Never run `bun build --compile` - it overwrites the shell wrapper and breaks sqlite-vec\n- The `qmd` file is a shell script that runs compiled JS from `dist/` - do not replace it\n- `npm run build` compiles TypeScript to `dist/` via `tsc -p tsconfig.build.json`\n\n## Releasing\n\nUse `/release <version>` to cut a release. Full changelog standards,\nrelease workflow, and git hook setup are documented in the\n[release skill](skills/release/SKILL.md).\n\nKey points:\n- Add changelog entries under `## [Unreleased]` **as you make changes**\n- The release script renames `[Unreleased]` → `[X.Y.Z] - date` at release time\n- Credit external PRs with `#NNN (thanks @username)`\n- GitHub releases roll up the full minor series (e.g. 1.2.0 through 1.2.3)\n","skills/qmd/SKILL.md":"---\nname: qmd\ndescription: Search local markdown knowledge bases, notes, docs, and wikis with QMD. Use when users ask to find notes, retrieve documents, inspect a wiki, answer from indexed markdown, or set up QMD access.\nlicense: MIT\ncompatibility: Requires qmd CLI or MCP server. Install via `npm install -g @tobilu/qmd`.\nmetadata:\n  author: tobi\n  version: \"2.3.0\"\nallowed-tools: Bash(qmd:*), mcp__qmd__*\n---\n\n# QMD - Query Markdown Documents\n\n## How search works\n\nQMD searches local markdown collections: notes, docs, wikis, transcripts, and\nproject knowledge bases. Use it before web search when the answer may already be\nin indexed local files.\n\nThe workflow is always:\n\n1. Search for candidate documents.\n2. Retrieve the full source with `qmd get` or `qmd multi-get`.\n3. Answer from retrieved text, citing paths or docids.\n\nDo not answer from snippets alone when the user needs facts, decisions, quotes,\nor nuance. Snippets are only leads.\n\nTypical loop:\n\n```bash\nqmd search \"merchant reality support interviews\" -n 5\n# leads: #abc123 concepts/customer-proximity.md; #def432 sources/merchant-call.md\nqmd multi-get \"#abc123,#def432\" --format md\n```\n\n**Default to structured `qmd query` with `intent:`, `lex:`, `vec:`, and `hyde:`\nfields that you write yourself.** You are a better query expander than the\nbuilt-in model: you know the user's actual goal, the domain vocabulary, and the\nnearby-but-wrong concepts to avoid. Do not just paste the user's words into\n`qmd query \"...\"` and hope the expansion model guesses right — supply the\n`intent:` and craft the lexical and semantic terms deliberately (see\n[Pick the right search mode](#pick-the-right-search-mode)).\n\nWhen reporting what you retrieved, a compact note is enough; do not paste whole\nfiles unless needed:\n\n```text\nRetrieved:\n- #abc123 concepts/customer-proximity.md\n- #def432 sources/merchant-call.md\n```\n\n## Pick the right search mode\n\nUse **BM25 lexical search** when you know exact words, titles, names, code\nsymbols, or rare phrases:\n\n```bash\nqmd search \"cockpit OKR Goodhart\" -n 10\nqmd search '\"AI Before Headcount\"' -c concepts -n 5\n```\n\nUse **`qmd query` with structured fields** when the user describes an idea\nindirectly, uses different wording than the source, or needs conceptual recall.\n**This is the default mode — write the fields yourself rather than leaning on\nquery expansion.** Combine exact anchors with semantic recall:\n\n```bash\nqmd query $'intent: Find the concept note about metrics as instruments without letting OKRs replace judgment.\\nlex: cockpit instruments OKR Goodhart metrics judgment\\nvec: data informed not metric driven product judgment\\nhyde: A concept note says metrics are useful like cockpit instruments, but leaders should remain data-informed rather than metric-driven because OKRs and dashboards can Goodhart product judgment.'\n```\n\nStructured query fields (you author each one — do not delegate this to the\nexpansion model):\n\n- `intent:` states what you are trying to find **and what to avoid**. Always\n  supply this. It steers ranking away from nearby-but-wrong concepts.\n- `lex:` exact terms, aliases, titles, code symbols, and rare words you expect\n  in the source. This is your own keyword expansion.\n- `vec:` paraphrases the idea in natural language, in source-like wording.\n- `hyde:` describes the document or answer that would satisfy the request.\n\nYou do not need all four every time, but you should almost always write at least\n`intent:` plus one of `lex:`/`vec:`. A bare `qmd query \"the user's sentence\"`\nthrows away the context only you have and relies on the built-in expander to\nreconstruct it — prefer the structured form.\n\nIf you genuinely have nothing to expand (a single rare token, a verbatim phrase),\nthat is a job for `qmd search`, not bare `qmd query`:\n\n```bash\nqmd query --format json --explain $'intent: ...\\nlex: ...\\nvec: ...'  # inspect ranking\n```\n\nIf `qmd query` is slow or model/GPU setup fails, fall back to `qmd search` with\nbetter lexical terms.\n\n## Retrieve sources\n\nSearch results include docids like `#abc123` and `qmd://...` paths. Fetch them:\n\n```bash\nqmd get \"#abc123\"\nqmd get qmd://concepts/ai-before-headcount.md\nqmd multi-get \"#abc123,#def432\" --format md\nqmd multi-get 'concepts/{ai-before-headcount.md,data-informed-not-metric-driven.md}' --format md\nqmd multi-get 'sources/podcast-2025-*.md' -l 80\n```\n\nUse `multi-get` when comparing several hits or gathering context across pages.\n\n### Output is line-numbered and carries the docid — cite both\n\n`get` and `multi-get` are **line-numbered by default** and always print the\ndocument's `#docid` and `qmd://` path. So `get` output looks like:\n\n```text\nqmd://concepts/note.md  #abc123\n---\n\n1: # Metrics as instruments\n2:\n3: Treat dashboards like cockpit instruments...\n```\n\nCite the docid and exact line numbers in your answer, and use the numbers to ask\nfor the next slice. Pass `--no-line-numbers` only when you need raw content to\ncopy verbatim (e.g. reproducing a code block).\n\nWhen you need to open or edit the underlying file (e.g. hand a path to `Read`,\n`Edit`, or an editor), add `--full-path`. It replaces the `qmd://` URL + docid\nheader with the document's on-disk path, falling back to the canonical header if\nthe file no longer exists on disk:\n\n```text\n$ qmd get \"#abc123\" --full-path\n/Users/you/notes/concepts/note.md\n---\n\n1: # Metrics as instruments\n```\n\n`--full-path` works the same way on `qmd search` and `qmd query`: result paths\nbecome the file's on-disk path — `./`-prefixed relative path when the file is\ninside `$PWD`, absolute realpath otherwise — and the per-result `#docid` is\ndropped because the path is the identifier. The leading `./` is intentional so\nthe output is unambiguously a filesystem path and cannot be mistaken for a bare\ncollection-relative string. Default search/query output still uses `qmd://`\nURIs; only opt into `--full-path` when you specifically need a path you can hand\nto a non-QMD tool.\n\n### Read line ranges with the `:from:count` suffix — never pipe through `sed`/`head`/`tail`\n\n`qmd get` slices files itself. Use the suffix or flags; do **not** shell out to\n`sed -n`, `head`, `tail`, or `awk` to pull a line range. Piping defeats docid\nresolution, virtual-path lookups, line numbering, and the header, and it is\nslower and more error-prone.\n\nThe most compact form is a `:from:count` suffix right on the path or docid —\nprefer it:\n\n```bash\nqmd get \"#abc123:120:40\"                  # 40 lines starting at line 120\nqmd get qmd://concepts/note.md:200:60     # lines 200–259\nqmd get \"#abc123:120\"                      # from line 120 to end of file\nqmd get \"#abc123\" --from 120 -l 40         # equivalent, using flags\n```\n\nSuffix and flags:\n\n- `<path>:<from>:<count>` — start at line `<from>`, read `<count>` lines. **Best\n  for reading around a search hit.**\n- `<path>:<from>` — start at `<from>`, read to end of file.\n- `--from <line>` / `-l <lines>` — flag equivalents. Explicit flags override the\n  suffix, so `... :5:2 -l 1` reads 1 line.\n- `--no-line-numbers` — drop the `N:` prefixes (line numbers are on by default).\n\nWrong: `qmd get \"#abc123\" | sed -n '120,160p'`\nRight: `qmd get \"#abc123:120:40\"`\n\nSearch results include a `:line` anchor on each hit — feed it straight into\n`qmd get path:line:<n>` to read a window around the match (line numbers in the\noutput will start at `line`).\n\n## Discover what is indexed\n\n```bash\nqmd collection list\nqmd ls\nqmd status\n```\n\nAdd collection filters when broad searches drift into the wrong corpus:\n\n```bash\nqmd search \"headcount autonomous agents\" -c concepts -n 10\nqmd query \"merchant support product reality\" -c concepts -c sources -n 10\n```\n\nOmit `-c` to search everything.\n\n## Filter by metadata\n\nDocuments can carry typed metadata in a `qmd.metadata` frontmatter block\n(strings, numbers, booleans, or flat arrays). `search`, `vsearch`, and `query`\naccept `--filter` with a recursive JSON AST, and every returned result satisfies\nit. Add a metadata filter when it expresses a constraint the user intends, not\nby default, and check coverage first: a condition on a key's value only sees\ndocuments that declare that key, and discovery (below) shows how many do.\n\n```bash\nqmd search \"authentication\" --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\nqmd query \"dependency injection\" --filter '{\"operator\":\"and\",\"operands\":[{\"field\":\"topics\",\"operator\":\"all\",\"value\":[\"typescript\"]},{\"field\":\"status\",\"operator\":\"nin\",\"value\":[\"draft\",\"archived\"]}]}'\n```\n\nNodes are discriminated by `operator`: groups `and`/`or` take `operands`, `not`\ntakes one `operand`, and conditions take `field` + `value` with operators\n`eq`/`ne`/`gt`/`gte`/`lt`/`lte` (comparison), `in`/`nin`/`all` (membership),\n`contains`/`prefix`/`suffix` (text), `type` (the value is a `string`, `number`,\nor `boolean`), or `exists` (presence). Matching is typed and exact; missing keys\ndo not match `ne`/`nin` (add an `exists: false` branch in an `or` group to\ninclude them). Conditions with a string value may add `\"caseInsensitive\": true`,\nwhich folds ASCII letters. The MCP `query` tool accepts the same AST as a\n`filter` object. JSON output includes each result's `metadata`.\n\n## Discover metadata before filtering\n\nDo not guess keys or values. `qmd collection show <name>` lists the top keys\nwith types and a value preview, and `qmd collection metadata` drills in. Same\nmetadata, different unit: `--filter` narrows documents (which are counted),\n`--match` narrows the metadata itself (which entries are reported). Both take\nthe predicate AST above. In a match, a condition's `field` is `\"key\"` (the\nentry's key name) or `\"value\"` (its value), and every operator applies except\n`exists` and `all`:\n\n```bash\nqmd collection metadata notes                                                                # top keys, top values\nqmd collection metadata notes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}'       # one key in depth\nqmd collection metadata notes --match '{\"field\":\"value\",\"operator\":\"eq\",\"value\":\"docs-team\"}'  # which keys hold this value\nqmd collection metadata notes --match '{\"operator\":\"and\",\"operands\":[{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"priority\"},{\"field\":\"value\",\"operator\":\"gte\",\"value\":3}]}'\nqmd collection metadata notes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}' --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\n```\n\nThe last form shows what a filter leaves behind before you commit to it in a\nquery. Read the output like this:\n\n- **The header is the decision.** `topics  string[]  388 of 480 documents\n  1,204 distinct` gives coverage and cardinality. With `--filter`, a\n  `filter: 312 of 480 documents` line opens the output and every count is out\n  of those 312. Counts are documents, not values.\n- **Quotes mark a string that reads like something else.** `\"42\"` is a string,\n  `42` a number, `\"\"` is empty. Paste a quoted value into a filter as the JSON\n  string it is.\n- **Footers mean windowed.** `N more values` or `N more keys` means the list\n  was cut. Page with `--value-offset`/`--key-offset` or raise\n  `--value-limit`/`--key-limit` rather than assuming the rest.\n- **Numbers print `min`, `median`, `max`**, enough to write a `gt`/`lt`\n  threshold in one call.\n- **`number | string` means documents disagree on type.** Each type reports its\n  own document count. Filter by the type that covers the documents you want.\n\nEvery value shown can be matched with `eq` under the same collection scope.\n\nOver MCP, call the `metadata` tool (same options, `collections` as an array)\nand read `totalKeys`, `remainingKeys`, `remainingValues`, and `range` from the\nstructured result. Page keys with `keyOffset` and values with `valueOffset`.\nThe `status` tool lists each collection's most covered keys, so check it first.\n\n## MCP Tool: `query`\n\nWhen using the MCP server, prefer structured searches:\n\n```json\n{\n  \"searches\": [\n    { \"type\": \"lex\", \"query\": \"cockpit OKR Goodhart\" },\n    { \"type\": \"vec\", \"query\": \"data informed not metric driven product judgment\" },\n    { \"type\": \"hyde\", \"query\": \"A concept note explains that metrics are useful as instruments, but leaders should not let OKRs or dashboards replace judgment.\" }\n  ],\n  \"intent\": \"Find the concept note about using metrics as instruments without becoming metric-driven.\",\n  \"collections\": [\"concepts\"],\n  \"limit\": 10\n}\n```\n\nQuery types:\n\n- `lex` — BM25 keyword search. Best for exact terms, names, titles, and code.\n- `vec` — vector semantic search. Best for natural-language concepts.\n- `hyde` — vector search using a hypothetical answer/document passage.\n\n## Query craft\n\nGood QMD searches mix three things:\n\n1. **Title/alias anchors:** exact page titles, named entities, phrases.\n2. **Semantic paraphrase:** how a human would describe the idea.\n3. **Negative space:** enough intent to avoid nearby-but-wrong concepts.\n\nExamples:\n\n```bash\n# Exact-ish title lookup\nqmd search '\"arm the rebels\" merchants tools big companies' -c concepts\n\n# Semantic concept lookup\nqmd query $'intent: Find the customer proximity concept, not generic customer delight.\\nlex: support pseudonymous merchant customer interviews\\nvec: founder stays close to merchant reality through support and product use'\n\n# Source lookup\nqmd search \"six-week cadence WhatsApp merchant relationships Shawn Ryan\" -c sources -n 10\n```\n\n## Setup and maintenance\n\nOnly mutate indexes when the user asked for setup or maintenance. Searching and\nretrieving are safe; collection/index mutation is not a casual first step.\n\n```bash\nnpm install -g @tobilu/qmd\nqmd collection add ~/notes --name notes\nqmd update\nqmd embed\n```\n\nHealth and diagnostics:\n\n```bash\nqmd doctor\nqmd status\nqmd pull\n```\n\n`qmd doctor` checks config, model cache, device/GPU setup, vector fingerprints,\nand common environment overrides. If a model-backed command fails, run it before\nchanging configuration.\n\n## MCP setup\n\nSee `references/mcp-setup.md` for Claude Code, Claude Desktop, OpenClaw, and HTTP\nserver configuration.\n\n## Pitfalls\n\n- **Do not stop at snippets.** Fetch documents before making claims.\n- **Do not slice files with `sed`/`head`/`tail`.** Use the `path:from:count`\n  suffix (e.g. `qmd get \"#abc123:120:40\"`) or `--from`/`-l`. Output is already\n  line-numbered; piping breaks docid resolution, the header, and virtual paths.\n- **Do not lean on query expansion.** Write `intent:`/`lex:`/`vec:`/`hyde:`\n  yourself. A bare `qmd query \"user sentence\"` discards the context only you\n  have. You expand the query; the model just ranks.\n- **Do not overuse semantic search.** If you know exact titles or terms, BM25 is\n  faster and often better.\n- **Do not filter blind.** Run `qmd collection metadata` first and read the\n  coverage. A key declared by 388 of 480 documents leaves 92 that no value\n  condition on it can reach; only `exists: false` selects them.\n- **Do not mutate indexes casually.** `qmd collection add`, `qmd update`, and\n  `qmd embed` change local state and can be expensive.\n- **Model-backed commands can be environment-sensitive.** If `qmd query`,\n  `qmd vsearch`, or reranking fails because local models/GPU are unavailable,\n  use `qmd search` and stronger lexical/structured terms.\n- **Ambiguous user wording needs intent.** Add `intent:` rather than hoping query\n  expansion guesses the right domain.\n- **Collection names matter.** Search `concepts` for synthesized wiki pages,\n  `sources` for transcripts/raw source pages, and docs collections for code or\n  project documentation.\n","skills/qmd/references/mcp-setup.md":"# QMD MCP Server Setup\n\n## Install\n\n```bash\nnpm install -g @tobilu/qmd\nqmd collection add ~/path/to/markdown --name myknowledge\nqmd embed\n```\n\n## Configure MCP Client\n\n**Claude Code** (`~/.claude/settings.json`):\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n  }\n}\n```\n\n**Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n  }\n}\n```\n\n**OpenClaw** (`~/.openclaw/openclaw.json`):\n```json\n{\n  \"mcp\": {\n    \"servers\": {\n      \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n    }\n  }\n}\n```\n\n## Protocol\n\nQMD speaks MCP **2026-07-28** (SDK 2.x) and dual-speaks 2025-era stdio clients.\n\n- **stdio** (`qmd mcp`): hosts still launch this as a subprocess. Opening\n  `initialize` (2025) or a `_meta`-enveloped request / `server/discover`\n  (2026) pins the connection's era.\n- **HTTP** (`qmd mcp --http`): sessionless Streamable HTTP. No\n  `Mcp-Session-Id`, no handshake. Each POST is independent. 2026 clients\n  MUST send `MCP-Protocol-Version`, `Mcp-Method`, and (for `tools/call`)\n  `Mcp-Name`. Version/caps travel in `_meta`. Call `server/discover` to\n  learn supported versions and capabilities. `tools/list` is cacheable\n  (`ttlMs` / `cacheScope`) and returns tools in a stable order.\n\n## HTTP Mode\n\n```bash\nqmd mcp --http              # Port 8181\nqmd mcp --http --daemon     # Background\nqmd mcp stop                # Stop daemon\n```\n\n`POST /mcp` is the MCP endpoint (JSON). `GET /health` is a liveness check.\nThere is no session GET stream and no idle-session TTL.\n\n## Tools\n\n### query\n\nSearch with pre-expanded queries.\n\n```json\n{\n  \"searches\": [\n    { \"type\": \"lex\", \"query\": \"keyword phrases\" },\n    { \"type\": \"vec\", \"query\": \"natural language question\" },\n    { \"type\": \"hyde\", \"query\": \"hypothetical answer passage...\" }\n  ],\n  \"limit\": 10,\n  \"collection\": \"optional\",\n  \"minScore\": 0.0\n}\n```\n\n| Type | Method | Input |\n|------|--------|-------|\n| `lex` | BM25 | Keywords (2-5 terms) |\n| `vec` | Vector | Question |\n| `hyde` | Vector | Answer passage (50-100 words) |\n\n### get\n\nRetrieve document by path or `#docid`.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `path` | string | File path or `#docid` |\n| `full` | bool? | Return full content |\n| `lineNumbers` | bool? | Add line numbers |\n\n### multi_get\n\nRetrieve multiple documents.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `pattern` | string | Glob or comma-separated list |\n| `maxBytes` | number? | Skip large files (default 64KB) |\n\n### status\n\nIndex health and collections. No params.\n\n## Troubleshooting\n\n- **Not starting**: `which qmd`, `qmd mcp` manually\n- **No results**: `qmd collection list`, `qmd embed`\n- **Slow first search**: Normal, models loading (~3GB)\n","finetune/CLAUDE.md":"# QMD Query Expansion Fine-Tuning\n\n## Overview\n\nTrain Qwen3-1.7B to expand search queries into structured `hyde:/lex:/vec:` output for QMD's hybrid retrieval pipeline.\n\n## Output Format\n\n```\nhyde: A hypothetical document passage that would answer the query.\nlex: keyword1\nlex: keyword2\nvec: semantic query reformulation\nvec: another semantic variation\n```\n\n- `hyde:` always comes FIRST (one line max)\n- `lex:` lines for BM25 keyword search (1-3 lines, short keywords)\n- `vec:` lines for vector similarity search (1-3 lines, natural language)\n\n## Training Data Format\n\n**There is exactly one JSONL format.** Every file in `data/*.jsonl` must match the strict Pydantic schema in `dataset/schema.py`:\n\n```json\n{\"query\": \"auth config\", \"output\": [[\"hyde\", \"...\"], [\"lex\", \"...\"], [\"vec\", \"...\"]]}\n```\n\n- `query`: non-empty string\n- `output`: list of `[type, text]` pairs where type is `\"lex\"`, `\"vec\"`, or `\"hyde\"`\n- Extra metadata fields (`category`, `intent`, `is_short`) are allowed but ignored\n\nThe schema is enforced by `dataset/schema.py:TrainingExample` (Pydantic model). All data loading goes through `load_examples()` which fails loudly on invalid data. No format alternatives, no legacy fallbacks.\n\n**All `.jsonl` files in `data/` are concatenated and deduplicated for training runs.** The prepared train/val files in `data/train/` are ephemeral build artifacts.\n\n## HuggingFace Repositories\n\n| Repository | Purpose |\n|------------|---------|\n| `tobil/qmd-query-expansion-1.7B` | Final merged model (SFT baseline) |\n| `tobil/qmd-query-expansion-1.7B-gguf` | GGUF quantized versions for deployment |\n| `tobil/qmd-query-expansion-1.7B-sft` | SFT adapter checkpoint (intermediate) |\n| `tobil/qmd-query-expansion-train` | Prepared training dataset |\n| `tobil/qmd-query-expansion-1.7B-grpo` | Experimental GRPO adapter (optional) |\n\n**Rules:**\n- No versioned repos (`-v1`, `-v2`, `-v4`, etc.) - update in place\n- Only push when eval scores improve over current deployed model\n- Always include eval results in model card when pushing\n\n## Dataset Tools\n\n| Script | Purpose |\n|--------|---------|\n| `dataset/schema.py` | Pydantic `TrainingExample` model + `load_examples()` |\n| `dataset/prepare_data.py` | Load via schema, apply Qwen3 chat template, dedup, split |\n| `dataset/validate_schema.py` | Validate all JSONL files against schema |\n| `dataset/score_data.py` | Score all examples using reward.py |\n| `dataset/analyze_data.py` | Analyze distribution and quality |\n\n## Training Pipeline\n\nAlways use **Qwen3-1.7B** as the base model unless explicitly stated otherwise.\n\n### Stage 0: Prepare Data\n\n```bash\nuv run dataset/prepare_data.py\n# Creates: data/train/train.jsonl, data/train/val.jsonl (ephemeral)\n```\n\n### Stage 1: SFT\n\n```bash\n# Local (requires CUDA)\nuv run train.py sft --config configs/sft.yaml\n\n# Cloud (HuggingFace Jobs)\nhf jobs uv run --flavor a10g-large --secrets HF_TOKEN --timeout 2h jobs/sft.py\n```\n\n### Stage 2: (Experimental) GRPO\n\n```bash\n# Experimental script\ncd finetune && HF_TOKEN=${HF_TOKEN} uv run python experiments/grpo/grpo.py\n```\n\n### HuggingFace Jobs\n\n```bash\nhf jobs ps                    # List running jobs\nhf jobs logs <job-id>         # Stream logs\nhf jobs inspect <job-id>      # Check status\nhf jobs cancel <job-id>       # Cancel a job\n```\n\n### Evaluation\n\n```bash\nuv run eval.py ./outputs/sft\nuv run eval.py tobil/qmd-query-expansion-1.7B\nuv run eval.py ./outputs/sft -o eval_results.json\n```\n\n## Quality Scoring\n\n`reward.py` is the single source of truth for scoring:\n\n```bash\nuv run reward.py   # Self-test\n```\n\nSee `SCORING.md` for the full rubric.\n\n## Experiments\n\nExperimental training configurations live in `experiments/`:\n\n```\nexperiments/\n├── lfm2/          # LiquidAI LFM2-1.2B (hybrid architecture, faster inference)\n│   ├── sft_lfm2.yaml\n│   └── sft_lfm2.py\n├── grpo/          # Experimental GRPO recipe and config\n│   ├── grpo.py\n│   └── grpo.yaml\n└── gepa/          # DSPy-based prompt optimization (GEPA)\n    ├── dspy_gepa.py\n    └── ...\n```\n\nThese are not part of the main training pipeline.\n\n## Key Files\n\n```\nfinetune/\n├── reward.py          # Scoring function (single source of truth)\n├── train.py           # SFT training entrypoint\n├── eval.py            # Generate and score expansions\n├── convert_gguf.py    # GGUF conversion\n├── SCORING.md         # Detailed scoring rubric\n├── CLAUDE.md          # This file\n├── Justfile           # Common commands\n├── data/              # All training JSONL files (strict schema)\n├── dataset/           # Schema + data tools (Pydantic-based)\n├── jobs/              # Self-contained HuggingFace Jobs scripts\n├── configs/           # Training configs (sft.yaml)\n├── evals/             # Test queries\n├── experiments/       # Experimental configs (LFM2, GEPA, GRPO)\n└── outputs/           # Local training outputs (gitignored)\n```\n"},"items":[{"name":"mcp-setup.md","path":"skills/qmd/references/mcp-setup.md","rawUrl":"https://raw.githubusercontent.com/tobi/qmd/HEAD/skills/qmd/references/mcp-setup.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# QMD MCP Server Setup\n\n## Install\n\n```bash\nnpm install -g @tobilu/qmd\nqmd collection add ~/path/to/markdown --name myknowledge\nqmd embed\n```\n\n## Configure MCP Client\n\n**Claude Code** (`~/.claude/settings.json`):\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n  }\n}\n```\n\n**Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n  }\n}\n```\n\n**OpenClaw** (`~/.openclaw/openclaw.json`):\n```json\n{\n  \"mcp\": {\n    \"servers\": {\n      \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n    }\n  }\n}\n```\n\n## Protocol\n\nQMD speaks MCP **2026-07-28** (SDK 2.x) and dual-speaks 2025-era stdio clients.\n\n- **stdio** (`qmd mcp`): hosts still launch this as a subprocess. Opening\n  `initialize` (2025) or a `_meta`-enveloped request / `server/discover`\n  (2026) pins the connection's era.\n- **HTTP** (`qmd mcp --http`): sessionless Streamable HTTP. No\n  `Mcp-Session-Id`, no handshake. Each POST is independent. 2026 clients\n  MUST send `MCP-Protocol-Version`, `Mcp-Method`, and (for `tools/call`)\n  `Mcp-Name`. Version/caps travel in `_meta`. Call `server/discover` to\n  learn supported versions and capabilities. `tools/list` is cacheable\n  (`ttlMs` / `cacheScope`) and returns tools in a stable order.\n\n## HTTP Mode\n\n```bash\nqmd mcp --http              # Port 8181\nqmd mcp --http --daemon     # Background\nqmd mcp stop                # Stop daemon\n```\n\n`POST /mcp` is the MCP endpoint (JSON). `GET /health` is a liveness check.\nThere is no session GET stream and no idle-session TTL.\n\n## Tools\n\n### query\n\nSearch with pre-expanded queries.\n\n```json\n{\n  \"searches\": [\n    { \"type\": \"lex\", \"query\": \"keyword phrases\" },\n    { \"type\": \"vec\", \"query\": \"natural language question\" },\n    { \"type\": \"hyde\", \"query\": \"hypothetical answer passage...\" }\n  ],\n  \"limit\": 10,\n  \"collection\": \"optional\",\n  \"minScore\": 0.0\n}\n```\n\n| Type | Method | Input |\n|------|--------|-------|\n| `lex` | BM25 | Keywords (2-5 terms) |\n| `vec` | Vector | Question |\n| `hyde` | Vector | Answer passage (50-100 words) |\n\n### get\n\nRetrieve document by path or `#docid`.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `path` | string | File path or `#docid` |\n| `full` | bool? | Return full content |\n| `lineNumbers` | bool? | Add line numbers |\n\n### multi_get\n\nRetrieve multiple documents.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `pattern` | string | Glob or comma-separated list |\n| `maxBytes` | number? | Skip large files (default 64KB) |\n\n### status\n\nIndex health and collections. No params.\n\n## Troubleshooting\n\n- **Not starting**: `which qmd`, `qmd mcp` manually\n- **No results**: `qmd collection list`, `qmd embed`\n- **Slow first search**: Normal, models loading (~3GB)\n","isInternal":false,"tokens":857,"sizeBytes":2808},{"name":"SKILL.md","path":"skills/qmd/SKILL.md","rawUrl":"https://raw.githubusercontent.com/tobi/qmd/HEAD/skills/qmd/SKILL.md","title":"qmd","category":"anthropic-skill","format":"markdown","content":"---\nname: qmd\ndescription: Search local markdown knowledge bases, notes, docs, and wikis with QMD. Use when users ask to find notes, retrieve documents, inspect a wiki, answer from indexed markdown, or set up QMD access.\nlicense: MIT\ncompatibility: Requires qmd CLI or MCP server. Install via `npm install -g @tobilu/qmd`.\nmetadata:\n  author: tobi\n  version: \"2.3.0\"\nallowed-tools: Bash(qmd:*), mcp__qmd__*\n---\n\n# QMD - Query Markdown Documents\n\n## How search works\n\nQMD searches local markdown collections: notes, docs, wikis, transcripts, and\nproject knowledge bases. Use it before web search when the answer may already be\nin indexed local files.\n\nThe workflow is always:\n\n1. Search for candidate documents.\n2. Retrieve the full source with `qmd get` or `qmd multi-get`.\n3. Answer from retrieved text, citing paths or docids.\n\nDo not answer from snippets alone when the user needs facts, decisions, quotes,\nor nuance. Snippets are only leads.\n\nTypical loop:\n\n```bash\nqmd search \"merchant reality support interviews\" -n 5\n# leads: #abc123 concepts/customer-proximity.md; #def432 sources/merchant-call.md\nqmd multi-get \"#abc123,#def432\" --format md\n```\n\n**Default to structured `qmd query` with `intent:`, `lex:`, `vec:`, and `hyde:`\nfields that you write yourself.** You are a better query expander than the\nbuilt-in model: you know the user's actual goal, the domain vocabulary, and the\nnearby-but-wrong concepts to avoid. Do not just paste the user's words into\n`qmd query \"...\"` and hope the expansion model guesses right — supply the\n`intent:` and craft the lexical and semantic terms deliberately (see\n[Pick the right search mode](#pick-the-right-search-mode)).\n\nWhen reporting what you retrieved, a compact note is enough; do not paste whole\nfiles unless needed:\n\n```text\nRetrieved:\n- #abc123 concepts/customer-proximity.md\n- #def432 sources/merchant-call.md\n```\n\n## Pick the right search mode\n\nUse **BM25 lexical search** when you know exact words, titles, names, code\nsymbols, or rare phrases:\n\n```bash\nqmd search \"cockpit OKR Goodhart\" -n 10\nqmd search '\"AI Before Headcount\"' -c concepts -n 5\n```\n\nUse **`qmd query` with structured fields** when the user describes an idea\nindirectly, uses different wording than the source, or needs conceptual recall.\n**This is the default mode — write the fields yourself rather than leaning on\nquery expansion.** Combine exact anchors with semantic recall:\n\n```bash\nqmd query $'intent: Find the concept note about metrics as instruments without letting OKRs replace judgment.\\nlex: cockpit instruments OKR Goodhart metrics judgment\\nvec: data informed not metric driven product judgment\\nhyde: A concept note says metrics are useful like cockpit instruments, but leaders should remain data-informed rather than metric-driven because OKRs and dashboards can Goodhart product judgment.'\n```\n\nStructured query fields (you author each one — do not delegate this to the\nexpansion model):\n\n- `intent:` states what you are trying to find **and what to avoid**. Always\n  supply this. It steers ranking away from nearby-but-wrong concepts.\n- `lex:` exact terms, aliases, titles, code symbols, and rare words you expect\n  in the source. This is your own keyword expansion.\n- `vec:` paraphrases the idea in natural language, in source-like wording.\n- `hyde:` describes the document or answer that would satisfy the request.\n\nYou do not need all four every time, but you should almost always write at least\n`intent:` plus one of `lex:`/`vec:`. A bare `qmd query \"the user's sentence\"`\nthrows away the context only you have and relies on the built-in expander to\nreconstruct it — prefer the structured form.\n\nIf you genuinely have nothing to expand (a single rare token, a verbatim phrase),\nthat is a job for `qmd search`, not bare `qmd query`:\n\n```bash\nqmd query --format json --explain $'intent: ...\\nlex: ...\\nvec: ...'  # inspect ranking\n```\n\nIf `qmd query` is slow or model/GPU setup fails, fall back to `qmd search` with\nbetter lexical terms.\n\n## Retrieve sources\n\nSearch results include docids like `#abc123` and `qmd://...` paths. Fetch them:\n\n```bash\nqmd get \"#abc123\"\nqmd get qmd://concepts/ai-before-headcount.md\nqmd multi-get \"#abc123,#def432\" --format md\nqmd multi-get 'concepts/{ai-before-headcount.md,data-informed-not-metric-driven.md}' --format md\nqmd multi-get 'sources/podcast-2025-*.md' -l 80\n```\n\nUse `multi-get` when comparing several hits or gathering context across pages.\n\n### Output is line-numbered and carries the docid — cite both\n\n`get` and `multi-get` are **line-numbered by default** and always print the\ndocument's `#docid` and `qmd://` path. So `get` output looks like:\n\n```text\nqmd://concepts/note.md  #abc123\n---\n\n1: # Metrics as instruments\n2:\n3: Treat dashboards like cockpit instruments...\n```\n\nCite the docid and exact line numbers in your answer, and use the numbers to ask\nfor the next slice. Pass `--no-line-numbers` only when you need raw content to\ncopy verbatim (e.g. reproducing a code block).\n\nWhen you need to open or edit the underlying file (e.g. hand a path to `Read`,\n`Edit`, or an editor), add `--full-path`. It replaces the `qmd://` URL + docid\nheader with the document's on-disk path, falling back to the canonical header if\nthe file no longer exists on disk:\n\n```text\n$ qmd get \"#abc123\" --full-path\n/Users/you/notes/concepts/note.md\n---\n\n1: # Metrics as instruments\n```\n\n`--full-path` works the same way on `qmd search` and `qmd query`: result paths\nbecome the file's on-disk path — `./`-prefixed relative path when the file is\ninside `$PWD`, absolute realpath otherwise — and the per-result `#docid` is\ndropped because the path is the identifier. The leading `./` is intentional so\nthe output is unambiguously a filesystem path and cannot be mistaken for a bare\ncollection-relative string. Default search/query output still uses `qmd://`\nURIs; only opt into `--full-path` when you specifically need a path you can hand\nto a non-QMD tool.\n\n### Read line ranges with the `:from:count` suffix — never pipe through `sed`/`head`/`tail`\n\n`qmd get` slices files itself. Use the suffix or flags; do **not** shell out to\n`sed -n`, `head`, `tail`, or `awk` to pull a line range. Piping defeats docid\nresolution, virtual-path lookups, line numbering, and the header, and it is\nslower and more error-prone.\n\nThe most compact form is a `:from:count` suffix right on the path or docid —\nprefer it:\n\n```bash\nqmd get \"#abc123:120:40\"                  # 40 lines starting at line 120\nqmd get qmd://concepts/note.md:200:60     # lines 200–259\nqmd get \"#abc123:120\"                      # from line 120 to end of file\nqmd get \"#abc123\" --from 120 -l 40         # equivalent, using flags\n```\n\nSuffix and flags:\n\n- `<path>:<from>:<count>` — start at line `<from>`, read `<count>` lines. **Best\n  for reading around a search hit.**\n- `<path>:<from>` — start at `<from>`, read to end of file.\n- `--from <line>` / `-l <lines>` — flag equivalents. Explicit flags override the\n  suffix, so `... :5:2 -l 1` reads 1 line.\n- `--no-line-numbers` — drop the `N:` prefixes (line numbers are on by default).\n\nWrong: `qmd get \"#abc123\" | sed -n '120,160p'`\nRight: `qmd get \"#abc123:120:40\"`\n\nSearch results include a `:line` anchor on each hit — feed it straight into\n`qmd get path:line:<n>` to read a window around the match (line numbers in the\noutput will start at `line`).\n\n## Discover what is indexed\n\n```bash\nqmd collection list\nqmd ls\nqmd status\n```\n\nAdd collection filters when broad searches drift into the wrong corpus:\n\n```bash\nqmd search \"headcount autonomous agents\" -c concepts -n 10\nqmd query \"merchant support product reality\" -c concepts -c sources -n 10\n```\n\nOmit `-c` to search everything.\n\n## Filter by metadata\n\nDocuments can carry typed metadata in a `qmd.metadata` frontmatter block\n(strings, numbers, booleans, or flat arrays). `search`, `vsearch`, and `query`\naccept `--filter` with a recursive JSON AST, and every returned result satisfies\nit. Add a metadata filter when it expresses a constraint the user intends, not\nby default, and check coverage first: a condition on a key's value only sees\ndocuments that declare that key, and discovery (below) shows how many do.\n\n```bash\nqmd search \"authentication\" --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\nqmd query \"dependency injection\" --filter '{\"operator\":\"and\",\"operands\":[{\"field\":\"topics\",\"operator\":\"all\",\"value\":[\"typescript\"]},{\"field\":\"status\",\"operator\":\"nin\",\"value\":[\"draft\",\"archived\"]}]}'\n```\n\nNodes are discriminated by `operator`: groups `and`/`or` take `operands`, `not`\ntakes one `operand`, and conditions take `field` + `value` with operators\n`eq`/`ne`/`gt`/`gte`/`lt`/`lte` (comparison), `in`/`nin`/`all` (membership),\n`contains`/`prefix`/`suffix` (text), `type` (the value is a `string`, `number`,\nor `boolean`), or `exists` (presence). Matching is typed and exact; missing keys\ndo not match `ne`/`nin` (add an `exists: false` branch in an `or` group to\ninclude them). Conditions with a string value may add `\"caseInsensitive\": true`,\nwhich folds ASCII letters. The MCP `query` tool accepts the same AST as a\n`filter` object. JSON output includes each result's `metadata`.\n\n## Discover metadata before filtering\n\nDo not guess keys or values. `qmd collection show <name>` lists the top keys\nwith types and a value preview, and `qmd collection metadata` drills in. Same\nmetadata, different unit: `--filter` narrows documents (which are counted),\n`--match` narrows the metadata itself (which entries are reported). Both take\nthe predicate AST above. In a match, a condition's `field` is `\"key\"` (the\nentry's key name) or `\"value\"` (its value), and every operator applies except\n`exists` and `all`:\n\n```bash\nqmd collection metadata notes                                                                # top keys, top values\nqmd collection metadata notes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}'       # one key in depth\nqmd collection metadata notes --match '{\"field\":\"value\",\"operator\":\"eq\",\"value\":\"docs-team\"}'  # which keys hold this value\nqmd collection metadata notes --match '{\"operator\":\"and\",\"operands\":[{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"priority\"},{\"field\":\"value\",\"operator\":\"gte\",\"value\":3}]}'\nqmd collection metadata notes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}' --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\n```\n\nThe last form shows what a filter leaves behind before you commit to it in a\nquery. Read the output like this:\n\n- **The header is the decision.** `topics  string[]  388 of 480 documents\n  1,204 distinct` gives coverage and cardinality. With `--filter`, a\n  `filter: 312 of 480 documents` line opens the output and every count is out\n  of those 312. Counts are documents, not values.\n- **Quotes mark a string that reads like something else.** `\"42\"` is a string,\n  `42` a number, `\"\"` is empty. Paste a quoted value into a filter as the JSON\n  string it is.\n- **Footers mean windowed.** `N more values` or `N more keys` means the list\n  was cut. Page with `--value-offset`/`--key-offset` or raise\n  `--value-limit`/`--key-limit` rather than assuming the rest.\n- **Numbers print `min`, `median`, `max`**, enough to write a `gt`/`lt`\n  threshold in one call.\n- **`number | string` means documents disagree on type.** Each type reports its\n  own document count. Filter by the type that covers the documents you want.\n\nEvery value shown can be matched with `eq` under the same collection scope.\n\nOver MCP, call the `metadata` tool (same options, `collections` as an array)\nand read `totalKeys`, `remainingKeys`, `remainingValues`, and `range` from the\nstructured result. Page keys with `keyOffset` and values with `valueOffset`.\nThe `status` tool lists each collection's most covered keys, so check it first.\n\n## MCP Tool: `query`\n\nWhen using the MCP server, prefer structured searches:\n\n```json\n{\n  \"searches\": [\n    { \"type\": \"lex\", \"query\": \"cockpit OKR Goodhart\" },\n    { \"type\": \"vec\", \"query\": \"data informed not metric driven product judgment\" },\n    { \"type\": \"hyde\", \"query\": \"A concept note explains that metrics are useful as instruments, but leaders should not let OKRs or dashboards replace judgment.\" }\n  ],\n  \"intent\": \"Find the concept note about using metrics as instruments without becoming metric-driven.\",\n  \"collections\": [\"concepts\"],\n  \"limit\": 10\n}\n```\n\nQuery types:\n\n- `lex` — BM25 keyword search. Best for exact terms, names, titles, and code.\n- `vec` — vector semantic search. Best for natural-language concepts.\n- `hyde` — vector search using a hypothetical answer/document passage.\n\n## Query craft\n\nGood QMD searches mix three things:\n\n1. **Title/alias anchors:** exact page titles, named entities, phrases.\n2. **Semantic paraphrase:** how a human would describe the idea.\n3. **Negative space:** enough intent to avoid nearby-but-wrong concepts.\n\nExamples:\n\n```bash\n# Exact-ish title lookup\nqmd search '\"arm the rebels\" merchants tools big companies' -c concepts\n\n# Semantic concept lookup\nqmd query $'intent: Find the customer proximity concept, not generic customer delight.\\nlex: support pseudonymous merchant customer interviews\\nvec: founder stays close to merchant reality through support and product use'\n\n# Source lookup\nqmd search \"six-week cadence WhatsApp merchant relationships Shawn Ryan\" -c sources -n 10\n```\n\n## Setup and maintenance\n\nOnly mutate indexes when the user asked for setup or maintenance. Searching and\nretrieving are safe; collection/index mutation is not a casual first step.\n\n```bash\nnpm install -g @tobilu/qmd\nqmd collection add ~/notes --name notes\nqmd update\nqmd embed\n```\n\nHealth and diagnostics:\n\n```bash\nqmd doctor\nqmd status\nqmd pull\n```\n\n`qmd doctor` checks config, model cache, device/GPU setup, vector fingerprints,\nand common environment overrides. If a model-backed command fails, run it before\nchanging configuration.\n\n## MCP setup\n\nSee `references/mcp-setup.md` for Claude Code, Claude Desktop, OpenClaw, and HTTP\nserver configuration.\n\n## Pitfalls\n\n- **Do not stop at snippets.** Fetch documents before making claims.\n- **Do not slice files with `sed`/`head`/`tail`.** Use the `path:from:count`\n  suffix (e.g. `qmd get \"#abc123:120:40\"`) or `--from`/`-l`. Output is already\n  line-numbered; piping breaks docid resolution, the header, and virtual paths.\n- **Do not lean on query expansion.** Write `intent:`/`lex:`/`vec:`/`hyde:`\n  yourself. A bare `qmd query \"user sentence\"` discards the context only you\n  have. You expand the query; the model just ranks.\n- **Do not overuse semantic search.** If you know exact titles or terms, BM25 is\n  faster and often better.\n- **Do not filter blind.** Run `qmd collection metadata` first and read the\n  coverage. A key declared by 388 of 480 documents leaves 92 that no value\n  condition on it can reach; only `exists: false` selects them.\n- **Do not mutate indexes casually.** `qmd collection add`, `qmd update`, and\n  `qmd embed` change local state and can be expensive.\n- **Model-backed commands can be environment-sensitive.** If `qmd query`,\n  `qmd vsearch`, or reranking fails because local models/GPU are unavailable,\n  use `qmd search` and stronger lexical/structured terms.\n- **Ambiguous user wording needs intent.** Add `intent:` rather than hoping query\n  expansion guesses the right domain.\n- **Collection names matter.** Search `concepts` for synthesized wiki pages,\n  `sources` for transcripts/raw source pages, and docs collections for code or\n  project documentation.\n","frontmatter":{"name":"qmd","description":"Search local markdown knowledge bases, notes, docs, and wikis with QMD. Use when users ask to find notes, retrieve documents, inspect a wiki, answer from indexed markdown, or set up QMD access.","license":"MIT","compatibility":"Requires qmd CLI or MCP server. Install via `npm install -g @tobilu/qmd`.","metadata":{"author":"tobi","version":"2.3.0"},"allowed-tools":"Bash(qmd:*), mcp__qmd__*","allowedTools":["Bash(qmd:*),","mcp__qmd__*"]},"allowedTools":["Bash(qmd:*),","mcp__qmd__*"],"isInternal":false,"tokens":3986,"sizeBytes":15495},{"name":"SKILL.md","path":"skills/release/SKILL.md","rawUrl":"https://raw.githubusercontent.com/tobi/qmd/HEAD/skills/release/SKILL.md","title":"release","category":"anthropic-skill","format":"markdown","content":"---\nname: release\ndescription: Manage releases for this project. Validates changelog, installs git hooks, and cuts releases. Use when user says \"/release\", \"release 1.0.5\", \"cut a release\", or asks about the release process. NOT auto-invoked by the model.\ndisable-model-invocation: true\n---\n\n# Release\n\nCut a release, validate the changelog, and ensure git hooks are installed.\n\n## Usage\n\n`/release 1.0.5` or `/release patch` (bumps patch from current version).\n\n## Process\n\nWhen the user triggers `/release <version>`:\n\n1. **Gather context** — run `skills/release/scripts/release-context.sh <version>`.\n   This silently installs git hooks and prints everything needed: version info,\n   working directory status, commits since last release, files changed, current\n   `[Unreleased]` content, and the previous release entry for style reference.\n\n2. **Commit outstanding work** — if the context shows staged, modified, or\n   untracked files that belong in this release, commit them first. Use the\n   /commit skill or make well-formed commits directly.\n\n3. **Write the changelog** — if `[Unreleased]` is empty, write it now using\n   the commits and file changes from the context output. Follow the changelog\n   standard below. Re-run the context script after committing if needed.\n\n4. **Check dependency updates** — before cutting the release, check for\n   updates to `sqlite-vec` (and platform packages), `node-llama-cpp`,\n   and `better-sqlite3`. Run `pnpm outdated` and report any available\n   updates for these packages. If updates exist, bump them (pinned, no\n   `^` ranges) and re-run tests before proceeding.\n\n5. **Cut the release** — run `scripts/release.sh <version>`. This renames\n   `[Unreleased]` → `[X.Y.Z] - date`, inserts a fresh `[Unreleased]`,\n   bumps `package.json` and the plugin version in\n   `.claude-plugin/marketplace.json` (so installed plugins see the update),\n   commits, and tags.\n\n6. **Show the final changelog** — print the full `[Unreleased]` +\n   minor series rollup via `scripts/extract-changelog.sh <version>`.\n   Ask the user to confirm before pushing.\n\n7. **Push** — after explicit confirmation, run `git push origin main --tags`.\n\n8. **Watch CI** — after the push, start a background dispatch to watch the\n   publish workflow. Use `interactive_shell` in dispatch mode with:\n   ```\n   gh run watch $(gh run list --workflow=publish.yml --limit=1 --json databaseId --jq '.[0].databaseId') --exit-status\n   ```\n   The agent will be notified when CI completes and should report the result.\n\nIf any step fails, stop and explain. Never force-push or skip validation.\n\n## Dependency Policy\n\nAll dependencies must be pinned to exact versions (no `^` or `~` ranges).\nThe lockfile ensures reproducible installs. When adding or updating any\ndependency, always use the exact version string (e.g. `\"3.18.1\"` not\n`\"^3.18.1\"`).\n\n## Changelog Standard\n\nThe changelog lives in `CHANGELOG.md` and follows [Keep a Changelog](https://keepachangelog.com/) conventions.\n\n### Heading format\n\n- `## [Unreleased]` — accumulates entries between releases\n- `## [X.Y.Z] - YYYY-MM-DD` — released versions\n\n### Structure of a release entry\n\nEach version entry has two parts:\n\n**1. Highlights (optional, 1-4 sentences of prose)**\n\nImmediately after the version heading, before any `###` section. The elevator\npitch — what would you tell someone in 30 seconds? Only for significant\nreleases; skip for small patches.\n\n```markdown\n## [1.1.0] - 2026-03-01\n\nQMD now runs on both Node.js and Bun, with up to 2.7x faster reranking\nthrough parallel contexts. GPU auto-detection replaces the unreliable\n`gpu: \"auto\"` with explicit CUDA/Metal/Vulkan probing.\n```\n\n**2. Detailed changelog (`### Changes` and `### Fixes`)**\n\n```markdown\n### Changes\n\n- Runtime: support Node.js (>=22) alongside Bun. The `qmd` wrapper\n  auto-detects a suitable install via PATH. #149 (thanks @igrigorik)\n- Performance: parallel embedding & reranking — up to 2.7x faster on\n  multi-core machines.\n\n### Fixes\n\n- Prevent VRAM waste from duplicate context creation during concurrent\n  `embedBatch` calls. #152 (thanks @jkrems)\n```\n\n### Writing guidelines\n\n- **Explain the why, not just the what.** The changelog is for users.\n- **Include numbers.** \"2.7x faster\", \"17x less memory\".\n- **Group by theme, not by file.** \"Performance\" not \"Changes to llm.ts\".\n- **Don't list every commit.** Aggregate related changes.\n- **Credit contributors:** end bullets with `#NNN (thanks @username)` for\n  external PRs. No need to credit the repo owner.\n\n### What not to include\n\n- Internal refactors with no user-visible effect\n- Dependency bumps (unless fixing a user-facing bug)\n- CI/tooling changes (unless affecting the release artifact)\n- Test additions (unless validating a fix worth mentioning)\n\n## GitHub Release Notes\n\nEach GitHub release includes the full changelog for the **minor series** back\nto x.x.0. The `scripts/extract-changelog.sh` script handles this, and the\npublish workflow (`publish.yml`) calls it to populate the GitHub release.\n\n## Git Hooks\n\nThe pre-push hook (`scripts/pre-push`) blocks `v*` tag pushes unless:\n\n1. `package.json` version matches the tag\n2. `CHANGELOG.md` has a `## [X.Y.Z] - date` entry for the version\n3. CI passed on GitHub (warns in non-interactive shells, blocks in terminals)\n\nHooks are installed silently by the context script. They can also be installed\nmanually via `skills/release/scripts/install-hooks.sh` or automatically via\n`bun install` (prepare script).\n","frontmatter":{"name":"release","description":"Manage releases for this project. Validates changelog, installs git hooks, and cuts releases. Use when user says \"/release\", \"release 1.0.5\", \"cut a release\", or asks about the release process. NOT auto-invoked by the model.","disable-model-invocation":true,"disableModelInvocation":true},"isInternal":false,"tokens":1355,"sizeBytes":5493},{"name":"CLAUDE.md","path":"CLAUDE.md","rawUrl":"https://raw.githubusercontent.com/tobi/qmd/HEAD/CLAUDE.md","title":"Claude Agent Guidelines & System Prompt","category":"claude-rule","format":"markdown","content":"# QMD - Query Markup Documents\n\nUse Bun instead of Node.js (`bun` not `node`, `bun install` not `npm install`).\n\n## Commands\n\n```sh\nqmd collection add . --name <n>   # Create/index collection\nqmd collection list               # List all collections with details\nqmd collection remove <name>      # Remove a collection by name\nqmd collection rename <old> <new> # Rename a collection\nqmd collection metadata [name...] # Discover metadata keys, types, and value counts (--match, --filter, --key-limit, --value-limit)\nqmd init                          # Create a project-local .qmd index\nqmd ls [collection[/path]]        # List collections or files in a collection\nqmd context add [path] \"text\"     # Add context for path (defaults to current dir)\nqmd context list                  # List all contexts\nqmd context check                 # Check for collections/paths missing context\nqmd context rm <path>             # Remove context\nqmd get <file>[:from[:count]]     # Get by path or docid (#abc123); optional line range\nqmd multi-get <pattern>           # Get multiple docs by glob or comma-separated list\nqmd status                        # Show index status and collections\nqmd doctor                        # Diagnose config, index, model, and device issues\nqmd update                        # Re-index collections; configured update hooks run first\nqmd trust [list|revoke]           # Approve a checked-in .qmd config's hooks/paths/models\nqmd embed                         # Generate vector embeddings (uses node-llama-cpp)\nqmd query <query>                 # Search with query expansion + reranking (recommended)\nqmd search <query>                # Full-text keyword search (BM25, no LLM)\nqmd vsearch <query>               # Vector similarity search (no reranking)\nqmd bench <fixture.json>          # Run search-quality benchmarks\nqmd mcp                           # Start MCP server (stdio transport)\nqmd mcp --http [--port N]         # Start MCP server (HTTP, default port 8181)\nqmd mcp --http --daemon           # Start as background daemon\nqmd mcp stop                      # Stop background MCP daemon\n```\n\n## Collection Management\n\n```sh\n# List all collections\nqmd collection list\n\n# Create a collection with explicit name\nqmd collection add ~/Documents/notes --name mynotes --mask '**/*.md'\n\n# Remove a collection\nqmd collection remove mynotes\n\n# Rename a collection\nqmd collection rename mynotes my-notes\n\n# Show collection details, including the top metadata keys\nqmd collection show mynotes\n\n# Discover metadata keys and values to filter on\nqmd collection metadata mynotes\nqmd collection metadata mynotes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}'\nqmd collection metadata mynotes --match '{\"field\":\"value\",\"operator\":\"eq\",\"value\":\"docs-team\"}'\nqmd collection metadata mynotes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}' --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\nqmd collection metadata mynotes --key-limit 50 --key-offset 50   # next page of keys\n\n# Set or clear the pre-update hook (runs before re-indexing on `qmd update`)\nqmd collection update-cmd mynotes 'git pull --ff-only'\nqmd collection update-cmd mynotes            # clear\n\n# Include or exclude from default (unscoped) queries\nqmd collection exclude mynotes\nqmd collection include mynotes\n\n# List all files in a collection\nqmd ls mynotes\n\n# List files with a path prefix\nqmd ls journals/2025\nqmd ls qmd://journals/2025\n```\n\n## Context Management\n\n```sh\n# Add context to current directory (auto-detects collection)\nqmd context add \"Description of these files\"\n\n# Add context to a specific path\nqmd context add /subfolder \"Description for subfolder\"\n\n# Add global context to all collections (system message)\nqmd context add / \"Always include this context\"\n\n# Add context using virtual paths\nqmd context add qmd://journals/ \"Context for entire journals collection\"\nqmd context add qmd://journals/2024 \"Journal entries from 2024\"\n\n# List all contexts\nqmd context list\n\n# Check for collections or paths without context\nqmd context check\n\n# Remove context\nqmd context rm qmd://journals/2024\nqmd context rm /  # Remove global context\n```\n\n## Document IDs (docid)\n\nEach document has a unique short ID (docid) - the first 6 characters of its content hash.\nDocids are shown in search results as `#abc123` and can be used with `get` and `multi-get`:\n\n```sh\n# Search returns docid in results\nqmd search \"query\" --json\n# Output: [{\"docid\": \"#abc123\", \"score\": 0.85, \"file\": \"docs/readme.md\", ...}]\n\n# Get document by docid\nqmd get \"#abc123\"\nqmd get abc123              # Leading # is optional\n\n# Docids also work in multi-get comma-separated lists\nqmd multi-get \"#abc123, #def456\"\n```\n\n## Options\n\n```sh\n# Search & retrieval\n-c, --collection <name>  # Restrict search to collection(s) (repeatable)\n-n <num>                 # Number of results\n--all                    # Return all matches\n--min-score <num>        # Minimum score threshold\n--full                   # Show full document content\n--intent <text>          # Describe what you're after to sharpen ranking (query)\n--no-rerank              # Skip LLM reranking (faster, lower quality)\n--full-path              # Show on-disk paths instead of qmd:// URIs\n\n# Get / multi-get\n-l <num>                 # Maximum lines per file\n--max-bytes <num>        # Skip files larger than this (default 10KB)\n--no-line-numbers        # Disable line numbers (on by default for get/multi-get)\n\n# Output format (search, query, multi-get)\n--format <kind>          # cli (default) | json | csv | md | xml | files\n                         # legacy --json/--csv/--md/--xml/--files still work as aliases\n```\n\n## Development\n\n```sh\nbun src/cli/qmd.ts <command>   # Run from source\nbun link               # Install globally as 'qmd'\n```\n\n## Tests\n\nAll tests live in `test/`. Run everything:\n\n```sh\nnpx vitest run --reporter=verbose test/\nbun test --preload ./src/test-preload.ts test/\n```\n\n## Architecture\n\n- SQLite FTS5 for full-text search (BM25)\n- sqlite-vec for vector similarity search\n- node-llama-cpp for embeddings (embeddinggemma), reranking (qwen3-reranker), and query expansion (Qwen3)\n- Reciprocal Rank Fusion (RRF) for combining results\n- Smart chunking: 900 tokens/chunk with 15% overlap, prefers markdown headings as boundaries\n- AST-aware chunking: use `--chunk-strategy auto` to chunk code files (.ts/.js/.py/.go/.rs) at function/class/import boundaries via tree-sitter. Default is `regex` (existing behavior). Markdown and unknown file types always use regex chunking.\n\n## Important: Do NOT run automatically\n\n- Never run `qmd collection add`, `qmd embed`, or `qmd update` automatically\n- Never modify the SQLite database directly\n- Write out example commands for the user to run manually\n- Index is stored at `~/.cache/qmd/index.sqlite`\n\n## Do NOT compile\n\n- Never run `bun build --compile` - it overwrites the shell wrapper and breaks sqlite-vec\n- The `qmd` file is a shell script that runs compiled JS from `dist/` - do not replace it\n- `npm run build` compiles TypeScript to `dist/` via `tsc -p tsconfig.build.json`\n\n## Releasing\n\nUse `/release <version>` to cut a release. Full changelog standards,\nrelease workflow, and git hook setup are documented in the\n[release skill](skills/release/SKILL.md).\n\nKey points:\n- Add changelog entries under `## [Unreleased]` **as you make changes**\n- The release script renames `[Unreleased]` → `[X.Y.Z] - date` at release time\n- Credit external PRs with `#NNN (thanks @username)`\n- GitHub releases roll up the full minor series (e.g. 1.2.0 through 1.2.3)\n","isInternal":false,"tokens":1884,"sizeBytes":7566},{"name":"CLAUDE.md","path":"finetune/CLAUDE.md","rawUrl":"https://raw.githubusercontent.com/tobi/qmd/HEAD/finetune/CLAUDE.md","title":"finetune - Claude Agent Guidelines & System Prompt","category":"claude-rule","format":"markdown","content":"# QMD Query Expansion Fine-Tuning\n\n## Overview\n\nTrain Qwen3-1.7B to expand search queries into structured `hyde:/lex:/vec:` output for QMD's hybrid retrieval pipeline.\n\n## Output Format\n\n```\nhyde: A hypothetical document passage that would answer the query.\nlex: keyword1\nlex: keyword2\nvec: semantic query reformulation\nvec: another semantic variation\n```\n\n- `hyde:` always comes FIRST (one line max)\n- `lex:` lines for BM25 keyword search (1-3 lines, short keywords)\n- `vec:` lines for vector similarity search (1-3 lines, natural language)\n\n## Training Data Format\n\n**There is exactly one JSONL format.** Every file in `data/*.jsonl` must match the strict Pydantic schema in `dataset/schema.py`:\n\n```json\n{\"query\": \"auth config\", \"output\": [[\"hyde\", \"...\"], [\"lex\", \"...\"], [\"vec\", \"...\"]]}\n```\n\n- `query`: non-empty string\n- `output`: list of `[type, text]` pairs where type is `\"lex\"`, `\"vec\"`, or `\"hyde\"`\n- Extra metadata fields (`category`, `intent`, `is_short`) are allowed but ignored\n\nThe schema is enforced by `dataset/schema.py:TrainingExample` (Pydantic model). All data loading goes through `load_examples()` which fails loudly on invalid data. No format alternatives, no legacy fallbacks.\n\n**All `.jsonl` files in `data/` are concatenated and deduplicated for training runs.** The prepared train/val files in `data/train/` are ephemeral build artifacts.\n\n## HuggingFace Repositories\n\n| Repository | Purpose |\n|------------|---------|\n| `tobil/qmd-query-expansion-1.7B` | Final merged model (SFT baseline) |\n| `tobil/qmd-query-expansion-1.7B-gguf` | GGUF quantized versions for deployment |\n| `tobil/qmd-query-expansion-1.7B-sft` | SFT adapter checkpoint (intermediate) |\n| `tobil/qmd-query-expansion-train` | Prepared training dataset |\n| `tobil/qmd-query-expansion-1.7B-grpo` | Experimental GRPO adapter (optional) |\n\n**Rules:**\n- No versioned repos (`-v1`, `-v2`, `-v4`, etc.) - update in place\n- Only push when eval scores improve over current deployed model\n- Always include eval results in model card when pushing\n\n## Dataset Tools\n\n| Script | Purpose |\n|--------|---------|\n| `dataset/schema.py` | Pydantic `TrainingExample` model + `load_examples()` |\n| `dataset/prepare_data.py` | Load via schema, apply Qwen3 chat template, dedup, split |\n| `dataset/validate_schema.py` | Validate all JSONL files against schema |\n| `dataset/score_data.py` | Score all examples using reward.py |\n| `dataset/analyze_data.py` | Analyze distribution and quality |\n\n## Training Pipeline\n\nAlways use **Qwen3-1.7B** as the base model unless explicitly stated otherwise.\n\n### Stage 0: Prepare Data\n\n```bash\nuv run dataset/prepare_data.py\n# Creates: data/train/train.jsonl, data/train/val.jsonl (ephemeral)\n```\n\n### Stage 1: SFT\n\n```bash\n# Local (requires CUDA)\nuv run train.py sft --config configs/sft.yaml\n\n# Cloud (HuggingFace Jobs)\nhf jobs uv run --flavor a10g-large --secrets HF_TOKEN --timeout 2h jobs/sft.py\n```\n\n### Stage 2: (Experimental) GRPO\n\n```bash\n# Experimental script\ncd finetune && HF_TOKEN=${HF_TOKEN} uv run python experiments/grpo/grpo.py\n```\n\n### HuggingFace Jobs\n\n```bash\nhf jobs ps                    # List running jobs\nhf jobs logs <job-id>         # Stream logs\nhf jobs inspect <job-id>      # Check status\nhf jobs cancel <job-id>       # Cancel a job\n```\n\n### Evaluation\n\n```bash\nuv run eval.py ./outputs/sft\nuv run eval.py tobil/qmd-query-expansion-1.7B\nuv run eval.py ./outputs/sft -o eval_results.json\n```\n\n## Quality Scoring\n\n`reward.py` is the single source of truth for scoring:\n\n```bash\nuv run reward.py   # Self-test\n```\n\nSee `SCORING.md` for the full rubric.\n\n## Experiments\n\nExperimental training configurations live in `experiments/`:\n\n```\nexperiments/\n├── lfm2/          # LiquidAI LFM2-1.2B (hybrid architecture, faster inference)\n│   ├── sft_lfm2.yaml\n│   └── sft_lfm2.py\n├── grpo/          # Experimental GRPO recipe and config\n│   ├── grpo.py\n│   └── grpo.yaml\n└── gepa/          # DSPy-based prompt optimization (GEPA)\n    ├── dspy_gepa.py\n    └── ...\n```\n\nThese are not part of the main training pipeline.\n\n## Key Files\n\n```\nfinetune/\n├── reward.py          # Scoring function (single source of truth)\n├── train.py           # SFT training entrypoint\n├── eval.py            # Generate and score expansions\n├── convert_gguf.py    # GGUF conversion\n├── SCORING.md         # Detailed scoring rubric\n├── CLAUDE.md          # This file\n├── Justfile           # Common commands\n├── data/              # All training JSONL files (strict schema)\n├── dataset/           # Schema + data tools (Pydantic-based)\n├── jobs/              # Self-contained HuggingFace Jobs scripts\n├── configs/           # Training configs (sft.yaml)\n├── evals/             # Test queries\n├── experiments/       # Experimental configs (LFM2, GEPA, GRPO)\n└── outputs/           # Local training outputs (gitignored)\n```\n","isInternal":false,"tokens":1323,"sizeBytes":4976},{"name":"marketplace.json","path":".claude-plugin/marketplace.json","rawUrl":"https://raw.githubusercontent.com/tobi/qmd/HEAD/.claude-plugin/marketplace.json","title":"qmd","category":"marketplace","format":"json","content":"{\n\t\"name\": \"qmd\",\n\t\"owner\": {\n\t\t\"name\": \"tobi\",\n\t\t\"email\": \"tobi@lutke.com\"\n\t},\n\t\"plugins\": [\n\t\t{\n\t\t\t\"name\": \"qmd\",\n\t\t\t\"source\": \"./skills\",\n\t\t\t\"description\": \"Search and retrieve documents from local markdown files.\",\n\t\t\t\"version\": \"2.8.3\",\n\t\t\t\"author\": {\n\t\t\t\t\"name\": \"tobi\",\n\t\t\t\t\"email\": \"tobi@lutke.com\"\n\t\t\t},\n\t\t\t\"repository\": \"https://github.com/tobi/qmd\",\n\t\t\t\"license\": \"MIT\",\n\t\t\t\"keywords\": [\"markdown\", \"search\", \"qmd\"],\n\t\t\t\"skills\": [\"./qmd\", \"./release\"],\n\t\t\t\"mcpServers\": {\n\t\t\t\t\"qmd\": {\n\t\t\t\t\t\"command\": \"qmd\",\n\t\t\t\t\t\"args\": [\"mcp\"]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t]\n}\n","frontmatter":{"name":"qmd","owner":{"name":"tobi","email":"tobi@lutke.com"},"plugins":[{"name":"qmd","source":"./skills","description":"Search and retrieve documents from local markdown files.","version":"2.8.3","author":{"name":"tobi","email":"tobi@lutke.com"},"repository":"https://github.com/tobi/qmd","license":"MIT","keywords":["markdown","search","qmd"],"skills":["./qmd","./release"],"mcpServers":{"qmd":{"command":"qmd","args":["mcp"]}}}]},"isInternal":false,"tokens":212,"sizeBytes":561}],"systemPromptSnippet":"<yakaai_skills repo=\"tobi/qmd\">\n<!-- File: skills/qmd/references/mcp-setup.md (Tokens: ~857 | Category: anthropic-skill) -->\n# QMD MCP Server Setup\n\n## Install\n\n```bash\nnpm install -g @tobilu/qmd\nqmd collection add ~/path/to/markdown --name myknowledge\nqmd embed\n```\n\n## Configure MCP Client\n\n**Claude Code** (`~/.claude/settings.json`):\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n  }\n}\n```\n\n**Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n  }\n}\n```\n\n**OpenClaw** (`~/.openclaw/openclaw.json`):\n```json\n{\n  \"mcp\": {\n    \"servers\": {\n      \"qmd\": { \"command\": \"qmd\", \"args\": [\"mcp\"] }\n    }\n  }\n}\n```\n\n## Protocol\n\nQMD speaks MCP **2026-07-28** (SDK 2.x) and dual-speaks 2025-era stdio clients.\n\n- **stdio** (`qmd mcp`): hosts still launch this as a subprocess. Opening\n  `initialize` (2025) or a `_meta`-enveloped request / `server/discover`\n  (2026) pins the connection's era.\n- **HTTP** (`qmd mcp --http`): sessionless Streamable HTTP. No\n  `Mcp-Session-Id`, no handshake. Each POST is independent. 2026 clients\n  MUST send `MCP-Protocol-Version`, `Mcp-Method`, and (for `tools/call`)\n  `Mcp-Name`. Version/caps travel in `_meta`. Call `server/discover` to\n  learn supported versions and capabilities. `tools/list` is cacheable\n  (`ttlMs` / `cacheScope`) and returns tools in a stable order.\n\n## HTTP Mode\n\n```bash\nqmd mcp --http              # Port 8181\nqmd mcp --http --daemon     # Background\nqmd mcp stop                # Stop daemon\n```\n\n`POST /mcp` is the MCP endpoint (JSON). `GET /health` is a liveness check.\nThere is no session GET stream and no idle-session TTL.\n\n## Tools\n\n### query\n\nSearch with pre-expanded queries.\n\n```json\n{\n  \"searches\": [\n    { \"type\": \"lex\", \"query\": \"keyword phrases\" },\n    { \"type\": \"vec\", \"query\": \"natural language question\" },\n    { \"type\": \"hyde\", \"query\": \"hypothetical answer passage...\" }\n  ],\n  \"limit\": 10,\n  \"collection\": \"optional\",\n  \"minScore\": 0.0\n}\n```\n\n| Type | Method | Input |\n|------|--------|-------|\n| `lex` | BM25 | Keywords (2-5 terms) |\n| `vec` | Vector | Question |\n| `hyde` | Vector | Answer passage (50-100 words) |\n\n### get\n\nRetrieve document by path or `#docid`.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `path` | string | File path or `#docid` |\n| `full` | bool? | Return full content |\n| `lineNumbers` | bool? | Add line numbers |\n\n### multi_get\n\nRetrieve multiple documents.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `pattern` | string | Glob or comma-separated list |\n| `maxBytes` | number? | Skip large files (default 64KB) |\n\n### status\n\nIndex health and collections. No params.\n\n## Troubleshooting\n\n- **Not starting**: `which qmd`, `qmd mcp` manually\n- **No results**: `qmd collection list`, `qmd embed`\n- **Slow first search**: Normal, models loading (~3GB)\n\n<!-- File: skills/qmd/SKILL.md (Tokens: ~3986 | Category: anthropic-skill) -->\n---\nname: qmd\ndescription: Search local markdown knowledge bases, notes, docs, and wikis with QMD. Use when users ask to find notes, retrieve documents, inspect a wiki, answer from indexed markdown, or set up QMD access.\nlicense: MIT\ncompatibility: Requires qmd CLI or MCP server. Install via `npm install -g @tobilu/qmd`.\nmetadata:\n  author: tobi\n  version: \"2.3.0\"\nallowed-tools: Bash(qmd:*), mcp__qmd__*\n---\n\n# QMD - Query Markdown Documents\n\n## How search works\n\nQMD searches local markdown collections: notes, docs, wikis, transcripts, and\nproject knowledge bases. Use it before web search when the answer may already be\nin indexed local files.\n\nThe workflow is always:\n\n1. Search for candidate documents.\n2. Retrieve the full source with `qmd get` or `qmd multi-get`.\n3. Answer from retrieved text, citing paths or docids.\n\nDo not answer from snippets alone when the user needs facts, decisions, quotes,\nor nuance. Snippets are only leads.\n\nTypical loop:\n\n```bash\nqmd search \"merchant reality support interviews\" -n 5\n# leads: #abc123 concepts/customer-proximity.md; #def432 sources/merchant-call.md\nqmd multi-get \"#abc123,#def432\" --format md\n```\n\n**Default to structured `qmd query` with `intent:`, `lex:`, `vec:`, and `hyde:`\nfields that you write yourself.** You are a better query expander than the\nbuilt-in model: you know the user's actual goal, the domain vocabulary, and the\nnearby-but-wrong concepts to avoid. Do not just paste the user's words into\n`qmd query \"...\"` and hope the expansion model guesses right — supply the\n`intent:` and craft the lexical and semantic terms deliberately (see\n[Pick the right search mode](#pick-the-right-search-mode)).\n\nWhen reporting what you retrieved, a compact note is enough; do not paste whole\nfiles unless needed:\n\n```text\nRetrieved:\n- #abc123 concepts/customer-proximity.md\n- #def432 sources/merchant-call.md\n```\n\n## Pick the right search mode\n\nUse **BM25 lexical search** when you know exact words, titles, names, code\nsymbols, or rare phrases:\n\n```bash\nqmd search \"cockpit OKR Goodhart\" -n 10\nqmd search '\"AI Before Headcount\"' -c concepts -n 5\n```\n\nUse **`qmd query` with structured fields** when the user describes an idea\nindirectly, uses different wording than the source, or needs conceptual recall.\n**This is the default mode — write the fields yourself rather than leaning on\nquery expansion.** Combine exact anchors with semantic recall:\n\n```bash\nqmd query $'intent: Find the concept note about metrics as instruments without letting OKRs replace judgment.\\nlex: cockpit instruments OKR Goodhart metrics judgment\\nvec: data informed not metric driven product judgment\\nhyde: A concept note says metrics are useful like cockpit instruments, but leaders should remain data-informed rather than metric-driven because OKRs and dashboards can Goodhart product judgment.'\n```\n\nStructured query fields (you author each one — do not delegate this to the\nexpansion model):\n\n- `intent:` states what you are trying to find **and what to avoid**. Always\n  supply this. It steers ranking away from nearby-but-wrong concepts.\n- `lex:` exact terms, aliases, titles, code symbols, and rare words you expect\n  in the source. This is your own keyword expansion.\n- `vec:` paraphrases the idea in natural language, in source-like wording.\n- `hyde:` describes the document or answer that would satisfy the request.\n\nYou do not need all four every time, but you should almost always write at least\n`intent:` plus one of `lex:`/`vec:`. A bare `qmd query \"the user's sentence\"`\nthrows away the context only you have and relies on the built-in expander to\nreconstruct it — prefer the structured form.\n\nIf you genuinely have nothing to expand (a single rare token, a verbatim phrase),\nthat is a job for `qmd search`, not bare `qmd query`:\n\n```bash\nqmd query --format json --explain $'intent: ...\\nlex: ...\\nvec: ...'  # inspect ranking\n```\n\nIf `qmd query` is slow or model/GPU setup fails, fall back to `qmd search` with\nbetter lexical terms.\n\n## Retrieve sources\n\nSearch results include docids like `#abc123` and `qmd://...` paths. Fetch them:\n\n```bash\nqmd get \"#abc123\"\nqmd get qmd://concepts/ai-before-headcount.md\nqmd multi-get \"#abc123,#def432\" --format md\nqmd multi-get 'concepts/{ai-before-headcount.md,data-informed-not-metric-driven.md}' --format md\nqmd multi-get 'sources/podcast-2025-*.md' -l 80\n```\n\nUse `multi-get` when comparing several hits or gathering context across pages.\n\n### Output is line-numbered and carries the docid — cite both\n\n`get` and `multi-get` are **line-numbered by default** and always print the\ndocument's `#docid` and `qmd://` path. So `get` output looks like:\n\n```text\nqmd://concepts/note.md  #abc123\n---\n\n1: # Metrics as instruments\n2:\n3: Treat dashboards like cockpit instruments...\n```\n\nCite the docid and exact line numbers in your answer, and use the numbers to ask\nfor the next slice. Pass `--no-line-numbers` only when you need raw content to\ncopy verbatim (e.g. reproducing a code block).\n\nWhen you need to open or edit the underlying file (e.g. hand a path to `Read`,\n`Edit`, or an editor), add `--full-path`. It replaces the `qmd://` URL + docid\nheader with the document's on-disk path, falling back to the canonical header if\nthe file no longer exists on disk:\n\n```text\n$ qmd get \"#abc123\" --full-path\n/Users/you/notes/concepts/note.md\n---\n\n1: # Metrics as instruments\n```\n\n`--full-path` works the same way on `qmd search` and `qmd query`: result paths\nbecome the file's on-disk path — `./`-prefixed relative path when the file is\ninside `$PWD`, absolute realpath otherwise — and the per-result `#docid` is\ndropped because the path is the identifier. The leading `./` is intentional so\nthe output is unambiguously a filesystem path and cannot be mistaken for a bare\ncollection-relative string. Default search/query output still uses `qmd://`\nURIs; only opt into `--full-path` when you specifically need a path you can hand\nto a non-QMD tool.\n\n### Read line ranges with the `:from:count` suffix — never pipe through `sed`/`head`/`tail`\n\n`qmd get` slices files itself. Use the suffix or flags; do **not** shell out to\n`sed -n`, `head`, `tail`, or `awk` to pull a line range. Piping defeats docid\nresolution, virtual-path lookups, line numbering, and the header, and it is\nslower and more error-prone.\n\nThe most compact form is a `:from:count` suffix right on the path or docid —\nprefer it:\n\n```bash\nqmd get \"#abc123:120:40\"                  # 40 lines starting at line 120\nqmd get qmd://concepts/note.md:200:60     # lines 200–259\nqmd get \"#abc123:120\"                      # from line 120 to end of file\nqmd get \"#abc123\" --from 120 -l 40         # equivalent, using flags\n```\n\nSuffix and flags:\n\n- `<path>:<from>:<count>` — start at line `<from>`, read `<count>` lines. **Best\n  for reading around a search hit.**\n- `<path>:<from>` — start at `<from>`, read to end of file.\n- `--from <line>` / `-l <lines>` — flag equivalents. Explicit flags override the\n  suffix, so `... :5:2 -l 1` reads 1 line.\n- `--no-line-numbers` — drop the `N:` prefixes (line numbers are on by default).\n\nWrong: `qmd get \"#abc123\" | sed -n '120,160p'`\nRight: `qmd get \"#abc123:120:40\"`\n\nSearch results include a `:line` anchor on each hit — feed it straight into\n`qmd get path:line:<n>` to read a window around the match (line numbers in the\noutput will start at `line`).\n\n## Discover what is indexed\n\n```bash\nqmd collection list\nqmd ls\nqmd status\n```\n\nAdd collection filters when broad searches drift into the wrong corpus:\n\n```bash\nqmd search \"headcount autonomous agents\" -c concepts -n 10\nqmd query \"merchant support product reality\" -c concepts -c sources -n 10\n```\n\nOmit `-c` to search everything.\n\n## Filter by metadata\n\nDocuments can carry typed metadata in a `qmd.metadata` frontmatter block\n(strings, numbers, booleans, or flat arrays). `search`, `vsearch`, and `query`\naccept `--filter` with a recursive JSON AST, and every returned result satisfies\nit. Add a metadata filter when it expresses a constraint the user intends, not\nby default, and check coverage first: a condition on a key's value only sees\ndocuments that declare that key, and discovery (below) shows how many do.\n\n```bash\nqmd search \"authentication\" --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\nqmd query \"dependency injection\" --filter '{\"operator\":\"and\",\"operands\":[{\"field\":\"topics\",\"operator\":\"all\",\"value\":[\"typescript\"]},{\"field\":\"status\",\"operator\":\"nin\",\"value\":[\"draft\",\"archived\"]}]}'\n```\n\nNodes are discriminated by `operator`: groups `and`/`or` take `operands`, `not`\ntakes one `operand`, and conditions take `field` + `value` with operators\n`eq`/`ne`/`gt`/`gte`/`lt`/`lte` (comparison), `in`/`nin`/`all` (membership),\n`contains`/`prefix`/`suffix` (text), `type` (the value is a `string`, `number`,\nor `boolean`), or `exists` (presence). Matching is typed and exact; missing keys\ndo not match `ne`/`nin` (add an `exists: false` branch in an `or` group to\ninclude them). Conditions with a string value may add `\"caseInsensitive\": true`,\nwhich folds ASCII letters. The MCP `query` tool accepts the same AST as a\n`filter` object. JSON output includes each result's `metadata`.\n\n## Discover metadata before filtering\n\nDo not guess keys or values. `qmd collection show <name>` lists the top keys\nwith types and a value preview, and `qmd collection metadata` drills in. Same\nmetadata, different unit: `--filter` narrows documents (which are counted),\n`--match` narrows the metadata itself (which entries are reported). Both take\nthe predicate AST above. In a match, a condition's `field` is `\"key\"` (the\nentry's key name) or `\"value\"` (its value), and every operator applies except\n`exists` and `all`:\n\n```bash\nqmd collection metadata notes                                                                # top keys, top values\nqmd collection metadata notes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}'       # one key in depth\nqmd collection metadata notes --match '{\"field\":\"value\",\"operator\":\"eq\",\"value\":\"docs-team\"}'  # which keys hold this value\nqmd collection metadata notes --match '{\"operator\":\"and\",\"operands\":[{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"priority\"},{\"field\":\"value\",\"operator\":\"gte\",\"value\":3}]}'\nqmd collection metadata notes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}' --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\n```\n\nThe last form shows what a filter leaves behind before you commit to it in a\nquery. Read the output like this:\n\n- **The header is the decision.** `topics  string[]  388 of 480 documents\n  1,204 distinct` gives coverage and cardinality. With `--filter`, a\n  `filter: 312 of 480 documents` line opens the output and every count is out\n  of those 312. Counts are documents, not values.\n- **Quotes mark a string that reads like something else.** `\"42\"` is a string,\n  `42` a number, `\"\"` is empty. Paste a quoted value into a filter as the JSON\n  string it is.\n- **Footers mean windowed.** `N more values` or `N more keys` means the list\n  was cut. Page with `--value-offset`/`--key-offset` or raise\n  `--value-limit`/`--key-limit` rather than assuming the rest.\n- **Numbers print `min`, `median`, `max`**, enough to write a `gt`/`lt`\n  threshold in one call.\n- **`number | string` means documents disagree on type.** Each type reports its\n  own document count. Filter by the type that covers the documents you want.\n\nEvery value shown can be matched with `eq` under the same collection scope.\n\nOver MCP, call the `metadata` tool (same options, `collections` as an array)\nand read `totalKeys`, `remainingKeys`, `remainingValues`, and `range` from the\nstructured result. Page keys with `keyOffset` and values with `valueOffset`.\nThe `status` tool lists each collection's most covered keys, so check it first.\n\n## MCP Tool: `query`\n\nWhen using the MCP server, prefer structured searches:\n\n```json\n{\n  \"searches\": [\n    { \"type\": \"lex\", \"query\": \"cockpit OKR Goodhart\" },\n    { \"type\": \"vec\", \"query\": \"data informed not metric driven product judgment\" },\n    { \"type\": \"hyde\", \"query\": \"A concept note explains that metrics are useful as instruments, but leaders should not let OKRs or dashboards replace judgment.\" }\n  ],\n  \"intent\": \"Find the concept note about using metrics as instruments without becoming metric-driven.\",\n  \"collections\": [\"concepts\"],\n  \"limit\": 10\n}\n```\n\nQuery types:\n\n- `lex` — BM25 keyword search. Best for exact terms, names, titles, and code.\n- `vec` — vector semantic search. Best for natural-language concepts.\n- `hyde` — vector search using a hypothetical answer/document passage.\n\n## Query craft\n\nGood QMD searches mix three things:\n\n1. **Title/alias anchors:** exact page titles, named entities, phrases.\n2. **Semantic paraphrase:** how a human would describe the idea.\n3. **Negative space:** enough intent to avoid nearby-but-wrong concepts.\n\nExamples:\n\n```bash\n# Exact-ish title lookup\nqmd search '\"arm the rebels\" merchants tools big companies' -c concepts\n\n# Semantic concept lookup\nqmd query $'intent: Find the customer proximity concept, not generic customer delight.\\nlex: support pseudonymous merchant customer interviews\\nvec: founder stays close to merchant reality through support and product use'\n\n# Source lookup\nqmd search \"six-week cadence WhatsApp merchant relationships Shawn Ryan\" -c sources -n 10\n```\n\n## Setup and maintenance\n\nOnly mutate indexes when the user asked for setup or maintenance. Searching and\nretrieving are safe; collection/index mutation is not a casual first step.\n\n```bash\nnpm install -g @tobilu/qmd\nqmd collection add ~/notes --name notes\nqmd update\nqmd embed\n```\n\nHealth and diagnostics:\n\n```bash\nqmd doctor\nqmd status\nqmd pull\n```\n\n`qmd doctor` checks config, model cache, device/GPU setup, vector fingerprints,\nand common environment overrides. If a model-backed command fails, run it before\nchanging configuration.\n\n## MCP setup\n\nSee `references/mcp-setup.md` for Claude Code, Claude Desktop, OpenClaw, and HTTP\nserver configuration.\n\n## Pitfalls\n\n- **Do not stop at snippets.** Fetch documents before making claims.\n- **Do not slice files with `sed`/`head`/`tail`.** Use the `path:from:count`\n  suffix (e.g. `qmd get \"#abc123:120:40\"`) or `--from`/`-l`. Output is already\n  line-numbered; piping breaks docid resolution, the header, and virtual paths.\n- **Do not lean on query expansion.** Write `intent:`/`lex:`/`vec:`/`hyde:`\n  yourself. A bare `qmd query \"user sentence\"` discards the context only you\n  have. You expand the query; the model just ranks.\n- **Do not overuse semantic search.** If you know exact titles or terms, BM25 is\n  faster and often better.\n- **Do not filter blind.** Run `qmd collection metadata` first and read the\n  coverage. A key declared by 388 of 480 documents leaves 92 that no value\n  condition on it can reach; only `exists: false` selects them.\n- **Do not mutate indexes casually.** `qmd collection add`, `qmd update`, and\n  `qmd embed` change local state and can be expensive.\n- **Model-backed commands can be environment-sensitive.** If `qmd query`,\n  `qmd vsearch`, or reranking fails because local models/GPU are unavailable,\n  use `qmd search` and stronger lexical/structured terms.\n- **Ambiguous user wording needs intent.** Add `intent:` rather than hoping query\n  expansion guesses the right domain.\n- **Collection names matter.** Search `concepts` for synthesized wiki pages,\n  `sources` for transcripts/raw source pages, and docs collections for code or\n  project documentation.\n\n<!-- File: skills/release/SKILL.md (Tokens: ~1355 | Category: anthropic-skill) -->\n---\nname: release\ndescription: Manage releases for this project. Validates changelog, installs git hooks, and cuts releases. Use when user says \"/release\", \"release 1.0.5\", \"cut a release\", or asks about the release process. NOT auto-invoked by the model.\ndisable-model-invocation: true\n---\n\n# Release\n\nCut a release, validate the changelog, and ensure git hooks are installed.\n\n## Usage\n\n`/release 1.0.5` or `/release patch` (bumps patch from current version).\n\n## Process\n\nWhen the user triggers `/release <version>`:\n\n1. **Gather context** — run `skills/release/scripts/release-context.sh <version>`.\n   This silently installs git hooks and prints everything needed: version info,\n   working directory status, commits since last release, files changed, current\n   `[Unreleased]` content, and the previous release entry for style reference.\n\n2. **Commit outstanding work** — if the context shows staged, modified, or\n   untracked files that belong in this release, commit them first. Use the\n   /commit skill or make well-formed commits directly.\n\n3. **Write the changelog** — if `[Unreleased]` is empty, write it now using\n   the commits and file changes from the context output. Follow the changelog\n   standard below. Re-run the context script after committing if needed.\n\n4. **Check dependency updates** — before cutting the release, check for\n   updates to `sqlite-vec` (and platform packages), `node-llama-cpp`,\n   and `better-sqlite3`. Run `pnpm outdated` and report any available\n   updates for these packages. If updates exist, bump them (pinned, no\n   `^` ranges) and re-run tests before proceeding.\n\n5. **Cut the release** — run `scripts/release.sh <version>`. This renames\n   `[Unreleased]` → `[X.Y.Z] - date`, inserts a fresh `[Unreleased]`,\n   bumps `package.json` and the plugin version in\n   `.claude-plugin/marketplace.json` (so installed plugins see the update),\n   commits, and tags.\n\n6. **Show the final changelog** — print the full `[Unreleased]` +\n   minor series rollup via `scripts/extract-changelog.sh <version>`.\n   Ask the user to confirm before pushing.\n\n7. **Push** — after explicit confirmation, run `git push origin main --tags`.\n\n8. **Watch CI** — after the push, start a background dispatch to watch the\n   publish workflow. Use `interactive_shell` in dispatch mode with:\n   ```\n   gh run watch $(gh run list --workflow=publish.yml --limit=1 --json databaseId --jq '.[0].databaseId') --exit-status\n   ```\n   The agent will be notified when CI completes and should report the result.\n\nIf any step fails, stop and explain. Never force-push or skip validation.\n\n## Dependency Policy\n\nAll dependencies must be pinned to exact versions (no `^` or `~` ranges).\nThe lockfile ensures reproducible installs. When adding or updating any\ndependency, always use the exact version string (e.g. `\"3.18.1\"` not\n`\"^3.18.1\"`).\n\n## Changelog Standard\n\nThe changelog lives in `CHANGELOG.md` and follows [Keep a Changelog](https://keepachangelog.com/) conventions.\n\n### Heading format\n\n- `## [Unreleased]` — accumulates entries between releases\n- `## [X.Y.Z] - YYYY-MM-DD` — released versions\n\n### Structure of a release entry\n\nEach version entry has two parts:\n\n**1. Highlights (optional, 1-4 sentences of prose)**\n\nImmediately after the version heading, before any `###` section. The elevator\npitch — what would you tell someone in 30 seconds? Only for significant\nreleases; skip for small patches.\n\n```markdown\n## [1.1.0] - 2026-03-01\n\nQMD now runs on both Node.js and Bun, with up to 2.7x faster reranking\nthrough parallel contexts. GPU auto-detection replaces the unreliable\n`gpu: \"auto\"` with explicit CUDA/Metal/Vulkan probing.\n```\n\n**2. Detailed changelog (`### Changes` and `### Fixes`)**\n\n```markdown\n### Changes\n\n- Runtime: support Node.js (>=22) alongside Bun. The `qmd` wrapper\n  auto-detects a suitable install via PATH. #149 (thanks @igrigorik)\n- Performance: parallel embedding & reranking — up to 2.7x faster on\n  multi-core machines.\n\n### Fixes\n\n- Prevent VRAM waste from duplicate context creation during concurrent\n  `embedBatch` calls. #152 (thanks @jkrems)\n```\n\n### Writing guidelines\n\n- **Explain the why, not just the what.** The changelog is for users.\n- **Include numbers.** \"2.7x faster\", \"17x less memory\".\n- **Group by theme, not by file.** \"Performance\" not \"Changes to llm.ts\".\n- **Don't list every commit.** Aggregate related changes.\n- **Credit contributors:** end bullets with `#NNN (thanks @username)` for\n  external PRs. No need to credit the repo owner.\n\n### What not to include\n\n- Internal refactors with no user-visible effect\n- Dependency bumps (unless fixing a user-facing bug)\n- CI/tooling changes (unless affecting the release artifact)\n- Test additions (unless validating a fix worth mentioning)\n\n## GitHub Release Notes\n\nEach GitHub release includes the full changelog for the **minor series** back\nto x.x.0. The `scripts/extract-changelog.sh` script handles this, and the\npublish workflow (`publish.yml`) calls it to populate the GitHub release.\n\n## Git Hooks\n\nThe pre-push hook (`scripts/pre-push`) blocks `v*` tag pushes unless:\n\n1. `package.json` version matches the tag\n2. `CHANGELOG.md` has a `## [X.Y.Z] - date` entry for the version\n3. CI passed on GitHub (warns in non-interactive shells, blocks in terminals)\n\nHooks are installed silently by the context script. They can also be installed\nmanually via `skills/release/scripts/install-hooks.sh` or automatically via\n`bun install` (prepare script).\n\n<!-- File: CLAUDE.md (Tokens: ~1884 | Category: claude-rule) -->\n# QMD - Query Markup Documents\n\nUse Bun instead of Node.js (`bun` not `node`, `bun install` not `npm install`).\n\n## Commands\n\n```sh\nqmd collection add . --name <n>   # Create/index collection\nqmd collection list               # List all collections with details\nqmd collection remove <name>      # Remove a collection by name\nqmd collection rename <old> <new> # Rename a collection\nqmd collection metadata [name...] # Discover metadata keys, types, and value counts (--match, --filter, --key-limit, --value-limit)\nqmd init                          # Create a project-local .qmd index\nqmd ls [collection[/path]]        # List collections or files in a collection\nqmd context add [path] \"text\"     # Add context for path (defaults to current dir)\nqmd context list                  # List all contexts\nqmd context check                 # Check for collections/paths missing context\nqmd context rm <path>             # Remove context\nqmd get <file>[:from[:count]]     # Get by path or docid (#abc123); optional line range\nqmd multi-get <pattern>           # Get multiple docs by glob or comma-separated list\nqmd status                        # Show index status and collections\nqmd doctor                        # Diagnose config, index, model, and device issues\nqmd update                        # Re-index collections; configured update hooks run first\nqmd trust [list|revoke]           # Approve a checked-in .qmd config's hooks/paths/models\nqmd embed                         # Generate vector embeddings (uses node-llama-cpp)\nqmd query <query>                 # Search with query expansion + reranking (recommended)\nqmd search <query>                # Full-text keyword search (BM25, no LLM)\nqmd vsearch <query>               # Vector similarity search (no reranking)\nqmd bench <fixture.json>          # Run search-quality benchmarks\nqmd mcp                           # Start MCP server (stdio transport)\nqmd mcp --http [--port N]         # Start MCP server (HTTP, default port 8181)\nqmd mcp --http --daemon           # Start as background daemon\nqmd mcp stop                      # Stop background MCP daemon\n```\n\n## Collection Management\n\n```sh\n# List all collections\nqmd collection list\n\n# Create a collection with explicit name\nqmd collection add ~/Documents/notes --name mynotes --mask '**/*.md'\n\n# Remove a collection\nqmd collection remove mynotes\n\n# Rename a collection\nqmd collection rename mynotes my-notes\n\n# Show collection details, including the top metadata keys\nqmd collection show mynotes\n\n# Discover metadata keys and values to filter on\nqmd collection metadata mynotes\nqmd collection metadata mynotes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}'\nqmd collection metadata mynotes --match '{\"field\":\"value\",\"operator\":\"eq\",\"value\":\"docs-team\"}'\nqmd collection metadata mynotes --match '{\"field\":\"key\",\"operator\":\"eq\",\"value\":\"topics\"}' --filter '{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"published\"}'\nqmd collection metadata mynotes --key-limit 50 --key-offset 50   # next page of keys\n\n# Set or clear the pre-update hook (runs before re-indexing on `qmd update`)\nqmd collection update-cmd mynotes 'git pull --ff-only'\nqmd collection update-cmd mynotes            # clear\n\n# Include or exclude from default (unscoped) queries\nqmd collection exclude mynotes\nqmd collection include mynotes\n\n# List all files in a collection\nqmd ls mynotes\n\n# List files with a path prefix\nqmd ls journals/2025\nqmd ls qmd://journals/2025\n```\n\n## Context Management\n\n```sh\n# Add context to current directory (auto-detects collection)\nqmd context add \"Description of these files\"\n\n# Add context to a specific path\nqmd context add /subfolder \"Description for subfolder\"\n\n# Add global context to all collections (system message)\nqmd context add / \"Always include this context\"\n\n# Add context using virtual paths\nqmd context add qmd://journals/ \"Context for entire journals collection\"\nqmd context add qmd://journals/2024 \"Journal entries from 2024\"\n\n# List all contexts\nqmd context list\n\n# Check for collections or paths without context\nqmd context check\n\n# Remove context\nqmd context rm qmd://journals/2024\nqmd context rm /  # Remove global context\n```\n\n## Document IDs (docid)\n\nEach document has a unique short ID (docid) - the first 6 characters of its content hash.\nDocids are shown in search results as `#abc123` and can be used with `get` and `multi-get`:\n\n```sh\n# Search returns docid in results\nqmd search \"query\" --json\n# Output: [{\"docid\": \"#abc123\", \"score\": 0.85, \"file\": \"docs/readme.md\", ...}]\n\n# Get document by docid\nqmd get \"#abc123\"\nqmd get abc123              # Leading # is optional\n\n# Docids also work in multi-get comma-separated lists\nqmd multi-get \"#abc123, #def456\"\n```\n\n## Options\n\n```sh\n# Search & retrieval\n-c, --collection <name>  # Restrict search to collection(s) (repeatable)\n-n <num>                 # Number of results\n--all                    # Return all matches\n--min-score <num>        # Minimum score threshold\n--full                   # Show full document content\n--intent <text>          # Describe what you're after to sharpen ranking (query)\n--no-rerank              # Skip LLM reranking (faster, lower quality)\n--full-path              # Show on-disk paths instead of qmd:// URIs\n\n# Get / multi-get\n-l <num>                 # Maximum lines per file\n--max-bytes <num>        # Skip files larger than this (default 10KB)\n--no-line-numbers        # Disable line numbers (on by default for get/multi-get)\n\n# Output format (search, query, multi-get)\n--format <kind>          # cli (default) | json | csv | md | xml | files\n                         # legacy --json/--csv/--md/--xml/--files still work as aliases\n```\n\n## Development\n\n```sh\nbun src/cli/qmd.ts <command>   # Run from source\nbun link               # Install globally as 'qmd'\n```\n\n## Tests\n\nAll tests live in `test/`. Run everything:\n\n```sh\nnpx vitest run --reporter=verbose test/\nbun test --preload ./src/test-preload.ts test/\n```\n\n## Architecture\n\n- SQLite FTS5 for full-text search (BM25)\n- sqlite-vec for vector similarity search\n- node-llama-cpp for embeddings (embeddinggemma), reranking (qwen3-reranker), and query expansion (Qwen3)\n- Reciprocal Rank Fusion (RRF) for combining results\n- Smart chunking: 900 tokens/chunk with 15% overlap, prefers markdown headings as boundaries\n- AST-aware chunking: use `--chunk-strategy auto` to chunk code files (.ts/.js/.py/.go/.rs) at function/class/import boundaries via tree-sitter. Default is `regex` (existing behavior). Markdown and unknown file types always use regex chunking.\n\n## Important: Do NOT run automatically\n\n- Never run `qmd collection add`, `qmd embed`, or `qmd update` automatically\n- Never modify the SQLite database directly\n- Write out example commands for the user to run manually\n- Index is stored at `~/.cache/qmd/index.sqlite`\n\n## Do NOT compile\n\n- Never run `bun build --compile` - it overwrites the shell wrapper and breaks sqlite-vec\n- The `qmd` file is a shell script that runs compiled JS from `dist/` - do not replace it\n- `npm run build` compiles TypeScript to `dist/` via `tsc -p tsconfig.build.json`\n\n## Releasing\n\nUse `/release <version>` to cut a release. Full changelog standards,\nrelease workflow, and git hook setup are documented in the\n[release skill](skills/release/SKILL.md).\n\nKey points:\n- Add changelog entries under `## [Unreleased]` **as you make changes**\n- The release script renames `[Unreleased]` → `[X.Y.Z] - date` at release time\n- Credit external PRs with `#NNN (thanks @username)`\n- GitHub releases roll up the full minor series (e.g. 1.2.0 through 1.2.3)\n\n<!-- File: finetune/CLAUDE.md (Tokens: ~1323 | Category: claude-rule) -->\n# QMD Query Expansion Fine-Tuning\n\n## Overview\n\nTrain Qwen3-1.7B to expand search queries into structured `hyde:/lex:/vec:` output for QMD's hybrid retrieval pipeline.\n\n## Output Format\n\n```\nhyde: A hypothetical document passage that would answer the query.\nlex: keyword1\nlex: keyword2\nvec: semantic query reformulation\nvec: another semantic variation\n```\n\n- `hyde:` always comes FIRST (one line max)\n- `lex:` lines for BM25 keyword search (1-3 lines, short keywords)\n- `vec:` lines for vector similarity search (1-3 lines, natural language)\n\n## Training Data Format\n\n**There is exactly one JSONL format.** Every file in `data/*.jsonl` must match the strict Pydantic schema in `dataset/schema.py`:\n\n```json\n{\"query\": \"auth config\", \"output\": [[\"hyde\", \"...\"], [\"lex\", \"...\"], [\"vec\", \"...\"]]}\n```\n\n- `query`: non-empty string\n- `output`: list of `[type, text]` pairs where type is `\"lex\"`, `\"vec\"`, or `\"hyde\"`\n- Extra metadata fields (`category`, `intent`, `is_short`) are allowed but ignored\n\nThe schema is enforced by `dataset/schema.py:TrainingExample` (Pydantic model). All data loading goes through `load_examples()` which fails loudly on invalid data. No format alternatives, no legacy fallbacks.\n\n**All `.jsonl` files in `data/` are concatenated and deduplicated for training runs.** The prepared train/val files in `data/train/` are ephemeral build artifacts.\n\n## HuggingFace Repositories\n\n| Repository | Purpose |\n|------------|---------|\n| `tobil/qmd-query-expansion-1.7B` | Final merged model (SFT baseline) |\n| `tobil/qmd-query-expansion-1.7B-gguf` | GGUF quantized versions for deployment |\n| `tobil/qmd-query-expansion-1.7B-sft` | SFT adapter checkpoint (intermediate) |\n| `tobil/qmd-query-expansion-train` | Prepared training dataset |\n| `tobil/qmd-query-expansion-1.7B-grpo` | Experimental GRPO adapter (optional) |\n\n**Rules:**\n- No versioned repos (`-v1`, `-v2`, `-v4`, etc.) - update in place\n- Only push when eval scores improve over current deployed model\n- Always include eval results in model card when pushing\n\n## Dataset Tools\n\n| Script | Purpose |\n|--------|---------|\n| `dataset/schema.py` | Pydantic `TrainingExample` model + `load_examples()` |\n| `dataset/prepare_data.py` | Load via schema, apply Qwen3 chat template, dedup, split |\n| `dataset/validate_schema.py` | Validate all JSONL files against schema |\n| `dataset/score_data.py` | Score all examples using reward.py |\n| `dataset/analyze_data.py` | Analyze distribution and quality |\n\n## Training Pipeline\n\nAlways use **Qwen3-1.7B** as the base model unless explicitly stated otherwise.\n\n### Stage 0: Prepare Data\n\n```bash\nuv run dataset/prepare_data.py\n# Creates: data/train/train.jsonl, data/train/val.jsonl (ephemeral)\n```\n\n### Stage 1: SFT\n\n```bash\n# Local (requires CUDA)\nuv run train.py sft --config configs/sft.yaml\n\n# Cloud (HuggingFace Jobs)\nhf jobs uv run --flavor a10g-large --secrets HF_TOKEN --timeout 2h jobs/sft.py\n```\n\n### Stage 2: (Experimental) GRPO\n\n```bash\n# Experimental script\ncd finetune && HF_TOKEN=${HF_TOKEN} uv run python experiments/grpo/grpo.py\n```\n\n### HuggingFace Jobs\n\n```bash\nhf jobs ps                    # List running jobs\nhf jobs logs <job-id>         # Stream logs\nhf jobs inspect <job-id>      # Check status\nhf jobs cancel <job-id>       # Cancel a job\n```\n\n### Evaluation\n\n```bash\nuv run eval.py ./outputs/sft\nuv run eval.py tobil/qmd-query-expansion-1.7B\nuv run eval.py ./outputs/sft -o eval_results.json\n```\n\n## Quality Scoring\n\n`reward.py` is the single source of truth for scoring:\n\n```bash\nuv run reward.py   # Self-test\n```\n\nSee `SCORING.md` for the full rubric.\n\n## Experiments\n\nExperimental training configurations live in `experiments/`:\n\n```\nexperiments/\n├── lfm2/          # LiquidAI LFM2-1.2B (hybrid architecture, faster inference)\n│   ├── sft_lfm2.yaml\n│   └── sft_lfm2.py\n├── grpo/          # Experimental GRPO recipe and config\n│   ├── grpo.py\n│   └── grpo.yaml\n└── gepa/          # DSPy-based prompt optimization (GEPA)\n    ├── dspy_gepa.py\n    └── ...\n```\n\nThese are not part of the main training pipeline.\n\n## Key Files\n\n```\nfinetune/\n├── reward.py          # Scoring function (single source of truth)\n├── train.py           # SFT training entrypoint\n├── eval.py            # Generate and score expansions\n├── convert_gguf.py    # GGUF conversion\n├── SCORING.md         # Detailed scoring rubric\n├── CLAUDE.md          # This file\n├── Justfile           # Common commands\n├── data/              # All training JSONL files (strict schema)\n├── dataset/           # Schema + data tools (Pydantic-based)\n├── jobs/              # Self-contained HuggingFace Jobs scripts\n├── configs/           # Training configs (sft.yaml)\n├── evals/             # Test queries\n├── experiments/       # Experimental configs (LFM2, GEPA, GRPO)\n└── outputs/           # Local training outputs (gitignored)\n```\n\n<!-- File: .claude-plugin/marketplace.json (Tokens: ~212 | Category: marketplace) -->\n{\n\t\"name\": \"qmd\",\n\t\"owner\": {\n\t\t\"name\": \"tobi\",\n\t\t\"email\": \"tobi@lutke.com\"\n\t},\n\t\"plugins\": [\n\t\t{\n\t\t\t\"name\": \"qmd\",\n\t\t\t\"source\": \"./skills\",\n\t\t\t\"description\": \"Search and retrieve documents from local markdown files.\",\n\t\t\t\"version\": \"2.8.3\",\n\t\t\t\"author\": {\n\t\t\t\t\"name\": \"tobi\",\n\t\t\t\t\"email\": \"tobi@lutke.com\"\n\t\t\t},\n\t\t\t\"repository\": \"https://github.com/tobi/qmd\",\n\t\t\t\"license\": \"MIT\",\n\t\t\t\"keywords\": [\"markdown\", \"search\", \"qmd\"],\n\t\t\t\"skills\": [\"./qmd\", \"./release\"],\n\t\t\t\"mcpServers\": {\n\t\t\t\t\"qmd\": {\n\t\t\t\t\t\"command\": \"qmd\",\n\t\t\t\t\t\"args\": [\"mcp\"]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t]\n}\n</yakaai_skills>"}