{"owner":"johnkerl","repo":"miller","hasSkills":true,"totalSkillsCount":2,"totalTokensCount":2424,"categories":["anthropic-skill","claude-rule"],"hasMcp":true,"mcpConfig":{"mcpServers":{"miller":{"command":"npx","args":["-y","@modelcontextprotocol/server-miller"]}}},"found":["pkg/terminals/mcp/SKILL.md","CLAUDE.md"],"skills":{"pkg/terminals/mcp/SKILL.md":"---\nname: miller\ndescription: >\n  Drive Miller (mlr) to process CSV/TSV/JSON/etc. data. Use when constructing\n  mlr command lines: discover capabilities from the catalog rather than\n  guessing, learn the data's shape before writing expressions, validate DSL\n  before running, and recover from failures via structured errors.\n---\n\n# Miller agent playbook\n\nMiller (`mlr`) is a command-line data processor for CSV, TSV, JSON, JSON\nLines, and other tabular/record formats, with SQL-like verbs (`cut`, `sort`,\n`join`, `stats1`, ...) and an awk-like DSL (`put`, `filter`).\n\nWork this loop. Each step exists to prevent a specific, common failure.\n\n## 1. Discover — never invent names\n\nEverything valid is in the catalog; anything not in the catalog does not\nexist. Hallucinated flag/function names are the top failure mode.\n\n- Route an intent: `which` with e.g. `\"join two files on a key\"` → ranked\n  candidates. `confident: true` means a name matched; trust the top hit.\n- Browse cheaply: `list_capabilities` with `index: true` → every\n  verb/function/flag/keyword with one-line summaries.\n- Drill in: `list_capabilities` with `kind: \"verb\", names: [\"join\"]` → the\n  full entry. Prefer the structured `options` list (flag, arg, type, enum\n  `values`) when present; `usage_text` is the prose fallback.\n- The whole catalog is cacheable against `(mlr_version,\n  catalog_schema_version)` — re-fetch only when either changes.\n\n## 2. Constrain — learn the data before touching it\n\nCall `describe_data` on the input first. It returns, per field: name, types\nseen with counts, occurrence count, null count, cardinality, min/max, and —\nfor low-cardinality fields — every distinct value.\n\n- Copy field names exactly from `describe_data`; never guess casing or\n  spelling.\n- For flags like `-g` (group-by) and DSL comparisons, use values from the\n  `values` array, not values you expect to exist.\n- Fields whose `count` is less than other fields' are absent in some records:\n  guard DSL with `is_present($field)`.\n\n## 3. Validate — check DSL before spending a run\n\nBefore any `run` that includes `put` or `filter`, call `validate_dsl` with the\nexpression. Cost: parse-only, no data read. On `valid: false`, the `error`\ndocument has `kind`, `hint`, and `did_you_mean` — apply the hint, don't\nre-guess syntax.\n\n## 4. Run — and read errors structurally\n\nCall `run` with argv as a list, one element per shell word (no shell quoting):\n\n    {\"args\": [\"--icsv\", \"--ojson\", \"cat\", \"data.csv\"]}\n\nCommand-line shape rules that prevent most argv errors:\n\n- Main flags (I/O formats etc.) come **before** the verb: `mlr --icsv sort -f name f.csv`.\n- Format shorthands: `--icsv --ojson` (separate in/out), `--csv`/`--c2j` etc. (combined).\n- Chain verbs with `then`: `[\"--icsv\", \"sort\", \"-f\", \"k\", \"then\", \"head\", \"-n\", \"3\", \"f.csv\"]`.\n- If a field value being compared in `filter` might collide with a verb flag,\n  end verb flags with `--` before filenames.\n- Inline data goes in `stdin_text`; files go at the end of `args`.\n\nOn failure, `exit_code` is nonzero and `error` (when present) carries `kind`,\n`hint`, and `did_you_mean` — `hint` is often a corrected command line; prefer\nexecuting it over reasoning from the message. `stdout_truncated: true` means\nthe output exceeded the server's cap: narrow the query (e.g. `head`, `cut`)\nrather than re-running the same command.\n\n## Notes\n\n- `run` cannot execute external commands (DSL `system`/`exec`, piped\n  redirects, `--prepipe`) unless the server was started with `--allow-shell`;\n  such calls fail cleanly. It **can** write files via `tee`, `split`, and DSL\n  output redirects — treat it as a write-capable tool.\n- Long inputs: prefer `describe_data` + targeted verbs over dumping whole\n  files through `run`.\n- One record format in, another out: Miller is format-to-format; there is no\n  separate conversion step.\n","CLAUDE.md":"# Claude Development Guide for Miller\n\n## Project Overview\n\nMiller is a command-line data processing tool for working with CSV, TSV, JSON, and other data formats. It's written in Go (v1.18+) and provides SQL-like operations on data.\n\n## Initial Setup\n\n### Setting Up staticcheck\n\nThe `make staticcheck` target requires the staticcheck tool. To set it up:\n\n```bash\ngo install honnef.co/go/tools/cmd/staticcheck@latest\n```\n\nThis installs staticcheck to `~/go/bin/` (the default Go binaries directory). For `make staticcheck` to work, you need `~/go/bin` in your `PATH`.\n\n**Add to your shell profile** (`.bashrc`, `.zshrc`, or equivalent):\n```bash\nexport PATH=\"$PATH:$HOME/go/bin\"\n```\n\n**Verify the installation:**\n```bash\nstaticcheck -version\n```\n\nIf this works, you can use `make staticcheck` without any further setup.\n\n### Setting Up golangci-lint\n\nCI runs `golangci-lint` (config in `.golangci.yml`) on every push and PR via\n`.github/workflows/golangci-lint.yml`. To run it locally, install the version\nmatching CI (currently v2.12.2) per the\n[official instructions](https://golangci-lint.run/welcome/install/#local-installation), e.g.:\n\n```bash\ncurl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.12.2\n```\n\nThis also installs to `~/go/bin/`, so the same `PATH` addition above covers it.\n\n**Verify the installation:**\n```bash\ngolangci-lint --version\n```\n\n## Build & Test\n\n### Building\n```bash\nmake build          # Build the mlr executable\nmake quiet          # Build silently (no output messages)\n```\n\n### Testing\n```bash\nmake check          # Run all tests (unit + regression)\nmake unit-test      # Run unit tests only\nmake regression-test # Run regression tests only\nmake bench          # Run benchmarks\n```\n\n### Code Quality\n```bash\nmake fmt            # Format code with go fmt\nmake staticcheck    # Run static analysis (see Initial Setup section above)\nmake lint           # Run golangci-lint, same invocation as CI (see Initial Setup section above)\n```\n\n`make lint` runs `golangci-lint run ./cmd/mlr ./pkg/...`, matching\n`.github/workflows/golangci-lint.yml` exactly, so a clean local run usually\nmeans CI's lint job will pass too. It is not part of `make check` or\n`make dev`, so run it explicitly before pushing.\n\n**Note:** CI's lint job restores a persistent `golangci-lint` results cache\nshared across commits and PRs (via `golangci-lint-action`), which can\noccasionally serve stale `staticcheck` results (e.g. spurious `SA5011`\nnil-check warnings) unrelated to your diff. The job is `continue-on-error:\ntrue` for this reason, so it won't block merging. If CI lint fails but\n`make lint` is clean locally, it's likely a stale cache — rerun the job, or a\nmaintainer can clear it: `gh cache list --key golangci-lint` then\n`gh cache delete <id>`.\n\n### Full Developer Workflow\n```bash\nmake dev            # Format, build, test, generate docs (comprehensive check before pushing)\n```\n\n## Project Structure\n\n- `cmd/mlr` - Main Miller executable entry point\n- `pkg/` - Core library code organized by functionality\n  - `pkg/lib/` - Utility libraries\n  - `pkg/scan/` - Input scanning\n  - `pkg/mlrval/` - Miller value types\n  - `pkg/bifs/` - Built-in functions\n  - `pkg/input/` - Input format handlers\n- `regression_test.go` - Regression test suite\n- `docs/` - Documentation (Markdown with live code samples)\n- `man/` - Man page generation\n\n## Development Conventions\n\n### Code Style\n- Follow Go conventions and `go fmt` output\n- Use meaningful variable and function names\n- Keep functions focused and testable\n\n### Testing\n- Add unit tests for new functionality in `pkg/*/` directories\n- Use `go test` for unit tests\n- Run `make regression-test` for integration testing\n- The `mlr regtest` command provides more control for interactive debugging\n\n### Documentation\n- Update relevant `.md.in` files in `docs/src/` when adding features\n- These are processed into live documentation with actual code examples\n- Run `make dev` to rebuild documentation\n\n### Git Workflow\n- Create descriptive commit messages\n- Reference issue numbers when fixing bugs or implementing features\n- Test locally with `make check` before committing\n- For major changes, run `make dev` to ensure docs and tests pass\n\n## Key Dependencies\n\nMiller uses the Go standard library. Check `go.mod` for specific versions.\n\n## Common Tasks\n\n### Adding a New Built-in Function\n1. Implement in appropriate package (likely `pkg/bifs/`)\n2. Add unit tests alongside\n3. Update documentation in `docs/src/`\n4. Run `make check` to verify\n\n### Fixing a Bug\n1. Create a minimal test case (unit test or regression test)\n2. Fix the code\n3. Verify with `make check`\n4. Commit with reference to issue number\n\n### Performance Optimization\n1. Run `make bench` to establish baseline\n2. Make changes\n3. Run benchmarks again to measure improvement\n4. Consider adding permanent benchmark in test files\n\n## Documentation Building\n\nDocumentation is built from `.md.in` template files that contain live code samples executed via Miller itself. When you make changes that affect command output or behavior, you may need to update these templates and rebuild docs with `make -C docs/src forcebuild`.\n\n## Before Pushing\n\nAlways run:\n```bash\nmake dev\nmake lint\n```\n\n`make dev` ensures code formatting, builds successfully, passes all tests, and documentation is up to date. `make lint` is separate (not part of `make dev`/`make check`) and mirrors the CI lint job.\n\n## Additional Resources\n\n- [Full documentation](https://miller.readthedocs.io/)\n- [Contributing guidelines](https://miller.readthedocs.io/en/latest/contributing/)\n- [Issue labeling notes](https://github.com/johnkerl/miller/wiki/Issue-labeling)\n"},"files":{"pkg/terminals/mcp/SKILL.md":"---\nname: miller\ndescription: >\n  Drive Miller (mlr) to process CSV/TSV/JSON/etc. data. Use when constructing\n  mlr command lines: discover capabilities from the catalog rather than\n  guessing, learn the data's shape before writing expressions, validate DSL\n  before running, and recover from failures via structured errors.\n---\n\n# Miller agent playbook\n\nMiller (`mlr`) is a command-line data processor for CSV, TSV, JSON, JSON\nLines, and other tabular/record formats, with SQL-like verbs (`cut`, `sort`,\n`join`, `stats1`, ...) and an awk-like DSL (`put`, `filter`).\n\nWork this loop. Each step exists to prevent a specific, common failure.\n\n## 1. Discover — never invent names\n\nEverything valid is in the catalog; anything not in the catalog does not\nexist. Hallucinated flag/function names are the top failure mode.\n\n- Route an intent: `which` with e.g. `\"join two files on a key\"` → ranked\n  candidates. `confident: true` means a name matched; trust the top hit.\n- Browse cheaply: `list_capabilities` with `index: true` → every\n  verb/function/flag/keyword with one-line summaries.\n- Drill in: `list_capabilities` with `kind: \"verb\", names: [\"join\"]` → the\n  full entry. Prefer the structured `options` list (flag, arg, type, enum\n  `values`) when present; `usage_text` is the prose fallback.\n- The whole catalog is cacheable against `(mlr_version,\n  catalog_schema_version)` — re-fetch only when either changes.\n\n## 2. Constrain — learn the data before touching it\n\nCall `describe_data` on the input first. It returns, per field: name, types\nseen with counts, occurrence count, null count, cardinality, min/max, and —\nfor low-cardinality fields — every distinct value.\n\n- Copy field names exactly from `describe_data`; never guess casing or\n  spelling.\n- For flags like `-g` (group-by) and DSL comparisons, use values from the\n  `values` array, not values you expect to exist.\n- Fields whose `count` is less than other fields' are absent in some records:\n  guard DSL with `is_present($field)`.\n\n## 3. Validate — check DSL before spending a run\n\nBefore any `run` that includes `put` or `filter`, call `validate_dsl` with the\nexpression. Cost: parse-only, no data read. On `valid: false`, the `error`\ndocument has `kind`, `hint`, and `did_you_mean` — apply the hint, don't\nre-guess syntax.\n\n## 4. Run — and read errors structurally\n\nCall `run` with argv as a list, one element per shell word (no shell quoting):\n\n    {\"args\": [\"--icsv\", \"--ojson\", \"cat\", \"data.csv\"]}\n\nCommand-line shape rules that prevent most argv errors:\n\n- Main flags (I/O formats etc.) come **before** the verb: `mlr --icsv sort -f name f.csv`.\n- Format shorthands: `--icsv --ojson` (separate in/out), `--csv`/`--c2j` etc. (combined).\n- Chain verbs with `then`: `[\"--icsv\", \"sort\", \"-f\", \"k\", \"then\", \"head\", \"-n\", \"3\", \"f.csv\"]`.\n- If a field value being compared in `filter` might collide with a verb flag,\n  end verb flags with `--` before filenames.\n- Inline data goes in `stdin_text`; files go at the end of `args`.\n\nOn failure, `exit_code` is nonzero and `error` (when present) carries `kind`,\n`hint`, and `did_you_mean` — `hint` is often a corrected command line; prefer\nexecuting it over reasoning from the message. `stdout_truncated: true` means\nthe output exceeded the server's cap: narrow the query (e.g. `head`, `cut`)\nrather than re-running the same command.\n\n## Notes\n\n- `run` cannot execute external commands (DSL `system`/`exec`, piped\n  redirects, `--prepipe`) unless the server was started with `--allow-shell`;\n  such calls fail cleanly. It **can** write files via `tee`, `split`, and DSL\n  output redirects — treat it as a write-capable tool.\n- Long inputs: prefer `describe_data` + targeted verbs over dumping whole\n  files through `run`.\n- One record format in, another out: Miller is format-to-format; there is no\n  separate conversion step.\n","CLAUDE.md":"# Claude Development Guide for Miller\n\n## Project Overview\n\nMiller is a command-line data processing tool for working with CSV, TSV, JSON, and other data formats. It's written in Go (v1.18+) and provides SQL-like operations on data.\n\n## Initial Setup\n\n### Setting Up staticcheck\n\nThe `make staticcheck` target requires the staticcheck tool. To set it up:\n\n```bash\ngo install honnef.co/go/tools/cmd/staticcheck@latest\n```\n\nThis installs staticcheck to `~/go/bin/` (the default Go binaries directory). For `make staticcheck` to work, you need `~/go/bin` in your `PATH`.\n\n**Add to your shell profile** (`.bashrc`, `.zshrc`, or equivalent):\n```bash\nexport PATH=\"$PATH:$HOME/go/bin\"\n```\n\n**Verify the installation:**\n```bash\nstaticcheck -version\n```\n\nIf this works, you can use `make staticcheck` without any further setup.\n\n### Setting Up golangci-lint\n\nCI runs `golangci-lint` (config in `.golangci.yml`) on every push and PR via\n`.github/workflows/golangci-lint.yml`. To run it locally, install the version\nmatching CI (currently v2.12.2) per the\n[official instructions](https://golangci-lint.run/welcome/install/#local-installation), e.g.:\n\n```bash\ncurl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.12.2\n```\n\nThis also installs to `~/go/bin/`, so the same `PATH` addition above covers it.\n\n**Verify the installation:**\n```bash\ngolangci-lint --version\n```\n\n## Build & Test\n\n### Building\n```bash\nmake build          # Build the mlr executable\nmake quiet          # Build silently (no output messages)\n```\n\n### Testing\n```bash\nmake check          # Run all tests (unit + regression)\nmake unit-test      # Run unit tests only\nmake regression-test # Run regression tests only\nmake bench          # Run benchmarks\n```\n\n### Code Quality\n```bash\nmake fmt            # Format code with go fmt\nmake staticcheck    # Run static analysis (see Initial Setup section above)\nmake lint           # Run golangci-lint, same invocation as CI (see Initial Setup section above)\n```\n\n`make lint` runs `golangci-lint run ./cmd/mlr ./pkg/...`, matching\n`.github/workflows/golangci-lint.yml` exactly, so a clean local run usually\nmeans CI's lint job will pass too. It is not part of `make check` or\n`make dev`, so run it explicitly before pushing.\n\n**Note:** CI's lint job restores a persistent `golangci-lint` results cache\nshared across commits and PRs (via `golangci-lint-action`), which can\noccasionally serve stale `staticcheck` results (e.g. spurious `SA5011`\nnil-check warnings) unrelated to your diff. The job is `continue-on-error:\ntrue` for this reason, so it won't block merging. If CI lint fails but\n`make lint` is clean locally, it's likely a stale cache — rerun the job, or a\nmaintainer can clear it: `gh cache list --key golangci-lint` then\n`gh cache delete <id>`.\n\n### Full Developer Workflow\n```bash\nmake dev            # Format, build, test, generate docs (comprehensive check before pushing)\n```\n\n## Project Structure\n\n- `cmd/mlr` - Main Miller executable entry point\n- `pkg/` - Core library code organized by functionality\n  - `pkg/lib/` - Utility libraries\n  - `pkg/scan/` - Input scanning\n  - `pkg/mlrval/` - Miller value types\n  - `pkg/bifs/` - Built-in functions\n  - `pkg/input/` - Input format handlers\n- `regression_test.go` - Regression test suite\n- `docs/` - Documentation (Markdown with live code samples)\n- `man/` - Man page generation\n\n## Development Conventions\n\n### Code Style\n- Follow Go conventions and `go fmt` output\n- Use meaningful variable and function names\n- Keep functions focused and testable\n\n### Testing\n- Add unit tests for new functionality in `pkg/*/` directories\n- Use `go test` for unit tests\n- Run `make regression-test` for integration testing\n- The `mlr regtest` command provides more control for interactive debugging\n\n### Documentation\n- Update relevant `.md.in` files in `docs/src/` when adding features\n- These are processed into live documentation with actual code examples\n- Run `make dev` to rebuild documentation\n\n### Git Workflow\n- Create descriptive commit messages\n- Reference issue numbers when fixing bugs or implementing features\n- Test locally with `make check` before committing\n- For major changes, run `make dev` to ensure docs and tests pass\n\n## Key Dependencies\n\nMiller uses the Go standard library. Check `go.mod` for specific versions.\n\n## Common Tasks\n\n### Adding a New Built-in Function\n1. Implement in appropriate package (likely `pkg/bifs/`)\n2. Add unit tests alongside\n3. Update documentation in `docs/src/`\n4. Run `make check` to verify\n\n### Fixing a Bug\n1. Create a minimal test case (unit test or regression test)\n2. Fix the code\n3. Verify with `make check`\n4. Commit with reference to issue number\n\n### Performance Optimization\n1. Run `make bench` to establish baseline\n2. Make changes\n3. Run benchmarks again to measure improvement\n4. Consider adding permanent benchmark in test files\n\n## Documentation Building\n\nDocumentation is built from `.md.in` template files that contain live code samples executed via Miller itself. When you make changes that affect command output or behavior, you may need to update these templates and rebuild docs with `make -C docs/src forcebuild`.\n\n## Before Pushing\n\nAlways run:\n```bash\nmake dev\nmake lint\n```\n\n`make dev` ensures code formatting, builds successfully, passes all tests, and documentation is up to date. `make lint` is separate (not part of `make dev`/`make check`) and mirrors the CI lint job.\n\n## Additional Resources\n\n- [Full documentation](https://miller.readthedocs.io/)\n- [Contributing guidelines](https://miller.readthedocs.io/en/latest/contributing/)\n- [Issue labeling notes](https://github.com/johnkerl/miller/wiki/Issue-labeling)\n"},"items":[{"name":"SKILL.md","path":"pkg/terminals/mcp/SKILL.md","rawUrl":"https://raw.githubusercontent.com/johnkerl/miller/HEAD/pkg/terminals/mcp/SKILL.md","title":"miller","category":"anthropic-skill","format":"markdown","content":"---\nname: miller\ndescription: >\n  Drive Miller (mlr) to process CSV/TSV/JSON/etc. data. Use when constructing\n  mlr command lines: discover capabilities from the catalog rather than\n  guessing, learn the data's shape before writing expressions, validate DSL\n  before running, and recover from failures via structured errors.\n---\n\n# Miller agent playbook\n\nMiller (`mlr`) is a command-line data processor for CSV, TSV, JSON, JSON\nLines, and other tabular/record formats, with SQL-like verbs (`cut`, `sort`,\n`join`, `stats1`, ...) and an awk-like DSL (`put`, `filter`).\n\nWork this loop. Each step exists to prevent a specific, common failure.\n\n## 1. Discover — never invent names\n\nEverything valid is in the catalog; anything not in the catalog does not\nexist. Hallucinated flag/function names are the top failure mode.\n\n- Route an intent: `which` with e.g. `\"join two files on a key\"` → ranked\n  candidates. `confident: true` means a name matched; trust the top hit.\n- Browse cheaply: `list_capabilities` with `index: true` → every\n  verb/function/flag/keyword with one-line summaries.\n- Drill in: `list_capabilities` with `kind: \"verb\", names: [\"join\"]` → the\n  full entry. Prefer the structured `options` list (flag, arg, type, enum\n  `values`) when present; `usage_text` is the prose fallback.\n- The whole catalog is cacheable against `(mlr_version,\n  catalog_schema_version)` — re-fetch only when either changes.\n\n## 2. Constrain — learn the data before touching it\n\nCall `describe_data` on the input first. It returns, per field: name, types\nseen with counts, occurrence count, null count, cardinality, min/max, and —\nfor low-cardinality fields — every distinct value.\n\n- Copy field names exactly from `describe_data`; never guess casing or\n  spelling.\n- For flags like `-g` (group-by) and DSL comparisons, use values from the\n  `values` array, not values you expect to exist.\n- Fields whose `count` is less than other fields' are absent in some records:\n  guard DSL with `is_present($field)`.\n\n## 3. Validate — check DSL before spending a run\n\nBefore any `run` that includes `put` or `filter`, call `validate_dsl` with the\nexpression. Cost: parse-only, no data read. On `valid: false`, the `error`\ndocument has `kind`, `hint`, and `did_you_mean` — apply the hint, don't\nre-guess syntax.\n\n## 4. Run — and read errors structurally\n\nCall `run` with argv as a list, one element per shell word (no shell quoting):\n\n    {\"args\": [\"--icsv\", \"--ojson\", \"cat\", \"data.csv\"]}\n\nCommand-line shape rules that prevent most argv errors:\n\n- Main flags (I/O formats etc.) come **before** the verb: `mlr --icsv sort -f name f.csv`.\n- Format shorthands: `--icsv --ojson` (separate in/out), `--csv`/`--c2j` etc. (combined).\n- Chain verbs with `then`: `[\"--icsv\", \"sort\", \"-f\", \"k\", \"then\", \"head\", \"-n\", \"3\", \"f.csv\"]`.\n- If a field value being compared in `filter` might collide with a verb flag,\n  end verb flags with `--` before filenames.\n- Inline data goes in `stdin_text`; files go at the end of `args`.\n\nOn failure, `exit_code` is nonzero and `error` (when present) carries `kind`,\n`hint`, and `did_you_mean` — `hint` is often a corrected command line; prefer\nexecuting it over reasoning from the message. `stdout_truncated: true` means\nthe output exceeded the server's cap: narrow the query (e.g. `head`, `cut`)\nrather than re-running the same command.\n\n## Notes\n\n- `run` cannot execute external commands (DSL `system`/`exec`, piped\n  redirects, `--prepipe`) unless the server was started with `--allow-shell`;\n  such calls fail cleanly. It **can** write files via `tee`, `split`, and DSL\n  output redirects — treat it as a write-capable tool.\n- Long inputs: prefer `describe_data` + targeted verbs over dumping whole\n  files through `run`.\n- One record format in, another out: Miller is format-to-format; there is no\n  separate conversion step.\n","frontmatter":{"name":"miller","description":"Drive Miller (mlr) to process CSV/TSV/JSON/etc. data. Use when constructing mlr command lines: discover capabilities from the catalog rather than guessing, learn the data's shape before writing expressions, validate DSL before running, and recover from failures via structured errors.\n"},"isInternal":false,"tokens":1006,"sizeBytes":3864},{"name":"CLAUDE.md","path":"CLAUDE.md","rawUrl":"https://raw.githubusercontent.com/johnkerl/miller/HEAD/CLAUDE.md","title":"Claude Agent Guidelines & System Prompt","category":"claude-rule","format":"markdown","content":"# Claude Development Guide for Miller\n\n## Project Overview\n\nMiller is a command-line data processing tool for working with CSV, TSV, JSON, and other data formats. It's written in Go (v1.18+) and provides SQL-like operations on data.\n\n## Initial Setup\n\n### Setting Up staticcheck\n\nThe `make staticcheck` target requires the staticcheck tool. To set it up:\n\n```bash\ngo install honnef.co/go/tools/cmd/staticcheck@latest\n```\n\nThis installs staticcheck to `~/go/bin/` (the default Go binaries directory). For `make staticcheck` to work, you need `~/go/bin` in your `PATH`.\n\n**Add to your shell profile** (`.bashrc`, `.zshrc`, or equivalent):\n```bash\nexport PATH=\"$PATH:$HOME/go/bin\"\n```\n\n**Verify the installation:**\n```bash\nstaticcheck -version\n```\n\nIf this works, you can use `make staticcheck` without any further setup.\n\n### Setting Up golangci-lint\n\nCI runs `golangci-lint` (config in `.golangci.yml`) on every push and PR via\n`.github/workflows/golangci-lint.yml`. To run it locally, install the version\nmatching CI (currently v2.12.2) per the\n[official instructions](https://golangci-lint.run/welcome/install/#local-installation), e.g.:\n\n```bash\ncurl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.12.2\n```\n\nThis also installs to `~/go/bin/`, so the same `PATH` addition above covers it.\n\n**Verify the installation:**\n```bash\ngolangci-lint --version\n```\n\n## Build & Test\n\n### Building\n```bash\nmake build          # Build the mlr executable\nmake quiet          # Build silently (no output messages)\n```\n\n### Testing\n```bash\nmake check          # Run all tests (unit + regression)\nmake unit-test      # Run unit tests only\nmake regression-test # Run regression tests only\nmake bench          # Run benchmarks\n```\n\n### Code Quality\n```bash\nmake fmt            # Format code with go fmt\nmake staticcheck    # Run static analysis (see Initial Setup section above)\nmake lint           # Run golangci-lint, same invocation as CI (see Initial Setup section above)\n```\n\n`make lint` runs `golangci-lint run ./cmd/mlr ./pkg/...`, matching\n`.github/workflows/golangci-lint.yml` exactly, so a clean local run usually\nmeans CI's lint job will pass too. It is not part of `make check` or\n`make dev`, so run it explicitly before pushing.\n\n**Note:** CI's lint job restores a persistent `golangci-lint` results cache\nshared across commits and PRs (via `golangci-lint-action`), which can\noccasionally serve stale `staticcheck` results (e.g. spurious `SA5011`\nnil-check warnings) unrelated to your diff. The job is `continue-on-error:\ntrue` for this reason, so it won't block merging. If CI lint fails but\n`make lint` is clean locally, it's likely a stale cache — rerun the job, or a\nmaintainer can clear it: `gh cache list --key golangci-lint` then\n`gh cache delete <id>`.\n\n### Full Developer Workflow\n```bash\nmake dev            # Format, build, test, generate docs (comprehensive check before pushing)\n```\n\n## Project Structure\n\n- `cmd/mlr` - Main Miller executable entry point\n- `pkg/` - Core library code organized by functionality\n  - `pkg/lib/` - Utility libraries\n  - `pkg/scan/` - Input scanning\n  - `pkg/mlrval/` - Miller value types\n  - `pkg/bifs/` - Built-in functions\n  - `pkg/input/` - Input format handlers\n- `regression_test.go` - Regression test suite\n- `docs/` - Documentation (Markdown with live code samples)\n- `man/` - Man page generation\n\n## Development Conventions\n\n### Code Style\n- Follow Go conventions and `go fmt` output\n- Use meaningful variable and function names\n- Keep functions focused and testable\n\n### Testing\n- Add unit tests for new functionality in `pkg/*/` directories\n- Use `go test` for unit tests\n- Run `make regression-test` for integration testing\n- The `mlr regtest` command provides more control for interactive debugging\n\n### Documentation\n- Update relevant `.md.in` files in `docs/src/` when adding features\n- These are processed into live documentation with actual code examples\n- Run `make dev` to rebuild documentation\n\n### Git Workflow\n- Create descriptive commit messages\n- Reference issue numbers when fixing bugs or implementing features\n- Test locally with `make check` before committing\n- For major changes, run `make dev` to ensure docs and tests pass\n\n## Key Dependencies\n\nMiller uses the Go standard library. Check `go.mod` for specific versions.\n\n## Common Tasks\n\n### Adding a New Built-in Function\n1. Implement in appropriate package (likely `pkg/bifs/`)\n2. Add unit tests alongside\n3. Update documentation in `docs/src/`\n4. Run `make check` to verify\n\n### Fixing a Bug\n1. Create a minimal test case (unit test or regression test)\n2. Fix the code\n3. Verify with `make check`\n4. Commit with reference to issue number\n\n### Performance Optimization\n1. Run `make bench` to establish baseline\n2. Make changes\n3. Run benchmarks again to measure improvement\n4. Consider adding permanent benchmark in test files\n\n## Documentation Building\n\nDocumentation is built from `.md.in` template files that contain live code samples executed via Miller itself. When you make changes that affect command output or behavior, you may need to update these templates and rebuild docs with `make -C docs/src forcebuild`.\n\n## Before Pushing\n\nAlways run:\n```bash\nmake dev\nmake lint\n```\n\n`make dev` ensures code formatting, builds successfully, passes all tests, and documentation is up to date. `make lint` is separate (not part of `make dev`/`make check`) and mirrors the CI lint job.\n\n## Additional Resources\n\n- [Full documentation](https://miller.readthedocs.io/)\n- [Contributing guidelines](https://miller.readthedocs.io/en/latest/contributing/)\n- [Issue labeling notes](https://github.com/johnkerl/miller/wiki/Issue-labeling)\n","isInternal":false,"tokens":1418,"sizeBytes":5722}],"systemPromptSnippet":"<yakaai_skills repo=\"johnkerl/miller\">\n<!-- File: pkg/terminals/mcp/SKILL.md (Tokens: ~1006 | Category: anthropic-skill) -->\n---\nname: miller\ndescription: >\n  Drive Miller (mlr) to process CSV/TSV/JSON/etc. data. Use when constructing\n  mlr command lines: discover capabilities from the catalog rather than\n  guessing, learn the data's shape before writing expressions, validate DSL\n  before running, and recover from failures via structured errors.\n---\n\n# Miller agent playbook\n\nMiller (`mlr`) is a command-line data processor for CSV, TSV, JSON, JSON\nLines, and other tabular/record formats, with SQL-like verbs (`cut`, `sort`,\n`join`, `stats1`, ...) and an awk-like DSL (`put`, `filter`).\n\nWork this loop. Each step exists to prevent a specific, common failure.\n\n## 1. Discover — never invent names\n\nEverything valid is in the catalog; anything not in the catalog does not\nexist. Hallucinated flag/function names are the top failure mode.\n\n- Route an intent: `which` with e.g. `\"join two files on a key\"` → ranked\n  candidates. `confident: true` means a name matched; trust the top hit.\n- Browse cheaply: `list_capabilities` with `index: true` → every\n  verb/function/flag/keyword with one-line summaries.\n- Drill in: `list_capabilities` with `kind: \"verb\", names: [\"join\"]` → the\n  full entry. Prefer the structured `options` list (flag, arg, type, enum\n  `values`) when present; `usage_text` is the prose fallback.\n- The whole catalog is cacheable against `(mlr_version,\n  catalog_schema_version)` — re-fetch only when either changes.\n\n## 2. Constrain — learn the data before touching it\n\nCall `describe_data` on the input first. It returns, per field: name, types\nseen with counts, occurrence count, null count, cardinality, min/max, and —\nfor low-cardinality fields — every distinct value.\n\n- Copy field names exactly from `describe_data`; never guess casing or\n  spelling.\n- For flags like `-g` (group-by) and DSL comparisons, use values from the\n  `values` array, not values you expect to exist.\n- Fields whose `count` is less than other fields' are absent in some records:\n  guard DSL with `is_present($field)`.\n\n## 3. Validate — check DSL before spending a run\n\nBefore any `run` that includes `put` or `filter`, call `validate_dsl` with the\nexpression. Cost: parse-only, no data read. On `valid: false`, the `error`\ndocument has `kind`, `hint`, and `did_you_mean` — apply the hint, don't\nre-guess syntax.\n\n## 4. Run — and read errors structurally\n\nCall `run` with argv as a list, one element per shell word (no shell quoting):\n\n    {\"args\": [\"--icsv\", \"--ojson\", \"cat\", \"data.csv\"]}\n\nCommand-line shape rules that prevent most argv errors:\n\n- Main flags (I/O formats etc.) come **before** the verb: `mlr --icsv sort -f name f.csv`.\n- Format shorthands: `--icsv --ojson` (separate in/out), `--csv`/`--c2j` etc. (combined).\n- Chain verbs with `then`: `[\"--icsv\", \"sort\", \"-f\", \"k\", \"then\", \"head\", \"-n\", \"3\", \"f.csv\"]`.\n- If a field value being compared in `filter` might collide with a verb flag,\n  end verb flags with `--` before filenames.\n- Inline data goes in `stdin_text`; files go at the end of `args`.\n\nOn failure, `exit_code` is nonzero and `error` (when present) carries `kind`,\n`hint`, and `did_you_mean` — `hint` is often a corrected command line; prefer\nexecuting it over reasoning from the message. `stdout_truncated: true` means\nthe output exceeded the server's cap: narrow the query (e.g. `head`, `cut`)\nrather than re-running the same command.\n\n## Notes\n\n- `run` cannot execute external commands (DSL `system`/`exec`, piped\n  redirects, `--prepipe`) unless the server was started with `--allow-shell`;\n  such calls fail cleanly. It **can** write files via `tee`, `split`, and DSL\n  output redirects — treat it as a write-capable tool.\n- Long inputs: prefer `describe_data` + targeted verbs over dumping whole\n  files through `run`.\n- One record format in, another out: Miller is format-to-format; there is no\n  separate conversion step.\n\n<!-- File: CLAUDE.md (Tokens: ~1418 | Category: claude-rule) -->\n# Claude Development Guide for Miller\n\n## Project Overview\n\nMiller is a command-line data processing tool for working with CSV, TSV, JSON, and other data formats. It's written in Go (v1.18+) and provides SQL-like operations on data.\n\n## Initial Setup\n\n### Setting Up staticcheck\n\nThe `make staticcheck` target requires the staticcheck tool. To set it up:\n\n```bash\ngo install honnef.co/go/tools/cmd/staticcheck@latest\n```\n\nThis installs staticcheck to `~/go/bin/` (the default Go binaries directory). For `make staticcheck` to work, you need `~/go/bin` in your `PATH`.\n\n**Add to your shell profile** (`.bashrc`, `.zshrc`, or equivalent):\n```bash\nexport PATH=\"$PATH:$HOME/go/bin\"\n```\n\n**Verify the installation:**\n```bash\nstaticcheck -version\n```\n\nIf this works, you can use `make staticcheck` without any further setup.\n\n### Setting Up golangci-lint\n\nCI runs `golangci-lint` (config in `.golangci.yml`) on every push and PR via\n`.github/workflows/golangci-lint.yml`. To run it locally, install the version\nmatching CI (currently v2.12.2) per the\n[official instructions](https://golangci-lint.run/welcome/install/#local-installation), e.g.:\n\n```bash\ncurl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.12.2\n```\n\nThis also installs to `~/go/bin/`, so the same `PATH` addition above covers it.\n\n**Verify the installation:**\n```bash\ngolangci-lint --version\n```\n\n## Build & Test\n\n### Building\n```bash\nmake build          # Build the mlr executable\nmake quiet          # Build silently (no output messages)\n```\n\n### Testing\n```bash\nmake check          # Run all tests (unit + regression)\nmake unit-test      # Run unit tests only\nmake regression-test # Run regression tests only\nmake bench          # Run benchmarks\n```\n\n### Code Quality\n```bash\nmake fmt            # Format code with go fmt\nmake staticcheck    # Run static analysis (see Initial Setup section above)\nmake lint           # Run golangci-lint, same invocation as CI (see Initial Setup section above)\n```\n\n`make lint` runs `golangci-lint run ./cmd/mlr ./pkg/...`, matching\n`.github/workflows/golangci-lint.yml` exactly, so a clean local run usually\nmeans CI's lint job will pass too. It is not part of `make check` or\n`make dev`, so run it explicitly before pushing.\n\n**Note:** CI's lint job restores a persistent `golangci-lint` results cache\nshared across commits and PRs (via `golangci-lint-action`), which can\noccasionally serve stale `staticcheck` results (e.g. spurious `SA5011`\nnil-check warnings) unrelated to your diff. The job is `continue-on-error:\ntrue` for this reason, so it won't block merging. If CI lint fails but\n`make lint` is clean locally, it's likely a stale cache — rerun the job, or a\nmaintainer can clear it: `gh cache list --key golangci-lint` then\n`gh cache delete <id>`.\n\n### Full Developer Workflow\n```bash\nmake dev            # Format, build, test, generate docs (comprehensive check before pushing)\n```\n\n## Project Structure\n\n- `cmd/mlr` - Main Miller executable entry point\n- `pkg/` - Core library code organized by functionality\n  - `pkg/lib/` - Utility libraries\n  - `pkg/scan/` - Input scanning\n  - `pkg/mlrval/` - Miller value types\n  - `pkg/bifs/` - Built-in functions\n  - `pkg/input/` - Input format handlers\n- `regression_test.go` - Regression test suite\n- `docs/` - Documentation (Markdown with live code samples)\n- `man/` - Man page generation\n\n## Development Conventions\n\n### Code Style\n- Follow Go conventions and `go fmt` output\n- Use meaningful variable and function names\n- Keep functions focused and testable\n\n### Testing\n- Add unit tests for new functionality in `pkg/*/` directories\n- Use `go test` for unit tests\n- Run `make regression-test` for integration testing\n- The `mlr regtest` command provides more control for interactive debugging\n\n### Documentation\n- Update relevant `.md.in` files in `docs/src/` when adding features\n- These are processed into live documentation with actual code examples\n- Run `make dev` to rebuild documentation\n\n### Git Workflow\n- Create descriptive commit messages\n- Reference issue numbers when fixing bugs or implementing features\n- Test locally with `make check` before committing\n- For major changes, run `make dev` to ensure docs and tests pass\n\n## Key Dependencies\n\nMiller uses the Go standard library. Check `go.mod` for specific versions.\n\n## Common Tasks\n\n### Adding a New Built-in Function\n1. Implement in appropriate package (likely `pkg/bifs/`)\n2. Add unit tests alongside\n3. Update documentation in `docs/src/`\n4. Run `make check` to verify\n\n### Fixing a Bug\n1. Create a minimal test case (unit test or regression test)\n2. Fix the code\n3. Verify with `make check`\n4. Commit with reference to issue number\n\n### Performance Optimization\n1. Run `make bench` to establish baseline\n2. Make changes\n3. Run benchmarks again to measure improvement\n4. Consider adding permanent benchmark in test files\n\n## Documentation Building\n\nDocumentation is built from `.md.in` template files that contain live code samples executed via Miller itself. When you make changes that affect command output or behavior, you may need to update these templates and rebuild docs with `make -C docs/src forcebuild`.\n\n## Before Pushing\n\nAlways run:\n```bash\nmake dev\nmake lint\n```\n\n`make dev` ensures code formatting, builds successfully, passes all tests, and documentation is up to date. `make lint` is separate (not part of `make dev`/`make check`) and mirrors the CI lint job.\n\n## Additional Resources\n\n- [Full documentation](https://miller.readthedocs.io/)\n- [Contributing guidelines](https://miller.readthedocs.io/en/latest/contributing/)\n- [Issue labeling notes](https://github.com/johnkerl/miller/wiki/Issue-labeling)\n</yakaai_skills>"}