{"owner":"mikefarah","repo":"yq","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md"],"skills":{"AGENTS.md":"# yq — agent instructions\n\n## ⚠️ MANDATORY: GitHub agent disclosure\n\n**Always required. No exceptions.**\n\nWhenever you perform **any** GitHub action on behalf of the user, you **must** disclose that an AI agent (Cursor) wrote the content and is acting on the user's behalf — **not the user personally**. Do this **before** submitting; never post first and add the disclosure later.\n\nApplies to **all** GitHub interactions, including:\n\n- Pull requests (titles, descriptions, and reviews)\n- PR comments and inline review comments\n- Issues (new issues, comments, and updates)\n- Any other post or reply on GitHub\n\n**How to disclose:** Put it prominently at the **top** of every PR description, review body, comment, or issue. Use wording like:\n\n\nInline review comments must include a short disclosure too (e.g. `> Generated by Cursor acting on the user's behalf, not the user personally.`).\n\n**Never** submit a GitHub action without this disclosure.\n\n---\n\nAlways run the spellcheck before raising a PR:\n\n```bash\nbash scripts/spelling.sh\n```\n\nThis is also included in the full CI pipeline via `make local test`.\n\n## Cursor Cloud specific instructions\n\n### Overview\n\n**yq** is a Go CLI for querying and transforming YAML, JSON, XML, INI, and other structured formats. There are no long-running services — development is build-and-test against a local `./yq` binary.\n\n### Prerequisites\n\n- **Go ≥ 1.25** (see `go.mod`)\n- **Bash** (acceptance tests)\n- **Docker/Podman** is optional; use `make local <target>` to run natively when containers are unavailable\n\n### PATH\n\nAfter `scripts/devtools.sh`, add Go tool binaries to PATH:\n\n```bash\nexport PATH=\"$HOME/go/bin:$PATH\"\n```\n\n`golangci-lint` and `typos` install to `$HOME/go/bin`; `gosec` installs to `./bin/gosec` in the repo root.\n\n### Common commands (local, no Docker)\n\n| Task | Command |\n|------|---------|\n| Install dev tools | `bash scripts/devtools.sh` |\n| Vendor dependencies | `make local vendor` |\n| Build binary | `go build -o yq .` or `make local build` |\n| Format | `make local format` |\n| Lint | `make local check` |\n| Unit tests | `make local test` or `bash scripts/test.sh` |\n| Acceptance (E2E) | `bash scripts/acceptance.sh` (requires `./yq` built first) |\n\n`make local build` runs the full CI chain (format → spelling → gosec → lint → unit tests → build → acceptance). For a faster loop, build with `go build -o yq .` and run `bash scripts/acceptance.sh`.\n\n### Caveats\n\n- **`make` without `local`** tries Docker/Podman (`Dockerfile.dev`). In Cloud Agent VMs without Docker, always prefix with `make local`.\n- **Spelling step** uses `typos` (installed by `scripts/devtools.sh`).\n- **`make local test` / `scripts/check.sh`** require `golangci-lint` on PATH (`devtools.sh`).\n\n---\n\n# General rules\n✅ **DO:**\n- You can use ./yq with the `--debug-node-info` flag to get a deeper understanding of the ast.\n- run ./scripts/format.sh to format the code; then ./scripts/check.sh lint and finally ./scripts/spelling.sh to check spelling.\n- Add comprehensive tests to cover the changes\n- Run test suite to ensure there is no regression\n- Use UK english spelling\n- **Follow the mandatory GitHub agent disclosure rule above** on every GitHub action — no exceptions\n\n❌ **DON'T:**\n- Git add or commit\n- Add comments to functions that are self-explanatory\n- **Post to GitHub without the mandatory agent disclosure** (PRs, reviews, comments, issues, or any other GitHub interaction)\n\n\n\n# Adding a New Encoder/Decoder\n\nThis guide explains how to add support for a new format (encoder/decoder) to yq without modifying `candidate_node.go`.\n\n## Overview\n\nThe encoder/decoder architecture in yq is based on two main interfaces:\n\n- **Encoder**: Converts a `CandidateNode` to output in a specific format\n- **Decoder**: Reads input in a specific format and creates a `CandidateNode`\n\nEach format is registered in `pkg/yqlib/format.go` and made available through factory functions.\n\n## Architecture\n\n### Key Files\n\n- `pkg/yqlib/encoder.go` - Defines the `Encoder` interface\n- `pkg/yqlib/decoder.go` - Defines the `Decoder` interface\n- `pkg/yqlib/format.go` - Format registry and factory functions\n- `pkg/yqlib/operator_encoder_decoder.go` - Encode/decode operators\n- `pkg/yqlib/encoder_*.go` - Encoder implementations\n- `pkg/yqlib/decoder_*.go` - Decoder implementations\n\n### Interfaces\n\n**Encoder Interface:**\n```go\ntype Encoder interface {\n    Encode(writer io.Writer, node *CandidateNode) error\n    PrintDocumentSeparator(writer io.Writer) error\n    PrintLeadingContent(writer io.Writer, content string) error\n    CanHandleAliases() bool\n}\n```\n\n**Decoder Interface:**\n```go\ntype Decoder interface {\n    Init(reader io.Reader) error\n    Decode() (*CandidateNode, error)\n}\n```\n\n## Step-by-Step: Adding a New Encoder/Decoder\n\n### Step 1: Create the Encoder File\n\nCreate `pkg/yqlib/encoder_<format>.go` implementing the `Encoder` interface:\n- `Encode()` - Convert a `CandidateNode` to your format and write to the output writer\n- `PrintDocumentSeparator()` - Handle document separators if your format requires them\n- `PrintLeadingContent()` - Handle leading content/comments if supported\n- `CanHandleAliases()` - Return whether your format supports YAML aliases\n\nSee `encoder_json.go` or `encoder_base64.go` for examples.\n\n### Step 2: Create the Decoder File\n\nCreate `pkg/yqlib/decoder_<format>.go` implementing the `Decoder` interface:\n- `Init()` - Initialize the decoder with the input reader and set up any needed state\n- `Decode()` - Decode one document from the input and return a `CandidateNode`, or `io.EOF` when finished\n\nSee `decoder_json.go` or `decoder_base64.go` for examples.\n\n### Step 3: Create Tests (Mandatory)\n\nCreate a test file `pkg/yqlib/<format>_test.go` using the `formatScenario` pattern:\n- Define test scenarios as `formatScenario` structs with fields: `description`, `input`, `expected`, `scenarioType`\n- `scenarioType` can be `\"decode\"` (test decoding to YAML) or `\"roundtrip\"` (encode/decode preservation)\n- Create a helper function `test<Format>Scenario()` that switches on `scenarioType`\n- Create main test function `Test<Format>FormatScenarios()` that iterates over scenarios\n- The main test function should use `documentScenarios` to ensure testcase documentation is generated.\n\nTest coverage must include:\n- Basic data types (scalars, arrays, objects/maps)\n- Nested structures\n- Edge cases (empty inputs, special characters, escape sequences)\n- Format-specific features or syntax\n- Round-trip tests: decode → encode → decode should preserve data\n\nSee `hcl_test.go` for a complete example.\n\n### Step 4: Register the Format in format.go\n\nEdit `pkg/yqlib/format.go`:\n\n1. Add a new format variable:\n   - `\"<format>\"` is the formal name (e.g., \"json\", \"yaml\")\n   - `[]string{...}` contains short aliases (can be empty)\n   - The first function creates an encoder (can be nil for encode-only formats)\n   - The second function creates a decoder (can be nil for decode-only formats)\n\n2. Add the format to the `Formats` slice in the same file\n\nSee existing formats in `format.go` for the exact structure.\n\n### Step 5: Handle Encoder Configuration (if needed)\n\nIf your format has preferences/configuration options:\n\n1. Create a preferences struct with your configuration fields\n2. Update the encoder to accept preferences in its factory function\n3. Update `format.go` to pass the configured preferences\n4. Update `operator_encoder_decoder.go` if special indent handling is needed (see existing formats like JSON and YAML for the pattern)\n\nThis pattern is optional and only needed if your format has user-configurable options.\n\n## Build Tags\n\nUse build tags to allow optional compilation of formats:\n- Add `//go:build !yq_no<format>` at the top of your encoder and decoder files\n- Create a no-build version in `pkg/yqlib/no_<format>.go` that returns nil for encoder/decoder factories\n\nThis allows users to compile yq without certain formats using: `go build -tags yq_no<format>`\n\n## Working with CandidateNode\n\nThe `CandidateNode` struct represents a YAML node with:\n- `Kind`: The node type (ScalarNode, SequenceNode, MappingNode)\n- `Tag`: The YAML tag (e.g., \"!!str\", \"!!int\", \"!!map\")\n- `Value`: The scalar value (for ScalarNode only)\n- `Content`: Child nodes (for SequenceNode and MappingNode)\n\nKey methods:\n- `node.guessTagFromCustomType()` - Infer the tag from Go type\n- `node.AsList()` - Convert to a list for processing\n- `node.CreateReplacement()` - Create a new replacement node\n- `NewCandidate()` - Create a new CandidateNode\n\n## Key Points\n\n✅ **DO:**\n- Implement only the `Encoder` and `Decoder` interfaces\n- Register your format in `format.go` only\n- Keep format-specific logic in your encoder/decoder files\n- Use the candidate_node style attribute to store style information for round-trip. Ask if this needs to be updated with new styles.\n- Use build tags for optional compilation\n- Add comprehensive tests\n- Run the specific encoder/decoder test (e.g. <format>_test.go) whenever you make ay changes to the encoder_<format> or decoder_<format>\n- Handle errors gracefully\n- Add the no build directive, like the xml encoder and decoder, that enables a minimal yq builds. e.g.  `//go:build !yq_<format>`. Be sure to also update the build_small-yq.sh and build-tinygo-yq.sh to not include the new format.\n\n❌ **DON'T:**\n- Modify `candidate_node.go` to add format-specific logic\n- Add format-specific fields to `CandidateNode`\n- Create special cases in core navigation or evaluation logic\n- Bypass the encoder/decoder interfaces\n- Use candidate_node tag attribute for anything other than indicate the data type\n\n## Examples\n\nRefer to existing format implementations for patterns:\n\n- **Simple encoder/decoder**: `encoder_json.go`, `decoder_json.go`\n- **Complex with preferences**: `encoder_yaml.go`, `decoder_yaml.go`\n- **Encoder-only**: `encoder_sh.go` (ShFormat has nil decoder)\n- **String-only operations**: `encoder_base64.go`, `decoder_base64.go`\n\n## Testing Your Implementation (Mandatory)\n\nTests must be implemented in `<format>_test.go` following the `formatScenario` pattern:\n\n1. **Create test scenarios** using the `formatScenario` struct with fields:\n   - `description`: Brief description of what's being tested\n   - `input`: Sample input in your format\n   - `expected`: Expected output (typically in YAML for decode tests)\n   - `scenarioType`: Either `\"decode\"` or `\"roundtrip\"`\n\n2. **Test coverage must include:**\n   - Basic data types (scalars, arrays, objects/maps)\n   - Nested structures\n   - Edge cases (empty inputs, special characters, escape sequences)\n   - Format-specific features or syntax\n   - Round-trip tests: decode → encode → decode should preserve data\n\n3. **Test function pattern:**\n   - `test<Format>Scenario()`: Helper function that switches on `scenarioType`\n   - `Test<Format>FormatScenarios()`: Main test function that iterates over scenarios\n\n4. **Example from existing formats:**\n   - See `hcl_test.go` for a complete example\n   - See `yaml_test.go` for YAML-specific patterns\n   - See `json_test.go` for more complex scenarios\n\n## Common Patterns\n\n### Format with Indentation\nUse preferences to control output formatting:\n```go\ntype <format>Preferences struct {\n    Indent int\n}\n\nfunc (prefs *<format>Preferences) Copy() <format>Preferences {\n    return *prefs\n}\n```\n\n### Multiple Documents\nDecoders should support reading multiple documents:\n```go\nfunc (dec *<format>Decoder) Decode() (*CandidateNode, error) {\n    if dec.finished {\n        return nil, io.EOF\n    }\n    // ... decode next document ...\n    if noMoreDocuments {\n        dec.finished = true\n    }\n    return candidate, nil\n}\n```\n\n---\n\n# Adding a New Operator\n\nThis guide explains how to add a new operator to yq. Operators are the core of yq's expression language and process `CandidateNode` objects without requiring modifications to `candidate_node.go` itself.\n\n## Overview\n\nOperators transform data by implementing a handler function that processes a `Context` containing `CandidateNode` objects. Each operator is:\n\n1. Defined as an `operationType` in `operation.go`\n2. Registered in the lexer in `lexer_participle.go`\n3. Implemented in its own `operator_<type>.go` file\n4. Tested in `operator_<type>_test.go`\n5. Documented in `pkg/yqlib/doc/operators/headers/<type>.md`\n\n## Architecture\n\n### Key Files\n\n- `pkg/yqlib/operation.go` - Defines `operationType` and operator registry\n- `pkg/yqlib/lexer_participle.go` - Registers operators with their syntax patterns\n- `pkg/yqlib/operator_<type>.go` - Operator implementation\n- `pkg/yqlib/operator_<type>_test.go` - Operator tests using `expressionScenario`\n- `pkg/yqlib/doc/operators/headers/<type>.md` - Documentation header\n\n### Core Types\n\n**operationType:**\n```go\ntype operationType struct {\n    Type                 string          // Unique operator name (e.g., \"REVERSE\")\n    NumArgs              uint            // Number of arguments (0 for no args)\n    Precedence           uint            // Operator precedence (higher = higher precedence)\n    Handler              operatorHandler // The function that executes the operator\n    CheckForPostTraverse bool            // Whether to apply post-traversal logic\n    ToString             func(*Operation) string // Custom string representation\n}\n```\n\n**operatorHandler signature:**\n```go\ntype operatorHandler func(*dataTreeNavigator, Context, *ExpressionNode) (Context, error)\n```\n\n**expressionScenario for tests:**\n```go\ntype expressionScenario struct {\n    description      string\n    subdescription   string\n    document         string\n    expression       string\n    expected         []string\n    skipDoc          bool\n    expectedError    string\n}\n```\n\n## Step-by-Step: Adding a New Operator\n\n### Step 1: Create the Operator Implementation File\n\nCreate `pkg/yqlib/operator_<type>.go` implementing the operator handler function:\n- Implement the `operatorHandler` function signature\n- Process nodes from `context.MatchingNodes`\n- Return a new `Context` with results using `context.ChildContext()`\n- Use `candidate.CreateReplacement()` or `candidate.CreateReplacementWithComments()` to create new nodes\n- Handle errors gracefully with meaningful error messages\n\nSee `operator_reverse.go` or `operator_keys.go` for examples.\n\n### Step 2: Register the Operator in operation.go\n\nAdd the operator type definition to `pkg/yqlib/operation.go`:\n\n```go\nvar <type>OpType = &operationType{\n    Type:       \"<TYPE>\",          // All caps, matches pattern in lexer\n    NumArgs:    0,                 // 0 for no args, 1+ for args\n    Precedence: 50,                // Typical range: 40-55\n    Handler:    <type>Operator,    // Reference to handler function\n}\n```\n\n**Precedence guidelines:**\n- 10-20: Logical operators (OR, AND, UNION)\n- 30: Pipe operator\n- 40: Assignment and comparison operators\n- 42: Arithmetic operators (ADD, SUBTRACT, MULTIPLY, DIVIDE)\n- 50-52: Most other operators\n- 55: High precedence (e.g., GET_VARIABLE)\n\n**Optional fields:**\n- `CheckForPostTraverse: true` - If your operator can have another directly after it without the pipe character. Most of the time this is false.\n- `ToString: customToString` - Custom string representation (rarely needed)\n\n### Step 3: Register the Operator in lexer_participle.go\n\nEdit `pkg/yqlib/lexer_participle.go` to add the operator to the lexer rules:\n- Use `simpleOp()` for simple keyword patterns\n- Use object syntax for regex patterns or complex syntax\n- Support optional characters with `_?` and aliases with `|`\n\nSee existing operators in `lexer_participle.go` for pattern examples.\n\n### Step 4: Create Tests (Mandatory)\n\nCreate `pkg/yqlib/operator_<type>_test.go` using the `expressionScenario` pattern:\n- Define test scenarios with `description`, `document`, `expression`, and `expected` fields\n- `expected` is a slice of strings showing output format: `\"D<doc>, P[<path>], (<tag>)::<value>\\n\"`\n- Set `skipDoc: true` for edge cases you don't want in generated documentation\n- Include `subdescription` for longer test names\n- Set `expectedError` if testing error cases\n- Create main test function that iterates over scenarios\n- The main test function should use `documentScenarios` to ensure testcase documentation is generated.\n\nTest coverage must include:\n- Basic data types and nested structures\n- Edge cases (empty inputs, special characters, type errors)\n- Multiple outputs if applicable\n- Format-specific features\n\nSee `operator_reverse_test.go` for a simple example and `operator_keys_test.go` for complex cases.\n\n### Step 5: Create Documentation Header\n\nCreate `pkg/yqlib/doc/operators/headers/<type>.md`:\n- Use the exact operator name as the title\n- Include a concise 1-2 sentence summary\n- Add additional context or examples if the operator is complex\n\nSee existing headers in `doc/operators/headers/` for examples.\n\n## Working with Context and CandidateNode\n\n### Context Management\n- `context.ChildContext(results)` - Create child context with results\n- `context.GetVariable(\"varName\")` - Get variables stored in context\n- `context.SetVariable(\"varName\", value)` - Set variables in context\n\n### CandidateNode Operations\n- `candidate.CreateReplacement(ScalarNode, \"!!str\", stringValue)` - Create a replacement node\n- `candidate.CreateReplacementWithComments(SequenceNode, \"!!seq\", candidate.Style)` - With style preserved\n- `candidate.Kind` - The node type (ScalarNode, SequenceNode, MappingNode)\n- `candidate.Tag` - The YAML tag (!!str, !!int, etc.)\n- `candidate.Value` - The scalar value (for ScalarNode only)\n- `candidate.Content` - Child nodes (for SequenceNode and MappingNode)\n- `candidate.guessTagFromCustomType()` - Infer the tag from Go type\n- `candidate.AsList()` - Convert to a list representation\n\n## Key Points\n\n✅ **DO:**\n- Implement the operator handler with the correct signature\n- Register in `operation.go` with appropriate precedence\n- Add the lexer pattern in `lexer_participle.go`\n- Write comprehensive tests covering normal and edge cases\n- Create a documentation header in `doc/operators/headers/`\n- Use `Context.ChildContext()` for proper context threading\n- Handle all node types gracefully\n- Return meaningful error messages\n\n❌ **DON'T:**\n- Modify `candidate_node.go` (operators shouldn't need this)\n- Modify core navigation or evaluation logic\n- Bypass the handler function pattern\n- Add format-specific or operator-specific fields to `CandidateNode`\n- Skip tests or documentation\n\n## Examples\n\nRefer to existing operator implementations for patterns:\n\n- **No-argument operator**: `operator_reverse.go` - Processes arrays/sequences\n- **Single-argument operator**: `operator_map.go` - Takes an expression argument\n- **Complex multi-output**: `operator_keys.go` - Produces multiple results\n- **With preferences**: `operator_to_number.go` - Configuration options\n- **Error handling**: `operator_error.go` - Control flow with errors\n- **String operations**: `operator_strings.go` - Multiple related operators\n\n## Testing Patterns\n\nRefer to existing test files for specific patterns:\n- Basic expression tests in `operator_reverse_test.go`\n- Multi-output tests in `operator_keys_test.go`\n- Error handling tests in `operator_error_test.go`\n- Tests with `skipDoc` flag to exclude from generated documentation\n\n## Common Patterns\n\nRefer to existing operator implementations for these patterns:\n- Simple transformation: see `operator_reverse.go`\n- Type checking: see `operator_error.go`\n- Working with arguments: see `operator_map.go`\n- Post-traversal operators: see `operator_with.go`\n"},"files":{"AGENTS.md":"# yq — agent instructions\n\n## ⚠️ MANDATORY: GitHub agent disclosure\n\n**Always required. No exceptions.**\n\nWhenever you perform **any** GitHub action on behalf of the user, you **must** disclose that an AI agent (Cursor) wrote the content and is acting on the user's behalf — **not the user personally**. Do this **before** submitting; never post first and add the disclosure later.\n\nApplies to **all** GitHub interactions, including:\n\n- Pull requests (titles, descriptions, and reviews)\n- PR comments and inline review comments\n- Issues (new issues, comments, and updates)\n- Any other post or reply on GitHub\n\n**How to disclose:** Put it prominently at the **top** of every PR description, review body, comment, or issue. Use wording like:\n\n\nInline review comments must include a short disclosure too (e.g. `> Generated by Cursor acting on the user's behalf, not the user personally.`).\n\n**Never** submit a GitHub action without this disclosure.\n\n---\n\nAlways run the spellcheck before raising a PR:\n\n```bash\nbash scripts/spelling.sh\n```\n\nThis is also included in the full CI pipeline via `make local test`.\n\n## Cursor Cloud specific instructions\n\n### Overview\n\n**yq** is a Go CLI for querying and transforming YAML, JSON, XML, INI, and other structured formats. There are no long-running services — development is build-and-test against a local `./yq` binary.\n\n### Prerequisites\n\n- **Go ≥ 1.25** (see `go.mod`)\n- **Bash** (acceptance tests)\n- **Docker/Podman** is optional; use `make local <target>` to run natively when containers are unavailable\n\n### PATH\n\nAfter `scripts/devtools.sh`, add Go tool binaries to PATH:\n\n```bash\nexport PATH=\"$HOME/go/bin:$PATH\"\n```\n\n`golangci-lint` and `typos` install to `$HOME/go/bin`; `gosec` installs to `./bin/gosec` in the repo root.\n\n### Common commands (local, no Docker)\n\n| Task | Command |\n|------|---------|\n| Install dev tools | `bash scripts/devtools.sh` |\n| Vendor dependencies | `make local vendor` |\n| Build binary | `go build -o yq .` or `make local build` |\n| Format | `make local format` |\n| Lint | `make local check` |\n| Unit tests | `make local test` or `bash scripts/test.sh` |\n| Acceptance (E2E) | `bash scripts/acceptance.sh` (requires `./yq` built first) |\n\n`make local build` runs the full CI chain (format → spelling → gosec → lint → unit tests → build → acceptance). For a faster loop, build with `go build -o yq .` and run `bash scripts/acceptance.sh`.\n\n### Caveats\n\n- **`make` without `local`** tries Docker/Podman (`Dockerfile.dev`). In Cloud Agent VMs without Docker, always prefix with `make local`.\n- **Spelling step** uses `typos` (installed by `scripts/devtools.sh`).\n- **`make local test` / `scripts/check.sh`** require `golangci-lint` on PATH (`devtools.sh`).\n\n---\n\n# General rules\n✅ **DO:**\n- You can use ./yq with the `--debug-node-info` flag to get a deeper understanding of the ast.\n- run ./scripts/format.sh to format the code; then ./scripts/check.sh lint and finally ./scripts/spelling.sh to check spelling.\n- Add comprehensive tests to cover the changes\n- Run test suite to ensure there is no regression\n- Use UK english spelling\n- **Follow the mandatory GitHub agent disclosure rule above** on every GitHub action — no exceptions\n\n❌ **DON'T:**\n- Git add or commit\n- Add comments to functions that are self-explanatory\n- **Post to GitHub without the mandatory agent disclosure** (PRs, reviews, comments, issues, or any other GitHub interaction)\n\n\n\n# Adding a New Encoder/Decoder\n\nThis guide explains how to add support for a new format (encoder/decoder) to yq without modifying `candidate_node.go`.\n\n## Overview\n\nThe encoder/decoder architecture in yq is based on two main interfaces:\n\n- **Encoder**: Converts a `CandidateNode` to output in a specific format\n- **Decoder**: Reads input in a specific format and creates a `CandidateNode`\n\nEach format is registered in `pkg/yqlib/format.go` and made available through factory functions.\n\n## Architecture\n\n### Key Files\n\n- `pkg/yqlib/encoder.go` - Defines the `Encoder` interface\n- `pkg/yqlib/decoder.go` - Defines the `Decoder` interface\n- `pkg/yqlib/format.go` - Format registry and factory functions\n- `pkg/yqlib/operator_encoder_decoder.go` - Encode/decode operators\n- `pkg/yqlib/encoder_*.go` - Encoder implementations\n- `pkg/yqlib/decoder_*.go` - Decoder implementations\n\n### Interfaces\n\n**Encoder Interface:**\n```go\ntype Encoder interface {\n    Encode(writer io.Writer, node *CandidateNode) error\n    PrintDocumentSeparator(writer io.Writer) error\n    PrintLeadingContent(writer io.Writer, content string) error\n    CanHandleAliases() bool\n}\n```\n\n**Decoder Interface:**\n```go\ntype Decoder interface {\n    Init(reader io.Reader) error\n    Decode() (*CandidateNode, error)\n}\n```\n\n## Step-by-Step: Adding a New Encoder/Decoder\n\n### Step 1: Create the Encoder File\n\nCreate `pkg/yqlib/encoder_<format>.go` implementing the `Encoder` interface:\n- `Encode()` - Convert a `CandidateNode` to your format and write to the output writer\n- `PrintDocumentSeparator()` - Handle document separators if your format requires them\n- `PrintLeadingContent()` - Handle leading content/comments if supported\n- `CanHandleAliases()` - Return whether your format supports YAML aliases\n\nSee `encoder_json.go` or `encoder_base64.go` for examples.\n\n### Step 2: Create the Decoder File\n\nCreate `pkg/yqlib/decoder_<format>.go` implementing the `Decoder` interface:\n- `Init()` - Initialize the decoder with the input reader and set up any needed state\n- `Decode()` - Decode one document from the input and return a `CandidateNode`, or `io.EOF` when finished\n\nSee `decoder_json.go` or `decoder_base64.go` for examples.\n\n### Step 3: Create Tests (Mandatory)\n\nCreate a test file `pkg/yqlib/<format>_test.go` using the `formatScenario` pattern:\n- Define test scenarios as `formatScenario` structs with fields: `description`, `input`, `expected`, `scenarioType`\n- `scenarioType` can be `\"decode\"` (test decoding to YAML) or `\"roundtrip\"` (encode/decode preservation)\n- Create a helper function `test<Format>Scenario()` that switches on `scenarioType`\n- Create main test function `Test<Format>FormatScenarios()` that iterates over scenarios\n- The main test function should use `documentScenarios` to ensure testcase documentation is generated.\n\nTest coverage must include:\n- Basic data types (scalars, arrays, objects/maps)\n- Nested structures\n- Edge cases (empty inputs, special characters, escape sequences)\n- Format-specific features or syntax\n- Round-trip tests: decode → encode → decode should preserve data\n\nSee `hcl_test.go` for a complete example.\n\n### Step 4: Register the Format in format.go\n\nEdit `pkg/yqlib/format.go`:\n\n1. Add a new format variable:\n   - `\"<format>\"` is the formal name (e.g., \"json\", \"yaml\")\n   - `[]string{...}` contains short aliases (can be empty)\n   - The first function creates an encoder (can be nil for encode-only formats)\n   - The second function creates a decoder (can be nil for decode-only formats)\n\n2. Add the format to the `Formats` slice in the same file\n\nSee existing formats in `format.go` for the exact structure.\n\n### Step 5: Handle Encoder Configuration (if needed)\n\nIf your format has preferences/configuration options:\n\n1. Create a preferences struct with your configuration fields\n2. Update the encoder to accept preferences in its factory function\n3. Update `format.go` to pass the configured preferences\n4. Update `operator_encoder_decoder.go` if special indent handling is needed (see existing formats like JSON and YAML for the pattern)\n\nThis pattern is optional and only needed if your format has user-configurable options.\n\n## Build Tags\n\nUse build tags to allow optional compilation of formats:\n- Add `//go:build !yq_no<format>` at the top of your encoder and decoder files\n- Create a no-build version in `pkg/yqlib/no_<format>.go` that returns nil for encoder/decoder factories\n\nThis allows users to compile yq without certain formats using: `go build -tags yq_no<format>`\n\n## Working with CandidateNode\n\nThe `CandidateNode` struct represents a YAML node with:\n- `Kind`: The node type (ScalarNode, SequenceNode, MappingNode)\n- `Tag`: The YAML tag (e.g., \"!!str\", \"!!int\", \"!!map\")\n- `Value`: The scalar value (for ScalarNode only)\n- `Content`: Child nodes (for SequenceNode and MappingNode)\n\nKey methods:\n- `node.guessTagFromCustomType()` - Infer the tag from Go type\n- `node.AsList()` - Convert to a list for processing\n- `node.CreateReplacement()` - Create a new replacement node\n- `NewCandidate()` - Create a new CandidateNode\n\n## Key Points\n\n✅ **DO:**\n- Implement only the `Encoder` and `Decoder` interfaces\n- Register your format in `format.go` only\n- Keep format-specific logic in your encoder/decoder files\n- Use the candidate_node style attribute to store style information for round-trip. Ask if this needs to be updated with new styles.\n- Use build tags for optional compilation\n- Add comprehensive tests\n- Run the specific encoder/decoder test (e.g. <format>_test.go) whenever you make ay changes to the encoder_<format> or decoder_<format>\n- Handle errors gracefully\n- Add the no build directive, like the xml encoder and decoder, that enables a minimal yq builds. e.g.  `//go:build !yq_<format>`. Be sure to also update the build_small-yq.sh and build-tinygo-yq.sh to not include the new format.\n\n❌ **DON'T:**\n- Modify `candidate_node.go` to add format-specific logic\n- Add format-specific fields to `CandidateNode`\n- Create special cases in core navigation or evaluation logic\n- Bypass the encoder/decoder interfaces\n- Use candidate_node tag attribute for anything other than indicate the data type\n\n## Examples\n\nRefer to existing format implementations for patterns:\n\n- **Simple encoder/decoder**: `encoder_json.go`, `decoder_json.go`\n- **Complex with preferences**: `encoder_yaml.go`, `decoder_yaml.go`\n- **Encoder-only**: `encoder_sh.go` (ShFormat has nil decoder)\n- **String-only operations**: `encoder_base64.go`, `decoder_base64.go`\n\n## Testing Your Implementation (Mandatory)\n\nTests must be implemented in `<format>_test.go` following the `formatScenario` pattern:\n\n1. **Create test scenarios** using the `formatScenario` struct with fields:\n   - `description`: Brief description of what's being tested\n   - `input`: Sample input in your format\n   - `expected`: Expected output (typically in YAML for decode tests)\n   - `scenarioType`: Either `\"decode\"` or `\"roundtrip\"`\n\n2. **Test coverage must include:**\n   - Basic data types (scalars, arrays, objects/maps)\n   - Nested structures\n   - Edge cases (empty inputs, special characters, escape sequences)\n   - Format-specific features or syntax\n   - Round-trip tests: decode → encode → decode should preserve data\n\n3. **Test function pattern:**\n   - `test<Format>Scenario()`: Helper function that switches on `scenarioType`\n   - `Test<Format>FormatScenarios()`: Main test function that iterates over scenarios\n\n4. **Example from existing formats:**\n   - See `hcl_test.go` for a complete example\n   - See `yaml_test.go` for YAML-specific patterns\n   - See `json_test.go` for more complex scenarios\n\n## Common Patterns\n\n### Format with Indentation\nUse preferences to control output formatting:\n```go\ntype <format>Preferences struct {\n    Indent int\n}\n\nfunc (prefs *<format>Preferences) Copy() <format>Preferences {\n    return *prefs\n}\n```\n\n### Multiple Documents\nDecoders should support reading multiple documents:\n```go\nfunc (dec *<format>Decoder) Decode() (*CandidateNode, error) {\n    if dec.finished {\n        return nil, io.EOF\n    }\n    // ... decode next document ...\n    if noMoreDocuments {\n        dec.finished = true\n    }\n    return candidate, nil\n}\n```\n\n---\n\n# Adding a New Operator\n\nThis guide explains how to add a new operator to yq. Operators are the core of yq's expression language and process `CandidateNode` objects without requiring modifications to `candidate_node.go` itself.\n\n## Overview\n\nOperators transform data by implementing a handler function that processes a `Context` containing `CandidateNode` objects. Each operator is:\n\n1. Defined as an `operationType` in `operation.go`\n2. Registered in the lexer in `lexer_participle.go`\n3. Implemented in its own `operator_<type>.go` file\n4. Tested in `operator_<type>_test.go`\n5. Documented in `pkg/yqlib/doc/operators/headers/<type>.md`\n\n## Architecture\n\n### Key Files\n\n- `pkg/yqlib/operation.go` - Defines `operationType` and operator registry\n- `pkg/yqlib/lexer_participle.go` - Registers operators with their syntax patterns\n- `pkg/yqlib/operator_<type>.go` - Operator implementation\n- `pkg/yqlib/operator_<type>_test.go` - Operator tests using `expressionScenario`\n- `pkg/yqlib/doc/operators/headers/<type>.md` - Documentation header\n\n### Core Types\n\n**operationType:**\n```go\ntype operationType struct {\n    Type                 string          // Unique operator name (e.g., \"REVERSE\")\n    NumArgs              uint            // Number of arguments (0 for no args)\n    Precedence           uint            // Operator precedence (higher = higher precedence)\n    Handler              operatorHandler // The function that executes the operator\n    CheckForPostTraverse bool            // Whether to apply post-traversal logic\n    ToString             func(*Operation) string // Custom string representation\n}\n```\n\n**operatorHandler signature:**\n```go\ntype operatorHandler func(*dataTreeNavigator, Context, *ExpressionNode) (Context, error)\n```\n\n**expressionScenario for tests:**\n```go\ntype expressionScenario struct {\n    description      string\n    subdescription   string\n    document         string\n    expression       string\n    expected         []string\n    skipDoc          bool\n    expectedError    string\n}\n```\n\n## Step-by-Step: Adding a New Operator\n\n### Step 1: Create the Operator Implementation File\n\nCreate `pkg/yqlib/operator_<type>.go` implementing the operator handler function:\n- Implement the `operatorHandler` function signature\n- Process nodes from `context.MatchingNodes`\n- Return a new `Context` with results using `context.ChildContext()`\n- Use `candidate.CreateReplacement()` or `candidate.CreateReplacementWithComments()` to create new nodes\n- Handle errors gracefully with meaningful error messages\n\nSee `operator_reverse.go` or `operator_keys.go` for examples.\n\n### Step 2: Register the Operator in operation.go\n\nAdd the operator type definition to `pkg/yqlib/operation.go`:\n\n```go\nvar <type>OpType = &operationType{\n    Type:       \"<TYPE>\",          // All caps, matches pattern in lexer\n    NumArgs:    0,                 // 0 for no args, 1+ for args\n    Precedence: 50,                // Typical range: 40-55\n    Handler:    <type>Operator,    // Reference to handler function\n}\n```\n\n**Precedence guidelines:**\n- 10-20: Logical operators (OR, AND, UNION)\n- 30: Pipe operator\n- 40: Assignment and comparison operators\n- 42: Arithmetic operators (ADD, SUBTRACT, MULTIPLY, DIVIDE)\n- 50-52: Most other operators\n- 55: High precedence (e.g., GET_VARIABLE)\n\n**Optional fields:**\n- `CheckForPostTraverse: true` - If your operator can have another directly after it without the pipe character. Most of the time this is false.\n- `ToString: customToString` - Custom string representation (rarely needed)\n\n### Step 3: Register the Operator in lexer_participle.go\n\nEdit `pkg/yqlib/lexer_participle.go` to add the operator to the lexer rules:\n- Use `simpleOp()` for simple keyword patterns\n- Use object syntax for regex patterns or complex syntax\n- Support optional characters with `_?` and aliases with `|`\n\nSee existing operators in `lexer_participle.go` for pattern examples.\n\n### Step 4: Create Tests (Mandatory)\n\nCreate `pkg/yqlib/operator_<type>_test.go` using the `expressionScenario` pattern:\n- Define test scenarios with `description`, `document`, `expression`, and `expected` fields\n- `expected` is a slice of strings showing output format: `\"D<doc>, P[<path>], (<tag>)::<value>\\n\"`\n- Set `skipDoc: true` for edge cases you don't want in generated documentation\n- Include `subdescription` for longer test names\n- Set `expectedError` if testing error cases\n- Create main test function that iterates over scenarios\n- The main test function should use `documentScenarios` to ensure testcase documentation is generated.\n\nTest coverage must include:\n- Basic data types and nested structures\n- Edge cases (empty inputs, special characters, type errors)\n- Multiple outputs if applicable\n- Format-specific features\n\nSee `operator_reverse_test.go` for a simple example and `operator_keys_test.go` for complex cases.\n\n### Step 5: Create Documentation Header\n\nCreate `pkg/yqlib/doc/operators/headers/<type>.md`:\n- Use the exact operator name as the title\n- Include a concise 1-2 sentence summary\n- Add additional context or examples if the operator is complex\n\nSee existing headers in `doc/operators/headers/` for examples.\n\n## Working with Context and CandidateNode\n\n### Context Management\n- `context.ChildContext(results)` - Create child context with results\n- `context.GetVariable(\"varName\")` - Get variables stored in context\n- `context.SetVariable(\"varName\", value)` - Set variables in context\n\n### CandidateNode Operations\n- `candidate.CreateReplacement(ScalarNode, \"!!str\", stringValue)` - Create a replacement node\n- `candidate.CreateReplacementWithComments(SequenceNode, \"!!seq\", candidate.Style)` - With style preserved\n- `candidate.Kind` - The node type (ScalarNode, SequenceNode, MappingNode)\n- `candidate.Tag` - The YAML tag (!!str, !!int, etc.)\n- `candidate.Value` - The scalar value (for ScalarNode only)\n- `candidate.Content` - Child nodes (for SequenceNode and MappingNode)\n- `candidate.guessTagFromCustomType()` - Infer the tag from Go type\n- `candidate.AsList()` - Convert to a list representation\n\n## Key Points\n\n✅ **DO:**\n- Implement the operator handler with the correct signature\n- Register in `operation.go` with appropriate precedence\n- Add the lexer pattern in `lexer_participle.go`\n- Write comprehensive tests covering normal and edge cases\n- Create a documentation header in `doc/operators/headers/`\n- Use `Context.ChildContext()` for proper context threading\n- Handle all node types gracefully\n- Return meaningful error messages\n\n❌ **DON'T:**\n- Modify `candidate_node.go` (operators shouldn't need this)\n- Modify core navigation or evaluation logic\n- Bypass the handler function pattern\n- Add format-specific or operator-specific fields to `CandidateNode`\n- Skip tests or documentation\n\n## Examples\n\nRefer to existing operator implementations for patterns:\n\n- **No-argument operator**: `operator_reverse.go` - Processes arrays/sequences\n- **Single-argument operator**: `operator_map.go` - Takes an expression argument\n- **Complex multi-output**: `operator_keys.go` - Produces multiple results\n- **With preferences**: `operator_to_number.go` - Configuration options\n- **Error handling**: `operator_error.go` - Control flow with errors\n- **String operations**: `operator_strings.go` - Multiple related operators\n\n## Testing Patterns\n\nRefer to existing test files for specific patterns:\n- Basic expression tests in `operator_reverse_test.go`\n- Multi-output tests in `operator_keys_test.go`\n- Error handling tests in `operator_error_test.go`\n- Tests with `skipDoc` flag to exclude from generated documentation\n\n## Common Patterns\n\nRefer to existing operator implementations for these patterns:\n- Simple transformation: see `operator_reverse.go`\n- Type checking: see `operator_error.go`\n- Working with arguments: see `operator_map.go`\n- Post-traversal operators: see `operator_with.go`\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"# yq — agent instructions\n\n## ⚠️ MANDATORY: GitHub agent disclosure\n\n**Always required. No exceptions.**\n\nWhenever you perform **any** GitHub action on behalf of the user, you **must** disclose that an AI agent (Cursor) wrote the content and is acting on the user's behalf — **not the user personally**. Do this **before** submitting; never post first and add the disclosure later.\n\nApplies to **all** GitHub interactions, including:\n\n- Pull requests (titles, descriptions, and reviews)\n- PR comments and inline review comments\n- Issues (new issues, comments, and updates)\n- Any other post or reply on GitHub\n\n**How to disclose:** Put it prominently at the **top** of every PR description, review body, comment, or issue. Use wording like:\n\n\nInline review comments must include a short disclosure too (e.g. `> Generated by Cursor acting on the user's behalf, not the user personally.`).\n\n**Never** submit a GitHub action without this disclosure.\n\n---\n\nAlways run the spellcheck before raising a PR:\n\n```bash\nbash scripts/spelling.sh\n```\n\nThis is also included in the full CI pipeline via `make local test`.\n\n## Cursor Cloud specific instructions\n\n### Overview\n\n**yq** is a Go CLI for querying and transforming YAML, JSON, XML, INI, and other structured formats. There are no long-running services — development is build-and-test against a local `./yq` binary.\n\n### Prerequisites\n\n- **Go ≥ 1.25** (see `go.mod`)\n- **Bash** (acceptance tests)\n- **Docker/Podman** is optional; use `make local <target>` to run natively when containers are unavailable\n\n### PATH\n\nAfter `scripts/devtools.sh`, add Go tool binaries to PATH:\n\n```bash\nexport PATH=\"$HOME/go/bin:$PATH\"\n```\n\n`golangci-lint` and `typos` install to `$HOME/go/bin`; `gosec` installs to `./bin/gosec` in the repo root.\n\n### Common commands (local, no Docker)\n\n| Task | Command |\n|------|---------|\n| Install dev tools | `bash scripts/devtools.sh` |\n| Vendor dependencies | `make local vendor` |\n| Build binary | `go build -o yq .` or `make local build` |\n| Format | `make local format` |\n| Lint | `make local check` |\n| Unit tests | `make local test` or `bash scripts/test.sh` |\n| Acceptance (E2E) | `bash scripts/acceptance.sh` (requires `./yq` built first) |\n\n`make local build` runs the full CI chain (format → spelling → gosec → lint → unit tests → build → acceptance). For a faster loop, build with `go build -o yq .` and run `bash scripts/acceptance.sh`.\n\n### Caveats\n\n- **`make` without `local`** tries Docker/Podman (`Dockerfile.dev`). In Cloud Agent VMs without Docker, always prefix with `make local`.\n- **Spelling step** uses `typos` (installed by `scripts/devtools.sh`).\n- **`make local test` / `scripts/check.sh`** require `golangci-lint` on PATH (`devtools.sh`).\n\n---\n\n# General rules\n✅ **DO:**\n- You can use ./yq with the `--debug-node-info` flag to get a deeper understanding of the ast.\n- run ./scripts/format.sh to format the code; then ./scripts/check.sh lint and finally ./scripts/spelling.sh to check spelling.\n- Add comprehensive tests to cover the changes\n- Run test suite to ensure there is no regression\n- Use UK english spelling\n- **Follow the mandatory GitHub agent disclosure rule above** on every GitHub action — no exceptions\n\n❌ **DON'T:**\n- Git add or commit\n- Add comments to functions that are self-explanatory\n- **Post to GitHub without the mandatory agent disclosure** (PRs, reviews, comments, issues, or any other GitHub interaction)\n\n\n\n# Adding a New Encoder/Decoder\n\nThis guide explains how to add support for a new format (encoder/decoder) to yq without modifying `candidate_node.go`.\n\n## Overview\n\nThe encoder/decoder architecture in yq is based on two main interfaces:\n\n- **Encoder**: Converts a `CandidateNode` to output in a specific format\n- **Decoder**: Reads input in a specific format and creates a `CandidateNode`\n\nEach format is registered in `pkg/yqlib/format.go` and made available through factory functions.\n\n## Architecture\n\n### Key Files\n\n- `pkg/yqlib/encoder.go` - Defines the `Encoder` interface\n- `pkg/yqlib/decoder.go` - Defines the `Decoder` interface\n- `pkg/yqlib/format.go` - Format registry and factory functions\n- `pkg/yqlib/operator_encoder_decoder.go` - Encode/decode operators\n- `pkg/yqlib/encoder_*.go` - Encoder implementations\n- `pkg/yqlib/decoder_*.go` - Decoder implementations\n\n### Interfaces\n\n**Encoder Interface:**\n```go\ntype Encoder interface {\n    Encode(writer io.Writer, node *CandidateNode) error\n    PrintDocumentSeparator(writer io.Writer) error\n    PrintLeadingContent(writer io.Writer, content string) error\n    CanHandleAliases() bool\n}\n```\n\n**Decoder Interface:**\n```go\ntype Decoder interface {\n    Init(reader io.Reader) error\n    Decode() (*CandidateNode, error)\n}\n```\n\n## Step-by-Step: Adding a New Encoder/Decoder\n\n### Step 1: Create the Encoder File\n\nCreate `pkg/yqlib/encoder_<format>.go` implementing the `Encoder` interface:\n- `Encode()` - Convert a `CandidateNode` to your format and write to the output writer\n- `PrintDocumentSeparator()` - Handle document separators if your format requires them\n- `PrintLeadingContent()` - Handle leading content/comments if supported\n- `CanHandleAliases()` - Return whether your format supports YAML aliases\n\nSee `encoder_json.go` or `encoder_base64.go` for examples.\n\n### Step 2: Create the Decoder File\n\nCreate `pkg/yqlib/decoder_<format>.go` implementing the `Decoder` interface:\n- `Init()` - Initialize the decoder with the input reader and set up any needed state\n- `Decode()` - Decode one document from the input and return a `CandidateNode`, or `io.EOF` when finished\n\nSee `decoder_json.go` or `decoder_base64.go` for examples.\n\n### Step 3: Create Tests (Mandatory)\n\nCreate a test file `pkg/yqlib/<format>_test.go` using the `formatScenario` pattern:\n- Define test scenarios as `formatScenario` structs with fields: `description`, `input`, `expected`, `scenarioType`\n- `scenarioType` can be `\"decode\"` (test decoding to YAML) or `\"roundtrip\"` (encode/decode preservation)\n- Create a helper function `test<Format>Scenario()` that switches on `scenarioType`\n- Create main test function `Test<Format>FormatScenarios()` that iterates over scenarios\n- The main test function should use `documentScenarios` to ensure testcase documentation is generated.\n\nTest coverage must include:\n- Basic data types (scalars, arrays, objects/maps)\n- Nested structures\n- Edge cases (empty inputs, special characters, escape sequences)\n- Format-specific features or syntax\n- Round-trip tests: decode → encode → decode should preserve data\n\nSee `hcl_test.go` for a complete example.\n\n### Step 4: Register the Format in format.go\n\nEdit `pkg/yqlib/format.go`:\n\n1. Add a new format variable:\n   - `\"<format>\"` is the formal name (e.g., \"json\", \"yaml\")\n   - `[]string{...}` contains short aliases (can be empty)\n   - The first function creates an encoder (can be nil for encode-only formats)\n   - The second function creates a decoder (can be nil for decode-only formats)\n\n2. Add the format to the `Formats` slice in the same file\n\nSee existing formats in `format.go` for the exact structure.\n\n### Step 5: Handle Encoder Configuration (if needed)\n\nIf your format has preferences/configuration options:\n\n1. Create a preferences struct with your configuration fields\n2. Update the encoder to accept preferences in its factory function\n3. Update `format.go` to pass the configured preferences\n4. Update `operator_encoder_decoder.go` if special indent handling is needed (see existing formats like JSON and YAML for the pattern)\n\nThis pattern is optional and only needed if your format has user-configurable options.\n\n## Build Tags\n\nUse build tags to allow optional compilation of formats:\n- Add `//go:build !yq_no<format>` at the top of your encoder and decoder files\n- Create a no-build version in `pkg/yqlib/no_<format>.go` that returns nil for encoder/decoder factories\n\nThis allows users to compile yq without certain formats using: `go build -tags yq_no<format>`\n\n## Working with CandidateNode\n\nThe `CandidateNode` struct represents a YAML node with:\n- `Kind`: The node type (ScalarNode, SequenceNode, MappingNode)\n- `Tag`: The YAML tag (e.g., \"!!str\", \"!!int\", \"!!map\")\n- `Value`: The scalar value (for ScalarNode only)\n- `Content`: Child nodes (for SequenceNode and MappingNode)\n\nKey methods:\n- `node.guessTagFromCustomType()` - Infer the tag from Go type\n- `node.AsList()` - Convert to a list for processing\n- `node.CreateReplacement()` - Create a new replacement node\n- `NewCandidate()` - Create a new CandidateNode\n\n## Key Points\n\n✅ **DO:**\n- Implement only the `Encoder` and `Decoder` interfaces\n- Register your format in `format.go` only\n- Keep format-specific logic in your encoder/decoder files\n- Use the candidate_node style attribute to store style information for round-trip. Ask if this needs to be updated with new styles.\n- Use build tags for optional compilation\n- Add comprehensive tests\n- Run the specific encoder/decoder test (e.g. <format>_test.go) whenever you make ay changes to the encoder_<format> or decoder_<format>\n- Handle errors gracefully\n- Add the no build directive, like the xml encoder and decoder, that enables a minimal yq builds. e.g.  `//go:build !yq_<format>`. Be sure to also update the build_small-yq.sh and build-tinygo-yq.sh to not include the new format.\n\n❌ **DON'T:**\n- Modify `candidate_node.go` to add format-specific logic\n- Add format-specific fields to `CandidateNode`\n- Create special cases in core navigation or evaluation logic\n- Bypass the encoder/decoder interfaces\n- Use candidate_node tag attribute for anything other than indicate the data type\n\n## Examples\n\nRefer to existing format implementations for patterns:\n\n- **Simple encoder/decoder**: `encoder_json.go`, `decoder_json.go`\n- **Complex with preferences**: `encoder_yaml.go`, `decoder_yaml.go`\n- **Encoder-only**: `encoder_sh.go` (ShFormat has nil decoder)\n- **String-only operations**: `encoder_base64.go`, `decoder_base64.go`\n\n## Testing Your Implementation (Mandatory)\n\nTests must be implemented in `<format>_test.go` following the `formatScenario` pattern:\n\n1. **Create test scenarios** using the `formatScenario` struct with fields:\n   - `description`: Brief description of what's being tested\n   - `input`: Sample input in your format\n   - `expected`: Expected output (typically in YAML for decode tests)\n   - `scenarioType`: Either `\"decode\"` or `\"roundtrip\"`\n\n2. **Test coverage must include:**\n   - Basic data types (scalars, arrays, objects/maps)\n   - Nested structures\n   - Edge cases (empty inputs, special characters, escape sequences)\n   - Format-specific features or syntax\n   - Round-trip tests: decode → encode → decode should preserve data\n\n3. **Test function pattern:**\n   - `test<Format>Scenario()`: Helper function that switches on `scenarioType`\n   - `Test<Format>FormatScenarios()`: Main test function that iterates over scenarios\n\n4. **Example from existing formats:**\n   - See `hcl_test.go` for a complete example\n   - See `yaml_test.go` for YAML-specific patterns\n   - See `json_test.go` for more complex scenarios\n\n## Common Patterns\n\n### Format with Indentation\nUse preferences to control output formatting:\n```go\ntype <format>Preferences struct {\n    Indent int\n}\n\nfunc (prefs *<format>Preferences) Copy() <format>Preferences {\n    return *prefs\n}\n```\n\n### Multiple Documents\nDecoders should support reading multiple documents:\n```go\nfunc (dec *<format>Decoder) Decode() (*CandidateNode, error) {\n    if dec.finished {\n        return nil, io.EOF\n    }\n    // ... decode next document ...\n    if noMoreDocuments {\n        dec.finished = true\n    }\n    return candidate, nil\n}\n```\n\n---\n\n# Adding a New Operator\n\nThis guide explains how to add a new operator to yq. Operators are the core of yq's expression language and process `CandidateNode` objects without requiring modifications to `candidate_node.go` itself.\n\n## Overview\n\nOperators transform data by implementing a handler function that processes a `Context` containing `CandidateNode` objects. Each operator is:\n\n1. Defined as an `operationType` in `operation.go`\n2. Registered in the lexer in `lexer_participle.go`\n3. Implemented in its own `operator_<type>.go` file\n4. Tested in `operator_<type>_test.go`\n5. Documented in `pkg/yqlib/doc/operators/headers/<type>.md`\n\n## Architecture\n\n### Key Files\n\n- `pkg/yqlib/operation.go` - Defines `operationType` and operator registry\n- `pkg/yqlib/lexer_participle.go` - Registers operators with their syntax patterns\n- `pkg/yqlib/operator_<type>.go` - Operator implementation\n- `pkg/yqlib/operator_<type>_test.go` - Operator tests using `expressionScenario`\n- `pkg/yqlib/doc/operators/headers/<type>.md` - Documentation header\n\n### Core Types\n\n**operationType:**\n```go\ntype operationType struct {\n    Type                 string          // Unique operator name (e.g., \"REVERSE\")\n    NumArgs              uint            // Number of arguments (0 for no args)\n    Precedence           uint            // Operator precedence (higher = higher precedence)\n    Handler              operatorHandler // The function that executes the operator\n    CheckForPostTraverse bool            // Whether to apply post-traversal logic\n    ToString             func(*Operation) string // Custom string representation\n}\n```\n\n**operatorHandler signature:**\n```go\ntype operatorHandler func(*dataTreeNavigator, Context, *ExpressionNode) (Context, error)\n```\n\n**expressionScenario for tests:**\n```go\ntype expressionScenario struct {\n    description      string\n    subdescription   string\n    document         string\n    expression       string\n    expected         []string\n    skipDoc          bool\n    expectedError    string\n}\n```\n\n## Step-by-Step: Adding a New Operator\n\n### Step 1: Create the Operator Implementation File\n\nCreate `pkg/yqlib/operator_<type>.go` implementing the operator handler function:\n- Implement the `operatorHandler` function signature\n- Process nodes from `context.MatchingNodes`\n- Return a new `Context` with results using `context.ChildContext()`\n- Use `candidate.CreateReplacement()` or `candidate.CreateReplacementWithComments()` to create new nodes\n- Handle errors gracefully with meaningful error messages\n\nSee `operator_reverse.go` or `operator_keys.go` for examples.\n\n### Step 2: Register the Operator in operation.go\n\nAdd the operator type definition to `pkg/yqlib/operation.go`:\n\n```go\nvar <type>OpType = &operationType{\n    Type:       \"<TYPE>\",          // All caps, matches pattern in lexer\n    NumArgs:    0,                 // 0 for no args, 1+ for args\n    Precedence: 50,                // Typical range: 40-55\n    Handler:    <type>Operator,    // Reference to handler function\n}\n```\n\n**Precedence guidelines:**\n- 10-20: Logical operators (OR, AND, UNION)\n- 30: Pipe operator\n- 40: Assignment and comparison operators\n- 42: Arithmetic operators (ADD, SUBTRACT, MULTIPLY, DIVIDE)\n- 50-52: Most other operators\n- 55: High precedence (e.g., GET_VARIABLE)\n\n**Optional fields:**\n- `CheckForPostTraverse: true` - If your operator can have another directly after it without the pipe character. Most of the time this is false.\n- `ToString: customToString` - Custom string representation (rarely needed)\n\n### Step 3: Register the Operator in lexer_participle.go\n\nEdit `pkg/yqlib/lexer_participle.go` to add the operator to the lexer rules:\n- Use `simpleOp()` for simple keyword patterns\n- Use object syntax for regex patterns or complex syntax\n- Support optional characters with `_?` and aliases with `|`\n\nSee existing operators in `lexer_participle.go` for pattern examples.\n\n### Step 4: Create Tests (Mandatory)\n\nCreate `pkg/yqlib/operator_<type>_test.go` using the `expressionScenario` pattern:\n- Define test scenarios with `description`, `document`, `expression`, and `expected` fields\n- `expected` is a slice of strings showing output format: `\"D<doc>, P[<path>], (<tag>)::<value>\\n\"`\n- Set `skipDoc: true` for edge cases you don't want in generated documentation\n- Include `subdescription` for longer test names\n- Set `expectedError` if testing error cases\n- Create main test function that iterates over scenarios\n- The main test function should use `documentScenarios` to ensure testcase documentation is generated.\n\nTest coverage must include:\n- Basic data types and nested structures\n- Edge cases (empty inputs, special characters, type errors)\n- Multiple outputs if applicable\n- Format-specific features\n\nSee `operator_reverse_test.go` for a simple example and `operator_keys_test.go` for complex cases.\n\n### Step 5: Create Documentation Header\n\nCreate `pkg/yqlib/doc/operators/headers/<type>.md`:\n- Use the exact operator name as the title\n- Include a concise 1-2 sentence summary\n- Add additional context or examples if the operator is complex\n\nSee existing headers in `doc/operators/headers/` for examples.\n\n## Working with Context and CandidateNode\n\n### Context Management\n- `context.ChildContext(results)` - Create child context with results\n- `context.GetVariable(\"varName\")` - Get variables stored in context\n- `context.SetVariable(\"varName\", value)` - Set variables in context\n\n### CandidateNode Operations\n- `candidate.CreateReplacement(ScalarNode, \"!!str\", stringValue)` - Create a replacement node\n- `candidate.CreateReplacementWithComments(SequenceNode, \"!!seq\", candidate.Style)` - With style preserved\n- `candidate.Kind` - The node type (ScalarNode, SequenceNode, MappingNode)\n- `candidate.Tag` - The YAML tag (!!str, !!int, etc.)\n- `candidate.Value` - The scalar value (for ScalarNode only)\n- `candidate.Content` - Child nodes (for SequenceNode and MappingNode)\n- `candidate.guessTagFromCustomType()` - Infer the tag from Go type\n- `candidate.AsList()` - Convert to a list representation\n\n## Key Points\n\n✅ **DO:**\n- Implement the operator handler with the correct signature\n- Register in `operation.go` with appropriate precedence\n- Add the lexer pattern in `lexer_participle.go`\n- Write comprehensive tests covering normal and edge cases\n- Create a documentation header in `doc/operators/headers/`\n- Use `Context.ChildContext()` for proper context threading\n- Handle all node types gracefully\n- Return meaningful error messages\n\n❌ **DON'T:**\n- Modify `candidate_node.go` (operators shouldn't need this)\n- Modify core navigation or evaluation logic\n- Bypass the handler function pattern\n- Add format-specific or operator-specific fields to `CandidateNode`\n- Skip tests or documentation\n\n## Examples\n\nRefer to existing operator implementations for patterns:\n\n- **No-argument operator**: `operator_reverse.go` - Processes arrays/sequences\n- **Single-argument operator**: `operator_map.go` - Takes an expression argument\n- **Complex multi-output**: `operator_keys.go` - Produces multiple results\n- **With preferences**: `operator_to_number.go` - Configuration options\n- **Error handling**: `operator_error.go` - Control flow with errors\n- **String operations**: `operator_strings.go` - Multiple related operators\n\n## Testing Patterns\n\nRefer to existing test files for specific patterns:\n- Basic expression tests in `operator_reverse_test.go`\n- Multi-output tests in `operator_keys_test.go`\n- Error handling tests in `operator_error_test.go`\n- Tests with `skipDoc` flag to exclude from generated documentation\n\n## Common Patterns\n\nRefer to existing operator implementations for these patterns:\n- Simple transformation: see `operator_reverse.go`\n- Type checking: see `operator_error.go`\n- Working with arguments: see `operator_map.go`\n- Post-traversal operators: see `operator_with.go`\n","category":"root","tokens":4858}]}