{"owner":"MontFerret","repo":"ferret","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md"],"skills":{"AGENTS.md":"# AGENTS.md\n\nThis file is the canonical operating guide for coding agents working in this repository. It is written for Ferret v2 only. If repository documentation conflicts with this file, prefer Makefile, go.mod, and .github/workflows/build.yml for commands, toolchain, and CI behavior.\n\n## Repo snapshot\n\n* Module path: github.com/MontFerret/ferret/v2\n* Go version: 1.23+\n* Toolchain in go.mod: go1.24.5\n* This repository root is Ferret v2. Do not mix assumptions from the separate v1 branch.\n* High-level flow: Engine -> compiler -> bytecode.Program -> vm.VM\n\n## Architectural mental model\n\nFerret v2 is a compiled query language and runtime.\n\nPrimary pipeline:\n\nsource -> parser -> diagnostics/AST -> compiler -> bytecode.Program -> vm.VM -> runtime values/results\n\nAgents should reason about changes by pipeline stage and ownership boundary:\n\n* Source identity, source ranges, and source-origin behavior usually begin in pkg/source and affect parser, diagnostics, compiler, formatter, or tooling call sites.\n* Syntax changes usually begin in grammar/parser and continue into compiler lowering.\n* Diagnostic changes usually involve pkg/diagnostics plus the parser/compiler/runtime call sites that create or wrap the diagnostic.\n* Semantic/runtime changes usually live in compiler, runtime, or VM.\n* Bytecode changes usually require coordinated updates in pkg/bytecode, compiler emission, VM execution, and low-level tooling such as pkg/asm or debugger metadata.\n* Runtime value behavior usually belongs in pkg/runtime; VM, stdlib, encoding, and debugger should consume those semantics rather than redefine them.\n* Output and materialization changes usually involve pkg/encoding, runtime values, and VM result handling.\n* Debugging changes usually involve pkg/debugger, VM execution hooks/state, compiler or bytecode metadata, and runtime value inspection contracts.\n* Built-in module/function changes usually belong in pkg/stdlib, while reusable module contracts belong in pkg/module or pkg/sdk.\n* Embedding/API changes usually affect the top-level package and integration boundaries.\n* File system access, sandboxing, or path-policy behavior usually belongs in pkg/fs, not parser/compiler logic directly.\n\n## Canonical invariants\n\n* Ferret v2 uses a register-based VM.\n* runtime.Value is the common runtime/VM value abstraction.\n* Parser-generated code is derived output, not the source of truth.\n* Compiler changes must preserve program semantics expected by the VM.\n* Optimizations must preserve correctness before performance.\n* Runtime execution errors and internal invariant violations are different classes of failure and should not be collapsed conceptually.\n* Do not assume behavior from old design notes or the v1 codebase unless it is reflected in the current v2 code.\n* Do not change FQL language semantics as a side effect of refactoring.\n\n## Package map\n\nAgents should begin with the package whose responsibility owns the requested behavior. Do not infer ownership from file names alone when a package in this map already describes the intended boundary.\n\n### Core execution pipeline\n\n* pkg/source\n    * Owns source text, source identity, source ranges, and source-origin metadata.\n    * Prefer this package when behavior depends on where code came from.\n    * Parser, diagnostics, compiler, formatter, and tooling may depend on it, but source identity should not be reimplemented in those packages.\n* pkg/parser\n    * Owns FQL syntax parsing, parse-tree processing, parser diagnostics, and parser-generated code integration.\n    * Grammar changes should begin under pkg/parser/antlr.\n    * pkg/parser/antlr contains grammar sources.\n    * pkg/parser/fql contains generated parser and lexer code.\n    * Do not hand-edit generated parser artifacts; edit grammar sources and regenerate.\n* pkg/diagnostics\n    * Owns shared diagnostic primitives and formatting support for errors, warnings, spans, labels, notes, hints, and user-facing messages.\n    * Parser, compiler, runtime, formatter, and tooling should use shared diagnostic concepts rather than inventing local diagnostic formats.\n    * Changes here should preserve diagnostic category, span, label, note, and hint quality.\n* pkg/compiler\n    * Owns semantic analysis, lowering from parsed FQL into bytecode.Program, bytecode emission, and optimization/code generation passes.\n    * Compiler changes must preserve the runtime semantics expected by the VM.\n    * Do not move runtime-only behavior into the compiler unless the behavior is explicitly compile-time validation or compile-time semantics.\n* pkg/bytecode\n    * Owns the executable program model consumed by the VM and produced by the compiler.\n    * Includes instructions, operands, programs, and related executable metadata.\n    * Changes here are cross-cutting and usually require coordinated updates in compiler emission, VM execution, debugger metadata, and low-level tooling such as pkg/asm.\n* pkg/vm\n    * Owns bytecode execution, VM state, instruction dispatch, runtime coordination, cleanup, and VM-facing result handling.\n    * This package is performance-sensitive and semantics-sensitive.\n    * Verify compiler, bytecode, and runtime assumptions before changing VM internals.\n* pkg/runtime\n    * Owns core runtime semantics: values, value comparison/equality behavior, type behavior, function registries, runtime contracts, and execution-facing interfaces.\n    * VM, stdlib, encoding, and debugger should consume runtime semantics rather than redefine them locally.\n    * Prefer this package for shared value behavior that must remain consistent across execution, materialization, stdlib functions, and debugging.\n\n### Output, formatting, and low-level tooling\n\n* pkg/encoding\n    * Owns output encoding and materialization infrastructure for turning runtime values into external representations.\n    * Changes here must account for runtime value semantics, resource ownership, cleanup behavior, and result lifetimes.\n* pkg/formatter\n    * Owns Ferret source formatting and pretty-printing behavior.\n    * Formatting changes should preserve source semantics and should be validated with targeted tests or fixtures.\n* pkg/asm\n    * Owns assembly-layer support for Ferret bytecode programs.\n    * Includes parsing, encoding, and low-level textual representations used by bytecode tooling and debugging.\n    * Opcode or bytecode shape changes may require corresponding updates here.\n\n### Debugging and developer tooling\n\n* pkg/debugger\n    * Owns Ferret debugging support: debugger state, breakpoints, stepping coordination, execution inspection, and debugger-facing protocol adaptation.\n    * It should consume VM execution hooks, compiler/bytecode metadata, and runtime value inspection contracts.\n    * It should not own core runtime value semantics.\n* pkg/logging\n    * Owns internal logging support.\n    * Logging should remain observational.\n    * Do not make language semantics, diagnostics, execution behavior, or control flow depend on log output.\n\n### Integration and extension surfaces\n\n* pkg/module\n    * Owns Ferret module API contracts, host module interfaces, and module registration boundaries.\n    * Use this package for reusable module integration concepts.\n    * Do not place stdlib-specific behavior here.\n* pkg/sdk\n    * Owns extension-facing helpers and contracts for developers building on top of Ferret internals.\n    * Prefer pkg/sdk when the goal is to support external integrations, tools, or custom runtime/module implementations.\n    * Do not move internals into pkg/sdk only to make cross-package access easier.\n* pkg/stdlib\n    * Owns built-in Ferret namespaces, modules, and host functions registered as the standard library.\n    * Built-in functions should delegate shared semantics to runtime-owned helpers when behavior must be consistent outside stdlib.\n    * Do not duplicate runtime value semantics inside stdlib functions.\n\n### File systems, internals, and support packages\n\n* pkg/fs\n    * Owns file system abstractions and security-aware file access.\n    * Use this package for controlled file access, virtual file systems, path policy, and sandbox-like behavior.\n    * Do not confuse it with pkg/source; source identity and file system access are related but separate concerns.\n* pkg/internal\n    * Owns implementation-only packages that are not intended for use outside of the Ferret project.\n    * Prefer pkg/internal for shared implementation details that should not become public or extension contracts.\n    * Do not expose APIs from here to external users.\n\n### Top-level package and regression suites\n\n* The top-level package owns the embedding surface used by applications:\n    * Engine compiles and runs FQL sources.\n    * Plan wraps compiled bytecode plus environment state.\n    * Session executes a plan.\n    * Module and ModuleRegistry expose extension points for host modules.\n* Changes here are API-sensitive and should be treated as embedding-facing behavior changes.\n* test/integration/compiler, test/integration/optimization, and test/integration/vm are the main regression suites.\n\n## Where to start by task\n\n* Add or change syntax:\n    * edit grammar under pkg/parser/antlr\n    * regenerate parser artifacts\n    * inspect parser diagnostics/code in pkg/parser\n    * update compiler lowering in pkg/compiler\n    * add or update integration coverage\n* Add or change bytecode/opcodes:\n    * inspect pkg/bytecode\n    * inspect compiler emission sites\n    * inspect VM execution in pkg/vm\n    * validate with VM/integration tests\n* Change runtime value semantics:\n    * inspect pkg/runtime\n    * inspect relevant assumptions in pkg/vm\n    * inspect result/materialization behavior if affected\n* Change diagnostics behavior:\n    * inspect pkg/diagnostics\n    * inspect parser/compiler call sites that construct or transform diagnostics\n    * validate both message content and span/label accuracy\n* Change output/materialization behavior:\n    * inspect pkg/encoding\n    * inspect runtime and VM call sites that feed values into encoders/materializers\n    * validate public-facing behavior and resource/cleanup interactions if relevant\n* Change formatting behavior:\n    * inspect pkg/formatter\n    * validate formatting stability with targeted tests or fixtures\n* Change file-backed source handling or source loading behavior:\n    * inspect pkg/source\n    * inspect parser/compiler call sites that consume source objects\n    * validate path-aware diagnostics and any embedding/tooling behavior that depends on source identity\n* Change embedding API:\n    * inspect top-level package (Engine, Plan, Session)\n    * inspect downstream compiler/runtime/VM interactions\n    * validate public behavior with integration coverage as appropriate\n* Change built-in functions/modules:\n    * inspect pkg/stdlib\n    * inspect host function/module registration and runtime contracts\n* Change extension or integration support for external developers/tools:\n    * inspect pkg/sdk\n    * validate that the change belongs to an extension-facing surface rather than the core pipeline\n* Change developer tooling or low-level program tooling:\n    * inspect pkg/asm, pkg/debugger, pkg/formatter, pkg/diagnostics, or pkg/sdk depending on which surface owns the behavior\n\n## Stability guide\n\nTreat these as relatively stable unless the task explicitly targets them:\n\n* the overall pipeline shape: parser -> compiler -> bytecode -> VM\n* the parser generation workflow\n* the top-level embedding entry points\n\nTreat these as implementation-sensitive and verify current code before proposing changes:\n\n* optimizer internals\n* diagnostics plumbing\n* VM execution internals\n* runtime value behavior and cleanup/resource semantics\n* encoding/materialization behavior\n* debugger integration points\n\nDo not treat historical discussion, stale comments, or old branches as authoritative.\n\n## Public API and package boundary rules\n\n* Treat the top-level package, pkg/module, pkg/runtime, and pkg/sdk as API-sensitive.\n* Do not export new symbols from API-sensitive packages unless the task explicitly requires an external contract.\n* Prefer unexported helpers inside the owning package before adding exported APIs.\n* If a new exported symbol is necessary, add a doc comment that explains the external contract and stability expectation.\n* Do not move internals into pkg/sdk only to make tests or cross-package access easier.\n* Do not expose debugger-only APIs through the public embedding surface unless explicitly requested.\n\n## Language behavior change rules\n\n* Do not change FQL language semantics as a side effect of refactoring.\n* Any intentional language behavior change must be called out explicitly in the final summary.\n* Backward-incompatible behavior changes require tests showing both the old edge case and the new expected behavior.\n* When behavior differs from v1 or from old docs, prefer current v2 tests and this file.\n* Preserve existing behavior unless the task explicitly requires changing it.\n\n## Runtime value correctness rules\n\n* Hashes are acceleration hints, not proof of equality.\n* Any uniqueness, DISTINCT, set, grouping, or deduplication behavior must verify equality after hash comparison.\n* Reuse shared runtime equality/comparison semantics instead of inventing local equality rules.\n* Do not introduce hash-only correctness paths unless the behavior is explicitly probabilistic, which language semantics normally are not.\n* Preserve Ferret type ordering and comparison semantics consistently across compiler, VM, stdlib, and encoding behavior.\n\n## Resource and lifecycle rules\n\n* Values or results that own resources must document ownership and cleanup behavior.\n* Cleanup must be deterministic where the API exposes Close or equivalent lifecycle methods.\n* VM execution must preserve cleanup behavior on normal return, runtime error, and cancellation paths.\n* Do not materialize lazy or streaming values eagerly unless the task explicitly requires it.\n* Encoding/materialization changes must consider resource ownership and whether values can outlive the current execution frame.\n\n## Debugging architecture rules\n\n* Debugger support must not change normal execution semantics.\n* Debugger-disabled execution paths must remain allocation-conscious and should avoid debugger-specific work on hot VM paths.\n* Runtime values may expose debugger-facing metadata or display information through runtime-owned contracts, not debugger-owned runtime type checks.\n* Debugger contracts should live near the value/runtime boundary when they describe value behavior.\n* pkg/debugger should consume those contracts and translate them into debugger state, protocol state, or user-facing inspection data; it should not own core runtime semantics.\n* Prefer optional interfaces over modifying every runtime value type.\n* Debugger integration in the VM must be explicit, measurable, and easy to bypass when debugging is disabled.\n\n## Diagnostic quality rules\n\n* User-facing errors should include accurate source spans whenever source information is available.\n* Prefer actionable hints when an error is likely caused by a common misuse.\n* Do not replace specific diagnostics with generic errors.\n* Parser/compiler diagnostics should distinguish syntax errors, semantic errors, runtime type errors, and internal invariants.\n* Tests for diagnostics should verify message category and span accuracy for behavior changes.\n\n## Standard library rules\n\n* Built-in functions should keep Ferret-facing argument validation close to the function boundary.\n* Prefer small host functions that delegate core behavior to runtime-owned helpers when behavior is shared.\n* Do not duplicate runtime semantics inside stdlib functions.\n* Stdlib errors should be user-facing and should preserve argument context where practical.\n* Test stdlib behavior at the Ferret-language level whenever practical.\n\n## Go type and file structure rules\n\nThese rules are mandatory unless the task explicitly requires otherwise.\n\n* Do not define multiple method-bearing structs in the same .go file.\n* Prefer declaring a method-bearing struct as a standalone type Name struct { ... }.\n* A method-bearing struct should usually live in its own file, named after the primary type or responsibility whenever practical, for example:\n    * result.go for Result\n    * register_file.go for RegisterFile\n    * call_stack.go for CallStack\n* Grouped type ( ... ) declarations are allowed for interfaces, passive data-only structs, and other small related helper/value types that belong to the same narrow concern.\n* A grouped type ( ... ) block may also contain exactly one method-bearing struct when:\n    * it is the only behavioral type in the file, and\n    * the other grouped types are passive helper/value types from the same narrow concern.\n* Do not use grouped type ( ... ) declarations to hide multiple substantial behavioral types.\n* If a helper struct later gains methods and would create more than one method-bearing struct in the file, extract it into its own file immediately.\n* Methods for a struct should live in the same file as the struct unless there is a strong, explicit reason to split by concern.\n* Do not place a new method-bearing struct into an existing file just because the code compiles.\n\nAllowed:\n\ntype (\n\tPassResult struct {\n\t\tMetadata map[string]any\n\t\tModified bool\n\t}\n\tPassContext struct {\n\t\tProgram  *bytecode.Program\n\t\tCFG      *ControlFlowGraph\n\t\tMetadata map[string]any\n\t}\n\tPass interface {\n\t\tName() string\n\t\tRequires() []string\n\t\tRun(ctx *PassContext) (*PassResult, error)\n\t}\n)\n\nAvoid:\n\ntype (\n\tResult struct {\n\t\t// ...\n\t}\n\texecState struct {\n\t\t// ...\n\t}\n)\n\nRationale:\n\n* one method-bearing type per file keeps ownership of behavior obvious\n* standalone method-bearing types make diffs and reviews clearer\n* grouped type blocks are fine for passive, closely related types, but should not hide substantial behavioral types\n\n## Function and method ownership rules\n\nThese rules are mandatory unless the task explicitly requires otherwise.\n\n* A file centered on a method-bearing type should contain the type, its methods, and its constructors only.\n* Do not mix package-level helper functions into a file that already contains methods for a primary type.\n* In type-centered files, constructor functions are the only normally allowed package-level functions.\n* If logic conceptually belongs to the primary type, implement it as a method.\n* If logic does not belong to the type and must remain a package-level function, place it in a separate helper-focused file.\n* Package-level functions are preferred only when there is no natural owning type or when the behavior is genuinely package-level.\n* If a file contains both methods and non-constructor package-level functions, that is usually a structure violation and should be refactored.\n\n## Comment rules for functions and methods\n\n* Do not add comments to every function or method by default.\n* Exported functions and methods should usually have doc comments, especially in public, embedding-facing, or extension-facing packages.\n* Unexported functions and methods should be commented only when they carry non-obvious behavior, invariants, side effects, ownership rules, cleanup expectations, or protocol/lifecycle constraints.\n* Comments must explain intent, contract, invariants, side effects, or lifecycle behavior.\n* Prefer comments that explain why the code exists, what must remain true, or how the method is meant to be used.\n* Do not write comments that merely restate the method name or signature.\n* For VM, runtime, compiler, encoding, diagnostics, and debugger internals, prefer comments on semantics and invariants over implementation narration.\n* Avoid comment wallpaper. Dense, meaningful comments are preferred over mechanically documenting obvious code.\n\nPreferred:\n\n// Close releases resources associated with the result.\n// It is safe to call multiple times. Once closed, the result must not be reused.\nfunc (r *Result) Close() error\n\nPreferred for internal code:\n\n// promoteEscaped ensures a value that may outlive the current register write\n// is no longer tied to the current ownership path.\nfunc (s *execState) promoteEscaped(...)\n\nAvoid:\n\n// Close closes the result.\nfunc (r *Result) Close() error\n\n## Go control-flow spacing rules\n\nThese rules are mandatory for handwritten Go code.\n\nBlank lines should separate logical units and make control-flow and termination boundaries visually obvious.\n\n### Immediate producer + check\n\nA declaration, assignment, function call, type assertion, lookup, parse operation, or similar statement may remain directly adjacent to a following `if` when the `if` immediately checks or consumes the value produced by that statement.\n\nThis includes error checks, boolean/result checks, type assertions, nil checks, bounds checks, and other immediate validation.\n\nPreferred:\n\n```go\nres, err := doSome()\nif err != nil {\n\treturn err\n}\n```\n\nPreferred:\n\n```go\nnamed, ok := typeOf.(*types.Named)\nif !ok || named.Obj().Pkg() == nil || !w.localPackage(named.Obj().Pkg().Path()) {\n\treturn w.source.errorAt(\n\t\tErrorUnsupportedRegistration,\n\t\texpression.Pos(),\n\t\t\"New selects a module root dynamically\",\n\t)\n}\n```\n\nPreferred:\n\n```go\nvalue := lookup(name)\nif value == nil {\n\treturn ErrNotFound\n}\n```\n\nPreferred:\n\n```go\ncount := len(items)\nif count == 0 {\n\treturn nil\n}\n```\n\nThe producer and its immediate check form one logical unit and should not be separated by a blank line.\n\n### Separation from preceding logic\n\nIf an immediate producer + check unit follows another statement or logical unit, separate it from the preceding code with a blank line.\n\nPreferred:\n\n```go\nprepareState()\n\nnamed, ok := typeOf.(*types.Named)\nif !ok {\n\treturn ErrUnsupported\n}\n```\n\nAvoid:\n\n```go\nprepareState()\nnamed, ok := typeOf.(*types.Named)\nif !ok {\n\treturn ErrUnsupported\n}\n```\n\nNo leading blank line is required when the producer begins the enclosing block:\n\n```go\nfunc inspect(typeOf types.Type) error {\n\tnamed, ok := typeOf.(*types.Named)\n\tif !ok {\n\t\treturn ErrUnsupported\n\t}\n\n\treturn inspectNamed(named)\n}\n```\n\n### Consecutive control-flow blocks\n\nSeparate independent `if` statements with a blank line.\n\nAvoid:\n\n```go\nif foo != nil {\n\tuseFoo(foo)\n}\nif bar != nil {\n\tuseBar(bar)\n}\n```\n\nPrefer:\n\n```go\nif foo != nil {\n\tuseFoo(foo)\n}\n\nif bar != nil {\n\tuseBar(bar)\n}\n```\n\nThis applies even when both conditions are short. Independent control-flow decisions should remain visually distinct.\n\n### Statements after control flow\n\nAdd a blank line after a completed `if` block before continuing with a separate statement or logical unit.\n\nAvoid:\n\n```go\nif foo == bar {\n\tdoFoo()\n}\ndoSomething()\n```\n\nPrefer:\n\n```go\nif foo == bar {\n\tdoFoo()\n}\n\ndoSomething()\n```\n\n### Return and break separation\n\n`return` and `break` are termination or control-transfer statements and should be visually separated from preceding statements.\n\nA `return` or `break` must begin a new logical group: when another statement precedes it in the same block, place a blank line immediately before it.\n\nThis rule applies inside nested control-flow blocks as well as at the function-body level.\n\nAvoid:\n\n```go\nif is(offending.Prev().Prev(), \"NOT\") {\n\treturn \"NOT EXISTS\", offending.Prev()\n}\nreturn \"EXISTS\", offending.Prev()\n```\n\nPrefer:\n\n```go\nif is(offending.Prev().Prev(), \"NOT\") {\n\treturn \"NOT EXISTS\", offending.Prev()\n}\n\nreturn \"EXISTS\", offending.Prev()\n```\n\nAvoid:\n\n```go\nif is(curr, \"WAITFOR\") {\n\tfoundWaitFor = true\n\tbreak\n}\n```\n\nPrefer:\n\n```go\nif is(curr, \"WAITFOR\") {\n\tfoundWaitFor = true\n\n\tbreak\n}\n```\n\nThe same rule applies when ordinary computation precedes a return:\n\nAvoid:\n\n```go\nresult := buildResult()\nreturn result\n```\n\nPrefer:\n\n```go\nresult := buildResult()\n\nreturn result\n```\n\nLikewise for `break`:\n\nAvoid:\n\n```go\nfound = true\nbreak\n```\n\nPrefer:\n\n```go\nfound = true\n\nbreak\n```\n\nNo blank line is required before a `return` when it is already the first statement in its block:\n\n```go\nif err != nil {\n\treturn err\n}\n```\n\nNo artificial leading blank line should be introduced:\n\n```go\nfunc value() int {\n\treturn 42\n}\n```\n\nThe intent is not to surround every `return` or `break` with whitespace. The rule specifically requires separation from a preceding statement in the same block.\n\n\n## Local type declarations\n\nLocal types declared inside functions are allowed, but should be used deliberately.\n\nPrefer a local type when all of the following are true:\n\n- it is small;\n- it is passive and method-free;\n- it is used only within that function;\n- it exists purely to support the local algorithm;\n- keeping it local makes the function easier to understand rather than harder to scan.\n\nPrefer a package-level unexported type when one or more of the following are true:\n\n- the type represents a meaningful domain or algorithmic concept;\n- the type is used across a substantial portion of a long or complex function;\n- moving the type declaration out of the control flow improves readability;\n- the type may reasonably gain methods or behavior;\n- the type is likely to be reused by nearby helpers;\n- the type name helps explain the algorithm or responsibility at package scope.\n\nDo not promote a tiny throwaway struct to package scope merely for consistency.\n\nDo not keep a meaningful concept local merely to avoid adding a package-level type.\n\nExample of an appropriate local type:\n\n```go\nfunc collect(...) {\n\ttype entry struct {\n\t\tname  string\n\t\tindex int\n\t}\n\n\t// Small, passive, function-local algorithm state.\n}\n```\n\nExample where a package-level type is preferable:\n\n```go\ntype waitForEmptyGroupCandidate struct {\n\tmode            string\n\tsynchronization string\n\tspan            source.Span\n\tdistance        int\n}\n```\n\nwhen that value represents a meaningful candidate selected and ranked throughout a substantial parsing or diagnostic algorithm.\n\nThe decision should be based on readability, conceptual ownership, and expected evolution, not on a blanket preference for either local or package-level types.\n\n## Response and code style\n\nWhen assisting with this repository, avoid large unstructured blocks of prose or code.\n\nPrefer responses that are easy to scan:\n\n* Use short sections with clear headings.\n* Use bullet points for decisions, trade-offs, and follow-up work.\n* Use code blocks only for actual code, commands, or configuration.\n* Prefer focused snippets or diffs over full-file dumps.\n* Explain why a change is needed before showing how to implement it.\n* Keep comments in code useful and minimal.\n* Avoid repeating the same context in multiple places.\n* When the change touches multiple files, summarize the role of each file first.\n\nThe expected tone is practical, concise, and engineering-focused.\n\n## Development practice expectations\n\nAgents must follow repository-specific engineering discipline rather than generic style preferences.\n\n### Core principles\n\n* Preserve correctness first.\n* Preserve subsystem boundaries and invariants.\n* Prefer the smallest local change that fully solves the task.\n* Avoid introducing abstractions, indirection, or refactors unless they are necessary for correctness, maintainability, or an explicitly requested design change.\n* Do not optimize by intuition alone; use measurements for performance-sensitive work.\n* Keep behavioral ownership obvious in code structure, naming, and file layout.\n* Do not treat the first working implementation as final.\n* A task is complete only after implementation, validation, self-review, necessary corrections, and final validation.\n\n### Mandatory expectations\n\n* Identify the owning subsystem before making a non-trivial change.\n* Preserve existing behavior unless the task explicitly requires changing it.\n* Add or update tests for any behavior change.\n* Add or update benchmarks for any significant change.\n* Run the narrowest relevant validation first, then broaden as appropriate.\n* Perform the mandatory final self-review for every non-trivial task.\n* Inspect the complete final diff before declaring the task complete.\n* Re-run affected validation after any changes made during self-review.\n* Do not claim tests, benchmarks, review, or validation were completed unless they were actually performed.\n* Do not treat historical discussions, abandoned directions, or old branches as authoritative over current code and repository guidance.\n* Do not perform opportunistic refactors unrelated to the requested task unless they are required for correctness.\n\n### Required workflow for non-trivial changes\n\nBefore and while making a non-trivial change, agents must:\n\n1. Identify the owning subsystem.\n2. Identify the contract, invariant, or behavior being preserved or changed.\n3. Choose the smallest reasonable implementation that fits the existing design.\n4. Determine whether the change is significant.\n5. Add or update correctness tests.\n6. Add or update benchmarks if the change is significant.\n7. Run the relevant validation.\n8. Perform the mandatory final self-review described below.\n9. Address issues discovered during the self-review.\n10. Re-run affected validation after review-driven changes.\n11. Re-run relevant benchmarks if review-driven changes affect benchmarked code.\n12. Inspect the complete final diff as a whole.\n13. Summarize the implementation, review, and validation results accurately.\n\nDo not consider a task complete merely because the implementation compiles and its tests pass.\n\n## Mandatory final self-review\n\nAfter completing the implementation and initial validation for any non-trivial task, agents must review the complete resulting change before considering the task finished.\n\nThe review must evaluate the implementation itself, not merely confirm that tests pass.\n\nThe purpose of the review is to catch correctness, design, quality, organization, and maintainability problems introduced or exposed by the task. It must not be used as justification for unrelated refactoring or redesign.\n\nReview the final change for:\n\n### Correctness\n\n* Verify that the implementation satisfies the task requirements completely.\n* Look for missing cases, incorrect assumptions, regressions, boundary conditions, and failure paths.\n* Check error handling, cancellation, cleanup, state transitions, ownership, and lifecycle behavior where applicable.\n* Verify that concurrency behavior remains correct where relevant.\n* Verify that public or language-visible semantics match the intended contract.\n* Verify that tests exercise the intended behavior rather than merely mirroring the implementation.\n* For bug fixes, ensure a regression test would fail without the fix whenever practical.\n\n### Code clarity and cleanliness\n\n* Look for unnecessary complexity, duplication, excessive nesting, awkward control flow, misleading naming, and code that is difficult to reason about.\n* Prefer straightforward, idiomatic Go over clever implementations.\n* Remove temporary implementation artifacts, dead branches, obsolete helpers, debugging code, and comments describing abandoned approaches.\n* Avoid unnecessary abstraction layers and indirection.\n* Ensure the main execution path remains easy to follow.\n\n### Repository and Go best practices\n\n* Verify that the implementation follows the conventions and mandatory rules in this file.\n* Check relevant Go practices for error handling, API shape, resource ownership, concurrency, synchronization, context propagation, and lifecycle management.\n* Check whether errors are wrapped or propagated appropriately.\n* Check whether resources can leak on failure, cancellation, or early return.\n* Check whether ownership expectations are explicit where they need to be.\n* Do not recommend or introduce a pattern merely because it is fashionable or common elsewhere; it must improve this repository specifically.\n\n### Architecture\n\n* Verify that responsibilities remain in the correct package, type, and layer.\n* Check dependency direction and existing architectural boundaries.\n* Look for unwanted coupling, leaked implementation details, duplicated semantics, misplaced behavior, or abstractions at the wrong level.\n* Verify that runtime-owned semantics remain in pkg/runtime rather than being redefined by VM, stdlib, encoding, or debugger consumers.\n* Verify that compile-time behavior and runtime behavior remain separated appropriately.\n* Verify that public or extension-facing APIs are introduced only when the task genuinely requires them.\n* Consider whether the design will remain understandable and maintainable as the feature evolves.\n\n### Code organization and split\n\n* Verify that files, types, methods, functions, and packages have clear responsibilities.\n* Check compliance with the Go type/file structure rules in this file.\n* Check compliance with function and method ownership rules.\n* Look for files, functions, or types doing too much.\n* Look for unrelated responsibilities grouped together.\n* Also avoid unnecessary fragmentation where tightly related behavior has been split into excessive helpers, files, or abstractions.\n* Verify that helpers exist at the narrowest appropriate ownership level.\n* Ensure behavioral ownership is obvious from code layout.\n\n### Tests\n\n* Review test coverage for meaningful behavioral gaps.\n* Look especially for missing negative cases, edge conditions, cleanup paths, cancellation paths, invalid states, and boundary inputs.\n* Check for brittle tests coupled unnecessarily to implementation details.\n* Check for redundant tests that add maintenance cost without meaningful coverage.\n* Check for weak assertions that would allow plausible regressions to pass.\n* Verify diagnostic tests check message category and span accuracy where relevant.\n* Verify integration coverage exists when user-visible behavior crosses package boundaries.\n\n## Performance\n\nFor significant changes:\n\n* Inspect the final implementation for accidental allocations, repeated work, unnecessary materialization, unnecessary synchronization, or additional hot-path overhead.\n* Compare required benchmark results against the recorded baseline.\n* Verify that benchmark changes are attributable to the implementation rather than a different benchmark setup.\n* Do not trade clear correctness or maintainability for speculative micro-optimization.\n* If performance regresses meaningfully, investigate before considering the task complete.\n\n### Review findings and remediation\n\nWhen the self-review finds a problem:\n\n1. Fix correctness issues and regressions.\n2. Fix meaningful architectural, ownership, lifecycle, API, or maintainability problems.\n3. Simplify unnecessarily complicated code when doing so clearly improves the implementation.\n4. Correct file, type, or method ownership violations.\n5. Add or improve tests when the review exposes a behavioral coverage gap.\n6. Re-run validation affected by the change.\n7. Re-run relevant benchmarks if the correction affects benchmarked code.\n\nDo not leave a known correctness, architecture, ownership, lifecycle, or significant test-coverage problem unresolved merely because the initial task implementation already works.\n\nMinor stylistic preferences do not require changes.\n\nDistinguish actual problems from optional preferences. Existing code that is already clear, correct, idiomatic, and appropriately structured should be left alone.\n\nDo not use the self-review as justification for:\n\n* speculative refactoring\n* unrelated cleanup\n* unrelated API redesign\n* rewriting existing code merely for stylistic consistency\n* introducing abstractions without a concrete need\n* broad package reshuffling\n* changing FQL semantics beyond the requested task\n\n### Final diff inspection\n\nImmediately before finishing a non-trivial task, inspect the complete final diff as a whole rather than reviewing only individual edited files.\n\nVerify that:\n\n* every changed line is relevant to the requested task or a necessary supporting change;\n* no temporary or debugging code remains;\n* no accidental behavior changes were introduced;\n* no accidental API changes were introduced;\n* no unrelated refactors slipped into the change;\n* generated files changed only when their source inputs required regeneration;\n* tests describe intended behavior rather than implementation details;\n* comments describe current contracts, behavior, and invariants rather than abandoned implementation ideas;\n* file, type, function, and package boundaries remain coherent;\n* resource ownership and lifecycle behavior remain correct;\n* the resulting implementation is the smallest coherent change that fully solves the task.\n\nIf final diff inspection reveals an issue, correct it and repeat the affected validation before finishing.\n\n## Significant changes\n\nA change is significant when it could reasonably affect:\n\n* execution throughput\n* compile-time performance\n* latency on common paths\n* allocation patterns\n* memory reuse, pooling, or cleanup behavior\n* result/materialization cost\n* optimizer or code generation output relevant to performance\n\nThis includes, but is not limited to, changes in:\n\n* pkg/vm\n* pkg/runtime\n* pkg/compiler\n* pkg/bytecode\n* pkg/encoding\n* parser/compiler hot paths\n* caching, pooling, register allocation, ownership tracking, or materialization logic\n* debugger hooks on execution hot paths\n\nThis usually does not include:\n\n* comment-only, docs-only, or formatting-only edits\n* pure renames with no behavior change\n* test-only changes\n* narrowly scoped refactors that do not affect behavior or hot paths\n\nWhen in doubt, treat the change as significant and benchmark it.\n\n### Benchmark workflow for significant changes\n\nFor significant changes, agents must:\n\n* run relevant benchmarks before making the change and save the results as a baseline\n* implement the change\n* run the same benchmarks again after the change\n* compare before/after results, preferably including ns/op, B/op, and allocs/op\n* report the benchmark command used and summarize the performance delta\n\nIf no relevant benchmark exists for the changed hot path, add one.\n\nIf benchmark tooling or environment is unavailable, state that explicitly and do not claim benchmark validation was completed.\n\n## Test placement rules\n\n* Parser syntax behavior should have parser-focused tests or fixtures.\n* Compiler semantic behavior should have compiler tests and diagnostics/span assertions when relevant.\n* Bytecode emission changes should include compiler or integration tests that verify emitted behavior, not just VM behavior.\n* VM opcode behavior should have VM-level tests plus integration coverage when user-visible.\n* Stdlib behavior should be tested at the Ferret-language level whenever practical.\n* Public embedding behavior should have top-level API tests, not only package-internal tests.\n* Debugger behavior should test protocol/inspection output separately from VM execution semantics when possible.\n\n## Validation and evidence\n\nWhen finishing a non-trivial change, agents must report:\n\n* owning subsystem\n* files changed\n* tests added or updated\n* benchmarks added or updated\n* validation commands run\n* benchmark commands run, if applicable\n* self-review completed\n* notable issues found and corrected during self-review, if any\n* notable invariants preserved or intentionally changed\n* remaining concerns or limitations, if any\n\nFor significant changes:\n\n* tests alone are not sufficient\n* both correctness tests and benchmarks are required\n* benchmark results must be compared against a baseline when the environment allows it\n\nDo not claim:\n\n* tests passed unless they were actually run;\n* benchmarks were completed unless they were actually run;\n* self-review was completed unless the final implementation and diff were actually inspected;\n* validation succeeded if commands failed or were skipped.\n\nIf validation, benchmarking, or review work could not be completed because of tooling or environment limitations, state that explicitly.\n\n## Change discipline\n\n* Prefer adapting an existing local pattern over introducing a new architectural pattern.\n* Do not add new helper layers, wrappers, interfaces, or abstractions only for aesthetic reasons.\n* Do not move code across packages unless the ownership boundary is genuinely wrong.\n* Keep diffs focused on the requested task.\n* If a cleanup is necessary to make the requested change safe, keep it tightly scoped and explain why it was needed.\n* Self-review must not expand task scope unless a discovered problem directly affects correctness, safety, architecture, lifecycle, or maintainability of the requested change.\n\n## Comment and documentation discipline\n\n* Add comments where semantics, invariants, side effects, ownership, lifecycle, or recovery behavior are non-obvious.\n* Do not add comment wallpaper.\n* Prefer comments that explain why, contract, or invariants rather than implementation narration.\n* Public and extension-facing behavior should be documented more carefully than local obvious helpers.\n\n## Decision bias when uncertain\n\nWhen uncertain:\n\n* preserve existing behavior\n* prefer the smaller local change\n* add a focused test\n* treat the change as significant if performance might be affected\n* verify ownership before introducing a new abstraction or package-level dependency\n* prefer fixing an actual review finding over performing speculative cleanup\n* leave already-correct code alone\n\n## Tooling prerequisites\n\n* Go must be installed.\n* make is optional but is the preferred entrypoint for repo-defined workflows.\n* Java plus ANTLR 4.13.2 are required when regenerating parser artifacts.\n* staticcheck, goimports, and revive are needed for lint/format flows; install them with make install-tools.\n\n## Command matrix\n\n* Broad validation: go test ./...\n* Race-heavy package and integration coverage: make test\n* Lint: make lint\n* Format: make fmt\n* Regenerate parser/codegen artifacts: make generate\n* Build the CLI binary: make compile\n\nRun make generate only when grammar or generator inputs change.\n\n## Editing rules\n\n* Never hand-edit generated files under pkg/parser/fql or pkg/parser/antlr/gen.\n* Parser generation is driven by pkg/parser/parser.go:\n    * antlr -Xexact-output-dir -o fql -package fql -visitor -Dlanguage=Go antlr/FqlLexer.g4 antlr/FqlParser.g4\n    * go run ./tools/patch_lexer.go\n* If you change grammar files in pkg/parser/antlr, run make generate and commit the generated output in the same change.\n* Treat Makefile and .github/workflows/build.yml as the source of truth for validation commands.\n* Prefer narrow validation first, then broaden:\n    * Package-local changes: run the affected go test package or packages.\n    * Compiler, optimizer, or VM changes: run the relevant integration suites.\n    * Cross-cutting changes: finish with go test ./... or make test.\n\n## Validation expectations\n\n* After code changes, run the narrowest tests that prove the behavior you touched.\n* Before finishing broader changes, run the relevant repo-level command from the matrix above.\n* If you changed formatting-sensitive files, run make fmt.\n* If you changed lint-sensitive code paths or public behavior, run make lint when the toolchain is available.\n* If you changed parser grammar, generated lexer/parser output must be included and reviewed.\n* After review-driven code changes, re-run the validation relevant to those changes.\n* Do not consider initial validation sufficient if the implementation changed afterward.\n\n### Expectations for non-trivial changes\n\nWhen proposing or implementing non-trivial changes:\n\n* identify the owning subsystem first\n* preserve invariants unless the task explicitly changes them\n* prefer local, comprehensible changes before introducing new abstractions\n* distinguish correctness work from performance work\n* do not perform opportunistic refactors unrelated to the requested task unless they are necessary for correctness\n* complete the mandatory final self-review before finishing\n* inspect the final diff after all review-driven corrections\n* re-run affected validation after the last implementation change\n\n## Secondary references\n\n* README.md for product context and links to the broader Ferret ecosystem.\n* CONTRIBUTING.md for human contributor process.\n* .github/workflows/build.yml for the current CI validation path.\n\n## Website documentation synchronization\n\nFerret's public documentation is maintained in the website repository.\n\nChanges to public behavior must include corresponding website documentation updates when applicable. In particular, always evaluate documentation impact when changing:\n\n* FQL syntax, grammar, operators, expressions, statements, or language semantics;\n* embedding APIs or embedding behavior;\n* public SDK APIs, contracts, helpers, or extension points;\n* other public behavior already documented on the website.\n\nWhen such a change affects existing documentation:\n\n* locate the corresponding documentation in the website repository;\n* update it as part of the same task when the repository is available;\n* keep examples, syntax descriptions, API descriptions, and behavioral notes consistent with the implementation;\n* remove or revise documentation that describes behavior made obsolete by the change.\n\nFor new public syntax, embedding features, or SDK capabilities, add documentation to the appropriate existing section rather than leaving the implementation as the only specification.\n\nDocumentation synchronization is part of completing the change, not optional follow-up work.\n\nIf the website repository is not available in the working environment, explicitly report the required documentation update in the final summary rather than silently skipping it.\n"},"files":{"AGENTS.md":"# AGENTS.md\n\nThis file is the canonical operating guide for coding agents working in this repository. It is written for Ferret v2 only. If repository documentation conflicts with this file, prefer Makefile, go.mod, and .github/workflows/build.yml for commands, toolchain, and CI behavior.\n\n## Repo snapshot\n\n* Module path: github.com/MontFerret/ferret/v2\n* Go version: 1.23+\n* Toolchain in go.mod: go1.24.5\n* This repository root is Ferret v2. Do not mix assumptions from the separate v1 branch.\n* High-level flow: Engine -> compiler -> bytecode.Program -> vm.VM\n\n## Architectural mental model\n\nFerret v2 is a compiled query language and runtime.\n\nPrimary pipeline:\n\nsource -> parser -> diagnostics/AST -> compiler -> bytecode.Program -> vm.VM -> runtime values/results\n\nAgents should reason about changes by pipeline stage and ownership boundary:\n\n* Source identity, source ranges, and source-origin behavior usually begin in pkg/source and affect parser, diagnostics, compiler, formatter, or tooling call sites.\n* Syntax changes usually begin in grammar/parser and continue into compiler lowering.\n* Diagnostic changes usually involve pkg/diagnostics plus the parser/compiler/runtime call sites that create or wrap the diagnostic.\n* Semantic/runtime changes usually live in compiler, runtime, or VM.\n* Bytecode changes usually require coordinated updates in pkg/bytecode, compiler emission, VM execution, and low-level tooling such as pkg/asm or debugger metadata.\n* Runtime value behavior usually belongs in pkg/runtime; VM, stdlib, encoding, and debugger should consume those semantics rather than redefine them.\n* Output and materialization changes usually involve pkg/encoding, runtime values, and VM result handling.\n* Debugging changes usually involve pkg/debugger, VM execution hooks/state, compiler or bytecode metadata, and runtime value inspection contracts.\n* Built-in module/function changes usually belong in pkg/stdlib, while reusable module contracts belong in pkg/module or pkg/sdk.\n* Embedding/API changes usually affect the top-level package and integration boundaries.\n* File system access, sandboxing, or path-policy behavior usually belongs in pkg/fs, not parser/compiler logic directly.\n\n## Canonical invariants\n\n* Ferret v2 uses a register-based VM.\n* runtime.Value is the common runtime/VM value abstraction.\n* Parser-generated code is derived output, not the source of truth.\n* Compiler changes must preserve program semantics expected by the VM.\n* Optimizations must preserve correctness before performance.\n* Runtime execution errors and internal invariant violations are different classes of failure and should not be collapsed conceptually.\n* Do not assume behavior from old design notes or the v1 codebase unless it is reflected in the current v2 code.\n* Do not change FQL language semantics as a side effect of refactoring.\n\n## Package map\n\nAgents should begin with the package whose responsibility owns the requested behavior. Do not infer ownership from file names alone when a package in this map already describes the intended boundary.\n\n### Core execution pipeline\n\n* pkg/source\n    * Owns source text, source identity, source ranges, and source-origin metadata.\n    * Prefer this package when behavior depends on where code came from.\n    * Parser, diagnostics, compiler, formatter, and tooling may depend on it, but source identity should not be reimplemented in those packages.\n* pkg/parser\n    * Owns FQL syntax parsing, parse-tree processing, parser diagnostics, and parser-generated code integration.\n    * Grammar changes should begin under pkg/parser/antlr.\n    * pkg/parser/antlr contains grammar sources.\n    * pkg/parser/fql contains generated parser and lexer code.\n    * Do not hand-edit generated parser artifacts; edit grammar sources and regenerate.\n* pkg/diagnostics\n    * Owns shared diagnostic primitives and formatting support for errors, warnings, spans, labels, notes, hints, and user-facing messages.\n    * Parser, compiler, runtime, formatter, and tooling should use shared diagnostic concepts rather than inventing local diagnostic formats.\n    * Changes here should preserve diagnostic category, span, label, note, and hint quality.\n* pkg/compiler\n    * Owns semantic analysis, lowering from parsed FQL into bytecode.Program, bytecode emission, and optimization/code generation passes.\n    * Compiler changes must preserve the runtime semantics expected by the VM.\n    * Do not move runtime-only behavior into the compiler unless the behavior is explicitly compile-time validation or compile-time semantics.\n* pkg/bytecode\n    * Owns the executable program model consumed by the VM and produced by the compiler.\n    * Includes instructions, operands, programs, and related executable metadata.\n    * Changes here are cross-cutting and usually require coordinated updates in compiler emission, VM execution, debugger metadata, and low-level tooling such as pkg/asm.\n* pkg/vm\n    * Owns bytecode execution, VM state, instruction dispatch, runtime coordination, cleanup, and VM-facing result handling.\n    * This package is performance-sensitive and semantics-sensitive.\n    * Verify compiler, bytecode, and runtime assumptions before changing VM internals.\n* pkg/runtime\n    * Owns core runtime semantics: values, value comparison/equality behavior, type behavior, function registries, runtime contracts, and execution-facing interfaces.\n    * VM, stdlib, encoding, and debugger should consume runtime semantics rather than redefine them locally.\n    * Prefer this package for shared value behavior that must remain consistent across execution, materialization, stdlib functions, and debugging.\n\n### Output, formatting, and low-level tooling\n\n* pkg/encoding\n    * Owns output encoding and materialization infrastructure for turning runtime values into external representations.\n    * Changes here must account for runtime value semantics, resource ownership, cleanup behavior, and result lifetimes.\n* pkg/formatter\n    * Owns Ferret source formatting and pretty-printing behavior.\n    * Formatting changes should preserve source semantics and should be validated with targeted tests or fixtures.\n* pkg/asm\n    * Owns assembly-layer support for Ferret bytecode programs.\n    * Includes parsing, encoding, and low-level textual representations used by bytecode tooling and debugging.\n    * Opcode or bytecode shape changes may require corresponding updates here.\n\n### Debugging and developer tooling\n\n* pkg/debugger\n    * Owns Ferret debugging support: debugger state, breakpoints, stepping coordination, execution inspection, and debugger-facing protocol adaptation.\n    * It should consume VM execution hooks, compiler/bytecode metadata, and runtime value inspection contracts.\n    * It should not own core runtime value semantics.\n* pkg/logging\n    * Owns internal logging support.\n    * Logging should remain observational.\n    * Do not make language semantics, diagnostics, execution behavior, or control flow depend on log output.\n\n### Integration and extension surfaces\n\n* pkg/module\n    * Owns Ferret module API contracts, host module interfaces, and module registration boundaries.\n    * Use this package for reusable module integration concepts.\n    * Do not place stdlib-specific behavior here.\n* pkg/sdk\n    * Owns extension-facing helpers and contracts for developers building on top of Ferret internals.\n    * Prefer pkg/sdk when the goal is to support external integrations, tools, or custom runtime/module implementations.\n    * Do not move internals into pkg/sdk only to make cross-package access easier.\n* pkg/stdlib\n    * Owns built-in Ferret namespaces, modules, and host functions registered as the standard library.\n    * Built-in functions should delegate shared semantics to runtime-owned helpers when behavior must be consistent outside stdlib.\n    * Do not duplicate runtime value semantics inside stdlib functions.\n\n### File systems, internals, and support packages\n\n* pkg/fs\n    * Owns file system abstractions and security-aware file access.\n    * Use this package for controlled file access, virtual file systems, path policy, and sandbox-like behavior.\n    * Do not confuse it with pkg/source; source identity and file system access are related but separate concerns.\n* pkg/internal\n    * Owns implementation-only packages that are not intended for use outside of the Ferret project.\n    * Prefer pkg/internal for shared implementation details that should not become public or extension contracts.\n    * Do not expose APIs from here to external users.\n\n### Top-level package and regression suites\n\n* The top-level package owns the embedding surface used by applications:\n    * Engine compiles and runs FQL sources.\n    * Plan wraps compiled bytecode plus environment state.\n    * Session executes a plan.\n    * Module and ModuleRegistry expose extension points for host modules.\n* Changes here are API-sensitive and should be treated as embedding-facing behavior changes.\n* test/integration/compiler, test/integration/optimization, and test/integration/vm are the main regression suites.\n\n## Where to start by task\n\n* Add or change syntax:\n    * edit grammar under pkg/parser/antlr\n    * regenerate parser artifacts\n    * inspect parser diagnostics/code in pkg/parser\n    * update compiler lowering in pkg/compiler\n    * add or update integration coverage\n* Add or change bytecode/opcodes:\n    * inspect pkg/bytecode\n    * inspect compiler emission sites\n    * inspect VM execution in pkg/vm\n    * validate with VM/integration tests\n* Change runtime value semantics:\n    * inspect pkg/runtime\n    * inspect relevant assumptions in pkg/vm\n    * inspect result/materialization behavior if affected\n* Change diagnostics behavior:\n    * inspect pkg/diagnostics\n    * inspect parser/compiler call sites that construct or transform diagnostics\n    * validate both message content and span/label accuracy\n* Change output/materialization behavior:\n    * inspect pkg/encoding\n    * inspect runtime and VM call sites that feed values into encoders/materializers\n    * validate public-facing behavior and resource/cleanup interactions if relevant\n* Change formatting behavior:\n    * inspect pkg/formatter\n    * validate formatting stability with targeted tests or fixtures\n* Change file-backed source handling or source loading behavior:\n    * inspect pkg/source\n    * inspect parser/compiler call sites that consume source objects\n    * validate path-aware diagnostics and any embedding/tooling behavior that depends on source identity\n* Change embedding API:\n    * inspect top-level package (Engine, Plan, Session)\n    * inspect downstream compiler/runtime/VM interactions\n    * validate public behavior with integration coverage as appropriate\n* Change built-in functions/modules:\n    * inspect pkg/stdlib\n    * inspect host function/module registration and runtime contracts\n* Change extension or integration support for external developers/tools:\n    * inspect pkg/sdk\n    * validate that the change belongs to an extension-facing surface rather than the core pipeline\n* Change developer tooling or low-level program tooling:\n    * inspect pkg/asm, pkg/debugger, pkg/formatter, pkg/diagnostics, or pkg/sdk depending on which surface owns the behavior\n\n## Stability guide\n\nTreat these as relatively stable unless the task explicitly targets them:\n\n* the overall pipeline shape: parser -> compiler -> bytecode -> VM\n* the parser generation workflow\n* the top-level embedding entry points\n\nTreat these as implementation-sensitive and verify current code before proposing changes:\n\n* optimizer internals\n* diagnostics plumbing\n* VM execution internals\n* runtime value behavior and cleanup/resource semantics\n* encoding/materialization behavior\n* debugger integration points\n\nDo not treat historical discussion, stale comments, or old branches as authoritative.\n\n## Public API and package boundary rules\n\n* Treat the top-level package, pkg/module, pkg/runtime, and pkg/sdk as API-sensitive.\n* Do not export new symbols from API-sensitive packages unless the task explicitly requires an external contract.\n* Prefer unexported helpers inside the owning package before adding exported APIs.\n* If a new exported symbol is necessary, add a doc comment that explains the external contract and stability expectation.\n* Do not move internals into pkg/sdk only to make tests or cross-package access easier.\n* Do not expose debugger-only APIs through the public embedding surface unless explicitly requested.\n\n## Language behavior change rules\n\n* Do not change FQL language semantics as a side effect of refactoring.\n* Any intentional language behavior change must be called out explicitly in the final summary.\n* Backward-incompatible behavior changes require tests showing both the old edge case and the new expected behavior.\n* When behavior differs from v1 or from old docs, prefer current v2 tests and this file.\n* Preserve existing behavior unless the task explicitly requires changing it.\n\n## Runtime value correctness rules\n\n* Hashes are acceleration hints, not proof of equality.\n* Any uniqueness, DISTINCT, set, grouping, or deduplication behavior must verify equality after hash comparison.\n* Reuse shared runtime equality/comparison semantics instead of inventing local equality rules.\n* Do not introduce hash-only correctness paths unless the behavior is explicitly probabilistic, which language semantics normally are not.\n* Preserve Ferret type ordering and comparison semantics consistently across compiler, VM, stdlib, and encoding behavior.\n\n## Resource and lifecycle rules\n\n* Values or results that own resources must document ownership and cleanup behavior.\n* Cleanup must be deterministic where the API exposes Close or equivalent lifecycle methods.\n* VM execution must preserve cleanup behavior on normal return, runtime error, and cancellation paths.\n* Do not materialize lazy or streaming values eagerly unless the task explicitly requires it.\n* Encoding/materialization changes must consider resource ownership and whether values can outlive the current execution frame.\n\n## Debugging architecture rules\n\n* Debugger support must not change normal execution semantics.\n* Debugger-disabled execution paths must remain allocation-conscious and should avoid debugger-specific work on hot VM paths.\n* Runtime values may expose debugger-facing metadata or display information through runtime-owned contracts, not debugger-owned runtime type checks.\n* Debugger contracts should live near the value/runtime boundary when they describe value behavior.\n* pkg/debugger should consume those contracts and translate them into debugger state, protocol state, or user-facing inspection data; it should not own core runtime semantics.\n* Prefer optional interfaces over modifying every runtime value type.\n* Debugger integration in the VM must be explicit, measurable, and easy to bypass when debugging is disabled.\n\n## Diagnostic quality rules\n\n* User-facing errors should include accurate source spans whenever source information is available.\n* Prefer actionable hints when an error is likely caused by a common misuse.\n* Do not replace specific diagnostics with generic errors.\n* Parser/compiler diagnostics should distinguish syntax errors, semantic errors, runtime type errors, and internal invariants.\n* Tests for diagnostics should verify message category and span accuracy for behavior changes.\n\n## Standard library rules\n\n* Built-in functions should keep Ferret-facing argument validation close to the function boundary.\n* Prefer small host functions that delegate core behavior to runtime-owned helpers when behavior is shared.\n* Do not duplicate runtime semantics inside stdlib functions.\n* Stdlib errors should be user-facing and should preserve argument context where practical.\n* Test stdlib behavior at the Ferret-language level whenever practical.\n\n## Go type and file structure rules\n\nThese rules are mandatory unless the task explicitly requires otherwise.\n\n* Do not define multiple method-bearing structs in the same .go file.\n* Prefer declaring a method-bearing struct as a standalone type Name struct { ... }.\n* A method-bearing struct should usually live in its own file, named after the primary type or responsibility whenever practical, for example:\n    * result.go for Result\n    * register_file.go for RegisterFile\n    * call_stack.go for CallStack\n* Grouped type ( ... ) declarations are allowed for interfaces, passive data-only structs, and other small related helper/value types that belong to the same narrow concern.\n* A grouped type ( ... ) block may also contain exactly one method-bearing struct when:\n    * it is the only behavioral type in the file, and\n    * the other grouped types are passive helper/value types from the same narrow concern.\n* Do not use grouped type ( ... ) declarations to hide multiple substantial behavioral types.\n* If a helper struct later gains methods and would create more than one method-bearing struct in the file, extract it into its own file immediately.\n* Methods for a struct should live in the same file as the struct unless there is a strong, explicit reason to split by concern.\n* Do not place a new method-bearing struct into an existing file just because the code compiles.\n\nAllowed:\n\ntype (\n\tPassResult struct {\n\t\tMetadata map[string]any\n\t\tModified bool\n\t}\n\tPassContext struct {\n\t\tProgram  *bytecode.Program\n\t\tCFG      *ControlFlowGraph\n\t\tMetadata map[string]any\n\t}\n\tPass interface {\n\t\tName() string\n\t\tRequires() []string\n\t\tRun(ctx *PassContext) (*PassResult, error)\n\t}\n)\n\nAvoid:\n\ntype (\n\tResult struct {\n\t\t// ...\n\t}\n\texecState struct {\n\t\t// ...\n\t}\n)\n\nRationale:\n\n* one method-bearing type per file keeps ownership of behavior obvious\n* standalone method-bearing types make diffs and reviews clearer\n* grouped type blocks are fine for passive, closely related types, but should not hide substantial behavioral types\n\n## Function and method ownership rules\n\nThese rules are mandatory unless the task explicitly requires otherwise.\n\n* A file centered on a method-bearing type should contain the type, its methods, and its constructors only.\n* Do not mix package-level helper functions into a file that already contains methods for a primary type.\n* In type-centered files, constructor functions are the only normally allowed package-level functions.\n* If logic conceptually belongs to the primary type, implement it as a method.\n* If logic does not belong to the type and must remain a package-level function, place it in a separate helper-focused file.\n* Package-level functions are preferred only when there is no natural owning type or when the behavior is genuinely package-level.\n* If a file contains both methods and non-constructor package-level functions, that is usually a structure violation and should be refactored.\n\n## Comment rules for functions and methods\n\n* Do not add comments to every function or method by default.\n* Exported functions and methods should usually have doc comments, especially in public, embedding-facing, or extension-facing packages.\n* Unexported functions and methods should be commented only when they carry non-obvious behavior, invariants, side effects, ownership rules, cleanup expectations, or protocol/lifecycle constraints.\n* Comments must explain intent, contract, invariants, side effects, or lifecycle behavior.\n* Prefer comments that explain why the code exists, what must remain true, or how the method is meant to be used.\n* Do not write comments that merely restate the method name or signature.\n* For VM, runtime, compiler, encoding, diagnostics, and debugger internals, prefer comments on semantics and invariants over implementation narration.\n* Avoid comment wallpaper. Dense, meaningful comments are preferred over mechanically documenting obvious code.\n\nPreferred:\n\n// Close releases resources associated with the result.\n// It is safe to call multiple times. Once closed, the result must not be reused.\nfunc (r *Result) Close() error\n\nPreferred for internal code:\n\n// promoteEscaped ensures a value that may outlive the current register write\n// is no longer tied to the current ownership path.\nfunc (s *execState) promoteEscaped(...)\n\nAvoid:\n\n// Close closes the result.\nfunc (r *Result) Close() error\n\n## Go control-flow spacing rules\n\nThese rules are mandatory for handwritten Go code.\n\nBlank lines should separate logical units and make control-flow and termination boundaries visually obvious.\n\n### Immediate producer + check\n\nA declaration, assignment, function call, type assertion, lookup, parse operation, or similar statement may remain directly adjacent to a following `if` when the `if` immediately checks or consumes the value produced by that statement.\n\nThis includes error checks, boolean/result checks, type assertions, nil checks, bounds checks, and other immediate validation.\n\nPreferred:\n\n```go\nres, err := doSome()\nif err != nil {\n\treturn err\n}\n```\n\nPreferred:\n\n```go\nnamed, ok := typeOf.(*types.Named)\nif !ok || named.Obj().Pkg() == nil || !w.localPackage(named.Obj().Pkg().Path()) {\n\treturn w.source.errorAt(\n\t\tErrorUnsupportedRegistration,\n\t\texpression.Pos(),\n\t\t\"New selects a module root dynamically\",\n\t)\n}\n```\n\nPreferred:\n\n```go\nvalue := lookup(name)\nif value == nil {\n\treturn ErrNotFound\n}\n```\n\nPreferred:\n\n```go\ncount := len(items)\nif count == 0 {\n\treturn nil\n}\n```\n\nThe producer and its immediate check form one logical unit and should not be separated by a blank line.\n\n### Separation from preceding logic\n\nIf an immediate producer + check unit follows another statement or logical unit, separate it from the preceding code with a blank line.\n\nPreferred:\n\n```go\nprepareState()\n\nnamed, ok := typeOf.(*types.Named)\nif !ok {\n\treturn ErrUnsupported\n}\n```\n\nAvoid:\n\n```go\nprepareState()\nnamed, ok := typeOf.(*types.Named)\nif !ok {\n\treturn ErrUnsupported\n}\n```\n\nNo leading blank line is required when the producer begins the enclosing block:\n\n```go\nfunc inspect(typeOf types.Type) error {\n\tnamed, ok := typeOf.(*types.Named)\n\tif !ok {\n\t\treturn ErrUnsupported\n\t}\n\n\treturn inspectNamed(named)\n}\n```\n\n### Consecutive control-flow blocks\n\nSeparate independent `if` statements with a blank line.\n\nAvoid:\n\n```go\nif foo != nil {\n\tuseFoo(foo)\n}\nif bar != nil {\n\tuseBar(bar)\n}\n```\n\nPrefer:\n\n```go\nif foo != nil {\n\tuseFoo(foo)\n}\n\nif bar != nil {\n\tuseBar(bar)\n}\n```\n\nThis applies even when both conditions are short. Independent control-flow decisions should remain visually distinct.\n\n### Statements after control flow\n\nAdd a blank line after a completed `if` block before continuing with a separate statement or logical unit.\n\nAvoid:\n\n```go\nif foo == bar {\n\tdoFoo()\n}\ndoSomething()\n```\n\nPrefer:\n\n```go\nif foo == bar {\n\tdoFoo()\n}\n\ndoSomething()\n```\n\n### Return and break separation\n\n`return` and `break` are termination or control-transfer statements and should be visually separated from preceding statements.\n\nA `return` or `break` must begin a new logical group: when another statement precedes it in the same block, place a blank line immediately before it.\n\nThis rule applies inside nested control-flow blocks as well as at the function-body level.\n\nAvoid:\n\n```go\nif is(offending.Prev().Prev(), \"NOT\") {\n\treturn \"NOT EXISTS\", offending.Prev()\n}\nreturn \"EXISTS\", offending.Prev()\n```\n\nPrefer:\n\n```go\nif is(offending.Prev().Prev(), \"NOT\") {\n\treturn \"NOT EXISTS\", offending.Prev()\n}\n\nreturn \"EXISTS\", offending.Prev()\n```\n\nAvoid:\n\n```go\nif is(curr, \"WAITFOR\") {\n\tfoundWaitFor = true\n\tbreak\n}\n```\n\nPrefer:\n\n```go\nif is(curr, \"WAITFOR\") {\n\tfoundWaitFor = true\n\n\tbreak\n}\n```\n\nThe same rule applies when ordinary computation precedes a return:\n\nAvoid:\n\n```go\nresult := buildResult()\nreturn result\n```\n\nPrefer:\n\n```go\nresult := buildResult()\n\nreturn result\n```\n\nLikewise for `break`:\n\nAvoid:\n\n```go\nfound = true\nbreak\n```\n\nPrefer:\n\n```go\nfound = true\n\nbreak\n```\n\nNo blank line is required before a `return` when it is already the first statement in its block:\n\n```go\nif err != nil {\n\treturn err\n}\n```\n\nNo artificial leading blank line should be introduced:\n\n```go\nfunc value() int {\n\treturn 42\n}\n```\n\nThe intent is not to surround every `return` or `break` with whitespace. The rule specifically requires separation from a preceding statement in the same block.\n\n\n## Local type declarations\n\nLocal types declared inside functions are allowed, but should be used deliberately.\n\nPrefer a local type when all of the following are true:\n\n- it is small;\n- it is passive and method-free;\n- it is used only within that function;\n- it exists purely to support the local algorithm;\n- keeping it local makes the function easier to understand rather than harder to scan.\n\nPrefer a package-level unexported type when one or more of the following are true:\n\n- the type represents a meaningful domain or algorithmic concept;\n- the type is used across a substantial portion of a long or complex function;\n- moving the type declaration out of the control flow improves readability;\n- the type may reasonably gain methods or behavior;\n- the type is likely to be reused by nearby helpers;\n- the type name helps explain the algorithm or responsibility at package scope.\n\nDo not promote a tiny throwaway struct to package scope merely for consistency.\n\nDo not keep a meaningful concept local merely to avoid adding a package-level type.\n\nExample of an appropriate local type:\n\n```go\nfunc collect(...) {\n\ttype entry struct {\n\t\tname  string\n\t\tindex int\n\t}\n\n\t// Small, passive, function-local algorithm state.\n}\n```\n\nExample where a package-level type is preferable:\n\n```go\ntype waitForEmptyGroupCandidate struct {\n\tmode            string\n\tsynchronization string\n\tspan            source.Span\n\tdistance        int\n}\n```\n\nwhen that value represents a meaningful candidate selected and ranked throughout a substantial parsing or diagnostic algorithm.\n\nThe decision should be based on readability, conceptual ownership, and expected evolution, not on a blanket preference for either local or package-level types.\n\n## Response and code style\n\nWhen assisting with this repository, avoid large unstructured blocks of prose or code.\n\nPrefer responses that are easy to scan:\n\n* Use short sections with clear headings.\n* Use bullet points for decisions, trade-offs, and follow-up work.\n* Use code blocks only for actual code, commands, or configuration.\n* Prefer focused snippets or diffs over full-file dumps.\n* Explain why a change is needed before showing how to implement it.\n* Keep comments in code useful and minimal.\n* Avoid repeating the same context in multiple places.\n* When the change touches multiple files, summarize the role of each file first.\n\nThe expected tone is practical, concise, and engineering-focused.\n\n## Development practice expectations\n\nAgents must follow repository-specific engineering discipline rather than generic style preferences.\n\n### Core principles\n\n* Preserve correctness first.\n* Preserve subsystem boundaries and invariants.\n* Prefer the smallest local change that fully solves the task.\n* Avoid introducing abstractions, indirection, or refactors unless they are necessary for correctness, maintainability, or an explicitly requested design change.\n* Do not optimize by intuition alone; use measurements for performance-sensitive work.\n* Keep behavioral ownership obvious in code structure, naming, and file layout.\n* Do not treat the first working implementation as final.\n* A task is complete only after implementation, validation, self-review, necessary corrections, and final validation.\n\n### Mandatory expectations\n\n* Identify the owning subsystem before making a non-trivial change.\n* Preserve existing behavior unless the task explicitly requires changing it.\n* Add or update tests for any behavior change.\n* Add or update benchmarks for any significant change.\n* Run the narrowest relevant validation first, then broaden as appropriate.\n* Perform the mandatory final self-review for every non-trivial task.\n* Inspect the complete final diff before declaring the task complete.\n* Re-run affected validation after any changes made during self-review.\n* Do not claim tests, benchmarks, review, or validation were completed unless they were actually performed.\n* Do not treat historical discussions, abandoned directions, or old branches as authoritative over current code and repository guidance.\n* Do not perform opportunistic refactors unrelated to the requested task unless they are required for correctness.\n\n### Required workflow for non-trivial changes\n\nBefore and while making a non-trivial change, agents must:\n\n1. Identify the owning subsystem.\n2. Identify the contract, invariant, or behavior being preserved or changed.\n3. Choose the smallest reasonable implementation that fits the existing design.\n4. Determine whether the change is significant.\n5. Add or update correctness tests.\n6. Add or update benchmarks if the change is significant.\n7. Run the relevant validation.\n8. Perform the mandatory final self-review described below.\n9. Address issues discovered during the self-review.\n10. Re-run affected validation after review-driven changes.\n11. Re-run relevant benchmarks if review-driven changes affect benchmarked code.\n12. Inspect the complete final diff as a whole.\n13. Summarize the implementation, review, and validation results accurately.\n\nDo not consider a task complete merely because the implementation compiles and its tests pass.\n\n## Mandatory final self-review\n\nAfter completing the implementation and initial validation for any non-trivial task, agents must review the complete resulting change before considering the task finished.\n\nThe review must evaluate the implementation itself, not merely confirm that tests pass.\n\nThe purpose of the review is to catch correctness, design, quality, organization, and maintainability problems introduced or exposed by the task. It must not be used as justification for unrelated refactoring or redesign.\n\nReview the final change for:\n\n### Correctness\n\n* Verify that the implementation satisfies the task requirements completely.\n* Look for missing cases, incorrect assumptions, regressions, boundary conditions, and failure paths.\n* Check error handling, cancellation, cleanup, state transitions, ownership, and lifecycle behavior where applicable.\n* Verify that concurrency behavior remains correct where relevant.\n* Verify that public or language-visible semantics match the intended contract.\n* Verify that tests exercise the intended behavior rather than merely mirroring the implementation.\n* For bug fixes, ensure a regression test would fail without the fix whenever practical.\n\n### Code clarity and cleanliness\n\n* Look for unnecessary complexity, duplication, excessive nesting, awkward control flow, misleading naming, and code that is difficult to reason about.\n* Prefer straightforward, idiomatic Go over clever implementations.\n* Remove temporary implementation artifacts, dead branches, obsolete helpers, debugging code, and comments describing abandoned approaches.\n* Avoid unnecessary abstraction layers and indirection.\n* Ensure the main execution path remains easy to follow.\n\n### Repository and Go best practices\n\n* Verify that the implementation follows the conventions and mandatory rules in this file.\n* Check relevant Go practices for error handling, API shape, resource ownership, concurrency, synchronization, context propagation, and lifecycle management.\n* Check whether errors are wrapped or propagated appropriately.\n* Check whether resources can leak on failure, cancellation, or early return.\n* Check whether ownership expectations are explicit where they need to be.\n* Do not recommend or introduce a pattern merely because it is fashionable or common elsewhere; it must improve this repository specifically.\n\n### Architecture\n\n* Verify that responsibilities remain in the correct package, type, and layer.\n* Check dependency direction and existing architectural boundaries.\n* Look for unwanted coupling, leaked implementation details, duplicated semantics, misplaced behavior, or abstractions at the wrong level.\n* Verify that runtime-owned semantics remain in pkg/runtime rather than being redefined by VM, stdlib, encoding, or debugger consumers.\n* Verify that compile-time behavior and runtime behavior remain separated appropriately.\n* Verify that public or extension-facing APIs are introduced only when the task genuinely requires them.\n* Consider whether the design will remain understandable and maintainable as the feature evolves.\n\n### Code organization and split\n\n* Verify that files, types, methods, functions, and packages have clear responsibilities.\n* Check compliance with the Go type/file structure rules in this file.\n* Check compliance with function and method ownership rules.\n* Look for files, functions, or types doing too much.\n* Look for unrelated responsibilities grouped together.\n* Also avoid unnecessary fragmentation where tightly related behavior has been split into excessive helpers, files, or abstractions.\n* Verify that helpers exist at the narrowest appropriate ownership level.\n* Ensure behavioral ownership is obvious from code layout.\n\n### Tests\n\n* Review test coverage for meaningful behavioral gaps.\n* Look especially for missing negative cases, edge conditions, cleanup paths, cancellation paths, invalid states, and boundary inputs.\n* Check for brittle tests coupled unnecessarily to implementation details.\n* Check for redundant tests that add maintenance cost without meaningful coverage.\n* Check for weak assertions that would allow plausible regressions to pass.\n* Verify diagnostic tests check message category and span accuracy where relevant.\n* Verify integration coverage exists when user-visible behavior crosses package boundaries.\n\n## Performance\n\nFor significant changes:\n\n* Inspect the final implementation for accidental allocations, repeated work, unnecessary materialization, unnecessary synchronization, or additional hot-path overhead.\n* Compare required benchmark results against the recorded baseline.\n* Verify that benchmark changes are attributable to the implementation rather than a different benchmark setup.\n* Do not trade clear correctness or maintainability for speculative micro-optimization.\n* If performance regresses meaningfully, investigate before considering the task complete.\n\n### Review findings and remediation\n\nWhen the self-review finds a problem:\n\n1. Fix correctness issues and regressions.\n2. Fix meaningful architectural, ownership, lifecycle, API, or maintainability problems.\n3. Simplify unnecessarily complicated code when doing so clearly improves the implementation.\n4. Correct file, type, or method ownership violations.\n5. Add or improve tests when the review exposes a behavioral coverage gap.\n6. Re-run validation affected by the change.\n7. Re-run relevant benchmarks if the correction affects benchmarked code.\n\nDo not leave a known correctness, architecture, ownership, lifecycle, or significant test-coverage problem unresolved merely because the initial task implementation already works.\n\nMinor stylistic preferences do not require changes.\n\nDistinguish actual problems from optional preferences. Existing code that is already clear, correct, idiomatic, and appropriately structured should be left alone.\n\nDo not use the self-review as justification for:\n\n* speculative refactoring\n* unrelated cleanup\n* unrelated API redesign\n* rewriting existing code merely for stylistic consistency\n* introducing abstractions without a concrete need\n* broad package reshuffling\n* changing FQL semantics beyond the requested task\n\n### Final diff inspection\n\nImmediately before finishing a non-trivial task, inspect the complete final diff as a whole rather than reviewing only individual edited files.\n\nVerify that:\n\n* every changed line is relevant to the requested task or a necessary supporting change;\n* no temporary or debugging code remains;\n* no accidental behavior changes were introduced;\n* no accidental API changes were introduced;\n* no unrelated refactors slipped into the change;\n* generated files changed only when their source inputs required regeneration;\n* tests describe intended behavior rather than implementation details;\n* comments describe current contracts, behavior, and invariants rather than abandoned implementation ideas;\n* file, type, function, and package boundaries remain coherent;\n* resource ownership and lifecycle behavior remain correct;\n* the resulting implementation is the smallest coherent change that fully solves the task.\n\nIf final diff inspection reveals an issue, correct it and repeat the affected validation before finishing.\n\n## Significant changes\n\nA change is significant when it could reasonably affect:\n\n* execution throughput\n* compile-time performance\n* latency on common paths\n* allocation patterns\n* memory reuse, pooling, or cleanup behavior\n* result/materialization cost\n* optimizer or code generation output relevant to performance\n\nThis includes, but is not limited to, changes in:\n\n* pkg/vm\n* pkg/runtime\n* pkg/compiler\n* pkg/bytecode\n* pkg/encoding\n* parser/compiler hot paths\n* caching, pooling, register allocation, ownership tracking, or materialization logic\n* debugger hooks on execution hot paths\n\nThis usually does not include:\n\n* comment-only, docs-only, or formatting-only edits\n* pure renames with no behavior change\n* test-only changes\n* narrowly scoped refactors that do not affect behavior or hot paths\n\nWhen in doubt, treat the change as significant and benchmark it.\n\n### Benchmark workflow for significant changes\n\nFor significant changes, agents must:\n\n* run relevant benchmarks before making the change and save the results as a baseline\n* implement the change\n* run the same benchmarks again after the change\n* compare before/after results, preferably including ns/op, B/op, and allocs/op\n* report the benchmark command used and summarize the performance delta\n\nIf no relevant benchmark exists for the changed hot path, add one.\n\nIf benchmark tooling or environment is unavailable, state that explicitly and do not claim benchmark validation was completed.\n\n## Test placement rules\n\n* Parser syntax behavior should have parser-focused tests or fixtures.\n* Compiler semantic behavior should have compiler tests and diagnostics/span assertions when relevant.\n* Bytecode emission changes should include compiler or integration tests that verify emitted behavior, not just VM behavior.\n* VM opcode behavior should have VM-level tests plus integration coverage when user-visible.\n* Stdlib behavior should be tested at the Ferret-language level whenever practical.\n* Public embedding behavior should have top-level API tests, not only package-internal tests.\n* Debugger behavior should test protocol/inspection output separately from VM execution semantics when possible.\n\n## Validation and evidence\n\nWhen finishing a non-trivial change, agents must report:\n\n* owning subsystem\n* files changed\n* tests added or updated\n* benchmarks added or updated\n* validation commands run\n* benchmark commands run, if applicable\n* self-review completed\n* notable issues found and corrected during self-review, if any\n* notable invariants preserved or intentionally changed\n* remaining concerns or limitations, if any\n\nFor significant changes:\n\n* tests alone are not sufficient\n* both correctness tests and benchmarks are required\n* benchmark results must be compared against a baseline when the environment allows it\n\nDo not claim:\n\n* tests passed unless they were actually run;\n* benchmarks were completed unless they were actually run;\n* self-review was completed unless the final implementation and diff were actually inspected;\n* validation succeeded if commands failed or were skipped.\n\nIf validation, benchmarking, or review work could not be completed because of tooling or environment limitations, state that explicitly.\n\n## Change discipline\n\n* Prefer adapting an existing local pattern over introducing a new architectural pattern.\n* Do not add new helper layers, wrappers, interfaces, or abstractions only for aesthetic reasons.\n* Do not move code across packages unless the ownership boundary is genuinely wrong.\n* Keep diffs focused on the requested task.\n* If a cleanup is necessary to make the requested change safe, keep it tightly scoped and explain why it was needed.\n* Self-review must not expand task scope unless a discovered problem directly affects correctness, safety, architecture, lifecycle, or maintainability of the requested change.\n\n## Comment and documentation discipline\n\n* Add comments where semantics, invariants, side effects, ownership, lifecycle, or recovery behavior are non-obvious.\n* Do not add comment wallpaper.\n* Prefer comments that explain why, contract, or invariants rather than implementation narration.\n* Public and extension-facing behavior should be documented more carefully than local obvious helpers.\n\n## Decision bias when uncertain\n\nWhen uncertain:\n\n* preserve existing behavior\n* prefer the smaller local change\n* add a focused test\n* treat the change as significant if performance might be affected\n* verify ownership before introducing a new abstraction or package-level dependency\n* prefer fixing an actual review finding over performing speculative cleanup\n* leave already-correct code alone\n\n## Tooling prerequisites\n\n* Go must be installed.\n* make is optional but is the preferred entrypoint for repo-defined workflows.\n* Java plus ANTLR 4.13.2 are required when regenerating parser artifacts.\n* staticcheck, goimports, and revive are needed for lint/format flows; install them with make install-tools.\n\n## Command matrix\n\n* Broad validation: go test ./...\n* Race-heavy package and integration coverage: make test\n* Lint: make lint\n* Format: make fmt\n* Regenerate parser/codegen artifacts: make generate\n* Build the CLI binary: make compile\n\nRun make generate only when grammar or generator inputs change.\n\n## Editing rules\n\n* Never hand-edit generated files under pkg/parser/fql or pkg/parser/antlr/gen.\n* Parser generation is driven by pkg/parser/parser.go:\n    * antlr -Xexact-output-dir -o fql -package fql -visitor -Dlanguage=Go antlr/FqlLexer.g4 antlr/FqlParser.g4\n    * go run ./tools/patch_lexer.go\n* If you change grammar files in pkg/parser/antlr, run make generate and commit the generated output in the same change.\n* Treat Makefile and .github/workflows/build.yml as the source of truth for validation commands.\n* Prefer narrow validation first, then broaden:\n    * Package-local changes: run the affected go test package or packages.\n    * Compiler, optimizer, or VM changes: run the relevant integration suites.\n    * Cross-cutting changes: finish with go test ./... or make test.\n\n## Validation expectations\n\n* After code changes, run the narrowest tests that prove the behavior you touched.\n* Before finishing broader changes, run the relevant repo-level command from the matrix above.\n* If you changed formatting-sensitive files, run make fmt.\n* If you changed lint-sensitive code paths or public behavior, run make lint when the toolchain is available.\n* If you changed parser grammar, generated lexer/parser output must be included and reviewed.\n* After review-driven code changes, re-run the validation relevant to those changes.\n* Do not consider initial validation sufficient if the implementation changed afterward.\n\n### Expectations for non-trivial changes\n\nWhen proposing or implementing non-trivial changes:\n\n* identify the owning subsystem first\n* preserve invariants unless the task explicitly changes them\n* prefer local, comprehensible changes before introducing new abstractions\n* distinguish correctness work from performance work\n* do not perform opportunistic refactors unrelated to the requested task unless they are necessary for correctness\n* complete the mandatory final self-review before finishing\n* inspect the final diff after all review-driven corrections\n* re-run affected validation after the last implementation change\n\n## Secondary references\n\n* README.md for product context and links to the broader Ferret ecosystem.\n* CONTRIBUTING.md for human contributor process.\n* .github/workflows/build.yml for the current CI validation path.\n\n## Website documentation synchronization\n\nFerret's public documentation is maintained in the website repository.\n\nChanges to public behavior must include corresponding website documentation updates when applicable. In particular, always evaluate documentation impact when changing:\n\n* FQL syntax, grammar, operators, expressions, statements, or language semantics;\n* embedding APIs or embedding behavior;\n* public SDK APIs, contracts, helpers, or extension points;\n* other public behavior already documented on the website.\n\nWhen such a change affects existing documentation:\n\n* locate the corresponding documentation in the website repository;\n* update it as part of the same task when the repository is available;\n* keep examples, syntax descriptions, API descriptions, and behavioral notes consistent with the implementation;\n* remove or revise documentation that describes behavior made obsolete by the change.\n\nFor new public syntax, embedding features, or SDK capabilities, add documentation to the appropriate existing section rather than leaving the implementation as the only specification.\n\nDocumentation synchronization is part of completing the change, not optional follow-up work.\n\nIf the website repository is not available in the working environment, explicitly report the required documentation update in the final summary rather than silently skipping it.\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"# AGENTS.md\n\nThis file is the canonical operating guide for coding agents working in this repository. It is written for Ferret v2 only. If repository documentation conflicts with this file, prefer Makefile, go.mod, and .github/workflows/build.yml for commands, toolchain, and CI behavior.\n\n## Repo snapshot\n\n* Module path: github.com/MontFerret/ferret/v2\n* Go version: 1.23+\n* Toolchain in go.mod: go1.24.5\n* This repository root is Ferret v2. Do not mix assumptions from the separate v1 branch.\n* High-level flow: Engine -> compiler -> bytecode.Program -> vm.VM\n\n## Architectural mental model\n\nFerret v2 is a compiled query language and runtime.\n\nPrimary pipeline:\n\nsource -> parser -> diagnostics/AST -> compiler -> bytecode.Program -> vm.VM -> runtime values/results\n\nAgents should reason about changes by pipeline stage and ownership boundary:\n\n* Source identity, source ranges, and source-origin behavior usually begin in pkg/source and affect parser, diagnostics, compiler, formatter, or tooling call sites.\n* Syntax changes usually begin in grammar/parser and continue into compiler lowering.\n* Diagnostic changes usually involve pkg/diagnostics plus the parser/compiler/runtime call sites that create or wrap the diagnostic.\n* Semantic/runtime changes usually live in compiler, runtime, or VM.\n* Bytecode changes usually require coordinated updates in pkg/bytecode, compiler emission, VM execution, and low-level tooling such as pkg/asm or debugger metadata.\n* Runtime value behavior usually belongs in pkg/runtime; VM, stdlib, encoding, and debugger should consume those semantics rather than redefine them.\n* Output and materialization changes usually involve pkg/encoding, runtime values, and VM result handling.\n* Debugging changes usually involve pkg/debugger, VM execution hooks/state, compiler or bytecode metadata, and runtime value inspection contracts.\n* Built-in module/function changes usually belong in pkg/stdlib, while reusable module contracts belong in pkg/module or pkg/sdk.\n* Embedding/API changes usually affect the top-level package and integration boundaries.\n* File system access, sandboxing, or path-policy behavior usually belongs in pkg/fs, not parser/compiler logic directly.\n\n## Canonical invariants\n\n* Ferret v2 uses a register-based VM.\n* runtime.Value is the common runtime/VM value abstraction.\n* Parser-generated code is derived output, not the source of truth.\n* Compiler changes must preserve program semantics expected by the VM.\n* Optimizations must preserve correctness before performance.\n* Runtime execution errors and internal invariant violations are different classes of failure and should not be collapsed conceptually.\n* Do not assume behavior from old design notes or the v1 codebase unless it is reflected in the current v2 code.\n* Do not change FQL language semantics as a side effect of refactoring.\n\n## Package map\n\nAgents should begin with the package whose responsibility owns the requested behavior. Do not infer ownership from file names alone when a package in this map already describes the intended boundary.\n\n### Core execution pipeline\n\n* pkg/source\n    * Owns source text, source identity, source ranges, and source-origin metadata.\n    * Prefer this package when behavior depends on where code came from.\n    * Parser, diagnostics, compiler, formatter, and tooling may depend on it, but source identity should not be reimplemented in those packages.\n* pkg/parser\n    * Owns FQL syntax parsing, parse-tree processing, parser diagnostics, and parser-generated code integration.\n    * Grammar changes should begin under pkg/parser/antlr.\n    * pkg/parser/antlr contains grammar sources.\n    * pkg/parser/fql contains generated parser and lexer code.\n    * Do not hand-edit generated parser artifacts; edit grammar sources and regenerate.\n* pkg/diagnostics\n    * Owns shared diagnostic primitives and formatting support for errors, warnings, spans, labels, notes, hints, and user-facing messages.\n    * Parser, compiler, runtime, formatter, and tooling should use shared diagnostic concepts rather than inventing local diagnostic formats.\n    * Changes here should preserve diagnostic category, span, label, note, and hint quality.\n* pkg/compiler\n    * Owns semantic analysis, lowering from parsed FQL into bytecode.Program, bytecode emission, and optimization/code generation passes.\n    * Compiler changes must preserve the runtime semantics expected by the VM.\n    * Do not move runtime-only behavior into the compiler unless the behavior is explicitly compile-time validation or compile-time semantics.\n* pkg/bytecode\n    * Owns the executable program model consumed by the VM and produced by the compiler.\n    * Includes instructions, operands, programs, and related executable metadata.\n    * Changes here are cross-cutting and usually require coordinated updates in compiler emission, VM execution, debugger metadata, and low-level tooling such as pkg/asm.\n* pkg/vm\n    * Owns bytecode execution, VM state, instruction dispatch, runtime coordination, cleanup, and VM-facing result handling.\n    * This package is performance-sensitive and semantics-sensitive.\n    * Verify compiler, bytecode, and runtime assumptions before changing VM internals.\n* pkg/runtime\n    * Owns core runtime semantics: values, value comparison/equality behavior, type behavior, function registries, runtime contracts, and execution-facing interfaces.\n    * VM, stdlib, encoding, and debugger should consume runtime semantics rather than redefine them locally.\n    * Prefer this package for shared value behavior that must remain consistent across execution, materialization, stdlib functions, and debugging.\n\n### Output, formatting, and low-level tooling\n\n* pkg/encoding\n    * Owns output encoding and materialization infrastructure for turning runtime values into external representations.\n    * Changes here must account for runtime value semantics, resource ownership, cleanup behavior, and result lifetimes.\n* pkg/formatter\n    * Owns Ferret source formatting and pretty-printing behavior.\n    * Formatting changes should preserve source semantics and should be validated with targeted tests or fixtures.\n* pkg/asm\n    * Owns assembly-layer support for Ferret bytecode programs.\n    * Includes parsing, encoding, and low-level textual representations used by bytecode tooling and debugging.\n    * Opcode or bytecode shape changes may require corresponding updates here.\n\n### Debugging and developer tooling\n\n* pkg/debugger\n    * Owns Ferret debugging support: debugger state, breakpoints, stepping coordination, execution inspection, and debugger-facing protocol adaptation.\n    * It should consume VM execution hooks, compiler/bytecode metadata, and runtime value inspection contracts.\n    * It should not own core runtime value semantics.\n* pkg/logging\n    * Owns internal logging support.\n    * Logging should remain observational.\n    * Do not make language semantics, diagnostics, execution behavior, or control flow depend on log output.\n\n### Integration and extension surfaces\n\n* pkg/module\n    * Owns Ferret module API contracts, host module interfaces, and module registration boundaries.\n    * Use this package for reusable module integration concepts.\n    * Do not place stdlib-specific behavior here.\n* pkg/sdk\n    * Owns extension-facing helpers and contracts for developers building on top of Ferret internals.\n    * Prefer pkg/sdk when the goal is to support external integrations, tools, or custom runtime/module implementations.\n    * Do not move internals into pkg/sdk only to make cross-package access easier.\n* pkg/stdlib\n    * Owns built-in Ferret namespaces, modules, and host functions registered as the standard library.\n    * Built-in functions should delegate shared semantics to runtime-owned helpers when behavior must be consistent outside stdlib.\n    * Do not duplicate runtime value semantics inside stdlib functions.\n\n### File systems, internals, and support packages\n\n* pkg/fs\n    * Owns file system abstractions and security-aware file access.\n    * Use this package for controlled file access, virtual file systems, path policy, and sandbox-like behavior.\n    * Do not confuse it with pkg/source; source identity and file system access are related but separate concerns.\n* pkg/internal\n    * Owns implementation-only packages that are not intended for use outside of the Ferret project.\n    * Prefer pkg/internal for shared implementation details that should not become public or extension contracts.\n    * Do not expose APIs from here to external users.\n\n### Top-level package and regression suites\n\n* The top-level package owns the embedding surface used by applications:\n    * Engine compiles and runs FQL sources.\n    * Plan wraps compiled bytecode plus environment state.\n    * Session executes a plan.\n    * Module and ModuleRegistry expose extension points for host modules.\n* Changes here are API-sensitive and should be treated as embedding-facing behavior changes.\n* test/integration/compiler, test/integration/optimization, and test/integration/vm are the main regression suites.\n\n## Where to start by task\n\n* Add or change syntax:\n    * edit grammar under pkg/parser/antlr\n    * regenerate parser artifacts\n    * inspect parser diagnostics/code in pkg/parser\n    * update compiler lowering in pkg/compiler\n    * add or update integration coverage\n* Add or change bytecode/opcodes:\n    * inspect pkg/bytecode\n    * inspect compiler emission sites\n    * inspect VM execution in pkg/vm\n    * validate with VM/integration tests\n* Change runtime value semantics:\n    * inspect pkg/runtime\n    * inspect relevant assumptions in pkg/vm\n    * inspect result/materialization behavior if affected\n* Change diagnostics behavior:\n    * inspect pkg/diagnostics\n    * inspect parser/compiler call sites that construct or transform diagnostics\n    * validate both message content and span/label accuracy\n* Change output/materialization behavior:\n    * inspect pkg/encoding\n    * inspect runtime and VM call sites that feed values into encoders/materializers\n    * validate public-facing behavior and resource/cleanup interactions if relevant\n* Change formatting behavior:\n    * inspect pkg/formatter\n    * validate formatting stability with targeted tests or fixtures\n* Change file-backed source handling or source loading behavior:\n    * inspect pkg/source\n    * inspect parser/compiler call sites that consume source objects\n    * validate path-aware diagnostics and any embedding/tooling behavior that depends on source identity\n* Change embedding API:\n    * inspect top-level package (Engine, Plan, Session)\n    * inspect downstream compiler/runtime/VM interactions\n    * validate public behavior with integration coverage as appropriate\n* Change built-in functions/modules:\n    * inspect pkg/stdlib\n    * inspect host function/module registration and runtime contracts\n* Change extension or integration support for external developers/tools:\n    * inspect pkg/sdk\n    * validate that the change belongs to an extension-facing surface rather than the core pipeline\n* Change developer tooling or low-level program tooling:\n    * inspect pkg/asm, pkg/debugger, pkg/formatter, pkg/diagnostics, or pkg/sdk depending on which surface owns the behavior\n\n## Stability guide\n\nTreat these as relatively stable unless the task explicitly targets them:\n\n* the overall pipeline shape: parser -> compiler -> bytecode -> VM\n* the parser generation workflow\n* the top-level embedding entry points\n\nTreat these as implementation-sensitive and verify current code before proposing changes:\n\n* optimizer internals\n* diagnostics plumbing\n* VM execution internals\n* runtime value behavior and cleanup/resource semantics\n* encoding/materialization behavior\n* debugger integration points\n\nDo not treat historical discussion, stale comments, or old branches as authoritative.\n\n## Public API and package boundary rules\n\n* Treat the top-level package, pkg/module, pkg/runtime, and pkg/sdk as API-sensitive.\n* Do not export new symbols from API-sensitive packages unless the task explicitly requires an external contract.\n* Prefer unexported helpers inside the owning package before adding exported APIs.\n* If a new exported symbol is necessary, add a doc comment that explains the external contract and stability expectation.\n* Do not move internals into pkg/sdk only to make tests or cross-package access easier.\n* Do not expose debugger-only APIs through the public embedding surface unless explicitly requested.\n\n## Language behavior change rules\n\n* Do not change FQL language semantics as a side effect of refactoring.\n* Any intentional language behavior change must be called out explicitly in the final summary.\n* Backward-incompatible behavior changes require tests showing both the old edge case and the new expected behavior.\n* When behavior differs from v1 or from old docs, prefer current v2 tests and this file.\n* Preserve existing behavior unless the task explicitly requires changing it.\n\n## Runtime value correctness rules\n\n* Hashes are acceleration hints, not proof of equality.\n* Any uniqueness, DISTINCT, set, grouping, or deduplication behavior must verify equality after hash comparison.\n* Reuse shared runtime equality/comparison semantics instead of inventing local equality rules.\n* Do not introduce hash-only correctness paths unless the behavior is explicitly probabilistic, which language semantics normally are not.\n* Preserve Ferret type ordering and comparison semantics consistently across compiler, VM, stdlib, and encoding behavior.\n\n## Resource and lifecycle rules\n\n* Values or results that own resources must document ownership and cleanup behavior.\n* Cleanup must be deterministic where the API exposes Close or equivalent lifecycle methods.\n* VM execution must preserve cleanup behavior on normal return, runtime error, and cancellation paths.\n* Do not materialize lazy or streaming values eagerly unless the task explicitly requires it.\n* Encoding/materialization changes must consider resource ownership and whether values can outlive the current execution frame.\n\n## Debugging architecture rules\n\n* Debugger support must not change normal execution semantics.\n* Debugger-disabled execution paths must remain allocation-conscious and should avoid debugger-specific work on hot VM paths.\n* Runtime values may expose debugger-facing metadata or display information through runtime-owned contracts, not debugger-owned runtime type checks.\n* Debugger contracts should live near the value/runtime boundary when they describe value behavior.\n* pkg/debugger should consume those contracts and translate them into debugger state, protocol state, or user-facing inspection data; it should not own core runtime semantics.\n* Prefer optional interfaces over modifying every runtime value type.\n* Debugger integration in the VM must be explicit, measurable, and easy to bypass when debugging is disabled.\n\n## Diagnostic quality rules\n\n* User-facing errors should include accurate source spans whenever source information is available.\n* Prefer actionable hints when an error is likely caused by a common misuse.\n* Do not replace specific diagnostics with generic errors.\n* Parser/compiler diagnostics should distinguish syntax errors, semantic errors, runtime type errors, and internal invariants.\n* Tests for diagnostics should verify message category and span accuracy for behavior changes.\n\n## Standard library rules\n\n* Built-in functions should keep Ferret-facing argument validation close to the function boundary.\n* Prefer small host functions that delegate core behavior to runtime-owned helpers when behavior is shared.\n* Do not duplicate runtime semantics inside stdlib functions.\n* Stdlib errors should be user-facing and should preserve argument context where practical.\n* Test stdlib behavior at the Ferret-language level whenever practical.\n\n## Go type and file structure rules\n\nThese rules are mandatory unless the task explicitly requires otherwise.\n\n* Do not define multiple method-bearing structs in the same .go file.\n* Prefer declaring a method-bearing struct as a standalone type Name struct { ... }.\n* A method-bearing struct should usually live in its own file, named after the primary type or responsibility whenever practical, for example:\n    * result.go for Result\n    * register_file.go for RegisterFile\n    * call_stack.go for CallStack\n* Grouped type ( ... ) declarations are allowed for interfaces, passive data-only structs, and other small related helper/value types that belong to the same narrow concern.\n* A grouped type ( ... ) block may also contain exactly one method-bearing struct when:\n    * it is the only behavioral type in the file, and\n    * the other grouped types are passive helper/value types from the same narrow concern.\n* Do not use grouped type ( ... ) declarations to hide multiple substantial behavioral types.\n* If a helper struct later gains methods and would create more than one method-bearing struct in the file, extract it into its own file immediately.\n* Methods for a struct should live in the same file as the struct unless there is a strong, explicit reason to split by concern.\n* Do not place a new method-bearing struct into an existing file just because the code compiles.\n\nAllowed:\n\ntype (\n\tPassResult struct {\n\t\tMetadata map[string]any\n\t\tModified bool\n\t}\n\tPassContext struct {\n\t\tProgram  *bytecode.Program\n\t\tCFG      *ControlFlowGraph\n\t\tMetadata map[string]any\n\t}\n\tPass interface {\n\t\tName() string\n\t\tRequires() []string\n\t\tRun(ctx *PassContext) (*PassResult, error)\n\t}\n)\n\nAvoid:\n\ntype (\n\tResult struct {\n\t\t// ...\n\t}\n\texecState struct {\n\t\t// ...\n\t}\n)\n\nRationale:\n\n* one method-bearing type per file keeps ownership of behavior obvious\n* standalone method-bearing types make diffs and reviews clearer\n* grouped type blocks are fine for passive, closely related types, but should not hide substantial behavioral types\n\n## Function and method ownership rules\n\nThese rules are mandatory unless the task explicitly requires otherwise.\n\n* A file centered on a method-bearing type should contain the type, its methods, and its constructors only.\n* Do not mix package-level helper functions into a file that already contains methods for a primary type.\n* In type-centered files, constructor functions are the only normally allowed package-level functions.\n* If logic conceptually belongs to the primary type, implement it as a method.\n* If logic does not belong to the type and must remain a package-level function, place it in a separate helper-focused file.\n* Package-level functions are preferred only when there is no natural owning type or when the behavior is genuinely package-level.\n* If a file contains both methods and non-constructor package-level functions, that is usually a structure violation and should be refactored.\n\n## Comment rules for functions and methods\n\n* Do not add comments to every function or method by default.\n* Exported functions and methods should usually have doc comments, especially in public, embedding-facing, or extension-facing packages.\n* Unexported functions and methods should be commented only when they carry non-obvious behavior, invariants, side effects, ownership rules, cleanup expectations, or protocol/lifecycle constraints.\n* Comments must explain intent, contract, invariants, side effects, or lifecycle behavior.\n* Prefer comments that explain why the code exists, what must remain true, or how the method is meant to be used.\n* Do not write comments that merely restate the method name or signature.\n* For VM, runtime, compiler, encoding, diagnostics, and debugger internals, prefer comments on semantics and invariants over implementation narration.\n* Avoid comment wallpaper. Dense, meaningful comments are preferred over mechanically documenting obvious code.\n\nPreferred:\n\n// Close releases resources associated with the result.\n// It is safe to call multiple times. Once closed, the result must not be reused.\nfunc (r *Result) Close() error\n\nPreferred for internal code:\n\n// promoteEscaped ensures a value that may outlive the current register write\n// is no longer tied to the current ownership path.\nfunc (s *execState) promoteEscaped(...)\n\nAvoid:\n\n// Close closes the result.\nfunc (r *Result) Close() error\n\n## Go control-flow spacing rules\n\nThese rules are mandatory for handwritten Go code.\n\nBlank lines should separate logical units and make control-flow and termination boundaries visually obvious.\n\n### Immediate producer + check\n\nA declaration, assignment, function call, type assertion, lookup, parse operation, or similar statement may remain directly adjacent to a following `if` when the `if` immediately checks or consumes the value produced by that statement.\n\nThis includes error checks, boolean/result checks, type assertions, nil checks, bounds checks, and other immediate validation.\n\nPreferred:\n\n```go\nres, err := doSome()\nif err != nil {\n\treturn err\n}\n```\n\nPreferred:\n\n```go\nnamed, ok := typeOf.(*types.Named)\nif !ok || named.Obj().Pkg() == nil || !w.localPackage(named.Obj().Pkg().Path()) {\n\treturn w.source.errorAt(\n\t\tErrorUnsupportedRegistration,\n\t\texpression.Pos(),\n\t\t\"New selects a module root dynamically\",\n\t)\n}\n```\n\nPreferred:\n\n```go\nvalue := lookup(name)\nif value == nil {\n\treturn ErrNotFound\n}\n```\n\nPreferred:\n\n```go\ncount := len(items)\nif count == 0 {\n\treturn nil\n}\n```\n\nThe producer and its immediate check form one logical unit and should not be separated by a blank line.\n\n### Separation from preceding logic\n\nIf an immediate producer + check unit follows another statement or logical unit, separate it from the preceding code with a blank line.\n\nPreferred:\n\n```go\nprepareState()\n\nnamed, ok := typeOf.(*types.Named)\nif !ok {\n\treturn ErrUnsupported\n}\n```\n\nAvoid:\n\n```go\nprepareState()\nnamed, ok := typeOf.(*types.Named)\nif !ok {\n\treturn ErrUnsupported\n}\n```\n\nNo leading blank line is required when the producer begins the enclosing block:\n\n```go\nfunc inspect(typeOf types.Type) error {\n\tnamed, ok := typeOf.(*types.Named)\n\tif !ok {\n\t\treturn ErrUnsupported\n\t}\n\n\treturn inspectNamed(named)\n}\n```\n\n### Consecutive control-flow blocks\n\nSeparate independent `if` statements with a blank line.\n\nAvoid:\n\n```go\nif foo != nil {\n\tuseFoo(foo)\n}\nif bar != nil {\n\tuseBar(bar)\n}\n```\n\nPrefer:\n\n```go\nif foo != nil {\n\tuseFoo(foo)\n}\n\nif bar != nil {\n\tuseBar(bar)\n}\n```\n\nThis applies even when both conditions are short. Independent control-flow decisions should remain visually distinct.\n\n### Statements after control flow\n\nAdd a blank line after a completed `if` block before continuing with a separate statement or logical unit.\n\nAvoid:\n\n```go\nif foo == bar {\n\tdoFoo()\n}\ndoSomething()\n```\n\nPrefer:\n\n```go\nif foo == bar {\n\tdoFoo()\n}\n\ndoSomething()\n```\n\n### Return and break separation\n\n`return` and `break` are termination or control-transfer statements and should be visually separated from preceding statements.\n\nA `return` or `break` must begin a new logical group: when another statement precedes it in the same block, place a blank line immediately before it.\n\nThis rule applies inside nested control-flow blocks as well as at the function-body level.\n\nAvoid:\n\n```go\nif is(offending.Prev().Prev(), \"NOT\") {\n\treturn \"NOT EXISTS\", offending.Prev()\n}\nreturn \"EXISTS\", offending.Prev()\n```\n\nPrefer:\n\n```go\nif is(offending.Prev().Prev(), \"NOT\") {\n\treturn \"NOT EXISTS\", offending.Prev()\n}\n\nreturn \"EXISTS\", offending.Prev()\n```\n\nAvoid:\n\n```go\nif is(curr, \"WAITFOR\") {\n\tfoundWaitFor = true\n\tbreak\n}\n```\n\nPrefer:\n\n```go\nif is(curr, \"WAITFOR\") {\n\tfoundWaitFor = true\n\n\tbreak\n}\n```\n\nThe same rule applies when ordinary computation precedes a return:\n\nAvoid:\n\n```go\nresult := buildResult()\nreturn result\n```\n\nPrefer:\n\n```go\nresult := buildResult()\n\nreturn result\n```\n\nLikewise for `break`:\n\nAvoid:\n\n```go\nfound = true\nbreak\n```\n\nPrefer:\n\n```go\nfound = true\n\nbreak\n```\n\nNo blank line is required before a `return` when it is already the first statement in its block:\n\n```go\nif err != nil {\n\treturn err\n}\n```\n\nNo artificial leading blank line should be introduced:\n\n```go\nfunc value() int {\n\treturn 42\n}\n```\n\nThe intent is not to surround every `return` or `break` with whitespace. The rule specifically requires separation from a preceding statement in the same block.\n\n\n## Local type declarations\n\nLocal types declared inside functions are allowed, but should be used deliberately.\n\nPrefer a local type when all of the following are true:\n\n- it is small;\n- it is passive and method-free;\n- it is used only within that function;\n- it exists purely to support the local algorithm;\n- keeping it local makes the function easier to understand rather than harder to scan.\n\nPrefer a package-level unexported type when one or more of the following are true:\n\n- the type represents a meaningful domain or algorithmic concept;\n- the type is used across a substantial portion of a long or complex function;\n- moving the type declaration out of the control flow improves readability;\n- the type may reasonably gain methods or behavior;\n- the type is likely to be reused by nearby helpers;\n- the type name helps explain the algorithm or responsibility at package scope.\n\nDo not promote a tiny throwaway struct to package scope merely for consistency.\n\nDo not keep a meaningful concept local merely to avoid adding a package-level type.\n\nExample of an appropriate local type:\n\n```go\nfunc collect(...) {\n\ttype entry struct {\n\t\tname  string\n\t\tindex int\n\t}\n\n\t// Small, passive, function-local algorithm state.\n}\n```\n\nExample where a package-level type is preferable:\n\n```go\ntype waitForEmptyGroupCandidate struct {\n\tmode            string\n\tsynchronization string\n\tspan            source.Span\n\tdistance        int\n}\n```\n\nwhen that value represents a meaningful candidate selected and ranked throughout a substantial parsing or diagnostic algorithm.\n\nThe decision should be based on readability, conceptual ownership, and expected evolution, not on a blanket preference for either local or package-level types.\n\n## Response and code style\n\nWhen assisting with this repository, avoid large unstructured blocks of prose or code.\n\nPrefer responses that are easy to scan:\n\n* Use short sections with clear headings.\n* Use bullet points for decisions, trade-offs, and follow-up work.\n* Use code blocks only for actual code, commands, or configuration.\n* Prefer focused snippets or diffs over full-file dumps.\n* Explain why a change is needed before showing how to implement it.\n* Keep comments in code useful and minimal.\n* Avoid repeating the same context in multiple places.\n* When the change touches multiple files, summarize the role of each file first.\n\nThe expected tone is practical, concise, and engineering-focused.\n\n## Development practice expectations\n\nAgents must follow repository-specific engineering discipline rather than generic style preferences.\n\n### Core principles\n\n* Preserve correctness first.\n* Preserve subsystem boundaries and invariants.\n* Prefer the smallest local change that fully solves the task.\n* Avoid introducing abstractions, indirection, or refactors unless they are necessary for correctness, maintainability, or an explicitly requested design change.\n* Do not optimize by intuition alone; use measurements for performance-sensitive work.\n* Keep behavioral ownership obvious in code structure, naming, and file layout.\n* Do not treat the first working implementation as final.\n* A task is complete only after implementation, validation, self-review, necessary corrections, and final validation.\n\n### Mandatory expectations\n\n* Identify the owning subsystem before making a non-trivial change.\n* Preserve existing behavior unless the task explicitly requires changing it.\n* Add or update tests for any behavior change.\n* Add or update benchmarks for any significant change.\n* Run the narrowest relevant validation first, then broaden as appropriate.\n* Perform the mandatory final self-review for every non-trivial task.\n* Inspect the complete final diff before declaring the task complete.\n* Re-run affected validation after any changes made during self-review.\n* Do not claim tests, benchmarks, review, or validation were completed unless they were actually performed.\n* Do not treat historical discussions, abandoned directions, or old branches as authoritative over current code and repository guidance.\n* Do not perform opportunistic refactors unrelated to the requested task unless they are required for correctness.\n\n### Required workflow for non-trivial changes\n\nBefore and while making a non-trivial change, agents must:\n\n1. Identify the owning subsystem.\n2. Identify the contract, invariant, or behavior being preserved or changed.\n3. Choose the smallest reasonable implementation that fits the existing design.\n4. Determine whether the change is significant.\n5. Add or update correctness tests.\n6. Add or update benchmarks if the change is significant.\n7. Run the relevant validation.\n8. Perform the mandatory final self-review described below.\n9. Address issues discovered during the self-review.\n10. Re-run affected validation after review-driven changes.\n11. Re-run relevant benchmarks if review-driven changes affect benchmarked code.\n12. Inspect the complete final diff as a whole.\n13. Summarize the implementation, review, and validation results accurately.\n\nDo not consider a task complete merely because the implementation compiles and its tests pass.\n\n## Mandatory final self-review\n\nAfter completing the implementation and initial validation for any non-trivial task, agents must review the complete resulting change before considering the task finished.\n\nThe review must evaluate the implementation itself, not merely confirm that tests pass.\n\nThe purpose of the review is to catch correctness, design, quality, organization, and maintainability problems introduced or exposed by the task. It must not be used as justification for unrelated refactoring or redesign.\n\nReview the final change for:\n\n### Correctness\n\n* Verify that the implementation satisfies the task requirements completely.\n* Look for missing cases, incorrect assumptions, regressions, boundary conditions, and failure paths.\n* Check error handling, cancellation, cleanup, state transitions, ownership, and lifecycle behavior where applicable.\n* Verify that concurrency behavior remains correct where relevant.\n* Verify that public or language-visible semantics match the intended contract.\n* Verify that tests exercise the intended behavior rather than merely mirroring the implementation.\n* For bug fixes, ensure a regression test would fail without the fix whenever practical.\n\n### Code clarity and cleanliness\n\n* Look for unnecessary complexity, duplication, excessive nesting, awkward control flow, misleading naming, and code that is difficult to reason about.\n* Prefer straightforward, idiomatic Go over clever implementations.\n* Remove temporary implementation artifacts, dead branches, obsolete helpers, debugging code, and comments describing abandoned approaches.\n* Avoid unnecessary abstraction layers and indirection.\n* Ensure the main execution path remains easy to follow.\n\n### Repository and Go best practices\n\n* Verify that the implementation follows the conventions and mandatory rules in this file.\n* Check relevant Go practices for error handling, API shape, resource ownership, concurrency, synchronization, context propagation, and lifecycle management.\n* Check whether errors are wrapped or propagated appropriately.\n* Check whether resources can leak on failure, cancellation, or early return.\n* Check whether ownership expectations are explicit where they need to be.\n* Do not recommend or introduce a pattern merely because it is fashionable or common elsewhere; it must improve this repository specifically.\n\n### Architecture\n\n* Verify that responsibilities remain in the correct package, type, and layer.\n* Check dependency direction and existing architectural boundaries.\n* Look for unwanted coupling, leaked implementation details, duplicated semantics, misplaced behavior, or abstractions at the wrong level.\n* Verify that runtime-owned semantics remain in pkg/runtime rather than being redefined by VM, stdlib, encoding, or debugger consumers.\n* Verify that compile-time behavior and runtime behavior remain separated appropriately.\n* Verify that public or extension-facing APIs are introduced only when the task genuinely requires them.\n* Consider whether the design will remain understandable and maintainable as the feature evolves.\n\n### Code organization and split\n\n* Verify that files, types, methods, functions, and packages have clear responsibilities.\n* Check compliance with the Go type/file structure rules in this file.\n* Check compliance with function and method ownership rules.\n* Look for files, functions, or types doing too much.\n* Look for unrelated responsibilities grouped together.\n* Also avoid unnecessary fragmentation where tightly related behavior has been split into excessive helpers, files, or abstractions.\n* Verify that helpers exist at the narrowest appropriate ownership level.\n* Ensure behavioral ownership is obvious from code layout.\n\n### Tests\n\n* Review test coverage for meaningful behavioral gaps.\n* Look especially for missing negative cases, edge conditions, cleanup paths, cancellation paths, invalid states, and boundary inputs.\n* Check for brittle tests coupled unnecessarily to implementation details.\n* Check for redundant tests that add maintenance cost without meaningful coverage.\n* Check for weak assertions that would allow plausible regressions to pass.\n* Verify diagnostic tests check message category and span accuracy where relevant.\n* Verify integration coverage exists when user-visible behavior crosses package boundaries.\n\n## Performance\n\nFor significant changes:\n\n* Inspect the final implementation for accidental allocations, repeated work, unnecessary materialization, unnecessary synchronization, or additional hot-path overhead.\n* Compare required benchmark results against the recorded baseline.\n* Verify that benchmark changes are attributable to the implementation rather than a different benchmark setup.\n* Do not trade clear correctness or maintainability for speculative micro-optimization.\n* If performance regresses meaningfully, investigate before considering the task complete.\n\n### Review findings and remediation\n\nWhen the self-review finds a problem:\n\n1. Fix correctness issues and regressions.\n2. Fix meaningful architectural, ownership, lifecycle, API, or maintainability problems.\n3. Simplify unnecessarily complicated code when doing so clearly improves the implementation.\n4. Correct file, type, or method ownership violations.\n5. Add or improve tests when the review exposes a behavioral coverage gap.\n6. Re-run validation affected by the change.\n7. Re-run relevant benchmarks if the correction affects benchmarked code.\n\nDo not leave a known correctness, architecture, ownership, lifecycle, or significant test-coverage problem unresolved merely because the initial task implementation already works.\n\nMinor stylistic preferences do not require changes.\n\nDistinguish actual problems from optional preferences. Existing code that is already clear, correct, idiomatic, and appropriately structured should be left alone.\n\nDo not use the self-review as justification for:\n\n* speculative refactoring\n* unrelated cleanup\n* unrelated API redesign\n* rewriting existing code merely for stylistic consistency\n* introducing abstractions without a concrete need\n* broad package reshuffling\n* changing FQL semantics beyond the requested task\n\n### Final diff inspection\n\nImmediately before finishing a non-trivial task, inspect the complete final diff as a whole rather than reviewing only individual edited files.\n\nVerify that:\n\n* every changed line is relevant to the requested task or a necessary supporting change;\n* no temporary or debugging code remains;\n* no accidental behavior changes were introduced;\n* no accidental API changes were introduced;\n* no unrelated refactors slipped into the change;\n* generated files changed only when their source inputs required regeneration;\n* tests describe intended behavior rather than implementation details;\n* comments describe current contracts, behavior, and invariants rather than abandoned implementation ideas;\n* file, type, function, and package boundaries remain coherent;\n* resource ownership and lifecycle behavior remain correct;\n* the resulting implementation is the smallest coherent change that fully solves the task.\n\nIf final diff inspection reveals an issue, correct it and repeat the affected validation before finishing.\n\n## Significant changes\n\nA change is significant when it could reasonably affect:\n\n* execution throughput\n* compile-time performance\n* latency on common paths\n* allocation patterns\n* memory reuse, pooling, or cleanup behavior\n* result/materialization cost\n* optimizer or code generation output relevant to performance\n\nThis includes, but is not limited to, changes in:\n\n* pkg/vm\n* pkg/runtime\n* pkg/compiler\n* pkg/bytecode\n* pkg/encoding\n* parser/compiler hot paths\n* caching, pooling, register allocation, ownership tracking, or materialization logic\n* debugger hooks on execution hot paths\n\nThis usually does not include:\n\n* comment-only, docs-only, or formatting-only edits\n* pure renames with no behavior change\n* test-only changes\n* narrowly scoped refactors that do not affect behavior or hot paths\n\nWhen in doubt, treat the change as significant and benchmark it.\n\n### Benchmark workflow for significant changes\n\nFor significant changes, agents must:\n\n* run relevant benchmarks before making the change and save the results as a baseline\n* implement the change\n* run the same benchmarks again after the change\n* compare before/after results, preferably including ns/op, B/op, and allocs/op\n* report the benchmark command used and summarize the performance delta\n\nIf no relevant benchmark exists for the changed hot path, add one.\n\nIf benchmark tooling or environment is unavailable, state that explicitly and do not claim benchmark validation was completed.\n\n## Test placement rules\n\n* Parser syntax behavior should have parser-focused tests or fixtures.\n* Compiler semantic behavior should have compiler tests and diagnostics/span assertions when relevant.\n* Bytecode emission changes should include compiler or integration tests that verify emitted behavior, not just VM behavior.\n* VM opcode behavior should have VM-level tests plus integration coverage when user-visible.\n* Stdlib behavior should be tested at the Ferret-language level whenever practical.\n* Public embedding behavior should have top-level API tests, not only package-internal tests.\n* Debugger behavior should test protocol/inspection output separately from VM execution semantics when possible.\n\n## Validation and evidence\n\nWhen finishing a non-trivial change, agents must report:\n\n* owning subsystem\n* files changed\n* tests added or updated\n* benchmarks added or updated\n* validation commands run\n* benchmark commands run, if applicable\n* self-review completed\n* notable issues found and corrected during self-review, if any\n* notable invariants preserved or intentionally changed\n* remaining concerns or limitations, if any\n\nFor significant changes:\n\n* tests alone are not sufficient\n* both correctness tests and benchmarks are required\n* benchmark results must be compared against a baseline when the environment allows it\n\nDo not claim:\n\n* tests passed unless they were actually run;\n* benchmarks were completed unless they were actually run;\n* self-review was completed unless the final implementation and diff were actually inspected;\n* validation succeeded if commands failed or were skipped.\n\nIf validation, benchmarking, or review work could not be completed because of tooling or environment limitations, state that explicitly.\n\n## Change discipline\n\n* Prefer adapting an existing local pattern over introducing a new architectural pattern.\n* Do not add new helper layers, wrappers, interfaces, or abstractions only for aesthetic reasons.\n* Do not move code across packages unless the ownership boundary is genuinely wrong.\n* Keep diffs focused on the requested task.\n* If a cleanup is necessary to make the requested change safe, keep it tightly scoped and explain why it was needed.\n* Self-review must not expand task scope unless a discovered problem directly affects correctness, safety, architecture, lifecycle, or maintainability of the requested change.\n\n## Comment and documentation discipline\n\n* Add comments where semantics, invariants, side effects, ownership, lifecycle, or recovery behavior are non-obvious.\n* Do not add comment wallpaper.\n* Prefer comments that explain why, contract, or invariants rather than implementation narration.\n* Public and extension-facing behavior should be documented more carefully than local obvious helpers.\n\n## Decision bias when uncertain\n\nWhen uncertain:\n\n* preserve existing behavior\n* prefer the smaller local change\n* add a focused test\n* treat the change as significant if performance might be affected\n* verify ownership before introducing a new abstraction or package-level dependency\n* prefer fixing an actual review finding over performing speculative cleanup\n* leave already-correct code alone\n\n## Tooling prerequisites\n\n* Go must be installed.\n* make is optional but is the preferred entrypoint for repo-defined workflows.\n* Java plus ANTLR 4.13.2 are required when regenerating parser artifacts.\n* staticcheck, goimports, and revive are needed for lint/format flows; install them with make install-tools.\n\n## Command matrix\n\n* Broad validation: go test ./...\n* Race-heavy package and integration coverage: make test\n* Lint: make lint\n* Format: make fmt\n* Regenerate parser/codegen artifacts: make generate\n* Build the CLI binary: make compile\n\nRun make generate only when grammar or generator inputs change.\n\n## Editing rules\n\n* Never hand-edit generated files under pkg/parser/fql or pkg/parser/antlr/gen.\n* Parser generation is driven by pkg/parser/parser.go:\n    * antlr -Xexact-output-dir -o fql -package fql -visitor -Dlanguage=Go antlr/FqlLexer.g4 antlr/FqlParser.g4\n    * go run ./tools/patch_lexer.go\n* If you change grammar files in pkg/parser/antlr, run make generate and commit the generated output in the same change.\n* Treat Makefile and .github/workflows/build.yml as the source of truth for validation commands.\n* Prefer narrow validation first, then broaden:\n    * Package-local changes: run the affected go test package or packages.\n    * Compiler, optimizer, or VM changes: run the relevant integration suites.\n    * Cross-cutting changes: finish with go test ./... or make test.\n\n## Validation expectations\n\n* After code changes, run the narrowest tests that prove the behavior you touched.\n* Before finishing broader changes, run the relevant repo-level command from the matrix above.\n* If you changed formatting-sensitive files, run make fmt.\n* If you changed lint-sensitive code paths or public behavior, run make lint when the toolchain is available.\n* If you changed parser grammar, generated lexer/parser output must be included and reviewed.\n* After review-driven code changes, re-run the validation relevant to those changes.\n* Do not consider initial validation sufficient if the implementation changed afterward.\n\n### Expectations for non-trivial changes\n\nWhen proposing or implementing non-trivial changes:\n\n* identify the owning subsystem first\n* preserve invariants unless the task explicitly changes them\n* prefer local, comprehensible changes before introducing new abstractions\n* distinguish correctness work from performance work\n* do not perform opportunistic refactors unrelated to the requested task unless they are necessary for correctness\n* complete the mandatory final self-review before finishing\n* inspect the final diff after all review-driven corrections\n* re-run affected validation after the last implementation change\n\n## Secondary references\n\n* README.md for product context and links to the broader Ferret ecosystem.\n* CONTRIBUTING.md for human contributor process.\n* .github/workflows/build.yml for the current CI validation path.\n\n## Website documentation synchronization\n\nFerret's public documentation is maintained in the website repository.\n\nChanges to public behavior must include corresponding website documentation updates when applicable. In particular, always evaluate documentation impact when changing:\n\n* FQL syntax, grammar, operators, expressions, statements, or language semantics;\n* embedding APIs or embedding behavior;\n* public SDK APIs, contracts, helpers, or extension points;\n* other public behavior already documented on the website.\n\nWhen such a change affects existing documentation:\n\n* locate the corresponding documentation in the website repository;\n* update it as part of the same task when the repository is available;\n* keep examples, syntax descriptions, API descriptions, and behavioral notes consistent with the implementation;\n* remove or revise documentation that describes behavior made obsolete by the change.\n\nFor new public syntax, embedding features, or SDK capabilities, add documentation to the appropriate existing section rather than leaving the implementation as the only specification.\n\nDocumentation synchronization is part of completing the change, not optional follow-up work.\n\nIf the website repository is not available in the working environment, explicitly report the required documentation update in the final summary rather than silently skipping it.\n","category":"root","tokens":11353}]}