{"owner":"apple","repo":"foundationdb","hasSkills":true,"totalSkillsCount":2,"totalTokensCount":2867,"categories":["root-instruction","copilot-instructions"],"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md",".github/copilot-instructions.md"],"skills":{"AGENTS.md":"# AGENTS.md\n\nThis file provides guidance to AI coding agents working in this repository.\n\n## Build\n\n```bash\nmkdir build && cd build\ncmake -G Ninja <SOURCE_DIR>\nninja\n# or: ninja fdbserver fdbcli fdbclient  (specific targets)\n```\n\nKey cmake options:\n- `-DCMAKE_BUILD_TYPE=Debug` (default Release)\n- `-DUSE_WERROR=ON` for development\n- `-DOPEN_FOR_IDE=ON` to generate IDE project without building\n\nmacOS: add `-DBUILD_SWIFT_BINDING=OFF` if Swift issues arise.\n\nLinux with Clang: `CC=clang CXX=clang++ cmake -DUSE_LD=LLD -DUSE_LIBCXX=1 -G Ninja ..`\n\n## Testing\n\n**Unit tests** use `TEST_CASE(\"/path/to/test\")` macros (defined in `flow/include/flow/UnitTest.h`). Prefer the narrowest dedicated unit-test target instead of `fdbserver -r unittests`; these targets compile and run faster. Use `fdbclient_test` for client tests or the relevant `fdbserver_<library>_test` target defined via `add_fdbserver_unit_test` (for example, `fdbserver_core_test`, `fdbserver_kvstore_test`, or `fdbserver_storageserver_test`). Find the applicable target in the affected component's `CMakeLists.txt`:\n```bash\nninja fdbclient_test\nbin/fdbclient_test                                  # all tests in the target\nbin/fdbclient_test -f \"<test-prefix>\"                # tests matching a prefix\n\nninja fdbserver_core_test\nbin/fdbserver_core_test\n```\n\nUse `fdbserver` for simulation or broader end-to-end validation, not as the default unit-test runner.\n\n**Simulation tests** use TOML workload definitions in `tests/fast/`, `tests/slow/`, etc.:\n```bash\nbin/fdbserver -r simulation -f tests/fast/CycleTest.toml\n```\n\n**Via ctest:**\n```bash\nctest -L fast                    # all fast tests\nctest -R \"StatusDuringOutage\"    # by name pattern\n```\n\nEnable simulation tests in cmake: `-DENABLE_SIMULATION_TESTS=ON`\n\n**Adding new tests:**\n- For a new otherwise-unreferenced translation unit containing `TEST_CASE`, add a `forceLinkXxxTests()` stub and call it from `fdbserver/workloads/UnitTests.cpp`, or the linker can drop the tests.\n- Register new `.toml` or `.txt` tests under `tests/` in `tests/CMakeLists.txt` via `add_fdb_test`. `configure_testing(... ERROR_ON_ADDITIONAL_FILES)` and `verify_testing()` should catch unregistered files during CMake configuration.\n\n**Joshua**: bulk correctness/simulation testing runs on the internal Joshua cluster against optimized Release builds. Local `fdbserver -r simulation` runs are for iteration; large-scale shake-out happens on Joshua.\n\n**Trace logs**: simulation and test runs emit `trace.*.xml` (or `.json`) files in the run directory. `TraceEvent(...)` is the canonical logging mechanism — search trace files by event name when debugging a simulation failure.\n\n- A `SevError` (Severity 40) `TraceEvent` **fails** a simulation/Joshua run. Use `SevWarnAlways` for \"shouldn't happen but non-fatal.\" Errors carrying an injected fault are auto-downgraded; genuine `SevError`s are not. Expected errors must be suppressed/allowlisted, never emitted at `SevError`.\n- Keep a `TraceEvent` small: any single `detail()` value over 495 bytes is truncated (`...`), and an event whose fields total over 4000 bytes is **dropped entirely** and re-logged as `TraceEventOverflow` — at `SevError` in simulation, so an oversized event *fails the test*. Chunk long payloads (joined lists, command lines, serialized blobs) across multiple events.\n- The `Time` field is `now()`, which does not advance between `co_await`s, so spans within one run-loop slice read as zero. `g_traceBatch` events also carry `MonotonicTime` for this.\n\n## Architecture\n\nFoundationDB is a distributed ordered key-value store with strict serializability. The codebase is organized into ~12 subsystems. For background on subsystems before diving into code, the `design/` directory holds human-authored design docs and `design/AI-generated/` holds subsystem maps and per-subsystem diagrams (start with `design/AI-generated/foundationdb_subsystem_map.md`).\n\n### Concurrency Model: C++ Coroutines\n\nFDB uses cooperative single-threaded concurrency. Asynchronous functions use\nstandard C++ coroutines with `co_await` and `co_return` in ordinary `.cpp` and\n`.h` files. See `design/coroutines.md` for coroutine and cancellation semantics.\n\nKey types: `Future<T>`, `Promise<T>`, `PromiseStream<T>`, `Reference<T>` (ref-counted pointer), `Optional<T>`, `ErrorOr<T>`, `Arena` (region-based allocation).\n\n#### Common pitfalls\n\n- C++ coroutines: `co_await` is not allowed inside a `catch` handler. Capture the error, exit the catch, then `co_await` outside.\n- Errors are integer codes (`flow/include/flow/error_definitions.h`), not exceptions with messages. When you `catch (Error& e)`, re-throw `actor_cancelled` (and never silently swallow `broken_promise`) — eating cancellation causes hangs and leaks. Transaction retry goes through `tr.onError(e)`, not a bare loop.\n- `StringRef`/`KeyRef`/`ValueRef` are non-owning views into an `Arena`. Returning or storing one past its arena's lifetime is a dangling-reference bug; use `Standalone<>` (or `Key`/`Value`) when you need to own the bytes.\n- `a && b` on `Future<Void>` is `quorum(x, 2, 2)`, not a join: the first error completes it and drops the other input, cancelling it if nothing else holds it, and an already-failed `lhs` discards `rhs` unreferenced. Never accumulate with `f = f && g`; use `waitForAllReady()` to wait for everything regardless of errors.\n- `Subspace::range()` spans `prefix + '\\x00'` to `prefix + '\\xff'`, so it excludes the subspace's own key: a value written at `s.pack(x)` is not returned by scanning `s.get(x).range()`.\n\n### Core Subsystems\n\n- **`flow/`** — Async runtime, event loop, tracing, deterministic random, arenas\n- **`fdbrpc/`** — Endpoint-addressed RPC, peer management, failure monitor, Sim2 (deterministic simulation network)\n- **`fdbclient/`** — Transaction API (`NativeAPI.cpp`), read-your-writes (`ReadYourWrites.cpp`), location cache, multi-version client\n- **`fdbserver/`** — All server roles, organized by subdirectory:\n  - `clustercontroller/` — Leader election, role recruitment, ServerDBInfo broadcasting\n  - `coordinator/` — Paxos-based coordination state (generation registers)\n  - `commitproxy/`, `grvproxy/`, `resolver/` — Transaction commit pipeline\n  - `tlog/`, `logsystem/` — Durable mutation log (tag-partitioned)\n  - `storageserver/` — Serves reads, applies mutations from TLogs\n  - `kvstore/` — Pluggable storage engines (RocksDB, SQLite, memory/DiskQueue)\n  - `datadistributor/` — Shard management, team building, data movement\n  - `ratekeeper/` — Back-pressure and throttling\n  - `workloads/` — Simulation test workloads\n\n### Simulation Testing\n\n`fdbserver -r simulation` runs the entire cluster in a single process using Sim2, a deterministic simulated network. `buggify()` calls (`flow/include/flow/Buggify.h`) inject faults (delays, failures, corruption). Tests are TOML files that compose workloads. This is FDB's primary testing strategy.\n\n- Determinism is per seed *and* binary. Any code change shifts the `deterministicRandom()` sequence, so the same seed on a patched binary is a different execution, not an A/B comparison. Reproduce a failure on the unpatched binary, and judge a fix by failure rates over many seeds.\n- A continuation can resume with a different simulated process current than the one that started waiting, e.g. when a future is shared between processes on one simulated machine. Code that binds to `g_simulator->getCurrentProcess()` should capture it on entry and `co_await g_simulator->onProcess(...)` back before binding.\n\n### Knobs\n\nRuntime-tunable parameters are declared in `fdbserver/core/include/fdbserver/core/Knobs.h`, `fdbclient/include/fdbclient/Knobs.h`, and `flow/include/flow/Knobs.h` (UPPER_CASE names). Access them via `SERVER_KNOBS`, `CLIENT_KNOBS`, or `FLOW_KNOBS`. Defaults and buggification ranges live in `fdbserver/core/ServerKnobs.cpp`, `fdbclient/ClientKnobs.cpp`, and `flow/Knobs.cpp`.\n\n## Compatibility\n\nBefore changing a serialized type that persists on disk, inspect its `serializer()` and adjacent versioning or downgrade comments. Do not reorder, remove, or change encoded fields without the appropriate version gate or migration and compatibility coverage.\n\n## Naming Conventions\n\n- **Variables, functions, methods**: camelCase (`smoothTotal`, `getWorkerForRole`)\n- **Classes, structs, enums**: PascalCase (`StorageServer`, `RecoveryState`)\n- **Knobs/constants**: UPPER_CASE (`MAX_BATCH_SIZE`, `VERSIONS_PER_SECOND`)\n- **Globals**: `g_` prefix for important ones (`g_network`, `g_simulator`)\n\n## Code Formatting\n\n`clang-format` is used. Python code uses `black` and `flake8` (pre-commit hooks: `pip install pre-commit && pre-commit install`).\n\nEdit tracked `.cpp` and `.h` sources, not generated output under the build directory.\n\n## Source File Headers\n\nEvery new `.cpp` / `.h` file starts with the standard Apache 2.0 license block, with the filename on line 2 and the current year on the copyright line. Copy from any existing file in the tree (e.g. `flow/Knobs.cpp`). Add file-purpose comments *after* the license block, not in place of it.\n\n## Code Comments\n\nGeneral guidance: comments should maximize clarity. They should be clear without being verbose. They should be concise without being cryptic.\n\nScope: the guidance here mainly governs inline and implementation comments. Comments on public-facing APIs are contract statements and must be clear, precise, and complete.\n\nAudience: assume the readers of FDB are senior engineers fluent in asynchronous and concurrent programming, distributed systems, and stateful complex systems generally. Calibrate \"non-obvious\" to that reader — do not explain the language, the standard library, Flow/actor idioms, or established patterns they already know. Reserve comments for what such a reader could not quickly infer from the code itself.\n\nImplementation comments should focus on important, non-obvious fundamental rationale, motivation, and explanation of extremely subtle phenomena. Writing down important invariants — and the preconditions a piece of code requires of its caller (e.g. a lock that must be held, or a version that must already be durable) — is also a good idea.\n\nDo not write comments about mundane day-to-day development refactorings — junk like `// This used to be part of xyzFunc in Foo.cpp but we moved it here so that aaaFunc could also call it`.\n\nDo not write caller-location / \"happening elsewhere\" comments — narration of who invokes this code or where it is used (\"called from X\", \"used by Y\"). They duplicate what a search answers instantly and rot when callers change; describe what the code requires and guarantees on its own terms. Stating a real invariant is fine — \"recovery is the only path that repairs\" — naming the callers that happen to exist today is not.\n\nAfter writing or changing a bunch of code, make a pass over it one last time to review the comments. Ask yourself: am I rambling? Am I repeating what the code is showing? Am I just telling stories here? Is the information I'm writing going to be relevant and important a week from now, a month from now, a year from now? Is it genuinely essential and **non-obvious from the code itself**?\n\n## Code Review\n\nWhen reviewing, state explicitly: what the change is for, whether it is correct, bugs, omissions, better approaches, and a clear LGTM / no / not-yet. Read a construct's history (`git log -L`, `git blame`) before proposing to change something that looks odd.\n\n## Branching\n\nPRs target `main`. Release branches receive cherry-picks rather than direct PRs — don't open backport PRs without confirming first.\n\n## Design Docs\n\nBefore a design doc is considered finished (not at the start), reconcile it against `design/design-doc-template.md`.\n",".github/copilot-instructions.md":"When reviewing PRs, address general code review criteria, but also specifically the following questions: What is it trying to do? Is it correct? Are there bugs? Are there omissions? Are there better ways of doing things? Should this CL be approved?\n"},"files":{"AGENTS.md":"# AGENTS.md\n\nThis file provides guidance to AI coding agents working in this repository.\n\n## Build\n\n```bash\nmkdir build && cd build\ncmake -G Ninja <SOURCE_DIR>\nninja\n# or: ninja fdbserver fdbcli fdbclient  (specific targets)\n```\n\nKey cmake options:\n- `-DCMAKE_BUILD_TYPE=Debug` (default Release)\n- `-DUSE_WERROR=ON` for development\n- `-DOPEN_FOR_IDE=ON` to generate IDE project without building\n\nmacOS: add `-DBUILD_SWIFT_BINDING=OFF` if Swift issues arise.\n\nLinux with Clang: `CC=clang CXX=clang++ cmake -DUSE_LD=LLD -DUSE_LIBCXX=1 -G Ninja ..`\n\n## Testing\n\n**Unit tests** use `TEST_CASE(\"/path/to/test\")` macros (defined in `flow/include/flow/UnitTest.h`). Prefer the narrowest dedicated unit-test target instead of `fdbserver -r unittests`; these targets compile and run faster. Use `fdbclient_test` for client tests or the relevant `fdbserver_<library>_test` target defined via `add_fdbserver_unit_test` (for example, `fdbserver_core_test`, `fdbserver_kvstore_test`, or `fdbserver_storageserver_test`). Find the applicable target in the affected component's `CMakeLists.txt`:\n```bash\nninja fdbclient_test\nbin/fdbclient_test                                  # all tests in the target\nbin/fdbclient_test -f \"<test-prefix>\"                # tests matching a prefix\n\nninja fdbserver_core_test\nbin/fdbserver_core_test\n```\n\nUse `fdbserver` for simulation or broader end-to-end validation, not as the default unit-test runner.\n\n**Simulation tests** use TOML workload definitions in `tests/fast/`, `tests/slow/`, etc.:\n```bash\nbin/fdbserver -r simulation -f tests/fast/CycleTest.toml\n```\n\n**Via ctest:**\n```bash\nctest -L fast                    # all fast tests\nctest -R \"StatusDuringOutage\"    # by name pattern\n```\n\nEnable simulation tests in cmake: `-DENABLE_SIMULATION_TESTS=ON`\n\n**Adding new tests:**\n- For a new otherwise-unreferenced translation unit containing `TEST_CASE`, add a `forceLinkXxxTests()` stub and call it from `fdbserver/workloads/UnitTests.cpp`, or the linker can drop the tests.\n- Register new `.toml` or `.txt` tests under `tests/` in `tests/CMakeLists.txt` via `add_fdb_test`. `configure_testing(... ERROR_ON_ADDITIONAL_FILES)` and `verify_testing()` should catch unregistered files during CMake configuration.\n\n**Joshua**: bulk correctness/simulation testing runs on the internal Joshua cluster against optimized Release builds. Local `fdbserver -r simulation` runs are for iteration; large-scale shake-out happens on Joshua.\n\n**Trace logs**: simulation and test runs emit `trace.*.xml` (or `.json`) files in the run directory. `TraceEvent(...)` is the canonical logging mechanism — search trace files by event name when debugging a simulation failure.\n\n- A `SevError` (Severity 40) `TraceEvent` **fails** a simulation/Joshua run. Use `SevWarnAlways` for \"shouldn't happen but non-fatal.\" Errors carrying an injected fault are auto-downgraded; genuine `SevError`s are not. Expected errors must be suppressed/allowlisted, never emitted at `SevError`.\n- Keep a `TraceEvent` small: any single `detail()` value over 495 bytes is truncated (`...`), and an event whose fields total over 4000 bytes is **dropped entirely** and re-logged as `TraceEventOverflow` — at `SevError` in simulation, so an oversized event *fails the test*. Chunk long payloads (joined lists, command lines, serialized blobs) across multiple events.\n- The `Time` field is `now()`, which does not advance between `co_await`s, so spans within one run-loop slice read as zero. `g_traceBatch` events also carry `MonotonicTime` for this.\n\n## Architecture\n\nFoundationDB is a distributed ordered key-value store with strict serializability. The codebase is organized into ~12 subsystems. For background on subsystems before diving into code, the `design/` directory holds human-authored design docs and `design/AI-generated/` holds subsystem maps and per-subsystem diagrams (start with `design/AI-generated/foundationdb_subsystem_map.md`).\n\n### Concurrency Model: C++ Coroutines\n\nFDB uses cooperative single-threaded concurrency. Asynchronous functions use\nstandard C++ coroutines with `co_await` and `co_return` in ordinary `.cpp` and\n`.h` files. See `design/coroutines.md` for coroutine and cancellation semantics.\n\nKey types: `Future<T>`, `Promise<T>`, `PromiseStream<T>`, `Reference<T>` (ref-counted pointer), `Optional<T>`, `ErrorOr<T>`, `Arena` (region-based allocation).\n\n#### Common pitfalls\n\n- C++ coroutines: `co_await` is not allowed inside a `catch` handler. Capture the error, exit the catch, then `co_await` outside.\n- Errors are integer codes (`flow/include/flow/error_definitions.h`), not exceptions with messages. When you `catch (Error& e)`, re-throw `actor_cancelled` (and never silently swallow `broken_promise`) — eating cancellation causes hangs and leaks. Transaction retry goes through `tr.onError(e)`, not a bare loop.\n- `StringRef`/`KeyRef`/`ValueRef` are non-owning views into an `Arena`. Returning or storing one past its arena's lifetime is a dangling-reference bug; use `Standalone<>` (or `Key`/`Value`) when you need to own the bytes.\n- `a && b` on `Future<Void>` is `quorum(x, 2, 2)`, not a join: the first error completes it and drops the other input, cancelling it if nothing else holds it, and an already-failed `lhs` discards `rhs` unreferenced. Never accumulate with `f = f && g`; use `waitForAllReady()` to wait for everything regardless of errors.\n- `Subspace::range()` spans `prefix + '\\x00'` to `prefix + '\\xff'`, so it excludes the subspace's own key: a value written at `s.pack(x)` is not returned by scanning `s.get(x).range()`.\n\n### Core Subsystems\n\n- **`flow/`** — Async runtime, event loop, tracing, deterministic random, arenas\n- **`fdbrpc/`** — Endpoint-addressed RPC, peer management, failure monitor, Sim2 (deterministic simulation network)\n- **`fdbclient/`** — Transaction API (`NativeAPI.cpp`), read-your-writes (`ReadYourWrites.cpp`), location cache, multi-version client\n- **`fdbserver/`** — All server roles, organized by subdirectory:\n  - `clustercontroller/` — Leader election, role recruitment, ServerDBInfo broadcasting\n  - `coordinator/` — Paxos-based coordination state (generation registers)\n  - `commitproxy/`, `grvproxy/`, `resolver/` — Transaction commit pipeline\n  - `tlog/`, `logsystem/` — Durable mutation log (tag-partitioned)\n  - `storageserver/` — Serves reads, applies mutations from TLogs\n  - `kvstore/` — Pluggable storage engines (RocksDB, SQLite, memory/DiskQueue)\n  - `datadistributor/` — Shard management, team building, data movement\n  - `ratekeeper/` — Back-pressure and throttling\n  - `workloads/` — Simulation test workloads\n\n### Simulation Testing\n\n`fdbserver -r simulation` runs the entire cluster in a single process using Sim2, a deterministic simulated network. `buggify()` calls (`flow/include/flow/Buggify.h`) inject faults (delays, failures, corruption). Tests are TOML files that compose workloads. This is FDB's primary testing strategy.\n\n- Determinism is per seed *and* binary. Any code change shifts the `deterministicRandom()` sequence, so the same seed on a patched binary is a different execution, not an A/B comparison. Reproduce a failure on the unpatched binary, and judge a fix by failure rates over many seeds.\n- A continuation can resume with a different simulated process current than the one that started waiting, e.g. when a future is shared between processes on one simulated machine. Code that binds to `g_simulator->getCurrentProcess()` should capture it on entry and `co_await g_simulator->onProcess(...)` back before binding.\n\n### Knobs\n\nRuntime-tunable parameters are declared in `fdbserver/core/include/fdbserver/core/Knobs.h`, `fdbclient/include/fdbclient/Knobs.h`, and `flow/include/flow/Knobs.h` (UPPER_CASE names). Access them via `SERVER_KNOBS`, `CLIENT_KNOBS`, or `FLOW_KNOBS`. Defaults and buggification ranges live in `fdbserver/core/ServerKnobs.cpp`, `fdbclient/ClientKnobs.cpp`, and `flow/Knobs.cpp`.\n\n## Compatibility\n\nBefore changing a serialized type that persists on disk, inspect its `serializer()` and adjacent versioning or downgrade comments. Do not reorder, remove, or change encoded fields without the appropriate version gate or migration and compatibility coverage.\n\n## Naming Conventions\n\n- **Variables, functions, methods**: camelCase (`smoothTotal`, `getWorkerForRole`)\n- **Classes, structs, enums**: PascalCase (`StorageServer`, `RecoveryState`)\n- **Knobs/constants**: UPPER_CASE (`MAX_BATCH_SIZE`, `VERSIONS_PER_SECOND`)\n- **Globals**: `g_` prefix for important ones (`g_network`, `g_simulator`)\n\n## Code Formatting\n\n`clang-format` is used. Python code uses `black` and `flake8` (pre-commit hooks: `pip install pre-commit && pre-commit install`).\n\nEdit tracked `.cpp` and `.h` sources, not generated output under the build directory.\n\n## Source File Headers\n\nEvery new `.cpp` / `.h` file starts with the standard Apache 2.0 license block, with the filename on line 2 and the current year on the copyright line. Copy from any existing file in the tree (e.g. `flow/Knobs.cpp`). Add file-purpose comments *after* the license block, not in place of it.\n\n## Code Comments\n\nGeneral guidance: comments should maximize clarity. They should be clear without being verbose. They should be concise without being cryptic.\n\nScope: the guidance here mainly governs inline and implementation comments. Comments on public-facing APIs are contract statements and must be clear, precise, and complete.\n\nAudience: assume the readers of FDB are senior engineers fluent in asynchronous and concurrent programming, distributed systems, and stateful complex systems generally. Calibrate \"non-obvious\" to that reader — do not explain the language, the standard library, Flow/actor idioms, or established patterns they already know. Reserve comments for what such a reader could not quickly infer from the code itself.\n\nImplementation comments should focus on important, non-obvious fundamental rationale, motivation, and explanation of extremely subtle phenomena. Writing down important invariants — and the preconditions a piece of code requires of its caller (e.g. a lock that must be held, or a version that must already be durable) — is also a good idea.\n\nDo not write comments about mundane day-to-day development refactorings — junk like `// This used to be part of xyzFunc in Foo.cpp but we moved it here so that aaaFunc could also call it`.\n\nDo not write caller-location / \"happening elsewhere\" comments — narration of who invokes this code or where it is used (\"called from X\", \"used by Y\"). They duplicate what a search answers instantly and rot when callers change; describe what the code requires and guarantees on its own terms. Stating a real invariant is fine — \"recovery is the only path that repairs\" — naming the callers that happen to exist today is not.\n\nAfter writing or changing a bunch of code, make a pass over it one last time to review the comments. Ask yourself: am I rambling? Am I repeating what the code is showing? Am I just telling stories here? Is the information I'm writing going to be relevant and important a week from now, a month from now, a year from now? Is it genuinely essential and **non-obvious from the code itself**?\n\n## Code Review\n\nWhen reviewing, state explicitly: what the change is for, whether it is correct, bugs, omissions, better approaches, and a clear LGTM / no / not-yet. Read a construct's history (`git log -L`, `git blame`) before proposing to change something that looks odd.\n\n## Branching\n\nPRs target `main`. Release branches receive cherry-picks rather than direct PRs — don't open backport PRs without confirming first.\n\n## Design Docs\n\nBefore a design doc is considered finished (not at the start), reconcile it against `design/design-doc-template.md`.\n",".github/copilot-instructions.md":"When reviewing PRs, address general code review criteria, but also specifically the following questions: What is it trying to do? Is it correct? Are there bugs? Are there omissions? Are there better ways of doing things? Should this CL be approved?\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","rawUrl":"https://raw.githubusercontent.com/apple/foundationdb/HEAD/AGENTS.md","title":"AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"# AGENTS.md\n\nThis file provides guidance to AI coding agents working in this repository.\n\n## Build\n\n```bash\nmkdir build && cd build\ncmake -G Ninja <SOURCE_DIR>\nninja\n# or: ninja fdbserver fdbcli fdbclient  (specific targets)\n```\n\nKey cmake options:\n- `-DCMAKE_BUILD_TYPE=Debug` (default Release)\n- `-DUSE_WERROR=ON` for development\n- `-DOPEN_FOR_IDE=ON` to generate IDE project without building\n\nmacOS: add `-DBUILD_SWIFT_BINDING=OFF` if Swift issues arise.\n\nLinux with Clang: `CC=clang CXX=clang++ cmake -DUSE_LD=LLD -DUSE_LIBCXX=1 -G Ninja ..`\n\n## Testing\n\n**Unit tests** use `TEST_CASE(\"/path/to/test\")` macros (defined in `flow/include/flow/UnitTest.h`). Prefer the narrowest dedicated unit-test target instead of `fdbserver -r unittests`; these targets compile and run faster. Use `fdbclient_test` for client tests or the relevant `fdbserver_<library>_test` target defined via `add_fdbserver_unit_test` (for example, `fdbserver_core_test`, `fdbserver_kvstore_test`, or `fdbserver_storageserver_test`). Find the applicable target in the affected component's `CMakeLists.txt`:\n```bash\nninja fdbclient_test\nbin/fdbclient_test                                  # all tests in the target\nbin/fdbclient_test -f \"<test-prefix>\"                # tests matching a prefix\n\nninja fdbserver_core_test\nbin/fdbserver_core_test\n```\n\nUse `fdbserver` for simulation or broader end-to-end validation, not as the default unit-test runner.\n\n**Simulation tests** use TOML workload definitions in `tests/fast/`, `tests/slow/`, etc.:\n```bash\nbin/fdbserver -r simulation -f tests/fast/CycleTest.toml\n```\n\n**Via ctest:**\n```bash\nctest -L fast                    # all fast tests\nctest -R \"StatusDuringOutage\"    # by name pattern\n```\n\nEnable simulation tests in cmake: `-DENABLE_SIMULATION_TESTS=ON`\n\n**Adding new tests:**\n- For a new otherwise-unreferenced translation unit containing `TEST_CASE`, add a `forceLinkXxxTests()` stub and call it from `fdbserver/workloads/UnitTests.cpp`, or the linker can drop the tests.\n- Register new `.toml` or `.txt` tests under `tests/` in `tests/CMakeLists.txt` via `add_fdb_test`. `configure_testing(... ERROR_ON_ADDITIONAL_FILES)` and `verify_testing()` should catch unregistered files during CMake configuration.\n\n**Joshua**: bulk correctness/simulation testing runs on the internal Joshua cluster against optimized Release builds. Local `fdbserver -r simulation` runs are for iteration; large-scale shake-out happens on Joshua.\n\n**Trace logs**: simulation and test runs emit `trace.*.xml` (or `.json`) files in the run directory. `TraceEvent(...)` is the canonical logging mechanism — search trace files by event name when debugging a simulation failure.\n\n- A `SevError` (Severity 40) `TraceEvent` **fails** a simulation/Joshua run. Use `SevWarnAlways` for \"shouldn't happen but non-fatal.\" Errors carrying an injected fault are auto-downgraded; genuine `SevError`s are not. Expected errors must be suppressed/allowlisted, never emitted at `SevError`.\n- Keep a `TraceEvent` small: any single `detail()` value over 495 bytes is truncated (`...`), and an event whose fields total over 4000 bytes is **dropped entirely** and re-logged as `TraceEventOverflow` — at `SevError` in simulation, so an oversized event *fails the test*. Chunk long payloads (joined lists, command lines, serialized blobs) across multiple events.\n- The `Time` field is `now()`, which does not advance between `co_await`s, so spans within one run-loop slice read as zero. `g_traceBatch` events also carry `MonotonicTime` for this.\n\n## Architecture\n\nFoundationDB is a distributed ordered key-value store with strict serializability. The codebase is organized into ~12 subsystems. For background on subsystems before diving into code, the `design/` directory holds human-authored design docs and `design/AI-generated/` holds subsystem maps and per-subsystem diagrams (start with `design/AI-generated/foundationdb_subsystem_map.md`).\n\n### Concurrency Model: C++ Coroutines\n\nFDB uses cooperative single-threaded concurrency. Asynchronous functions use\nstandard C++ coroutines with `co_await` and `co_return` in ordinary `.cpp` and\n`.h` files. See `design/coroutines.md` for coroutine and cancellation semantics.\n\nKey types: `Future<T>`, `Promise<T>`, `PromiseStream<T>`, `Reference<T>` (ref-counted pointer), `Optional<T>`, `ErrorOr<T>`, `Arena` (region-based allocation).\n\n#### Common pitfalls\n\n- C++ coroutines: `co_await` is not allowed inside a `catch` handler. Capture the error, exit the catch, then `co_await` outside.\n- Errors are integer codes (`flow/include/flow/error_definitions.h`), not exceptions with messages. When you `catch (Error& e)`, re-throw `actor_cancelled` (and never silently swallow `broken_promise`) — eating cancellation causes hangs and leaks. Transaction retry goes through `tr.onError(e)`, not a bare loop.\n- `StringRef`/`KeyRef`/`ValueRef` are non-owning views into an `Arena`. Returning or storing one past its arena's lifetime is a dangling-reference bug; use `Standalone<>` (or `Key`/`Value`) when you need to own the bytes.\n- `a && b` on `Future<Void>` is `quorum(x, 2, 2)`, not a join: the first error completes it and drops the other input, cancelling it if nothing else holds it, and an already-failed `lhs` discards `rhs` unreferenced. Never accumulate with `f = f && g`; use `waitForAllReady()` to wait for everything regardless of errors.\n- `Subspace::range()` spans `prefix + '\\x00'` to `prefix + '\\xff'`, so it excludes the subspace's own key: a value written at `s.pack(x)` is not returned by scanning `s.get(x).range()`.\n\n### Core Subsystems\n\n- **`flow/`** — Async runtime, event loop, tracing, deterministic random, arenas\n- **`fdbrpc/`** — Endpoint-addressed RPC, peer management, failure monitor, Sim2 (deterministic simulation network)\n- **`fdbclient/`** — Transaction API (`NativeAPI.cpp`), read-your-writes (`ReadYourWrites.cpp`), location cache, multi-version client\n- **`fdbserver/`** — All server roles, organized by subdirectory:\n  - `clustercontroller/` — Leader election, role recruitment, ServerDBInfo broadcasting\n  - `coordinator/` — Paxos-based coordination state (generation registers)\n  - `commitproxy/`, `grvproxy/`, `resolver/` — Transaction commit pipeline\n  - `tlog/`, `logsystem/` — Durable mutation log (tag-partitioned)\n  - `storageserver/` — Serves reads, applies mutations from TLogs\n  - `kvstore/` — Pluggable storage engines (RocksDB, SQLite, memory/DiskQueue)\n  - `datadistributor/` — Shard management, team building, data movement\n  - `ratekeeper/` — Back-pressure and throttling\n  - `workloads/` — Simulation test workloads\n\n### Simulation Testing\n\n`fdbserver -r simulation` runs the entire cluster in a single process using Sim2, a deterministic simulated network. `buggify()` calls (`flow/include/flow/Buggify.h`) inject faults (delays, failures, corruption). Tests are TOML files that compose workloads. This is FDB's primary testing strategy.\n\n- Determinism is per seed *and* binary. Any code change shifts the `deterministicRandom()` sequence, so the same seed on a patched binary is a different execution, not an A/B comparison. Reproduce a failure on the unpatched binary, and judge a fix by failure rates over many seeds.\n- A continuation can resume with a different simulated process current than the one that started waiting, e.g. when a future is shared between processes on one simulated machine. Code that binds to `g_simulator->getCurrentProcess()` should capture it on entry and `co_await g_simulator->onProcess(...)` back before binding.\n\n### Knobs\n\nRuntime-tunable parameters are declared in `fdbserver/core/include/fdbserver/core/Knobs.h`, `fdbclient/include/fdbclient/Knobs.h`, and `flow/include/flow/Knobs.h` (UPPER_CASE names). Access them via `SERVER_KNOBS`, `CLIENT_KNOBS`, or `FLOW_KNOBS`. Defaults and buggification ranges live in `fdbserver/core/ServerKnobs.cpp`, `fdbclient/ClientKnobs.cpp`, and `flow/Knobs.cpp`.\n\n## Compatibility\n\nBefore changing a serialized type that persists on disk, inspect its `serializer()` and adjacent versioning or downgrade comments. Do not reorder, remove, or change encoded fields without the appropriate version gate or migration and compatibility coverage.\n\n## Naming Conventions\n\n- **Variables, functions, methods**: camelCase (`smoothTotal`, `getWorkerForRole`)\n- **Classes, structs, enums**: PascalCase (`StorageServer`, `RecoveryState`)\n- **Knobs/constants**: UPPER_CASE (`MAX_BATCH_SIZE`, `VERSIONS_PER_SECOND`)\n- **Globals**: `g_` prefix for important ones (`g_network`, `g_simulator`)\n\n## Code Formatting\n\n`clang-format` is used. Python code uses `black` and `flake8` (pre-commit hooks: `pip install pre-commit && pre-commit install`).\n\nEdit tracked `.cpp` and `.h` sources, not generated output under the build directory.\n\n## Source File Headers\n\nEvery new `.cpp` / `.h` file starts with the standard Apache 2.0 license block, with the filename on line 2 and the current year on the copyright line. Copy from any existing file in the tree (e.g. `flow/Knobs.cpp`). Add file-purpose comments *after* the license block, not in place of it.\n\n## Code Comments\n\nGeneral guidance: comments should maximize clarity. They should be clear without being verbose. They should be concise without being cryptic.\n\nScope: the guidance here mainly governs inline and implementation comments. Comments on public-facing APIs are contract statements and must be clear, precise, and complete.\n\nAudience: assume the readers of FDB are senior engineers fluent in asynchronous and concurrent programming, distributed systems, and stateful complex systems generally. Calibrate \"non-obvious\" to that reader — do not explain the language, the standard library, Flow/actor idioms, or established patterns they already know. Reserve comments for what such a reader could not quickly infer from the code itself.\n\nImplementation comments should focus on important, non-obvious fundamental rationale, motivation, and explanation of extremely subtle phenomena. Writing down important invariants — and the preconditions a piece of code requires of its caller (e.g. a lock that must be held, or a version that must already be durable) — is also a good idea.\n\nDo not write comments about mundane day-to-day development refactorings — junk like `// This used to be part of xyzFunc in Foo.cpp but we moved it here so that aaaFunc could also call it`.\n\nDo not write caller-location / \"happening elsewhere\" comments — narration of who invokes this code or where it is used (\"called from X\", \"used by Y\"). They duplicate what a search answers instantly and rot when callers change; describe what the code requires and guarantees on its own terms. Stating a real invariant is fine — \"recovery is the only path that repairs\" — naming the callers that happen to exist today is not.\n\nAfter writing or changing a bunch of code, make a pass over it one last time to review the comments. Ask yourself: am I rambling? Am I repeating what the code is showing? Am I just telling stories here? Is the information I'm writing going to be relevant and important a week from now, a month from now, a year from now? Is it genuinely essential and **non-obvious from the code itself**?\n\n## Code Review\n\nWhen reviewing, state explicitly: what the change is for, whether it is correct, bugs, omissions, better approaches, and a clear LGTM / no / not-yet. Read a construct's history (`git log -L`, `git blame`) before proposing to change something that looks odd.\n\n## Branching\n\nPRs target `main`. Release branches receive cherry-picks rather than direct PRs — don't open backport PRs without confirming first.\n\n## Design Docs\n\nBefore a design doc is considered finished (not at the start), reconcile it against `design/design-doc-template.md`.\n","isInternal":false,"tokens":2815,"sizeBytes":11768},{"name":"copilot-instructions.md","path":".github/copilot-instructions.md","rawUrl":"https://raw.githubusercontent.com/apple/foundationdb/HEAD/.github/copilot-instructions.md","title":"GitHub Copilot Instructions","category":"copilot-instructions","format":"markdown","content":"When reviewing PRs, address general code review criteria, but also specifically the following questions: What is it trying to do? Is it correct? Are there bugs? Are there omissions? Are there better ways of doing things? Should this CL be approved?\n","isInternal":false,"tokens":52,"sizeBytes":249}],"systemPromptSnippet":"<yakaai_skills repo=\"apple/foundationdb\">\n<!-- File: AGENTS.md (Tokens: ~2815 | Category: root-instruction) -->\n# AGENTS.md\n\nThis file provides guidance to AI coding agents working in this repository.\n\n## Build\n\n```bash\nmkdir build && cd build\ncmake -G Ninja <SOURCE_DIR>\nninja\n# or: ninja fdbserver fdbcli fdbclient  (specific targets)\n```\n\nKey cmake options:\n- `-DCMAKE_BUILD_TYPE=Debug` (default Release)\n- `-DUSE_WERROR=ON` for development\n- `-DOPEN_FOR_IDE=ON` to generate IDE project without building\n\nmacOS: add `-DBUILD_SWIFT_BINDING=OFF` if Swift issues arise.\n\nLinux with Clang: `CC=clang CXX=clang++ cmake -DUSE_LD=LLD -DUSE_LIBCXX=1 -G Ninja ..`\n\n## Testing\n\n**Unit tests** use `TEST_CASE(\"/path/to/test\")` macros (defined in `flow/include/flow/UnitTest.h`). Prefer the narrowest dedicated unit-test target instead of `fdbserver -r unittests`; these targets compile and run faster. Use `fdbclient_test` for client tests or the relevant `fdbserver_<library>_test` target defined via `add_fdbserver_unit_test` (for example, `fdbserver_core_test`, `fdbserver_kvstore_test`, or `fdbserver_storageserver_test`). Find the applicable target in the affected component's `CMakeLists.txt`:\n```bash\nninja fdbclient_test\nbin/fdbclient_test                                  # all tests in the target\nbin/fdbclient_test -f \"<test-prefix>\"                # tests matching a prefix\n\nninja fdbserver_core_test\nbin/fdbserver_core_test\n```\n\nUse `fdbserver` for simulation or broader end-to-end validation, not as the default unit-test runner.\n\n**Simulation tests** use TOML workload definitions in `tests/fast/`, `tests/slow/`, etc.:\n```bash\nbin/fdbserver -r simulation -f tests/fast/CycleTest.toml\n```\n\n**Via ctest:**\n```bash\nctest -L fast                    # all fast tests\nctest -R \"StatusDuringOutage\"    # by name pattern\n```\n\nEnable simulation tests in cmake: `-DENABLE_SIMULATION_TESTS=ON`\n\n**Adding new tests:**\n- For a new otherwise-unreferenced translation unit containing `TEST_CASE`, add a `forceLinkXxxTests()` stub and call it from `fdbserver/workloads/UnitTests.cpp`, or the linker can drop the tests.\n- Register new `.toml` or `.txt` tests under `tests/` in `tests/CMakeLists.txt` via `add_fdb_test`. `configure_testing(... ERROR_ON_ADDITIONAL_FILES)` and `verify_testing()` should catch unregistered files during CMake configuration.\n\n**Joshua**: bulk correctness/simulation testing runs on the internal Joshua cluster against optimized Release builds. Local `fdbserver -r simulation` runs are for iteration; large-scale shake-out happens on Joshua.\n\n**Trace logs**: simulation and test runs emit `trace.*.xml` (or `.json`) files in the run directory. `TraceEvent(...)` is the canonical logging mechanism — search trace files by event name when debugging a simulation failure.\n\n- A `SevError` (Severity 40) `TraceEvent` **fails** a simulation/Joshua run. Use `SevWarnAlways` for \"shouldn't happen but non-fatal.\" Errors carrying an injected fault are auto-downgraded; genuine `SevError`s are not. Expected errors must be suppressed/allowlisted, never emitted at `SevError`.\n- Keep a `TraceEvent` small: any single `detail()` value over 495 bytes is truncated (`...`), and an event whose fields total over 4000 bytes is **dropped entirely** and re-logged as `TraceEventOverflow` — at `SevError` in simulation, so an oversized event *fails the test*. Chunk long payloads (joined lists, command lines, serialized blobs) across multiple events.\n- The `Time` field is `now()`, which does not advance between `co_await`s, so spans within one run-loop slice read as zero. `g_traceBatch` events also carry `MonotonicTime` for this.\n\n## Architecture\n\nFoundationDB is a distributed ordered key-value store with strict serializability. The codebase is organized into ~12 subsystems. For background on subsystems before diving into code, the `design/` directory holds human-authored design docs and `design/AI-generated/` holds subsystem maps and per-subsystem diagrams (start with `design/AI-generated/foundationdb_subsystem_map.md`).\n\n### Concurrency Model: C++ Coroutines\n\nFDB uses cooperative single-threaded concurrency. Asynchronous functions use\nstandard C++ coroutines with `co_await` and `co_return` in ordinary `.cpp` and\n`.h` files. See `design/coroutines.md` for coroutine and cancellation semantics.\n\nKey types: `Future<T>`, `Promise<T>`, `PromiseStream<T>`, `Reference<T>` (ref-counted pointer), `Optional<T>`, `ErrorOr<T>`, `Arena` (region-based allocation).\n\n#### Common pitfalls\n\n- C++ coroutines: `co_await` is not allowed inside a `catch` handler. Capture the error, exit the catch, then `co_await` outside.\n- Errors are integer codes (`flow/include/flow/error_definitions.h`), not exceptions with messages. When you `catch (Error& e)`, re-throw `actor_cancelled` (and never silently swallow `broken_promise`) — eating cancellation causes hangs and leaks. Transaction retry goes through `tr.onError(e)`, not a bare loop.\n- `StringRef`/`KeyRef`/`ValueRef` are non-owning views into an `Arena`. Returning or storing one past its arena's lifetime is a dangling-reference bug; use `Standalone<>` (or `Key`/`Value`) when you need to own the bytes.\n- `a && b` on `Future<Void>` is `quorum(x, 2, 2)`, not a join: the first error completes it and drops the other input, cancelling it if nothing else holds it, and an already-failed `lhs` discards `rhs` unreferenced. Never accumulate with `f = f && g`; use `waitForAllReady()` to wait for everything regardless of errors.\n- `Subspace::range()` spans `prefix + '\\x00'` to `prefix + '\\xff'`, so it excludes the subspace's own key: a value written at `s.pack(x)` is not returned by scanning `s.get(x).range()`.\n\n### Core Subsystems\n\n- **`flow/`** — Async runtime, event loop, tracing, deterministic random, arenas\n- **`fdbrpc/`** — Endpoint-addressed RPC, peer management, failure monitor, Sim2 (deterministic simulation network)\n- **`fdbclient/`** — Transaction API (`NativeAPI.cpp`), read-your-writes (`ReadYourWrites.cpp`), location cache, multi-version client\n- **`fdbserver/`** — All server roles, organized by subdirectory:\n  - `clustercontroller/` — Leader election, role recruitment, ServerDBInfo broadcasting\n  - `coordinator/` — Paxos-based coordination state (generation registers)\n  - `commitproxy/`, `grvproxy/`, `resolver/` — Transaction commit pipeline\n  - `tlog/`, `logsystem/` — Durable mutation log (tag-partitioned)\n  - `storageserver/` — Serves reads, applies mutations from TLogs\n  - `kvstore/` — Pluggable storage engines (RocksDB, SQLite, memory/DiskQueue)\n  - `datadistributor/` — Shard management, team building, data movement\n  - `ratekeeper/` — Back-pressure and throttling\n  - `workloads/` — Simulation test workloads\n\n### Simulation Testing\n\n`fdbserver -r simulation` runs the entire cluster in a single process using Sim2, a deterministic simulated network. `buggify()` calls (`flow/include/flow/Buggify.h`) inject faults (delays, failures, corruption). Tests are TOML files that compose workloads. This is FDB's primary testing strategy.\n\n- Determinism is per seed *and* binary. Any code change shifts the `deterministicRandom()` sequence, so the same seed on a patched binary is a different execution, not an A/B comparison. Reproduce a failure on the unpatched binary, and judge a fix by failure rates over many seeds.\n- A continuation can resume with a different simulated process current than the one that started waiting, e.g. when a future is shared between processes on one simulated machine. Code that binds to `g_simulator->getCurrentProcess()` should capture it on entry and `co_await g_simulator->onProcess(...)` back before binding.\n\n### Knobs\n\nRuntime-tunable parameters are declared in `fdbserver/core/include/fdbserver/core/Knobs.h`, `fdbclient/include/fdbclient/Knobs.h`, and `flow/include/flow/Knobs.h` (UPPER_CASE names). Access them via `SERVER_KNOBS`, `CLIENT_KNOBS`, or `FLOW_KNOBS`. Defaults and buggification ranges live in `fdbserver/core/ServerKnobs.cpp`, `fdbclient/ClientKnobs.cpp`, and `flow/Knobs.cpp`.\n\n## Compatibility\n\nBefore changing a serialized type that persists on disk, inspect its `serializer()` and adjacent versioning or downgrade comments. Do not reorder, remove, or change encoded fields without the appropriate version gate or migration and compatibility coverage.\n\n## Naming Conventions\n\n- **Variables, functions, methods**: camelCase (`smoothTotal`, `getWorkerForRole`)\n- **Classes, structs, enums**: PascalCase (`StorageServer`, `RecoveryState`)\n- **Knobs/constants**: UPPER_CASE (`MAX_BATCH_SIZE`, `VERSIONS_PER_SECOND`)\n- **Globals**: `g_` prefix for important ones (`g_network`, `g_simulator`)\n\n## Code Formatting\n\n`clang-format` is used. Python code uses `black` and `flake8` (pre-commit hooks: `pip install pre-commit && pre-commit install`).\n\nEdit tracked `.cpp` and `.h` sources, not generated output under the build directory.\n\n## Source File Headers\n\nEvery new `.cpp` / `.h` file starts with the standard Apache 2.0 license block, with the filename on line 2 and the current year on the copyright line. Copy from any existing file in the tree (e.g. `flow/Knobs.cpp`). Add file-purpose comments *after* the license block, not in place of it.\n\n## Code Comments\n\nGeneral guidance: comments should maximize clarity. They should be clear without being verbose. They should be concise without being cryptic.\n\nScope: the guidance here mainly governs inline and implementation comments. Comments on public-facing APIs are contract statements and must be clear, precise, and complete.\n\nAudience: assume the readers of FDB are senior engineers fluent in asynchronous and concurrent programming, distributed systems, and stateful complex systems generally. Calibrate \"non-obvious\" to that reader — do not explain the language, the standard library, Flow/actor idioms, or established patterns they already know. Reserve comments for what such a reader could not quickly infer from the code itself.\n\nImplementation comments should focus on important, non-obvious fundamental rationale, motivation, and explanation of extremely subtle phenomena. Writing down important invariants — and the preconditions a piece of code requires of its caller (e.g. a lock that must be held, or a version that must already be durable) — is also a good idea.\n\nDo not write comments about mundane day-to-day development refactorings — junk like `// This used to be part of xyzFunc in Foo.cpp but we moved it here so that aaaFunc could also call it`.\n\nDo not write caller-location / \"happening elsewhere\" comments — narration of who invokes this code or where it is used (\"called from X\", \"used by Y\"). They duplicate what a search answers instantly and rot when callers change; describe what the code requires and guarantees on its own terms. Stating a real invariant is fine — \"recovery is the only path that repairs\" — naming the callers that happen to exist today is not.\n\nAfter writing or changing a bunch of code, make a pass over it one last time to review the comments. Ask yourself: am I rambling? Am I repeating what the code is showing? Am I just telling stories here? Is the information I'm writing going to be relevant and important a week from now, a month from now, a year from now? Is it genuinely essential and **non-obvious from the code itself**?\n\n## Code Review\n\nWhen reviewing, state explicitly: what the change is for, whether it is correct, bugs, omissions, better approaches, and a clear LGTM / no / not-yet. Read a construct's history (`git log -L`, `git blame`) before proposing to change something that looks odd.\n\n## Branching\n\nPRs target `main`. Release branches receive cherry-picks rather than direct PRs — don't open backport PRs without confirming first.\n\n## Design Docs\n\nBefore a design doc is considered finished (not at the start), reconcile it against `design/design-doc-template.md`.\n\n<!-- File: .github/copilot-instructions.md (Tokens: ~52 | Category: copilot-instructions) -->\nWhen reviewing PRs, address general code review criteria, but also specifically the following questions: What is it trying to do? Is it correct? Are there bugs? Are there omissions? Are there better ways of doing things? Should this CL be approved?\n</yakaai_skills>"}