{"owner":"arvidn","repo":"libtorrent","hasSkills":true,"totalSkillsCount":12,"totalTokensCount":23340,"categories":["anthropic-skill","claude-rule"],"hasMcp":false,"mcpConfig":null,"found":[".claude/rules/documentation.md","tools/vcpkg.json",".claude/rules/disk-io.md",".claude/CLAUDE.md",".claude/rules/dht.md",".claude/rules/disk-cache.md",".claude/rules/fuzzers.md",".claude/rules/python-bindings.md",".claude/rules/strong-types.md",".claude/rules/piece-picker.md",".claude/rules/protocol-encryption.md",".claude/rules/v2-torrents.md"],"skills":{".claude/rules/documentation.md":"# Documentation Generation and Spell Checking\n\nThe reference documentation is generated from the C++ public headers; the\nprose manual is hand-written reStructuredText (RST). Both are built and\nspell-checked from the `docs/` directory via `docs/makefile`, and the same\ntargets run in CI (`.github/workflows/docs.yml`).\n\n## Reference Doc Generation (`docs/gen_reference_doc.py`)\n\n`gen_reference_doc.py` parses the public headers and emits RST. It scans\n`include/libtorrent/*.hpp`, `include/libtorrent/kademlia/*.hpp`, and\n`include/libtorrent/extensions/*.hpp` (add `--internal` to also include\n`include/libtorrent/aux_/*.hpp`). For each documented class, function, enum,\nand constant it produces an RST file per category (`reference-Core.rst`,\n`reference-Session.rst`, ...) plus a table of contents (`reference.rst`) and an\noptional `single-page-ref.rst` (`--single-page`).\n\n### How comments become docs\n\n- A documentation comment is the run of `//` comment lines immediately\n **above** a declaration, with no blank line between the comment and the\n declaration. A blank line, or a line that ends a declaration (`;`, `{`,\n `}`), resets the pending comment.\n- **Comment bodies are reStructuredText.** You can use RST markup directly in\n the `//` comments: `.. code::` blocks, `.. note::`, bullet lists,\n `parsed-literal`, headings, etc. The generator renders them verbatim into\n the output RST.\n- **Symbol cross-linking** is automatic: any word in a comment that matches a\n known symbol name (class, function, enum, `settings_pack::` setting, or one\n of the pre-defined manual anchors) is turned into a hyperlink by\n `linkify_symbols()`. A trailing underscore (`foo_`) forces a link the same\n way RST reference syntax does.\n\n### Grouping functions under one comment (man-page style)\n\nMultiple overloads or related functions can share a **single** documentation\ncomment. The rule is **adjacency with no blank line between the\ndeclarations**: when a function (or field) declaration immediately follows the\nprevious one with no blank line and no intervening comment, its signature is\nappended to the previous entry's signature list rather than starting a new\nentry. So a comment above the first of a group documents the whole group, the\nway a man page documents a family of related calls together. Inserting a blank\nline between declarations splits them into separately-documented entries (and\nthe second will then warn if it has no comment of its own).\n\n### Visibility and warnings\n\n- A comment beginning with `hidden` excludes the symbol entirely. A comment\n beginning with `internal` excludes it from the public docs (but `--internal`\n includes it).\n- Every public, non-trivial member must be documented. Undocumented\n symbols print `WARNING: ... is not documented`. Trivial members\n (destructors, default/copy/move ctors, assignment, comparison/stream/\n conversion operators) are exempt via `is_trivial_member()`.\n- A `TODO:` in a public doc comment is a hard error (exits non-zero).\n- `#if` blocks gated on debug-only macros (asserts, invariant checks, and the\n other debugging macros) and on old `TORRENT_ABI_VERSION` guards are skipped\n entirely, so debug-only or deprecated members do not appear in docs. A\n non-ABI `TORRENT_*` `#if` inside a public struct warns about possible ABI\n breakage.\n- `// OVERVIEW` introduces a section overview block for a header's category.\n\n### Plain-text output for spell checking\n\nWith `--plain-output`, `add_desc()` writes a `plain_text_out.txt` containing\n**only the prose** from doc comments, with the C++ signatures and `.. code::`\nblocks stripped out. This is what the spell checker runs over.\n\n## Other Generators\n\n| Script | Output | Source |\n|--------|--------|--------|\n| `gen_reference_doc.py` | `reference-*.rst`, `reference.rst`, `single-page-ref.rst`, `plain_text_out.txt` | public headers |\n| `gen_settings_doc.py` | `settings.rst` + appends settings names to `hunspell/settings.dic` | `settings_pack.hpp` |\n| `gen_stats_doc.py` | `stats_counters.rst` | `session_stats.cpp`, `performance_counters.hpp` |\n| `gen_todo.py` | `todo.html` | `src/*.cpp`, headers |\n| `filter-rst.py` | `*-plain.txt` (prose-only) | a hand-written `.rst` file |\n\n`gen_settings_doc.py` doubles as a dictionary source: it splits every setting\nname on `_` and adds the parts to `hunspell/settings.dic`, so setting-name\nfragments are not flagged as misspellings.\n\n## Spell Checking (hunspell)\n\nSpell checking uses **hunspell**, run via `make spell-check` from `docs/`:\n\n```sh\ncd docs && make spell-check\n```\n\nThe flow:\n1. Generated reference prose is in `plain_text_out.txt` (from\n `gen_reference_doc.py --plain-output`).\n2. Hand-written manuals are converted to prose-only text with\n `filter-rst.py` (e.g. `manual.rst` -> `manual-plain.txt`). `filter-rst.py`\n strips RST directives (`.. ` lines) and indented literal/code blocks so\n only natural-language text remains.\n3. `hunspell -l` lists misspelled words from each plain-text file into\n `hunspell-report.txt`, using dictionary `hunspell/en_US` plus the project\n word list `hunspell/libtorrent.dic` (settings text uses the generated\n `hunspell/settings.dic`). HTML manuals are checked with `-H` (HTML mode).\n4. If `hunspell-report.txt` is non-empty the target **fails** and prints the\n offending words.\n\n### Fixing spell-check failures\n\nEither correct the typo in the source, or -- if the word is a real term,\nidentifier, or acronym -- add it to **`docs/hunspell/libtorrent.dic`** (one\nword per line). `hunspell/en_US.{aff,dic}` is the stock English dictionary and\nshould not be edited. `hunspell/settings.dic` is generated (en_US + the\nsettings names + a copy of `libtorrent.dic`); do not hand-edit it.\n\n## Building the HTML\n\n```sh\ncd docs && make html # all RST -> HTML via rst2html, plus figures\ncd docs && make # default target builds the docs\n```\n\nHTML is produced by `rst2html` (docutils) with `template.txt`/`style.css`;\nfigures are rendered from `.dot` (graphviz), `.diagram` (aafigure), and `.svg`\n(imagemagick `convert`). `make stage` copies the built site to `$WEB_PATH`.\n`make clean` removes generated `.rst`/`.html`/figures (it does **not** delete\nthe checked-in hand-written `.rst` manuals).\n\n## CI\n\n`.github/workflows/docs.yml` installs `python3-docutils`, `hunspell`,\n`graphviz`, `imagemagick`, `aafigure`, etc., then runs\n`make spell-check html`. A documentation warning, an undocumented public\nsymbol, a `TODO:` in a doc comment, or a spelling error all fail the build.\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `docs/gen_reference_doc.py` | parse headers -> reference RST + plain text |\n| `docs/gen_settings_doc.py` | `settings.rst` + settings dictionary words |\n| `docs/gen_stats_doc.py` | `stats_counters.rst` |\n| `docs/filter-rst.py` | strip RST markup to prose for spell checking |\n| `docs/makefile` | `html`, `rst`, `spell-check`, `stage`, `clean` targets |\n| `docs/hunspell/libtorrent.dic` | project word list (edit to whitelist terms) |\n| `docs/hunspell/en_US.{aff,dic}` | stock English dictionary (do not edit) |\n| `.github/workflows/docs.yml` | CI: spell-check + build |\n","tools/vcpkg.json":"{\n \"$schema\": \"https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json\",\n \"builtin-baseline\": \"d90a9b159c08169f39adcd1b0f1ac0ca12c4b96c\",\n \"dependencies\": [\"openssl\"],\n \"overrides\": [\n {\n \"name\": \"openssl\",\n \"version-string\": \"3.6.1\"\n }\n ]\n}",".claude/rules/disk-io.md":"---\npaths:\n - \"include/libtorrent/disk_interface.hpp\"\n - \"src/mmap_disk_io*\"\n - \"src/posix_disk_io*\"\n - \"src/pread_disk_io*\"\n - \"src/disk_job_fence*\"\n---\n\n## `disk_interface` contract\n\n`disk_interface` (`include/libtorrent/disk_interface.hpp`) is the session-level\ncustomization point for disk I/O, selected via\n`session_params::disk_io_constructor`. Built-in implementations: `mmap_disk_io`,\n`posix_disk_io`, `pread_disk_io`.\n\n**Threading.** Every `async_*` method is called from libtorrent's single network\nthread. The work may be performed asynchronously on the implementation's own\nthreads, but each completion handler **must be posted back to the network\nthread** via the `io_context` given to the constructor; handlers run on the\nnetwork thread. An implementation may use any number of threads and service\nqueued jobs concurrently and out of order, except where the contract below\nrequires ordering.\n\n**`async_hash` reflects all prior `async_write`s.** For a given piece, the\nengine posts all of the piece's blocks via `async_write`, then posts\n`async_hash` for that piece. The hash returned **must be computed over the data\nof every `async_write` for that piece that was posted before it.** This is the\nkey constraint for a multi-threaded back end that services jobs out of order: it\nmust order a piece's hash after that piece's writes and must never hash stale\nor missing data. The piece picker treats the returned hash as authoritative for\nthe data it handed over. `async_hash2` (single v2 block hash) has the same\nrequirement for the block it covers.\n\n**Fence jobs.** Several operations must run with exclusive access to a storage,\nsynchronized against all other I/O on it. These are *fence jobs*. A fence job:\n\n- waits for all jobs currently outstanding **on that storage** to finish, then\n executes -- so it runs alone, with no other operation on the storage in\n flight;\n- *fences* everything posted after it: any job posted to the same storage while\n the fence is raised is queued and not executed until the fence job completes.\n\nFences are per-storage -- a fence on one storage does not block jobs on another.\nThe fence operations are:\n\n- `async_clear_piece` -- synchronize with all outstanding I/O to the piece\n before the engine re-requests its blocks\n- `async_move_storage`\n- `async_release_files`\n- `async_stop_torrent`\n- `async_rename_file`\n- `async_delete_files`\n- `async_set_file_priority`\n- `async_check_files`\n\n`async_read`, `async_write`, `async_hash`, and `async_hash2` are ordinary\n(non-fence) jobs. The built-in back ends implement the fence semantics with\n`disk_job_fence` (`src/disk_job_fence.cpp`). A storage's outstanding-job count\nis what the fence waits to reach zero before it runs; a job stays counted from\nthe moment it is accepted until its completion is posted back.\n\n**Every ordinary job must honor the fence -- including a buffered write.** Each\njob, with no exceptions, has to participate in the outstanding-job count for its\nwhole lifetime, so the fence cannot run while any work on the storage is still\nin flight. This is easy to overlook for `async_write` on a write-back back end:\neven though the write is inserted into a cache and its handler is deferred, it\nmust be counted as outstanding from the moment it is accepted until the flush\nthat retires it has finished and its completion is posted -- *including the\nwindow while a disk thread is mid-flush writing it out*. It must not take a fast\npath that skips the count. If a buffered write (or the thread flushing it) is\ninvisible to the fence, the fence can be raised and complete while that write is\nstill being written to disk, and a teardown fence (`async_stop_torrent`,\n`async_release_files`, ...) can then tear the storage down underneath the\nin-flight flush -- a use-after-free. So the rule is symmetric: a write counts\nexactly like a read or a hash.\n\n### Fences and deferred (write-back) writes\n\nThe fence contract above is straightforward for an implementation that runs\neach job to completion immediately. An implementation that *defers* writes --\nreporting an `async_write` complete before the data is durably on disk, as a\nwrite-back cache does -- has to uphold two further invariants. Both are easy to\nget wrong, and a violation of either is a deadlock or silent data corruption,\nnot a degraded-performance bug.\n\n**A fence must not wait on writes that nothing will flush (forward progress).**\nA fence waits for every job outstanding on the storage to complete before it\nruns. When a write is only reported complete once it is durably stored, that\nwrite stays outstanding -- and keeps the fence waiting -- the whole time it sits\nbuffered. The flush that would retire it normally happens when the piece\ncompletes (its `async_hash`) or under memory pressure; but the completing hash\nis itself an ordinary job, now queued *behind* the very fence that is waiting\nfor it, and memory pressure may never arrive. The invariant: **raising a fence\nmust guarantee the storage's buffered writes get flushed, rather than waiting\npassively for a flush trigger the fence is itself blocking.** Otherwise the\nfence and its own backlog deadlock.\n\n**Resuming a fenced backlog must preserve posting order against new jobs.** When\na fence completes, the jobs that queued behind it are resumed. If resuming a job\nis observable -- a buffered write becomes visible, a read is served -- then a job\nthe network thread posts *during* that resume must be ordered after the whole\nresumed backlog, never interleaved ahead of it. The invariant: **a job arriving\nwhile the backlog is being resumed must not bypass the backlog.** Violate it and\na freshly-posted `async_read` overtakes a still-queued `async_write` for the\nsame block and returns stale data (or a hole).\n\n**A read resumed from a fence must reflect writes resumed ahead of it.** A\ncorollary of the ordering invariant: a read unblocked from the backlog may be\npreceded in that same backlog by a write to the very block it reads (both were\nblocked by the fence). That read must observe the write -- i.e. it must be\nserved from the buffered data, not from disk where the superseding write has not\nyet landed. Put plainly: the reads and writes a fence releases must take effect\nin the order they were posted, exactly as if the fence had never been raised.\n\n**Back-pressure.** `async_write` returns `true` when the write queue is over the\nhigh watermark; the caller then stops issuing writes for that peer until the\n`disk_observer` passed in is notified that the queue has drained. `false` means\nthe caller may keep writing.\n\n**Batching.** `async_*` calls only enqueue work; `submit_jobs()` is called once\nafter a batch to wake the disk thread(s). An implementation may also wake on each\n`async_*`, but should tolerate the batched notification.\n",".claude/CLAUDE.md":"# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Overview\n\nlibtorrent-rasterbar is a C++17 BitTorrent library (version 2.1.0). It supports v1 and v2 torrents, DHT, WebTorrent (WebRTC), and optional I2P/SSL.\n\n## Build Systems\n\nThe project has **two build systems**: boost-build (b2/bjam) and CMake. **Boost-build is the preferred build system** per the contributing guidelines.\n\n### Boost-build (b2) - preferred\n\n**Never add `-j` (parallel jobs) flags to b2 commands.** b2 manages parallelism on its own.\n\nBuild the library (from repo root):\n```sh\nb2\n```\n\nRun unit tests:\n```sh\ncd test && b2\n```\n\nRun a single test:\n```sh\ncd test && b2 test_session\n```\n\nRun deterministic tests (no flaky networking):\n```sh\ncd test && b2 deterministic-tests\n```\n\nRun simulations:\n```sh\ncd simulation && b2\n```\n\nBuild with developer options (typical for development):\n```sh\ncd test && b2 asserts=on invariant-checks=full debug-iterators=on picker-debugging=on\n```\n\nBuild with sanitizers:\n```sh\ncd test && b2 address-sanitizer=norecover undefined-sanitizer=norecover asserts=on\n```\n\nBuild examples, tools, fuzzers, python bindings:\n```sh\ncd examples && b2\ncd tools && b2\ncd fuzzers && b2 clang\ncd bindings/python && b2\n```\n\n### CMake — alternative\n\n```sh\ncmake -B build -DCMAKE_BUILD_TYPE=Debug -Dbuild_tests=ON\ncmake --build build\ncd build && ctest\n```\n\nRun a single test via CMake:\n```sh\ncd build && ./test/test_session\n```\n\n## Pre-commit\n\nThe repo uses [pre-commit](https://pre-commit.com). Before pushing:\n```sh\npip install pre-commit\npre-commit install\npre-commit run --all-files\n```\n\nPre-commit hooks include trailing whitespace, YAML/TOML/XML checks, RST formatting, Python formatting (black, isort, flake8, mypy, autoflake), and auto-generation of:\n- `include/libtorrent/fwd.hpp` and `include/libtorrent/libtorrent.hpp` via `tools/gen_fwd.py` and `tools/gen_convenience_header.py`\n- C binding headers (`bindings/c/include/libtorrent_settings.h`, `libtorrent_alerts.h`) via `bindings/c/tools/gen_header.py` and `bindings/c/tools/gen_alert_header.py`\n\nThe C++ format check (`git-clang-format-18`) is the only hook that requires a\nsystem install -- `apt install clang-format-18` (or platform equivalent). It\nformats only the lines that differ from HEAD, not whole files; the codebase is\nbeing migrated to the `.clang-format` style incrementally rather than via a\nflag-day reformat.\n\n## Key Architecture\n\n### Threading Model\n\nThree thread categories:\n1. **Main network thread** — all sockets, session/torrent/peer state, boost.asio event loop\n2. **Disk I/O thread(s)** — reads, writes, and SHA-1/SHA-256 piece verification (count controlled by `settings_pack::aio_threads`)\n3. **Resolver thread** — spawned by boost.asio for async DNS on platforms without native async getaddrinfo\n\nAll interaction with session/torrent state from outside must go through the network thread (via `session_handle` and `torrent_handle`).\n\n### Core Classes\n\n- **`session`** (`include/libtorrent/session.hpp`) — public API, pimpl over `session_impl`\n- **`session_impl`** (`include/libtorrent/aux_/session_impl.hpp`) — all session state: torrent list, connection list, global rate limits, DHT state, port mapping\n- **`torrent`** (`include/libtorrent/aux_/torrent.hpp`, `src/torrent.cpp`) — all state for a single swarm: piece picker, peer connections, file storage\n- **`torrent_handle`** (`include/libtorrent/torrent_handle.hpp`) — public pimpl handle, weak reference to `torrent`; sends messages to network thread\n- **`peer_connection`** / **`bt_peer_connection`** — BitTorrent protocol implementation\n- **`piece_picker`** — download strategy (which blocks to request from which peers). See `.claude/rules/piece-picker.md` for a detailed description.\n- **`peer_list`** — list of known (not necessarily connected) peers for a swarm\n\n### Disk I/O\n\nThree disk backends:\n- `mmap_disk_io` — mmap-based (default on 64-bit when mmap is available)\n- `posix_disk_io` — fallback single-threaded POSIX I/O (used on 32-bit or without mmap)\n- `pread_disk_io` — multi-threaded backend using `pread()`/`pwrite()`. See `.claude/rules/disk-cache.md` for a detailed description.\n\n### DHT (Kademlia)\n\nSource in `src/kademlia/`, headers in `include/libtorrent/kademlia/`. Includes ed25519 signing (`src/ed25519/`).\n\n### Extensions\n\nProtocol extensions live in `src/` and `include/libtorrent/extensions/`:\n- `ut_metadata` — magnet link metadata exchange\n- `ut_pex` — peer exchange\n- `smart_ban` — ban peers that send corrupt data\n- `i2p_pex` — peer exchange for I2P peers\n\n### WebTorrent (WebRTC)\n\nOptional feature controlled by `webtorrent=on` / `-Dwebtorrent=ON`. Uses `deps/libdatachannel` and the `rtc_stream`/`rtc_signaling` subsystem.\n\n### Python Bindings\n\nSource in `bindings/python/src/*.cpp`, built as `libtorrent.so` using boost.python. See `.claude/rules/python-bindings.md` for detailed conventions.\n\n### Simulations\n\n`simulation/` contains network simulation tests that use `libsimulator` (a deterministic virtual network). These test high-level behaviors (swarms, DHT, session management) without real network access. All simulation tests run with `invariant-checks=full`, `asserts=on`, `debug-iterators=on`.\n\n### Terminology\n\n- **piece** — SHA-1/SHA-256 verified chunk of torrent data (typically power-of-two size)\n- **block** — 16 KiB sub-unit of a piece (the transfer unit in the protocol)\n- **torrent_peer** — a known-but-not-connected peer entry\n- **peer_connection** — an active connection to a peer\n\n## Key Build Options (b2 features)\n\n| Feature | Values |\n|---------|--------|\n| `crypto` | `built-in`, `openssl`, `openssl-shared`, `wolfssl`, `gcrypt`, `gnutls`, `libcrypto` |\n| `asserts` | `on`, `off`, `production`, `system` |\n| `invariant-checks` | `off`, `on`, `full` |\n| `logging` | `on`, `off` |\n| `deprecated-functions` | `on`, `off`, `1`, `2`, `3`, `4` |\n| `webtorrent` | `on`, `off` |\n| `address-sanitizer` | `norecover`, `recover`, `off` |\n| `undefined-sanitizer` | `norecover`, `recover`, `off` |\n| `thread-sanitizer` | `norecover`, `recover`, `off` |\n| `picker-debugging` | `on`, `off` |\n| `debug-iterators` | `default`, `off`, `on`, `harden` |\n\n`deprecated-functions` selects the ABI/API version via `TORRENT_ABI_VERSION`. Different values are **link-incompatible**. Higher numbers expose a more modern interface and remove older deprecated APIs:\n\n| Value | `TORRENT_ABI_VERSION` | Corresponds to |\n|-------|-----------------------|----------------|\n| `on` | (oldest supported) | libtorrent 1.x |\n| `1` | 1 | libtorrent 1.x |\n| `2` | 2 | libtorrent 2.0 |\n| `3` | 3 | libtorrent 2.x |\n| `4` | 4 | libtorrent 2.1 |\n| `off` | 100 (newest) | latest API |\n\n## Directory Layout\n\n```\nsrc/ main library source\nsrc/kademlia/ DHT implementation\nsrc/ed25519/ ed25519 signing (vendored)\ninclude/libtorrent/ public API headers\ninclude/libtorrent/aux_/ internal headers (not public API)\ninclude/libtorrent/kademlia/ DHT headers\ninclude/libtorrent/extensions/ extension headers\ntest/ unit tests (test_*.cpp)\nsimulation/ network simulation tests (test_*.cpp)\nfuzzers/src/ fuzz targets\nexamples/ example programs\ntools/ utility scripts and programs\nbindings/python/ Python bindings (boost.python)\nbindings/c/ C API bindings\ndeps/ vendored dependencies\n deps/try_signal/ signal handling\n deps/asio-gnutls/ GnuTLS adapter for asio\n deps/json/ nlohmann/json (for WebTorrent)\n deps/libdatachannel/ WebRTC (for WebTorrent)\n```\n\n## Adding New Source Files\n\nWhen adding a new `.cpp` or `.hpp` file, it must be added to **all three** build systems:\n1. `Jamfile` (boost-build)\n2. `CMakeLists.txt`\n3. `Makefile` (if applicable)\n\n## Code-Generation Tools\n\nAfter modifying public API headers, regenerate the derived headers by running these scripts from the repo root:\n\n- `tools/gen_fwd.py` — regenerates `include/libtorrent/fwd.hpp` (forward declarations for all public types)\n- `tools/gen_convenience_header.py` — regenerates `include/libtorrent/libtorrent.hpp` (the convenience header that includes all public headers)\n\nThese are also run automatically by the pre-commit hooks.\n\n## Coding Conventions\n\n- Comments must use ASCII characters only (no Unicode, smart quotes, em-dashes, etc.)\n- Use a single space after a period in comments (not two spaces)\n- Declare variables `const` whenever they are not reassigned after initialization\n- C++17 throughout; no C++20 features yet\n- The code must build with standard C++ and must not require compiler language extensions; optional extensions (e.g. `__builtin_*`, platform-specific pragmas) may be used when guarded by feature detection\n- `namespace lt = libtorrent` alias is always available\n- Asserts: use `TORRENT_ASSERT(cond)` (active when `TORRENT_USE_ASSERTS` is defined, i.e. debug builds)\n- Invariant checks: expensive checks inside `#if TORRENT_USE_INVARIANT_CHECKS`\n- Public API headers use `TORRENT_EXPORT` macro; internal symbols use hidden visibility\n- Internal functions and classes use `TORRENT_EXPORT_EXTRA` macro; to grant access to tests\n- ABI versioning via `TORRENT_VERSION_NAMESPACE_2/3/4` inline namespace macros (defined in `include/libtorrent/aux_/export.hpp`); `_2` = v1.2, `_3` = v2, `_4` = v2.1\n- Warnings are treated as errors in CI (both gcc and clang)\n- Changes to ABI (fields/ordering of public classes) must target `master`, not `RC_*` stable branches\n- `TORRENT_EXPORT_EXTRA` marks a symbol exported only for test access; it is private/internal, not public API, and is exempt from the ABI-stability rules above (safe to change on `RC_*` branches). Do not treat `TORRENT_EXPORT_EXTRA` as indicating public or ABI-sensitive status; `TORRENT_EXPORT` is the public one\n- `settings_pack` enum values must be appended at the end of each enum group (int, bool, string) — never inserted in the middle — to avoid changing the numeric values of existing settings and breaking ABI\n- prefer the C++ counterparts to C headers\n- do not use using-statements in header files\n- prefer to fully qualify standard types and functions\n- prefer the short lt namespace alias when fully qualifying libtorrent types\n- prefer using default member initializers over initializer list\n\n### Strong Types\n\nAvoid raw `int` for indices and flags; use `aux::strong_typedef` (`include/libtorrent/units.hpp`) and `flags::bitfield_flag` (`include/libtorrent/flags.hpp`). See `.claude/rules/strong-types.md` for details.\n",".claude/rules/dht.md":"---\npaths:\n - \"src/kademlia/**\"\n - \"include/libtorrent/kademlia/**\"\n---\n\n# DHT (Kademlia) Implementation\n\nSource: `src/kademlia/`, `include/libtorrent/kademlia/`\n\n## BEP Support\n\n| BEP | Description | Status |\n|-----|-------------|--------|\n| BEP 5 | DHT Protocol (ping, find_node, get_peers, announce_peer) | Full |\n| BEP 32 | IPv6 extension (dual-stack, nodes6) | Full |\n| BEP 33 | DHT Scrape (bloom filters BFpe/BFsd) | Full |\n| BEP 42 | Node ID Verification (IP-derived IDs) | Full |\n| BEP 44 | Arbitrary data storage (immutable + mutable items, Ed25519) | Full |\n| BEP 51 | DHT Infohash Indexing (sample_infohashes) | Full |\n\n---\n\n## File Overview\n\n| File | Purpose |\n|------|---------|\n| `node.cpp` / `node.hpp` | Central DHT node — routing, dispatch, traversal lifecycle |\n| `dht_tracker.cpp` / `dht_tracker.hpp` | Session-level coordinator, one `node` per listen socket |\n| `routing_table.cpp` / `routing_table.hpp` | K-bucket routing table |\n| `rpc_manager.cpp` / `rpc_manager.hpp` | Tracks outstanding RPCs, calls observers on response/timeout |\n| `observer.hpp` | Base observer class for pending RPC state |\n| `traversal_algorithm.cpp` / `.hpp` | Base iterative search algorithm |\n| `find_data.cpp` / `.hpp` | Find-nodes traversal, base for get_peers and get_item |\n| `get_peers.cpp` / `.hpp` | BEP 5 peer lookup traversal |\n| `get_item.cpp` / `.hpp` | BEP 44 item retrieval traversal |\n| `put_data.cpp` / `.hpp` | BEP 44 item storage traversal |\n| `sample_infohashes.cpp` / `.hpp` | BEP 51 traversal |\n| `refresh.cpp` / `.hpp` | Bootstrap / bucket-refresh traversal |\n| `direct_request.hpp` | Single-message direct request |\n| `node_id.cpp` / `node_id.hpp` | `node_id` type (alias for `sha1_hash`), XOR distance, BEP 42 generation |\n| `node_entry.cpp` / `node_entry.hpp` | Single routing-table entry |\n| `item.cpp` / `item.hpp` | Mutable/immutable item type |\n| `types.hpp` | `public_key`, `secret_key`, `signature`, `sequence_number` |\n| `msg.hpp` | Incoming message structure |\n| `dht_storage.cpp` / `dht_storage.hpp` | Pure-virtual storage interface + default RAM implementation |\n| `dht_state.cpp` / `dht_state.hpp` | Persistent state (node IDs, bootstrap nodes) |\n| `dht_settings.hpp` | Per-DHT settings struct |\n| `dht_observer.hpp` | Callbacks from DHT into `session_impl` |\n| `dos_blocker.cpp` / `dos_blocker.hpp` | Per-IP rate limiting (5 pkt/s, 5-min block) |\n| `ed25519.hpp` | Ed25519 sign/verify API (vendored impl in `src/ed25519/`) |\n| `announce_flags.hpp` | Announce flags: `seed`, `implied_port`, `ssl_torrent` |\n\n---\n\n## Threading\n\nAll DHT code runs on the **session network thread** (boost.asio). No locking needed inside DHT classes. `rpc_manager` uses a mutex only for its observer-pool allocator.\n\n---\n\n## Core Data Structures\n\n### `node_id`\nType alias for `sha1_hash` (160 bits).\n\nKey functions:\n- `distance(a, b)` — XOR metric\n- `distance_exp(a, b)` — index of highest differing bit (0–160)\n- `compare_ref(a, b, ref)` — compare distances to reference\n- `generate_id(external_ip)` — BEP 42 derivation\n- `verify_id(nid, source_ip)` — BEP 42 check\n\n### `node_entry`\nOne routing-table peer entry:\n- `node_id id`, `udp::endpoint endpoint`\n- `uint16_t rtt` (0xffff = unknown)\n- `uint8_t timeout_count` (0 = confirmed, 0xff = never pinged)\n- `bool verified` (BEP 42)\n- `time_point last_queried`\n\n### `routing_table`\n160 k-buckets (one per XOR distance bit). Extended routing table (default on) gives larger buckets close to us:\n\n| Bucket index | Multiplier | Max live nodes (k=20) |\n|---|---|---|\n| 0 | 16× | 320 |\n| 1 | 8× | 160 |\n| 2 | 4× | 80 |\n| 3 | 2× | 40 |\n| 4–159 | 1× | 20 (k) |\n\nKey behaviours:\n- Last (closest) bucket splits when it exceeds k entries\n- Each bucket has a **replacement cache**; candidates promoted when live node is removed\n- `ip_set` restricts one node per IP per bucket (BEP 42, `restrict_routing_ips`)\n- Periodic refresh: random query in each bucket's range every ~15 min\n\n### `item` (BEP 44)\n- **Immutable** — key is `sha1(value)`, no signature\n- **Mutable** — key is `sha1(salt || pk)`, Ed25519-signed, has `sequence_number`\n- CAS: `cas` field lets clients do compare-and-swap on sequence number\n\n### `dht_storage_interface`\nPure-virtual; default RAM implementation (`dht_default_storage_constructor`):\n\n```\nmap m_torrents (BEP 5 peers)\nmap m_immutable_items\nmap m_mutable_items\n```\n\nEviction uses distance from our node IDs — items far from us are evicted first.\nLimits: `max_torrents` (2000), `max_peers` (500), `max_dht_items` (700).\nPeer entries expire after 30 min (announce_interval).\n\n---\n\n## Key Classes\n\n### `node`\nOne per listen socket. Owns `routing_table` and `rpc_manager`.\n\nImportant methods:\n```cpp\nvoid bootstrap(endpoints, done_cb);\nvoid get_peers(info_hash, callback, flags);\nvoid announce(info_hash, port, flags, callback);\nvoid get_item(target, callback); // immutable\nvoid get_item(pk, salt, callback); // mutable\nvoid put_item(data, callback); // immutable\nvoid put_item(pk, sk, salt, callback); // mutable\nvoid sample_infohashes(endpoint, target, callback);\nvoid incoming(udp::endpoint, msg); // dispatch received UDP\nvoid tick(); // token rotation, bucket refresh\n```\n\nWrite tokens: `hash(requester_ip + secret + info_hash)`. Two secrets kept; rotated periodically to allow stale tokens for one rotation period.\n\n### `dht_tracker`\nOwns `std::map` (one `node` per socket).\nAggregates results across all nodes. Manages DOS blocking. Saves/restores `dht_state`.\n\n### `rpc_manager`\n- `invoke(entry, endpoint, observer_ptr)` — sends bencoded RPC, tracks by 2-byte transaction ID\n- `incoming(msg, &sender_id)` — looks up observer by txn-id, calls `observer::reply()`\n- `tick()` — fires `observer::timeout()` / `observer::short_timeout()` for stale requests\n- Uses an object pool for observers to reduce heap fragmentation\n\n### `observer`\nOne per pending RPC. Flags:\n\n| Flag | Meaning |\n|------|---------|\n| `flag_queried` | Request sent |\n| `flag_initial` | First attempt |\n| `flag_short_timeout` | Intermediate timeout fired |\n| `flag_failed` | Node failed |\n| `flag_alive` | Response received |\n| `flag_done` | Finished, no longer tracked |\n\nObserver hierarchy:\n```\nobserver\n ├─ traversal_observer (parses \"nodes\"/\"nodes6\" from responses)\n │ ├─ find_data_observer\n │ │ ├─ get_peers_observer (also parses \"values\", bloom filters)\n │ │ └─ get_item_observer (parses v/k/sig/seq)\n │ ├─ put_data_observer\n │ └─ sample_infohashes_observer\n ├─ announce_observer\n ├─ direct_observer\n └─ null_observer\n```\n\n---\n\n## Traversal Algorithms\n\nAll searches inherit from `traversal_algorithm`. The base implements the Kademlia iterative lookup:\n\n1. Seed result set with k closest nodes from routing table\n2. Issue α (default 3) concurrent requests (`add_requests()`)\n3. Each response: add new closer nodes, sort result set\n4. Repeat until k closest nodes all responded or timeout\n5. Call `done()`\n\n| Class | BEP | RPC sent | Callback delivers |\n|-------|-----|----------|-------------------|\n| `find_data` | 5 | `find_node` | closest nodes + write tokens |\n| `get_peers` | 5 | `get_peers` | `vector` (peers) |\n| `get_item` | 44 | `get` | `item`, `bool authoritative` |\n| `put_data` | 44 | first `get`, then `put` | response count |\n| `sample_infohashes` | 51 | `sample_infohashes` | samples, interval, total |\n| `bootstrap` / `refresh` | 5 | `get_peers` (our own ID) | (fires bucket refresh) |\n| `direct_traversal` | — | arbitrary | single `msg` |\n\n`obfuscated_get_peers`: wraps `get_peers`, uses a random target until convergence, then switches to the real info_hash (BEP 5 privacy mode, controlled by `dht_privacy_lookups`).\n\n---\n\n## Message Format (bencode)\n\nAll messages are bencoded dicts with mandatory keys `\"t\"` (transaction id), `\"y\"` (`\"q\"/\"r\"/\"e\"`), optional `\"v\"` (version), `\"ip\"` (external address, BEP 42), `\"ro\"` (read-only, BEP 5).\n\n**Queries** (`y=\"q\"`): `\"q\"` = method name, `\"a\"` = argument dict with `\"id\"` (our node id, 20 bytes).\n\n**Responses** (`y=\"r\"`): `\"r\"` = response dict.\n\n**Errors** (`y=\"e\"`): `\"e\"` = `[error_code, \"message\"]`.\n\n### Method summary\n\n| Method | Key request fields | Key response fields |\n|--------|--------------------|---------------------|\n| `ping` | `id` | `id` |\n| `find_node` | `id`, `target`, `want` | `id`, `nodes`, `nodes6` |\n| `get_peers` | `id`, `info_hash`, `noseed`, `scrape`, `want` | `id`, `token`, `values`\\|`nodes`, `BFpe`, `BFsd` |\n| `announce_peer` | `id`, `info_hash`, `port`, `token`, `implied_port`, `seed` | `id` |\n| `get` (BEP 44) | `id`, `target`, [`seq`] | `id`, `v`, [`k`, `sig`, `seq`] |\n| `put` (BEP 44) | `id`, `v`, [`k`, `sig`, `seq`, `salt`, `cas`] | `id` |\n| `sample_infohashes` | `id`, `target` | `id`, `samples`, `num`, `interval`, `nodes` |\n\nCompact node encoding: 26 bytes per IPv4 node (20-byte id + 4-byte IP + 2-byte port), 38 bytes for IPv6.\n\n---\n\n## Bootstrap\n\n1. Load `dht_state` (saved node IDs + endpoints)\n2. Assign node ID: saved → BEP 42 from external IP → random\n3. Add router nodes from settings (`router.bittorrent.com:6881`, etc.)\n4. Run `bootstrap` traversal against saved/configured endpoints (queries `get_peers` for own ID)\n5. On completion, refresh all non-contiguous buckets\n6. `dht_observer::get_peers()` notifies `session_impl` to query for all active torrents\n\n---\n\n## Integration with `session_impl`\n\n`session_impl` owns `shared_ptr m_dht` and implements `dht_observer`:\n\n| Observer method | When called |\n|-----------------|-------------|\n| `set_external_address()` | DHT detects our external IP via `\"ip\"` field |\n| `get_listen_port(socket)` | Node asks for our announce port |\n| `get_peers(info_hash, peer)` | Another node announced a peer to us |\n| `announce(info_hash, endpoint)` | We received a get_peers for one of our torrents |\n| `on_dht_request(method, msg, reply)` | Custom extension message handler |\n\nSession calls:\n- `dht_tracker::incoming_packet(socket, endpoint, buffer)` — for each received DHT UDP packet\n- `dht_tracker::new_socket(handle)` / `delete_socket(handle)` — listen port changes\n- `dht_tracker::tick()` — periodic maintenance\n\nRelevant `settings_pack` keys:\n\n| Setting | Effect |\n|---------|--------|\n| `enable_dht` | Toggle DHT |\n| `dht_search_branching` | Alpha (concurrent requests) |\n| `dht_max_peers_reply` | Peers returned per get_peers response |\n| `dht_enforce_node_id` | Reject nodes failing BEP 42 check |\n| `dht_restrict_routing_ips` | One node per /24 IPv4 per bucket |\n| `dht_restrict_search_ips` | Sybil mitigation in traversals |\n| `dht_privacy_lookups` | Enable obfuscated_get_peers |\n| `dht_read_only` | Set `ro=1` flag, don't accept queries |\n| `dht_upload_rate_limit` | Response bytes/sec cap |\n| `dht_item_lifetime` | BEP 44 item expiry (0 = never) |\n| `dht_sample_infohashes_interval` | BEP 51 cache TTL |\n\n---\n\n## Constants and Tunables\n\n| Parameter | Value | Notes |\n|-----------|-------|-------|\n| Node ID | 160 bits | SHA-1 |\n| k (bucket size) | 20 | Configurable |\n| Alpha | 3 | Concurrent requests per traversal |\n| RPC full timeout | 15 s | |\n| RPC short timeout | ~6 s | Triggers next request early |\n| Token rotation | ~7 min | Two tokens kept |\n| Bucket refresh | 15 min | Per-bucket, measured from last query |\n| Peer announce validity | 30 min | |\n| DOS block rate | 5 pkt/s | Per source IP |\n| DOS block duration | 5 min | |\n| DOS tracker size | 20 | Most-recent offenders |\n",".claude/rules/disk-cache.md":"---\npaths:\n - \"src/disk_cache*\"\n - \"src/pread_disk_io*\"\n - \"include/libtorrent/aux_/disk_cache*\"\n---\n\n## Disk Cache (`pread_disk_io`)\n\nThe disk cache (`src/disk_cache.cpp`, `include/libtorrent/aux_/disk_cache.hpp`) is used exclusively by `pread_disk_io`. It is a write-back cache for incoming blocks, deferred until pieces are complete or memory pressure forces a flush.\n\n**Data structures:**\n\n- `disk_cache` — the cache itself; holds a `boost::multi_index_container` of `cached_piece_entry` objects, protected by a single `std::mutex`. The container has five indices: by `piece_location` (ordered), by `cheap_to_flush()` (ordered descending), by `need_force_flush()` (ordered descending), by `piece_location` (hashed, for fast lookup), and by `needs_hasher_kick()` (ordered descending).\n- `cached_piece_entry` — one entry per in-flight piece. Contains an array of `cached_block_entry` (one per block), an optional `piece_hasher` (`ph`, v1 only), an optional `block_hashes` array (`sha256_hash[]`, v2 only), and the cursors/flags described below.\n- `cached_block_entry::write_state` — `std::variant` with three mutually exclusive states:\n - `std::monostate` — no data present (the block has never been written, or its data has been fully released).\n - `disk_job*` — a pending write job is queued. The buffer is owned by the job; `take_write_job()` consumes the job during flushing.\n - `disk_buffer_ref` — the write has been flushed. The buffer may be non-null (`has_buf()`, \"held alive\" for future hashing) or null (already released). `is_flushed()` is true in either case and contributes to `flushed_cursor`.\n\n**Cursors:**\n\n- `hasher_cursor` — index of the first block not yet hashed (v1 SHA-1 only; hashing is always contiguous from block 0). Owned by the hashing thread while `hashing_flag` is set.\n- `flushed_cursor` — index of the first block not yet confirmed flushed to disk. Updated under the mutex after each flush completes.\n- `cheap_to_flush()` returns `hasher_cursor - flushed_cursor`: blocks that have been hashed but not yet flushed can be flushed without ever needing read-back.\n\n**Flags on `cached_piece_entry`:**\n\nFlags are stored in `cached_piece_flags flags` (a `bitfield_flag` bitfield). The mutex must be held to read or modify flags; updates go through `view.modify()` (required by `boost::multi_index`). Once a flag is set, the owning thread may access the protected state **without the mutex**:\n\n- `hashing_flag` — set under the mutex by the thread about to hash blocks. While set, that thread has exclusive access to `ph` (the v1 `piece_hasher`) and may advance `hasher_cursor`. No other thread may touch these. Cleared under the mutex when hashing is complete.\n- `flushing_flag` — set under the mutex by the thread about to write blocks to disk. While set, that thread has exclusive access to the block buffers being flushed. No other thread will attempt to flush the same piece. Cleared under the mutex when flushing is complete.\n- `force_flush_flag` — set when the piece has all blocks populated; prioritises this piece for flushing. Cleared once all blocks are flushed.\n- `piece_hash_returned_flag` — set once the piece hash has been computed and returned to the BitTorrent engine. The hash job is also the only signal that a piece is complete: it is what allows `flush_to_disk` to evict the `cached_piece_entry`, and what gets a piece force-flushed when the v1 hasher could not reach the end on its own (pure-v2 pieces, or pieces partially on disk at startup). When `disable_hash_checks` is set the engine still posts the job; the setting is latched into the storage (and into `torrent`) when the storage is created, at add time or when metadata arrives for a magnet link, so a later change never applies to it. In that mode `insert_write` caches the piece as neither v1 nor v2, so nothing is hashed on write, and `do_job(hash)` runs `hash_piece()` with a no-op callback purely for that bookkeeping and returns zero hashes.\n- `v1_hashes_flag` — set if this piece requires v1 SHA-1 hashing (`ph` is allocated).\n- `v2_hashes_flag` — set if this piece requires v2 SHA-256 block hashing (`block_hashes` is allocated).\n- `needs_hasher_kick_flag` — set when a block is inserted and the hasher thread should be woken; cleared when the hasher picks it up. Coalesces multiple insertions into one wakeup.\n- `notify_flushed_flag` — set by a `flush_storage()` caller that is waiting on a piece another thread is mid-flush on; `flush_piece_impl`'s `scope_end` checks it and signals `m_flushing_cv`. Only a wakeup signal supporting a single waiter, not a pin — the waiter re-looks-up the piece after each wakeup because it may have been flushed, hashed, or evicted while it slept.\n\nIn both hashing and flushing cases, the lock is **released** for the duration of the actual I/O (hashing or `pwrite()`), then re-acquired to update cursors/flags and potentially free buffers.\n\n**Deferred clear:** If `try_clear_piece()` is called while `flushing_flag` or `hashing_flag` is set, the clear job is stored in `cached_piece_entry::clear_piece` and executed by the owning thread once it clears its flag.\n\n**Held-alive buffers and the flush/hasher race:**\n\nWhen `flush_piece_impl` flushes a block, it checks `needed_by_hasher = (hashing_flag && block_index >= hasher_cursor)`. If a hasher is mid-snapshot (the `kick_hasher` window between snapshot and re-lock), the pointer to the buffer is in `blocks_storage[]` and must stay live. In that case the block transitions to `disk_buffer_ref{buf}` (non-null), and `m_blocks` is *not* decremented. Otherwise the buffer is freed immediately and `m_blocks` is decremented.\n\n`m_blocks` therefore counts both pending writes and held-alive buffers — both forms of memory pressure. `back_pressure` and `flush_request()` are driven off `m_blocks`.\n\nThe held-alive buffer is reclaimed when `kick_hasher` next advances the cursor past it (its cleanup loop frees blocks in `[old_cursor, new_cursor)` via `take_buf`). However, if v1 SHA-1 is blocked by a missing earlier block, the cursor cannot advance and the buffer stays held until either the missing block arrives or the piece is cleared. The \"drop held-alive\" flush pass (below) handles the case where no further inserts will arrive — without it, back-pressure can pin the cache above the low watermark indefinitely (the back-pressure observer is only notified once `m_blocks` drops to the low watermark, so the producer that tripped back-pressure would never be woken up). Note that v2 hashes for held-alive blocks are typically already in `block_hashes[]` (computed by the kick that triggered the keep-alive), so freeing the buffer only loses the v1 read-back optimization, not v2 hash work.\n\n**Flush strategy (`flush_to_disk`), in order:**\n\n1. Force-flush pass (`view`) — flush pieces with `force_flush` set (piece fully downloaded and hashed); ordered by `need_force_flush()`.\n2. Cheap flush pass (`view2`) — flush blocks in the range `[flushed_cursor, hasher_cursor)` (already hashed, no future read-back needed); pieces ordered by `cheap_to_flush()` descending.\n3. Drop held-alive pass (`view3`) — for each piece without `flushing_flag` or `hashing_flag`, free all blocks where `has_buf()` (held-alive `disk_buffer_ref{buf}`). No I/O — the data is already on disk. Done before the expensive pass because it's free; only skipped if the target is already met. Loses the v1 read-back optimization for these blocks; a subsequent `async_hash` will read them from disk.\n4. Expensive flush pass (`view4`) — flush any dirty blocks even if they haven't been hashed yet, requiring a read-back later to complete the piece hash. Used only when the cache exceeds the high-water mark.\n\nPasses 2-4 run only when `optimistic=false`. The optimistic flush at the top of each disk thread's loop runs only pass 1.\n\n**`flush_storage()`:** Called as a fence job during torrent teardown (`release_files`, `delete_files`, etc.) to flush every cached piece of one storage to disk. For each piece it flushes any still-dirty blocks (completing their write jobs via `flush_piece_impl`). It does **not** free or erase the piece — eviction is left to `flush_to_disk`. Keeping the piece preserves its partial hash state and lets another thread erase it concurrently without racing the flush. If a piece is mid-flush on another thread (`flushing_flag` set), it sets `notify_flushed_flag`, waits on `m_flushing_cv`, and re-looks-up the piece on each wakeup (it may be flushed, hashed, or evicted while waiting); if it is gone, it moves on. A concurrent `flush_storage()` already waiting on a piece (`notify_flushed_flag` already set) is left to that waiter, since the flag supports only one waiter.\n\n**Fences and the cache (`pread_disk_io`):** the general fence contract is in `.claude/rules/disk-io.md`; this is how `pread_disk_io` honours it with a write-back cache.\n\n- **Writes during a fence are queued, not aborted.** `async_write` calls `is_blocked()`, which takes ownership of the job and parks it in the fence's blocked queue. When the fence lowers, `disk_job_fence::job_complete` calls the `repost` callback (still under the fence mutex) once per blocked job in posting order; `add_completed_jobs_impl` inserts each reposted write into the cache there, and serves reposted reads from the cache (a normal disk read would return stale data, since the superseding write is still buffered). Because the resume runs under the fence mutex, a job posted concurrently is ordered behind the whole backlog. Work that can re-enter the fence (hasher kick, flush, completions) is deferred to after the mutex is released.\n- **`async_read` never consults the cache; `add_job` is the single funnel.** `async_read` only allocates a `job::read` (carrying the request verbatim, including an unaligned, block-spanning start) and hands it to `add_job`. `add_job` runs `is_blocked()` first, so a read posted while a fence is up is parked like any other job and cannot bypass the fence to serve stale data ahead of a queued write. The cache is consulted exactly once, by `prepare_read()`, and only after the job is past the fence gate: in `add_job` when the fence is down, or in the `repost` callback when the fence lowers. Both call `prepare_read()`; reposted jobs must **not** re-enter `is_blocked()` (a stacked fence still raises `m_has_fence`, so it would wrongly re-park them behind a later fence), which is why the consult is shared *after* the gate rather than by routing reposts back through `add_job`. `prepare_read()` mutates the job in place, so the single outstanding-job accounting `is_blocked()`/`job_complete()` set up is preserved.\n- **Every write is counted; writes during a fence are queued, not aborted.** `async_write` always routes through `is_blocked()` (never a fast path around it), so the write counts as an outstanding job on the storage whether or not a fence is up -- a fence then waits for buffered writes and their in-flight flush, exactly like a read or hash (see `.claude/rules/disk-io.md`). With no fence up, `is_blocked()` marks the job in-progress and increments the count, and the write is inserted into the cache inline; it stays counted until it is flushed and completed (or aborted). With a fence up, `is_blocked()` instead takes ownership of the job and parks it in the fence's blocked queue. When the fence lowers, `disk_job_fence::job_complete` calls the `repost` callback (still under the fence mutex) once per blocked job in posting order; `add_completed_jobs_impl` inserts each reposted write into the cache there, and serves reposted reads from the cache (a normal disk read would return stale data, since the superseding write is still buffered). Because the resume runs under the fence mutex, a job posted concurrently is ordered behind the whole backlog. Work that can re-enter the fence (hasher kick, flush, completions) is deferred to after the mutex is released.\n- **Reads racing the unblock.** `async_read`'s lock-free cache probe may run just before the backlog's writes land. `add_job` re-checks the cache once the fence is observed down, so a read is never dispatched to disk for a block whose write is sitting buffered.\n- **Forward progress (`m_fence_flush`).** A fence waits for the storage's outstanding jobs (including buffered writes) to reach zero, but nothing else flushes a partial piece under the watermark — so a raised fence hands its storage to a generic thread to flush (`flush_fenced_storages`), draining the writes it is waiting on. `add_fence_job` does this when a fence is raised over outstanding work (`fence_post_none`).\n- **Stacked fences.** When a fence lowers and reposts writes that sit ahead of a still-raised second fence, those partial-piece writes are now in the cache with no flush trigger. `add_completed_jobs_impl` detects a completed fence whose storage still `has_fence()` and re-arms `m_fence_flush` for it, so the next fence's wait can also make progress. (With no generic threads it flushes inline.)\n- **Unaligned reads and `job::partial_read`.** A peer read request whose start is not block-aligned can span two blocks. `prepare_read()` probes both with `disk_cache::get2`: if both blocks are in the cache it serves the whole request; if neither is, the job stays a `job::read` and `do_job(read)` reads the whole span from disk; if exactly one block is in the cache, it copies that side into the buffer and rewrites the job in place into a `job::partial_read` whose `offset`/`buffer_size`/`buffer_offset` describe just the missing side. `do_job(partial_read)` then reads only that side from disk into the rest of the buffer. Because `prepare_read()` runs past the fence gate against the *current* cache, the cached side it copies is never stale, and a `partial_read` therefore reads only from disk and never re-consults the cache. `prepare_read()` is the only place `partial_read` jobs are created -- they are never posted directly.\n",".claude/rules/fuzzers.md":"---\npaths:\n - \"fuzzers/**\"\n - \"tools/gen_corpus.py\"\n---\n\n## Fuzzers\n\n### Build\n\nFuzzers require clang and are built separately from the main library:\n\n```sh\ncd fuzzers && b2 clang\n```\n\nBuilt with ASAN + UBSAN (norecover), debug symbols, asserts on, debug\niterators (`_GLIBCXX_DEBUG`), and WebTorrent enabled (per the default-build\nin `fuzzers/Jamfile`; note that `invariant-checks` is *not* turned on). Each\ntarget produces a standalone binary under `fuzzers/bin//` (e.g.\n`fuzzers/bin/clang-linux-20/...`). The explicit `stage` / `stage-large`\ninstall targets copy the binaries into the `fuzzers/fuzzers/` subdirectory\n(the `fuzzers` in `fuzzers/Jamfile` is relative to the Jamfile's\nown directory).\n\nTwo optional build features (defined in `fuzzers/Jamfile`):\n\n- `debug-logging=on` -- defines `DEBUG_LOGGING=1`, enabling extra alert\n logging in the session-level fuzzers (see peer_conn below).\n- `variant=build_coverage` -- a fast replay build with sanitizers, asserts,\n and invariant checks all off. Combine with `test-coverage=on` to measure\n code coverage from an existing corpus.\n\nThe default-build also pins `1` (`TORRENT_ABI_VERSION=1`,\nthe oldest/most inclusive ABI -- see the table in the top-level `CLAUDE.md`),\nso every fuzzer target, not just `bdecode_entry` (below), compiles and fuzzes\nthe full set of deprecated APIs alongside the current ones. `TORRENT_ABI_VERSION`\nguards in the codebase are exact-match (`== 1`) or lower-bound (`>= N`) checks,\nnever upper-bound, so ABI 1 is a strict superset of what any higher ABI value\ncompiles; this is safe project-wide.\n\n### Running a fuzzer\n\nRun a specific fuzzer against a corpus (single job):\n\n```sh\n./fuzzers/bin/.../peer_conn fuzzers/corpus/peer_conn/\n```\n\nUseful libFuzzer flags:\n\n| Flag | Purpose |\n|------|---------|\n| `-max_len=N` | Cap input size (default: 4096) |\n| `-timeout=N` | Seconds before a single run is declared a timeout (default: 1200) |\n| `-max_total_time=N` | Stop after N seconds total |\n| `-artifact_prefix=dir/` | Write crash/timeout inputs to this directory |\n\n### Running in parallel\n\nlibFuzzer supports in-process parallelism via `-jobs` and `-workers`:\n\n```sh\n./fuzzers/bin/.../peer_conn fuzzers/corpus/peer_conn/ \\\n -jobs=8 -workers=8\n```\n\n- `-jobs=N` — total number of fuzzing jobs to complete before the driver exits\n- `-workers=N` — number of concurrent fuzzer processes (defaults to\n `min(jobs, num_cpus/2)` when omitted)\n\nEach worker writes its own log to `fuzz-N.log` in the current directory.\nCrashes are written to the current directory by default; use\n`-artifact_prefix=crashes/` to redirect them.\n\nAfter parallel runs, merge all worker corpora back into the main corpus to\ndeduplicate and minimize:\n\n```sh\n./fuzzers/bin/.../peer_conn -merge=1 \\\n fuzzers/corpus/peer_conn/ fuzzers/corpus/peer_conn/\n```\n\n(Passing the same directory as both source and destination is safe; libFuzzer\nreads all inputs first, then writes the minimized set back.)\n\n### bdecode_node.cpp vs bdecode_entry.cpp\n\n`bdecode_node.cpp` fuzzes the modern `lt::bdecode(span, ...)`\nfrom `bdecode.hpp`; `bdecode_entry.cpp` fuzzes the deprecated,\n`TORRENT_ABI_VERSION == 1`-only `lt::bdecode()` overloads in\n`bencode.hpp`. They share no code path. `bdecode_entry` degrades to a\nno-op stub if built at any other ABI, rather than failing to compile.\n\n### Structure\n\nEvery fuzzer in `fuzzers/src/` implements `LLVMFuzzerTestOneInput`; the\nsession-level fuzzers additionally implement `LLVMFuzzerInitialize`:\n\n- `LLVMFuzzerTestOneInput(data, size)` — implemented by every fuzzer;\n called for every fuzz input; must be stateless with respect to prior\n inputs. Returns 0 on success, -1 to signal libFuzzer that the input was\n uninteresting (used for timeouts, not errors).\n\n- `LLVMFuzzerInitialize(argc, argv)` — implemented only by the\n session-level fuzzers (`add_torrent`, `pe_conn`, `peer_conn`,\n `piece_layers`, `rtc_peer_conn`, `ut_metadata`, `ut_pex`, `web_seed`).\n Called once at startup to set up global state (sessions, sockets,\n servers) that persists across all test inputs. The many pure-function\n fuzzers (e.g. `bdecode_node`, `parse_magnet_uri`, `gzip`,\n `torrent_info`) have no global state and omit it.\n\n### peer_conn.cpp\n\nConnects to a live libtorrent session as a BitTorrent peer and sends\nfuzz-controlled protocol messages.\n\nThe session and torrent setup is shared via `peer_session.hpp`\n(`configure_fuzz_session()`, `make_fuzz_torrent_params()`, and the\n`peer_fuzz_session` struct `g_fz`), and reused by the rtc_peer_conn,\nut_metadata, and ut_pex fuzzers.\n\n**Session setup (`LLVMFuzzerInitialize`):**\n\n- Creates a session on `127.0.0.1:0` with all outbound connections and\n ancillary services (outgoing TCP/uTP, DHT, LSD, UPnP, NAT-PMP, IP\n notifier) disabled, encryption disabled, and all peer timeouts set to\n 1 second.\n- Adds a **hybrid (v1 + v2)** torrent built from four files (a 0-byte\n file, a 100-byte file, a 50 KiB file, and a 97 MiB + 200 KiB file).\n After canonicalization (alphabetical sort + inserted pad files) this is\n exactly 100 pieces of 1 MiB. Piece hashes (SHA-1) and per-file block\n hashes (SHA-256) are filled with dummy values. `save_path` is `.` and\n no data is on disk, so the torrent stays in downloading state.\n- Waits for both the TCP `listen_succeeded_alert` (recording\n `g_fz.listen_port`) and the `torrent_resumed_alert` before returning.\n- Records `g_fz.info_hash` (an `info_hash_t` holding both the v1 SHA-1\n and v2 SHA-256 info-hashes).\n- Also adds a **second** torrent as a **magnet** (`info_hashes` only, no\n `ti`), built via the shared `add_fuzz_magnet(false, 'a')` helper (which\n wraps `build_fuzz_torrent()` from `peer_session.hpp`) from the identical\n file layout but with no v2 (SHA-256) hashes, so it's v1-only. Its\n info-hash and pre-encoded ut_metadata piece message (from\n `torrent_info::info_section()`) are stored in\n `g_v1_only_hash`/`g_v1_only_metadata_piece`. Because this metadata is\n fixed at startup rather than derived from fuzz input, delivering it via\n ut_metadata (see below) always succeeds, unlike a corpus byte sequence,\n which would have to find a SHA-1 preimage to ever make\n `torrent::set_metadata()` return true. Without this second torrent, the\n whole torrent-acquires-metadata code path (`set_metadata()`,\n `initialize_merkle_trees()`, `on_metadata_impl()`) would be unreachable:\n the first, pre-built torrent always has valid metadata, so that code\n never runs for it.\n- Also adds a **third** torrent, added as a **hybrid (v1 + v2) magnet**\n (`info_hashes` holds both the v1 and v2 hash, no `ti`), built via the\n same `add_fuzz_magnet(true, 'c')` helper from the identical file layout\n with a different dummy-hash fill byte (`'c'` instead of the default\n `'a'`) so it gets its own info-hash instead of colliding with `g_fz`'s.\n Its info-hash and pre-encoded ut_metadata piece message are stored in\n `g_v2_hybrid_hash`/`g_v2_hybrid_metadata_piece`. This is the counterpart\n to the v1-only magnet: metadata resolves to a genuine hybrid, so a peer's\n speculative v2 claim turns out honest and must survive metadata\n resolution rather than being corrected, exercising the same code as\n the v1-only magnet from the opposite direction, to guard against the\n fix over-triggering on legitimate v2 peers. More variants (a different\n file layout, etc.) can be added the same way if a future case needs\n one.\n\n**Per-input flow (`LLVMFuzzerTestOneInput`):**\n\n1. Opens a TCP connection to `127.0.0.1:g_fz.listen_port`.\n2. Sends a 68-byte BT handshake:\n - `data[0..7]` are used as the extension-flags (reserved) bytes.\n - Bits for BEP 10 (byte 5, bit 0x10) and BEP 6 FAST (byte 7, bit\n 0x04) are always forced on so those handlers are always reachable.\n - Byte 0 bit 0x01 selects the peer's protocol version, for torrents\n that have a v1 and a v2 hash to choose between (torrent selector 0\n and 2 below): clear = v2 (advertises the torrent's\n `info_hashes().get_best()`, `protocol_v2` true), set = v1\n (advertises the torrent's `info_hashes().v1`, `protocol_v2` false).\n - Byte 0 bits 0x02 and 0x04 form a 2-bit torrent selector: 0 = the\n first, pre-built torrent (metadata already valid, bit 0x01 applies\n as above); 1 = the second, v1-only magnet (always presents\n `g_v1_only_hash.v1`; bit 0x01 doesn't apply since there's no v2\n hash to choose between); 2 = the third, hybrid magnet (bit 0x01\n applies exactly as it does for selector 0); 3 = unused, falls back\n to selector 0. Every other flag bit, including byte 7 bit 0x10 (the\n v2-upgrade bit), is independent of the torrent selector and free to\n combine with any of them, e.g. a handshake claim that doesn't match\n what the connected torrent's info-hash actually supports.\n - Peer-ID is all zeros.\n3. Sends a fixed BEP 10 extended handshake (`k_extended_handshake`,\n ext_id 0) registering:\n\n | Extension | ext_id |\n |-------------|--------|\n | ut_pex | 1 |\n | ut_metadata | 2 |\n | upload_only | 3 |\n\n4. If the torrent selector picked the v1-only or hybrid magnet, sends two\n more fixed (not fuzz-derived) messages before continuing: a second\n extended handshake (BEP 10 supports re-negotiation) announcing\n `metadata_size`, which makes the ut_metadata plugin queue a request\n for piece 0, then a ut_metadata piece message carrying that magnet's\n metadata (`g_v1_only_metadata_piece` or `g_v2_hybrid_metadata_piece`),\n so `torrent::set_metadata()` succeeds deterministically on every run\n that picks either branch.\n5. Parses `data[8..]` as a sequence of messages (see **Corpus wire\n format** below) and sends each with a correct 4-byte BT length\n prefix. This is unchanged regardless of which torrent step 2 picked,\n so the same fuzzed message space (including BEP 78 hash messages)\n now also runs against a torrent whose metadata was just acquired\n rather than always pre-loaded.\n6. Closes the socket and waits for a `peer_error_alert` or\n `peer_disconnected_alert` (3-second timeout).\n\n**DEBUG_LOGGING:** defined by `fuzzers/Jamfile` via the `debug-logging`\nfeature (default `off` -> `DEBUG_LOGGING=0`). Build with\n`b2 clang debug-logging=on` to set it to 1, which adds `peer_log` alerts to\nthe alert mask and makes `wait_for_disconnect` print the last 10 interesting\nalerts (connect and routine peer-log noise excluded) before each disconnect\n-- useful for tracing which message paths were hit.\n\n### Corpus wire format (peer_conn)\n\nEach corpus file is binary-structured:\n\n```\n[8 bytes : extension flags]\n[repeated until EOF:\n 1 byte : msg_type\n 2 bytes : payload_len (big-endian, max 65535)\n N bytes : payload (N = min(payload_len, bytes_remaining))\n]\n```\n\nMinimum file size: 11 bytes (8 flag bytes + 3-byte message header).\n\nFor **msg_type 20** (BEP 10 extended protocol), the first byte of the\npayload is the extended message ID and the rest is the extension body.\nThe fuzzer splits this automatically before calling\n`send_extended_message`.\n\n**Extension flag byte layout** (8 reserved bytes, big-endian):\n\n| Byte | Bit | Protocol |\n|------|------|---------------------|\n| 0 | 0x01 | protocol version, when the selected torrent has a v1 and a v2 hash (set = v1 SHA-1, clear = v2) |\n| 0 | 0x02 + 0x04 | 2-bit torrent selector: 0 = pre-built hybrid, 1 = v1-only magnet, 2 = hybrid magnet, 3 = unused (falls back to 0) |\n| 5 | 0x10 | BEP 10 extended |\n| 7 | 0x04 | BEP 6 FAST |\n| 7 | 0x01 | BEP 5 DHT |\n| 7 | 0x10 | v2-upgrade bit (peer claims v2 support) |\n\n**libtorrent's fixed incoming ext_ids** (from `bt_peer_connection.hpp`\nenum; these are the IDs libtorrent expects in messages FROM us):\n\n| ext_id | Handler | Notes |\n|--------|------------------|--------------------------------------------|\n| 0 | extended HS | re-negotiation supported |\n| 1 | ut_pex plugin | as registered in `k_extended_handshake` |\n| 2 | ut_metadata | as registered in `k_extended_handshake` |\n| 3 | upload_only | `upload_only_msg` constant |\n| 4 | holepunch | `holepunch_msg`; requires `m_holepunch_id` |\n| 7 | dont_have | `dont_have_msg`; 4-byte piece index |\n| 8 | share_mode | `share_mode_msg` |\n| other | disconnect | `errors::invalid_message` |\n\nHolepunch (ext_id 4) silently returns if `m_holepunch_id == 0`. To\nreach the holepunch handler body, a corpus file must first send an\nadditional extended handshake (ext_id 0) that includes\n`\"ut_holepunch\": ` in its `m` dict.\n\n**BEP 78 hash messages** (21 = hash_request, 22 = hashes, 23 =\nhash_reject) require the peer's `protocol_v2` flag (set from the\nhandshake) and, separately, the connected torrent's info-hash to actually\nhave a v2 component, since a peer can set the flag on a torrent that never\ngains one (see the v1-only magnet above), so both are checked. Either\nfailing disconnects with `errors::invalid_message`.\n\n### Corpus generation tool\n\n`tools/gen_corpus.py` generates seed corpora for all session-level fuzzers.\nRun it from the repo root:\n\n```sh\npython3 tools/gen_corpus.py\n```\n\nThis writes into:\n- `fuzzers/corpus/peer_conn/`\n- `fuzzers/corpus/natpmp/`\n- `fuzzers/corpus/udp_tracker/`\n- `fuzzers/corpus/upnp/`\n- `fuzzers/corpus/pe_crypto_state/`\n- `fuzzers/corpus/pe_conn/`\n- `fuzzers/corpus/ut_metadata/`\n- `fuzzers/corpus/ut_pex/`\n- `fuzzers/corpus/web_seed/`\n- `fuzzers/corpus/utp_stream/`\n- `fuzzers/corpus/piece_layers/`\n- `fuzzers/corpus/torrent_info/`\n- `fuzzers/corpus/lsd/`\n- `fuzzers/corpus/add_torrent/`\n\n**Sync requirement:** the Python constants `EXT_UT_PEX`,\n`EXT_UT_METADATA`, `EXT_UPLOAD_ONLY`, and `FUZZER_EXT_HANDSHAKE` in\nthe tool must match `k_extended_handshake` in `peer_conn.cpp`. If the\nC++ extended handshake is changed, update both files together.\n\nThe tool also uses `NUM_PIECES = 100`, `PIECE_SIZE = 1 MiB`, and\n`BLOCK_SIZE = 16 KiB` -- these must match the torrent that\n`make_fuzz_torrent_params()` (equivalently `build_fuzz_torrent(true)`) in\n`fuzzers/src/peer_session.hpp` builds. `build_fuzz_torrent(false)` builds\nthe second, v1-only torrent, and `build_fuzz_torrent(true, 'c')` builds\nthe third, hybrid-magnet torrent, both from the identical file layout, so\nthese constants apply to them too.\n\nA handful of seeds set byte 0 bits 0x02/0x04 to connect to the second or\nthird torrent instead of the first, combined with other flag bits (e.g.\nthe v2-upgrade bit) and a follow-up message. These exist to give the\nfuzzer a foothold in those branches, not to exhaustively cover them;\ncoverage-guided mutation explores the rest.\n\n### pe_conn.cpp\n\nConnects to a live libtorrent session and drives the full PE (protocol\nencryption / MSE) handshake as the initiating (client) side. Requires\nthe session to have `in_enc_policy = pe_enabled` and\n`allowed_enc_level = pe_both`, which pe_conn's `LLVMFuzzerInitialize`\nsets.\n\nThe fuzzer performs a real DH key exchange (so the server can locate the\ntorrent), then uses fuzz-controlled bytes to drive all encrypted fields.\nEvery server-side error path is reachable by setting the appropriate bit\nin the flags byte.\n\n**Session setup (`LLVMFuzzerInitialize`):**\n\n- Same hybrid v1+v2 torrent as peer_conn (100 pieces, 1 MiB each).\n- `in_enc_policy = pe_enabled`, `allowed_enc_level = pe_both`.\n- All outbound services disabled (same as peer_conn).\n\n**Per-input flow (`LLVMFuzzerTestOneInput`):**\n\n1. Connect TCP socket to session.\n2. Generate a fresh DH key pair; send DH public key + PadA (from fuzz bytes).\n3. Read the server's DH public key (96 bytes, blocking).\n4. Compute shared secret, RC4 keys, sync hash, and obfuscated SKEY hash.\n5. Build the PE payload with fuzz-controlled fields and optional corruptions.\n6. Send the payload, close socket, wait for disconnect.\n\n**Corpus wire format:**\n\n```\nByte 0 : pad_a_size (PadA bytes before sync hash, 0-255)\nByte 1 : flags\n bit 0 : corrupt_sync_hash -> sync_hash_not_found\n bit 1 : corrupt_skey -> invalid_info_hash\n bit 2 : corrupt_vc -> invalid_encryption_constant\n bits 3-4 : crypto_provide\n 00 -> pe_both (0x03)\n 01 -> pe_plaintext (0x01)\n 10 -> pe_rc4 (0x02)\n 11 -> 0x00 (invalid) -> unsupported_encryption_mode\nByte 2 : len_pad_c * 2 (encrypted padding, 0-510 bytes)\nByte 3 : len_ia (IA size 0-255; server rejects > 68)\nBytes 4+ : first pad_a_size bytes = PadA content\n then up to len_ia bytes = IA (BT handshake portion)\n```\n\nSee `.claude/rules/protocol-encryption.md` for details of the PE protocol.\n",".claude/rules/python-bindings.md":"---\npaths:\n - \"bindings/python/**\"\n---\n\n## Python Bindings\n\nSource in `bindings/python/src/*.cpp`; one `.cpp` file per exposed subsystem (e.g. `session.cpp`, `torrent_handle.cpp`, `alert.cpp`). Built as a Python extension module (`libtorrent.so`) using boost.python.\n\n**Building:**\n```sh\ncd bindings/python && b2\n```\nCMake also builds them when `-Dpython-bindings=ON` is passed.\n\n**Testing:**\n\nTests live in `bindings/python/tests/*_test.py` and are run with pytest:\n```sh\ncd bindings/python && python -m pytest tests/\n```\nThere is a roughly one-to-one correspondence between `src/foo.cpp` and `tests/foo_test.py`. Tests must be isolated — no real network access, no persistent filesystem side-effects. Use `get_isolated_settings()` from `tests/lib.py` when constructing a session, and `tempfile.TemporaryDirectory()` for any disk I/O. See `bindings/python/tests/guidelines.txt` for the full testing conventions.\n\n**Type stubs:**\n\nThe type stub file is `bindings/python/libtorrent/__init__.pyi`. It was originally bootstrapped with `stubgen` but is now **maintained entirely by hand** — do not regenerate it with stubgen, as that would lose the carefully written annotations. When adding a new binding, add the corresponding type annotation to this file manually. The header comment in the file documents the conventions used (e.g. how boost.python enums, positional-only args, and the `_BoostBaseClass` metaclass are represented).\n\n**Design philosophy — classes over dicts:**\n\nPrefer exposing real C++ classes via boost.python rather than converting them to plain Python dicts. Earlier versions of the bindings returned plain `dict` objects for things like peer info and torrent status; these have been replaced with proper bound classes (`peer_info`, `torrent_status`, etc.) that mirror the C++ types. When adding new bindings, follow this pattern: expose the C++ struct/class directly rather than unpacking it into a dict.\n\nNote: boost.python strictly requires `dict` and `list` (not duck-typed `Mapping`/`Sequence` abstractions) for function arguments that cross the C++ boundary. This is a boost.python limitation and not a design choice; the type stubs reflect this with `dict` and `list` rather than `Mapping` or `Sequence`.\n",".claude/rules/strong-types.md":"---\npaths:\n - \"include/libtorrent/units.hpp\"\n - \"include/libtorrent/flags.hpp\"\n - \"include/libtorrent/index_range.hpp\"\n - \"include/libtorrent/**\"\n - \"src/**\"\n---\n\n## Strong Types\n\nThe codebase avoids using raw `int` or bare integers for indices and flags. Instead, use the strong type and flag facilities.\n\n**Index types** (`include/libtorrent/units.hpp`):\n\n`aux::strong_typedef` wraps an integer so it is incompatible with other integers and other strong types. Arithmetic with raw integers is not allowed; only arithmetic with the same type or its `diff_type` is permitted.\n\nTo define a new index type:\n```cpp\nusing my_index_t = aux::strong_typedef;\n```\n\nTo convert to/from the underlying integer, use an explicit cast:\n```cpp\npiece_index_t p{42};\nint raw = static_cast(p);\n```\n\nTo iterate over a range of indices, use `index_range` (`include/libtorrent/index_range.hpp`):\n```cpp\nfor (piece_index_t i : index_range{begin, end}) { ... }\n```\n\n**Flag types** (`include/libtorrent/flags.hpp`):\n\n`flags::bitfield_flag` wraps an unsigned integer as a type-safe bitfield. Flags of different types cannot be combined. Individual bit constants are defined using the `_bit` user-defined literal:\n\n```cpp\nusing torrent_flags_t = flags::bitfield_flag;\nstatic constexpr torrent_flags_t seed_mode = 0_bit;\nstatic constexpr torrent_flags_t upload_mode = 1_bit;\n```\n\nTo define a new flag type:\n```cpp\nusing my_flags_t = flags::bitfield_flag;\n```\n\nFlag values support `|`, `&`, `^`, `~`, `|=`, `&=`, `^=`, and `operator bool()` (true if any bit is set). To test a flag:\n```cpp\nif (flags & torrent_flags::seed_mode) { ... }\n```\n",".claude/rules/piece-picker.md":"---\npaths:\n - \"src/piece_picker.cpp\"\n - \"include/libtorrent/aux_/piece_picker.hpp\"\n - \"test/test_piece_picker.cpp\"\n---\n\n## Piece Picker\n\nThe piece picker (`src/piece_picker.cpp`, `include/libtorrent/aux_/piece_picker.hpp`) decides\nwhich blocks to request from which peers. It tracks availability (how many peers have each piece),\nper-piece priority, and the state of every block in every partially-downloaded piece.\n\n### Core data structures\n\n**`piece_pos`** — 8 bytes per piece, tightly packed:\n- `peer_count` (26 bits) — number of peers that have this piece\n- `download_state` (3 bits) — one of the `download_queue_t` constants below\n- `piece_priority` (3 bits) — 0 = filtered (dont-download), 1–7 user priority (4 = default)\n- `index` (`prio_index_t`) — position in `m_pieces`, or `we_have_index` (-1) if we have it\n\n**`m_piece_map`** — `aux::vector`, one entry per piece.\n\n**`m_pieces`** — flat `aux::vector` of all pickable (not filtered,\nnot have) pieces, sorted by effective priority. Within each priority band pieces are in random\norder (rarest-first is approximated by the priority formula, not by sorting on availability alone).\n\n**`m_priority_boundaries`** — cumulative end-indices into `m_pieces`. Priority band `p` spans\n`[m_priority_boundaries[p-1], m_priority_boundaries[p])`. Priority 0 starts at index 0.\n\n**`m_downloads`** — `aux::array` of 4 sorted `vector` (one per download\ncategory), indexed by `download_queue_t`:\n- `piece_downloading` (0) — some blocks still open (not yet requested)\n- `piece_full` (1) — all blocks requested, at least one not yet writing/finished\n- `piece_finished` (2) — all blocks writing or finished\n- `piece_zero_prio` (3) — downloading but priority = 0\n\n`piece_downloading_reverse` and `piece_full_reverse` are pseudo-states stored in `download_state`\nto de-prioritize pieces only requested from reverse peers; they map to the same download vectors\nas their non-reverse counterparts.\n\n**`downloading_piece`** — one per in-flight piece:\n- `info_idx` (`uint16_t`) — index into `m_block_info`; the slice `[info_idx * blocks_per_piece, (info_idx+1) * blocks_per_piece)` holds the block states for this piece\n- `finished`, `writing`, `requested` (15 bits each) — block state counters\n- `passed_hash_check` (1 bit) — hash job returned OK (piece may not yet be on disk)\n- `locked` (1 bit) — blocks cannot be picked; set during error recovery, cleared by `restore_piece()`\n- `hashing` (1 bit) — outstanding hash request in flight\n\n**`m_block_info`** — flat `aux::vector`. Each downloading piece owns\n`blocks_per_piece()` consecutive entries identified by `info_idx`. Free ranges are recycled\nvia `m_free_block_infos` (a free-list of `uint16_t` indices).\n\n**`block_info`** — per block:\n- `peer` — last peer to request/write this block\n- `num_peers` (14 bits) — peers with this block outstanding\n- `state` (2 bits) — `state_none → state_requested → state_writing → state_finished`\n\n### Priority formula\n\n`piece_pos::priority(picker)` returns -1 (not pickable) if: filtered, have, no availability\n(`peer_count + m_seeds == 0`), `piece_full`, or `piece_finished`.\n\nOtherwise:\n```\npriority = availability * (priority_levels - piece_priority) * prio_factor + adjustment\n```\nwhere `priority_levels = 8`, `prio_factor = 3`, and:\n- `adjustment = -3` for actively downloading pieces (non-reverse)\n- `adjustment = -2` for open pieces\n- `adjustment = -1` for reverse-downloading pieces\n\nLower value = picked first. The `prio_factor = 3` creates three sub-levels per availability\ntier, letting downloading pieces beat open ones at the same availability.\n\n### Seeds optimization\n\nPeers with all pieces are tracked in `m_seeds` rather than incrementing every `peer_count`.\nTrue availability of a piece is `peer_count + m_seeds`. When a seed sends DONT_HAVE,\n`break_one_seed()` decrements `m_seeds` and increments every `piece_pos::peer_count`.\n\n### Lazy rebuild (`m_dirty`)\n\n`m_pieces` and `m_priority_boundaries` are rebuilt lazily. When `m_dirty` is true, they are\nstale; `update_pieces()` (called automatically before any pick) rebuilds them:\n1. Walk `m_piece_map`, count pieces per priority band → `m_priority_boundaries` holds deltas\n2. Make boundaries cumulative, resize `m_pieces`\n3. Fill `m_pieces` using per-piece relative offsets stored in `piece_pos::index`\n4. Shuffle each priority band randomly\n5. Fix up `piece_pos::index` to point back into the final positions in `m_pieces`\n\nBulk operations (e.g. `inc_refcount(bitfield)` touching many pieces) set `m_dirty = true`\nrather than updating incrementally. For small bitfield changes (< 50 pieces), the picker\nupdates incrementally via `update()` / `add()` / `remove()` without dirtying.\n\n### Sequential download cursors\n\n`m_cursor` — lowest piece index not yet had.\n`m_reverse_cursor` — one past the highest piece index not yet had (all pieces from here to end\nare had). `set_sequential_range(first, last)` constrains the cursor window.\n\n### Piece lifecycle\n\n```\nopen → downloading → full → finished → (removed, piece_pos::have set)\n```\n\n- `mark_as_downloading(block, peer)` — block: none→requested; piece may transition open→downloading\n- `mark_as_writing(block, peer)` — block: requested→writing\n- `mark_as_finished(block, peer)` — block: writing→finished; calls `update_piece_state()` which\n may advance the piece from downloading→full→finished\n- `piece_passed(index)` — hash check passed; sets `passed_hash_check`, calls `account_have()`,\n calls `piece_flushed()` if all blocks are also finished\n- `piece_flushed(index)` — piece is on disk; removes from `m_downloads`, removes from `m_pieces`\n via `remove()`, sets `piece_pos::index = we_have_index`\n- `restore_piece(index, blocks)` — after hash failure or write error; clears `locked`, optionally\n resets specific blocks back to `state_none`, re-opens the piece for downloading\n\n### `pick_pieces` options\n\n`picker_options_t` flags passed to `pick_pieces()`:\n- `rarest_first` — pick lowest-availability pieces (the default; driven by the priority formula)\n- `reverse` — pick most-common first, or last pieces if sequential\n- `sequential` — pick within the `[m_cursor, m_reverse_cursor)` window in order\n- `on_parole` — only pick pieces exclusively requested by this peer (after suspected bad data)\n- `prioritize_partials` — prefer pieces already in `m_downloads` over fresh ones\n- `piece_extent_affinity` — create affinity for 4 MiB extents of adjacent pieces\n- `align_expanded_pieces` — align contiguous-block expansion to natural boundaries\n\nAuto partial prioritization: if `num_partials > num_peers * 3 / 2` or\n`num_partials * blocks_per_piece > 2048`, `prioritize_partials` is forced on and\n`prefer_contiguous_blocks` is set to 0 to avoid unbounded partial piece growth.\n\n`prefer_contiguous_blocks` (int, not a flag) — request this many consecutive blocks from the\nsame piece; used by web peers to pipeline larger requests.\n",".claude/rules/protocol-encryption.md":"# Protocol Encryption (MSE/PE) Handshake\n\nlibtorrent implements Message Stream Encryption (MSE), also called Protocol\nEncryption (PE). The implementation lives in `src/bt_peer_connection.cpp` and\n`src/pe_crypto.cpp` (crypto primitives in `include/libtorrent/aux_/pe_crypto.hpp`).\n\n## Overview\n\nMSE provides obfuscation (not strong security) against deep-packet inspection.\nIt uses a Diffie-Hellman key exchange to derive RC4 stream keys and wraps the\nBitTorrent handshake. The torrent info hash is obfuscated in the handshake so\nthe connection cannot be trivially identified.\n\nAfter the handshake, the connection body is RC4-encrypted only if\n`crypto_select` ended on `pe_rc4`. If `pe_plaintext` was selected, only the\nhandshake is obfuscated and post-handshake traffic is sent in the clear. This\nis tracked by the `m_rc4_encrypted` flag on `bt_peer_connection` and gates the\nencrypt/decrypt path in every send and receive routine.\n\n## Settings\n\n| Setting | Meaning |\n|---------|---------|\n| `pe_forced` | Always use PE (outgoing) / require PE (incoming) |\n| `pe_enabled` | Use PE if peer supports it, fall back to plaintext |\n| `pe_disabled` | Never use PE |\n\n`settings_pack::out_enc_policy` controls outgoing connections.\n`settings_pack::in_enc_policy` controls acceptance of incoming connections.\n`settings_pack::allowed_enc_level` is a bitmask (`pe_plaintext=1`, `pe_rc4=2`, `pe_both=3`).\n`settings_pack::prefer_rc4` selects RC4 over plaintext when both are offered.\n\n## Handshake States\n\nThe state machine lives in `bt_peer_connection::on_receive_impl()`. The\nhandshake differs between the initiating side (outgoing) and the accepting\nside (incoming).\n\n### Incoming (server) path\n\n1. **`read_pe_dhkey`** (96 bytes) -- read client's DH public key, respond\n with our DH public key + random PadB, transition to `read_pe_synchash`.\n2. **`read_pe_synchash`** (scans up to 512 bytes of PadA + the 20-byte hash) --\n scan received bytes for `hash('req1', S)` where S is the shared secret. The\n client may insert up to 512 bytes of PadA before this hash. Failure:\n `sync_hash_not_found`.\n3. **`read_pe_skey_vc`** (28 bytes) -- read the obfuscated SKEY hash (20\n bytes) and the encrypted VC (8 bytes). The SKEY hash is\n `hash('req2', info_hash) XOR hash('req3', S)` which reveals which torrent\n the client wants. Failure if torrent not found: `invalid_info_hash`.\n Failure if VC decrypts to non-zero: `invalid_encryption_constant`.\n4. **`read_pe_cryptofield`** (6 bytes) -- read encrypted\n `crypto_provide(4) + len_pad_c(2)`. Server selects a crypto mode and\n sends back its own VC + `crypto_select + pad`. Failure if no mode overlaps:\n `unsupported_encryption_mode`. Failure if `len_pad_c > 512`:\n `invalid_pad_size`.\n5. **`read_pe_pad`** (`len_pad_c + 2` bytes) -- skip encrypted padding, read\n `len_ia` (2 bytes).\n6. **`read_pe_ia`** (`len_ia` bytes) -- read Initial Application data (IA),\n the encrypted start of the BitTorrent handshake. Failure if `len_ia > 68`:\n `invalid_encrypt_handshake`.\n7. Falls through to **`read_protocol_identifier`** (regular BT handshake).\n\n### Outgoing (client) path\n\n1. **`on_connected`** -- send DH public key + random PadA,\n transition to `read_pe_dhkey`.\n2. **`read_pe_dhkey`** (96 bytes) -- read server's DH public key, then send\n sync hash + SKEY hash + encrypted(VC + crypto_provide + len_pad + PadC +\n len_IA) + encrypted(IA = BT handshake). Also send the BT handshake\n encrypted as IA. Transition to `read_pe_syncvc`.\n3. **`read_pe_syncvc`** (scans up to 512 bytes of PadB + the 8-byte VC) --\n scan for the encrypted 8-byte verification constant (all zeros encrypted\n with RC4). Failure if not found within the 512-byte pad window:\n `invalid_encryption_constant`.\n4. **`read_pe_cryptofield`** (6 bytes) -- read server's `crypto_select(4) +\n len_pad(2)`. Failure if selected mode not in our `allowed_enc_level`:\n `unsupported_encryption_mode_selected`.\n5. **`read_pe_pad`** (`len_pad` bytes) -- skip server's padding.\n6. Falls through to **`read_protocol_identifier`** (regular BT handshake).\n\n## Crypto Primitives\n\n### DH Key Exchange (`dh_key_exchange`)\n\n768-bit Diffie-Hellman with a fixed prime (MSE standard prime from BEP 10/MSE\nspec). `key = 2^random % prime`.\n\n- `dh_key_exchange()` -- generate local key pair\n- `compute_secret(remote_pubkey)` -- compute shared secret S; also computes\n `m_xor_mask = hash('req3', S)` used for SKEY obfuscation\n- `get_local_key()` -- 96-byte DH public key to send\n- `get_secret()` -- shared secret S\n- `get_hash_xor_mask()` -- `hash('req3', S)` for SKEY obfuscation\n\n### RC4 Key Derivation\n\nFor outgoing connections (client):\n```\nencrypt_key = hash(\"keyA\", S, SKEY)\ndecrypt_key = hash(\"keyB\", S, SKEY)\n```\nFor incoming connections (server):\n```\nencrypt_key = hash(\"keyB\", S, SKEY)\ndecrypt_key = hash(\"keyA\", S, SKEY)\n```\nwhere SKEY is the SHA-1 info hash of the torrent (or first 20 bytes of SHA-256\nfor v2 torrents). The RC4 stream is initialized by discarding the first 1024\nbytes (standard MSE requirement).\n\n### Message Layout (client -> server)\n\n```\n[96 bytes] client DH public key\n[0-512 bytes] PadA (random, encrypted sync hash follows)\n[20 bytes] sync_hash = hash('req1', S)\n[20 bytes] skey_obfusc = hash('req2', SKEY) XOR hash('req3', S)\nencrypt([8 bytes] VC = 0x00 * 8\n [4 bytes] crypto_provide (bit flags: 1=plaintext, 2=RC4)\n [2 bytes] len_PadC\n [len_PadC] PadC (random)\n [2 bytes] len_IA)\nencrypt([len_IA bytes] IA = start of BT handshake, always RC4 encrypted)\n```\n\n### Message Layout (server -> client)\n\n```\n[96 bytes] server DH public key\n[0-512 bytes] PadB (random)\nencrypt([8 bytes] VC = 0x00 * 8\n [4 bytes] crypto_select (chosen mode: 1 or 2)\n [2 bytes] len_PadD\n [len_PadD] PadD (random))\n[regular BT handshake, encrypted or not depending on crypto_select]\n```\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `src/pe_crypto.cpp` + `include/libtorrent/aux_/pe_crypto.hpp` | DH exchange, RC4 handler, encryption_handler |\n| `src/bt_peer_connection.cpp` | PE state machine and write helpers |\n| `fuzzers/src/pe_conn.cpp` | Session-level fuzzer for the PE handshake |\n| `fuzzers/src/pe_crypto_state.cpp` | Low-level fuzzer for DH + RC4 primitives |\n\n## Error Codes\n\n| Error | Trigger |\n|-------|---------|\n| `sync_hash_not_found` | sync hash not found within 512-byte window |\n| `invalid_info_hash` | SKEY hash doesn't match any known torrent |\n| `invalid_encryption_constant` | VC decrypts to non-zero bytes |\n| `unsupported_encryption_mode` | No overlap in crypto_provide vs allowed_enc_level |\n| `unsupported_encryption_mode_selected` | Server selected a mode not in our allowed list |\n| `invalid_pad_size` | len_pad_c > 512 |\n| `invalid_encrypt_handshake` | len_ia > 68 |\n",".claude/rules/v2-torrents.md":"# BitTorrent v2 Torrents (BEP 52)\n\n## Info Dict Structure\n\nV2 uses a nested **`file tree`** dict instead of a flat `files` list:\n- Keys are filename path components; non-leaf = directory (nested dict)\n- Leaf nodes have empty-string key `\"\"` → file metadata dict with `length`, `pieces root`, etc.\n- **`pieces root`** (32 bytes): root of the per-file merkle tree (SHA-256)\n- **`meta version`**: integer `2` identifies v2 metadata\n- **`piece layers`**: (outside info dict) maps `pieces root` → concatenated piece-layer hashes\n\nV2 torrents have a **SHA-256 info hash** (vs v1's SHA-1). `torrent_info::m_info_hash` holds both.\n\n## Per-File Merkle Tree\n\nEach file has an independent merkle tree (SHA-256, not SHA-1):\n\n```\nroot (pieces root)\n └── interior nodes\n └── piece layer ← hash of one piece's block subtree\n └── block layer (leaf) ← hash of 16 KiB block\n```\n\n- **Block** = 16 KiB (fixed), the unit of transfer\n- **Piece** = `piece_length` bytes; must be a power-of-2 multiple of 16 KiB\n- Tree is **padded to a power of 2** in leaf count using `merkle_pad()` (zeros hashed up)\n- Interior nodes: `parent = SHA-256(left_child || right_child)`\n- Flat array indexing: node at layer `l`, offset `o` → index `(1 << l) - 1 + o`; parent = `(i-1)/2`\n\nKey helpers in `src/merkle.cpp`:\n- `merkle_num_leafs()`, `merkle_num_nodes()`, `merkle_num_layers()`\n- `merkle_fill_tree()` — compute interior nodes from leaves\n- `merkle_validate_and_insert_proofs()` — validate a proof chain against a known root\n- `merkle_pad()` — padding hash for odd tree levels\n\n## merkle_tree Class (`include/libtorrent/aux_/merkle_tree.hpp`)\n\nOne instance per file, with four storage modes (optimization):\n\n| Mode | Stored nodes | When used |\n|------|-------------|-----------|\n| `empty_tree` | root only | single-block file, or tree completely unknown |\n| `piece_layer` | piece-layer hashes | piece hashes received, blocks not yet |\n| `block_layer` | block hashes | download complete, all blocks verified |\n| `full_tree` | all nodes | during partial verification |\n\n`m_block_verified` bitfield tracks which block hashes have been validated against their parent chain up to the known root.\n\nKey methods: `load_tree()`, `load_piece_layer()`, `add_hashes()`, `set_block()`\n\n## Hybrid Torrents (v1 + v2)\n\nA single `.torrent` file can contain both:\n- v1 info dict: `files` list + `pieces` SHA-1 concatenation\n- v2 `file tree` + `piece layers`\n\nConstraints: identical file layout, same `piece_length`. Results in two info hashes.\n`torrent_info` parsing uses `extract_files()` for v1 and `extract_files2()` for v2.\n\n## Hash Picker (`src/hash_picker.cpp`)\n\nDecides which merkle hashes to request from peers (BitTorrent v2 hash exchange protocol).\n\n**`hash_request` struct:**\n```cpp\nstruct hash_request {\n file_index_t file;\n int base; // tree layer (0 = block layer)\n int index; // starting offset in layer\n int count; // number of hashes (≤ 8192)\n int proof_layers; // uncle hashes needed to prove against known root\n};\n```\n\n**Strategy:**\n1. Request piece-layer hashes in 512-piece chunks until all known\n2. On piece failure, request all block hashes for that piece (3s minimum interval)\n\n**Validation flow:**\n- `add_hashes()`: receives hashes + uncle proof chain, calls `merkle_validate_and_insert_proofs()`, inserts valid nodes, reports `pass`/`fail` per piece\n- `set_block()`: inserts a single block hash, finds largest verifiable subtree, returns pass/fail\n\n## Resume Data\n\nMerkle trees are persisted across sessions in resume data (`src/write_resume_data.cpp`, `src/read_resume_data.cpp`):\n\n```\n[\"merkle trees\"][file_index] → sparse tree nodes (non-zero nodes only)\n[\"merkle tree mask\"][file_index] → bitfield of which nodes are stored\n[\"verified bits\"][file_index] → m_block_verified bitfield\n[\"piece layers\"][pieces_root] → concatenated piece-layer hashes\n```\n\n- `write_resume_data()` uses `build_sparse_vector()` to encode only known nodes\n- `read_resume_data()` calls `load_sparse_tree()` to restore the partial tree\n- Piece layers are re-derived from the stored tree during write\n\n## Verification Flow\n\n1. File added → `merkle_tree` created with root hash from `pieces root`\n2. Block arrives → `set_block()` inserts hash, attempts validation up to root\n3. Piece complete → all block hashes in piece set; piece passes if tree validates\n4. Piece fails → clear invalid nodes; hash picker schedules block-hash requests\n5. Hashes received → `add_hashes()` validates proof chain, inserts valid nodes, marks failures\n6. Download complete → all blocks verified; block layer mode; tree fully populated\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `src/merkle.cpp` | Tree math, proof validation, padding |\n| `src/merkle_tree.cpp` + `include/libtorrent/aux_/merkle_tree.hpp` | Per-file tree storage |\n| `src/hash_picker.cpp` + `include/libtorrent/aux_/hash_picker.hpp` | Hash request scheduling |\n| `src/torrent_info.cpp` | Parsing `file tree`, `piece layers`, hybrid logic |\n| `src/create_torrent.cpp` | Building v2/hybrid `.torrent` files |\n| `src/write_resume_data.cpp` + `src/read_resume_data.cpp` | Persisting trees |\n"},"files":{".claude/rules/documentation.md":"# Documentation Generation and Spell Checking\n\nThe reference documentation is generated from the C++ public headers; the\nprose manual is hand-written reStructuredText (RST). Both are built and\nspell-checked from the `docs/` directory via `docs/makefile`, and the same\ntargets run in CI (`.github/workflows/docs.yml`).\n\n## Reference Doc Generation (`docs/gen_reference_doc.py`)\n\n`gen_reference_doc.py` parses the public headers and emits RST. It scans\n`include/libtorrent/*.hpp`, `include/libtorrent/kademlia/*.hpp`, and\n`include/libtorrent/extensions/*.hpp` (add `--internal` to also include\n`include/libtorrent/aux_/*.hpp`). For each documented class, function, enum,\nand constant it produces an RST file per category (`reference-Core.rst`,\n`reference-Session.rst`, ...) plus a table of contents (`reference.rst`) and an\noptional `single-page-ref.rst` (`--single-page`).\n\n### How comments become docs\n\n- A documentation comment is the run of `//` comment lines immediately\n **above** a declaration, with no blank line between the comment and the\n declaration. A blank line, or a line that ends a declaration (`;`, `{`,\n `}`), resets the pending comment.\n- **Comment bodies are reStructuredText.** You can use RST markup directly in\n the `//` comments: `.. code::` blocks, `.. note::`, bullet lists,\n `parsed-literal`, headings, etc. The generator renders them verbatim into\n the output RST.\n- **Symbol cross-linking** is automatic: any word in a comment that matches a\n known symbol name (class, function, enum, `settings_pack::` setting, or one\n of the pre-defined manual anchors) is turned into a hyperlink by\n `linkify_symbols()`. A trailing underscore (`foo_`) forces a link the same\n way RST reference syntax does.\n\n### Grouping functions under one comment (man-page style)\n\nMultiple overloads or related functions can share a **single** documentation\ncomment. The rule is **adjacency with no blank line between the\ndeclarations**: when a function (or field) declaration immediately follows the\nprevious one with no blank line and no intervening comment, its signature is\nappended to the previous entry's signature list rather than starting a new\nentry. So a comment above the first of a group documents the whole group, the\nway a man page documents a family of related calls together. Inserting a blank\nline between declarations splits them into separately-documented entries (and\nthe second will then warn if it has no comment of its own).\n\n### Visibility and warnings\n\n- A comment beginning with `hidden` excludes the symbol entirely. A comment\n beginning with `internal` excludes it from the public docs (but `--internal`\n includes it).\n- Every public, non-trivial member must be documented. Undocumented\n symbols print `WARNING: ... is not documented`. Trivial members\n (destructors, default/copy/move ctors, assignment, comparison/stream/\n conversion operators) are exempt via `is_trivial_member()`.\n- A `TODO:` in a public doc comment is a hard error (exits non-zero).\n- `#if` blocks gated on debug-only macros (asserts, invariant checks, and the\n other debugging macros) and on old `TORRENT_ABI_VERSION` guards are skipped\n entirely, so debug-only or deprecated members do not appear in docs. A\n non-ABI `TORRENT_*` `#if` inside a public struct warns about possible ABI\n breakage.\n- `// OVERVIEW` introduces a section overview block for a header's category.\n\n### Plain-text output for spell checking\n\nWith `--plain-output`, `add_desc()` writes a `plain_text_out.txt` containing\n**only the prose** from doc comments, with the C++ signatures and `.. code::`\nblocks stripped out. This is what the spell checker runs over.\n\n## Other Generators\n\n| Script | Output | Source |\n|--------|--------|--------|\n| `gen_reference_doc.py` | `reference-*.rst`, `reference.rst`, `single-page-ref.rst`, `plain_text_out.txt` | public headers |\n| `gen_settings_doc.py` | `settings.rst` + appends settings names to `hunspell/settings.dic` | `settings_pack.hpp` |\n| `gen_stats_doc.py` | `stats_counters.rst` | `session_stats.cpp`, `performance_counters.hpp` |\n| `gen_todo.py` | `todo.html` | `src/*.cpp`, headers |\n| `filter-rst.py` | `*-plain.txt` (prose-only) | a hand-written `.rst` file |\n\n`gen_settings_doc.py` doubles as a dictionary source: it splits every setting\nname on `_` and adds the parts to `hunspell/settings.dic`, so setting-name\nfragments are not flagged as misspellings.\n\n## Spell Checking (hunspell)\n\nSpell checking uses **hunspell**, run via `make spell-check` from `docs/`:\n\n```sh\ncd docs && make spell-check\n```\n\nThe flow:\n1. Generated reference prose is in `plain_text_out.txt` (from\n `gen_reference_doc.py --plain-output`).\n2. Hand-written manuals are converted to prose-only text with\n `filter-rst.py` (e.g. `manual.rst` -> `manual-plain.txt`). `filter-rst.py`\n strips RST directives (`.. ` lines) and indented literal/code blocks so\n only natural-language text remains.\n3. `hunspell -l` lists misspelled words from each plain-text file into\n `hunspell-report.txt`, using dictionary `hunspell/en_US` plus the project\n word list `hunspell/libtorrent.dic` (settings text uses the generated\n `hunspell/settings.dic`). HTML manuals are checked with `-H` (HTML mode).\n4. If `hunspell-report.txt` is non-empty the target **fails** and prints the\n offending words.\n\n### Fixing spell-check failures\n\nEither correct the typo in the source, or -- if the word is a real term,\nidentifier, or acronym -- add it to **`docs/hunspell/libtorrent.dic`** (one\nword per line). `hunspell/en_US.{aff,dic}` is the stock English dictionary and\nshould not be edited. `hunspell/settings.dic` is generated (en_US + the\nsettings names + a copy of `libtorrent.dic`); do not hand-edit it.\n\n## Building the HTML\n\n```sh\ncd docs && make html # all RST -> HTML via rst2html, plus figures\ncd docs && make # default target builds the docs\n```\n\nHTML is produced by `rst2html` (docutils) with `template.txt`/`style.css`;\nfigures are rendered from `.dot` (graphviz), `.diagram` (aafigure), and `.svg`\n(imagemagick `convert`). `make stage` copies the built site to `$WEB_PATH`.\n`make clean` removes generated `.rst`/`.html`/figures (it does **not** delete\nthe checked-in hand-written `.rst` manuals).\n\n## CI\n\n`.github/workflows/docs.yml` installs `python3-docutils`, `hunspell`,\n`graphviz`, `imagemagick`, `aafigure`, etc., then runs\n`make spell-check html`. A documentation warning, an undocumented public\nsymbol, a `TODO:` in a doc comment, or a spelling error all fail the build.\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `docs/gen_reference_doc.py` | parse headers -> reference RST + plain text |\n| `docs/gen_settings_doc.py` | `settings.rst` + settings dictionary words |\n| `docs/gen_stats_doc.py` | `stats_counters.rst` |\n| `docs/filter-rst.py` | strip RST markup to prose for spell checking |\n| `docs/makefile` | `html`, `rst`, `spell-check`, `stage`, `clean` targets |\n| `docs/hunspell/libtorrent.dic` | project word list (edit to whitelist terms) |\n| `docs/hunspell/en_US.{aff,dic}` | stock English dictionary (do not edit) |\n| `.github/workflows/docs.yml` | CI: spell-check + build |\n","tools/vcpkg.json":"{\n \"$schema\": \"https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json\",\n \"builtin-baseline\": \"d90a9b159c08169f39adcd1b0f1ac0ca12c4b96c\",\n \"dependencies\": [\"openssl\"],\n \"overrides\": [\n {\n \"name\": \"openssl\",\n \"version-string\": \"3.6.1\"\n }\n ]\n}",".claude/rules/disk-io.md":"---\npaths:\n - \"include/libtorrent/disk_interface.hpp\"\n - \"src/mmap_disk_io*\"\n - \"src/posix_disk_io*\"\n - \"src/pread_disk_io*\"\n - \"src/disk_job_fence*\"\n---\n\n## `disk_interface` contract\n\n`disk_interface` (`include/libtorrent/disk_interface.hpp`) is the session-level\ncustomization point for disk I/O, selected via\n`session_params::disk_io_constructor`. Built-in implementations: `mmap_disk_io`,\n`posix_disk_io`, `pread_disk_io`.\n\n**Threading.** Every `async_*` method is called from libtorrent's single network\nthread. The work may be performed asynchronously on the implementation's own\nthreads, but each completion handler **must be posted back to the network\nthread** via the `io_context` given to the constructor; handlers run on the\nnetwork thread. An implementation may use any number of threads and service\nqueued jobs concurrently and out of order, except where the contract below\nrequires ordering.\n\n**`async_hash` reflects all prior `async_write`s.** For a given piece, the\nengine posts all of the piece's blocks via `async_write`, then posts\n`async_hash` for that piece. The hash returned **must be computed over the data\nof every `async_write` for that piece that was posted before it.** This is the\nkey constraint for a multi-threaded back end that services jobs out of order: it\nmust order a piece's hash after that piece's writes and must never hash stale\nor missing data. The piece picker treats the returned hash as authoritative for\nthe data it handed over. `async_hash2` (single v2 block hash) has the same\nrequirement for the block it covers.\n\n**Fence jobs.** Several operations must run with exclusive access to a storage,\nsynchronized against all other I/O on it. These are *fence jobs*. A fence job:\n\n- waits for all jobs currently outstanding **on that storage** to finish, then\n executes -- so it runs alone, with no other operation on the storage in\n flight;\n- *fences* everything posted after it: any job posted to the same storage while\n the fence is raised is queued and not executed until the fence job completes.\n\nFences are per-storage -- a fence on one storage does not block jobs on another.\nThe fence operations are:\n\n- `async_clear_piece` -- synchronize with all outstanding I/O to the piece\n before the engine re-requests its blocks\n- `async_move_storage`\n- `async_release_files`\n- `async_stop_torrent`\n- `async_rename_file`\n- `async_delete_files`\n- `async_set_file_priority`\n- `async_check_files`\n\n`async_read`, `async_write`, `async_hash`, and `async_hash2` are ordinary\n(non-fence) jobs. The built-in back ends implement the fence semantics with\n`disk_job_fence` (`src/disk_job_fence.cpp`). A storage's outstanding-job count\nis what the fence waits to reach zero before it runs; a job stays counted from\nthe moment it is accepted until its completion is posted back.\n\n**Every ordinary job must honor the fence -- including a buffered write.** Each\njob, with no exceptions, has to participate in the outstanding-job count for its\nwhole lifetime, so the fence cannot run while any work on the storage is still\nin flight. This is easy to overlook for `async_write` on a write-back back end:\neven though the write is inserted into a cache and its handler is deferred, it\nmust be counted as outstanding from the moment it is accepted until the flush\nthat retires it has finished and its completion is posted -- *including the\nwindow while a disk thread is mid-flush writing it out*. It must not take a fast\npath that skips the count. If a buffered write (or the thread flushing it) is\ninvisible to the fence, the fence can be raised and complete while that write is\nstill being written to disk, and a teardown fence (`async_stop_torrent`,\n`async_release_files`, ...) can then tear the storage down underneath the\nin-flight flush -- a use-after-free. So the rule is symmetric: a write counts\nexactly like a read or a hash.\n\n### Fences and deferred (write-back) writes\n\nThe fence contract above is straightforward for an implementation that runs\neach job to completion immediately. An implementation that *defers* writes --\nreporting an `async_write` complete before the data is durably on disk, as a\nwrite-back cache does -- has to uphold two further invariants. Both are easy to\nget wrong, and a violation of either is a deadlock or silent data corruption,\nnot a degraded-performance bug.\n\n**A fence must not wait on writes that nothing will flush (forward progress).**\nA fence waits for every job outstanding on the storage to complete before it\nruns. When a write is only reported complete once it is durably stored, that\nwrite stays outstanding -- and keeps the fence waiting -- the whole time it sits\nbuffered. The flush that would retire it normally happens when the piece\ncompletes (its `async_hash`) or under memory pressure; but the completing hash\nis itself an ordinary job, now queued *behind* the very fence that is waiting\nfor it, and memory pressure may never arrive. The invariant: **raising a fence\nmust guarantee the storage's buffered writes get flushed, rather than waiting\npassively for a flush trigger the fence is itself blocking.** Otherwise the\nfence and its own backlog deadlock.\n\n**Resuming a fenced backlog must preserve posting order against new jobs.** When\na fence completes, the jobs that queued behind it are resumed. If resuming a job\nis observable -- a buffered write becomes visible, a read is served -- then a job\nthe network thread posts *during* that resume must be ordered after the whole\nresumed backlog, never interleaved ahead of it. The invariant: **a job arriving\nwhile the backlog is being resumed must not bypass the backlog.** Violate it and\na freshly-posted `async_read` overtakes a still-queued `async_write` for the\nsame block and returns stale data (or a hole).\n\n**A read resumed from a fence must reflect writes resumed ahead of it.** A\ncorollary of the ordering invariant: a read unblocked from the backlog may be\npreceded in that same backlog by a write to the very block it reads (both were\nblocked by the fence). That read must observe the write -- i.e. it must be\nserved from the buffered data, not from disk where the superseding write has not\nyet landed. Put plainly: the reads and writes a fence releases must take effect\nin the order they were posted, exactly as if the fence had never been raised.\n\n**Back-pressure.** `async_write` returns `true` when the write queue is over the\nhigh watermark; the caller then stops issuing writes for that peer until the\n`disk_observer` passed in is notified that the queue has drained. `false` means\nthe caller may keep writing.\n\n**Batching.** `async_*` calls only enqueue work; `submit_jobs()` is called once\nafter a batch to wake the disk thread(s). An implementation may also wake on each\n`async_*`, but should tolerate the batched notification.\n",".claude/CLAUDE.md":"# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Overview\n\nlibtorrent-rasterbar is a C++17 BitTorrent library (version 2.1.0). It supports v1 and v2 torrents, DHT, WebTorrent (WebRTC), and optional I2P/SSL.\n\n## Build Systems\n\nThe project has **two build systems**: boost-build (b2/bjam) and CMake. **Boost-build is the preferred build system** per the contributing guidelines.\n\n### Boost-build (b2) - preferred\n\n**Never add `-j` (parallel jobs) flags to b2 commands.** b2 manages parallelism on its own.\n\nBuild the library (from repo root):\n```sh\nb2\n```\n\nRun unit tests:\n```sh\ncd test && b2\n```\n\nRun a single test:\n```sh\ncd test && b2 test_session\n```\n\nRun deterministic tests (no flaky networking):\n```sh\ncd test && b2 deterministic-tests\n```\n\nRun simulations:\n```sh\ncd simulation && b2\n```\n\nBuild with developer options (typical for development):\n```sh\ncd test && b2 asserts=on invariant-checks=full debug-iterators=on picker-debugging=on\n```\n\nBuild with sanitizers:\n```sh\ncd test && b2 address-sanitizer=norecover undefined-sanitizer=norecover asserts=on\n```\n\nBuild examples, tools, fuzzers, python bindings:\n```sh\ncd examples && b2\ncd tools && b2\ncd fuzzers && b2 clang\ncd bindings/python && b2\n```\n\n### CMake — alternative\n\n```sh\ncmake -B build -DCMAKE_BUILD_TYPE=Debug -Dbuild_tests=ON\ncmake --build build\ncd build && ctest\n```\n\nRun a single test via CMake:\n```sh\ncd build && ./test/test_session\n```\n\n## Pre-commit\n\nThe repo uses [pre-commit](https://pre-commit.com). Before pushing:\n```sh\npip install pre-commit\npre-commit install\npre-commit run --all-files\n```\n\nPre-commit hooks include trailing whitespace, YAML/TOML/XML checks, RST formatting, Python formatting (black, isort, flake8, mypy, autoflake), and auto-generation of:\n- `include/libtorrent/fwd.hpp` and `include/libtorrent/libtorrent.hpp` via `tools/gen_fwd.py` and `tools/gen_convenience_header.py`\n- C binding headers (`bindings/c/include/libtorrent_settings.h`, `libtorrent_alerts.h`) via `bindings/c/tools/gen_header.py` and `bindings/c/tools/gen_alert_header.py`\n\nThe C++ format check (`git-clang-format-18`) is the only hook that requires a\nsystem install -- `apt install clang-format-18` (or platform equivalent). It\nformats only the lines that differ from HEAD, not whole files; the codebase is\nbeing migrated to the `.clang-format` style incrementally rather than via a\nflag-day reformat.\n\n## Key Architecture\n\n### Threading Model\n\nThree thread categories:\n1. **Main network thread** — all sockets, session/torrent/peer state, boost.asio event loop\n2. **Disk I/O thread(s)** — reads, writes, and SHA-1/SHA-256 piece verification (count controlled by `settings_pack::aio_threads`)\n3. **Resolver thread** — spawned by boost.asio for async DNS on platforms without native async getaddrinfo\n\nAll interaction with session/torrent state from outside must go through the network thread (via `session_handle` and `torrent_handle`).\n\n### Core Classes\n\n- **`session`** (`include/libtorrent/session.hpp`) — public API, pimpl over `session_impl`\n- **`session_impl`** (`include/libtorrent/aux_/session_impl.hpp`) — all session state: torrent list, connection list, global rate limits, DHT state, port mapping\n- **`torrent`** (`include/libtorrent/aux_/torrent.hpp`, `src/torrent.cpp`) — all state for a single swarm: piece picker, peer connections, file storage\n- **`torrent_handle`** (`include/libtorrent/torrent_handle.hpp`) — public pimpl handle, weak reference to `torrent`; sends messages to network thread\n- **`peer_connection`** / **`bt_peer_connection`** — BitTorrent protocol implementation\n- **`piece_picker`** — download strategy (which blocks to request from which peers). See `.claude/rules/piece-picker.md` for a detailed description.\n- **`peer_list`** — list of known (not necessarily connected) peers for a swarm\n\n### Disk I/O\n\nThree disk backends:\n- `mmap_disk_io` — mmap-based (default on 64-bit when mmap is available)\n- `posix_disk_io` — fallback single-threaded POSIX I/O (used on 32-bit or without mmap)\n- `pread_disk_io` — multi-threaded backend using `pread()`/`pwrite()`. See `.claude/rules/disk-cache.md` for a detailed description.\n\n### DHT (Kademlia)\n\nSource in `src/kademlia/`, headers in `include/libtorrent/kademlia/`. Includes ed25519 signing (`src/ed25519/`).\n\n### Extensions\n\nProtocol extensions live in `src/` and `include/libtorrent/extensions/`:\n- `ut_metadata` — magnet link metadata exchange\n- `ut_pex` — peer exchange\n- `smart_ban` — ban peers that send corrupt data\n- `i2p_pex` — peer exchange for I2P peers\n\n### WebTorrent (WebRTC)\n\nOptional feature controlled by `webtorrent=on` / `-Dwebtorrent=ON`. Uses `deps/libdatachannel` and the `rtc_stream`/`rtc_signaling` subsystem.\n\n### Python Bindings\n\nSource in `bindings/python/src/*.cpp`, built as `libtorrent.so` using boost.python. See `.claude/rules/python-bindings.md` for detailed conventions.\n\n### Simulations\n\n`simulation/` contains network simulation tests that use `libsimulator` (a deterministic virtual network). These test high-level behaviors (swarms, DHT, session management) without real network access. All simulation tests run with `invariant-checks=full`, `asserts=on`, `debug-iterators=on`.\n\n### Terminology\n\n- **piece** — SHA-1/SHA-256 verified chunk of torrent data (typically power-of-two size)\n- **block** — 16 KiB sub-unit of a piece (the transfer unit in the protocol)\n- **torrent_peer** — a known-but-not-connected peer entry\n- **peer_connection** — an active connection to a peer\n\n## Key Build Options (b2 features)\n\n| Feature | Values |\n|---------|--------|\n| `crypto` | `built-in`, `openssl`, `openssl-shared`, `wolfssl`, `gcrypt`, `gnutls`, `libcrypto` |\n| `asserts` | `on`, `off`, `production`, `system` |\n| `invariant-checks` | `off`, `on`, `full` |\n| `logging` | `on`, `off` |\n| `deprecated-functions` | `on`, `off`, `1`, `2`, `3`, `4` |\n| `webtorrent` | `on`, `off` |\n| `address-sanitizer` | `norecover`, `recover`, `off` |\n| `undefined-sanitizer` | `norecover`, `recover`, `off` |\n| `thread-sanitizer` | `norecover`, `recover`, `off` |\n| `picker-debugging` | `on`, `off` |\n| `debug-iterators` | `default`, `off`, `on`, `harden` |\n\n`deprecated-functions` selects the ABI/API version via `TORRENT_ABI_VERSION`. Different values are **link-incompatible**. Higher numbers expose a more modern interface and remove older deprecated APIs:\n\n| Value | `TORRENT_ABI_VERSION` | Corresponds to |\n|-------|-----------------------|----------------|\n| `on` | (oldest supported) | libtorrent 1.x |\n| `1` | 1 | libtorrent 1.x |\n| `2` | 2 | libtorrent 2.0 |\n| `3` | 3 | libtorrent 2.x |\n| `4` | 4 | libtorrent 2.1 |\n| `off` | 100 (newest) | latest API |\n\n## Directory Layout\n\n```\nsrc/ main library source\nsrc/kademlia/ DHT implementation\nsrc/ed25519/ ed25519 signing (vendored)\ninclude/libtorrent/ public API headers\ninclude/libtorrent/aux_/ internal headers (not public API)\ninclude/libtorrent/kademlia/ DHT headers\ninclude/libtorrent/extensions/ extension headers\ntest/ unit tests (test_*.cpp)\nsimulation/ network simulation tests (test_*.cpp)\nfuzzers/src/ fuzz targets\nexamples/ example programs\ntools/ utility scripts and programs\nbindings/python/ Python bindings (boost.python)\nbindings/c/ C API bindings\ndeps/ vendored dependencies\n deps/try_signal/ signal handling\n deps/asio-gnutls/ GnuTLS adapter for asio\n deps/json/ nlohmann/json (for WebTorrent)\n deps/libdatachannel/ WebRTC (for WebTorrent)\n```\n\n## Adding New Source Files\n\nWhen adding a new `.cpp` or `.hpp` file, it must be added to **all three** build systems:\n1. `Jamfile` (boost-build)\n2. `CMakeLists.txt`\n3. `Makefile` (if applicable)\n\n## Code-Generation Tools\n\nAfter modifying public API headers, regenerate the derived headers by running these scripts from the repo root:\n\n- `tools/gen_fwd.py` — regenerates `include/libtorrent/fwd.hpp` (forward declarations for all public types)\n- `tools/gen_convenience_header.py` — regenerates `include/libtorrent/libtorrent.hpp` (the convenience header that includes all public headers)\n\nThese are also run automatically by the pre-commit hooks.\n\n## Coding Conventions\n\n- Comments must use ASCII characters only (no Unicode, smart quotes, em-dashes, etc.)\n- Use a single space after a period in comments (not two spaces)\n- Declare variables `const` whenever they are not reassigned after initialization\n- C++17 throughout; no C++20 features yet\n- The code must build with standard C++ and must not require compiler language extensions; optional extensions (e.g. `__builtin_*`, platform-specific pragmas) may be used when guarded by feature detection\n- `namespace lt = libtorrent` alias is always available\n- Asserts: use `TORRENT_ASSERT(cond)` (active when `TORRENT_USE_ASSERTS` is defined, i.e. debug builds)\n- Invariant checks: expensive checks inside `#if TORRENT_USE_INVARIANT_CHECKS`\n- Public API headers use `TORRENT_EXPORT` macro; internal symbols use hidden visibility\n- Internal functions and classes use `TORRENT_EXPORT_EXTRA` macro; to grant access to tests\n- ABI versioning via `TORRENT_VERSION_NAMESPACE_2/3/4` inline namespace macros (defined in `include/libtorrent/aux_/export.hpp`); `_2` = v1.2, `_3` = v2, `_4` = v2.1\n- Warnings are treated as errors in CI (both gcc and clang)\n- Changes to ABI (fields/ordering of public classes) must target `master`, not `RC_*` stable branches\n- `TORRENT_EXPORT_EXTRA` marks a symbol exported only for test access; it is private/internal, not public API, and is exempt from the ABI-stability rules above (safe to change on `RC_*` branches). Do not treat `TORRENT_EXPORT_EXTRA` as indicating public or ABI-sensitive status; `TORRENT_EXPORT` is the public one\n- `settings_pack` enum values must be appended at the end of each enum group (int, bool, string) — never inserted in the middle — to avoid changing the numeric values of existing settings and breaking ABI\n- prefer the C++ counterparts to C headers\n- do not use using-statements in header files\n- prefer to fully qualify standard types and functions\n- prefer the short lt namespace alias when fully qualifying libtorrent types\n- prefer using default member initializers over initializer list\n\n### Strong Types\n\nAvoid raw `int` for indices and flags; use `aux::strong_typedef` (`include/libtorrent/units.hpp`) and `flags::bitfield_flag` (`include/libtorrent/flags.hpp`). See `.claude/rules/strong-types.md` for details.\n",".claude/rules/dht.md":"---\npaths:\n - \"src/kademlia/**\"\n - \"include/libtorrent/kademlia/**\"\n---\n\n# DHT (Kademlia) Implementation\n\nSource: `src/kademlia/`, `include/libtorrent/kademlia/`\n\n## BEP Support\n\n| BEP | Description | Status |\n|-----|-------------|--------|\n| BEP 5 | DHT Protocol (ping, find_node, get_peers, announce_peer) | Full |\n| BEP 32 | IPv6 extension (dual-stack, nodes6) | Full |\n| BEP 33 | DHT Scrape (bloom filters BFpe/BFsd) | Full |\n| BEP 42 | Node ID Verification (IP-derived IDs) | Full |\n| BEP 44 | Arbitrary data storage (immutable + mutable items, Ed25519) | Full |\n| BEP 51 | DHT Infohash Indexing (sample_infohashes) | Full |\n\n---\n\n## File Overview\n\n| File | Purpose |\n|------|---------|\n| `node.cpp` / `node.hpp` | Central DHT node — routing, dispatch, traversal lifecycle |\n| `dht_tracker.cpp` / `dht_tracker.hpp` | Session-level coordinator, one `node` per listen socket |\n| `routing_table.cpp` / `routing_table.hpp` | K-bucket routing table |\n| `rpc_manager.cpp` / `rpc_manager.hpp` | Tracks outstanding RPCs, calls observers on response/timeout |\n| `observer.hpp` | Base observer class for pending RPC state |\n| `traversal_algorithm.cpp` / `.hpp` | Base iterative search algorithm |\n| `find_data.cpp` / `.hpp` | Find-nodes traversal, base for get_peers and get_item |\n| `get_peers.cpp` / `.hpp` | BEP 5 peer lookup traversal |\n| `get_item.cpp` / `.hpp` | BEP 44 item retrieval traversal |\n| `put_data.cpp` / `.hpp` | BEP 44 item storage traversal |\n| `sample_infohashes.cpp` / `.hpp` | BEP 51 traversal |\n| `refresh.cpp` / `.hpp` | Bootstrap / bucket-refresh traversal |\n| `direct_request.hpp` | Single-message direct request |\n| `node_id.cpp` / `node_id.hpp` | `node_id` type (alias for `sha1_hash`), XOR distance, BEP 42 generation |\n| `node_entry.cpp` / `node_entry.hpp` | Single routing-table entry |\n| `item.cpp` / `item.hpp` | Mutable/immutable item type |\n| `types.hpp` | `public_key`, `secret_key`, `signature`, `sequence_number` |\n| `msg.hpp` | Incoming message structure |\n| `dht_storage.cpp` / `dht_storage.hpp` | Pure-virtual storage interface + default RAM implementation |\n| `dht_state.cpp` / `dht_state.hpp` | Persistent state (node IDs, bootstrap nodes) |\n| `dht_settings.hpp` | Per-DHT settings struct |\n| `dht_observer.hpp` | Callbacks from DHT into `session_impl` |\n| `dos_blocker.cpp` / `dos_blocker.hpp` | Per-IP rate limiting (5 pkt/s, 5-min block) |\n| `ed25519.hpp` | Ed25519 sign/verify API (vendored impl in `src/ed25519/`) |\n| `announce_flags.hpp` | Announce flags: `seed`, `implied_port`, `ssl_torrent` |\n\n---\n\n## Threading\n\nAll DHT code runs on the **session network thread** (boost.asio). No locking needed inside DHT classes. `rpc_manager` uses a mutex only for its observer-pool allocator.\n\n---\n\n## Core Data Structures\n\n### `node_id`\nType alias for `sha1_hash` (160 bits).\n\nKey functions:\n- `distance(a, b)` — XOR metric\n- `distance_exp(a, b)` — index of highest differing bit (0–160)\n- `compare_ref(a, b, ref)` — compare distances to reference\n- `generate_id(external_ip)` — BEP 42 derivation\n- `verify_id(nid, source_ip)` — BEP 42 check\n\n### `node_entry`\nOne routing-table peer entry:\n- `node_id id`, `udp::endpoint endpoint`\n- `uint16_t rtt` (0xffff = unknown)\n- `uint8_t timeout_count` (0 = confirmed, 0xff = never pinged)\n- `bool verified` (BEP 42)\n- `time_point last_queried`\n\n### `routing_table`\n160 k-buckets (one per XOR distance bit). Extended routing table (default on) gives larger buckets close to us:\n\n| Bucket index | Multiplier | Max live nodes (k=20) |\n|---|---|---|\n| 0 | 16× | 320 |\n| 1 | 8× | 160 |\n| 2 | 4× | 80 |\n| 3 | 2× | 40 |\n| 4–159 | 1× | 20 (k) |\n\nKey behaviours:\n- Last (closest) bucket splits when it exceeds k entries\n- Each bucket has a **replacement cache**; candidates promoted when live node is removed\n- `ip_set` restricts one node per IP per bucket (BEP 42, `restrict_routing_ips`)\n- Periodic refresh: random query in each bucket's range every ~15 min\n\n### `item` (BEP 44)\n- **Immutable** — key is `sha1(value)`, no signature\n- **Mutable** — key is `sha1(salt || pk)`, Ed25519-signed, has `sequence_number`\n- CAS: `cas` field lets clients do compare-and-swap on sequence number\n\n### `dht_storage_interface`\nPure-virtual; default RAM implementation (`dht_default_storage_constructor`):\n\n```\nmap m_torrents (BEP 5 peers)\nmap m_immutable_items\nmap m_mutable_items\n```\n\nEviction uses distance from our node IDs — items far from us are evicted first.\nLimits: `max_torrents` (2000), `max_peers` (500), `max_dht_items` (700).\nPeer entries expire after 30 min (announce_interval).\n\n---\n\n## Key Classes\n\n### `node`\nOne per listen socket. Owns `routing_table` and `rpc_manager`.\n\nImportant methods:\n```cpp\nvoid bootstrap(endpoints, done_cb);\nvoid get_peers(info_hash, callback, flags);\nvoid announce(info_hash, port, flags, callback);\nvoid get_item(target, callback); // immutable\nvoid get_item(pk, salt, callback); // mutable\nvoid put_item(data, callback); // immutable\nvoid put_item(pk, sk, salt, callback); // mutable\nvoid sample_infohashes(endpoint, target, callback);\nvoid incoming(udp::endpoint, msg); // dispatch received UDP\nvoid tick(); // token rotation, bucket refresh\n```\n\nWrite tokens: `hash(requester_ip + secret + info_hash)`. Two secrets kept; rotated periodically to allow stale tokens for one rotation period.\n\n### `dht_tracker`\nOwns `std::map` (one `node` per socket).\nAggregates results across all nodes. Manages DOS blocking. Saves/restores `dht_state`.\n\n### `rpc_manager`\n- `invoke(entry, endpoint, observer_ptr)` — sends bencoded RPC, tracks by 2-byte transaction ID\n- `incoming(msg, &sender_id)` — looks up observer by txn-id, calls `observer::reply()`\n- `tick()` — fires `observer::timeout()` / `observer::short_timeout()` for stale requests\n- Uses an object pool for observers to reduce heap fragmentation\n\n### `observer`\nOne per pending RPC. Flags:\n\n| Flag | Meaning |\n|------|---------|\n| `flag_queried` | Request sent |\n| `flag_initial` | First attempt |\n| `flag_short_timeout` | Intermediate timeout fired |\n| `flag_failed` | Node failed |\n| `flag_alive` | Response received |\n| `flag_done` | Finished, no longer tracked |\n\nObserver hierarchy:\n```\nobserver\n ├─ traversal_observer (parses \"nodes\"/\"nodes6\" from responses)\n │ ├─ find_data_observer\n │ │ ├─ get_peers_observer (also parses \"values\", bloom filters)\n │ │ └─ get_item_observer (parses v/k/sig/seq)\n │ ├─ put_data_observer\n │ └─ sample_infohashes_observer\n ├─ announce_observer\n ├─ direct_observer\n └─ null_observer\n```\n\n---\n\n## Traversal Algorithms\n\nAll searches inherit from `traversal_algorithm`. The base implements the Kademlia iterative lookup:\n\n1. Seed result set with k closest nodes from routing table\n2. Issue α (default 3) concurrent requests (`add_requests()`)\n3. Each response: add new closer nodes, sort result set\n4. Repeat until k closest nodes all responded or timeout\n5. Call `done()`\n\n| Class | BEP | RPC sent | Callback delivers |\n|-------|-----|----------|-------------------|\n| `find_data` | 5 | `find_node` | closest nodes + write tokens |\n| `get_peers` | 5 | `get_peers` | `vector` (peers) |\n| `get_item` | 44 | `get` | `item`, `bool authoritative` |\n| `put_data` | 44 | first `get`, then `put` | response count |\n| `sample_infohashes` | 51 | `sample_infohashes` | samples, interval, total |\n| `bootstrap` / `refresh` | 5 | `get_peers` (our own ID) | (fires bucket refresh) |\n| `direct_traversal` | — | arbitrary | single `msg` |\n\n`obfuscated_get_peers`: wraps `get_peers`, uses a random target until convergence, then switches to the real info_hash (BEP 5 privacy mode, controlled by `dht_privacy_lookups`).\n\n---\n\n## Message Format (bencode)\n\nAll messages are bencoded dicts with mandatory keys `\"t\"` (transaction id), `\"y\"` (`\"q\"/\"r\"/\"e\"`), optional `\"v\"` (version), `\"ip\"` (external address, BEP 42), `\"ro\"` (read-only, BEP 5).\n\n**Queries** (`y=\"q\"`): `\"q\"` = method name, `\"a\"` = argument dict with `\"id\"` (our node id, 20 bytes).\n\n**Responses** (`y=\"r\"`): `\"r\"` = response dict.\n\n**Errors** (`y=\"e\"`): `\"e\"` = `[error_code, \"message\"]`.\n\n### Method summary\n\n| Method | Key request fields | Key response fields |\n|--------|--------------------|---------------------|\n| `ping` | `id` | `id` |\n| `find_node` | `id`, `target`, `want` | `id`, `nodes`, `nodes6` |\n| `get_peers` | `id`, `info_hash`, `noseed`, `scrape`, `want` | `id`, `token`, `values`\\|`nodes`, `BFpe`, `BFsd` |\n| `announce_peer` | `id`, `info_hash`, `port`, `token`, `implied_port`, `seed` | `id` |\n| `get` (BEP 44) | `id`, `target`, [`seq`] | `id`, `v`, [`k`, `sig`, `seq`] |\n| `put` (BEP 44) | `id`, `v`, [`k`, `sig`, `seq`, `salt`, `cas`] | `id` |\n| `sample_infohashes` | `id`, `target` | `id`, `samples`, `num`, `interval`, `nodes` |\n\nCompact node encoding: 26 bytes per IPv4 node (20-byte id + 4-byte IP + 2-byte port), 38 bytes for IPv6.\n\n---\n\n## Bootstrap\n\n1. Load `dht_state` (saved node IDs + endpoints)\n2. Assign node ID: saved → BEP 42 from external IP → random\n3. Add router nodes from settings (`router.bittorrent.com:6881`, etc.)\n4. Run `bootstrap` traversal against saved/configured endpoints (queries `get_peers` for own ID)\n5. On completion, refresh all non-contiguous buckets\n6. `dht_observer::get_peers()` notifies `session_impl` to query for all active torrents\n\n---\n\n## Integration with `session_impl`\n\n`session_impl` owns `shared_ptr m_dht` and implements `dht_observer`:\n\n| Observer method | When called |\n|-----------------|-------------|\n| `set_external_address()` | DHT detects our external IP via `\"ip\"` field |\n| `get_listen_port(socket)` | Node asks for our announce port |\n| `get_peers(info_hash, peer)` | Another node announced a peer to us |\n| `announce(info_hash, endpoint)` | We received a get_peers for one of our torrents |\n| `on_dht_request(method, msg, reply)` | Custom extension message handler |\n\nSession calls:\n- `dht_tracker::incoming_packet(socket, endpoint, buffer)` — for each received DHT UDP packet\n- `dht_tracker::new_socket(handle)` / `delete_socket(handle)` — listen port changes\n- `dht_tracker::tick()` — periodic maintenance\n\nRelevant `settings_pack` keys:\n\n| Setting | Effect |\n|---------|--------|\n| `enable_dht` | Toggle DHT |\n| `dht_search_branching` | Alpha (concurrent requests) |\n| `dht_max_peers_reply` | Peers returned per get_peers response |\n| `dht_enforce_node_id` | Reject nodes failing BEP 42 check |\n| `dht_restrict_routing_ips` | One node per /24 IPv4 per bucket |\n| `dht_restrict_search_ips` | Sybil mitigation in traversals |\n| `dht_privacy_lookups` | Enable obfuscated_get_peers |\n| `dht_read_only` | Set `ro=1` flag, don't accept queries |\n| `dht_upload_rate_limit` | Response bytes/sec cap |\n| `dht_item_lifetime` | BEP 44 item expiry (0 = never) |\n| `dht_sample_infohashes_interval` | BEP 51 cache TTL |\n\n---\n\n## Constants and Tunables\n\n| Parameter | Value | Notes |\n|-----------|-------|-------|\n| Node ID | 160 bits | SHA-1 |\n| k (bucket size) | 20 | Configurable |\n| Alpha | 3 | Concurrent requests per traversal |\n| RPC full timeout | 15 s | |\n| RPC short timeout | ~6 s | Triggers next request early |\n| Token rotation | ~7 min | Two tokens kept |\n| Bucket refresh | 15 min | Per-bucket, measured from last query |\n| Peer announce validity | 30 min | |\n| DOS block rate | 5 pkt/s | Per source IP |\n| DOS block duration | 5 min | |\n| DOS tracker size | 20 | Most-recent offenders |\n",".claude/rules/disk-cache.md":"---\npaths:\n - \"src/disk_cache*\"\n - \"src/pread_disk_io*\"\n - \"include/libtorrent/aux_/disk_cache*\"\n---\n\n## Disk Cache (`pread_disk_io`)\n\nThe disk cache (`src/disk_cache.cpp`, `include/libtorrent/aux_/disk_cache.hpp`) is used exclusively by `pread_disk_io`. It is a write-back cache for incoming blocks, deferred until pieces are complete or memory pressure forces a flush.\n\n**Data structures:**\n\n- `disk_cache` — the cache itself; holds a `boost::multi_index_container` of `cached_piece_entry` objects, protected by a single `std::mutex`. The container has five indices: by `piece_location` (ordered), by `cheap_to_flush()` (ordered descending), by `need_force_flush()` (ordered descending), by `piece_location` (hashed, for fast lookup), and by `needs_hasher_kick()` (ordered descending).\n- `cached_piece_entry` — one entry per in-flight piece. Contains an array of `cached_block_entry` (one per block), an optional `piece_hasher` (`ph`, v1 only), an optional `block_hashes` array (`sha256_hash[]`, v2 only), and the cursors/flags described below.\n- `cached_block_entry::write_state` — `std::variant` with three mutually exclusive states:\n - `std::monostate` — no data present (the block has never been written, or its data has been fully released).\n - `disk_job*` — a pending write job is queued. The buffer is owned by the job; `take_write_job()` consumes the job during flushing.\n - `disk_buffer_ref` — the write has been flushed. The buffer may be non-null (`has_buf()`, \"held alive\" for future hashing) or null (already released). `is_flushed()` is true in either case and contributes to `flushed_cursor`.\n\n**Cursors:**\n\n- `hasher_cursor` — index of the first block not yet hashed (v1 SHA-1 only; hashing is always contiguous from block 0). Owned by the hashing thread while `hashing_flag` is set.\n- `flushed_cursor` — index of the first block not yet confirmed flushed to disk. Updated under the mutex after each flush completes.\n- `cheap_to_flush()` returns `hasher_cursor - flushed_cursor`: blocks that have been hashed but not yet flushed can be flushed without ever needing read-back.\n\n**Flags on `cached_piece_entry`:**\n\nFlags are stored in `cached_piece_flags flags` (a `bitfield_flag` bitfield). The mutex must be held to read or modify flags; updates go through `view.modify()` (required by `boost::multi_index`). Once a flag is set, the owning thread may access the protected state **without the mutex**:\n\n- `hashing_flag` — set under the mutex by the thread about to hash blocks. While set, that thread has exclusive access to `ph` (the v1 `piece_hasher`) and may advance `hasher_cursor`. No other thread may touch these. Cleared under the mutex when hashing is complete.\n- `flushing_flag` — set under the mutex by the thread about to write blocks to disk. While set, that thread has exclusive access to the block buffers being flushed. No other thread will attempt to flush the same piece. Cleared under the mutex when flushing is complete.\n- `force_flush_flag` — set when the piece has all blocks populated; prioritises this piece for flushing. Cleared once all blocks are flushed.\n- `piece_hash_returned_flag` — set once the piece hash has been computed and returned to the BitTorrent engine. The hash job is also the only signal that a piece is complete: it is what allows `flush_to_disk` to evict the `cached_piece_entry`, and what gets a piece force-flushed when the v1 hasher could not reach the end on its own (pure-v2 pieces, or pieces partially on disk at startup). When `disable_hash_checks` is set the engine still posts the job; the setting is latched into the storage (and into `torrent`) when the storage is created, at add time or when metadata arrives for a magnet link, so a later change never applies to it. In that mode `insert_write` caches the piece as neither v1 nor v2, so nothing is hashed on write, and `do_job(hash)` runs `hash_piece()` with a no-op callback purely for that bookkeeping and returns zero hashes.\n- `v1_hashes_flag` — set if this piece requires v1 SHA-1 hashing (`ph` is allocated).\n- `v2_hashes_flag` — set if this piece requires v2 SHA-256 block hashing (`block_hashes` is allocated).\n- `needs_hasher_kick_flag` — set when a block is inserted and the hasher thread should be woken; cleared when the hasher picks it up. Coalesces multiple insertions into one wakeup.\n- `notify_flushed_flag` — set by a `flush_storage()` caller that is waiting on a piece another thread is mid-flush on; `flush_piece_impl`'s `scope_end` checks it and signals `m_flushing_cv`. Only a wakeup signal supporting a single waiter, not a pin — the waiter re-looks-up the piece after each wakeup because it may have been flushed, hashed, or evicted while it slept.\n\nIn both hashing and flushing cases, the lock is **released** for the duration of the actual I/O (hashing or `pwrite()`), then re-acquired to update cursors/flags and potentially free buffers.\n\n**Deferred clear:** If `try_clear_piece()` is called while `flushing_flag` or `hashing_flag` is set, the clear job is stored in `cached_piece_entry::clear_piece` and executed by the owning thread once it clears its flag.\n\n**Held-alive buffers and the flush/hasher race:**\n\nWhen `flush_piece_impl` flushes a block, it checks `needed_by_hasher = (hashing_flag && block_index >= hasher_cursor)`. If a hasher is mid-snapshot (the `kick_hasher` window between snapshot and re-lock), the pointer to the buffer is in `blocks_storage[]` and must stay live. In that case the block transitions to `disk_buffer_ref{buf}` (non-null), and `m_blocks` is *not* decremented. Otherwise the buffer is freed immediately and `m_blocks` is decremented.\n\n`m_blocks` therefore counts both pending writes and held-alive buffers — both forms of memory pressure. `back_pressure` and `flush_request()` are driven off `m_blocks`.\n\nThe held-alive buffer is reclaimed when `kick_hasher` next advances the cursor past it (its cleanup loop frees blocks in `[old_cursor, new_cursor)` via `take_buf`). However, if v1 SHA-1 is blocked by a missing earlier block, the cursor cannot advance and the buffer stays held until either the missing block arrives or the piece is cleared. The \"drop held-alive\" flush pass (below) handles the case where no further inserts will arrive — without it, back-pressure can pin the cache above the low watermark indefinitely (the back-pressure observer is only notified once `m_blocks` drops to the low watermark, so the producer that tripped back-pressure would never be woken up). Note that v2 hashes for held-alive blocks are typically already in `block_hashes[]` (computed by the kick that triggered the keep-alive), so freeing the buffer only loses the v1 read-back optimization, not v2 hash work.\n\n**Flush strategy (`flush_to_disk`), in order:**\n\n1. Force-flush pass (`view`) — flush pieces with `force_flush` set (piece fully downloaded and hashed); ordered by `need_force_flush()`.\n2. Cheap flush pass (`view2`) — flush blocks in the range `[flushed_cursor, hasher_cursor)` (already hashed, no future read-back needed); pieces ordered by `cheap_to_flush()` descending.\n3. Drop held-alive pass (`view3`) — for each piece without `flushing_flag` or `hashing_flag`, free all blocks where `has_buf()` (held-alive `disk_buffer_ref{buf}`). No I/O — the data is already on disk. Done before the expensive pass because it's free; only skipped if the target is already met. Loses the v1 read-back optimization for these blocks; a subsequent `async_hash` will read them from disk.\n4. Expensive flush pass (`view4`) — flush any dirty blocks even if they haven't been hashed yet, requiring a read-back later to complete the piece hash. Used only when the cache exceeds the high-water mark.\n\nPasses 2-4 run only when `optimistic=false`. The optimistic flush at the top of each disk thread's loop runs only pass 1.\n\n**`flush_storage()`:** Called as a fence job during torrent teardown (`release_files`, `delete_files`, etc.) to flush every cached piece of one storage to disk. For each piece it flushes any still-dirty blocks (completing their write jobs via `flush_piece_impl`). It does **not** free or erase the piece — eviction is left to `flush_to_disk`. Keeping the piece preserves its partial hash state and lets another thread erase it concurrently without racing the flush. If a piece is mid-flush on another thread (`flushing_flag` set), it sets `notify_flushed_flag`, waits on `m_flushing_cv`, and re-looks-up the piece on each wakeup (it may be flushed, hashed, or evicted while waiting); if it is gone, it moves on. A concurrent `flush_storage()` already waiting on a piece (`notify_flushed_flag` already set) is left to that waiter, since the flag supports only one waiter.\n\n**Fences and the cache (`pread_disk_io`):** the general fence contract is in `.claude/rules/disk-io.md`; this is how `pread_disk_io` honours it with a write-back cache.\n\n- **Writes during a fence are queued, not aborted.** `async_write` calls `is_blocked()`, which takes ownership of the job and parks it in the fence's blocked queue. When the fence lowers, `disk_job_fence::job_complete` calls the `repost` callback (still under the fence mutex) once per blocked job in posting order; `add_completed_jobs_impl` inserts each reposted write into the cache there, and serves reposted reads from the cache (a normal disk read would return stale data, since the superseding write is still buffered). Because the resume runs under the fence mutex, a job posted concurrently is ordered behind the whole backlog. Work that can re-enter the fence (hasher kick, flush, completions) is deferred to after the mutex is released.\n- **`async_read` never consults the cache; `add_job` is the single funnel.** `async_read` only allocates a `job::read` (carrying the request verbatim, including an unaligned, block-spanning start) and hands it to `add_job`. `add_job` runs `is_blocked()` first, so a read posted while a fence is up is parked like any other job and cannot bypass the fence to serve stale data ahead of a queued write. The cache is consulted exactly once, by `prepare_read()`, and only after the job is past the fence gate: in `add_job` when the fence is down, or in the `repost` callback when the fence lowers. Both call `prepare_read()`; reposted jobs must **not** re-enter `is_blocked()` (a stacked fence still raises `m_has_fence`, so it would wrongly re-park them behind a later fence), which is why the consult is shared *after* the gate rather than by routing reposts back through `add_job`. `prepare_read()` mutates the job in place, so the single outstanding-job accounting `is_blocked()`/`job_complete()` set up is preserved.\n- **Every write is counted; writes during a fence are queued, not aborted.** `async_write` always routes through `is_blocked()` (never a fast path around it), so the write counts as an outstanding job on the storage whether or not a fence is up -- a fence then waits for buffered writes and their in-flight flush, exactly like a read or hash (see `.claude/rules/disk-io.md`). With no fence up, `is_blocked()` marks the job in-progress and increments the count, and the write is inserted into the cache inline; it stays counted until it is flushed and completed (or aborted). With a fence up, `is_blocked()` instead takes ownership of the job and parks it in the fence's blocked queue. When the fence lowers, `disk_job_fence::job_complete` calls the `repost` callback (still under the fence mutex) once per blocked job in posting order; `add_completed_jobs_impl` inserts each reposted write into the cache there, and serves reposted reads from the cache (a normal disk read would return stale data, since the superseding write is still buffered). Because the resume runs under the fence mutex, a job posted concurrently is ordered behind the whole backlog. Work that can re-enter the fence (hasher kick, flush, completions) is deferred to after the mutex is released.\n- **Reads racing the unblock.** `async_read`'s lock-free cache probe may run just before the backlog's writes land. `add_job` re-checks the cache once the fence is observed down, so a read is never dispatched to disk for a block whose write is sitting buffered.\n- **Forward progress (`m_fence_flush`).** A fence waits for the storage's outstanding jobs (including buffered writes) to reach zero, but nothing else flushes a partial piece under the watermark — so a raised fence hands its storage to a generic thread to flush (`flush_fenced_storages`), draining the writes it is waiting on. `add_fence_job` does this when a fence is raised over outstanding work (`fence_post_none`).\n- **Stacked fences.** When a fence lowers and reposts writes that sit ahead of a still-raised second fence, those partial-piece writes are now in the cache with no flush trigger. `add_completed_jobs_impl` detects a completed fence whose storage still `has_fence()` and re-arms `m_fence_flush` for it, so the next fence's wait can also make progress. (With no generic threads it flushes inline.)\n- **Unaligned reads and `job::partial_read`.** A peer read request whose start is not block-aligned can span two blocks. `prepare_read()` probes both with `disk_cache::get2`: if both blocks are in the cache it serves the whole request; if neither is, the job stays a `job::read` and `do_job(read)` reads the whole span from disk; if exactly one block is in the cache, it copies that side into the buffer and rewrites the job in place into a `job::partial_read` whose `offset`/`buffer_size`/`buffer_offset` describe just the missing side. `do_job(partial_read)` then reads only that side from disk into the rest of the buffer. Because `prepare_read()` runs past the fence gate against the *current* cache, the cached side it copies is never stale, and a `partial_read` therefore reads only from disk and never re-consults the cache. `prepare_read()` is the only place `partial_read` jobs are created -- they are never posted directly.\n",".claude/rules/fuzzers.md":"---\npaths:\n - \"fuzzers/**\"\n - \"tools/gen_corpus.py\"\n---\n\n## Fuzzers\n\n### Build\n\nFuzzers require clang and are built separately from the main library:\n\n```sh\ncd fuzzers && b2 clang\n```\n\nBuilt with ASAN + UBSAN (norecover), debug symbols, asserts on, debug\niterators (`_GLIBCXX_DEBUG`), and WebTorrent enabled (per the default-build\nin `fuzzers/Jamfile`; note that `invariant-checks` is *not* turned on). Each\ntarget produces a standalone binary under `fuzzers/bin//` (e.g.\n`fuzzers/bin/clang-linux-20/...`). The explicit `stage` / `stage-large`\ninstall targets copy the binaries into the `fuzzers/fuzzers/` subdirectory\n(the `fuzzers` in `fuzzers/Jamfile` is relative to the Jamfile's\nown directory).\n\nTwo optional build features (defined in `fuzzers/Jamfile`):\n\n- `debug-logging=on` -- defines `DEBUG_LOGGING=1`, enabling extra alert\n logging in the session-level fuzzers (see peer_conn below).\n- `variant=build_coverage` -- a fast replay build with sanitizers, asserts,\n and invariant checks all off. Combine with `test-coverage=on` to measure\n code coverage from an existing corpus.\n\nThe default-build also pins `1` (`TORRENT_ABI_VERSION=1`,\nthe oldest/most inclusive ABI -- see the table in the top-level `CLAUDE.md`),\nso every fuzzer target, not just `bdecode_entry` (below), compiles and fuzzes\nthe full set of deprecated APIs alongside the current ones. `TORRENT_ABI_VERSION`\nguards in the codebase are exact-match (`== 1`) or lower-bound (`>= N`) checks,\nnever upper-bound, so ABI 1 is a strict superset of what any higher ABI value\ncompiles; this is safe project-wide.\n\n### Running a fuzzer\n\nRun a specific fuzzer against a corpus (single job):\n\n```sh\n./fuzzers/bin/.../peer_conn fuzzers/corpus/peer_conn/\n```\n\nUseful libFuzzer flags:\n\n| Flag | Purpose |\n|------|---------|\n| `-max_len=N` | Cap input size (default: 4096) |\n| `-timeout=N` | Seconds before a single run is declared a timeout (default: 1200) |\n| `-max_total_time=N` | Stop after N seconds total |\n| `-artifact_prefix=dir/` | Write crash/timeout inputs to this directory |\n\n### Running in parallel\n\nlibFuzzer supports in-process parallelism via `-jobs` and `-workers`:\n\n```sh\n./fuzzers/bin/.../peer_conn fuzzers/corpus/peer_conn/ \\\n -jobs=8 -workers=8\n```\n\n- `-jobs=N` — total number of fuzzing jobs to complete before the driver exits\n- `-workers=N` — number of concurrent fuzzer processes (defaults to\n `min(jobs, num_cpus/2)` when omitted)\n\nEach worker writes its own log to `fuzz-N.log` in the current directory.\nCrashes are written to the current directory by default; use\n`-artifact_prefix=crashes/` to redirect them.\n\nAfter parallel runs, merge all worker corpora back into the main corpus to\ndeduplicate and minimize:\n\n```sh\n./fuzzers/bin/.../peer_conn -merge=1 \\\n fuzzers/corpus/peer_conn/ fuzzers/corpus/peer_conn/\n```\n\n(Passing the same directory as both source and destination is safe; libFuzzer\nreads all inputs first, then writes the minimized set back.)\n\n### bdecode_node.cpp vs bdecode_entry.cpp\n\n`bdecode_node.cpp` fuzzes the modern `lt::bdecode(span, ...)`\nfrom `bdecode.hpp`; `bdecode_entry.cpp` fuzzes the deprecated,\n`TORRENT_ABI_VERSION == 1`-only `lt::bdecode()` overloads in\n`bencode.hpp`. They share no code path. `bdecode_entry` degrades to a\nno-op stub if built at any other ABI, rather than failing to compile.\n\n### Structure\n\nEvery fuzzer in `fuzzers/src/` implements `LLVMFuzzerTestOneInput`; the\nsession-level fuzzers additionally implement `LLVMFuzzerInitialize`:\n\n- `LLVMFuzzerTestOneInput(data, size)` — implemented by every fuzzer;\n called for every fuzz input; must be stateless with respect to prior\n inputs. Returns 0 on success, -1 to signal libFuzzer that the input was\n uninteresting (used for timeouts, not errors).\n\n- `LLVMFuzzerInitialize(argc, argv)` — implemented only by the\n session-level fuzzers (`add_torrent`, `pe_conn`, `peer_conn`,\n `piece_layers`, `rtc_peer_conn`, `ut_metadata`, `ut_pex`, `web_seed`).\n Called once at startup to set up global state (sessions, sockets,\n servers) that persists across all test inputs. The many pure-function\n fuzzers (e.g. `bdecode_node`, `parse_magnet_uri`, `gzip`,\n `torrent_info`) have no global state and omit it.\n\n### peer_conn.cpp\n\nConnects to a live libtorrent session as a BitTorrent peer and sends\nfuzz-controlled protocol messages.\n\nThe session and torrent setup is shared via `peer_session.hpp`\n(`configure_fuzz_session()`, `make_fuzz_torrent_params()`, and the\n`peer_fuzz_session` struct `g_fz`), and reused by the rtc_peer_conn,\nut_metadata, and ut_pex fuzzers.\n\n**Session setup (`LLVMFuzzerInitialize`):**\n\n- Creates a session on `127.0.0.1:0` with all outbound connections and\n ancillary services (outgoing TCP/uTP, DHT, LSD, UPnP, NAT-PMP, IP\n notifier) disabled, encryption disabled, and all peer timeouts set to\n 1 second.\n- Adds a **hybrid (v1 + v2)** torrent built from four files (a 0-byte\n file, a 100-byte file, a 50 KiB file, and a 97 MiB + 200 KiB file).\n After canonicalization (alphabetical sort + inserted pad files) this is\n exactly 100 pieces of 1 MiB. Piece hashes (SHA-1) and per-file block\n hashes (SHA-256) are filled with dummy values. `save_path` is `.` and\n no data is on disk, so the torrent stays in downloading state.\n- Waits for both the TCP `listen_succeeded_alert` (recording\n `g_fz.listen_port`) and the `torrent_resumed_alert` before returning.\n- Records `g_fz.info_hash` (an `info_hash_t` holding both the v1 SHA-1\n and v2 SHA-256 info-hashes).\n- Also adds a **second** torrent as a **magnet** (`info_hashes` only, no\n `ti`), built via the shared `add_fuzz_magnet(false, 'a')` helper (which\n wraps `build_fuzz_torrent()` from `peer_session.hpp`) from the identical\n file layout but with no v2 (SHA-256) hashes, so it's v1-only. Its\n info-hash and pre-encoded ut_metadata piece message (from\n `torrent_info::info_section()`) are stored in\n `g_v1_only_hash`/`g_v1_only_metadata_piece`. Because this metadata is\n fixed at startup rather than derived from fuzz input, delivering it via\n ut_metadata (see below) always succeeds, unlike a corpus byte sequence,\n which would have to find a SHA-1 preimage to ever make\n `torrent::set_metadata()` return true. Without this second torrent, the\n whole torrent-acquires-metadata code path (`set_metadata()`,\n `initialize_merkle_trees()`, `on_metadata_impl()`) would be unreachable:\n the first, pre-built torrent always has valid metadata, so that code\n never runs for it.\n- Also adds a **third** torrent, added as a **hybrid (v1 + v2) magnet**\n (`info_hashes` holds both the v1 and v2 hash, no `ti`), built via the\n same `add_fuzz_magnet(true, 'c')` helper from the identical file layout\n with a different dummy-hash fill byte (`'c'` instead of the default\n `'a'`) so it gets its own info-hash instead of colliding with `g_fz`'s.\n Its info-hash and pre-encoded ut_metadata piece message are stored in\n `g_v2_hybrid_hash`/`g_v2_hybrid_metadata_piece`. This is the counterpart\n to the v1-only magnet: metadata resolves to a genuine hybrid, so a peer's\n speculative v2 claim turns out honest and must survive metadata\n resolution rather than being corrected, exercising the same code as\n the v1-only magnet from the opposite direction, to guard against the\n fix over-triggering on legitimate v2 peers. More variants (a different\n file layout, etc.) can be added the same way if a future case needs\n one.\n\n**Per-input flow (`LLVMFuzzerTestOneInput`):**\n\n1. Opens a TCP connection to `127.0.0.1:g_fz.listen_port`.\n2. Sends a 68-byte BT handshake:\n - `data[0..7]` are used as the extension-flags (reserved) bytes.\n - Bits for BEP 10 (byte 5, bit 0x10) and BEP 6 FAST (byte 7, bit\n 0x04) are always forced on so those handlers are always reachable.\n - Byte 0 bit 0x01 selects the peer's protocol version, for torrents\n that have a v1 and a v2 hash to choose between (torrent selector 0\n and 2 below): clear = v2 (advertises the torrent's\n `info_hashes().get_best()`, `protocol_v2` true), set = v1\n (advertises the torrent's `info_hashes().v1`, `protocol_v2` false).\n - Byte 0 bits 0x02 and 0x04 form a 2-bit torrent selector: 0 = the\n first, pre-built torrent (metadata already valid, bit 0x01 applies\n as above); 1 = the second, v1-only magnet (always presents\n `g_v1_only_hash.v1`; bit 0x01 doesn't apply since there's no v2\n hash to choose between); 2 = the third, hybrid magnet (bit 0x01\n applies exactly as it does for selector 0); 3 = unused, falls back\n to selector 0. Every other flag bit, including byte 7 bit 0x10 (the\n v2-upgrade bit), is independent of the torrent selector and free to\n combine with any of them, e.g. a handshake claim that doesn't match\n what the connected torrent's info-hash actually supports.\n - Peer-ID is all zeros.\n3. Sends a fixed BEP 10 extended handshake (`k_extended_handshake`,\n ext_id 0) registering:\n\n | Extension | ext_id |\n |-------------|--------|\n | ut_pex | 1 |\n | ut_metadata | 2 |\n | upload_only | 3 |\n\n4. If the torrent selector picked the v1-only or hybrid magnet, sends two\n more fixed (not fuzz-derived) messages before continuing: a second\n extended handshake (BEP 10 supports re-negotiation) announcing\n `metadata_size`, which makes the ut_metadata plugin queue a request\n for piece 0, then a ut_metadata piece message carrying that magnet's\n metadata (`g_v1_only_metadata_piece` or `g_v2_hybrid_metadata_piece`),\n so `torrent::set_metadata()` succeeds deterministically on every run\n that picks either branch.\n5. Parses `data[8..]` as a sequence of messages (see **Corpus wire\n format** below) and sends each with a correct 4-byte BT length\n prefix. This is unchanged regardless of which torrent step 2 picked,\n so the same fuzzed message space (including BEP 78 hash messages)\n now also runs against a torrent whose metadata was just acquired\n rather than always pre-loaded.\n6. Closes the socket and waits for a `peer_error_alert` or\n `peer_disconnected_alert` (3-second timeout).\n\n**DEBUG_LOGGING:** defined by `fuzzers/Jamfile` via the `debug-logging`\nfeature (default `off` -> `DEBUG_LOGGING=0`). Build with\n`b2 clang debug-logging=on` to set it to 1, which adds `peer_log` alerts to\nthe alert mask and makes `wait_for_disconnect` print the last 10 interesting\nalerts (connect and routine peer-log noise excluded) before each disconnect\n-- useful for tracing which message paths were hit.\n\n### Corpus wire format (peer_conn)\n\nEach corpus file is binary-structured:\n\n```\n[8 bytes : extension flags]\n[repeated until EOF:\n 1 byte : msg_type\n 2 bytes : payload_len (big-endian, max 65535)\n N bytes : payload (N = min(payload_len, bytes_remaining))\n]\n```\n\nMinimum file size: 11 bytes (8 flag bytes + 3-byte message header).\n\nFor **msg_type 20** (BEP 10 extended protocol), the first byte of the\npayload is the extended message ID and the rest is the extension body.\nThe fuzzer splits this automatically before calling\n`send_extended_message`.\n\n**Extension flag byte layout** (8 reserved bytes, big-endian):\n\n| Byte | Bit | Protocol |\n|------|------|---------------------|\n| 0 | 0x01 | protocol version, when the selected torrent has a v1 and a v2 hash (set = v1 SHA-1, clear = v2) |\n| 0 | 0x02 + 0x04 | 2-bit torrent selector: 0 = pre-built hybrid, 1 = v1-only magnet, 2 = hybrid magnet, 3 = unused (falls back to 0) |\n| 5 | 0x10 | BEP 10 extended |\n| 7 | 0x04 | BEP 6 FAST |\n| 7 | 0x01 | BEP 5 DHT |\n| 7 | 0x10 | v2-upgrade bit (peer claims v2 support) |\n\n**libtorrent's fixed incoming ext_ids** (from `bt_peer_connection.hpp`\nenum; these are the IDs libtorrent expects in messages FROM us):\n\n| ext_id | Handler | Notes |\n|--------|------------------|--------------------------------------------|\n| 0 | extended HS | re-negotiation supported |\n| 1 | ut_pex plugin | as registered in `k_extended_handshake` |\n| 2 | ut_metadata | as registered in `k_extended_handshake` |\n| 3 | upload_only | `upload_only_msg` constant |\n| 4 | holepunch | `holepunch_msg`; requires `m_holepunch_id` |\n| 7 | dont_have | `dont_have_msg`; 4-byte piece index |\n| 8 | share_mode | `share_mode_msg` |\n| other | disconnect | `errors::invalid_message` |\n\nHolepunch (ext_id 4) silently returns if `m_holepunch_id == 0`. To\nreach the holepunch handler body, a corpus file must first send an\nadditional extended handshake (ext_id 0) that includes\n`\"ut_holepunch\": ` in its `m` dict.\n\n**BEP 78 hash messages** (21 = hash_request, 22 = hashes, 23 =\nhash_reject) require the peer's `protocol_v2` flag (set from the\nhandshake) and, separately, the connected torrent's info-hash to actually\nhave a v2 component, since a peer can set the flag on a torrent that never\ngains one (see the v1-only magnet above), so both are checked. Either\nfailing disconnects with `errors::invalid_message`.\n\n### Corpus generation tool\n\n`tools/gen_corpus.py` generates seed corpora for all session-level fuzzers.\nRun it from the repo root:\n\n```sh\npython3 tools/gen_corpus.py\n```\n\nThis writes into:\n- `fuzzers/corpus/peer_conn/`\n- `fuzzers/corpus/natpmp/`\n- `fuzzers/corpus/udp_tracker/`\n- `fuzzers/corpus/upnp/`\n- `fuzzers/corpus/pe_crypto_state/`\n- `fuzzers/corpus/pe_conn/`\n- `fuzzers/corpus/ut_metadata/`\n- `fuzzers/corpus/ut_pex/`\n- `fuzzers/corpus/web_seed/`\n- `fuzzers/corpus/utp_stream/`\n- `fuzzers/corpus/piece_layers/`\n- `fuzzers/corpus/torrent_info/`\n- `fuzzers/corpus/lsd/`\n- `fuzzers/corpus/add_torrent/`\n\n**Sync requirement:** the Python constants `EXT_UT_PEX`,\n`EXT_UT_METADATA`, `EXT_UPLOAD_ONLY`, and `FUZZER_EXT_HANDSHAKE` in\nthe tool must match `k_extended_handshake` in `peer_conn.cpp`. If the\nC++ extended handshake is changed, update both files together.\n\nThe tool also uses `NUM_PIECES = 100`, `PIECE_SIZE = 1 MiB`, and\n`BLOCK_SIZE = 16 KiB` -- these must match the torrent that\n`make_fuzz_torrent_params()` (equivalently `build_fuzz_torrent(true)`) in\n`fuzzers/src/peer_session.hpp` builds. `build_fuzz_torrent(false)` builds\nthe second, v1-only torrent, and `build_fuzz_torrent(true, 'c')` builds\nthe third, hybrid-magnet torrent, both from the identical file layout, so\nthese constants apply to them too.\n\nA handful of seeds set byte 0 bits 0x02/0x04 to connect to the second or\nthird torrent instead of the first, combined with other flag bits (e.g.\nthe v2-upgrade bit) and a follow-up message. These exist to give the\nfuzzer a foothold in those branches, not to exhaustively cover them;\ncoverage-guided mutation explores the rest.\n\n### pe_conn.cpp\n\nConnects to a live libtorrent session and drives the full PE (protocol\nencryption / MSE) handshake as the initiating (client) side. Requires\nthe session to have `in_enc_policy = pe_enabled` and\n`allowed_enc_level = pe_both`, which pe_conn's `LLVMFuzzerInitialize`\nsets.\n\nThe fuzzer performs a real DH key exchange (so the server can locate the\ntorrent), then uses fuzz-controlled bytes to drive all encrypted fields.\nEvery server-side error path is reachable by setting the appropriate bit\nin the flags byte.\n\n**Session setup (`LLVMFuzzerInitialize`):**\n\n- Same hybrid v1+v2 torrent as peer_conn (100 pieces, 1 MiB each).\n- `in_enc_policy = pe_enabled`, `allowed_enc_level = pe_both`.\n- All outbound services disabled (same as peer_conn).\n\n**Per-input flow (`LLVMFuzzerTestOneInput`):**\n\n1. Connect TCP socket to session.\n2. Generate a fresh DH key pair; send DH public key + PadA (from fuzz bytes).\n3. Read the server's DH public key (96 bytes, blocking).\n4. Compute shared secret, RC4 keys, sync hash, and obfuscated SKEY hash.\n5. Build the PE payload with fuzz-controlled fields and optional corruptions.\n6. Send the payload, close socket, wait for disconnect.\n\n**Corpus wire format:**\n\n```\nByte 0 : pad_a_size (PadA bytes before sync hash, 0-255)\nByte 1 : flags\n bit 0 : corrupt_sync_hash -> sync_hash_not_found\n bit 1 : corrupt_skey -> invalid_info_hash\n bit 2 : corrupt_vc -> invalid_encryption_constant\n bits 3-4 : crypto_provide\n 00 -> pe_both (0x03)\n 01 -> pe_plaintext (0x01)\n 10 -> pe_rc4 (0x02)\n 11 -> 0x00 (invalid) -> unsupported_encryption_mode\nByte 2 : len_pad_c * 2 (encrypted padding, 0-510 bytes)\nByte 3 : len_ia (IA size 0-255; server rejects > 68)\nBytes 4+ : first pad_a_size bytes = PadA content\n then up to len_ia bytes = IA (BT handshake portion)\n```\n\nSee `.claude/rules/protocol-encryption.md` for details of the PE protocol.\n",".claude/rules/python-bindings.md":"---\npaths:\n - \"bindings/python/**\"\n---\n\n## Python Bindings\n\nSource in `bindings/python/src/*.cpp`; one `.cpp` file per exposed subsystem (e.g. `session.cpp`, `torrent_handle.cpp`, `alert.cpp`). Built as a Python extension module (`libtorrent.so`) using boost.python.\n\n**Building:**\n```sh\ncd bindings/python && b2\n```\nCMake also builds them when `-Dpython-bindings=ON` is passed.\n\n**Testing:**\n\nTests live in `bindings/python/tests/*_test.py` and are run with pytest:\n```sh\ncd bindings/python && python -m pytest tests/\n```\nThere is a roughly one-to-one correspondence between `src/foo.cpp` and `tests/foo_test.py`. Tests must be isolated — no real network access, no persistent filesystem side-effects. Use `get_isolated_settings()` from `tests/lib.py` when constructing a session, and `tempfile.TemporaryDirectory()` for any disk I/O. See `bindings/python/tests/guidelines.txt` for the full testing conventions.\n\n**Type stubs:**\n\nThe type stub file is `bindings/python/libtorrent/__init__.pyi`. It was originally bootstrapped with `stubgen` but is now **maintained entirely by hand** — do not regenerate it with stubgen, as that would lose the carefully written annotations. When adding a new binding, add the corresponding type annotation to this file manually. The header comment in the file documents the conventions used (e.g. how boost.python enums, positional-only args, and the `_BoostBaseClass` metaclass are represented).\n\n**Design philosophy — classes over dicts:**\n\nPrefer exposing real C++ classes via boost.python rather than converting them to plain Python dicts. Earlier versions of the bindings returned plain `dict` objects for things like peer info and torrent status; these have been replaced with proper bound classes (`peer_info`, `torrent_status`, etc.) that mirror the C++ types. When adding new bindings, follow this pattern: expose the C++ struct/class directly rather than unpacking it into a dict.\n\nNote: boost.python strictly requires `dict` and `list` (not duck-typed `Mapping`/`Sequence` abstractions) for function arguments that cross the C++ boundary. This is a boost.python limitation and not a design choice; the type stubs reflect this with `dict` and `list` rather than `Mapping` or `Sequence`.\n",".claude/rules/strong-types.md":"---\npaths:\n - \"include/libtorrent/units.hpp\"\n - \"include/libtorrent/flags.hpp\"\n - \"include/libtorrent/index_range.hpp\"\n - \"include/libtorrent/**\"\n - \"src/**\"\n---\n\n## Strong Types\n\nThe codebase avoids using raw `int` or bare integers for indices and flags. Instead, use the strong type and flag facilities.\n\n**Index types** (`include/libtorrent/units.hpp`):\n\n`aux::strong_typedef` wraps an integer so it is incompatible with other integers and other strong types. Arithmetic with raw integers is not allowed; only arithmetic with the same type or its `diff_type` is permitted.\n\nTo define a new index type:\n```cpp\nusing my_index_t = aux::strong_typedef;\n```\n\nTo convert to/from the underlying integer, use an explicit cast:\n```cpp\npiece_index_t p{42};\nint raw = static_cast(p);\n```\n\nTo iterate over a range of indices, use `index_range` (`include/libtorrent/index_range.hpp`):\n```cpp\nfor (piece_index_t i : index_range{begin, end}) { ... }\n```\n\n**Flag types** (`include/libtorrent/flags.hpp`):\n\n`flags::bitfield_flag` wraps an unsigned integer as a type-safe bitfield. Flags of different types cannot be combined. Individual bit constants are defined using the `_bit` user-defined literal:\n\n```cpp\nusing torrent_flags_t = flags::bitfield_flag;\nstatic constexpr torrent_flags_t seed_mode = 0_bit;\nstatic constexpr torrent_flags_t upload_mode = 1_bit;\n```\n\nTo define a new flag type:\n```cpp\nusing my_flags_t = flags::bitfield_flag;\n```\n\nFlag values support `|`, `&`, `^`, `~`, `|=`, `&=`, `^=`, and `operator bool()` (true if any bit is set). To test a flag:\n```cpp\nif (flags & torrent_flags::seed_mode) { ... }\n```\n",".claude/rules/piece-picker.md":"---\npaths:\n - \"src/piece_picker.cpp\"\n - \"include/libtorrent/aux_/piece_picker.hpp\"\n - \"test/test_piece_picker.cpp\"\n---\n\n## Piece Picker\n\nThe piece picker (`src/piece_picker.cpp`, `include/libtorrent/aux_/piece_picker.hpp`) decides\nwhich blocks to request from which peers. It tracks availability (how many peers have each piece),\nper-piece priority, and the state of every block in every partially-downloaded piece.\n\n### Core data structures\n\n**`piece_pos`** — 8 bytes per piece, tightly packed:\n- `peer_count` (26 bits) — number of peers that have this piece\n- `download_state` (3 bits) — one of the `download_queue_t` constants below\n- `piece_priority` (3 bits) — 0 = filtered (dont-download), 1–7 user priority (4 = default)\n- `index` (`prio_index_t`) — position in `m_pieces`, or `we_have_index` (-1) if we have it\n\n**`m_piece_map`** — `aux::vector`, one entry per piece.\n\n**`m_pieces`** — flat `aux::vector` of all pickable (not filtered,\nnot have) pieces, sorted by effective priority. Within each priority band pieces are in random\norder (rarest-first is approximated by the priority formula, not by sorting on availability alone).\n\n**`m_priority_boundaries`** — cumulative end-indices into `m_pieces`. Priority band `p` spans\n`[m_priority_boundaries[p-1], m_priority_boundaries[p])`. Priority 0 starts at index 0.\n\n**`m_downloads`** — `aux::array` of 4 sorted `vector` (one per download\ncategory), indexed by `download_queue_t`:\n- `piece_downloading` (0) — some blocks still open (not yet requested)\n- `piece_full` (1) — all blocks requested, at least one not yet writing/finished\n- `piece_finished` (2) — all blocks writing or finished\n- `piece_zero_prio` (3) — downloading but priority = 0\n\n`piece_downloading_reverse` and `piece_full_reverse` are pseudo-states stored in `download_state`\nto de-prioritize pieces only requested from reverse peers; they map to the same download vectors\nas their non-reverse counterparts.\n\n**`downloading_piece`** — one per in-flight piece:\n- `info_idx` (`uint16_t`) — index into `m_block_info`; the slice `[info_idx * blocks_per_piece, (info_idx+1) * blocks_per_piece)` holds the block states for this piece\n- `finished`, `writing`, `requested` (15 bits each) — block state counters\n- `passed_hash_check` (1 bit) — hash job returned OK (piece may not yet be on disk)\n- `locked` (1 bit) — blocks cannot be picked; set during error recovery, cleared by `restore_piece()`\n- `hashing` (1 bit) — outstanding hash request in flight\n\n**`m_block_info`** — flat `aux::vector`. Each downloading piece owns\n`blocks_per_piece()` consecutive entries identified by `info_idx`. Free ranges are recycled\nvia `m_free_block_infos` (a free-list of `uint16_t` indices).\n\n**`block_info`** — per block:\n- `peer` — last peer to request/write this block\n- `num_peers` (14 bits) — peers with this block outstanding\n- `state` (2 bits) — `state_none → state_requested → state_writing → state_finished`\n\n### Priority formula\n\n`piece_pos::priority(picker)` returns -1 (not pickable) if: filtered, have, no availability\n(`peer_count + m_seeds == 0`), `piece_full`, or `piece_finished`.\n\nOtherwise:\n```\npriority = availability * (priority_levels - piece_priority) * prio_factor + adjustment\n```\nwhere `priority_levels = 8`, `prio_factor = 3`, and:\n- `adjustment = -3` for actively downloading pieces (non-reverse)\n- `adjustment = -2` for open pieces\n- `adjustment = -1` for reverse-downloading pieces\n\nLower value = picked first. The `prio_factor = 3` creates three sub-levels per availability\ntier, letting downloading pieces beat open ones at the same availability.\n\n### Seeds optimization\n\nPeers with all pieces are tracked in `m_seeds` rather than incrementing every `peer_count`.\nTrue availability of a piece is `peer_count + m_seeds`. When a seed sends DONT_HAVE,\n`break_one_seed()` decrements `m_seeds` and increments every `piece_pos::peer_count`.\n\n### Lazy rebuild (`m_dirty`)\n\n`m_pieces` and `m_priority_boundaries` are rebuilt lazily. When `m_dirty` is true, they are\nstale; `update_pieces()` (called automatically before any pick) rebuilds them:\n1. Walk `m_piece_map`, count pieces per priority band → `m_priority_boundaries` holds deltas\n2. Make boundaries cumulative, resize `m_pieces`\n3. Fill `m_pieces` using per-piece relative offsets stored in `piece_pos::index`\n4. Shuffle each priority band randomly\n5. Fix up `piece_pos::index` to point back into the final positions in `m_pieces`\n\nBulk operations (e.g. `inc_refcount(bitfield)` touching many pieces) set `m_dirty = true`\nrather than updating incrementally. For small bitfield changes (< 50 pieces), the picker\nupdates incrementally via `update()` / `add()` / `remove()` without dirtying.\n\n### Sequential download cursors\n\n`m_cursor` — lowest piece index not yet had.\n`m_reverse_cursor` — one past the highest piece index not yet had (all pieces from here to end\nare had). `set_sequential_range(first, last)` constrains the cursor window.\n\n### Piece lifecycle\n\n```\nopen → downloading → full → finished → (removed, piece_pos::have set)\n```\n\n- `mark_as_downloading(block, peer)` — block: none→requested; piece may transition open→downloading\n- `mark_as_writing(block, peer)` — block: requested→writing\n- `mark_as_finished(block, peer)` — block: writing→finished; calls `update_piece_state()` which\n may advance the piece from downloading→full→finished\n- `piece_passed(index)` — hash check passed; sets `passed_hash_check`, calls `account_have()`,\n calls `piece_flushed()` if all blocks are also finished\n- `piece_flushed(index)` — piece is on disk; removes from `m_downloads`, removes from `m_pieces`\n via `remove()`, sets `piece_pos::index = we_have_index`\n- `restore_piece(index, blocks)` — after hash failure or write error; clears `locked`, optionally\n resets specific blocks back to `state_none`, re-opens the piece for downloading\n\n### `pick_pieces` options\n\n`picker_options_t` flags passed to `pick_pieces()`:\n- `rarest_first` — pick lowest-availability pieces (the default; driven by the priority formula)\n- `reverse` — pick most-common first, or last pieces if sequential\n- `sequential` — pick within the `[m_cursor, m_reverse_cursor)` window in order\n- `on_parole` — only pick pieces exclusively requested by this peer (after suspected bad data)\n- `prioritize_partials` — prefer pieces already in `m_downloads` over fresh ones\n- `piece_extent_affinity` — create affinity for 4 MiB extents of adjacent pieces\n- `align_expanded_pieces` — align contiguous-block expansion to natural boundaries\n\nAuto partial prioritization: if `num_partials > num_peers * 3 / 2` or\n`num_partials * blocks_per_piece > 2048`, `prioritize_partials` is forced on and\n`prefer_contiguous_blocks` is set to 0 to avoid unbounded partial piece growth.\n\n`prefer_contiguous_blocks` (int, not a flag) — request this many consecutive blocks from the\nsame piece; used by web peers to pipeline larger requests.\n",".claude/rules/protocol-encryption.md":"# Protocol Encryption (MSE/PE) Handshake\n\nlibtorrent implements Message Stream Encryption (MSE), also called Protocol\nEncryption (PE). The implementation lives in `src/bt_peer_connection.cpp` and\n`src/pe_crypto.cpp` (crypto primitives in `include/libtorrent/aux_/pe_crypto.hpp`).\n\n## Overview\n\nMSE provides obfuscation (not strong security) against deep-packet inspection.\nIt uses a Diffie-Hellman key exchange to derive RC4 stream keys and wraps the\nBitTorrent handshake. The torrent info hash is obfuscated in the handshake so\nthe connection cannot be trivially identified.\n\nAfter the handshake, the connection body is RC4-encrypted only if\n`crypto_select` ended on `pe_rc4`. If `pe_plaintext` was selected, only the\nhandshake is obfuscated and post-handshake traffic is sent in the clear. This\nis tracked by the `m_rc4_encrypted` flag on `bt_peer_connection` and gates the\nencrypt/decrypt path in every send and receive routine.\n\n## Settings\n\n| Setting | Meaning |\n|---------|---------|\n| `pe_forced` | Always use PE (outgoing) / require PE (incoming) |\n| `pe_enabled` | Use PE if peer supports it, fall back to plaintext |\n| `pe_disabled` | Never use PE |\n\n`settings_pack::out_enc_policy` controls outgoing connections.\n`settings_pack::in_enc_policy` controls acceptance of incoming connections.\n`settings_pack::allowed_enc_level` is a bitmask (`pe_plaintext=1`, `pe_rc4=2`, `pe_both=3`).\n`settings_pack::prefer_rc4` selects RC4 over plaintext when both are offered.\n\n## Handshake States\n\nThe state machine lives in `bt_peer_connection::on_receive_impl()`. The\nhandshake differs between the initiating side (outgoing) and the accepting\nside (incoming).\n\n### Incoming (server) path\n\n1. **`read_pe_dhkey`** (96 bytes) -- read client's DH public key, respond\n with our DH public key + random PadB, transition to `read_pe_synchash`.\n2. **`read_pe_synchash`** (scans up to 512 bytes of PadA + the 20-byte hash) --\n scan received bytes for `hash('req1', S)` where S is the shared secret. The\n client may insert up to 512 bytes of PadA before this hash. Failure:\n `sync_hash_not_found`.\n3. **`read_pe_skey_vc`** (28 bytes) -- read the obfuscated SKEY hash (20\n bytes) and the encrypted VC (8 bytes). The SKEY hash is\n `hash('req2', info_hash) XOR hash('req3', S)` which reveals which torrent\n the client wants. Failure if torrent not found: `invalid_info_hash`.\n Failure if VC decrypts to non-zero: `invalid_encryption_constant`.\n4. **`read_pe_cryptofield`** (6 bytes) -- read encrypted\n `crypto_provide(4) + len_pad_c(2)`. Server selects a crypto mode and\n sends back its own VC + `crypto_select + pad`. Failure if no mode overlaps:\n `unsupported_encryption_mode`. Failure if `len_pad_c > 512`:\n `invalid_pad_size`.\n5. **`read_pe_pad`** (`len_pad_c + 2` bytes) -- skip encrypted padding, read\n `len_ia` (2 bytes).\n6. **`read_pe_ia`** (`len_ia` bytes) -- read Initial Application data (IA),\n the encrypted start of the BitTorrent handshake. Failure if `len_ia > 68`:\n `invalid_encrypt_handshake`.\n7. Falls through to **`read_protocol_identifier`** (regular BT handshake).\n\n### Outgoing (client) path\n\n1. **`on_connected`** -- send DH public key + random PadA,\n transition to `read_pe_dhkey`.\n2. **`read_pe_dhkey`** (96 bytes) -- read server's DH public key, then send\n sync hash + SKEY hash + encrypted(VC + crypto_provide + len_pad + PadC +\n len_IA) + encrypted(IA = BT handshake). Also send the BT handshake\n encrypted as IA. Transition to `read_pe_syncvc`.\n3. **`read_pe_syncvc`** (scans up to 512 bytes of PadB + the 8-byte VC) --\n scan for the encrypted 8-byte verification constant (all zeros encrypted\n with RC4). Failure if not found within the 512-byte pad window:\n `invalid_encryption_constant`.\n4. **`read_pe_cryptofield`** (6 bytes) -- read server's `crypto_select(4) +\n len_pad(2)`. Failure if selected mode not in our `allowed_enc_level`:\n `unsupported_encryption_mode_selected`.\n5. **`read_pe_pad`** (`len_pad` bytes) -- skip server's padding.\n6. Falls through to **`read_protocol_identifier`** (regular BT handshake).\n\n## Crypto Primitives\n\n### DH Key Exchange (`dh_key_exchange`)\n\n768-bit Diffie-Hellman with a fixed prime (MSE standard prime from BEP 10/MSE\nspec). `key = 2^random % prime`.\n\n- `dh_key_exchange()` -- generate local key pair\n- `compute_secret(remote_pubkey)` -- compute shared secret S; also computes\n `m_xor_mask = hash('req3', S)` used for SKEY obfuscation\n- `get_local_key()` -- 96-byte DH public key to send\n- `get_secret()` -- shared secret S\n- `get_hash_xor_mask()` -- `hash('req3', S)` for SKEY obfuscation\n\n### RC4 Key Derivation\n\nFor outgoing connections (client):\n```\nencrypt_key = hash(\"keyA\", S, SKEY)\ndecrypt_key = hash(\"keyB\", S, SKEY)\n```\nFor incoming connections (server):\n```\nencrypt_key = hash(\"keyB\", S, SKEY)\ndecrypt_key = hash(\"keyA\", S, SKEY)\n```\nwhere SKEY is the SHA-1 info hash of the torrent (or first 20 bytes of SHA-256\nfor v2 torrents). The RC4 stream is initialized by discarding the first 1024\nbytes (standard MSE requirement).\n\n### Message Layout (client -> server)\n\n```\n[96 bytes] client DH public key\n[0-512 bytes] PadA (random, encrypted sync hash follows)\n[20 bytes] sync_hash = hash('req1', S)\n[20 bytes] skey_obfusc = hash('req2', SKEY) XOR hash('req3', S)\nencrypt([8 bytes] VC = 0x00 * 8\n [4 bytes] crypto_provide (bit flags: 1=plaintext, 2=RC4)\n [2 bytes] len_PadC\n [len_PadC] PadC (random)\n [2 bytes] len_IA)\nencrypt([len_IA bytes] IA = start of BT handshake, always RC4 encrypted)\n```\n\n### Message Layout (server -> client)\n\n```\n[96 bytes] server DH public key\n[0-512 bytes] PadB (random)\nencrypt([8 bytes] VC = 0x00 * 8\n [4 bytes] crypto_select (chosen mode: 1 or 2)\n [2 bytes] len_PadD\n [len_PadD] PadD (random))\n[regular BT handshake, encrypted or not depending on crypto_select]\n```\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `src/pe_crypto.cpp` + `include/libtorrent/aux_/pe_crypto.hpp` | DH exchange, RC4 handler, encryption_handler |\n| `src/bt_peer_connection.cpp` | PE state machine and write helpers |\n| `fuzzers/src/pe_conn.cpp` | Session-level fuzzer for the PE handshake |\n| `fuzzers/src/pe_crypto_state.cpp` | Low-level fuzzer for DH + RC4 primitives |\n\n## Error Codes\n\n| Error | Trigger |\n|-------|---------|\n| `sync_hash_not_found` | sync hash not found within 512-byte window |\n| `invalid_info_hash` | SKEY hash doesn't match any known torrent |\n| `invalid_encryption_constant` | VC decrypts to non-zero bytes |\n| `unsupported_encryption_mode` | No overlap in crypto_provide vs allowed_enc_level |\n| `unsupported_encryption_mode_selected` | Server selected a mode not in our allowed list |\n| `invalid_pad_size` | len_pad_c > 512 |\n| `invalid_encrypt_handshake` | len_ia > 68 |\n",".claude/rules/v2-torrents.md":"# BitTorrent v2 Torrents (BEP 52)\n\n## Info Dict Structure\n\nV2 uses a nested **`file tree`** dict instead of a flat `files` list:\n- Keys are filename path components; non-leaf = directory (nested dict)\n- Leaf nodes have empty-string key `\"\"` → file metadata dict with `length`, `pieces root`, etc.\n- **`pieces root`** (32 bytes): root of the per-file merkle tree (SHA-256)\n- **`meta version`**: integer `2` identifies v2 metadata\n- **`piece layers`**: (outside info dict) maps `pieces root` → concatenated piece-layer hashes\n\nV2 torrents have a **SHA-256 info hash** (vs v1's SHA-1). `torrent_info::m_info_hash` holds both.\n\n## Per-File Merkle Tree\n\nEach file has an independent merkle tree (SHA-256, not SHA-1):\n\n```\nroot (pieces root)\n └── interior nodes\n └── piece layer ← hash of one piece's block subtree\n └── block layer (leaf) ← hash of 16 KiB block\n```\n\n- **Block** = 16 KiB (fixed), the unit of transfer\n- **Piece** = `piece_length` bytes; must be a power-of-2 multiple of 16 KiB\n- Tree is **padded to a power of 2** in leaf count using `merkle_pad()` (zeros hashed up)\n- Interior nodes: `parent = SHA-256(left_child || right_child)`\n- Flat array indexing: node at layer `l`, offset `o` → index `(1 << l) - 1 + o`; parent = `(i-1)/2`\n\nKey helpers in `src/merkle.cpp`:\n- `merkle_num_leafs()`, `merkle_num_nodes()`, `merkle_num_layers()`\n- `merkle_fill_tree()` — compute interior nodes from leaves\n- `merkle_validate_and_insert_proofs()` — validate a proof chain against a known root\n- `merkle_pad()` — padding hash for odd tree levels\n\n## merkle_tree Class (`include/libtorrent/aux_/merkle_tree.hpp`)\n\nOne instance per file, with four storage modes (optimization):\n\n| Mode | Stored nodes | When used |\n|------|-------------|-----------|\n| `empty_tree` | root only | single-block file, or tree completely unknown |\n| `piece_layer` | piece-layer hashes | piece hashes received, blocks not yet |\n| `block_layer` | block hashes | download complete, all blocks verified |\n| `full_tree` | all nodes | during partial verification |\n\n`m_block_verified` bitfield tracks which block hashes have been validated against their parent chain up to the known root.\n\nKey methods: `load_tree()`, `load_piece_layer()`, `add_hashes()`, `set_block()`\n\n## Hybrid Torrents (v1 + v2)\n\nA single `.torrent` file can contain both:\n- v1 info dict: `files` list + `pieces` SHA-1 concatenation\n- v2 `file tree` + `piece layers`\n\nConstraints: identical file layout, same `piece_length`. Results in two info hashes.\n`torrent_info` parsing uses `extract_files()` for v1 and `extract_files2()` for v2.\n\n## Hash Picker (`src/hash_picker.cpp`)\n\nDecides which merkle hashes to request from peers (BitTorrent v2 hash exchange protocol).\n\n**`hash_request` struct:**\n```cpp\nstruct hash_request {\n file_index_t file;\n int base; // tree layer (0 = block layer)\n int index; // starting offset in layer\n int count; // number of hashes (≤ 8192)\n int proof_layers; // uncle hashes needed to prove against known root\n};\n```\n\n**Strategy:**\n1. Request piece-layer hashes in 512-piece chunks until all known\n2. On piece failure, request all block hashes for that piece (3s minimum interval)\n\n**Validation flow:**\n- `add_hashes()`: receives hashes + uncle proof chain, calls `merkle_validate_and_insert_proofs()`, inserts valid nodes, reports `pass`/`fail` per piece\n- `set_block()`: inserts a single block hash, finds largest verifiable subtree, returns pass/fail\n\n## Resume Data\n\nMerkle trees are persisted across sessions in resume data (`src/write_resume_data.cpp`, `src/read_resume_data.cpp`):\n\n```\n[\"merkle trees\"][file_index] → sparse tree nodes (non-zero nodes only)\n[\"merkle tree mask\"][file_index] → bitfield of which nodes are stored\n[\"verified bits\"][file_index] → m_block_verified bitfield\n[\"piece layers\"][pieces_root] → concatenated piece-layer hashes\n```\n\n- `write_resume_data()` uses `build_sparse_vector()` to encode only known nodes\n- `read_resume_data()` calls `load_sparse_tree()` to restore the partial tree\n- Piece layers are re-derived from the stored tree during write\n\n## Verification Flow\n\n1. File added → `merkle_tree` created with root hash from `pieces root`\n2. Block arrives → `set_block()` inserts hash, attempts validation up to root\n3. Piece complete → all block hashes in piece set; piece passes if tree validates\n4. Piece fails → clear invalid nodes; hash picker schedules block-hash requests\n5. Hashes received → `add_hashes()` validates proof chain, inserts valid nodes, marks failures\n6. Download complete → all blocks verified; block layer mode; tree fully populated\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `src/merkle.cpp` | Tree math, proof validation, padding |\n| `src/merkle_tree.cpp` + `include/libtorrent/aux_/merkle_tree.hpp` | Per-file tree storage |\n| `src/hash_picker.cpp` + `include/libtorrent/aux_/hash_picker.hpp` | Hash request scheduling |\n| `src/torrent_info.cpp` | Parsing `file tree`, `piece layers`, hybrid logic |\n| `src/create_torrent.cpp` | Building v2/hybrid `.torrent` files |\n| `src/write_resume_data.cpp` + `src/read_resume_data.cpp` | Persisting trees |\n"},"items":[{"name":"dht.md","path":".claude/rules/dht.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/dht.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"---\npaths:\n - \"src/kademlia/**\"\n - \"include/libtorrent/kademlia/**\"\n---\n\n# DHT (Kademlia) Implementation\n\nSource: `src/kademlia/`, `include/libtorrent/kademlia/`\n\n## BEP Support\n\n| BEP | Description | Status |\n|-----|-------------|--------|\n| BEP 5 | DHT Protocol (ping, find_node, get_peers, announce_peer) | Full |\n| BEP 32 | IPv6 extension (dual-stack, nodes6) | Full |\n| BEP 33 | DHT Scrape (bloom filters BFpe/BFsd) | Full |\n| BEP 42 | Node ID Verification (IP-derived IDs) | Full |\n| BEP 44 | Arbitrary data storage (immutable + mutable items, Ed25519) | Full |\n| BEP 51 | DHT Infohash Indexing (sample_infohashes) | Full |\n\n---\n\n## File Overview\n\n| File | Purpose |\n|------|---------|\n| `node.cpp` / `node.hpp` | Central DHT node — routing, dispatch, traversal lifecycle |\n| `dht_tracker.cpp` / `dht_tracker.hpp` | Session-level coordinator, one `node` per listen socket |\n| `routing_table.cpp` / `routing_table.hpp` | K-bucket routing table |\n| `rpc_manager.cpp` / `rpc_manager.hpp` | Tracks outstanding RPCs, calls observers on response/timeout |\n| `observer.hpp` | Base observer class for pending RPC state |\n| `traversal_algorithm.cpp` / `.hpp` | Base iterative search algorithm |\n| `find_data.cpp` / `.hpp` | Find-nodes traversal, base for get_peers and get_item |\n| `get_peers.cpp` / `.hpp` | BEP 5 peer lookup traversal |\n| `get_item.cpp` / `.hpp` | BEP 44 item retrieval traversal |\n| `put_data.cpp` / `.hpp` | BEP 44 item storage traversal |\n| `sample_infohashes.cpp` / `.hpp` | BEP 51 traversal |\n| `refresh.cpp` / `.hpp` | Bootstrap / bucket-refresh traversal |\n| `direct_request.hpp` | Single-message direct request |\n| `node_id.cpp` / `node_id.hpp` | `node_id` type (alias for `sha1_hash`), XOR distance, BEP 42 generation |\n| `node_entry.cpp` / `node_entry.hpp` | Single routing-table entry |\n| `item.cpp` / `item.hpp` | Mutable/immutable item type |\n| `types.hpp` | `public_key`, `secret_key`, `signature`, `sequence_number` |\n| `msg.hpp` | Incoming message structure |\n| `dht_storage.cpp` / `dht_storage.hpp` | Pure-virtual storage interface + default RAM implementation |\n| `dht_state.cpp` / `dht_state.hpp` | Persistent state (node IDs, bootstrap nodes) |\n| `dht_settings.hpp` | Per-DHT settings struct |\n| `dht_observer.hpp` | Callbacks from DHT into `session_impl` |\n| `dos_blocker.cpp` / `dos_blocker.hpp` | Per-IP rate limiting (5 pkt/s, 5-min block) |\n| `ed25519.hpp` | Ed25519 sign/verify API (vendored impl in `src/ed25519/`) |\n| `announce_flags.hpp` | Announce flags: `seed`, `implied_port`, `ssl_torrent` |\n\n---\n\n## Threading\n\nAll DHT code runs on the **session network thread** (boost.asio). No locking needed inside DHT classes. `rpc_manager` uses a mutex only for its observer-pool allocator.\n\n---\n\n## Core Data Structures\n\n### `node_id`\nType alias for `sha1_hash` (160 bits).\n\nKey functions:\n- `distance(a, b)` — XOR metric\n- `distance_exp(a, b)` — index of highest differing bit (0–160)\n- `compare_ref(a, b, ref)` — compare distances to reference\n- `generate_id(external_ip)` — BEP 42 derivation\n- `verify_id(nid, source_ip)` — BEP 42 check\n\n### `node_entry`\nOne routing-table peer entry:\n- `node_id id`, `udp::endpoint endpoint`\n- `uint16_t rtt` (0xffff = unknown)\n- `uint8_t timeout_count` (0 = confirmed, 0xff = never pinged)\n- `bool verified` (BEP 42)\n- `time_point last_queried`\n\n### `routing_table`\n160 k-buckets (one per XOR distance bit). Extended routing table (default on) gives larger buckets close to us:\n\n| Bucket index | Multiplier | Max live nodes (k=20) |\n|---|---|---|\n| 0 | 16× | 320 |\n| 1 | 8× | 160 |\n| 2 | 4× | 80 |\n| 3 | 2× | 40 |\n| 4–159 | 1× | 20 (k) |\n\nKey behaviours:\n- Last (closest) bucket splits when it exceeds k entries\n- Each bucket has a **replacement cache**; candidates promoted when live node is removed\n- `ip_set` restricts one node per IP per bucket (BEP 42, `restrict_routing_ips`)\n- Periodic refresh: random query in each bucket's range every ~15 min\n\n### `item` (BEP 44)\n- **Immutable** — key is `sha1(value)`, no signature\n- **Mutable** — key is `sha1(salt || pk)`, Ed25519-signed, has `sequence_number`\n- CAS: `cas` field lets clients do compare-and-swap on sequence number\n\n### `dht_storage_interface`\nPure-virtual; default RAM implementation (`dht_default_storage_constructor`):\n\n```\nmap m_torrents (BEP 5 peers)\nmap m_immutable_items\nmap m_mutable_items\n```\n\nEviction uses distance from our node IDs — items far from us are evicted first.\nLimits: `max_torrents` (2000), `max_peers` (500), `max_dht_items` (700).\nPeer entries expire after 30 min (announce_interval).\n\n---\n\n## Key Classes\n\n### `node`\nOne per listen socket. Owns `routing_table` and `rpc_manager`.\n\nImportant methods:\n```cpp\nvoid bootstrap(endpoints, done_cb);\nvoid get_peers(info_hash, callback, flags);\nvoid announce(info_hash, port, flags, callback);\nvoid get_item(target, callback); // immutable\nvoid get_item(pk, salt, callback); // mutable\nvoid put_item(data, callback); // immutable\nvoid put_item(pk, sk, salt, callback); // mutable\nvoid sample_infohashes(endpoint, target, callback);\nvoid incoming(udp::endpoint, msg); // dispatch received UDP\nvoid tick(); // token rotation, bucket refresh\n```\n\nWrite tokens: `hash(requester_ip + secret + info_hash)`. Two secrets kept; rotated periodically to allow stale tokens for one rotation period.\n\n### `dht_tracker`\nOwns `std::map` (one `node` per socket).\nAggregates results across all nodes. Manages DOS blocking. Saves/restores `dht_state`.\n\n### `rpc_manager`\n- `invoke(entry, endpoint, observer_ptr)` — sends bencoded RPC, tracks by 2-byte transaction ID\n- `incoming(msg, &sender_id)` — looks up observer by txn-id, calls `observer::reply()`\n- `tick()` — fires `observer::timeout()` / `observer::short_timeout()` for stale requests\n- Uses an object pool for observers to reduce heap fragmentation\n\n### `observer`\nOne per pending RPC. Flags:\n\n| Flag | Meaning |\n|------|---------|\n| `flag_queried` | Request sent |\n| `flag_initial` | First attempt |\n| `flag_short_timeout` | Intermediate timeout fired |\n| `flag_failed` | Node failed |\n| `flag_alive` | Response received |\n| `flag_done` | Finished, no longer tracked |\n\nObserver hierarchy:\n```\nobserver\n ├─ traversal_observer (parses \"nodes\"/\"nodes6\" from responses)\n │ ├─ find_data_observer\n │ │ ├─ get_peers_observer (also parses \"values\", bloom filters)\n │ │ └─ get_item_observer (parses v/k/sig/seq)\n │ ├─ put_data_observer\n │ └─ sample_infohashes_observer\n ├─ announce_observer\n ├─ direct_observer\n └─ null_observer\n```\n\n---\n\n## Traversal Algorithms\n\nAll searches inherit from `traversal_algorithm`. The base implements the Kademlia iterative lookup:\n\n1. Seed result set with k closest nodes from routing table\n2. Issue α (default 3) concurrent requests (`add_requests()`)\n3. Each response: add new closer nodes, sort result set\n4. Repeat until k closest nodes all responded or timeout\n5. Call `done()`\n\n| Class | BEP | RPC sent | Callback delivers |\n|-------|-----|----------|-------------------|\n| `find_data` | 5 | `find_node` | closest nodes + write tokens |\n| `get_peers` | 5 | `get_peers` | `vector` (peers) |\n| `get_item` | 44 | `get` | `item`, `bool authoritative` |\n| `put_data` | 44 | first `get`, then `put` | response count |\n| `sample_infohashes` | 51 | `sample_infohashes` | samples, interval, total |\n| `bootstrap` / `refresh` | 5 | `get_peers` (our own ID) | (fires bucket refresh) |\n| `direct_traversal` | — | arbitrary | single `msg` |\n\n`obfuscated_get_peers`: wraps `get_peers`, uses a random target until convergence, then switches to the real info_hash (BEP 5 privacy mode, controlled by `dht_privacy_lookups`).\n\n---\n\n## Message Format (bencode)\n\nAll messages are bencoded dicts with mandatory keys `\"t\"` (transaction id), `\"y\"` (`\"q\"/\"r\"/\"e\"`), optional `\"v\"` (version), `\"ip\"` (external address, BEP 42), `\"ro\"` (read-only, BEP 5).\n\n**Queries** (`y=\"q\"`): `\"q\"` = method name, `\"a\"` = argument dict with `\"id\"` (our node id, 20 bytes).\n\n**Responses** (`y=\"r\"`): `\"r\"` = response dict.\n\n**Errors** (`y=\"e\"`): `\"e\"` = `[error_code, \"message\"]`.\n\n### Method summary\n\n| Method | Key request fields | Key response fields |\n|--------|--------------------|---------------------|\n| `ping` | `id` | `id` |\n| `find_node` | `id`, `target`, `want` | `id`, `nodes`, `nodes6` |\n| `get_peers` | `id`, `info_hash`, `noseed`, `scrape`, `want` | `id`, `token`, `values`\\|`nodes`, `BFpe`, `BFsd` |\n| `announce_peer` | `id`, `info_hash`, `port`, `token`, `implied_port`, `seed` | `id` |\n| `get` (BEP 44) | `id`, `target`, [`seq`] | `id`, `v`, [`k`, `sig`, `seq`] |\n| `put` (BEP 44) | `id`, `v`, [`k`, `sig`, `seq`, `salt`, `cas`] | `id` |\n| `sample_infohashes` | `id`, `target` | `id`, `samples`, `num`, `interval`, `nodes` |\n\nCompact node encoding: 26 bytes per IPv4 node (20-byte id + 4-byte IP + 2-byte port), 38 bytes for IPv6.\n\n---\n\n## Bootstrap\n\n1. Load `dht_state` (saved node IDs + endpoints)\n2. Assign node ID: saved → BEP 42 from external IP → random\n3. Add router nodes from settings (`router.bittorrent.com:6881`, etc.)\n4. Run `bootstrap` traversal against saved/configured endpoints (queries `get_peers` for own ID)\n5. On completion, refresh all non-contiguous buckets\n6. `dht_observer::get_peers()` notifies `session_impl` to query for all active torrents\n\n---\n\n## Integration with `session_impl`\n\n`session_impl` owns `shared_ptr m_dht` and implements `dht_observer`:\n\n| Observer method | When called |\n|-----------------|-------------|\n| `set_external_address()` | DHT detects our external IP via `\"ip\"` field |\n| `get_listen_port(socket)` | Node asks for our announce port |\n| `get_peers(info_hash, peer)` | Another node announced a peer to us |\n| `announce(info_hash, endpoint)` | We received a get_peers for one of our torrents |\n| `on_dht_request(method, msg, reply)` | Custom extension message handler |\n\nSession calls:\n- `dht_tracker::incoming_packet(socket, endpoint, buffer)` — for each received DHT UDP packet\n- `dht_tracker::new_socket(handle)` / `delete_socket(handle)` — listen port changes\n- `dht_tracker::tick()` — periodic maintenance\n\nRelevant `settings_pack` keys:\n\n| Setting | Effect |\n|---------|--------|\n| `enable_dht` | Toggle DHT |\n| `dht_search_branching` | Alpha (concurrent requests) |\n| `dht_max_peers_reply` | Peers returned per get_peers response |\n| `dht_enforce_node_id` | Reject nodes failing BEP 42 check |\n| `dht_restrict_routing_ips` | One node per /24 IPv4 per bucket |\n| `dht_restrict_search_ips` | Sybil mitigation in traversals |\n| `dht_privacy_lookups` | Enable obfuscated_get_peers |\n| `dht_read_only` | Set `ro=1` flag, don't accept queries |\n| `dht_upload_rate_limit` | Response bytes/sec cap |\n| `dht_item_lifetime` | BEP 44 item expiry (0 = never) |\n| `dht_sample_infohashes_interval` | BEP 51 cache TTL |\n\n---\n\n## Constants and Tunables\n\n| Parameter | Value | Notes |\n|-----------|-------|-------|\n| Node ID | 160 bits | SHA-1 |\n| k (bucket size) | 20 | Configurable |\n| Alpha | 3 | Concurrent requests per traversal |\n| RPC full timeout | 15 s | |\n| RPC short timeout | ~6 s | Triggers next request early |\n| Token rotation | ~7 min | Two tokens kept |\n| Bucket refresh | 15 min | Per-bucket, measured from last query |\n| Peer announce validity | 30 min | |\n| DOS block rate | 5 pkt/s | Per source IP |\n| DOS block duration | 5 min | |\n| DOS tracker size | 20 | Most-recent offenders |\n","frontmatter":{"paths":["src/kademlia/**","include/libtorrent/kademlia/**"]},"isInternal":false,"tokens":3301,"sizeBytes":11683},{"name":"disk-cache.md","path":".claude/rules/disk-cache.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/disk-cache.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"---\npaths:\n - \"src/disk_cache*\"\n - \"src/pread_disk_io*\"\n - \"include/libtorrent/aux_/disk_cache*\"\n---\n\n## Disk Cache (`pread_disk_io`)\n\nThe disk cache (`src/disk_cache.cpp`, `include/libtorrent/aux_/disk_cache.hpp`) is used exclusively by `pread_disk_io`. It is a write-back cache for incoming blocks, deferred until pieces are complete or memory pressure forces a flush.\n\n**Data structures:**\n\n- `disk_cache` — the cache itself; holds a `boost::multi_index_container` of `cached_piece_entry` objects, protected by a single `std::mutex`. The container has five indices: by `piece_location` (ordered), by `cheap_to_flush()` (ordered descending), by `need_force_flush()` (ordered descending), by `piece_location` (hashed, for fast lookup), and by `needs_hasher_kick()` (ordered descending).\n- `cached_piece_entry` — one entry per in-flight piece. Contains an array of `cached_block_entry` (one per block), an optional `piece_hasher` (`ph`, v1 only), an optional `block_hashes` array (`sha256_hash[]`, v2 only), and the cursors/flags described below.\n- `cached_block_entry::write_state` — `std::variant` with three mutually exclusive states:\n - `std::monostate` — no data present (the block has never been written, or its data has been fully released).\n - `disk_job*` — a pending write job is queued. The buffer is owned by the job; `take_write_job()` consumes the job during flushing.\n - `disk_buffer_ref` — the write has been flushed. The buffer may be non-null (`has_buf()`, \"held alive\" for future hashing) or null (already released). `is_flushed()` is true in either case and contributes to `flushed_cursor`.\n\n**Cursors:**\n\n- `hasher_cursor` — index of the first block not yet hashed (v1 SHA-1 only; hashing is always contiguous from block 0). Owned by the hashing thread while `hashing_flag` is set.\n- `flushed_cursor` — index of the first block not yet confirmed flushed to disk. Updated under the mutex after each flush completes.\n- `cheap_to_flush()` returns `hasher_cursor - flushed_cursor`: blocks that have been hashed but not yet flushed can be flushed without ever needing read-back.\n\n**Flags on `cached_piece_entry`:**\n\nFlags are stored in `cached_piece_flags flags` (a `bitfield_flag` bitfield). The mutex must be held to read or modify flags; updates go through `view.modify()` (required by `boost::multi_index`). Once a flag is set, the owning thread may access the protected state **without the mutex**:\n\n- `hashing_flag` — set under the mutex by the thread about to hash blocks. While set, that thread has exclusive access to `ph` (the v1 `piece_hasher`) and may advance `hasher_cursor`. No other thread may touch these. Cleared under the mutex when hashing is complete.\n- `flushing_flag` — set under the mutex by the thread about to write blocks to disk. While set, that thread has exclusive access to the block buffers being flushed. No other thread will attempt to flush the same piece. Cleared under the mutex when flushing is complete.\n- `force_flush_flag` — set when the piece has all blocks populated; prioritises this piece for flushing. Cleared once all blocks are flushed.\n- `piece_hash_returned_flag` — set once the piece hash has been computed and returned to the BitTorrent engine. The hash job is also the only signal that a piece is complete: it is what allows `flush_to_disk` to evict the `cached_piece_entry`, and what gets a piece force-flushed when the v1 hasher could not reach the end on its own (pure-v2 pieces, or pieces partially on disk at startup). When `disable_hash_checks` is set the engine still posts the job; the setting is latched into the storage (and into `torrent`) when the storage is created, at add time or when metadata arrives for a magnet link, so a later change never applies to it. In that mode `insert_write` caches the piece as neither v1 nor v2, so nothing is hashed on write, and `do_job(hash)` runs `hash_piece()` with a no-op callback purely for that bookkeeping and returns zero hashes.\n- `v1_hashes_flag` — set if this piece requires v1 SHA-1 hashing (`ph` is allocated).\n- `v2_hashes_flag` — set if this piece requires v2 SHA-256 block hashing (`block_hashes` is allocated).\n- `needs_hasher_kick_flag` — set when a block is inserted and the hasher thread should be woken; cleared when the hasher picks it up. Coalesces multiple insertions into one wakeup.\n- `notify_flushed_flag` — set by a `flush_storage()` caller that is waiting on a piece another thread is mid-flush on; `flush_piece_impl`'s `scope_end` checks it and signals `m_flushing_cv`. Only a wakeup signal supporting a single waiter, not a pin — the waiter re-looks-up the piece after each wakeup because it may have been flushed, hashed, or evicted while it slept.\n\nIn both hashing and flushing cases, the lock is **released** for the duration of the actual I/O (hashing or `pwrite()`), then re-acquired to update cursors/flags and potentially free buffers.\n\n**Deferred clear:** If `try_clear_piece()` is called while `flushing_flag` or `hashing_flag` is set, the clear job is stored in `cached_piece_entry::clear_piece` and executed by the owning thread once it clears its flag.\n\n**Held-alive buffers and the flush/hasher race:**\n\nWhen `flush_piece_impl` flushes a block, it checks `needed_by_hasher = (hashing_flag && block_index >= hasher_cursor)`. If a hasher is mid-snapshot (the `kick_hasher` window between snapshot and re-lock), the pointer to the buffer is in `blocks_storage[]` and must stay live. In that case the block transitions to `disk_buffer_ref{buf}` (non-null), and `m_blocks` is *not* decremented. Otherwise the buffer is freed immediately and `m_blocks` is decremented.\n\n`m_blocks` therefore counts both pending writes and held-alive buffers — both forms of memory pressure. `back_pressure` and `flush_request()` are driven off `m_blocks`.\n\nThe held-alive buffer is reclaimed when `kick_hasher` next advances the cursor past it (its cleanup loop frees blocks in `[old_cursor, new_cursor)` via `take_buf`). However, if v1 SHA-1 is blocked by a missing earlier block, the cursor cannot advance and the buffer stays held until either the missing block arrives or the piece is cleared. The \"drop held-alive\" flush pass (below) handles the case where no further inserts will arrive — without it, back-pressure can pin the cache above the low watermark indefinitely (the back-pressure observer is only notified once `m_blocks` drops to the low watermark, so the producer that tripped back-pressure would never be woken up). Note that v2 hashes for held-alive blocks are typically already in `block_hashes[]` (computed by the kick that triggered the keep-alive), so freeing the buffer only loses the v1 read-back optimization, not v2 hash work.\n\n**Flush strategy (`flush_to_disk`), in order:**\n\n1. Force-flush pass (`view`) — flush pieces with `force_flush` set (piece fully downloaded and hashed); ordered by `need_force_flush()`.\n2. Cheap flush pass (`view2`) — flush blocks in the range `[flushed_cursor, hasher_cursor)` (already hashed, no future read-back needed); pieces ordered by `cheap_to_flush()` descending.\n3. Drop held-alive pass (`view3`) — for each piece without `flushing_flag` or `hashing_flag`, free all blocks where `has_buf()` (held-alive `disk_buffer_ref{buf}`). No I/O — the data is already on disk. Done before the expensive pass because it's free; only skipped if the target is already met. Loses the v1 read-back optimization for these blocks; a subsequent `async_hash` will read them from disk.\n4. Expensive flush pass (`view4`) — flush any dirty blocks even if they haven't been hashed yet, requiring a read-back later to complete the piece hash. Used only when the cache exceeds the high-water mark.\n\nPasses 2-4 run only when `optimistic=false`. The optimistic flush at the top of each disk thread's loop runs only pass 1.\n\n**`flush_storage()`:** Called as a fence job during torrent teardown (`release_files`, `delete_files`, etc.) to flush every cached piece of one storage to disk. For each piece it flushes any still-dirty blocks (completing their write jobs via `flush_piece_impl`). It does **not** free or erase the piece — eviction is left to `flush_to_disk`. Keeping the piece preserves its partial hash state and lets another thread erase it concurrently without racing the flush. If a piece is mid-flush on another thread (`flushing_flag` set), it sets `notify_flushed_flag`, waits on `m_flushing_cv`, and re-looks-up the piece on each wakeup (it may be flushed, hashed, or evicted while waiting); if it is gone, it moves on. A concurrent `flush_storage()` already waiting on a piece (`notify_flushed_flag` already set) is left to that waiter, since the flag supports only one waiter.\n\n**Fences and the cache (`pread_disk_io`):** the general fence contract is in `.claude/rules/disk-io.md`; this is how `pread_disk_io` honours it with a write-back cache.\n\n- **Writes during a fence are queued, not aborted.** `async_write` calls `is_blocked()`, which takes ownership of the job and parks it in the fence's blocked queue. When the fence lowers, `disk_job_fence::job_complete` calls the `repost` callback (still under the fence mutex) once per blocked job in posting order; `add_completed_jobs_impl` inserts each reposted write into the cache there, and serves reposted reads from the cache (a normal disk read would return stale data, since the superseding write is still buffered). Because the resume runs under the fence mutex, a job posted concurrently is ordered behind the whole backlog. Work that can re-enter the fence (hasher kick, flush, completions) is deferred to after the mutex is released.\n- **`async_read` never consults the cache; `add_job` is the single funnel.** `async_read` only allocates a `job::read` (carrying the request verbatim, including an unaligned, block-spanning start) and hands it to `add_job`. `add_job` runs `is_blocked()` first, so a read posted while a fence is up is parked like any other job and cannot bypass the fence to serve stale data ahead of a queued write. The cache is consulted exactly once, by `prepare_read()`, and only after the job is past the fence gate: in `add_job` when the fence is down, or in the `repost` callback when the fence lowers. Both call `prepare_read()`; reposted jobs must **not** re-enter `is_blocked()` (a stacked fence still raises `m_has_fence`, so it would wrongly re-park them behind a later fence), which is why the consult is shared *after* the gate rather than by routing reposts back through `add_job`. `prepare_read()` mutates the job in place, so the single outstanding-job accounting `is_blocked()`/`job_complete()` set up is preserved.\n- **Every write is counted; writes during a fence are queued, not aborted.** `async_write` always routes through `is_blocked()` (never a fast path around it), so the write counts as an outstanding job on the storage whether or not a fence is up -- a fence then waits for buffered writes and their in-flight flush, exactly like a read or hash (see `.claude/rules/disk-io.md`). With no fence up, `is_blocked()` marks the job in-progress and increments the count, and the write is inserted into the cache inline; it stays counted until it is flushed and completed (or aborted). With a fence up, `is_blocked()` instead takes ownership of the job and parks it in the fence's blocked queue. When the fence lowers, `disk_job_fence::job_complete` calls the `repost` callback (still under the fence mutex) once per blocked job in posting order; `add_completed_jobs_impl` inserts each reposted write into the cache there, and serves reposted reads from the cache (a normal disk read would return stale data, since the superseding write is still buffered). Because the resume runs under the fence mutex, a job posted concurrently is ordered behind the whole backlog. Work that can re-enter the fence (hasher kick, flush, completions) is deferred to after the mutex is released.\n- **Reads racing the unblock.** `async_read`'s lock-free cache probe may run just before the backlog's writes land. `add_job` re-checks the cache once the fence is observed down, so a read is never dispatched to disk for a block whose write is sitting buffered.\n- **Forward progress (`m_fence_flush`).** A fence waits for the storage's outstanding jobs (including buffered writes) to reach zero, but nothing else flushes a partial piece under the watermark — so a raised fence hands its storage to a generic thread to flush (`flush_fenced_storages`), draining the writes it is waiting on. `add_fence_job` does this when a fence is raised over outstanding work (`fence_post_none`).\n- **Stacked fences.** When a fence lowers and reposts writes that sit ahead of a still-raised second fence, those partial-piece writes are now in the cache with no flush trigger. `add_completed_jobs_impl` detects a completed fence whose storage still `has_fence()` and re-arms `m_fence_flush` for it, so the next fence's wait can also make progress. (With no generic threads it flushes inline.)\n- **Unaligned reads and `job::partial_read`.** A peer read request whose start is not block-aligned can span two blocks. `prepare_read()` probes both with `disk_cache::get2`: if both blocks are in the cache it serves the whole request; if neither is, the job stays a `job::read` and `do_job(read)` reads the whole span from disk; if exactly one block is in the cache, it copies that side into the buffer and rewrites the job in place into a `job::partial_read` whose `offset`/`buffer_size`/`buffer_offset` describe just the missing side. `do_job(partial_read)` then reads only that side from disk into the rest of the buffer. Because `prepare_read()` runs past the fence gate against the *current* cache, the cached side it copies is never stale, and a `partial_read` therefore reads only from disk and never re-consults the cache. `prepare_read()` is the only place `partial_read` jobs are created -- they are never posted directly.\n","frontmatter":{"paths":["src/disk_cache*","src/pread_disk_io*","include/libtorrent/aux_/disk_cache*"]},"isInternal":false,"tokens":3260,"sizeBytes":13948},{"name":"disk-io.md","path":".claude/rules/disk-io.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/disk-io.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"---\npaths:\n - \"include/libtorrent/disk_interface.hpp\"\n - \"src/mmap_disk_io*\"\n - \"src/posix_disk_io*\"\n - \"src/pread_disk_io*\"\n - \"src/disk_job_fence*\"\n---\n\n## `disk_interface` contract\n\n`disk_interface` (`include/libtorrent/disk_interface.hpp`) is the session-level\ncustomization point for disk I/O, selected via\n`session_params::disk_io_constructor`. Built-in implementations: `mmap_disk_io`,\n`posix_disk_io`, `pread_disk_io`.\n\n**Threading.** Every `async_*` method is called from libtorrent's single network\nthread. The work may be performed asynchronously on the implementation's own\nthreads, but each completion handler **must be posted back to the network\nthread** via the `io_context` given to the constructor; handlers run on the\nnetwork thread. An implementation may use any number of threads and service\nqueued jobs concurrently and out of order, except where the contract below\nrequires ordering.\n\n**`async_hash` reflects all prior `async_write`s.** For a given piece, the\nengine posts all of the piece's blocks via `async_write`, then posts\n`async_hash` for that piece. The hash returned **must be computed over the data\nof every `async_write` for that piece that was posted before it.** This is the\nkey constraint for a multi-threaded back end that services jobs out of order: it\nmust order a piece's hash after that piece's writes and must never hash stale\nor missing data. The piece picker treats the returned hash as authoritative for\nthe data it handed over. `async_hash2` (single v2 block hash) has the same\nrequirement for the block it covers.\n\n**Fence jobs.** Several operations must run with exclusive access to a storage,\nsynchronized against all other I/O on it. These are *fence jobs*. A fence job:\n\n- waits for all jobs currently outstanding **on that storage** to finish, then\n executes -- so it runs alone, with no other operation on the storage in\n flight;\n- *fences* everything posted after it: any job posted to the same storage while\n the fence is raised is queued and not executed until the fence job completes.\n\nFences are per-storage -- a fence on one storage does not block jobs on another.\nThe fence operations are:\n\n- `async_clear_piece` -- synchronize with all outstanding I/O to the piece\n before the engine re-requests its blocks\n- `async_move_storage`\n- `async_release_files`\n- `async_stop_torrent`\n- `async_rename_file`\n- `async_delete_files`\n- `async_set_file_priority`\n- `async_check_files`\n\n`async_read`, `async_write`, `async_hash`, and `async_hash2` are ordinary\n(non-fence) jobs. The built-in back ends implement the fence semantics with\n`disk_job_fence` (`src/disk_job_fence.cpp`). A storage's outstanding-job count\nis what the fence waits to reach zero before it runs; a job stays counted from\nthe moment it is accepted until its completion is posted back.\n\n**Every ordinary job must honor the fence -- including a buffered write.** Each\njob, with no exceptions, has to participate in the outstanding-job count for its\nwhole lifetime, so the fence cannot run while any work on the storage is still\nin flight. This is easy to overlook for `async_write` on a write-back back end:\neven though the write is inserted into a cache and its handler is deferred, it\nmust be counted as outstanding from the moment it is accepted until the flush\nthat retires it has finished and its completion is posted -- *including the\nwindow while a disk thread is mid-flush writing it out*. It must not take a fast\npath that skips the count. If a buffered write (or the thread flushing it) is\ninvisible to the fence, the fence can be raised and complete while that write is\nstill being written to disk, and a teardown fence (`async_stop_torrent`,\n`async_release_files`, ...) can then tear the storage down underneath the\nin-flight flush -- a use-after-free. So the rule is symmetric: a write counts\nexactly like a read or a hash.\n\n### Fences and deferred (write-back) writes\n\nThe fence contract above is straightforward for an implementation that runs\neach job to completion immediately. An implementation that *defers* writes --\nreporting an `async_write` complete before the data is durably on disk, as a\nwrite-back cache does -- has to uphold two further invariants. Both are easy to\nget wrong, and a violation of either is a deadlock or silent data corruption,\nnot a degraded-performance bug.\n\n**A fence must not wait on writes that nothing will flush (forward progress).**\nA fence waits for every job outstanding on the storage to complete before it\nruns. When a write is only reported complete once it is durably stored, that\nwrite stays outstanding -- and keeps the fence waiting -- the whole time it sits\nbuffered. The flush that would retire it normally happens when the piece\ncompletes (its `async_hash`) or under memory pressure; but the completing hash\nis itself an ordinary job, now queued *behind* the very fence that is waiting\nfor it, and memory pressure may never arrive. The invariant: **raising a fence\nmust guarantee the storage's buffered writes get flushed, rather than waiting\npassively for a flush trigger the fence is itself blocking.** Otherwise the\nfence and its own backlog deadlock.\n\n**Resuming a fenced backlog must preserve posting order against new jobs.** When\na fence completes, the jobs that queued behind it are resumed. If resuming a job\nis observable -- a buffered write becomes visible, a read is served -- then a job\nthe network thread posts *during* that resume must be ordered after the whole\nresumed backlog, never interleaved ahead of it. The invariant: **a job arriving\nwhile the backlog is being resumed must not bypass the backlog.** Violate it and\na freshly-posted `async_read` overtakes a still-queued `async_write` for the\nsame block and returns stale data (or a hole).\n\n**A read resumed from a fence must reflect writes resumed ahead of it.** A\ncorollary of the ordering invariant: a read unblocked from the backlog may be\npreceded in that same backlog by a write to the very block it reads (both were\nblocked by the fence). That read must observe the write -- i.e. it must be\nserved from the buffered data, not from disk where the superseding write has not\nyet landed. Put plainly: the reads and writes a fence releases must take effect\nin the order they were posted, exactly as if the fence had never been raised.\n\n**Back-pressure.** `async_write` returns `true` when the write queue is over the\nhigh watermark; the caller then stops issuing writes for that peer until the\n`disk_observer` passed in is notified that the queue has drained. `false` means\nthe caller may keep writing.\n\n**Batching.** `async_*` calls only enqueue work; `submit_jobs()` is called once\nafter a batch to wake the disk thread(s). An implementation may also wake on each\n`async_*`, but should tolerate the batched notification.\n","frontmatter":{"paths":["include/libtorrent/disk_interface.hpp","src/mmap_disk_io*","src/posix_disk_io*","src/pread_disk_io*","src/disk_job_fence*"]},"isInternal":false,"tokens":1550,"sizeBytes":6789},{"name":"documentation.md","path":".claude/rules/documentation.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/documentation.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"# Documentation Generation and Spell Checking\n\nThe reference documentation is generated from the C++ public headers; the\nprose manual is hand-written reStructuredText (RST). Both are built and\nspell-checked from the `docs/` directory via `docs/makefile`, and the same\ntargets run in CI (`.github/workflows/docs.yml`).\n\n## Reference Doc Generation (`docs/gen_reference_doc.py`)\n\n`gen_reference_doc.py` parses the public headers and emits RST. It scans\n`include/libtorrent/*.hpp`, `include/libtorrent/kademlia/*.hpp`, and\n`include/libtorrent/extensions/*.hpp` (add `--internal` to also include\n`include/libtorrent/aux_/*.hpp`). For each documented class, function, enum,\nand constant it produces an RST file per category (`reference-Core.rst`,\n`reference-Session.rst`, ...) plus a table of contents (`reference.rst`) and an\noptional `single-page-ref.rst` (`--single-page`).\n\n### How comments become docs\n\n- A documentation comment is the run of `//` comment lines immediately\n **above** a declaration, with no blank line between the comment and the\n declaration. A blank line, or a line that ends a declaration (`;`, `{`,\n `}`), resets the pending comment.\n- **Comment bodies are reStructuredText.** You can use RST markup directly in\n the `//` comments: `.. code::` blocks, `.. note::`, bullet lists,\n `parsed-literal`, headings, etc. The generator renders them verbatim into\n the output RST.\n- **Symbol cross-linking** is automatic: any word in a comment that matches a\n known symbol name (class, function, enum, `settings_pack::` setting, or one\n of the pre-defined manual anchors) is turned into a hyperlink by\n `linkify_symbols()`. A trailing underscore (`foo_`) forces a link the same\n way RST reference syntax does.\n\n### Grouping functions under one comment (man-page style)\n\nMultiple overloads or related functions can share a **single** documentation\ncomment. The rule is **adjacency with no blank line between the\ndeclarations**: when a function (or field) declaration immediately follows the\nprevious one with no blank line and no intervening comment, its signature is\nappended to the previous entry's signature list rather than starting a new\nentry. So a comment above the first of a group documents the whole group, the\nway a man page documents a family of related calls together. Inserting a blank\nline between declarations splits them into separately-documented entries (and\nthe second will then warn if it has no comment of its own).\n\n### Visibility and warnings\n\n- A comment beginning with `hidden` excludes the symbol entirely. A comment\n beginning with `internal` excludes it from the public docs (but `--internal`\n includes it).\n- Every public, non-trivial member must be documented. Undocumented\n symbols print `WARNING: ... is not documented`. Trivial members\n (destructors, default/copy/move ctors, assignment, comparison/stream/\n conversion operators) are exempt via `is_trivial_member()`.\n- A `TODO:` in a public doc comment is a hard error (exits non-zero).\n- `#if` blocks gated on debug-only macros (asserts, invariant checks, and the\n other debugging macros) and on old `TORRENT_ABI_VERSION` guards are skipped\n entirely, so debug-only or deprecated members do not appear in docs. A\n non-ABI `TORRENT_*` `#if` inside a public struct warns about possible ABI\n breakage.\n- `// OVERVIEW` introduces a section overview block for a header's category.\n\n### Plain-text output for spell checking\n\nWith `--plain-output`, `add_desc()` writes a `plain_text_out.txt` containing\n**only the prose** from doc comments, with the C++ signatures and `.. code::`\nblocks stripped out. This is what the spell checker runs over.\n\n## Other Generators\n\n| Script | Output | Source |\n|--------|--------|--------|\n| `gen_reference_doc.py` | `reference-*.rst`, `reference.rst`, `single-page-ref.rst`, `plain_text_out.txt` | public headers |\n| `gen_settings_doc.py` | `settings.rst` + appends settings names to `hunspell/settings.dic` | `settings_pack.hpp` |\n| `gen_stats_doc.py` | `stats_counters.rst` | `session_stats.cpp`, `performance_counters.hpp` |\n| `gen_todo.py` | `todo.html` | `src/*.cpp`, headers |\n| `filter-rst.py` | `*-plain.txt` (prose-only) | a hand-written `.rst` file |\n\n`gen_settings_doc.py` doubles as a dictionary source: it splits every setting\nname on `_` and adds the parts to `hunspell/settings.dic`, so setting-name\nfragments are not flagged as misspellings.\n\n## Spell Checking (hunspell)\n\nSpell checking uses **hunspell**, run via `make spell-check` from `docs/`:\n\n```sh\ncd docs && make spell-check\n```\n\nThe flow:\n1. Generated reference prose is in `plain_text_out.txt` (from\n `gen_reference_doc.py --plain-output`).\n2. Hand-written manuals are converted to prose-only text with\n `filter-rst.py` (e.g. `manual.rst` -> `manual-plain.txt`). `filter-rst.py`\n strips RST directives (`.. ` lines) and indented literal/code blocks so\n only natural-language text remains.\n3. `hunspell -l` lists misspelled words from each plain-text file into\n `hunspell-report.txt`, using dictionary `hunspell/en_US` plus the project\n word list `hunspell/libtorrent.dic` (settings text uses the generated\n `hunspell/settings.dic`). HTML manuals are checked with `-H` (HTML mode).\n4. If `hunspell-report.txt` is non-empty the target **fails** and prints the\n offending words.\n\n### Fixing spell-check failures\n\nEither correct the typo in the source, or -- if the word is a real term,\nidentifier, or acronym -- add it to **`docs/hunspell/libtorrent.dic`** (one\nword per line). `hunspell/en_US.{aff,dic}` is the stock English dictionary and\nshould not be edited. `hunspell/settings.dic` is generated (en_US + the\nsettings names + a copy of `libtorrent.dic`); do not hand-edit it.\n\n## Building the HTML\n\n```sh\ncd docs && make html # all RST -> HTML via rst2html, plus figures\ncd docs && make # default target builds the docs\n```\n\nHTML is produced by `rst2html` (docutils) with `template.txt`/`style.css`;\nfigures are rendered from `.dot` (graphviz), `.diagram` (aafigure), and `.svg`\n(imagemagick `convert`). `make stage` copies the built site to `$WEB_PATH`.\n`make clean` removes generated `.rst`/`.html`/figures (it does **not** delete\nthe checked-in hand-written `.rst` manuals).\n\n## CI\n\n`.github/workflows/docs.yml` installs `python3-docutils`, `hunspell`,\n`graphviz`, `imagemagick`, `aafigure`, etc., then runs\n`make spell-check html`. A documentation warning, an undocumented public\nsymbol, a `TODO:` in a doc comment, or a spelling error all fail the build.\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `docs/gen_reference_doc.py` | parse headers -> reference RST + plain text |\n| `docs/gen_settings_doc.py` | `settings.rst` + settings dictionary words |\n| `docs/gen_stats_doc.py` | `stats_counters.rst` |\n| `docs/filter-rst.py` | strip RST markup to prose for spell checking |\n| `docs/makefile` | `html`, `rst`, `spell-check`, `stage`, `clean` targets |\n| `docs/hunspell/libtorrent.dic` | project word list (edit to whitelist terms) |\n| `docs/hunspell/en_US.{aff,dic}` | stock English dictionary (do not edit) |\n| `.github/workflows/docs.yml` | CI: spell-check + build |\n","isInternal":false,"tokens":1812,"sizeBytes":7139},{"name":"fuzzers.md","path":".claude/rules/fuzzers.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/fuzzers.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"---\npaths:\n - \"fuzzers/**\"\n - \"tools/gen_corpus.py\"\n---\n\n## Fuzzers\n\n### Build\n\nFuzzers require clang and are built separately from the main library:\n\n```sh\ncd fuzzers && b2 clang\n```\n\nBuilt with ASAN + UBSAN (norecover), debug symbols, asserts on, debug\niterators (`_GLIBCXX_DEBUG`), and WebTorrent enabled (per the default-build\nin `fuzzers/Jamfile`; note that `invariant-checks` is *not* turned on). Each\ntarget produces a standalone binary under `fuzzers/bin//` (e.g.\n`fuzzers/bin/clang-linux-20/...`). The explicit `stage` / `stage-large`\ninstall targets copy the binaries into the `fuzzers/fuzzers/` subdirectory\n(the `fuzzers` in `fuzzers/Jamfile` is relative to the Jamfile's\nown directory).\n\nTwo optional build features (defined in `fuzzers/Jamfile`):\n\n- `debug-logging=on` -- defines `DEBUG_LOGGING=1`, enabling extra alert\n logging in the session-level fuzzers (see peer_conn below).\n- `variant=build_coverage` -- a fast replay build with sanitizers, asserts,\n and invariant checks all off. Combine with `test-coverage=on` to measure\n code coverage from an existing corpus.\n\nThe default-build also pins `1` (`TORRENT_ABI_VERSION=1`,\nthe oldest/most inclusive ABI -- see the table in the top-level `CLAUDE.md`),\nso every fuzzer target, not just `bdecode_entry` (below), compiles and fuzzes\nthe full set of deprecated APIs alongside the current ones. `TORRENT_ABI_VERSION`\nguards in the codebase are exact-match (`== 1`) or lower-bound (`>= N`) checks,\nnever upper-bound, so ABI 1 is a strict superset of what any higher ABI value\ncompiles; this is safe project-wide.\n\n### Running a fuzzer\n\nRun a specific fuzzer against a corpus (single job):\n\n```sh\n./fuzzers/bin/.../peer_conn fuzzers/corpus/peer_conn/\n```\n\nUseful libFuzzer flags:\n\n| Flag | Purpose |\n|------|---------|\n| `-max_len=N` | Cap input size (default: 4096) |\n| `-timeout=N` | Seconds before a single run is declared a timeout (default: 1200) |\n| `-max_total_time=N` | Stop after N seconds total |\n| `-artifact_prefix=dir/` | Write crash/timeout inputs to this directory |\n\n### Running in parallel\n\nlibFuzzer supports in-process parallelism via `-jobs` and `-workers`:\n\n```sh\n./fuzzers/bin/.../peer_conn fuzzers/corpus/peer_conn/ \\\n -jobs=8 -workers=8\n```\n\n- `-jobs=N` — total number of fuzzing jobs to complete before the driver exits\n- `-workers=N` — number of concurrent fuzzer processes (defaults to\n `min(jobs, num_cpus/2)` when omitted)\n\nEach worker writes its own log to `fuzz-N.log` in the current directory.\nCrashes are written to the current directory by default; use\n`-artifact_prefix=crashes/` to redirect them.\n\nAfter parallel runs, merge all worker corpora back into the main corpus to\ndeduplicate and minimize:\n\n```sh\n./fuzzers/bin/.../peer_conn -merge=1 \\\n fuzzers/corpus/peer_conn/ fuzzers/corpus/peer_conn/\n```\n\n(Passing the same directory as both source and destination is safe; libFuzzer\nreads all inputs first, then writes the minimized set back.)\n\n### bdecode_node.cpp vs bdecode_entry.cpp\n\n`bdecode_node.cpp` fuzzes the modern `lt::bdecode(span, ...)`\nfrom `bdecode.hpp`; `bdecode_entry.cpp` fuzzes the deprecated,\n`TORRENT_ABI_VERSION == 1`-only `lt::bdecode()` overloads in\n`bencode.hpp`. They share no code path. `bdecode_entry` degrades to a\nno-op stub if built at any other ABI, rather than failing to compile.\n\n### Structure\n\nEvery fuzzer in `fuzzers/src/` implements `LLVMFuzzerTestOneInput`; the\nsession-level fuzzers additionally implement `LLVMFuzzerInitialize`:\n\n- `LLVMFuzzerTestOneInput(data, size)` — implemented by every fuzzer;\n called for every fuzz input; must be stateless with respect to prior\n inputs. Returns 0 on success, -1 to signal libFuzzer that the input was\n uninteresting (used for timeouts, not errors).\n\n- `LLVMFuzzerInitialize(argc, argv)` — implemented only by the\n session-level fuzzers (`add_torrent`, `pe_conn`, `peer_conn`,\n `piece_layers`, `rtc_peer_conn`, `ut_metadata`, `ut_pex`, `web_seed`).\n Called once at startup to set up global state (sessions, sockets,\n servers) that persists across all test inputs. The many pure-function\n fuzzers (e.g. `bdecode_node`, `parse_magnet_uri`, `gzip`,\n `torrent_info`) have no global state and omit it.\n\n### peer_conn.cpp\n\nConnects to a live libtorrent session as a BitTorrent peer and sends\nfuzz-controlled protocol messages.\n\nThe session and torrent setup is shared via `peer_session.hpp`\n(`configure_fuzz_session()`, `make_fuzz_torrent_params()`, and the\n`peer_fuzz_session` struct `g_fz`), and reused by the rtc_peer_conn,\nut_metadata, and ut_pex fuzzers.\n\n**Session setup (`LLVMFuzzerInitialize`):**\n\n- Creates a session on `127.0.0.1:0` with all outbound connections and\n ancillary services (outgoing TCP/uTP, DHT, LSD, UPnP, NAT-PMP, IP\n notifier) disabled, encryption disabled, and all peer timeouts set to\n 1 second.\n- Adds a **hybrid (v1 + v2)** torrent built from four files (a 0-byte\n file, a 100-byte file, a 50 KiB file, and a 97 MiB + 200 KiB file).\n After canonicalization (alphabetical sort + inserted pad files) this is\n exactly 100 pieces of 1 MiB. Piece hashes (SHA-1) and per-file block\n hashes (SHA-256) are filled with dummy values. `save_path` is `.` and\n no data is on disk, so the torrent stays in downloading state.\n- Waits for both the TCP `listen_succeeded_alert` (recording\n `g_fz.listen_port`) and the `torrent_resumed_alert` before returning.\n- Records `g_fz.info_hash` (an `info_hash_t` holding both the v1 SHA-1\n and v2 SHA-256 info-hashes).\n- Also adds a **second** torrent as a **magnet** (`info_hashes` only, no\n `ti`), built via the shared `add_fuzz_magnet(false, 'a')` helper (which\n wraps `build_fuzz_torrent()` from `peer_session.hpp`) from the identical\n file layout but with no v2 (SHA-256) hashes, so it's v1-only. Its\n info-hash and pre-encoded ut_metadata piece message (from\n `torrent_info::info_section()`) are stored in\n `g_v1_only_hash`/`g_v1_only_metadata_piece`. Because this metadata is\n fixed at startup rather than derived from fuzz input, delivering it via\n ut_metadata (see below) always succeeds, unlike a corpus byte sequence,\n which would have to find a SHA-1 preimage to ever make\n `torrent::set_metadata()` return true. Without this second torrent, the\n whole torrent-acquires-metadata code path (`set_metadata()`,\n `initialize_merkle_trees()`, `on_metadata_impl()`) would be unreachable:\n the first, pre-built torrent always has valid metadata, so that code\n never runs for it.\n- Also adds a **third** torrent, added as a **hybrid (v1 + v2) magnet**\n (`info_hashes` holds both the v1 and v2 hash, no `ti`), built via the\n same `add_fuzz_magnet(true, 'c')` helper from the identical file layout\n with a different dummy-hash fill byte (`'c'` instead of the default\n `'a'`) so it gets its own info-hash instead of colliding with `g_fz`'s.\n Its info-hash and pre-encoded ut_metadata piece message are stored in\n `g_v2_hybrid_hash`/`g_v2_hybrid_metadata_piece`. This is the counterpart\n to the v1-only magnet: metadata resolves to a genuine hybrid, so a peer's\n speculative v2 claim turns out honest and must survive metadata\n resolution rather than being corrected, exercising the same code as\n the v1-only magnet from the opposite direction, to guard against the\n fix over-triggering on legitimate v2 peers. More variants (a different\n file layout, etc.) can be added the same way if a future case needs\n one.\n\n**Per-input flow (`LLVMFuzzerTestOneInput`):**\n\n1. Opens a TCP connection to `127.0.0.1:g_fz.listen_port`.\n2. Sends a 68-byte BT handshake:\n - `data[0..7]` are used as the extension-flags (reserved) bytes.\n - Bits for BEP 10 (byte 5, bit 0x10) and BEP 6 FAST (byte 7, bit\n 0x04) are always forced on so those handlers are always reachable.\n - Byte 0 bit 0x01 selects the peer's protocol version, for torrents\n that have a v1 and a v2 hash to choose between (torrent selector 0\n and 2 below): clear = v2 (advertises the torrent's\n `info_hashes().get_best()`, `protocol_v2` true), set = v1\n (advertises the torrent's `info_hashes().v1`, `protocol_v2` false).\n - Byte 0 bits 0x02 and 0x04 form a 2-bit torrent selector: 0 = the\n first, pre-built torrent (metadata already valid, bit 0x01 applies\n as above); 1 = the second, v1-only magnet (always presents\n `g_v1_only_hash.v1`; bit 0x01 doesn't apply since there's no v2\n hash to choose between); 2 = the third, hybrid magnet (bit 0x01\n applies exactly as it does for selector 0); 3 = unused, falls back\n to selector 0. Every other flag bit, including byte 7 bit 0x10 (the\n v2-upgrade bit), is independent of the torrent selector and free to\n combine with any of them, e.g. a handshake claim that doesn't match\n what the connected torrent's info-hash actually supports.\n - Peer-ID is all zeros.\n3. Sends a fixed BEP 10 extended handshake (`k_extended_handshake`,\n ext_id 0) registering:\n\n | Extension | ext_id |\n |-------------|--------|\n | ut_pex | 1 |\n | ut_metadata | 2 |\n | upload_only | 3 |\n\n4. If the torrent selector picked the v1-only or hybrid magnet, sends two\n more fixed (not fuzz-derived) messages before continuing: a second\n extended handshake (BEP 10 supports re-negotiation) announcing\n `metadata_size`, which makes the ut_metadata plugin queue a request\n for piece 0, then a ut_metadata piece message carrying that magnet's\n metadata (`g_v1_only_metadata_piece` or `g_v2_hybrid_metadata_piece`),\n so `torrent::set_metadata()` succeeds deterministically on every run\n that picks either branch.\n5. Parses `data[8..]` as a sequence of messages (see **Corpus wire\n format** below) and sends each with a correct 4-byte BT length\n prefix. This is unchanged regardless of which torrent step 2 picked,\n so the same fuzzed message space (including BEP 78 hash messages)\n now also runs against a torrent whose metadata was just acquired\n rather than always pre-loaded.\n6. Closes the socket and waits for a `peer_error_alert` or\n `peer_disconnected_alert` (3-second timeout).\n\n**DEBUG_LOGGING:** defined by `fuzzers/Jamfile` via the `debug-logging`\nfeature (default `off` -> `DEBUG_LOGGING=0`). Build with\n`b2 clang debug-logging=on` to set it to 1, which adds `peer_log` alerts to\nthe alert mask and makes `wait_for_disconnect` print the last 10 interesting\nalerts (connect and routine peer-log noise excluded) before each disconnect\n-- useful for tracing which message paths were hit.\n\n### Corpus wire format (peer_conn)\n\nEach corpus file is binary-structured:\n\n```\n[8 bytes : extension flags]\n[repeated until EOF:\n 1 byte : msg_type\n 2 bytes : payload_len (big-endian, max 65535)\n N bytes : payload (N = min(payload_len, bytes_remaining))\n]\n```\n\nMinimum file size: 11 bytes (8 flag bytes + 3-byte message header).\n\nFor **msg_type 20** (BEP 10 extended protocol), the first byte of the\npayload is the extended message ID and the rest is the extension body.\nThe fuzzer splits this automatically before calling\n`send_extended_message`.\n\n**Extension flag byte layout** (8 reserved bytes, big-endian):\n\n| Byte | Bit | Protocol |\n|------|------|---------------------|\n| 0 | 0x01 | protocol version, when the selected torrent has a v1 and a v2 hash (set = v1 SHA-1, clear = v2) |\n| 0 | 0x02 + 0x04 | 2-bit torrent selector: 0 = pre-built hybrid, 1 = v1-only magnet, 2 = hybrid magnet, 3 = unused (falls back to 0) |\n| 5 | 0x10 | BEP 10 extended |\n| 7 | 0x04 | BEP 6 FAST |\n| 7 | 0x01 | BEP 5 DHT |\n| 7 | 0x10 | v2-upgrade bit (peer claims v2 support) |\n\n**libtorrent's fixed incoming ext_ids** (from `bt_peer_connection.hpp`\nenum; these are the IDs libtorrent expects in messages FROM us):\n\n| ext_id | Handler | Notes |\n|--------|------------------|--------------------------------------------|\n| 0 | extended HS | re-negotiation supported |\n| 1 | ut_pex plugin | as registered in `k_extended_handshake` |\n| 2 | ut_metadata | as registered in `k_extended_handshake` |\n| 3 | upload_only | `upload_only_msg` constant |\n| 4 | holepunch | `holepunch_msg`; requires `m_holepunch_id` |\n| 7 | dont_have | `dont_have_msg`; 4-byte piece index |\n| 8 | share_mode | `share_mode_msg` |\n| other | disconnect | `errors::invalid_message` |\n\nHolepunch (ext_id 4) silently returns if `m_holepunch_id == 0`. To\nreach the holepunch handler body, a corpus file must first send an\nadditional extended handshake (ext_id 0) that includes\n`\"ut_holepunch\": ` in its `m` dict.\n\n**BEP 78 hash messages** (21 = hash_request, 22 = hashes, 23 =\nhash_reject) require the peer's `protocol_v2` flag (set from the\nhandshake) and, separately, the connected torrent's info-hash to actually\nhave a v2 component, since a peer can set the flag on a torrent that never\ngains one (see the v1-only magnet above), so both are checked. Either\nfailing disconnects with `errors::invalid_message`.\n\n### Corpus generation tool\n\n`tools/gen_corpus.py` generates seed corpora for all session-level fuzzers.\nRun it from the repo root:\n\n```sh\npython3 tools/gen_corpus.py\n```\n\nThis writes into:\n- `fuzzers/corpus/peer_conn/`\n- `fuzzers/corpus/natpmp/`\n- `fuzzers/corpus/udp_tracker/`\n- `fuzzers/corpus/upnp/`\n- `fuzzers/corpus/pe_crypto_state/`\n- `fuzzers/corpus/pe_conn/`\n- `fuzzers/corpus/ut_metadata/`\n- `fuzzers/corpus/ut_pex/`\n- `fuzzers/corpus/web_seed/`\n- `fuzzers/corpus/utp_stream/`\n- `fuzzers/corpus/piece_layers/`\n- `fuzzers/corpus/torrent_info/`\n- `fuzzers/corpus/lsd/`\n- `fuzzers/corpus/add_torrent/`\n\n**Sync requirement:** the Python constants `EXT_UT_PEX`,\n`EXT_UT_METADATA`, `EXT_UPLOAD_ONLY`, and `FUZZER_EXT_HANDSHAKE` in\nthe tool must match `k_extended_handshake` in `peer_conn.cpp`. If the\nC++ extended handshake is changed, update both files together.\n\nThe tool also uses `NUM_PIECES = 100`, `PIECE_SIZE = 1 MiB`, and\n`BLOCK_SIZE = 16 KiB` -- these must match the torrent that\n`make_fuzz_torrent_params()` (equivalently `build_fuzz_torrent(true)`) in\n`fuzzers/src/peer_session.hpp` builds. `build_fuzz_torrent(false)` builds\nthe second, v1-only torrent, and `build_fuzz_torrent(true, 'c')` builds\nthe third, hybrid-magnet torrent, both from the identical file layout, so\nthese constants apply to them too.\n\nA handful of seeds set byte 0 bits 0x02/0x04 to connect to the second or\nthird torrent instead of the first, combined with other flag bits (e.g.\nthe v2-upgrade bit) and a follow-up message. These exist to give the\nfuzzer a foothold in those branches, not to exhaustively cover them;\ncoverage-guided mutation explores the rest.\n\n### pe_conn.cpp\n\nConnects to a live libtorrent session and drives the full PE (protocol\nencryption / MSE) handshake as the initiating (client) side. Requires\nthe session to have `in_enc_policy = pe_enabled` and\n`allowed_enc_level = pe_both`, which pe_conn's `LLVMFuzzerInitialize`\nsets.\n\nThe fuzzer performs a real DH key exchange (so the server can locate the\ntorrent), then uses fuzz-controlled bytes to drive all encrypted fields.\nEvery server-side error path is reachable by setting the appropriate bit\nin the flags byte.\n\n**Session setup (`LLVMFuzzerInitialize`):**\n\n- Same hybrid v1+v2 torrent as peer_conn (100 pieces, 1 MiB each).\n- `in_enc_policy = pe_enabled`, `allowed_enc_level = pe_both`.\n- All outbound services disabled (same as peer_conn).\n\n**Per-input flow (`LLVMFuzzerTestOneInput`):**\n\n1. Connect TCP socket to session.\n2. Generate a fresh DH key pair; send DH public key + PadA (from fuzz bytes).\n3. Read the server's DH public key (96 bytes, blocking).\n4. Compute shared secret, RC4 keys, sync hash, and obfuscated SKEY hash.\n5. Build the PE payload with fuzz-controlled fields and optional corruptions.\n6. Send the payload, close socket, wait for disconnect.\n\n**Corpus wire format:**\n\n```\nByte 0 : pad_a_size (PadA bytes before sync hash, 0-255)\nByte 1 : flags\n bit 0 : corrupt_sync_hash -> sync_hash_not_found\n bit 1 : corrupt_skey -> invalid_info_hash\n bit 2 : corrupt_vc -> invalid_encryption_constant\n bits 3-4 : crypto_provide\n 00 -> pe_both (0x03)\n 01 -> pe_plaintext (0x01)\n 10 -> pe_rc4 (0x02)\n 11 -> 0x00 (invalid) -> unsupported_encryption_mode\nByte 2 : len_pad_c * 2 (encrypted padding, 0-510 bytes)\nByte 3 : len_ia (IA size 0-255; server rejects > 68)\nBytes 4+ : first pad_a_size bytes = PadA content\n then up to len_ia bytes = IA (BT handshake portion)\n```\n\nSee `.claude/rules/protocol-encryption.md` for details of the PE protocol.\n","frontmatter":{"paths":["fuzzers/**","tools/gen_corpus.py"]},"isInternal":false,"tokens":4631,"sizeBytes":16893},{"name":"piece-picker.md","path":".claude/rules/piece-picker.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/piece-picker.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"---\npaths:\n - \"src/piece_picker.cpp\"\n - \"include/libtorrent/aux_/piece_picker.hpp\"\n - \"test/test_piece_picker.cpp\"\n---\n\n## Piece Picker\n\nThe piece picker (`src/piece_picker.cpp`, `include/libtorrent/aux_/piece_picker.hpp`) decides\nwhich blocks to request from which peers. It tracks availability (how many peers have each piece),\nper-piece priority, and the state of every block in every partially-downloaded piece.\n\n### Core data structures\n\n**`piece_pos`** — 8 bytes per piece, tightly packed:\n- `peer_count` (26 bits) — number of peers that have this piece\n- `download_state` (3 bits) — one of the `download_queue_t` constants below\n- `piece_priority` (3 bits) — 0 = filtered (dont-download), 1–7 user priority (4 = default)\n- `index` (`prio_index_t`) — position in `m_pieces`, or `we_have_index` (-1) if we have it\n\n**`m_piece_map`** — `aux::vector`, one entry per piece.\n\n**`m_pieces`** — flat `aux::vector` of all pickable (not filtered,\nnot have) pieces, sorted by effective priority. Within each priority band pieces are in random\norder (rarest-first is approximated by the priority formula, not by sorting on availability alone).\n\n**`m_priority_boundaries`** — cumulative end-indices into `m_pieces`. Priority band `p` spans\n`[m_priority_boundaries[p-1], m_priority_boundaries[p])`. Priority 0 starts at index 0.\n\n**`m_downloads`** — `aux::array` of 4 sorted `vector` (one per download\ncategory), indexed by `download_queue_t`:\n- `piece_downloading` (0) — some blocks still open (not yet requested)\n- `piece_full` (1) — all blocks requested, at least one not yet writing/finished\n- `piece_finished` (2) — all blocks writing or finished\n- `piece_zero_prio` (3) — downloading but priority = 0\n\n`piece_downloading_reverse` and `piece_full_reverse` are pseudo-states stored in `download_state`\nto de-prioritize pieces only requested from reverse peers; they map to the same download vectors\nas their non-reverse counterparts.\n\n**`downloading_piece`** — one per in-flight piece:\n- `info_idx` (`uint16_t`) — index into `m_block_info`; the slice `[info_idx * blocks_per_piece, (info_idx+1) * blocks_per_piece)` holds the block states for this piece\n- `finished`, `writing`, `requested` (15 bits each) — block state counters\n- `passed_hash_check` (1 bit) — hash job returned OK (piece may not yet be on disk)\n- `locked` (1 bit) — blocks cannot be picked; set during error recovery, cleared by `restore_piece()`\n- `hashing` (1 bit) — outstanding hash request in flight\n\n**`m_block_info`** — flat `aux::vector`. Each downloading piece owns\n`blocks_per_piece()` consecutive entries identified by `info_idx`. Free ranges are recycled\nvia `m_free_block_infos` (a free-list of `uint16_t` indices).\n\n**`block_info`** — per block:\n- `peer` — last peer to request/write this block\n- `num_peers` (14 bits) — peers with this block outstanding\n- `state` (2 bits) — `state_none → state_requested → state_writing → state_finished`\n\n### Priority formula\n\n`piece_pos::priority(picker)` returns -1 (not pickable) if: filtered, have, no availability\n(`peer_count + m_seeds == 0`), `piece_full`, or `piece_finished`.\n\nOtherwise:\n```\npriority = availability * (priority_levels - piece_priority) * prio_factor + adjustment\n```\nwhere `priority_levels = 8`, `prio_factor = 3`, and:\n- `adjustment = -3` for actively downloading pieces (non-reverse)\n- `adjustment = -2` for open pieces\n- `adjustment = -1` for reverse-downloading pieces\n\nLower value = picked first. The `prio_factor = 3` creates three sub-levels per availability\ntier, letting downloading pieces beat open ones at the same availability.\n\n### Seeds optimization\n\nPeers with all pieces are tracked in `m_seeds` rather than incrementing every `peer_count`.\nTrue availability of a piece is `peer_count + m_seeds`. When a seed sends DONT_HAVE,\n`break_one_seed()` decrements `m_seeds` and increments every `piece_pos::peer_count`.\n\n### Lazy rebuild (`m_dirty`)\n\n`m_pieces` and `m_priority_boundaries` are rebuilt lazily. When `m_dirty` is true, they are\nstale; `update_pieces()` (called automatically before any pick) rebuilds them:\n1. Walk `m_piece_map`, count pieces per priority band → `m_priority_boundaries` holds deltas\n2. Make boundaries cumulative, resize `m_pieces`\n3. Fill `m_pieces` using per-piece relative offsets stored in `piece_pos::index`\n4. Shuffle each priority band randomly\n5. Fix up `piece_pos::index` to point back into the final positions in `m_pieces`\n\nBulk operations (e.g. `inc_refcount(bitfield)` touching many pieces) set `m_dirty = true`\nrather than updating incrementally. For small bitfield changes (< 50 pieces), the picker\nupdates incrementally via `update()` / `add()` / `remove()` without dirtying.\n\n### Sequential download cursors\n\n`m_cursor` — lowest piece index not yet had.\n`m_reverse_cursor` — one past the highest piece index not yet had (all pieces from here to end\nare had). `set_sequential_range(first, last)` constrains the cursor window.\n\n### Piece lifecycle\n\n```\nopen → downloading → full → finished → (removed, piece_pos::have set)\n```\n\n- `mark_as_downloading(block, peer)` — block: none→requested; piece may transition open→downloading\n- `mark_as_writing(block, peer)` — block: requested→writing\n- `mark_as_finished(block, peer)` — block: writing→finished; calls `update_piece_state()` which\n may advance the piece from downloading→full→finished\n- `piece_passed(index)` — hash check passed; sets `passed_hash_check`, calls `account_have()`,\n calls `piece_flushed()` if all blocks are also finished\n- `piece_flushed(index)` — piece is on disk; removes from `m_downloads`, removes from `m_pieces`\n via `remove()`, sets `piece_pos::index = we_have_index`\n- `restore_piece(index, blocks)` — after hash failure or write error; clears `locked`, optionally\n resets specific blocks back to `state_none`, re-opens the piece for downloading\n\n### `pick_pieces` options\n\n`picker_options_t` flags passed to `pick_pieces()`:\n- `rarest_first` — pick lowest-availability pieces (the default; driven by the priority formula)\n- `reverse` — pick most-common first, or last pieces if sequential\n- `sequential` — pick within the `[m_cursor, m_reverse_cursor)` window in order\n- `on_parole` — only pick pieces exclusively requested by this peer (after suspected bad data)\n- `prioritize_partials` — prefer pieces already in `m_downloads` over fresh ones\n- `piece_extent_affinity` — create affinity for 4 MiB extents of adjacent pieces\n- `align_expanded_pieces` — align contiguous-block expansion to natural boundaries\n\nAuto partial prioritization: if `num_partials > num_peers * 3 / 2` or\n`num_partials * blocks_per_piece > 2048`, `prioritize_partials` is forced on and\n`prefer_contiguous_blocks` is set to 0 to avoid unbounded partial piece growth.\n\n`prefer_contiguous_blocks` (int, not a flag) — request this many consecutive blocks from the\nsame piece; used by web peers to pipeline larger requests.\n","frontmatter":{"paths":["src/piece_picker.cpp","include/libtorrent/aux_/piece_picker.hpp","test/test_piece_picker.cpp"]},"isInternal":false,"tokens":1771,"sizeBytes":7070},{"name":"protocol-encryption.md","path":".claude/rules/protocol-encryption.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/protocol-encryption.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"# Protocol Encryption (MSE/PE) Handshake\n\nlibtorrent implements Message Stream Encryption (MSE), also called Protocol\nEncryption (PE). The implementation lives in `src/bt_peer_connection.cpp` and\n`src/pe_crypto.cpp` (crypto primitives in `include/libtorrent/aux_/pe_crypto.hpp`).\n\n## Overview\n\nMSE provides obfuscation (not strong security) against deep-packet inspection.\nIt uses a Diffie-Hellman key exchange to derive RC4 stream keys and wraps the\nBitTorrent handshake. The torrent info hash is obfuscated in the handshake so\nthe connection cannot be trivially identified.\n\nAfter the handshake, the connection body is RC4-encrypted only if\n`crypto_select` ended on `pe_rc4`. If `pe_plaintext` was selected, only the\nhandshake is obfuscated and post-handshake traffic is sent in the clear. This\nis tracked by the `m_rc4_encrypted` flag on `bt_peer_connection` and gates the\nencrypt/decrypt path in every send and receive routine.\n\n## Settings\n\n| Setting | Meaning |\n|---------|---------|\n| `pe_forced` | Always use PE (outgoing) / require PE (incoming) |\n| `pe_enabled` | Use PE if peer supports it, fall back to plaintext |\n| `pe_disabled` | Never use PE |\n\n`settings_pack::out_enc_policy` controls outgoing connections.\n`settings_pack::in_enc_policy` controls acceptance of incoming connections.\n`settings_pack::allowed_enc_level` is a bitmask (`pe_plaintext=1`, `pe_rc4=2`, `pe_both=3`).\n`settings_pack::prefer_rc4` selects RC4 over plaintext when both are offered.\n\n## Handshake States\n\nThe state machine lives in `bt_peer_connection::on_receive_impl()`. The\nhandshake differs between the initiating side (outgoing) and the accepting\nside (incoming).\n\n### Incoming (server) path\n\n1. **`read_pe_dhkey`** (96 bytes) -- read client's DH public key, respond\n with our DH public key + random PadB, transition to `read_pe_synchash`.\n2. **`read_pe_synchash`** (scans up to 512 bytes of PadA + the 20-byte hash) --\n scan received bytes for `hash('req1', S)` where S is the shared secret. The\n client may insert up to 512 bytes of PadA before this hash. Failure:\n `sync_hash_not_found`.\n3. **`read_pe_skey_vc`** (28 bytes) -- read the obfuscated SKEY hash (20\n bytes) and the encrypted VC (8 bytes). The SKEY hash is\n `hash('req2', info_hash) XOR hash('req3', S)` which reveals which torrent\n the client wants. Failure if torrent not found: `invalid_info_hash`.\n Failure if VC decrypts to non-zero: `invalid_encryption_constant`.\n4. **`read_pe_cryptofield`** (6 bytes) -- read encrypted\n `crypto_provide(4) + len_pad_c(2)`. Server selects a crypto mode and\n sends back its own VC + `crypto_select + pad`. Failure if no mode overlaps:\n `unsupported_encryption_mode`. Failure if `len_pad_c > 512`:\n `invalid_pad_size`.\n5. **`read_pe_pad`** (`len_pad_c + 2` bytes) -- skip encrypted padding, read\n `len_ia` (2 bytes).\n6. **`read_pe_ia`** (`len_ia` bytes) -- read Initial Application data (IA),\n the encrypted start of the BitTorrent handshake. Failure if `len_ia > 68`:\n `invalid_encrypt_handshake`.\n7. Falls through to **`read_protocol_identifier`** (regular BT handshake).\n\n### Outgoing (client) path\n\n1. **`on_connected`** -- send DH public key + random PadA,\n transition to `read_pe_dhkey`.\n2. **`read_pe_dhkey`** (96 bytes) -- read server's DH public key, then send\n sync hash + SKEY hash + encrypted(VC + crypto_provide + len_pad + PadC +\n len_IA) + encrypted(IA = BT handshake). Also send the BT handshake\n encrypted as IA. Transition to `read_pe_syncvc`.\n3. **`read_pe_syncvc`** (scans up to 512 bytes of PadB + the 8-byte VC) --\n scan for the encrypted 8-byte verification constant (all zeros encrypted\n with RC4). Failure if not found within the 512-byte pad window:\n `invalid_encryption_constant`.\n4. **`read_pe_cryptofield`** (6 bytes) -- read server's `crypto_select(4) +\n len_pad(2)`. Failure if selected mode not in our `allowed_enc_level`:\n `unsupported_encryption_mode_selected`.\n5. **`read_pe_pad`** (`len_pad` bytes) -- skip server's padding.\n6. Falls through to **`read_protocol_identifier`** (regular BT handshake).\n\n## Crypto Primitives\n\n### DH Key Exchange (`dh_key_exchange`)\n\n768-bit Diffie-Hellman with a fixed prime (MSE standard prime from BEP 10/MSE\nspec). `key = 2^random % prime`.\n\n- `dh_key_exchange()` -- generate local key pair\n- `compute_secret(remote_pubkey)` -- compute shared secret S; also computes\n `m_xor_mask = hash('req3', S)` used for SKEY obfuscation\n- `get_local_key()` -- 96-byte DH public key to send\n- `get_secret()` -- shared secret S\n- `get_hash_xor_mask()` -- `hash('req3', S)` for SKEY obfuscation\n\n### RC4 Key Derivation\n\nFor outgoing connections (client):\n```\nencrypt_key = hash(\"keyA\", S, SKEY)\ndecrypt_key = hash(\"keyB\", S, SKEY)\n```\nFor incoming connections (server):\n```\nencrypt_key = hash(\"keyB\", S, SKEY)\ndecrypt_key = hash(\"keyA\", S, SKEY)\n```\nwhere SKEY is the SHA-1 info hash of the torrent (or first 20 bytes of SHA-256\nfor v2 torrents). The RC4 stream is initialized by discarding the first 1024\nbytes (standard MSE requirement).\n\n### Message Layout (client -> server)\n\n```\n[96 bytes] client DH public key\n[0-512 bytes] PadA (random, encrypted sync hash follows)\n[20 bytes] sync_hash = hash('req1', S)\n[20 bytes] skey_obfusc = hash('req2', SKEY) XOR hash('req3', S)\nencrypt([8 bytes] VC = 0x00 * 8\n [4 bytes] crypto_provide (bit flags: 1=plaintext, 2=RC4)\n [2 bytes] len_PadC\n [len_PadC] PadC (random)\n [2 bytes] len_IA)\nencrypt([len_IA bytes] IA = start of BT handshake, always RC4 encrypted)\n```\n\n### Message Layout (server -> client)\n\n```\n[96 bytes] server DH public key\n[0-512 bytes] PadB (random)\nencrypt([8 bytes] VC = 0x00 * 8\n [4 bytes] crypto_select (chosen mode: 1 or 2)\n [2 bytes] len_PadD\n [len_PadD] PadD (random))\n[regular BT handshake, encrypted or not depending on crypto_select]\n```\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `src/pe_crypto.cpp` + `include/libtorrent/aux_/pe_crypto.hpp` | DH exchange, RC4 handler, encryption_handler |\n| `src/bt_peer_connection.cpp` | PE state machine and write helpers |\n| `fuzzers/src/pe_conn.cpp` | Session-level fuzzer for the PE handshake |\n| `fuzzers/src/pe_crypto_state.cpp` | Low-level fuzzer for DH + RC4 primitives |\n\n## Error Codes\n\n| Error | Trigger |\n|-------|---------|\n| `sync_hash_not_found` | sync hash not found within 512-byte window |\n| `invalid_info_hash` | SKEY hash doesn't match any known torrent |\n| `invalid_encryption_constant` | VC decrypts to non-zero bytes |\n| `unsupported_encryption_mode` | No overlap in crypto_provide vs allowed_enc_level |\n| `unsupported_encryption_mode_selected` | Server selected a mode not in our allowed list |\n| `invalid_pad_size` | len_pad_c > 512 |\n| `invalid_encrypt_handshake` | len_ia > 68 |\n","isInternal":false,"tokens":1888,"sizeBytes":6790},{"name":"python-bindings.md","path":".claude/rules/python-bindings.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/python-bindings.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"---\npaths:\n - \"bindings/python/**\"\n---\n\n## Python Bindings\n\nSource in `bindings/python/src/*.cpp`; one `.cpp` file per exposed subsystem (e.g. `session.cpp`, `torrent_handle.cpp`, `alert.cpp`). Built as a Python extension module (`libtorrent.so`) using boost.python.\n\n**Building:**\n```sh\ncd bindings/python && b2\n```\nCMake also builds them when `-Dpython-bindings=ON` is passed.\n\n**Testing:**\n\nTests live in `bindings/python/tests/*_test.py` and are run with pytest:\n```sh\ncd bindings/python && python -m pytest tests/\n```\nThere is a roughly one-to-one correspondence between `src/foo.cpp` and `tests/foo_test.py`. Tests must be isolated — no real network access, no persistent filesystem side-effects. Use `get_isolated_settings()` from `tests/lib.py` when constructing a session, and `tempfile.TemporaryDirectory()` for any disk I/O. See `bindings/python/tests/guidelines.txt` for the full testing conventions.\n\n**Type stubs:**\n\nThe type stub file is `bindings/python/libtorrent/__init__.pyi`. It was originally bootstrapped with `stubgen` but is now **maintained entirely by hand** — do not regenerate it with stubgen, as that would lose the carefully written annotations. When adding a new binding, add the corresponding type annotation to this file manually. The header comment in the file documents the conventions used (e.g. how boost.python enums, positional-only args, and the `_BoostBaseClass` metaclass are represented).\n\n**Design philosophy — classes over dicts:**\n\nPrefer exposing real C++ classes via boost.python rather than converting them to plain Python dicts. Earlier versions of the bindings returned plain `dict` objects for things like peer info and torrent status; these have been replaced with proper bound classes (`peer_info`, `torrent_status`, etc.) that mirror the C++ types. When adding new bindings, follow this pattern: expose the C++ struct/class directly rather than unpacking it into a dict.\n\nNote: boost.python strictly requires `dict` and `list` (not duck-typed `Mapping`/`Sequence` abstractions) for function arguments that cross the C++ boundary. This is a boost.python limitation and not a design choice; the type stubs reflect this with `dict` and `list` rather than `Mapping` or `Sequence`.\n","frontmatter":{"paths":["bindings/python/**"]},"isInternal":false,"tokens":503,"sizeBytes":2239},{"name":"strong-types.md","path":".claude/rules/strong-types.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/strong-types.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"---\npaths:\n - \"include/libtorrent/units.hpp\"\n - \"include/libtorrent/flags.hpp\"\n - \"include/libtorrent/index_range.hpp\"\n - \"include/libtorrent/**\"\n - \"src/**\"\n---\n\n## Strong Types\n\nThe codebase avoids using raw `int` or bare integers for indices and flags. Instead, use the strong type and flag facilities.\n\n**Index types** (`include/libtorrent/units.hpp`):\n\n`aux::strong_typedef` wraps an integer so it is incompatible with other integers and other strong types. Arithmetic with raw integers is not allowed; only arithmetic with the same type or its `diff_type` is permitted.\n\nTo define a new index type:\n```cpp\nusing my_index_t = aux::strong_typedef;\n```\n\nTo convert to/from the underlying integer, use an explicit cast:\n```cpp\npiece_index_t p{42};\nint raw = static_cast(p);\n```\n\nTo iterate over a range of indices, use `index_range` (`include/libtorrent/index_range.hpp`):\n```cpp\nfor (piece_index_t i : index_range{begin, end}) { ... }\n```\n\n**Flag types** (`include/libtorrent/flags.hpp`):\n\n`flags::bitfield_flag` wraps an unsigned integer as a type-safe bitfield. Flags of different types cannot be combined. Individual bit constants are defined using the `_bit` user-defined literal:\n\n```cpp\nusing torrent_flags_t = flags::bitfield_flag;\nstatic constexpr torrent_flags_t seed_mode = 0_bit;\nstatic constexpr torrent_flags_t upload_mode = 1_bit;\n```\n\nTo define a new flag type:\n```cpp\nusing my_flags_t = flags::bitfield_flag;\n```\n\nFlag values support `|`, `&`, `^`, `~`, `|=`, `&=`, `^=`, and `operator bool()` (true if any bit is set). To test a flag:\n```cpp\nif (flags & torrent_flags::seed_mode) { ... }\n```\n","frontmatter":{"paths":["include/libtorrent/units.hpp","include/libtorrent/flags.hpp","include/libtorrent/index_range.hpp","include/libtorrent/**","src/**"]},"isInternal":false,"tokens":462,"sizeBytes":1791},{"name":"v2-torrents.md","path":".claude/rules/v2-torrents.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/rules/v2-torrents.md","title":"Rules Skill","category":"anthropic-skill","format":"markdown","content":"# BitTorrent v2 Torrents (BEP 52)\n\n## Info Dict Structure\n\nV2 uses a nested **`file tree`** dict instead of a flat `files` list:\n- Keys are filename path components; non-leaf = directory (nested dict)\n- Leaf nodes have empty-string key `\"\"` → file metadata dict with `length`, `pieces root`, etc.\n- **`pieces root`** (32 bytes): root of the per-file merkle tree (SHA-256)\n- **`meta version`**: integer `2` identifies v2 metadata\n- **`piece layers`**: (outside info dict) maps `pieces root` → concatenated piece-layer hashes\n\nV2 torrents have a **SHA-256 info hash** (vs v1's SHA-1). `torrent_info::m_info_hash` holds both.\n\n## Per-File Merkle Tree\n\nEach file has an independent merkle tree (SHA-256, not SHA-1):\n\n```\nroot (pieces root)\n └── interior nodes\n └── piece layer ← hash of one piece's block subtree\n └── block layer (leaf) ← hash of 16 KiB block\n```\n\n- **Block** = 16 KiB (fixed), the unit of transfer\n- **Piece** = `piece_length` bytes; must be a power-of-2 multiple of 16 KiB\n- Tree is **padded to a power of 2** in leaf count using `merkle_pad()` (zeros hashed up)\n- Interior nodes: `parent = SHA-256(left_child || right_child)`\n- Flat array indexing: node at layer `l`, offset `o` → index `(1 << l) - 1 + o`; parent = `(i-1)/2`\n\nKey helpers in `src/merkle.cpp`:\n- `merkle_num_leafs()`, `merkle_num_nodes()`, `merkle_num_layers()`\n- `merkle_fill_tree()` — compute interior nodes from leaves\n- `merkle_validate_and_insert_proofs()` — validate a proof chain against a known root\n- `merkle_pad()` — padding hash for odd tree levels\n\n## merkle_tree Class (`include/libtorrent/aux_/merkle_tree.hpp`)\n\nOne instance per file, with four storage modes (optimization):\n\n| Mode | Stored nodes | When used |\n|------|-------------|-----------|\n| `empty_tree` | root only | single-block file, or tree completely unknown |\n| `piece_layer` | piece-layer hashes | piece hashes received, blocks not yet |\n| `block_layer` | block hashes | download complete, all blocks verified |\n| `full_tree` | all nodes | during partial verification |\n\n`m_block_verified` bitfield tracks which block hashes have been validated against their parent chain up to the known root.\n\nKey methods: `load_tree()`, `load_piece_layer()`, `add_hashes()`, `set_block()`\n\n## Hybrid Torrents (v1 + v2)\n\nA single `.torrent` file can contain both:\n- v1 info dict: `files` list + `pieces` SHA-1 concatenation\n- v2 `file tree` + `piece layers`\n\nConstraints: identical file layout, same `piece_length`. Results in two info hashes.\n`torrent_info` parsing uses `extract_files()` for v1 and `extract_files2()` for v2.\n\n## Hash Picker (`src/hash_picker.cpp`)\n\nDecides which merkle hashes to request from peers (BitTorrent v2 hash exchange protocol).\n\n**`hash_request` struct:**\n```cpp\nstruct hash_request {\n file_index_t file;\n int base; // tree layer (0 = block layer)\n int index; // starting offset in layer\n int count; // number of hashes (≤ 8192)\n int proof_layers; // uncle hashes needed to prove against known root\n};\n```\n\n**Strategy:**\n1. Request piece-layer hashes in 512-piece chunks until all known\n2. On piece failure, request all block hashes for that piece (3s minimum interval)\n\n**Validation flow:**\n- `add_hashes()`: receives hashes + uncle proof chain, calls `merkle_validate_and_insert_proofs()`, inserts valid nodes, reports `pass`/`fail` per piece\n- `set_block()`: inserts a single block hash, finds largest verifiable subtree, returns pass/fail\n\n## Resume Data\n\nMerkle trees are persisted across sessions in resume data (`src/write_resume_data.cpp`, `src/read_resume_data.cpp`):\n\n```\n[\"merkle trees\"][file_index] → sparse tree nodes (non-zero nodes only)\n[\"merkle tree mask\"][file_index] → bitfield of which nodes are stored\n[\"verified bits\"][file_index] → m_block_verified bitfield\n[\"piece layers\"][pieces_root] → concatenated piece-layer hashes\n```\n\n- `write_resume_data()` uses `build_sparse_vector()` to encode only known nodes\n- `read_resume_data()` calls `load_sparse_tree()` to restore the partial tree\n- Piece layers are re-derived from the stored tree during write\n\n## Verification Flow\n\n1. File added → `merkle_tree` created with root hash from `pieces root`\n2. Block arrives → `set_block()` inserts hash, attempts validation up to root\n3. Piece complete → all block hashes in piece set; piece passes if tree validates\n4. Piece fails → clear invalid nodes; hash picker schedules block-hash requests\n5. Hashes received → `add_hashes()` validates proof chain, inserts valid nodes, marks failures\n6. Download complete → all blocks verified; block layer mode; tree fully populated\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `src/merkle.cpp` | Tree math, proof validation, padding |\n| `src/merkle_tree.cpp` + `include/libtorrent/aux_/merkle_tree.hpp` | Per-file tree storage |\n| `src/hash_picker.cpp` + `include/libtorrent/aux_/hash_picker.hpp` | Hash request scheduling |\n| `src/torrent_info.cpp` | Parsing `file tree`, `piece layers`, hybrid logic |\n| `src/create_torrent.cpp` | Building v2/hybrid `.torrent` files |\n| `src/write_resume_data.cpp` + `src/read_resume_data.cpp` | Persisting trees |\n","isInternal":false,"tokens":1328,"sizeBytes":5211},{"name":"vcpkg.json","path":"tools/vcpkg.json","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/tools/vcpkg.json","title":"Tools Skill","category":"anthropic-skill","format":"json","content":"{\n \"$schema\": \"https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json\",\n \"builtin-baseline\": \"d90a9b159c08169f39adcd1b0f1ac0ca12c4b96c\",\n \"dependencies\": [\"openssl\"],\n \"overrides\": [\n {\n \"name\": \"openssl\",\n \"version-string\": \"3.6.1\"\n }\n ]\n}","frontmatter":{"$schema":"https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json","builtin-baseline":"d90a9b159c08169f39adcd1b0f1ac0ca12c4b96c","dependencies":["openssl"],"overrides":[{"name":"openssl","version-string":"3.6.1"}]},"isInternal":false,"tokens":98,"sizeBytes":289},{"name":"CLAUDE.md","path":".claude/CLAUDE.md","rawUrl":"https://raw.githubusercontent.com/arvidn/libtorrent/HEAD/.claude/CLAUDE.md","title":".claude - Claude Agent Guidelines & System Prompt","category":"claude-rule","format":"markdown","content":"# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Overview\n\nlibtorrent-rasterbar is a C++17 BitTorrent library (version 2.1.0). It supports v1 and v2 torrents, DHT, WebTorrent (WebRTC), and optional I2P/SSL.\n\n## Build Systems\n\nThe project has **two build systems**: boost-build (b2/bjam) and CMake. **Boost-build is the preferred build system** per the contributing guidelines.\n\n### Boost-build (b2) - preferred\n\n**Never add `-j` (parallel jobs) flags to b2 commands.** b2 manages parallelism on its own.\n\nBuild the library (from repo root):\n```sh\nb2\n```\n\nRun unit tests:\n```sh\ncd test && b2\n```\n\nRun a single test:\n```sh\ncd test && b2 test_session\n```\n\nRun deterministic tests (no flaky networking):\n```sh\ncd test && b2 deterministic-tests\n```\n\nRun simulations:\n```sh\ncd simulation && b2\n```\n\nBuild with developer options (typical for development):\n```sh\ncd test && b2 asserts=on invariant-checks=full debug-iterators=on picker-debugging=on\n```\n\nBuild with sanitizers:\n```sh\ncd test && b2 address-sanitizer=norecover undefined-sanitizer=norecover asserts=on\n```\n\nBuild examples, tools, fuzzers, python bindings:\n```sh\ncd examples && b2\ncd tools && b2\ncd fuzzers && b2 clang\ncd bindings/python && b2\n```\n\n### CMake — alternative\n\n```sh\ncmake -B build -DCMAKE_BUILD_TYPE=Debug -Dbuild_tests=ON\ncmake --build build\ncd build && ctest\n```\n\nRun a single test via CMake:\n```sh\ncd build && ./test/test_session\n```\n\n## Pre-commit\n\nThe repo uses [pre-commit](https://pre-commit.com). Before pushing:\n```sh\npip install pre-commit\npre-commit install\npre-commit run --all-files\n```\n\nPre-commit hooks include trailing whitespace, YAML/TOML/XML checks, RST formatting, Python formatting (black, isort, flake8, mypy, autoflake), and auto-generation of:\n- `include/libtorrent/fwd.hpp` and `include/libtorrent/libtorrent.hpp` via `tools/gen_fwd.py` and `tools/gen_convenience_header.py`\n- C binding headers (`bindings/c/include/libtorrent_settings.h`, `libtorrent_alerts.h`) via `bindings/c/tools/gen_header.py` and `bindings/c/tools/gen_alert_header.py`\n\nThe C++ format check (`git-clang-format-18`) is the only hook that requires a\nsystem install -- `apt install clang-format-18` (or platform equivalent). It\nformats only the lines that differ from HEAD, not whole files; the codebase is\nbeing migrated to the `.clang-format` style incrementally rather than via a\nflag-day reformat.\n\n## Key Architecture\n\n### Threading Model\n\nThree thread categories:\n1. **Main network thread** — all sockets, session/torrent/peer state, boost.asio event loop\n2. **Disk I/O thread(s)** — reads, writes, and SHA-1/SHA-256 piece verification (count controlled by `settings_pack::aio_threads`)\n3. **Resolver thread** — spawned by boost.asio for async DNS on platforms without native async getaddrinfo\n\nAll interaction with session/torrent state from outside must go through the network thread (via `session_handle` and `torrent_handle`).\n\n### Core Classes\n\n- **`session`** (`include/libtorrent/session.hpp`) — public API, pimpl over `session_impl`\n- **`session_impl`** (`include/libtorrent/aux_/session_impl.hpp`) — all session state: torrent list, connection list, global rate limits, DHT state, port mapping\n- **`torrent`** (`include/libtorrent/aux_/torrent.hpp`, `src/torrent.cpp`) — all state for a single swarm: piece picker, peer connections, file storage\n- **`torrent_handle`** (`include/libtorrent/torrent_handle.hpp`) — public pimpl handle, weak reference to `torrent`; sends messages to network thread\n- **`peer_connection`** / **`bt_peer_connection`** — BitTorrent protocol implementation\n- **`piece_picker`** — download strategy (which blocks to request from which peers). See `.claude/rules/piece-picker.md` for a detailed description.\n- **`peer_list`** — list of known (not necessarily connected) peers for a swarm\n\n### Disk I/O\n\nThree disk backends:\n- `mmap_disk_io` — mmap-based (default on 64-bit when mmap is available)\n- `posix_disk_io` — fallback single-threaded POSIX I/O (used on 32-bit or without mmap)\n- `pread_disk_io` — multi-threaded backend using `pread()`/`pwrite()`. See `.claude/rules/disk-cache.md` for a detailed description.\n\n### DHT (Kademlia)\n\nSource in `src/kademlia/`, headers in `include/libtorrent/kademlia/`. Includes ed25519 signing (`src/ed25519/`).\n\n### Extensions\n\nProtocol extensions live in `src/` and `include/libtorrent/extensions/`:\n- `ut_metadata` — magnet link metadata exchange\n- `ut_pex` — peer exchange\n- `smart_ban` — ban peers that send corrupt data\n- `i2p_pex` — peer exchange for I2P peers\n\n### WebTorrent (WebRTC)\n\nOptional feature controlled by `webtorrent=on` / `-Dwebtorrent=ON`. Uses `deps/libdatachannel` and the `rtc_stream`/`rtc_signaling` subsystem.\n\n### Python Bindings\n\nSource in `bindings/python/src/*.cpp`, built as `libtorrent.so` using boost.python. See `.claude/rules/python-bindings.md` for detailed conventions.\n\n### Simulations\n\n`simulation/` contains network simulation tests that use `libsimulator` (a deterministic virtual network). These test high-level behaviors (swarms, DHT, session management) without real network access. All simulation tests run with `invariant-checks=full`, `asserts=on`, `debug-iterators=on`.\n\n### Terminology\n\n- **piece** — SHA-1/SHA-256 verified chunk of torrent data (typically power-of-two size)\n- **block** — 16 KiB sub-unit of a piece (the transfer unit in the protocol)\n- **torrent_peer** — a known-but-not-connected peer entry\n- **peer_connection** — an active connection to a peer\n\n## Key Build Options (b2 features)\n\n| Feature | Values |\n|---------|--------|\n| `crypto` | `built-in`, `openssl`, `openssl-shared`, `wolfssl`, `gcrypt`, `gnutls`, `libcrypto` |\n| `asserts` | `on`, `off`, `production`, `system` |\n| `invariant-checks` | `off`, `on`, `full` |\n| `logging` | `on`, `off` |\n| `deprecated-functions` | `on`, `off`, `1`, `2`, `3`, `4` |\n| `webtorrent` | `on`, `off` |\n| `address-sanitizer` | `norecover`, `recover`, `off` |\n| `undefined-sanitizer` | `norecover`, `recover`, `off` |\n| `thread-sanitizer` | `norecover`, `recover`, `off` |\n| `picker-debugging` | `on`, `off` |\n| `debug-iterators` | `default`, `off`, `on`, `harden` |\n\n`deprecated-functions` selects the ABI/API version via `TORRENT_ABI_VERSION`. Different values are **link-incompatible**. Higher numbers expose a more modern interface and remove older deprecated APIs:\n\n| Value | `TORRENT_ABI_VERSION` | Corresponds to |\n|-------|-----------------------|----------------|\n| `on` | (oldest supported) | libtorrent 1.x |\n| `1` | 1 | libtorrent 1.x |\n| `2` | 2 | libtorrent 2.0 |\n| `3` | 3 | libtorrent 2.x |\n| `4` | 4 | libtorrent 2.1 |\n| `off` | 100 (newest) | latest API |\n\n## Directory Layout\n\n```\nsrc/ main library source\nsrc/kademlia/ DHT implementation\nsrc/ed25519/ ed25519 signing (vendored)\ninclude/libtorrent/ public API headers\ninclude/libtorrent/aux_/ internal headers (not public API)\ninclude/libtorrent/kademlia/ DHT headers\ninclude/libtorrent/extensions/ extension headers\ntest/ unit tests (test_*.cpp)\nsimulation/ network simulation tests (test_*.cpp)\nfuzzers/src/ fuzz targets\nexamples/ example programs\ntools/ utility scripts and programs\nbindings/python/ Python bindings (boost.python)\nbindings/c/ C API bindings\ndeps/ vendored dependencies\n deps/try_signal/ signal handling\n deps/asio-gnutls/ GnuTLS adapter for asio\n deps/json/ nlohmann/json (for WebTorrent)\n deps/libdatachannel/ WebRTC (for WebTorrent)\n```\n\n## Adding New Source Files\n\nWhen adding a new `.cpp` or `.hpp` file, it must be added to **all three** build systems:\n1. `Jamfile` (boost-build)\n2. `CMakeLists.txt`\n3. `Makefile` (if applicable)\n\n## Code-Generation Tools\n\nAfter modifying public API headers, regenerate the derived headers by running these scripts from the repo root:\n\n- `tools/gen_fwd.py` — regenerates `include/libtorrent/fwd.hpp` (forward declarations for all public types)\n- `tools/gen_convenience_header.py` — regenerates `include/libtorrent/libtorrent.hpp` (the convenience header that includes all public headers)\n\nThese are also run automatically by the pre-commit hooks.\n\n## Coding Conventions\n\n- Comments must use ASCII characters only (no Unicode, smart quotes, em-dashes, etc.)\n- Use a single space after a period in comments (not two spaces)\n- Declare variables `const` whenever they are not reassigned after initialization\n- C++17 throughout; no C++20 features yet\n- The code must build with standard C++ and must not require compiler language extensions; optional extensions (e.g. `__builtin_*`, platform-specific pragmas) may be used when guarded by feature detection\n- `namespace lt = libtorrent` alias is always available\n- Asserts: use `TORRENT_ASSERT(cond)` (active when `TORRENT_USE_ASSERTS` is defined, i.e. debug builds)\n- Invariant checks: expensive checks inside `#if TORRENT_USE_INVARIANT_CHECKS`\n- Public API headers use `TORRENT_EXPORT` macro; internal symbols use hidden visibility\n- Internal functions and classes use `TORRENT_EXPORT_EXTRA` macro; to grant access to tests\n- ABI versioning via `TORRENT_VERSION_NAMESPACE_2/3/4` inline namespace macros (defined in `include/libtorrent/aux_/export.hpp`); `_2` = v1.2, `_3` = v2, `_4` = v2.1\n- Warnings are treated as errors in CI (both gcc and clang)\n- Changes to ABI (fields/ordering of public classes) must target `master`, not `RC_*` stable branches\n- `TORRENT_EXPORT_EXTRA` marks a symbol exported only for test access; it is private/internal, not public API, and is exempt from the ABI-stability rules above (safe to change on `RC_*` branches). Do not treat `TORRENT_EXPORT_EXTRA` as indicating public or ABI-sensitive status; `TORRENT_EXPORT` is the public one\n- `settings_pack` enum values must be appended at the end of each enum group (int, bool, string) — never inserted in the middle — to avoid changing the numeric values of existing settings and breaking ABI\n- prefer the C++ counterparts to C headers\n- do not use using-statements in header files\n- prefer to fully qualify standard types and functions\n- prefer the short lt namespace alias when fully qualifying libtorrent types\n- prefer using default member initializers over initializer list\n\n### Strong Types\n\nAvoid raw `int` for indices and flags; use `aux::strong_typedef` (`include/libtorrent/units.hpp`) and `flags::bitfield_flag` (`include/libtorrent/flags.hpp`). See `.claude/rules/strong-types.md` for details.\n","isInternal":false,"tokens":2736,"sizeBytes":10720}],"systemPromptSnippet":"\n\n---\npaths:\n - \"src/kademlia/**\"\n - \"include/libtorrent/kademlia/**\"\n---\n\n# DHT (Kademlia) Implementation\n\nSource: `src/kademlia/`, `include/libtorrent/kademlia/`\n\n## BEP Support\n\n| BEP | Description | Status |\n|-----|-------------|--------|\n| BEP 5 | DHT Protocol (ping, find_node, get_peers, announce_peer) | Full |\n| BEP 32 | IPv6 extension (dual-stack, nodes6) | Full |\n| BEP 33 | DHT Scrape (bloom filters BFpe/BFsd) | Full |\n| BEP 42 | Node ID Verification (IP-derived IDs) | Full |\n| BEP 44 | Arbitrary data storage (immutable + mutable items, Ed25519) | Full |\n| BEP 51 | DHT Infohash Indexing (sample_infohashes) | Full |\n\n---\n\n## File Overview\n\n| File | Purpose |\n|------|---------|\n| `node.cpp` / `node.hpp` | Central DHT node — routing, dispatch, traversal lifecycle |\n| `dht_tracker.cpp` / `dht_tracker.hpp` | Session-level coordinator, one `node` per listen socket |\n| `routing_table.cpp` / `routing_table.hpp` | K-bucket routing table |\n| `rpc_manager.cpp` / `rpc_manager.hpp` | Tracks outstanding RPCs, calls observers on response/timeout |\n| `observer.hpp` | Base observer class for pending RPC state |\n| `traversal_algorithm.cpp` / `.hpp` | Base iterative search algorithm |\n| `find_data.cpp` / `.hpp` | Find-nodes traversal, base for get_peers and get_item |\n| `get_peers.cpp` / `.hpp` | BEP 5 peer lookup traversal |\n| `get_item.cpp` / `.hpp` | BEP 44 item retrieval traversal |\n| `put_data.cpp` / `.hpp` | BEP 44 item storage traversal |\n| `sample_infohashes.cpp` / `.hpp` | BEP 51 traversal |\n| `refresh.cpp` / `.hpp` | Bootstrap / bucket-refresh traversal |\n| `direct_request.hpp` | Single-message direct request |\n| `node_id.cpp` / `node_id.hpp` | `node_id` type (alias for `sha1_hash`), XOR distance, BEP 42 generation |\n| `node_entry.cpp` / `node_entry.hpp` | Single routing-table entry |\n| `item.cpp` / `item.hpp` | Mutable/immutable item type |\n| `types.hpp` | `public_key`, `secret_key`, `signature`, `sequence_number` |\n| `msg.hpp` | Incoming message structure |\n| `dht_storage.cpp` / `dht_storage.hpp` | Pure-virtual storage interface + default RAM implementation |\n| `dht_state.cpp` / `dht_state.hpp` | Persistent state (node IDs, bootstrap nodes) |\n| `dht_settings.hpp` | Per-DHT settings struct |\n| `dht_observer.hpp` | Callbacks from DHT into `session_impl` |\n| `dos_blocker.cpp` / `dos_blocker.hpp` | Per-IP rate limiting (5 pkt/s, 5-min block) |\n| `ed25519.hpp` | Ed25519 sign/verify API (vendored impl in `src/ed25519/`) |\n| `announce_flags.hpp` | Announce flags: `seed`, `implied_port`, `ssl_torrent` |\n\n---\n\n## Threading\n\nAll DHT code runs on the **session network thread** (boost.asio). No locking needed inside DHT classes. `rpc_manager` uses a mutex only for its observer-pool allocator.\n\n---\n\n## Core Data Structures\n\n### `node_id`\nType alias for `sha1_hash` (160 bits).\n\nKey functions:\n- `distance(a, b)` — XOR metric\n- `distance_exp(a, b)` — index of highest differing bit (0–160)\n- `compare_ref(a, b, ref)` — compare distances to reference\n- `generate_id(external_ip)` — BEP 42 derivation\n- `verify_id(nid, source_ip)` — BEP 42 check\n\n### `node_entry`\nOne routing-table peer entry:\n- `node_id id`, `udp::endpoint endpoint`\n- `uint16_t rtt` (0xffff = unknown)\n- `uint8_t timeout_count` (0 = confirmed, 0xff = never pinged)\n- `bool verified` (BEP 42)\n- `time_point last_queried`\n\n### `routing_table`\n160 k-buckets (one per XOR distance bit). Extended routing table (default on) gives larger buckets close to us:\n\n| Bucket index | Multiplier | Max live nodes (k=20) |\n|---|---|---|\n| 0 | 16× | 320 |\n| 1 | 8× | 160 |\n| 2 | 4× | 80 |\n| 3 | 2× | 40 |\n| 4–159 | 1× | 20 (k) |\n\nKey behaviours:\n- Last (closest) bucket splits when it exceeds k entries\n- Each bucket has a **replacement cache**; candidates promoted when live node is removed\n- `ip_set` restricts one node per IP per bucket (BEP 42, `restrict_routing_ips`)\n- Periodic refresh: random query in each bucket's range every ~15 min\n\n### `item` (BEP 44)\n- **Immutable** — key is `sha1(value)`, no signature\n- **Mutable** — key is `sha1(salt || pk)`, Ed25519-signed, has `sequence_number`\n- CAS: `cas` field lets clients do compare-and-swap on sequence number\n\n### `dht_storage_interface`\nPure-virtual; default RAM implementation (`dht_default_storage_constructor`):\n\n```\nmap m_torrents (BEP 5 peers)\nmap m_immutable_items\nmap m_mutable_items\n```\n\nEviction uses distance from our node IDs — items far from us are evicted first.\nLimits: `max_torrents` (2000), `max_peers` (500), `max_dht_items` (700).\nPeer entries expire after 30 min (announce_interval).\n\n---\n\n## Key Classes\n\n### `node`\nOne per listen socket. Owns `routing_table` and `rpc_manager`.\n\nImportant methods:\n```cpp\nvoid bootstrap(endpoints, done_cb);\nvoid get_peers(info_hash, callback, flags);\nvoid announce(info_hash, port, flags, callback);\nvoid get_item(target, callback); // immutable\nvoid get_item(pk, salt, callback); // mutable\nvoid put_item(data, callback); // immutable\nvoid put_item(pk, sk, salt, callback); // mutable\nvoid sample_infohashes(endpoint, target, callback);\nvoid incoming(udp::endpoint, msg); // dispatch received UDP\nvoid tick(); // token rotation, bucket refresh\n```\n\nWrite tokens: `hash(requester_ip + secret + info_hash)`. Two secrets kept; rotated periodically to allow stale tokens for one rotation period.\n\n### `dht_tracker`\nOwns `std::map` (one `node` per socket).\nAggregates results across all nodes. Manages DOS blocking. Saves/restores `dht_state`.\n\n### `rpc_manager`\n- `invoke(entry, endpoint, observer_ptr)` — sends bencoded RPC, tracks by 2-byte transaction ID\n- `incoming(msg, &sender_id)` — looks up observer by txn-id, calls `observer::reply()`\n- `tick()` — fires `observer::timeout()` / `observer::short_timeout()` for stale requests\n- Uses an object pool for observers to reduce heap fragmentation\n\n### `observer`\nOne per pending RPC. Flags:\n\n| Flag | Meaning |\n|------|---------|\n| `flag_queried` | Request sent |\n| `flag_initial` | First attempt |\n| `flag_short_timeout` | Intermediate timeout fired |\n| `flag_failed` | Node failed |\n| `flag_alive` | Response received |\n| `flag_done` | Finished, no longer tracked |\n\nObserver hierarchy:\n```\nobserver\n ├─ traversal_observer (parses \"nodes\"/\"nodes6\" from responses)\n │ ├─ find_data_observer\n │ │ ├─ get_peers_observer (also parses \"values\", bloom filters)\n │ │ └─ get_item_observer (parses v/k/sig/seq)\n │ ├─ put_data_observer\n │ └─ sample_infohashes_observer\n ├─ announce_observer\n ├─ direct_observer\n └─ null_observer\n```\n\n---\n\n## Traversal Algorithms\n\nAll searches inherit from `traversal_algorithm`. The base implements the Kademlia iterative lookup:\n\n1. Seed result set with k closest nodes from routing table\n2. Issue α (default 3) concurrent requests (`add_requests()`)\n3. Each response: add new closer nodes, sort result set\n4. Repeat until k closest nodes all responded or timeout\n5. Call `done()`\n\n| Class | BEP | RPC sent | Callback delivers |\n|-------|-----|----------|-------------------|\n| `find_data` | 5 | `find_node` | closest nodes + write tokens |\n| `get_peers` | 5 | `get_peers` | `vector` (peers) |\n| `get_item` | 44 | `get` | `item`, `bool authoritative` |\n| `put_data` | 44 | first `get`, then `put` | response count |\n| `sample_infohashes` | 51 | `sample_infohashes` | samples, interval, total |\n| `bootstrap` / `refresh` | 5 | `get_peers` (our own ID) | (fires bucket refresh) |\n| `direct_traversal` | — | arbitrary | single `msg` |\n\n`obfuscated_get_peers`: wraps `get_peers`, uses a random target until convergence, then switches to the real info_hash (BEP 5 privacy mode, controlled by `dht_privacy_lookups`).\n\n---\n\n## Message Format (bencode)\n\nAll messages are bencoded dicts with mandatory keys `\"t\"` (transaction id), `\"y\"` (`\"q\"/\"r\"/\"e\"`), optional `\"v\"` (version), `\"ip\"` (external address, BEP 42), `\"ro\"` (read-only, BEP 5).\n\n**Queries** (`y=\"q\"`): `\"q\"` = method name, `\"a\"` = argument dict with `\"id\"` (our node id, 20 bytes).\n\n**Responses** (`y=\"r\"`): `\"r\"` = response dict.\n\n**Errors** (`y=\"e\"`): `\"e\"` = `[error_code, \"message\"]`.\n\n### Method summary\n\n| Method | Key request fields | Key response fields |\n|--------|--------------------|---------------------|\n| `ping` | `id` | `id` |\n| `find_node` | `id`, `target`, `want` | `id`, `nodes`, `nodes6` |\n| `get_peers` | `id`, `info_hash`, `noseed`, `scrape`, `want` | `id`, `token`, `values`\\|`nodes`, `BFpe`, `BFsd` |\n| `announce_peer` | `id`, `info_hash`, `port`, `token`, `implied_port`, `seed` | `id` |\n| `get` (BEP 44) | `id`, `target`, [`seq`] | `id`, `v`, [`k`, `sig`, `seq`] |\n| `put` (BEP 44) | `id`, `v`, [`k`, `sig`, `seq`, `salt`, `cas`] | `id` |\n| `sample_infohashes` | `id`, `target` | `id`, `samples`, `num`, `interval`, `nodes` |\n\nCompact node encoding: 26 bytes per IPv4 node (20-byte id + 4-byte IP + 2-byte port), 38 bytes for IPv6.\n\n---\n\n## Bootstrap\n\n1. Load `dht_state` (saved node IDs + endpoints)\n2. Assign node ID: saved → BEP 42 from external IP → random\n3. Add router nodes from settings (`router.bittorrent.com:6881`, etc.)\n4. Run `bootstrap` traversal against saved/configured endpoints (queries `get_peers` for own ID)\n5. On completion, refresh all non-contiguous buckets\n6. `dht_observer::get_peers()` notifies `session_impl` to query for all active torrents\n\n---\n\n## Integration with `session_impl`\n\n`session_impl` owns `shared_ptr m_dht` and implements `dht_observer`:\n\n| Observer method | When called |\n|-----------------|-------------|\n| `set_external_address()` | DHT detects our external IP via `\"ip\"` field |\n| `get_listen_port(socket)` | Node asks for our announce port |\n| `get_peers(info_hash, peer)` | Another node announced a peer to us |\n| `announce(info_hash, endpoint)` | We received a get_peers for one of our torrents |\n| `on_dht_request(method, msg, reply)` | Custom extension message handler |\n\nSession calls:\n- `dht_tracker::incoming_packet(socket, endpoint, buffer)` — for each received DHT UDP packet\n- `dht_tracker::new_socket(handle)` / `delete_socket(handle)` — listen port changes\n- `dht_tracker::tick()` — periodic maintenance\n\nRelevant `settings_pack` keys:\n\n| Setting | Effect |\n|---------|--------|\n| `enable_dht` | Toggle DHT |\n| `dht_search_branching` | Alpha (concurrent requests) |\n| `dht_max_peers_reply` | Peers returned per get_peers response |\n| `dht_enforce_node_id` | Reject nodes failing BEP 42 check |\n| `dht_restrict_routing_ips` | One node per /24 IPv4 per bucket |\n| `dht_restrict_search_ips` | Sybil mitigation in traversals |\n| `dht_privacy_lookups` | Enable obfuscated_get_peers |\n| `dht_read_only` | Set `ro=1` flag, don't accept queries |\n| `dht_upload_rate_limit` | Response bytes/sec cap |\n| `dht_item_lifetime` | BEP 44 item expiry (0 = never) |\n| `dht_sample_infohashes_interval` | BEP 51 cache TTL |\n\n---\n\n## Constants and Tunables\n\n| Parameter | Value | Notes |\n|-----------|-------|-------|\n| Node ID | 160 bits | SHA-1 |\n| k (bucket size) | 20 | Configurable |\n| Alpha | 3 | Concurrent requests per traversal |\n| RPC full timeout | 15 s | |\n| RPC short timeout | ~6 s | Triggers next request early |\n| Token rotation | ~7 min | Two tokens kept |\n| Bucket refresh | 15 min | Per-bucket, measured from last query |\n| Peer announce validity | 30 min | |\n| DOS block rate | 5 pkt/s | Per source IP |\n| DOS block duration | 5 min | |\n| DOS tracker size | 20 | Most-recent offenders |\n\n\n---\npaths:\n - \"src/disk_cache*\"\n - \"src/pread_disk_io*\"\n - \"include/libtorrent/aux_/disk_cache*\"\n---\n\n## Disk Cache (`pread_disk_io`)\n\nThe disk cache (`src/disk_cache.cpp`, `include/libtorrent/aux_/disk_cache.hpp`) is used exclusively by `pread_disk_io`. It is a write-back cache for incoming blocks, deferred until pieces are complete or memory pressure forces a flush.\n\n**Data structures:**\n\n- `disk_cache` — the cache itself; holds a `boost::multi_index_container` of `cached_piece_entry` objects, protected by a single `std::mutex`. The container has five indices: by `piece_location` (ordered), by `cheap_to_flush()` (ordered descending), by `need_force_flush()` (ordered descending), by `piece_location` (hashed, for fast lookup), and by `needs_hasher_kick()` (ordered descending).\n- `cached_piece_entry` — one entry per in-flight piece. Contains an array of `cached_block_entry` (one per block), an optional `piece_hasher` (`ph`, v1 only), an optional `block_hashes` array (`sha256_hash[]`, v2 only), and the cursors/flags described below.\n- `cached_block_entry::write_state` — `std::variant` with three mutually exclusive states:\n - `std::monostate` — no data present (the block has never been written, or its data has been fully released).\n - `disk_job*` — a pending write job is queued. The buffer is owned by the job; `take_write_job()` consumes the job during flushing.\n - `disk_buffer_ref` — the write has been flushed. The buffer may be non-null (`has_buf()`, \"held alive\" for future hashing) or null (already released). `is_flushed()` is true in either case and contributes to `flushed_cursor`.\n\n**Cursors:**\n\n- `hasher_cursor` — index of the first block not yet hashed (v1 SHA-1 only; hashing is always contiguous from block 0). Owned by the hashing thread while `hashing_flag` is set.\n- `flushed_cursor` — index of the first block not yet confirmed flushed to disk. Updated under the mutex after each flush completes.\n- `cheap_to_flush()` returns `hasher_cursor - flushed_cursor`: blocks that have been hashed but not yet flushed can be flushed without ever needing read-back.\n\n**Flags on `cached_piece_entry`:**\n\nFlags are stored in `cached_piece_flags flags` (a `bitfield_flag` bitfield). The mutex must be held to read or modify flags; updates go through `view.modify()` (required by `boost::multi_index`). Once a flag is set, the owning thread may access the protected state **without the mutex**:\n\n- `hashing_flag` — set under the mutex by the thread about to hash blocks. While set, that thread has exclusive access to `ph` (the v1 `piece_hasher`) and may advance `hasher_cursor`. No other thread may touch these. Cleared under the mutex when hashing is complete.\n- `flushing_flag` — set under the mutex by the thread about to write blocks to disk. While set, that thread has exclusive access to the block buffers being flushed. No other thread will attempt to flush the same piece. Cleared under the mutex when flushing is complete.\n- `force_flush_flag` — set when the piece has all blocks populated; prioritises this piece for flushing. Cleared once all blocks are flushed.\n- `piece_hash_returned_flag` — set once the piece hash has been computed and returned to the BitTorrent engine. The hash job is also the only signal that a piece is complete: it is what allows `flush_to_disk` to evict the `cached_piece_entry`, and what gets a piece force-flushed when the v1 hasher could not reach the end on its own (pure-v2 pieces, or pieces partially on disk at startup). When `disable_hash_checks` is set the engine still posts the job; the setting is latched into the storage (and into `torrent`) when the storage is created, at add time or when metadata arrives for a magnet link, so a later change never applies to it. In that mode `insert_write` caches the piece as neither v1 nor v2, so nothing is hashed on write, and `do_job(hash)` runs `hash_piece()` with a no-op callback purely for that bookkeeping and returns zero hashes.\n- `v1_hashes_flag` — set if this piece requires v1 SHA-1 hashing (`ph` is allocated).\n- `v2_hashes_flag` — set if this piece requires v2 SHA-256 block hashing (`block_hashes` is allocated).\n- `needs_hasher_kick_flag` — set when a block is inserted and the hasher thread should be woken; cleared when the hasher picks it up. Coalesces multiple insertions into one wakeup.\n- `notify_flushed_flag` — set by a `flush_storage()` caller that is waiting on a piece another thread is mid-flush on; `flush_piece_impl`'s `scope_end` checks it and signals `m_flushing_cv`. Only a wakeup signal supporting a single waiter, not a pin — the waiter re-looks-up the piece after each wakeup because it may have been flushed, hashed, or evicted while it slept.\n\nIn both hashing and flushing cases, the lock is **released** for the duration of the actual I/O (hashing or `pwrite()`), then re-acquired to update cursors/flags and potentially free buffers.\n\n**Deferred clear:** If `try_clear_piece()` is called while `flushing_flag` or `hashing_flag` is set, the clear job is stored in `cached_piece_entry::clear_piece` and executed by the owning thread once it clears its flag.\n\n**Held-alive buffers and the flush/hasher race:**\n\nWhen `flush_piece_impl` flushes a block, it checks `needed_by_hasher = (hashing_flag && block_index >= hasher_cursor)`. If a hasher is mid-snapshot (the `kick_hasher` window between snapshot and re-lock), the pointer to the buffer is in `blocks_storage[]` and must stay live. In that case the block transitions to `disk_buffer_ref{buf}` (non-null), and `m_blocks` is *not* decremented. Otherwise the buffer is freed immediately and `m_blocks` is decremented.\n\n`m_blocks` therefore counts both pending writes and held-alive buffers — both forms of memory pressure. `back_pressure` and `flush_request()` are driven off `m_blocks`.\n\nThe held-alive buffer is reclaimed when `kick_hasher` next advances the cursor past it (its cleanup loop frees blocks in `[old_cursor, new_cursor)` via `take_buf`). However, if v1 SHA-1 is blocked by a missing earlier block, the cursor cannot advance and the buffer stays held until either the missing block arrives or the piece is cleared. The \"drop held-alive\" flush pass (below) handles the case where no further inserts will arrive — without it, back-pressure can pin the cache above the low watermark indefinitely (the back-pressure observer is only notified once `m_blocks` drops to the low watermark, so the producer that tripped back-pressure would never be woken up). Note that v2 hashes for held-alive blocks are typically already in `block_hashes[]` (computed by the kick that triggered the keep-alive), so freeing the buffer only loses the v1 read-back optimization, not v2 hash work.\n\n**Flush strategy (`flush_to_disk`), in order:**\n\n1. Force-flush pass (`view`) — flush pieces with `force_flush` set (piece fully downloaded and hashed); ordered by `need_force_flush()`.\n2. Cheap flush pass (`view2`) — flush blocks in the range `[flushed_cursor, hasher_cursor)` (already hashed, no future read-back needed); pieces ordered by `cheap_to_flush()` descending.\n3. Drop held-alive pass (`view3`) — for each piece without `flushing_flag` or `hashing_flag`, free all blocks where `has_buf()` (held-alive `disk_buffer_ref{buf}`). No I/O — the data is already on disk. Done before the expensive pass because it's free; only skipped if the target is already met. Loses the v1 read-back optimization for these blocks; a subsequent `async_hash` will read them from disk.\n4. Expensive flush pass (`view4`) — flush any dirty blocks even if they haven't been hashed yet, requiring a read-back later to complete the piece hash. Used only when the cache exceeds the high-water mark.\n\nPasses 2-4 run only when `optimistic=false`. The optimistic flush at the top of each disk thread's loop runs only pass 1.\n\n**`flush_storage()`:** Called as a fence job during torrent teardown (`release_files`, `delete_files`, etc.) to flush every cached piece of one storage to disk. For each piece it flushes any still-dirty blocks (completing their write jobs via `flush_piece_impl`). It does **not** free or erase the piece — eviction is left to `flush_to_disk`. Keeping the piece preserves its partial hash state and lets another thread erase it concurrently without racing the flush. If a piece is mid-flush on another thread (`flushing_flag` set), it sets `notify_flushed_flag`, waits on `m_flushing_cv`, and re-looks-up the piece on each wakeup (it may be flushed, hashed, or evicted while waiting); if it is gone, it moves on. A concurrent `flush_storage()` already waiting on a piece (`notify_flushed_flag` already set) is left to that waiter, since the flag supports only one waiter.\n\n**Fences and the cache (`pread_disk_io`):** the general fence contract is in `.claude/rules/disk-io.md`; this is how `pread_disk_io` honours it with a write-back cache.\n\n- **Writes during a fence are queued, not aborted.** `async_write` calls `is_blocked()`, which takes ownership of the job and parks it in the fence's blocked queue. When the fence lowers, `disk_job_fence::job_complete` calls the `repost` callback (still under the fence mutex) once per blocked job in posting order; `add_completed_jobs_impl` inserts each reposted write into the cache there, and serves reposted reads from the cache (a normal disk read would return stale data, since the superseding write is still buffered). Because the resume runs under the fence mutex, a job posted concurrently is ordered behind the whole backlog. Work that can re-enter the fence (hasher kick, flush, completions) is deferred to after the mutex is released.\n- **`async_read` never consults the cache; `add_job` is the single funnel.** `async_read` only allocates a `job::read` (carrying the request verbatim, including an unaligned, block-spanning start) and hands it to `add_job`. `add_job` runs `is_blocked()` first, so a read posted while a fence is up is parked like any other job and cannot bypass the fence to serve stale data ahead of a queued write. The cache is consulted exactly once, by `prepare_read()`, and only after the job is past the fence gate: in `add_job` when the fence is down, or in the `repost` callback when the fence lowers. Both call `prepare_read()`; reposted jobs must **not** re-enter `is_blocked()` (a stacked fence still raises `m_has_fence`, so it would wrongly re-park them behind a later fence), which is why the consult is shared *after* the gate rather than by routing reposts back through `add_job`. `prepare_read()` mutates the job in place, so the single outstanding-job accounting `is_blocked()`/`job_complete()` set up is preserved.\n- **Every write is counted; writes during a fence are queued, not aborted.** `async_write` always routes through `is_blocked()` (never a fast path around it), so the write counts as an outstanding job on the storage whether or not a fence is up -- a fence then waits for buffered writes and their in-flight flush, exactly like a read or hash (see `.claude/rules/disk-io.md`). With no fence up, `is_blocked()` marks the job in-progress and increments the count, and the write is inserted into the cache inline; it stays counted until it is flushed and completed (or aborted). With a fence up, `is_blocked()` instead takes ownership of the job and parks it in the fence's blocked queue. When the fence lowers, `disk_job_fence::job_complete` calls the `repost` callback (still under the fence mutex) once per blocked job in posting order; `add_completed_jobs_impl` inserts each reposted write into the cache there, and serves reposted reads from the cache (a normal disk read would return stale data, since the superseding write is still buffered). Because the resume runs under the fence mutex, a job posted concurrently is ordered behind the whole backlog. Work that can re-enter the fence (hasher kick, flush, completions) is deferred to after the mutex is released.\n- **Reads racing the unblock.** `async_read`'s lock-free cache probe may run just before the backlog's writes land. `add_job` re-checks the cache once the fence is observed down, so a read is never dispatched to disk for a block whose write is sitting buffered.\n- **Forward progress (`m_fence_flush`).** A fence waits for the storage's outstanding jobs (including buffered writes) to reach zero, but nothing else flushes a partial piece under the watermark — so a raised fence hands its storage to a generic thread to flush (`flush_fenced_storages`), draining the writes it is waiting on. `add_fence_job` does this when a fence is raised over outstanding work (`fence_post_none`).\n- **Stacked fences.** When a fence lowers and reposts writes that sit ahead of a still-raised second fence, those partial-piece writes are now in the cache with no flush trigger. `add_completed_jobs_impl` detects a completed fence whose storage still `has_fence()` and re-arms `m_fence_flush` for it, so the next fence's wait can also make progress. (With no generic threads it flushes inline.)\n- **Unaligned reads and `job::partial_read`.** A peer read request whose start is not block-aligned can span two blocks. `prepare_read()` probes both with `disk_cache::get2`: if both blocks are in the cache it serves the whole request; if neither is, the job stays a `job::read` and `do_job(read)` reads the whole span from disk; if exactly one block is in the cache, it copies that side into the buffer and rewrites the job in place into a `job::partial_read` whose `offset`/`buffer_size`/`buffer_offset` describe just the missing side. `do_job(partial_read)` then reads only that side from disk into the rest of the buffer. Because `prepare_read()` runs past the fence gate against the *current* cache, the cached side it copies is never stale, and a `partial_read` therefore reads only from disk and never re-consults the cache. `prepare_read()` is the only place `partial_read` jobs are created -- they are never posted directly.\n\n\n---\npaths:\n - \"include/libtorrent/disk_interface.hpp\"\n - \"src/mmap_disk_io*\"\n - \"src/posix_disk_io*\"\n - \"src/pread_disk_io*\"\n - \"src/disk_job_fence*\"\n---\n\n## `disk_interface` contract\n\n`disk_interface` (`include/libtorrent/disk_interface.hpp`) is the session-level\ncustomization point for disk I/O, selected via\n`session_params::disk_io_constructor`. Built-in implementations: `mmap_disk_io`,\n`posix_disk_io`, `pread_disk_io`.\n\n**Threading.** Every `async_*` method is called from libtorrent's single network\nthread. The work may be performed asynchronously on the implementation's own\nthreads, but each completion handler **must be posted back to the network\nthread** via the `io_context` given to the constructor; handlers run on the\nnetwork thread. An implementation may use any number of threads and service\nqueued jobs concurrently and out of order, except where the contract below\nrequires ordering.\n\n**`async_hash` reflects all prior `async_write`s.** For a given piece, the\nengine posts all of the piece's blocks via `async_write`, then posts\n`async_hash` for that piece. The hash returned **must be computed over the data\nof every `async_write` for that piece that was posted before it.** This is the\nkey constraint for a multi-threaded back end that services jobs out of order: it\nmust order a piece's hash after that piece's writes and must never hash stale\nor missing data. The piece picker treats the returned hash as authoritative for\nthe data it handed over. `async_hash2` (single v2 block hash) has the same\nrequirement for the block it covers.\n\n**Fence jobs.** Several operations must run with exclusive access to a storage,\nsynchronized against all other I/O on it. These are *fence jobs*. A fence job:\n\n- waits for all jobs currently outstanding **on that storage** to finish, then\n executes -- so it runs alone, with no other operation on the storage in\n flight;\n- *fences* everything posted after it: any job posted to the same storage while\n the fence is raised is queued and not executed until the fence job completes.\n\nFences are per-storage -- a fence on one storage does not block jobs on another.\nThe fence operations are:\n\n- `async_clear_piece` -- synchronize with all outstanding I/O to the piece\n before the engine re-requests its blocks\n- `async_move_storage`\n- `async_release_files`\n- `async_stop_torrent`\n- `async_rename_file`\n- `async_delete_files`\n- `async_set_file_priority`\n- `async_check_files`\n\n`async_read`, `async_write`, `async_hash`, and `async_hash2` are ordinary\n(non-fence) jobs. The built-in back ends implement the fence semantics with\n`disk_job_fence` (`src/disk_job_fence.cpp`). A storage's outstanding-job count\nis what the fence waits to reach zero before it runs; a job stays counted from\nthe moment it is accepted until its completion is posted back.\n\n**Every ordinary job must honor the fence -- including a buffered write.** Each\njob, with no exceptions, has to participate in the outstanding-job count for its\nwhole lifetime, so the fence cannot run while any work on the storage is still\nin flight. This is easy to overlook for `async_write` on a write-back back end:\neven though the write is inserted into a cache and its handler is deferred, it\nmust be counted as outstanding from the moment it is accepted until the flush\nthat retires it has finished and its completion is posted -- *including the\nwindow while a disk thread is mid-flush writing it out*. It must not take a fast\npath that skips the count. If a buffered write (or the thread flushing it) is\ninvisible to the fence, the fence can be raised and complete while that write is\nstill being written to disk, and a teardown fence (`async_stop_torrent`,\n`async_release_files`, ...) can then tear the storage down underneath the\nin-flight flush -- a use-after-free. So the rule is symmetric: a write counts\nexactly like a read or a hash.\n\n### Fences and deferred (write-back) writes\n\nThe fence contract above is straightforward for an implementation that runs\neach job to completion immediately. An implementation that *defers* writes --\nreporting an `async_write` complete before the data is durably on disk, as a\nwrite-back cache does -- has to uphold two further invariants. Both are easy to\nget wrong, and a violation of either is a deadlock or silent data corruption,\nnot a degraded-performance bug.\n\n**A fence must not wait on writes that nothing will flush (forward progress).**\nA fence waits for every job outstanding on the storage to complete before it\nruns. When a write is only reported complete once it is durably stored, that\nwrite stays outstanding -- and keeps the fence waiting -- the whole time it sits\nbuffered. The flush that would retire it normally happens when the piece\ncompletes (its `async_hash`) or under memory pressure; but the completing hash\nis itself an ordinary job, now queued *behind* the very fence that is waiting\nfor it, and memory pressure may never arrive. The invariant: **raising a fence\nmust guarantee the storage's buffered writes get flushed, rather than waiting\npassively for a flush trigger the fence is itself blocking.** Otherwise the\nfence and its own backlog deadlock.\n\n**Resuming a fenced backlog must preserve posting order against new jobs.** When\na fence completes, the jobs that queued behind it are resumed. If resuming a job\nis observable -- a buffered write becomes visible, a read is served -- then a job\nthe network thread posts *during* that resume must be ordered after the whole\nresumed backlog, never interleaved ahead of it. The invariant: **a job arriving\nwhile the backlog is being resumed must not bypass the backlog.** Violate it and\na freshly-posted `async_read` overtakes a still-queued `async_write` for the\nsame block and returns stale data (or a hole).\n\n**A read resumed from a fence must reflect writes resumed ahead of it.** A\ncorollary of the ordering invariant: a read unblocked from the backlog may be\npreceded in that same backlog by a write to the very block it reads (both were\nblocked by the fence). That read must observe the write -- i.e. it must be\nserved from the buffered data, not from disk where the superseding write has not\nyet landed. Put plainly: the reads and writes a fence releases must take effect\nin the order they were posted, exactly as if the fence had never been raised.\n\n**Back-pressure.** `async_write` returns `true` when the write queue is over the\nhigh watermark; the caller then stops issuing writes for that peer until the\n`disk_observer` passed in is notified that the queue has drained. `false` means\nthe caller may keep writing.\n\n**Batching.** `async_*` calls only enqueue work; `submit_jobs()` is called once\nafter a batch to wake the disk thread(s). An implementation may also wake on each\n`async_*`, but should tolerate the batched notification.\n\n\n# Documentation Generation and Spell Checking\n\nThe reference documentation is generated from the C++ public headers; the\nprose manual is hand-written reStructuredText (RST). Both are built and\nspell-checked from the `docs/` directory via `docs/makefile`, and the same\ntargets run in CI (`.github/workflows/docs.yml`).\n\n## Reference Doc Generation (`docs/gen_reference_doc.py`)\n\n`gen_reference_doc.py` parses the public headers and emits RST. It scans\n`include/libtorrent/*.hpp`, `include/libtorrent/kademlia/*.hpp`, and\n`include/libtorrent/extensions/*.hpp` (add `--internal` to also include\n`include/libtorrent/aux_/*.hpp`). For each documented class, function, enum,\nand constant it produces an RST file per category (`reference-Core.rst`,\n`reference-Session.rst`, ...) plus a table of contents (`reference.rst`) and an\noptional `single-page-ref.rst` (`--single-page`).\n\n### How comments become docs\n\n- A documentation comment is the run of `//` comment lines immediately\n **above** a declaration, with no blank line between the comment and the\n declaration. A blank line, or a line that ends a declaration (`;`, `{`,\n `}`), resets the pending comment.\n- **Comment bodies are reStructuredText.** You can use RST markup directly in\n the `//` comments: `.. code::` blocks, `.. note::`, bullet lists,\n `parsed-literal`, headings, etc. The generator renders them verbatim into\n the output RST.\n- **Symbol cross-linking** is automatic: any word in a comment that matches a\n known symbol name (class, function, enum, `settings_pack::` setting, or one\n of the pre-defined manual anchors) is turned into a hyperlink by\n `linkify_symbols()`. A trailing underscore (`foo_`) forces a link the same\n way RST reference syntax does.\n\n### Grouping functions under one comment (man-page style)\n\nMultiple overloads or related functions can share a **single** documentation\ncomment. The rule is **adjacency with no blank line between the\ndeclarations**: when a function (or field) declaration immediately follows the\nprevious one with no blank line and no intervening comment, its signature is\nappended to the previous entry's signature list rather than starting a new\nentry. So a comment above the first of a group documents the whole group, the\nway a man page documents a family of related calls together. Inserting a blank\nline between declarations splits them into separately-documented entries (and\nthe second will then warn if it has no comment of its own).\n\n### Visibility and warnings\n\n- A comment beginning with `hidden` excludes the symbol entirely. A comment\n beginning with `internal` excludes it from the public docs (but `--internal`\n includes it).\n- Every public, non-trivial member must be documented. Undocumented\n symbols print `WARNING: ... is not documented`. Trivial members\n (destructors, default/copy/move ctors, assignment, comparison/stream/\n conversion operators) are exempt via `is_trivial_member()`.\n- A `TODO:` in a public doc comment is a hard error (exits non-zero).\n- `#if` blocks gated on debug-only macros (asserts, invariant checks, and the\n other debugging macros) and on old `TORRENT_ABI_VERSION` guards are skipped\n entirely, so debug-only or deprecated members do not appear in docs. A\n non-ABI `TORRENT_*` `#if` inside a public struct warns about possible ABI\n breakage.\n- `// OVERVIEW` introduces a section overview block for a header's category.\n\n### Plain-text output for spell checking\n\nWith `--plain-output`, `add_desc()` writes a `plain_text_out.txt` containing\n**only the prose** from doc comments, with the C++ signatures and `.. code::`\nblocks stripped out. This is what the spell checker runs over.\n\n## Other Generators\n\n| Script | Output | Source |\n|--------|--------|--------|\n| `gen_reference_doc.py` | `reference-*.rst`, `reference.rst`, `single-page-ref.rst`, `plain_text_out.txt` | public headers |\n| `gen_settings_doc.py` | `settings.rst` + appends settings names to `hunspell/settings.dic` | `settings_pack.hpp` |\n| `gen_stats_doc.py` | `stats_counters.rst` | `session_stats.cpp`, `performance_counters.hpp` |\n| `gen_todo.py` | `todo.html` | `src/*.cpp`, headers |\n| `filter-rst.py` | `*-plain.txt` (prose-only) | a hand-written `.rst` file |\n\n`gen_settings_doc.py` doubles as a dictionary source: it splits every setting\nname on `_` and adds the parts to `hunspell/settings.dic`, so setting-name\nfragments are not flagged as misspellings.\n\n## Spell Checking (hunspell)\n\nSpell checking uses **hunspell**, run via `make spell-check` from `docs/`:\n\n```sh\ncd docs && make spell-check\n```\n\nThe flow:\n1. Generated reference prose is in `plain_text_out.txt` (from\n `gen_reference_doc.py --plain-output`).\n2. Hand-written manuals are converted to prose-only text with\n `filter-rst.py` (e.g. `manual.rst` -> `manual-plain.txt`). `filter-rst.py`\n strips RST directives (`.. ` lines) and indented literal/code blocks so\n only natural-language text remains.\n3. `hunspell -l` lists misspelled words from each plain-text file into\n `hunspell-report.txt`, using dictionary `hunspell/en_US` plus the project\n word list `hunspell/libtorrent.dic` (settings text uses the generated\n `hunspell/settings.dic`). HTML manuals are checked with `-H` (HTML mode).\n4. If `hunspell-report.txt` is non-empty the target **fails** and prints the\n offending words.\n\n### Fixing spell-check failures\n\nEither correct the typo in the source, or -- if the word is a real term,\nidentifier, or acronym -- add it to **`docs/hunspell/libtorrent.dic`** (one\nword per line). `hunspell/en_US.{aff,dic}` is the stock English dictionary and\nshould not be edited. `hunspell/settings.dic` is generated (en_US + the\nsettings names + a copy of `libtorrent.dic`); do not hand-edit it.\n\n## Building the HTML\n\n```sh\ncd docs && make html # all RST -> HTML via rst2html, plus figures\ncd docs && make # default target builds the docs\n```\n\nHTML is produced by `rst2html` (docutils) with `template.txt`/`style.css`;\nfigures are rendered from `.dot` (graphviz), `.diagram` (aafigure), and `.svg`\n(imagemagick `convert`). `make stage` copies the built site to `$WEB_PATH`.\n`make clean` removes generated `.rst`/`.html`/figures (it does **not** delete\nthe checked-in hand-written `.rst` manuals).\n\n## CI\n\n`.github/workflows/docs.yml` installs `python3-docutils`, `hunspell`,\n`graphviz`, `imagemagick`, `aafigure`, etc., then runs\n`make spell-check html`. A documentation warning, an undocumented public\nsymbol, a `TODO:` in a doc comment, or a spelling error all fail the build.\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `docs/gen_reference_doc.py` | parse headers -> reference RST + plain text |\n| `docs/gen_settings_doc.py` | `settings.rst` + settings dictionary words |\n| `docs/gen_stats_doc.py` | `stats_counters.rst` |\n| `docs/filter-rst.py` | strip RST markup to prose for spell checking |\n| `docs/makefile` | `html`, `rst`, `spell-check`, `stage`, `clean` targets |\n| `docs/hunspell/libtorrent.dic` | project word list (edit to whitelist terms) |\n| `docs/hunspell/en_US.{aff,dic}` | stock English dictionary (do not edit) |\n| `.github/workflows/docs.yml` | CI: spell-check + build |\n\n\n---\npaths:\n - \"fuzzers/**\"\n - \"tools/gen_corpus.py\"\n---\n\n## Fuzzers\n\n### Build\n\nFuzzers require clang and are built separately from the main library:\n\n```sh\ncd fuzzers && b2 clang\n```\n\nBuilt with ASAN + UBSAN (norecover), debug symbols, asserts on, debug\niterators (`_GLIBCXX_DEBUG`), and WebTorrent enabled (per the default-build\nin `fuzzers/Jamfile`; note that `invariant-checks` is *not* turned on). Each\ntarget produces a standalone binary under `fuzzers/bin//` (e.g.\n`fuzzers/bin/clang-linux-20/...`). The explicit `stage` / `stage-large`\ninstall targets copy the binaries into the `fuzzers/fuzzers/` subdirectory\n(the `fuzzers` in `fuzzers/Jamfile` is relative to the Jamfile's\nown directory).\n\nTwo optional build features (defined in `fuzzers/Jamfile`):\n\n- `debug-logging=on` -- defines `DEBUG_LOGGING=1`, enabling extra alert\n logging in the session-level fuzzers (see peer_conn below).\n- `variant=build_coverage` -- a fast replay build with sanitizers, asserts,\n and invariant checks all off. Combine with `test-coverage=on` to measure\n code coverage from an existing corpus.\n\nThe default-build also pins `1` (`TORRENT_ABI_VERSION=1`,\nthe oldest/most inclusive ABI -- see the table in the top-level `CLAUDE.md`),\nso every fuzzer target, not just `bdecode_entry` (below), compiles and fuzzes\nthe full set of deprecated APIs alongside the current ones. `TORRENT_ABI_VERSION`\nguards in the codebase are exact-match (`== 1`) or lower-bound (`>= N`) checks,\nnever upper-bound, so ABI 1 is a strict superset of what any higher ABI value\ncompiles; this is safe project-wide.\n\n### Running a fuzzer\n\nRun a specific fuzzer against a corpus (single job):\n\n```sh\n./fuzzers/bin/.../peer_conn fuzzers/corpus/peer_conn/\n```\n\nUseful libFuzzer flags:\n\n| Flag | Purpose |\n|------|---------|\n| `-max_len=N` | Cap input size (default: 4096) |\n| `-timeout=N` | Seconds before a single run is declared a timeout (default: 1200) |\n| `-max_total_time=N` | Stop after N seconds total |\n| `-artifact_prefix=dir/` | Write crash/timeout inputs to this directory |\n\n### Running in parallel\n\nlibFuzzer supports in-process parallelism via `-jobs` and `-workers`:\n\n```sh\n./fuzzers/bin/.../peer_conn fuzzers/corpus/peer_conn/ \\\n -jobs=8 -workers=8\n```\n\n- `-jobs=N` — total number of fuzzing jobs to complete before the driver exits\n- `-workers=N` — number of concurrent fuzzer processes (defaults to\n `min(jobs, num_cpus/2)` when omitted)\n\nEach worker writes its own log to `fuzz-N.log` in the current directory.\nCrashes are written to the current directory by default; use\n`-artifact_prefix=crashes/` to redirect them.\n\nAfter parallel runs, merge all worker corpora back into the main corpus to\ndeduplicate and minimize:\n\n```sh\n./fuzzers/bin/.../peer_conn -merge=1 \\\n fuzzers/corpus/peer_conn/ fuzzers/corpus/peer_conn/\n```\n\n(Passing the same directory as both source and destination is safe; libFuzzer\nreads all inputs first, then writes the minimized set back.)\n\n### bdecode_node.cpp vs bdecode_entry.cpp\n\n`bdecode_node.cpp` fuzzes the modern `lt::bdecode(span, ...)`\nfrom `bdecode.hpp`; `bdecode_entry.cpp` fuzzes the deprecated,\n`TORRENT_ABI_VERSION == 1`-only `lt::bdecode()` overloads in\n`bencode.hpp`. They share no code path. `bdecode_entry` degrades to a\nno-op stub if built at any other ABI, rather than failing to compile.\n\n### Structure\n\nEvery fuzzer in `fuzzers/src/` implements `LLVMFuzzerTestOneInput`; the\nsession-level fuzzers additionally implement `LLVMFuzzerInitialize`:\n\n- `LLVMFuzzerTestOneInput(data, size)` — implemented by every fuzzer;\n called for every fuzz input; must be stateless with respect to prior\n inputs. Returns 0 on success, -1 to signal libFuzzer that the input was\n uninteresting (used for timeouts, not errors).\n\n- `LLVMFuzzerInitialize(argc, argv)` — implemented only by the\n session-level fuzzers (`add_torrent`, `pe_conn`, `peer_conn`,\n `piece_layers`, `rtc_peer_conn`, `ut_metadata`, `ut_pex`, `web_seed`).\n Called once at startup to set up global state (sessions, sockets,\n servers) that persists across all test inputs. The many pure-function\n fuzzers (e.g. `bdecode_node`, `parse_magnet_uri`, `gzip`,\n `torrent_info`) have no global state and omit it.\n\n### peer_conn.cpp\n\nConnects to a live libtorrent session as a BitTorrent peer and sends\nfuzz-controlled protocol messages.\n\nThe session and torrent setup is shared via `peer_session.hpp`\n(`configure_fuzz_session()`, `make_fuzz_torrent_params()`, and the\n`peer_fuzz_session` struct `g_fz`), and reused by the rtc_peer_conn,\nut_metadata, and ut_pex fuzzers.\n\n**Session setup (`LLVMFuzzerInitialize`):**\n\n- Creates a session on `127.0.0.1:0` with all outbound connections and\n ancillary services (outgoing TCP/uTP, DHT, LSD, UPnP, NAT-PMP, IP\n notifier) disabled, encryption disabled, and all peer timeouts set to\n 1 second.\n- Adds a **hybrid (v1 + v2)** torrent built from four files (a 0-byte\n file, a 100-byte file, a 50 KiB file, and a 97 MiB + 200 KiB file).\n After canonicalization (alphabetical sort + inserted pad files) this is\n exactly 100 pieces of 1 MiB. Piece hashes (SHA-1) and per-file block\n hashes (SHA-256) are filled with dummy values. `save_path` is `.` and\n no data is on disk, so the torrent stays in downloading state.\n- Waits for both the TCP `listen_succeeded_alert` (recording\n `g_fz.listen_port`) and the `torrent_resumed_alert` before returning.\n- Records `g_fz.info_hash` (an `info_hash_t` holding both the v1 SHA-1\n and v2 SHA-256 info-hashes).\n- Also adds a **second** torrent as a **magnet** (`info_hashes` only, no\n `ti`), built via the shared `add_fuzz_magnet(false, 'a')` helper (which\n wraps `build_fuzz_torrent()` from `peer_session.hpp`) from the identical\n file layout but with no v2 (SHA-256) hashes, so it's v1-only. Its\n info-hash and pre-encoded ut_metadata piece message (from\n `torrent_info::info_section()`) are stored in\n `g_v1_only_hash`/`g_v1_only_metadata_piece`. Because this metadata is\n fixed at startup rather than derived from fuzz input, delivering it via\n ut_metadata (see below) always succeeds, unlike a corpus byte sequence,\n which would have to find a SHA-1 preimage to ever make\n `torrent::set_metadata()` return true. Without this second torrent, the\n whole torrent-acquires-metadata code path (`set_metadata()`,\n `initialize_merkle_trees()`, `on_metadata_impl()`) would be unreachable:\n the first, pre-built torrent always has valid metadata, so that code\n never runs for it.\n- Also adds a **third** torrent, added as a **hybrid (v1 + v2) magnet**\n (`info_hashes` holds both the v1 and v2 hash, no `ti`), built via the\n same `add_fuzz_magnet(true, 'c')` helper from the identical file layout\n with a different dummy-hash fill byte (`'c'` instead of the default\n `'a'`) so it gets its own info-hash instead of colliding with `g_fz`'s.\n Its info-hash and pre-encoded ut_metadata piece message are stored in\n `g_v2_hybrid_hash`/`g_v2_hybrid_metadata_piece`. This is the counterpart\n to the v1-only magnet: metadata resolves to a genuine hybrid, so a peer's\n speculative v2 claim turns out honest and must survive metadata\n resolution rather than being corrected, exercising the same code as\n the v1-only magnet from the opposite direction, to guard against the\n fix over-triggering on legitimate v2 peers. More variants (a different\n file layout, etc.) can be added the same way if a future case needs\n one.\n\n**Per-input flow (`LLVMFuzzerTestOneInput`):**\n\n1. Opens a TCP connection to `127.0.0.1:g_fz.listen_port`.\n2. Sends a 68-byte BT handshake:\n - `data[0..7]` are used as the extension-flags (reserved) bytes.\n - Bits for BEP 10 (byte 5, bit 0x10) and BEP 6 FAST (byte 7, bit\n 0x04) are always forced on so those handlers are always reachable.\n - Byte 0 bit 0x01 selects the peer's protocol version, for torrents\n that have a v1 and a v2 hash to choose between (torrent selector 0\n and 2 below): clear = v2 (advertises the torrent's\n `info_hashes().get_best()`, `protocol_v2` true), set = v1\n (advertises the torrent's `info_hashes().v1`, `protocol_v2` false).\n - Byte 0 bits 0x02 and 0x04 form a 2-bit torrent selector: 0 = the\n first, pre-built torrent (metadata already valid, bit 0x01 applies\n as above); 1 = the second, v1-only magnet (always presents\n `g_v1_only_hash.v1`; bit 0x01 doesn't apply since there's no v2\n hash to choose between); 2 = the third, hybrid magnet (bit 0x01\n applies exactly as it does for selector 0); 3 = unused, falls back\n to selector 0. Every other flag bit, including byte 7 bit 0x10 (the\n v2-upgrade bit), is independent of the torrent selector and free to\n combine with any of them, e.g. a handshake claim that doesn't match\n what the connected torrent's info-hash actually supports.\n - Peer-ID is all zeros.\n3. Sends a fixed BEP 10 extended handshake (`k_extended_handshake`,\n ext_id 0) registering:\n\n | Extension | ext_id |\n |-------------|--------|\n | ut_pex | 1 |\n | ut_metadata | 2 |\n | upload_only | 3 |\n\n4. If the torrent selector picked the v1-only or hybrid magnet, sends two\n more fixed (not fuzz-derived) messages before continuing: a second\n extended handshake (BEP 10 supports re-negotiation) announcing\n `metadata_size`, which makes the ut_metadata plugin queue a request\n for piece 0, then a ut_metadata piece message carrying that magnet's\n metadata (`g_v1_only_metadata_piece` or `g_v2_hybrid_metadata_piece`),\n so `torrent::set_metadata()` succeeds deterministically on every run\n that picks either branch.\n5. Parses `data[8..]` as a sequence of messages (see **Corpus wire\n format** below) and sends each with a correct 4-byte BT length\n prefix. This is unchanged regardless of which torrent step 2 picked,\n so the same fuzzed message space (including BEP 78 hash messages)\n now also runs against a torrent whose metadata was just acquired\n rather than always pre-loaded.\n6. Closes the socket and waits for a `peer_error_alert` or\n `peer_disconnected_alert` (3-second timeout).\n\n**DEBUG_LOGGING:** defined by `fuzzers/Jamfile` via the `debug-logging`\nfeature (default `off` -> `DEBUG_LOGGING=0`). Build with\n`b2 clang debug-logging=on` to set it to 1, which adds `peer_log` alerts to\nthe alert mask and makes `wait_for_disconnect` print the last 10 interesting\nalerts (connect and routine peer-log noise excluded) before each disconnect\n-- useful for tracing which message paths were hit.\n\n### Corpus wire format (peer_conn)\n\nEach corpus file is binary-structured:\n\n```\n[8 bytes : extension flags]\n[repeated until EOF:\n 1 byte : msg_type\n 2 bytes : payload_len (big-endian, max 65535)\n N bytes : payload (N = min(payload_len, bytes_remaining))\n]\n```\n\nMinimum file size: 11 bytes (8 flag bytes + 3-byte message header).\n\nFor **msg_type 20** (BEP 10 extended protocol), the first byte of the\npayload is the extended message ID and the rest is the extension body.\nThe fuzzer splits this automatically before calling\n`send_extended_message`.\n\n**Extension flag byte layout** (8 reserved bytes, big-endian):\n\n| Byte | Bit | Protocol |\n|------|------|---------------------|\n| 0 | 0x01 | protocol version, when the selected torrent has a v1 and a v2 hash (set = v1 SHA-1, clear = v2) |\n| 0 | 0x02 + 0x04 | 2-bit torrent selector: 0 = pre-built hybrid, 1 = v1-only magnet, 2 = hybrid magnet, 3 = unused (falls back to 0) |\n| 5 | 0x10 | BEP 10 extended |\n| 7 | 0x04 | BEP 6 FAST |\n| 7 | 0x01 | BEP 5 DHT |\n| 7 | 0x10 | v2-upgrade bit (peer claims v2 support) |\n\n**libtorrent's fixed incoming ext_ids** (from `bt_peer_connection.hpp`\nenum; these are the IDs libtorrent expects in messages FROM us):\n\n| ext_id | Handler | Notes |\n|--------|------------------|--------------------------------------------|\n| 0 | extended HS | re-negotiation supported |\n| 1 | ut_pex plugin | as registered in `k_extended_handshake` |\n| 2 | ut_metadata | as registered in `k_extended_handshake` |\n| 3 | upload_only | `upload_only_msg` constant |\n| 4 | holepunch | `holepunch_msg`; requires `m_holepunch_id` |\n| 7 | dont_have | `dont_have_msg`; 4-byte piece index |\n| 8 | share_mode | `share_mode_msg` |\n| other | disconnect | `errors::invalid_message` |\n\nHolepunch (ext_id 4) silently returns if `m_holepunch_id == 0`. To\nreach the holepunch handler body, a corpus file must first send an\nadditional extended handshake (ext_id 0) that includes\n`\"ut_holepunch\": ` in its `m` dict.\n\n**BEP 78 hash messages** (21 = hash_request, 22 = hashes, 23 =\nhash_reject) require the peer's `protocol_v2` flag (set from the\nhandshake) and, separately, the connected torrent's info-hash to actually\nhave a v2 component, since a peer can set the flag on a torrent that never\ngains one (see the v1-only magnet above), so both are checked. Either\nfailing disconnects with `errors::invalid_message`.\n\n### Corpus generation tool\n\n`tools/gen_corpus.py` generates seed corpora for all session-level fuzzers.\nRun it from the repo root:\n\n```sh\npython3 tools/gen_corpus.py\n```\n\nThis writes into:\n- `fuzzers/corpus/peer_conn/`\n- `fuzzers/corpus/natpmp/`\n- `fuzzers/corpus/udp_tracker/`\n- `fuzzers/corpus/upnp/`\n- `fuzzers/corpus/pe_crypto_state/`\n- `fuzzers/corpus/pe_conn/`\n- `fuzzers/corpus/ut_metadata/`\n- `fuzzers/corpus/ut_pex/`\n- `fuzzers/corpus/web_seed/`\n- `fuzzers/corpus/utp_stream/`\n- `fuzzers/corpus/piece_layers/`\n- `fuzzers/corpus/torrent_info/`\n- `fuzzers/corpus/lsd/`\n- `fuzzers/corpus/add_torrent/`\n\n**Sync requirement:** the Python constants `EXT_UT_PEX`,\n`EXT_UT_METADATA`, `EXT_UPLOAD_ONLY`, and `FUZZER_EXT_HANDSHAKE` in\nthe tool must match `k_extended_handshake` in `peer_conn.cpp`. If the\nC++ extended handshake is changed, update both files together.\n\nThe tool also uses `NUM_PIECES = 100`, `PIECE_SIZE = 1 MiB`, and\n`BLOCK_SIZE = 16 KiB` -- these must match the torrent that\n`make_fuzz_torrent_params()` (equivalently `build_fuzz_torrent(true)`) in\n`fuzzers/src/peer_session.hpp` builds. `build_fuzz_torrent(false)` builds\nthe second, v1-only torrent, and `build_fuzz_torrent(true, 'c')` builds\nthe third, hybrid-magnet torrent, both from the identical file layout, so\nthese constants apply to them too.\n\nA handful of seeds set byte 0 bits 0x02/0x04 to connect to the second or\nthird torrent instead of the first, combined with other flag bits (e.g.\nthe v2-upgrade bit) and a follow-up message. These exist to give the\nfuzzer a foothold in those branches, not to exhaustively cover them;\ncoverage-guided mutation explores the rest.\n\n### pe_conn.cpp\n\nConnects to a live libtorrent session and drives the full PE (protocol\nencryption / MSE) handshake as the initiating (client) side. Requires\nthe session to have `in_enc_policy = pe_enabled` and\n`allowed_enc_level = pe_both`, which pe_conn's `LLVMFuzzerInitialize`\nsets.\n\nThe fuzzer performs a real DH key exchange (so the server can locate the\ntorrent), then uses fuzz-controlled bytes to drive all encrypted fields.\nEvery server-side error path is reachable by setting the appropriate bit\nin the flags byte.\n\n**Session setup (`LLVMFuzzerInitialize`):**\n\n- Same hybrid v1+v2 torrent as peer_conn (100 pieces, 1 MiB each).\n- `in_enc_policy = pe_enabled`, `allowed_enc_level = pe_both`.\n- All outbound services disabled (same as peer_conn).\n\n**Per-input flow (`LLVMFuzzerTestOneInput`):**\n\n1. Connect TCP socket to session.\n2. Generate a fresh DH key pair; send DH public key + PadA (from fuzz bytes).\n3. Read the server's DH public key (96 bytes, blocking).\n4. Compute shared secret, RC4 keys, sync hash, and obfuscated SKEY hash.\n5. Build the PE payload with fuzz-controlled fields and optional corruptions.\n6. Send the payload, close socket, wait for disconnect.\n\n**Corpus wire format:**\n\n```\nByte 0 : pad_a_size (PadA bytes before sync hash, 0-255)\nByte 1 : flags\n bit 0 : corrupt_sync_hash -> sync_hash_not_found\n bit 1 : corrupt_skey -> invalid_info_hash\n bit 2 : corrupt_vc -> invalid_encryption_constant\n bits 3-4 : crypto_provide\n 00 -> pe_both (0x03)\n 01 -> pe_plaintext (0x01)\n 10 -> pe_rc4 (0x02)\n 11 -> 0x00 (invalid) -> unsupported_encryption_mode\nByte 2 : len_pad_c * 2 (encrypted padding, 0-510 bytes)\nByte 3 : len_ia (IA size 0-255; server rejects > 68)\nBytes 4+ : first pad_a_size bytes = PadA content\n then up to len_ia bytes = IA (BT handshake portion)\n```\n\nSee `.claude/rules/protocol-encryption.md` for details of the PE protocol.\n\n\n---\npaths:\n - \"src/piece_picker.cpp\"\n - \"include/libtorrent/aux_/piece_picker.hpp\"\n - \"test/test_piece_picker.cpp\"\n---\n\n## Piece Picker\n\nThe piece picker (`src/piece_picker.cpp`, `include/libtorrent/aux_/piece_picker.hpp`) decides\nwhich blocks to request from which peers. It tracks availability (how many peers have each piece),\nper-piece priority, and the state of every block in every partially-downloaded piece.\n\n### Core data structures\n\n**`piece_pos`** — 8 bytes per piece, tightly packed:\n- `peer_count` (26 bits) — number of peers that have this piece\n- `download_state` (3 bits) — one of the `download_queue_t` constants below\n- `piece_priority` (3 bits) — 0 = filtered (dont-download), 1–7 user priority (4 = default)\n- `index` (`prio_index_t`) — position in `m_pieces`, or `we_have_index` (-1) if we have it\n\n**`m_piece_map`** — `aux::vector`, one entry per piece.\n\n**`m_pieces`** — flat `aux::vector` of all pickable (not filtered,\nnot have) pieces, sorted by effective priority. Within each priority band pieces are in random\norder (rarest-first is approximated by the priority formula, not by sorting on availability alone).\n\n**`m_priority_boundaries`** — cumulative end-indices into `m_pieces`. Priority band `p` spans\n`[m_priority_boundaries[p-1], m_priority_boundaries[p])`. Priority 0 starts at index 0.\n\n**`m_downloads`** — `aux::array` of 4 sorted `vector` (one per download\ncategory), indexed by `download_queue_t`:\n- `piece_downloading` (0) — some blocks still open (not yet requested)\n- `piece_full` (1) — all blocks requested, at least one not yet writing/finished\n- `piece_finished` (2) — all blocks writing or finished\n- `piece_zero_prio` (3) — downloading but priority = 0\n\n`piece_downloading_reverse` and `piece_full_reverse` are pseudo-states stored in `download_state`\nto de-prioritize pieces only requested from reverse peers; they map to the same download vectors\nas their non-reverse counterparts.\n\n**`downloading_piece`** — one per in-flight piece:\n- `info_idx` (`uint16_t`) — index into `m_block_info`; the slice `[info_idx * blocks_per_piece, (info_idx+1) * blocks_per_piece)` holds the block states for this piece\n- `finished`, `writing`, `requested` (15 bits each) — block state counters\n- `passed_hash_check` (1 bit) — hash job returned OK (piece may not yet be on disk)\n- `locked` (1 bit) — blocks cannot be picked; set during error recovery, cleared by `restore_piece()`\n- `hashing` (1 bit) — outstanding hash request in flight\n\n**`m_block_info`** — flat `aux::vector`. Each downloading piece owns\n`blocks_per_piece()` consecutive entries identified by `info_idx`. Free ranges are recycled\nvia `m_free_block_infos` (a free-list of `uint16_t` indices).\n\n**`block_info`** — per block:\n- `peer` — last peer to request/write this block\n- `num_peers` (14 bits) — peers with this block outstanding\n- `state` (2 bits) — `state_none → state_requested → state_writing → state_finished`\n\n### Priority formula\n\n`piece_pos::priority(picker)` returns -1 (not pickable) if: filtered, have, no availability\n(`peer_count + m_seeds == 0`), `piece_full`, or `piece_finished`.\n\nOtherwise:\n```\npriority = availability * (priority_levels - piece_priority) * prio_factor + adjustment\n```\nwhere `priority_levels = 8`, `prio_factor = 3`, and:\n- `adjustment = -3` for actively downloading pieces (non-reverse)\n- `adjustment = -2` for open pieces\n- `adjustment = -1` for reverse-downloading pieces\n\nLower value = picked first. The `prio_factor = 3` creates three sub-levels per availability\ntier, letting downloading pieces beat open ones at the same availability.\n\n### Seeds optimization\n\nPeers with all pieces are tracked in `m_seeds` rather than incrementing every `peer_count`.\nTrue availability of a piece is `peer_count + m_seeds`. When a seed sends DONT_HAVE,\n`break_one_seed()` decrements `m_seeds` and increments every `piece_pos::peer_count`.\n\n### Lazy rebuild (`m_dirty`)\n\n`m_pieces` and `m_priority_boundaries` are rebuilt lazily. When `m_dirty` is true, they are\nstale; `update_pieces()` (called automatically before any pick) rebuilds them:\n1. Walk `m_piece_map`, count pieces per priority band → `m_priority_boundaries` holds deltas\n2. Make boundaries cumulative, resize `m_pieces`\n3. Fill `m_pieces` using per-piece relative offsets stored in `piece_pos::index`\n4. Shuffle each priority band randomly\n5. Fix up `piece_pos::index` to point back into the final positions in `m_pieces`\n\nBulk operations (e.g. `inc_refcount(bitfield)` touching many pieces) set `m_dirty = true`\nrather than updating incrementally. For small bitfield changes (< 50 pieces), the picker\nupdates incrementally via `update()` / `add()` / `remove()` without dirtying.\n\n### Sequential download cursors\n\n`m_cursor` — lowest piece index not yet had.\n`m_reverse_cursor` — one past the highest piece index not yet had (all pieces from here to end\nare had). `set_sequential_range(first, last)` constrains the cursor window.\n\n### Piece lifecycle\n\n```\nopen → downloading → full → finished → (removed, piece_pos::have set)\n```\n\n- `mark_as_downloading(block, peer)` — block: none→requested; piece may transition open→downloading\n- `mark_as_writing(block, peer)` — block: requested→writing\n- `mark_as_finished(block, peer)` — block: writing→finished; calls `update_piece_state()` which\n may advance the piece from downloading→full→finished\n- `piece_passed(index)` — hash check passed; sets `passed_hash_check`, calls `account_have()`,\n calls `piece_flushed()` if all blocks are also finished\n- `piece_flushed(index)` — piece is on disk; removes from `m_downloads`, removes from `m_pieces`\n via `remove()`, sets `piece_pos::index = we_have_index`\n- `restore_piece(index, blocks)` — after hash failure or write error; clears `locked`, optionally\n resets specific blocks back to `state_none`, re-opens the piece for downloading\n\n### `pick_pieces` options\n\n`picker_options_t` flags passed to `pick_pieces()`:\n- `rarest_first` — pick lowest-availability pieces (the default; driven by the priority formula)\n- `reverse` — pick most-common first, or last pieces if sequential\n- `sequential` — pick within the `[m_cursor, m_reverse_cursor)` window in order\n- `on_parole` — only pick pieces exclusively requested by this peer (after suspected bad data)\n- `prioritize_partials` — prefer pieces already in `m_downloads` over fresh ones\n- `piece_extent_affinity` — create affinity for 4 MiB extents of adjacent pieces\n- `align_expanded_pieces` — align contiguous-block expansion to natural boundaries\n\nAuto partial prioritization: if `num_partials > num_peers * 3 / 2` or\n`num_partials * blocks_per_piece > 2048`, `prioritize_partials` is forced on and\n`prefer_contiguous_blocks` is set to 0 to avoid unbounded partial piece growth.\n\n`prefer_contiguous_blocks` (int, not a flag) — request this many consecutive blocks from the\nsame piece; used by web peers to pipeline larger requests.\n\n\n# Protocol Encryption (MSE/PE) Handshake\n\nlibtorrent implements Message Stream Encryption (MSE), also called Protocol\nEncryption (PE). The implementation lives in `src/bt_peer_connection.cpp` and\n`src/pe_crypto.cpp` (crypto primitives in `include/libtorrent/aux_/pe_crypto.hpp`).\n\n## Overview\n\nMSE provides obfuscation (not strong security) against deep-packet inspection.\nIt uses a Diffie-Hellman key exchange to derive RC4 stream keys and wraps the\nBitTorrent handshake. The torrent info hash is obfuscated in the handshake so\nthe connection cannot be trivially identified.\n\nAfter the handshake, the connection body is RC4-encrypted only if\n`crypto_select` ended on `pe_rc4`. If `pe_plaintext` was selected, only the\nhandshake is obfuscated and post-handshake traffic is sent in the clear. This\nis tracked by the `m_rc4_encrypted` flag on `bt_peer_connection` and gates the\nencrypt/decrypt path in every send and receive routine.\n\n## Settings\n\n| Setting | Meaning |\n|---------|---------|\n| `pe_forced` | Always use PE (outgoing) / require PE (incoming) |\n| `pe_enabled` | Use PE if peer supports it, fall back to plaintext |\n| `pe_disabled` | Never use PE |\n\n`settings_pack::out_enc_policy` controls outgoing connections.\n`settings_pack::in_enc_policy` controls acceptance of incoming connections.\n`settings_pack::allowed_enc_level` is a bitmask (`pe_plaintext=1`, `pe_rc4=2`, `pe_both=3`).\n`settings_pack::prefer_rc4` selects RC4 over plaintext when both are offered.\n\n## Handshake States\n\nThe state machine lives in `bt_peer_connection::on_receive_impl()`. The\nhandshake differs between the initiating side (outgoing) and the accepting\nside (incoming).\n\n### Incoming (server) path\n\n1. **`read_pe_dhkey`** (96 bytes) -- read client's DH public key, respond\n with our DH public key + random PadB, transition to `read_pe_synchash`.\n2. **`read_pe_synchash`** (scans up to 512 bytes of PadA + the 20-byte hash) --\n scan received bytes for `hash('req1', S)` where S is the shared secret. The\n client may insert up to 512 bytes of PadA before this hash. Failure:\n `sync_hash_not_found`.\n3. **`read_pe_skey_vc`** (28 bytes) -- read the obfuscated SKEY hash (20\n bytes) and the encrypted VC (8 bytes). The SKEY hash is\n `hash('req2', info_hash) XOR hash('req3', S)` which reveals which torrent\n the client wants. Failure if torrent not found: `invalid_info_hash`.\n Failure if VC decrypts to non-zero: `invalid_encryption_constant`.\n4. **`read_pe_cryptofield`** (6 bytes) -- read encrypted\n `crypto_provide(4) + len_pad_c(2)`. Server selects a crypto mode and\n sends back its own VC + `crypto_select + pad`. Failure if no mode overlaps:\n `unsupported_encryption_mode`. Failure if `len_pad_c > 512`:\n `invalid_pad_size`.\n5. **`read_pe_pad`** (`len_pad_c + 2` bytes) -- skip encrypted padding, read\n `len_ia` (2 bytes).\n6. **`read_pe_ia`** (`len_ia` bytes) -- read Initial Application data (IA),\n the encrypted start of the BitTorrent handshake. Failure if `len_ia > 68`:\n `invalid_encrypt_handshake`.\n7. Falls through to **`read_protocol_identifier`** (regular BT handshake).\n\n### Outgoing (client) path\n\n1. **`on_connected`** -- send DH public key + random PadA,\n transition to `read_pe_dhkey`.\n2. **`read_pe_dhkey`** (96 bytes) -- read server's DH public key, then send\n sync hash + SKEY hash + encrypted(VC + crypto_provide + len_pad + PadC +\n len_IA) + encrypted(IA = BT handshake). Also send the BT handshake\n encrypted as IA. Transition to `read_pe_syncvc`.\n3. **`read_pe_syncvc`** (scans up to 512 bytes of PadB + the 8-byte VC) --\n scan for the encrypted 8-byte verification constant (all zeros encrypted\n with RC4). Failure if not found within the 512-byte pad window:\n `invalid_encryption_constant`.\n4. **`read_pe_cryptofield`** (6 bytes) -- read server's `crypto_select(4) +\n len_pad(2)`. Failure if selected mode not in our `allowed_enc_level`:\n `unsupported_encryption_mode_selected`.\n5. **`read_pe_pad`** (`len_pad` bytes) -- skip server's padding.\n6. Falls through to **`read_protocol_identifier`** (regular BT handshake).\n\n## Crypto Primitives\n\n### DH Key Exchange (`dh_key_exchange`)\n\n768-bit Diffie-Hellman with a fixed prime (MSE standard prime from BEP 10/MSE\nspec). `key = 2^random % prime`.\n\n- `dh_key_exchange()` -- generate local key pair\n- `compute_secret(remote_pubkey)` -- compute shared secret S; also computes\n `m_xor_mask = hash('req3', S)` used for SKEY obfuscation\n- `get_local_key()` -- 96-byte DH public key to send\n- `get_secret()` -- shared secret S\n- `get_hash_xor_mask()` -- `hash('req3', S)` for SKEY obfuscation\n\n### RC4 Key Derivation\n\nFor outgoing connections (client):\n```\nencrypt_key = hash(\"keyA\", S, SKEY)\ndecrypt_key = hash(\"keyB\", S, SKEY)\n```\nFor incoming connections (server):\n```\nencrypt_key = hash(\"keyB\", S, SKEY)\ndecrypt_key = hash(\"keyA\", S, SKEY)\n```\nwhere SKEY is the SHA-1 info hash of the torrent (or first 20 bytes of SHA-256\nfor v2 torrents). The RC4 stream is initialized by discarding the first 1024\nbytes (standard MSE requirement).\n\n### Message Layout (client -> server)\n\n```\n[96 bytes] client DH public key\n[0-512 bytes] PadA (random, encrypted sync hash follows)\n[20 bytes] sync_hash = hash('req1', S)\n[20 bytes] skey_obfusc = hash('req2', SKEY) XOR hash('req3', S)\nencrypt([8 bytes] VC = 0x00 * 8\n [4 bytes] crypto_provide (bit flags: 1=plaintext, 2=RC4)\n [2 bytes] len_PadC\n [len_PadC] PadC (random)\n [2 bytes] len_IA)\nencrypt([len_IA bytes] IA = start of BT handshake, always RC4 encrypted)\n```\n\n### Message Layout (server -> client)\n\n```\n[96 bytes] server DH public key\n[0-512 bytes] PadB (random)\nencrypt([8 bytes] VC = 0x00 * 8\n [4 bytes] crypto_select (chosen mode: 1 or 2)\n [2 bytes] len_PadD\n [len_PadD] PadD (random))\n[regular BT handshake, encrypted or not depending on crypto_select]\n```\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `src/pe_crypto.cpp` + `include/libtorrent/aux_/pe_crypto.hpp` | DH exchange, RC4 handler, encryption_handler |\n| `src/bt_peer_connection.cpp` | PE state machine and write helpers |\n| `fuzzers/src/pe_conn.cpp` | Session-level fuzzer for the PE handshake |\n| `fuzzers/src/pe_crypto_state.cpp` | Low-level fuzzer for DH + RC4 primitives |\n\n## Error Codes\n\n| Error | Trigger |\n|-------|---------|\n| `sync_hash_not_found` | sync hash not found within 512-byte window |\n| `invalid_info_hash` | SKEY hash doesn't match any known torrent |\n| `invalid_encryption_constant` | VC decrypts to non-zero bytes |\n| `unsupported_encryption_mode` | No overlap in crypto_provide vs allowed_enc_level |\n| `unsupported_encryption_mode_selected` | Server selected a mode not in our allowed list |\n| `invalid_pad_size` | len_pad_c > 512 |\n| `invalid_encrypt_handshake` | len_ia > 68 |\n\n\n---\npaths:\n - \"bindings/python/**\"\n---\n\n## Python Bindings\n\nSource in `bindings/python/src/*.cpp`; one `.cpp` file per exposed subsystem (e.g. `session.cpp`, `torrent_handle.cpp`, `alert.cpp`). Built as a Python extension module (`libtorrent.so`) using boost.python.\n\n**Building:**\n```sh\ncd bindings/python && b2\n```\nCMake also builds them when `-Dpython-bindings=ON` is passed.\n\n**Testing:**\n\nTests live in `bindings/python/tests/*_test.py` and are run with pytest:\n```sh\ncd bindings/python && python -m pytest tests/\n```\nThere is a roughly one-to-one correspondence between `src/foo.cpp` and `tests/foo_test.py`. Tests must be isolated — no real network access, no persistent filesystem side-effects. Use `get_isolated_settings()` from `tests/lib.py` when constructing a session, and `tempfile.TemporaryDirectory()` for any disk I/O. See `bindings/python/tests/guidelines.txt` for the full testing conventions.\n\n**Type stubs:**\n\nThe type stub file is `bindings/python/libtorrent/__init__.pyi`. It was originally bootstrapped with `stubgen` but is now **maintained entirely by hand** — do not regenerate it with stubgen, as that would lose the carefully written annotations. When adding a new binding, add the corresponding type annotation to this file manually. The header comment in the file documents the conventions used (e.g. how boost.python enums, positional-only args, and the `_BoostBaseClass` metaclass are represented).\n\n**Design philosophy — classes over dicts:**\n\nPrefer exposing real C++ classes via boost.python rather than converting them to plain Python dicts. Earlier versions of the bindings returned plain `dict` objects for things like peer info and torrent status; these have been replaced with proper bound classes (`peer_info`, `torrent_status`, etc.) that mirror the C++ types. When adding new bindings, follow this pattern: expose the C++ struct/class directly rather than unpacking it into a dict.\n\nNote: boost.python strictly requires `dict` and `list` (not duck-typed `Mapping`/`Sequence` abstractions) for function arguments that cross the C++ boundary. This is a boost.python limitation and not a design choice; the type stubs reflect this with `dict` and `list` rather than `Mapping` or `Sequence`.\n\n\n---\npaths:\n - \"include/libtorrent/units.hpp\"\n - \"include/libtorrent/flags.hpp\"\n - \"include/libtorrent/index_range.hpp\"\n - \"include/libtorrent/**\"\n - \"src/**\"\n---\n\n## Strong Types\n\nThe codebase avoids using raw `int` or bare integers for indices and flags. Instead, use the strong type and flag facilities.\n\n**Index types** (`include/libtorrent/units.hpp`):\n\n`aux::strong_typedef` wraps an integer so it is incompatible with other integers and other strong types. Arithmetic with raw integers is not allowed; only arithmetic with the same type or its `diff_type` is permitted.\n\nTo define a new index type:\n```cpp\nusing my_index_t = aux::strong_typedef;\n```\n\nTo convert to/from the underlying integer, use an explicit cast:\n```cpp\npiece_index_t p{42};\nint raw = static_cast(p);\n```\n\nTo iterate over a range of indices, use `index_range` (`include/libtorrent/index_range.hpp`):\n```cpp\nfor (piece_index_t i : index_range{begin, end}) { ... }\n```\n\n**Flag types** (`include/libtorrent/flags.hpp`):\n\n`flags::bitfield_flag` wraps an unsigned integer as a type-safe bitfield. Flags of different types cannot be combined. Individual bit constants are defined using the `_bit` user-defined literal:\n\n```cpp\nusing torrent_flags_t = flags::bitfield_flag;\nstatic constexpr torrent_flags_t seed_mode = 0_bit;\nstatic constexpr torrent_flags_t upload_mode = 1_bit;\n```\n\nTo define a new flag type:\n```cpp\nusing my_flags_t = flags::bitfield_flag;\n```\n\nFlag values support `|`, `&`, `^`, `~`, `|=`, `&=`, `^=`, and `operator bool()` (true if any bit is set). To test a flag:\n```cpp\nif (flags & torrent_flags::seed_mode) { ... }\n```\n\n\n# BitTorrent v2 Torrents (BEP 52)\n\n## Info Dict Structure\n\nV2 uses a nested **`file tree`** dict instead of a flat `files` list:\n- Keys are filename path components; non-leaf = directory (nested dict)\n- Leaf nodes have empty-string key `\"\"` → file metadata dict with `length`, `pieces root`, etc.\n- **`pieces root`** (32 bytes): root of the per-file merkle tree (SHA-256)\n- **`meta version`**: integer `2` identifies v2 metadata\n- **`piece layers`**: (outside info dict) maps `pieces root` → concatenated piece-layer hashes\n\nV2 torrents have a **SHA-256 info hash** (vs v1's SHA-1). `torrent_info::m_info_hash` holds both.\n\n## Per-File Merkle Tree\n\nEach file has an independent merkle tree (SHA-256, not SHA-1):\n\n```\nroot (pieces root)\n └── interior nodes\n └── piece layer ← hash of one piece's block subtree\n └── block layer (leaf) ← hash of 16 KiB block\n```\n\n- **Block** = 16 KiB (fixed), the unit of transfer\n- **Piece** = `piece_length` bytes; must be a power-of-2 multiple of 16 KiB\n- Tree is **padded to a power of 2** in leaf count using `merkle_pad()` (zeros hashed up)\n- Interior nodes: `parent = SHA-256(left_child || right_child)`\n- Flat array indexing: node at layer `l`, offset `o` → index `(1 << l) - 1 + o`; parent = `(i-1)/2`\n\nKey helpers in `src/merkle.cpp`:\n- `merkle_num_leafs()`, `merkle_num_nodes()`, `merkle_num_layers()`\n- `merkle_fill_tree()` — compute interior nodes from leaves\n- `merkle_validate_and_insert_proofs()` — validate a proof chain against a known root\n- `merkle_pad()` — padding hash for odd tree levels\n\n## merkle_tree Class (`include/libtorrent/aux_/merkle_tree.hpp`)\n\nOne instance per file, with four storage modes (optimization):\n\n| Mode | Stored nodes | When used |\n|------|-------------|-----------|\n| `empty_tree` | root only | single-block file, or tree completely unknown |\n| `piece_layer` | piece-layer hashes | piece hashes received, blocks not yet |\n| `block_layer` | block hashes | download complete, all blocks verified |\n| `full_tree` | all nodes | during partial verification |\n\n`m_block_verified` bitfield tracks which block hashes have been validated against their parent chain up to the known root.\n\nKey methods: `load_tree()`, `load_piece_layer()`, `add_hashes()`, `set_block()`\n\n## Hybrid Torrents (v1 + v2)\n\nA single `.torrent` file can contain both:\n- v1 info dict: `files` list + `pieces` SHA-1 concatenation\n- v2 `file tree` + `piece layers`\n\nConstraints: identical file layout, same `piece_length`. Results in two info hashes.\n`torrent_info` parsing uses `extract_files()` for v1 and `extract_files2()` for v2.\n\n## Hash Picker (`src/hash_picker.cpp`)\n\nDecides which merkle hashes to request from peers (BitTorrent v2 hash exchange protocol).\n\n**`hash_request` struct:**\n```cpp\nstruct hash_request {\n file_index_t file;\n int base; // tree layer (0 = block layer)\n int index; // starting offset in layer\n int count; // number of hashes (≤ 8192)\n int proof_layers; // uncle hashes needed to prove against known root\n};\n```\n\n**Strategy:**\n1. Request piece-layer hashes in 512-piece chunks until all known\n2. On piece failure, request all block hashes for that piece (3s minimum interval)\n\n**Validation flow:**\n- `add_hashes()`: receives hashes + uncle proof chain, calls `merkle_validate_and_insert_proofs()`, inserts valid nodes, reports `pass`/`fail` per piece\n- `set_block()`: inserts a single block hash, finds largest verifiable subtree, returns pass/fail\n\n## Resume Data\n\nMerkle trees are persisted across sessions in resume data (`src/write_resume_data.cpp`, `src/read_resume_data.cpp`):\n\n```\n[\"merkle trees\"][file_index] → sparse tree nodes (non-zero nodes only)\n[\"merkle tree mask\"][file_index] → bitfield of which nodes are stored\n[\"verified bits\"][file_index] → m_block_verified bitfield\n[\"piece layers\"][pieces_root] → concatenated piece-layer hashes\n```\n\n- `write_resume_data()` uses `build_sparse_vector()` to encode only known nodes\n- `read_resume_data()` calls `load_sparse_tree()` to restore the partial tree\n- Piece layers are re-derived from the stored tree during write\n\n## Verification Flow\n\n1. File added → `merkle_tree` created with root hash from `pieces root`\n2. Block arrives → `set_block()` inserts hash, attempts validation up to root\n3. Piece complete → all block hashes in piece set; piece passes if tree validates\n4. Piece fails → clear invalid nodes; hash picker schedules block-hash requests\n5. Hashes received → `add_hashes()` validates proof chain, inserts valid nodes, marks failures\n6. Download complete → all blocks verified; block layer mode; tree fully populated\n\n## Key Files\n\n| File | Purpose |\n|------|---------|\n| `src/merkle.cpp` | Tree math, proof validation, padding |\n| `src/merkle_tree.cpp` + `include/libtorrent/aux_/merkle_tree.hpp` | Per-file tree storage |\n| `src/hash_picker.cpp` + `include/libtorrent/aux_/hash_picker.hpp` | Hash request scheduling |\n| `src/torrent_info.cpp` | Parsing `file tree`, `piece layers`, hybrid logic |\n| `src/create_torrent.cpp` | Building v2/hybrid `.torrent` files |\n| `src/write_resume_data.cpp` + `src/read_resume_data.cpp` | Persisting trees |\n\n\n{\n \"$schema\": \"https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json\",\n \"builtin-baseline\": \"d90a9b159c08169f39adcd1b0f1ac0ca12c4b96c\",\n \"dependencies\": [\"openssl\"],\n \"overrides\": [\n {\n \"name\": \"openssl\",\n \"version-string\": \"3.6.1\"\n }\n ]\n}\n\n\n# CLAUDE.md\n\nThis file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.\n\n## Overview\n\nlibtorrent-rasterbar is a C++17 BitTorrent library (version 2.1.0). It supports v1 and v2 torrents, DHT, WebTorrent (WebRTC), and optional I2P/SSL.\n\n## Build Systems\n\nThe project has **two build systems**: boost-build (b2/bjam) and CMake. **Boost-build is the preferred build system** per the contributing guidelines.\n\n### Boost-build (b2) - preferred\n\n**Never add `-j` (parallel jobs) flags to b2 commands.** b2 manages parallelism on its own.\n\nBuild the library (from repo root):\n```sh\nb2\n```\n\nRun unit tests:\n```sh\ncd test && b2\n```\n\nRun a single test:\n```sh\ncd test && b2 test_session\n```\n\nRun deterministic tests (no flaky networking):\n```sh\ncd test && b2 deterministic-tests\n```\n\nRun simulations:\n```sh\ncd simulation && b2\n```\n\nBuild with developer options (typical for development):\n```sh\ncd test && b2 asserts=on invariant-checks=full debug-iterators=on picker-debugging=on\n```\n\nBuild with sanitizers:\n```sh\ncd test && b2 address-sanitizer=norecover undefined-sanitizer=norecover asserts=on\n```\n\nBuild examples, tools, fuzzers, python bindings:\n```sh\ncd examples && b2\ncd tools && b2\ncd fuzzers && b2 clang\ncd bindings/python && b2\n```\n\n### CMake — alternative\n\n```sh\ncmake -B build -DCMAKE_BUILD_TYPE=Debug -Dbuild_tests=ON\ncmake --build build\ncd build && ctest\n```\n\nRun a single test via CMake:\n```sh\ncd build && ./test/test_session\n```\n\n## Pre-commit\n\nThe repo uses [pre-commit](https://pre-commit.com). Before pushing:\n```sh\npip install pre-commit\npre-commit install\npre-commit run --all-files\n```\n\nPre-commit hooks include trailing whitespace, YAML/TOML/XML checks, RST formatting, Python formatting (black, isort, flake8, mypy, autoflake), and auto-generation of:\n- `include/libtorrent/fwd.hpp` and `include/libtorrent/libtorrent.hpp` via `tools/gen_fwd.py` and `tools/gen_convenience_header.py`\n- C binding headers (`bindings/c/include/libtorrent_settings.h`, `libtorrent_alerts.h`) via `bindings/c/tools/gen_header.py` and `bindings/c/tools/gen_alert_header.py`\n\nThe C++ format check (`git-clang-format-18`) is the only hook that requires a\nsystem install -- `apt install clang-format-18` (or platform equivalent). It\nformats only the lines that differ from HEAD, not whole files; the codebase is\nbeing migrated to the `.clang-format` style incrementally rather than via a\nflag-day reformat.\n\n## Key Architecture\n\n### Threading Model\n\nThree thread categories:\n1. **Main network thread** — all sockets, session/torrent/peer state, boost.asio event loop\n2. **Disk I/O thread(s)** — reads, writes, and SHA-1/SHA-256 piece verification (count controlled by `settings_pack::aio_threads`)\n3. **Resolver thread** — spawned by boost.asio for async DNS on platforms without native async getaddrinfo\n\nAll interaction with session/torrent state from outside must go through the network thread (via `session_handle` and `torrent_handle`).\n\n### Core Classes\n\n- **`session`** (`include/libtorrent/session.hpp`) — public API, pimpl over `session_impl`\n- **`session_impl`** (`include/libtorrent/aux_/session_impl.hpp`) — all session state: torrent list, connection list, global rate limits, DHT state, port mapping\n- **`torrent`** (`include/libtorrent/aux_/torrent.hpp`, `src/torrent.cpp`) — all state for a single swarm: piece picker, peer connections, file storage\n- **`torrent_handle`** (`include/libtorrent/torrent_handle.hpp`) — public pimpl handle, weak reference to `torrent`; sends messages to network thread\n- **`peer_connection`** / **`bt_peer_connection`** — BitTorrent protocol implementation\n- **`piece_picker`** — download strategy (which blocks to request from which peers). See `.claude/rules/piece-picker.md` for a detailed description.\n- **`peer_list`** — list of known (not necessarily connected) peers for a swarm\n\n### Disk I/O\n\nThree disk backends:\n- `mmap_disk_io` — mmap-based (default on 64-bit when mmap is available)\n- `posix_disk_io` — fallback single-threaded POSIX I/O (used on 32-bit or without mmap)\n- `pread_disk_io` — multi-threaded backend using `pread()`/`pwrite()`. See `.claude/rules/disk-cache.md` for a detailed description.\n\n### DHT (Kademlia)\n\nSource in `src/kademlia/`, headers in `include/libtorrent/kademlia/`. Includes ed25519 signing (`src/ed25519/`).\n\n### Extensions\n\nProtocol extensions live in `src/` and `include/libtorrent/extensions/`:\n- `ut_metadata` — magnet link metadata exchange\n- `ut_pex` — peer exchange\n- `smart_ban` — ban peers that send corrupt data\n- `i2p_pex` — peer exchange for I2P peers\n\n### WebTorrent (WebRTC)\n\nOptional feature controlled by `webtorrent=on` / `-Dwebtorrent=ON`. Uses `deps/libdatachannel` and the `rtc_stream`/`rtc_signaling` subsystem.\n\n### Python Bindings\n\nSource in `bindings/python/src/*.cpp`, built as `libtorrent.so` using boost.python. See `.claude/rules/python-bindings.md` for detailed conventions.\n\n### Simulations\n\n`simulation/` contains network simulation tests that use `libsimulator` (a deterministic virtual network). These test high-level behaviors (swarms, DHT, session management) without real network access. All simulation tests run with `invariant-checks=full`, `asserts=on`, `debug-iterators=on`.\n\n### Terminology\n\n- **piece** — SHA-1/SHA-256 verified chunk of torrent data (typically power-of-two size)\n- **block** — 16 KiB sub-unit of a piece (the transfer unit in the protocol)\n- **torrent_peer** — a known-but-not-connected peer entry\n- **peer_connection** — an active connection to a peer\n\n## Key Build Options (b2 features)\n\n| Feature | Values |\n|---------|--------|\n| `crypto` | `built-in`, `openssl`, `openssl-shared`, `wolfssl`, `gcrypt`, `gnutls`, `libcrypto` |\n| `asserts` | `on`, `off`, `production`, `system` |\n| `invariant-checks` | `off`, `on`, `full` |\n| `logging` | `on`, `off` |\n| `deprecated-functions` | `on`, `off`, `1`, `2`, `3`, `4` |\n| `webtorrent` | `on`, `off` |\n| `address-sanitizer` | `norecover`, `recover`, `off` |\n| `undefined-sanitizer` | `norecover`, `recover`, `off` |\n| `thread-sanitizer` | `norecover`, `recover`, `off` |\n| `picker-debugging` | `on`, `off` |\n| `debug-iterators` | `default`, `off`, `on`, `harden` |\n\n`deprecated-functions` selects the ABI/API version via `TORRENT_ABI_VERSION`. Different values are **link-incompatible**. Higher numbers expose a more modern interface and remove older deprecated APIs:\n\n| Value | `TORRENT_ABI_VERSION` | Corresponds to |\n|-------|-----------------------|----------------|\n| `on` | (oldest supported) | libtorrent 1.x |\n| `1` | 1 | libtorrent 1.x |\n| `2` | 2 | libtorrent 2.0 |\n| `3` | 3 | libtorrent 2.x |\n| `4` | 4 | libtorrent 2.1 |\n| `off` | 100 (newest) | latest API |\n\n## Directory Layout\n\n```\nsrc/ main library source\nsrc/kademlia/ DHT implementation\nsrc/ed25519/ ed25519 signing (vendored)\ninclude/libtorrent/ public API headers\ninclude/libtorrent/aux_/ internal headers (not public API)\ninclude/libtorrent/kademlia/ DHT headers\ninclude/libtorrent/extensions/ extension headers\ntest/ unit tests (test_*.cpp)\nsimulation/ network simulation tests (test_*.cpp)\nfuzzers/src/ fuzz targets\nexamples/ example programs\ntools/ utility scripts and programs\nbindings/python/ Python bindings (boost.python)\nbindings/c/ C API bindings\ndeps/ vendored dependencies\n deps/try_signal/ signal handling\n deps/asio-gnutls/ GnuTLS adapter for asio\n deps/json/ nlohmann/json (for WebTorrent)\n deps/libdatachannel/ WebRTC (for WebTorrent)\n```\n\n## Adding New Source Files\n\nWhen adding a new `.cpp` or `.hpp` file, it must be added to **all three** build systems:\n1. `Jamfile` (boost-build)\n2. `CMakeLists.txt`\n3. `Makefile` (if applicable)\n\n## Code-Generation Tools\n\nAfter modifying public API headers, regenerate the derived headers by running these scripts from the repo root:\n\n- `tools/gen_fwd.py` — regenerates `include/libtorrent/fwd.hpp` (forward declarations for all public types)\n- `tools/gen_convenience_header.py` — regenerates `include/libtorrent/libtorrent.hpp` (the convenience header that includes all public headers)\n\nThese are also run automatically by the pre-commit hooks.\n\n## Coding Conventions\n\n- Comments must use ASCII characters only (no Unicode, smart quotes, em-dashes, etc.)\n- Use a single space after a period in comments (not two spaces)\n- Declare variables `const` whenever they are not reassigned after initialization\n- C++17 throughout; no C++20 features yet\n- The code must build with standard C++ and must not require compiler language extensions; optional extensions (e.g. `__builtin_*`, platform-specific pragmas) may be used when guarded by feature detection\n- `namespace lt = libtorrent` alias is always available\n- Asserts: use `TORRENT_ASSERT(cond)` (active when `TORRENT_USE_ASSERTS` is defined, i.e. debug builds)\n- Invariant checks: expensive checks inside `#if TORRENT_USE_INVARIANT_CHECKS`\n- Public API headers use `TORRENT_EXPORT` macro; internal symbols use hidden visibility\n- Internal functions and classes use `TORRENT_EXPORT_EXTRA` macro; to grant access to tests\n- ABI versioning via `TORRENT_VERSION_NAMESPACE_2/3/4` inline namespace macros (defined in `include/libtorrent/aux_/export.hpp`); `_2` = v1.2, `_3` = v2, `_4` = v2.1\n- Warnings are treated as errors in CI (both gcc and clang)\n- Changes to ABI (fields/ordering of public classes) must target `master`, not `RC_*` stable branches\n- `TORRENT_EXPORT_EXTRA` marks a symbol exported only for test access; it is private/internal, not public API, and is exempt from the ABI-stability rules above (safe to change on `RC_*` branches). Do not treat `TORRENT_EXPORT_EXTRA` as indicating public or ABI-sensitive status; `TORRENT_EXPORT` is the public one\n- `settings_pack` enum values must be appended at the end of each enum group (int, bool, string) — never inserted in the middle — to avoid changing the numeric values of existing settings and breaking ABI\n- prefer the C++ counterparts to C headers\n- do not use using-statements in header files\n- prefer to fully qualify standard types and functions\n- prefer the short lt namespace alias when fully qualifying libtorrent types\n- prefer using default member initializers over initializer list\n\n### Strong Types\n\nAvoid raw `int` for indices and flags; use `aux::strong_typedef` (`include/libtorrent/units.hpp`) and `flags::bitfield_flag` (`include/libtorrent/flags.hpp`). See `.claude/rules/strong-types.md` for details.\n"}