{"owner":"QuipNetwork","repo":"quip-protocol","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md"],"skills":{"AGENTS.md":"# AGENTS.md — QuIP Protocol Project Instructions\n\nCross-tool instructions for AI coding assistants (Claude Code, Codex, Cursor, Gemini CLI).\n\nFor the **runtime architecture**, read [`ARCHITECTURE.md`](./ARCHITECTURE.md).\nThis file is the developer-facing how-to: commands, dependencies, code\nstyle. Anything about how the system *runs* belongs in\n`ARCHITECTURE.md`.\n\n## Environment\n\nThe `.quip` virtualenv is already active in development shells —\ndon't prefix shell commands with `source .quip/bin/activate`.\n\n```bash\n# Fresh install (only if .quip doesn't exist yet)\npython3 -m venv .quip\nsource .quip/bin/activate\npip install -U pip setuptools wheel\npip install -e .            # core + CPU\npip install -e .[cuda]      # CUDA backend\npip install -e .[metal]     # Apple Silicon backend\npip install -e .[dev]       # pytest + pytest-asyncio\n```\n\n`.env` holds `DWAVE_API_KEY` and other credentials. **Never read or\ndisplay its contents.**\n\n## Running the miner\n\nThe CLI is `quip-miner` (defined in `quip_cli.py`, entry point in\n`pyproject.toml`). It attaches to a substrate validator over WS or\nHTTP; there is no longer any in-process P2P node to run.\n\n```bash\n# Generate a hybrid sr25519 + ML-DSA-44 keystore\nquip-miner keygen --out ~/.quip-miner/signing.json\n\n# Bootstrap (one-shot reachability + funding check against a validator)\nquip-miner bootstrap --validator ws://127.0.0.1:9944 \\\n  --signer-key ~/.quip-miner/signing.json\n\n# CPU PoW miner\nquip-miner cpu --validator ws://127.0.0.1:9944 \\\n  --signer-key ~/.quip-miner/signing.json\n\n# CUDA / Metal / D-Wave\nquip-miner gpu --validator ws://127.0.0.1:9944 --gpu-backend local --signer-key ...\nquip-miner gpu --validator ws://127.0.0.1:9944 --gpu-backend metal --signer-key ...\nquip-miner qpu --validator ws://127.0.0.1:9944 --daily-budget 30s --signer-key ...\n\n# Multiple CPU workers (PoW + mempool jobs share the same workers)\nquip-miner cpu --validator ws://... --num-cpus 4 --signer-key ...\n\n# TOML config (see docker/quip-miner.cpu.toml, docker/quip-miner.cuda.toml)\nquip-miner cpu --config ./docker/quip-miner.cpu.toml\n\n# Production: run everything the config declares, supervised\nquip-miner --config ./docker/quip-miner.cpu.toml\n\n# Narrow a multi-backend config to one miner type (CLI-only; the\n# supervisor echoes which configured types were dropped)\nquip-miner --config config.toml --mode gpu\n```\n\nMiner-type selection is CLI-only: the supervisor's `--mode cpu|gpu|qpu`\nkeeps one configured type (warning about the dropped ones), and a\ndirect `quip-miner cpu|gpu|qpu` run does the same narrowing with the\nsame warning. There is no config key for it — a legacy `[miner] mode`\nkey still loads but is ignored.\n\n**Mempool participation is config-only and per-miner** — `mempool` is\nan unquoted TOML bool set INSIDE each backend section\n(`[cpu] mempool = false`, `[gpu]`/`[metal]`/`[modal]`, or a qpu vendor\nsection like `[dwave] mempool = true`); defaults: cpu/gpu on, qpu off —\npaid QPU samples are opt-in. A `mempool` key in `[miner]` is rejected\nat load time; `[miner] mempool_min_reward` (0 = accept all) stays\nglobal. There is no CLI flag for it and no mempool-only mode (`--mode`\nselects miner types, not the work source): every worker mines PoW\ncontinuously, mempool jobs preempt PoW on the same workers, and PoW\nresumes afterward. Solver\nregistration is automatic at miner startup (query-first, never\nauto-deregisters — switching solver type requires an explicit\n`quip-miner deregister-solver` and restart). A mempool-fatal submit\nreceipt parks the mempool side for the run while PoW mining continues.\nOn a multi-backend config the mempool owner is derived from the\nper-section keys: an explicit `mempool = true` outranks default-on\ngroups, then the first default-on group in canonical cpu,gpu,qpu order\nowns; every other child resolves mempool off from the same TOML (one\nsubstrate account can only register one solver type on chain). Set\n`mempool = false` in a section to move ownership to the next group.\nNothing is transported out-of-band, so supervised, direct-subcommand,\nand `--mode`-narrowed runs all agree; the supervisor echoes the\nelection so operators see why a child is pow-only.\n\nLive integration uses the docker-compose validator under `docker/`\n(`docker compose up quip-validator`); the validator listens on\n`ws://127.0.0.1:9944` by default.\n\n**Metal interactive cap (Apple Silicon):** the `[metal]` section runs an\nadaptive governor when `yielding` is on (default). It senses HID-idle /\nthermal / battery / displays and caps **GPU occupancy** — the jank lever is\nconcurrent threads per command buffer (`problems × reads`), not core count or\nduty cycle. While you're present it splits reads so each command buffer stays\nunder `active_util` % of the GPU's max thread capacity\n(`maxTotalThreadsPerThreadgroup × cores`; default 85); idle/headless runs\nuncapped (full speed); thermal-serious halves it; battery / critical thermal\npause. Total reads and sweeps are always preserved. (On an M4 Max steady-state\nmining is smooth even at full saturation, so the cap is mainly insurance for\nweaker GPUs / sustained thermal load.) This path\nis **independent of the CUDA util monitor** — it lives in\n`GPU/metal_scheduler.py` + `GPU/macos_sensors.py` and shares no utilization\nmachinery with `GPU/util_monitor.py`. See `docs/metal-gpu-governor.md`.\n\n## Testing\n\n```bash\n# All tests\npython -m pytest tests/ -v\n\n# Single file / single test\npython -m pytest tests/test_pool_client.py -v\npython -m pytest tests/test_pool_client.py::test_get_head_forwards_empty_args -v\n```\n\nThere are no `smoke_node_*.py` scripts in `tests/` anymore — the old\nin-process P2P node smoke tests are gone. Live integration is via\nthe docker-compose validator described above.\n\n## Benchmarking and tools\n\n```bash\n# CPU baseline\npython tools/cpu_baseline.py --quick\npython tools/cpu_baseline.py --quick \\\n  --topology dwave_topologies/topologies/advantage2_system1.json.gz\n\n# Topology analysis\npython tools/analyze_topology_sizes.py --configs \"8,2\" --samples 10\npython tools/validate_mined_topology.py --all\n\n# Precompute embedding for QPU (slow)\npython tools/analyze_topology_sizes.py --configs \"9,2\" \\\n  --precompute-embedding --embedding-timeout 1w\n\n# GPU benchmarks (Modal Labs)\nmodal run benchmarks/gpu_benchmark_modal.py\n\n# Download + re-validate every on-chain win (walks the proof chain)\npython tools/download_and_validate_wins.py \\\n  --url wss://qpu-1.nodes.quip.network/rpc --out quip_wins\n```\n\n**Never run QPU benchmarks in the background.** Provide the command;\nlet the operator execute it.\n\n### Downloading winning-solution BQMs\n\n`submit_proof` stores a compact seed (nonce + topology hash), not the full\nBinary Quadratic Model. Add `--dump-bqm` to also reconstruct each win's Ising\nmodel and write it to `<out>.bqms.jsonl`:\n\n```bash\npython tools/download_and_validate_wins.py \\\n  --url wss://qpu-1.nodes.quip.network/rpc \\\n  --max 50 --dump-bqm --out quip_wins\n```\n\nEach line is one model: `block_number`, `nonce`, `topology_hash`, plus the\nreconstructed Ising model as flat lists — `h: [[node_id, bias], ...]` and\n`j: [[u, v, coupling], ...]`. Reload into the dicts the energy functions\nexpect with:\n\n```python\nh = {n: b for n, b in rec[\"h\"]}\nJ = {(u, v): c for u, v, c in rec[\"j\"]}\n```\n\nThe model is re-derived from the nonce + topology snapshot via\n`generate_ising_model_from_nonce` — the same function `_finalize_sample` and\nthe validator use, so a green validation run proves the dumped BQMs are\ncorrect.\n\n### Modal Labs (cloud GPU)\n\n```bash\npip install modal\nmodal token new  # opens a browser for authentication\n```\n\n## Dependencies\n\nFrom `pyproject.toml`:\n\n- **Core**: `dwave-ocean-sdk`, `numpy`, `aiohttp`, `click`, `blake3`,\n  `substrate-interface`, `scalecodec`, `dilithium-py` (ML-DSA-44),\n  `python-dotenv`, `tomli` (on 3.10).\n- **CUDA** (optional): `cupy-cuda12x`, `nvidia-ml-py`.\n- **Metal** (optional): `pyobjc-framework-Metal`,\n  `pyobjc-framework-MetalPerformanceShaders`.\n- **fast** (optional): `pyzmq`, `uvloop`.\n- **dev**: `pytest`, `pytest-asyncio`.\n\nPython 3.10+ required.\n\n**Removed in v0.2**: `aioquic`, `hashsigs` (legacy SPHINCS+),\n`cryptography` (was for self-signed TLS). The QUIC P2P stack went\nwith them.\n\n## Topology management (`dwave_topologies/`)\n\n- `topologies/*.json.gz` — hardware topology files (Advantage2, Chimera, Pegasus, Zephyr).\n- `embeddings/*.json.gz` — precomputed embeddings for QPU hardware mapping.\n- `embedded_topology.py`, `embedding_loader.py`, `smart_embedding.py` — loaders and embedding utilities.\n- **Default**: Zephyr Z(9,2) — 1,368 nodes, 7,692 edges.\n- For full Advantage2 hardware: `advantage2_system1.json.gz` (~4,579 qubits).\n\n**QPU solver revision updates** (when D-Wave recalibrates):\n1. Replace the topology file in `topologies/`.\n2. Verify existing `embeddings/` still match the new graph; copy over compatible ones.\n3. Delete stale topology files and incompatible embeddings.\n\n## Critical parameters\n\n**Genesis block defaults** (`genesis_block.json`):\n```python\ndifficulty_energy = -2500.0\nmin_diversity = 0.2\nmin_solutions = 5\nh_values = [-1.0, 0.0, 1.0]\n```\n\n**Production Z(9,2) targets:**\n```python\ndifficulty_energy = -4100.0\nmin_diversity = 0.15\nmin_solutions = 5\n```\n\n**Energy ranges by topology (GSE):**\n\n| Topology | Range | Size |\n|---|---|---|\n| Z(8,2) | -2869 to -2677 | 192 |\n| Z(9,2) | -4100 to -3870 | 230 (default) |\n| Z(10,2) | -5470 to -5200 | 270 |\n| Z(11,4) | -15170 to -14158 | 1012 |\n| Advantage2 (full) | similar to Z(11,4) | ~4579 qubits |\n\n**QPU h/J ranges:** See [`docs/dwave-solver-ranges.md`](docs/dwave-solver-ranges.md)\nfor per-solver `h_range`, `j_range`, `extended_j_range`, and\n`per_qubit_coupling_range`. Regenerate with\n`python tools/dump_solver_ranges.py`.\n\n**Per-miner adaptive parameters:**\n\n| Backend | num_sweeps | num_reads |\n|---|---|---|\n| CPU/SA | 64–4096 | 64–512 |\n| GPU/CUDA | 256–2048 | (adaptive) |\n| GPU/Metal | 64–512 | (adaptive) |\n| Modal | 128–4096 | (adaptive) |\n| QPU | annealing 5–20µs | 32–64 |\n\n## Identifiers: solution number vs `dispatch_id`\n\nTwo distinct identifiers exist; **never conflate them**, and never persist\non `dispatch_id`.\n\n- **Solution number** — the chain-global *ordinal* of the QPoW solution\n  being mined: `count(QuantumPow.WinningSolutions) + 1`. The chain has no\n  stored solution counter — solutions are keyed in `WinningSolutions` by\n  the block number they won at (`submitted_at == key`), so the ordinal is\n  derived by counting that map's keys (cheap: one paged `state_getKeysPaged`\n  over the storage prefix, keys only). It is **determinable at round\n  start** — a round mines toward a specific upcoming solution number; the\n  round's `last_proof_block_hash` (= `block_hash(LastProofBlock)`) stays\n  constant until a proof wins and advances it. Compute the count **once per\n  round and cache it** (recount only when `last_proof_block_hash` changes);\n  it is global, monotonic, and never repeats.\n  - **Not the same as a block number.** `LastProofBlock` (e.g. 52507) is\n    the block number of the most recent winning proof — the round anchor,\n    not the solution ordinal (e.g. 196). Don't conflate them.\n  - **This is the on-disk key for the mining-attempts archive**\n    (`{base}/{solution_number}/…`). Because it tracks the logical\n    solution, a controller/worker restart mid-round correctly *resumes*\n    writing into the same solution dir — that is not stale accretion, it\n    is the same solution. Different solution number ⇒ different dir.\n\n- **`dispatch_id`** — an **internal-only** scheduler↔worker coordination\n  handle: `_dispatch_contexts[(handle_id, dispatch_id)]`, cancel-ack\n  (`_await_done_sentinel`), and the preview channel (see `ARCHITECTURE.md`\n  §3.3). It is a process-local monotonic counter that **resets to 0 on\n  restart**, so it collides across runs and MUST NOT be used as a durable\n  or on-disk identity. Keep it in memory for pairing responses to\n  contexts; never name persisted artifacts after it.\n\nThe old `SubmissionLogger` `next_solution_id` local counter was removed;\nthe on-disk archive is keyed by the chain-derived solution number.\n\n## Code style\n\n- **Imports** at top of file, in stdlib → third-party → local order. Absolute imports only. No inline imports inside functions/methods. Exception: optional-dependency `try/except` at module level.\n- **Concurrency**: NEVER use threads. Multiprocessing only. Async via `asyncio` for network operations. Mining runs in worker processes (`shared/miner_worker.py:MinerHandle`).\n- **Type hints** on public APIs. Google-style docstrings on non-trivial public functions.\n- **Logging** via `logging.getLogger(__name__)` at module level. The custom formatter and component classification live in `shared/logging_config.py`.\n- **No `Co-Authored-By: <assistant> ...` lines in commits.**\n\nSee [`ARCHITECTURE.md`](./ARCHITECTURE.md) §9 for the cleanup-candidates\nlist (vestigial code flagged for removal).\n\n## Deployment\n\n- **Docker** (`docker/`): `Dockerfile.cpu`, `Dockerfile.cuda`, `docker-compose.yml`, `entrypoint.sh`, plus the example TOML configs (`quip-miner.cpu.toml`, `quip-miner.cuda.toml`).\n- **Cloud**: `akash/` and `aws/` contain deployment configs.\n- **GitLab CI** (`.gitlab-ci.yml`): builds CPU + CUDA Docker images on main/tags.\n\n## Concurrency: processes, not threads\n\nUse `multiprocessing` for concurrency. Do **not** introduce `threading.Thread`\nin our own code. Threads share the GIL; a CPU-bound thread starves its\nsiblings. This stalled the QPU stream pump in v0.2 — the per-attempt consumer\nheld the GIL ~1.2 s and the pump thread could not drive the sampler, so the\nQPU pipeline drained (throughput 0.6 vs ~8 subs/sec).\n\nRules:\n- New background work → a `multiprocessing` process (use `spawn`).\n- Share scalars via `multiprocessing.Value`; share large numpy buffers via\n  `multiprocessing.shared_memory.SharedMemory` (zero-copy), never by pickling\n  per item on a hot path.\n- `threading.Lock`/`RLock` for intra-process state is fine (a lock is not a\n  thread). Cross-process correctness uses `os.replace`/PIPE_BUF atomicity or\n  `mp` primitives, not locks.\n- Exceptions we don't control: third-party internal threads (D-Wave SDK,\n  asyncio executors, stdlib `QueueListener` if ever reused). Document any such\n  exception inline with the reason.\n\n## Debugging a hung process (get a traceback)\n\n`SIGINT` (Ctrl-C) only helps when the **main thread is running Python\nbytecode** — it raises `KeyboardInterrupt` at the next bytecode boundary. A\nprocess wedged in a C-level call (a lock/`join`/`wait`, an `mp.Queue`\nfeeder-thread join at interpreter shutdown, a blocking syscall) won't unwind,\nso `SIGINT` just kills it with **no traceback**. Don't reach for it on a hang.\n\nUse **`faulthandler`** + **`SIGABRT`** — the repo convention (CI already runs\n`timeout --signal=ABRT … python -X faulthandler -m pytest`):\n\n```bash\n# 1. Start the process with faulthandler enabled (installs handlers for the\n#    fatal signals SIGSEGV/SIGFPE/SIGABRT/SIGBUS/SIGILL).\nPYTHONFAULTHANDLER=1 python tools/whatever.py …      # or: python -X faulthandler …\n\n# 2. When it hangs, dump every thread's Python (and C) stack to stderr:\nkill -ABRT <pid>          # SIGABRT -> faulthandler dumps, then the process aborts (exit 134)\n```\n\nThis prints the exact frame each thread is stuck in (e.g. `threading.py:wait`\n→ an unjoined queue feeder thread). Notes:\n\n- **Pre-arm it.** Faulthandler must be enabled *before* the hang. For tools we\n  expect to run interactively against the QPU/long pipelines, prefer enabling\n  it (env var or `faulthandler.enable()` at startup) so a hang is debuggable.\n- **Dump without killing:** `faulthandler.register(signal.SIGUSR1)` in code,\n  then `kill -USR1 <pid>` dumps and **continues** (repeatable). `SIGABRT` is\n  fatal; `SIGUSR1` (registered) is not.\n- **No instrumentation available?** `py-spy dump --pid <pid>` attaches to any\n  running CPython and prints all-thread tracebacks without a signal or restart.\n- **Inspect the tree first:** `ps -o pid,ppid,stat,command -p <pid>` and\n  `pgrep -P <pid>` reveal stuck children / orphans (`PPID 1` = reparented after\n  a parent crash). An interpreter that won't exit is usually blocked joining a\n  non-daemon child or an `mp.Queue` feeder thread (call `cancel_join_thread()`\n  on queues whose buffered data is worthless at teardown).\n"},"files":{"AGENTS.md":"# AGENTS.md — QuIP Protocol Project Instructions\n\nCross-tool instructions for AI coding assistants (Claude Code, Codex, Cursor, Gemini CLI).\n\nFor the **runtime architecture**, read [`ARCHITECTURE.md`](./ARCHITECTURE.md).\nThis file is the developer-facing how-to: commands, dependencies, code\nstyle. Anything about how the system *runs* belongs in\n`ARCHITECTURE.md`.\n\n## Environment\n\nThe `.quip` virtualenv is already active in development shells —\ndon't prefix shell commands with `source .quip/bin/activate`.\n\n```bash\n# Fresh install (only if .quip doesn't exist yet)\npython3 -m venv .quip\nsource .quip/bin/activate\npip install -U pip setuptools wheel\npip install -e .            # core + CPU\npip install -e .[cuda]      # CUDA backend\npip install -e .[metal]     # Apple Silicon backend\npip install -e .[dev]       # pytest + pytest-asyncio\n```\n\n`.env` holds `DWAVE_API_KEY` and other credentials. **Never read or\ndisplay its contents.**\n\n## Running the miner\n\nThe CLI is `quip-miner` (defined in `quip_cli.py`, entry point in\n`pyproject.toml`). It attaches to a substrate validator over WS or\nHTTP; there is no longer any in-process P2P node to run.\n\n```bash\n# Generate a hybrid sr25519 + ML-DSA-44 keystore\nquip-miner keygen --out ~/.quip-miner/signing.json\n\n# Bootstrap (one-shot reachability + funding check against a validator)\nquip-miner bootstrap --validator ws://127.0.0.1:9944 \\\n  --signer-key ~/.quip-miner/signing.json\n\n# CPU PoW miner\nquip-miner cpu --validator ws://127.0.0.1:9944 \\\n  --signer-key ~/.quip-miner/signing.json\n\n# CUDA / Metal / D-Wave\nquip-miner gpu --validator ws://127.0.0.1:9944 --gpu-backend local --signer-key ...\nquip-miner gpu --validator ws://127.0.0.1:9944 --gpu-backend metal --signer-key ...\nquip-miner qpu --validator ws://127.0.0.1:9944 --daily-budget 30s --signer-key ...\n\n# Multiple CPU workers (PoW + mempool jobs share the same workers)\nquip-miner cpu --validator ws://... --num-cpus 4 --signer-key ...\n\n# TOML config (see docker/quip-miner.cpu.toml, docker/quip-miner.cuda.toml)\nquip-miner cpu --config ./docker/quip-miner.cpu.toml\n\n# Production: run everything the config declares, supervised\nquip-miner --config ./docker/quip-miner.cpu.toml\n\n# Narrow a multi-backend config to one miner type (CLI-only; the\n# supervisor echoes which configured types were dropped)\nquip-miner --config config.toml --mode gpu\n```\n\nMiner-type selection is CLI-only: the supervisor's `--mode cpu|gpu|qpu`\nkeeps one configured type (warning about the dropped ones), and a\ndirect `quip-miner cpu|gpu|qpu` run does the same narrowing with the\nsame warning. There is no config key for it — a legacy `[miner] mode`\nkey still loads but is ignored.\n\n**Mempool participation is config-only and per-miner** — `mempool` is\nan unquoted TOML bool set INSIDE each backend section\n(`[cpu] mempool = false`, `[gpu]`/`[metal]`/`[modal]`, or a qpu vendor\nsection like `[dwave] mempool = true`); defaults: cpu/gpu on, qpu off —\npaid QPU samples are opt-in. A `mempool` key in `[miner]` is rejected\nat load time; `[miner] mempool_min_reward` (0 = accept all) stays\nglobal. There is no CLI flag for it and no mempool-only mode (`--mode`\nselects miner types, not the work source): every worker mines PoW\ncontinuously, mempool jobs preempt PoW on the same workers, and PoW\nresumes afterward. Solver\nregistration is automatic at miner startup (query-first, never\nauto-deregisters — switching solver type requires an explicit\n`quip-miner deregister-solver` and restart). A mempool-fatal submit\nreceipt parks the mempool side for the run while PoW mining continues.\nOn a multi-backend config the mempool owner is derived from the\nper-section keys: an explicit `mempool = true` outranks default-on\ngroups, then the first default-on group in canonical cpu,gpu,qpu order\nowns; every other child resolves mempool off from the same TOML (one\nsubstrate account can only register one solver type on chain). Set\n`mempool = false` in a section to move ownership to the next group.\nNothing is transported out-of-band, so supervised, direct-subcommand,\nand `--mode`-narrowed runs all agree; the supervisor echoes the\nelection so operators see why a child is pow-only.\n\nLive integration uses the docker-compose validator under `docker/`\n(`docker compose up quip-validator`); the validator listens on\n`ws://127.0.0.1:9944` by default.\n\n**Metal interactive cap (Apple Silicon):** the `[metal]` section runs an\nadaptive governor when `yielding` is on (default). It senses HID-idle /\nthermal / battery / displays and caps **GPU occupancy** — the jank lever is\nconcurrent threads per command buffer (`problems × reads`), not core count or\nduty cycle. While you're present it splits reads so each command buffer stays\nunder `active_util` % of the GPU's max thread capacity\n(`maxTotalThreadsPerThreadgroup × cores`; default 85); idle/headless runs\nuncapped (full speed); thermal-serious halves it; battery / critical thermal\npause. Total reads and sweeps are always preserved. (On an M4 Max steady-state\nmining is smooth even at full saturation, so the cap is mainly insurance for\nweaker GPUs / sustained thermal load.) This path\nis **independent of the CUDA util monitor** — it lives in\n`GPU/metal_scheduler.py` + `GPU/macos_sensors.py` and shares no utilization\nmachinery with `GPU/util_monitor.py`. See `docs/metal-gpu-governor.md`.\n\n## Testing\n\n```bash\n# All tests\npython -m pytest tests/ -v\n\n# Single file / single test\npython -m pytest tests/test_pool_client.py -v\npython -m pytest tests/test_pool_client.py::test_get_head_forwards_empty_args -v\n```\n\nThere are no `smoke_node_*.py` scripts in `tests/` anymore — the old\nin-process P2P node smoke tests are gone. Live integration is via\nthe docker-compose validator described above.\n\n## Benchmarking and tools\n\n```bash\n# CPU baseline\npython tools/cpu_baseline.py --quick\npython tools/cpu_baseline.py --quick \\\n  --topology dwave_topologies/topologies/advantage2_system1.json.gz\n\n# Topology analysis\npython tools/analyze_topology_sizes.py --configs \"8,2\" --samples 10\npython tools/validate_mined_topology.py --all\n\n# Precompute embedding for QPU (slow)\npython tools/analyze_topology_sizes.py --configs \"9,2\" \\\n  --precompute-embedding --embedding-timeout 1w\n\n# GPU benchmarks (Modal Labs)\nmodal run benchmarks/gpu_benchmark_modal.py\n\n# Download + re-validate every on-chain win (walks the proof chain)\npython tools/download_and_validate_wins.py \\\n  --url wss://qpu-1.nodes.quip.network/rpc --out quip_wins\n```\n\n**Never run QPU benchmarks in the background.** Provide the command;\nlet the operator execute it.\n\n### Downloading winning-solution BQMs\n\n`submit_proof` stores a compact seed (nonce + topology hash), not the full\nBinary Quadratic Model. Add `--dump-bqm` to also reconstruct each win's Ising\nmodel and write it to `<out>.bqms.jsonl`:\n\n```bash\npython tools/download_and_validate_wins.py \\\n  --url wss://qpu-1.nodes.quip.network/rpc \\\n  --max 50 --dump-bqm --out quip_wins\n```\n\nEach line is one model: `block_number`, `nonce`, `topology_hash`, plus the\nreconstructed Ising model as flat lists — `h: [[node_id, bias], ...]` and\n`j: [[u, v, coupling], ...]`. Reload into the dicts the energy functions\nexpect with:\n\n```python\nh = {n: b for n, b in rec[\"h\"]}\nJ = {(u, v): c for u, v, c in rec[\"j\"]}\n```\n\nThe model is re-derived from the nonce + topology snapshot via\n`generate_ising_model_from_nonce` — the same function `_finalize_sample` and\nthe validator use, so a green validation run proves the dumped BQMs are\ncorrect.\n\n### Modal Labs (cloud GPU)\n\n```bash\npip install modal\nmodal token new  # opens a browser for authentication\n```\n\n## Dependencies\n\nFrom `pyproject.toml`:\n\n- **Core**: `dwave-ocean-sdk`, `numpy`, `aiohttp`, `click`, `blake3`,\n  `substrate-interface`, `scalecodec`, `dilithium-py` (ML-DSA-44),\n  `python-dotenv`, `tomli` (on 3.10).\n- **CUDA** (optional): `cupy-cuda12x`, `nvidia-ml-py`.\n- **Metal** (optional): `pyobjc-framework-Metal`,\n  `pyobjc-framework-MetalPerformanceShaders`.\n- **fast** (optional): `pyzmq`, `uvloop`.\n- **dev**: `pytest`, `pytest-asyncio`.\n\nPython 3.10+ required.\n\n**Removed in v0.2**: `aioquic`, `hashsigs` (legacy SPHINCS+),\n`cryptography` (was for self-signed TLS). The QUIC P2P stack went\nwith them.\n\n## Topology management (`dwave_topologies/`)\n\n- `topologies/*.json.gz` — hardware topology files (Advantage2, Chimera, Pegasus, Zephyr).\n- `embeddings/*.json.gz` — precomputed embeddings for QPU hardware mapping.\n- `embedded_topology.py`, `embedding_loader.py`, `smart_embedding.py` — loaders and embedding utilities.\n- **Default**: Zephyr Z(9,2) — 1,368 nodes, 7,692 edges.\n- For full Advantage2 hardware: `advantage2_system1.json.gz` (~4,579 qubits).\n\n**QPU solver revision updates** (when D-Wave recalibrates):\n1. Replace the topology file in `topologies/`.\n2. Verify existing `embeddings/` still match the new graph; copy over compatible ones.\n3. Delete stale topology files and incompatible embeddings.\n\n## Critical parameters\n\n**Genesis block defaults** (`genesis_block.json`):\n```python\ndifficulty_energy = -2500.0\nmin_diversity = 0.2\nmin_solutions = 5\nh_values = [-1.0, 0.0, 1.0]\n```\n\n**Production Z(9,2) targets:**\n```python\ndifficulty_energy = -4100.0\nmin_diversity = 0.15\nmin_solutions = 5\n```\n\n**Energy ranges by topology (GSE):**\n\n| Topology | Range | Size |\n|---|---|---|\n| Z(8,2) | -2869 to -2677 | 192 |\n| Z(9,2) | -4100 to -3870 | 230 (default) |\n| Z(10,2) | -5470 to -5200 | 270 |\n| Z(11,4) | -15170 to -14158 | 1012 |\n| Advantage2 (full) | similar to Z(11,4) | ~4579 qubits |\n\n**QPU h/J ranges:** See [`docs/dwave-solver-ranges.md`](docs/dwave-solver-ranges.md)\nfor per-solver `h_range`, `j_range`, `extended_j_range`, and\n`per_qubit_coupling_range`. Regenerate with\n`python tools/dump_solver_ranges.py`.\n\n**Per-miner adaptive parameters:**\n\n| Backend | num_sweeps | num_reads |\n|---|---|---|\n| CPU/SA | 64–4096 | 64–512 |\n| GPU/CUDA | 256–2048 | (adaptive) |\n| GPU/Metal | 64–512 | (adaptive) |\n| Modal | 128–4096 | (adaptive) |\n| QPU | annealing 5–20µs | 32–64 |\n\n## Identifiers: solution number vs `dispatch_id`\n\nTwo distinct identifiers exist; **never conflate them**, and never persist\non `dispatch_id`.\n\n- **Solution number** — the chain-global *ordinal* of the QPoW solution\n  being mined: `count(QuantumPow.WinningSolutions) + 1`. The chain has no\n  stored solution counter — solutions are keyed in `WinningSolutions` by\n  the block number they won at (`submitted_at == key`), so the ordinal is\n  derived by counting that map's keys (cheap: one paged `state_getKeysPaged`\n  over the storage prefix, keys only). It is **determinable at round\n  start** — a round mines toward a specific upcoming solution number; the\n  round's `last_proof_block_hash` (= `block_hash(LastProofBlock)`) stays\n  constant until a proof wins and advances it. Compute the count **once per\n  round and cache it** (recount only when `last_proof_block_hash` changes);\n  it is global, monotonic, and never repeats.\n  - **Not the same as a block number.** `LastProofBlock` (e.g. 52507) is\n    the block number of the most recent winning proof — the round anchor,\n    not the solution ordinal (e.g. 196). Don't conflate them.\n  - **This is the on-disk key for the mining-attempts archive**\n    (`{base}/{solution_number}/…`). Because it tracks the logical\n    solution, a controller/worker restart mid-round correctly *resumes*\n    writing into the same solution dir — that is not stale accretion, it\n    is the same solution. Different solution number ⇒ different dir.\n\n- **`dispatch_id`** — an **internal-only** scheduler↔worker coordination\n  handle: `_dispatch_contexts[(handle_id, dispatch_id)]`, cancel-ack\n  (`_await_done_sentinel`), and the preview channel (see `ARCHITECTURE.md`\n  §3.3). It is a process-local monotonic counter that **resets to 0 on\n  restart**, so it collides across runs and MUST NOT be used as a durable\n  or on-disk identity. Keep it in memory for pairing responses to\n  contexts; never name persisted artifacts after it.\n\nThe old `SubmissionLogger` `next_solution_id` local counter was removed;\nthe on-disk archive is keyed by the chain-derived solution number.\n\n## Code style\n\n- **Imports** at top of file, in stdlib → third-party → local order. Absolute imports only. No inline imports inside functions/methods. Exception: optional-dependency `try/except` at module level.\n- **Concurrency**: NEVER use threads. Multiprocessing only. Async via `asyncio` for network operations. Mining runs in worker processes (`shared/miner_worker.py:MinerHandle`).\n- **Type hints** on public APIs. Google-style docstrings on non-trivial public functions.\n- **Logging** via `logging.getLogger(__name__)` at module level. The custom formatter and component classification live in `shared/logging_config.py`.\n- **No `Co-Authored-By: <assistant> ...` lines in commits.**\n\nSee [`ARCHITECTURE.md`](./ARCHITECTURE.md) §9 for the cleanup-candidates\nlist (vestigial code flagged for removal).\n\n## Deployment\n\n- **Docker** (`docker/`): `Dockerfile.cpu`, `Dockerfile.cuda`, `docker-compose.yml`, `entrypoint.sh`, plus the example TOML configs (`quip-miner.cpu.toml`, `quip-miner.cuda.toml`).\n- **Cloud**: `akash/` and `aws/` contain deployment configs.\n- **GitLab CI** (`.gitlab-ci.yml`): builds CPU + CUDA Docker images on main/tags.\n\n## Concurrency: processes, not threads\n\nUse `multiprocessing` for concurrency. Do **not** introduce `threading.Thread`\nin our own code. Threads share the GIL; a CPU-bound thread starves its\nsiblings. This stalled the QPU stream pump in v0.2 — the per-attempt consumer\nheld the GIL ~1.2 s and the pump thread could not drive the sampler, so the\nQPU pipeline drained (throughput 0.6 vs ~8 subs/sec).\n\nRules:\n- New background work → a `multiprocessing` process (use `spawn`).\n- Share scalars via `multiprocessing.Value`; share large numpy buffers via\n  `multiprocessing.shared_memory.SharedMemory` (zero-copy), never by pickling\n  per item on a hot path.\n- `threading.Lock`/`RLock` for intra-process state is fine (a lock is not a\n  thread). Cross-process correctness uses `os.replace`/PIPE_BUF atomicity or\n  `mp` primitives, not locks.\n- Exceptions we don't control: third-party internal threads (D-Wave SDK,\n  asyncio executors, stdlib `QueueListener` if ever reused). Document any such\n  exception inline with the reason.\n\n## Debugging a hung process (get a traceback)\n\n`SIGINT` (Ctrl-C) only helps when the **main thread is running Python\nbytecode** — it raises `KeyboardInterrupt` at the next bytecode boundary. A\nprocess wedged in a C-level call (a lock/`join`/`wait`, an `mp.Queue`\nfeeder-thread join at interpreter shutdown, a blocking syscall) won't unwind,\nso `SIGINT` just kills it with **no traceback**. Don't reach for it on a hang.\n\nUse **`faulthandler`** + **`SIGABRT`** — the repo convention (CI already runs\n`timeout --signal=ABRT … python -X faulthandler -m pytest`):\n\n```bash\n# 1. Start the process with faulthandler enabled (installs handlers for the\n#    fatal signals SIGSEGV/SIGFPE/SIGABRT/SIGBUS/SIGILL).\nPYTHONFAULTHANDLER=1 python tools/whatever.py …      # or: python -X faulthandler …\n\n# 2. When it hangs, dump every thread's Python (and C) stack to stderr:\nkill -ABRT <pid>          # SIGABRT -> faulthandler dumps, then the process aborts (exit 134)\n```\n\nThis prints the exact frame each thread is stuck in (e.g. `threading.py:wait`\n→ an unjoined queue feeder thread). Notes:\n\n- **Pre-arm it.** Faulthandler must be enabled *before* the hang. For tools we\n  expect to run interactively against the QPU/long pipelines, prefer enabling\n  it (env var or `faulthandler.enable()` at startup) so a hang is debuggable.\n- **Dump without killing:** `faulthandler.register(signal.SIGUSR1)` in code,\n  then `kill -USR1 <pid>` dumps and **continues** (repeatable). `SIGABRT` is\n  fatal; `SIGUSR1` (registered) is not.\n- **No instrumentation available?** `py-spy dump --pid <pid>` attaches to any\n  running CPython and prints all-thread tracebacks without a signal or restart.\n- **Inspect the tree first:** `ps -o pid,ppid,stat,command -p <pid>` and\n  `pgrep -P <pid>` reveal stuck children / orphans (`PPID 1` = reparented after\n  a parent crash). An interpreter that won't exit is usually blocked joining a\n  non-daemon child or an `mp.Queue` feeder thread (call `cancel_join_thread()`\n  on queues whose buffered data is worthless at teardown).\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"# AGENTS.md — QuIP Protocol Project Instructions\n\nCross-tool instructions for AI coding assistants (Claude Code, Codex, Cursor, Gemini CLI).\n\nFor the **runtime architecture**, read [`ARCHITECTURE.md`](./ARCHITECTURE.md).\nThis file is the developer-facing how-to: commands, dependencies, code\nstyle. Anything about how the system *runs* belongs in\n`ARCHITECTURE.md`.\n\n## Environment\n\nThe `.quip` virtualenv is already active in development shells —\ndon't prefix shell commands with `source .quip/bin/activate`.\n\n```bash\n# Fresh install (only if .quip doesn't exist yet)\npython3 -m venv .quip\nsource .quip/bin/activate\npip install -U pip setuptools wheel\npip install -e .            # core + CPU\npip install -e .[cuda]      # CUDA backend\npip install -e .[metal]     # Apple Silicon backend\npip install -e .[dev]       # pytest + pytest-asyncio\n```\n\n`.env` holds `DWAVE_API_KEY` and other credentials. **Never read or\ndisplay its contents.**\n\n## Running the miner\n\nThe CLI is `quip-miner` (defined in `quip_cli.py`, entry point in\n`pyproject.toml`). It attaches to a substrate validator over WS or\nHTTP; there is no longer any in-process P2P node to run.\n\n```bash\n# Generate a hybrid sr25519 + ML-DSA-44 keystore\nquip-miner keygen --out ~/.quip-miner/signing.json\n\n# Bootstrap (one-shot reachability + funding check against a validator)\nquip-miner bootstrap --validator ws://127.0.0.1:9944 \\\n  --signer-key ~/.quip-miner/signing.json\n\n# CPU PoW miner\nquip-miner cpu --validator ws://127.0.0.1:9944 \\\n  --signer-key ~/.quip-miner/signing.json\n\n# CUDA / Metal / D-Wave\nquip-miner gpu --validator ws://127.0.0.1:9944 --gpu-backend local --signer-key ...\nquip-miner gpu --validator ws://127.0.0.1:9944 --gpu-backend metal --signer-key ...\nquip-miner qpu --validator ws://127.0.0.1:9944 --daily-budget 30s --signer-key ...\n\n# Multiple CPU workers (PoW + mempool jobs share the same workers)\nquip-miner cpu --validator ws://... --num-cpus 4 --signer-key ...\n\n# TOML config (see docker/quip-miner.cpu.toml, docker/quip-miner.cuda.toml)\nquip-miner cpu --config ./docker/quip-miner.cpu.toml\n\n# Production: run everything the config declares, supervised\nquip-miner --config ./docker/quip-miner.cpu.toml\n\n# Narrow a multi-backend config to one miner type (CLI-only; the\n# supervisor echoes which configured types were dropped)\nquip-miner --config config.toml --mode gpu\n```\n\nMiner-type selection is CLI-only: the supervisor's `--mode cpu|gpu|qpu`\nkeeps one configured type (warning about the dropped ones), and a\ndirect `quip-miner cpu|gpu|qpu` run does the same narrowing with the\nsame warning. There is no config key for it — a legacy `[miner] mode`\nkey still loads but is ignored.\n\n**Mempool participation is config-only and per-miner** — `mempool` is\nan unquoted TOML bool set INSIDE each backend section\n(`[cpu] mempool = false`, `[gpu]`/`[metal]`/`[modal]`, or a qpu vendor\nsection like `[dwave] mempool = true`); defaults: cpu/gpu on, qpu off —\npaid QPU samples are opt-in. A `mempool` key in `[miner]` is rejected\nat load time; `[miner] mempool_min_reward` (0 = accept all) stays\nglobal. There is no CLI flag for it and no mempool-only mode (`--mode`\nselects miner types, not the work source): every worker mines PoW\ncontinuously, mempool jobs preempt PoW on the same workers, and PoW\nresumes afterward. Solver\nregistration is automatic at miner startup (query-first, never\nauto-deregisters — switching solver type requires an explicit\n`quip-miner deregister-solver` and restart). A mempool-fatal submit\nreceipt parks the mempool side for the run while PoW mining continues.\nOn a multi-backend config the mempool owner is derived from the\nper-section keys: an explicit `mempool = true` outranks default-on\ngroups, then the first default-on group in canonical cpu,gpu,qpu order\nowns; every other child resolves mempool off from the same TOML (one\nsubstrate account can only register one solver type on chain). Set\n`mempool = false` in a section to move ownership to the next group.\nNothing is transported out-of-band, so supervised, direct-subcommand,\nand `--mode`-narrowed runs all agree; the supervisor echoes the\nelection so operators see why a child is pow-only.\n\nLive integration uses the docker-compose validator under `docker/`\n(`docker compose up quip-validator`); the validator listens on\n`ws://127.0.0.1:9944` by default.\n\n**Metal interactive cap (Apple Silicon):** the `[metal]` section runs an\nadaptive governor when `yielding` is on (default). It senses HID-idle /\nthermal / battery / displays and caps **GPU occupancy** — the jank lever is\nconcurrent threads per command buffer (`problems × reads`), not core count or\nduty cycle. While you're present it splits reads so each command buffer stays\nunder `active_util` % of the GPU's max thread capacity\n(`maxTotalThreadsPerThreadgroup × cores`; default 85); idle/headless runs\nuncapped (full speed); thermal-serious halves it; battery / critical thermal\npause. Total reads and sweeps are always preserved. (On an M4 Max steady-state\nmining is smooth even at full saturation, so the cap is mainly insurance for\nweaker GPUs / sustained thermal load.) This path\nis **independent of the CUDA util monitor** — it lives in\n`GPU/metal_scheduler.py` + `GPU/macos_sensors.py` and shares no utilization\nmachinery with `GPU/util_monitor.py`. See `docs/metal-gpu-governor.md`.\n\n## Testing\n\n```bash\n# All tests\npython -m pytest tests/ -v\n\n# Single file / single test\npython -m pytest tests/test_pool_client.py -v\npython -m pytest tests/test_pool_client.py::test_get_head_forwards_empty_args -v\n```\n\nThere are no `smoke_node_*.py` scripts in `tests/` anymore — the old\nin-process P2P node smoke tests are gone. Live integration is via\nthe docker-compose validator described above.\n\n## Benchmarking and tools\n\n```bash\n# CPU baseline\npython tools/cpu_baseline.py --quick\npython tools/cpu_baseline.py --quick \\\n  --topology dwave_topologies/topologies/advantage2_system1.json.gz\n\n# Topology analysis\npython tools/analyze_topology_sizes.py --configs \"8,2\" --samples 10\npython tools/validate_mined_topology.py --all\n\n# Precompute embedding for QPU (slow)\npython tools/analyze_topology_sizes.py --configs \"9,2\" \\\n  --precompute-embedding --embedding-timeout 1w\n\n# GPU benchmarks (Modal Labs)\nmodal run benchmarks/gpu_benchmark_modal.py\n\n# Download + re-validate every on-chain win (walks the proof chain)\npython tools/download_and_validate_wins.py \\\n  --url wss://qpu-1.nodes.quip.network/rpc --out quip_wins\n```\n\n**Never run QPU benchmarks in the background.** Provide the command;\nlet the operator execute it.\n\n### Downloading winning-solution BQMs\n\n`submit_proof` stores a compact seed (nonce + topology hash), not the full\nBinary Quadratic Model. Add `--dump-bqm` to also reconstruct each win's Ising\nmodel and write it to `<out>.bqms.jsonl`:\n\n```bash\npython tools/download_and_validate_wins.py \\\n  --url wss://qpu-1.nodes.quip.network/rpc \\\n  --max 50 --dump-bqm --out quip_wins\n```\n\nEach line is one model: `block_number`, `nonce`, `topology_hash`, plus the\nreconstructed Ising model as flat lists — `h: [[node_id, bias], ...]` and\n`j: [[u, v, coupling], ...]`. Reload into the dicts the energy functions\nexpect with:\n\n```python\nh = {n: b for n, b in rec[\"h\"]}\nJ = {(u, v): c for u, v, c in rec[\"j\"]}\n```\n\nThe model is re-derived from the nonce + topology snapshot via\n`generate_ising_model_from_nonce` — the same function `_finalize_sample` and\nthe validator use, so a green validation run proves the dumped BQMs are\ncorrect.\n\n### Modal Labs (cloud GPU)\n\n```bash\npip install modal\nmodal token new  # opens a browser for authentication\n```\n\n## Dependencies\n\nFrom `pyproject.toml`:\n\n- **Core**: `dwave-ocean-sdk`, `numpy`, `aiohttp`, `click`, `blake3`,\n  `substrate-interface`, `scalecodec`, `dilithium-py` (ML-DSA-44),\n  `python-dotenv`, `tomli` (on 3.10).\n- **CUDA** (optional): `cupy-cuda12x`, `nvidia-ml-py`.\n- **Metal** (optional): `pyobjc-framework-Metal`,\n  `pyobjc-framework-MetalPerformanceShaders`.\n- **fast** (optional): `pyzmq`, `uvloop`.\n- **dev**: `pytest`, `pytest-asyncio`.\n\nPython 3.10+ required.\n\n**Removed in v0.2**: `aioquic`, `hashsigs` (legacy SPHINCS+),\n`cryptography` (was for self-signed TLS). The QUIC P2P stack went\nwith them.\n\n## Topology management (`dwave_topologies/`)\n\n- `topologies/*.json.gz` — hardware topology files (Advantage2, Chimera, Pegasus, Zephyr).\n- `embeddings/*.json.gz` — precomputed embeddings for QPU hardware mapping.\n- `embedded_topology.py`, `embedding_loader.py`, `smart_embedding.py` — loaders and embedding utilities.\n- **Default**: Zephyr Z(9,2) — 1,368 nodes, 7,692 edges.\n- For full Advantage2 hardware: `advantage2_system1.json.gz` (~4,579 qubits).\n\n**QPU solver revision updates** (when D-Wave recalibrates):\n1. Replace the topology file in `topologies/`.\n2. Verify existing `embeddings/` still match the new graph; copy over compatible ones.\n3. Delete stale topology files and incompatible embeddings.\n\n## Critical parameters\n\n**Genesis block defaults** (`genesis_block.json`):\n```python\ndifficulty_energy = -2500.0\nmin_diversity = 0.2\nmin_solutions = 5\nh_values = [-1.0, 0.0, 1.0]\n```\n\n**Production Z(9,2) targets:**\n```python\ndifficulty_energy = -4100.0\nmin_diversity = 0.15\nmin_solutions = 5\n```\n\n**Energy ranges by topology (GSE):**\n\n| Topology | Range | Size |\n|---|---|---|\n| Z(8,2) | -2869 to -2677 | 192 |\n| Z(9,2) | -4100 to -3870 | 230 (default) |\n| Z(10,2) | -5470 to -5200 | 270 |\n| Z(11,4) | -15170 to -14158 | 1012 |\n| Advantage2 (full) | similar to Z(11,4) | ~4579 qubits |\n\n**QPU h/J ranges:** See [`docs/dwave-solver-ranges.md`](docs/dwave-solver-ranges.md)\nfor per-solver `h_range`, `j_range`, `extended_j_range`, and\n`per_qubit_coupling_range`. Regenerate with\n`python tools/dump_solver_ranges.py`.\n\n**Per-miner adaptive parameters:**\n\n| Backend | num_sweeps | num_reads |\n|---|---|---|\n| CPU/SA | 64–4096 | 64–512 |\n| GPU/CUDA | 256–2048 | (adaptive) |\n| GPU/Metal | 64–512 | (adaptive) |\n| Modal | 128–4096 | (adaptive) |\n| QPU | annealing 5–20µs | 32–64 |\n\n## Identifiers: solution number vs `dispatch_id`\n\nTwo distinct identifiers exist; **never conflate them**, and never persist\non `dispatch_id`.\n\n- **Solution number** — the chain-global *ordinal* of the QPoW solution\n  being mined: `count(QuantumPow.WinningSolutions) + 1`. The chain has no\n  stored solution counter — solutions are keyed in `WinningSolutions` by\n  the block number they won at (`submitted_at == key`), so the ordinal is\n  derived by counting that map's keys (cheap: one paged `state_getKeysPaged`\n  over the storage prefix, keys only). It is **determinable at round\n  start** — a round mines toward a specific upcoming solution number; the\n  round's `last_proof_block_hash` (= `block_hash(LastProofBlock)`) stays\n  constant until a proof wins and advances it. Compute the count **once per\n  round and cache it** (recount only when `last_proof_block_hash` changes);\n  it is global, monotonic, and never repeats.\n  - **Not the same as a block number.** `LastProofBlock` (e.g. 52507) is\n    the block number of the most recent winning proof — the round anchor,\n    not the solution ordinal (e.g. 196). Don't conflate them.\n  - **This is the on-disk key for the mining-attempts archive**\n    (`{base}/{solution_number}/…`). Because it tracks the logical\n    solution, a controller/worker restart mid-round correctly *resumes*\n    writing into the same solution dir — that is not stale accretion, it\n    is the same solution. Different solution number ⇒ different dir.\n\n- **`dispatch_id`** — an **internal-only** scheduler↔worker coordination\n  handle: `_dispatch_contexts[(handle_id, dispatch_id)]`, cancel-ack\n  (`_await_done_sentinel`), and the preview channel (see `ARCHITECTURE.md`\n  §3.3). It is a process-local monotonic counter that **resets to 0 on\n  restart**, so it collides across runs and MUST NOT be used as a durable\n  or on-disk identity. Keep it in memory for pairing responses to\n  contexts; never name persisted artifacts after it.\n\nThe old `SubmissionLogger` `next_solution_id` local counter was removed;\nthe on-disk archive is keyed by the chain-derived solution number.\n\n## Code style\n\n- **Imports** at top of file, in stdlib → third-party → local order. Absolute imports only. No inline imports inside functions/methods. Exception: optional-dependency `try/except` at module level.\n- **Concurrency**: NEVER use threads. Multiprocessing only. Async via `asyncio` for network operations. Mining runs in worker processes (`shared/miner_worker.py:MinerHandle`).\n- **Type hints** on public APIs. Google-style docstrings on non-trivial public functions.\n- **Logging** via `logging.getLogger(__name__)` at module level. The custom formatter and component classification live in `shared/logging_config.py`.\n- **No `Co-Authored-By: <assistant> ...` lines in commits.**\n\nSee [`ARCHITECTURE.md`](./ARCHITECTURE.md) §9 for the cleanup-candidates\nlist (vestigial code flagged for removal).\n\n## Deployment\n\n- **Docker** (`docker/`): `Dockerfile.cpu`, `Dockerfile.cuda`, `docker-compose.yml`, `entrypoint.sh`, plus the example TOML configs (`quip-miner.cpu.toml`, `quip-miner.cuda.toml`).\n- **Cloud**: `akash/` and `aws/` contain deployment configs.\n- **GitLab CI** (`.gitlab-ci.yml`): builds CPU + CUDA Docker images on main/tags.\n\n## Concurrency: processes, not threads\n\nUse `multiprocessing` for concurrency. Do **not** introduce `threading.Thread`\nin our own code. Threads share the GIL; a CPU-bound thread starves its\nsiblings. This stalled the QPU stream pump in v0.2 — the per-attempt consumer\nheld the GIL ~1.2 s and the pump thread could not drive the sampler, so the\nQPU pipeline drained (throughput 0.6 vs ~8 subs/sec).\n\nRules:\n- New background work → a `multiprocessing` process (use `spawn`).\n- Share scalars via `multiprocessing.Value`; share large numpy buffers via\n  `multiprocessing.shared_memory.SharedMemory` (zero-copy), never by pickling\n  per item on a hot path.\n- `threading.Lock`/`RLock` for intra-process state is fine (a lock is not a\n  thread). Cross-process correctness uses `os.replace`/PIPE_BUF atomicity or\n  `mp` primitives, not locks.\n- Exceptions we don't control: third-party internal threads (D-Wave SDK,\n  asyncio executors, stdlib `QueueListener` if ever reused). Document any such\n  exception inline with the reason.\n\n## Debugging a hung process (get a traceback)\n\n`SIGINT` (Ctrl-C) only helps when the **main thread is running Python\nbytecode** — it raises `KeyboardInterrupt` at the next bytecode boundary. A\nprocess wedged in a C-level call (a lock/`join`/`wait`, an `mp.Queue`\nfeeder-thread join at interpreter shutdown, a blocking syscall) won't unwind,\nso `SIGINT` just kills it with **no traceback**. Don't reach for it on a hang.\n\nUse **`faulthandler`** + **`SIGABRT`** — the repo convention (CI already runs\n`timeout --signal=ABRT … python -X faulthandler -m pytest`):\n\n```bash\n# 1. Start the process with faulthandler enabled (installs handlers for the\n#    fatal signals SIGSEGV/SIGFPE/SIGABRT/SIGBUS/SIGILL).\nPYTHONFAULTHANDLER=1 python tools/whatever.py …      # or: python -X faulthandler …\n\n# 2. When it hangs, dump every thread's Python (and C) stack to stderr:\nkill -ABRT <pid>          # SIGABRT -> faulthandler dumps, then the process aborts (exit 134)\n```\n\nThis prints the exact frame each thread is stuck in (e.g. `threading.py:wait`\n→ an unjoined queue feeder thread). Notes:\n\n- **Pre-arm it.** Faulthandler must be enabled *before* the hang. For tools we\n  expect to run interactively against the QPU/long pipelines, prefer enabling\n  it (env var or `faulthandler.enable()` at startup) so a hang is debuggable.\n- **Dump without killing:** `faulthandler.register(signal.SIGUSR1)` in code,\n  then `kill -USR1 <pid>` dumps and **continues** (repeatable). `SIGABRT` is\n  fatal; `SIGUSR1` (registered) is not.\n- **No instrumentation available?** `py-spy dump --pid <pid>` attaches to any\n  running CPython and prints all-thread tracebacks without a signal or restart.\n- **Inspect the tree first:** `ps -o pid,ppid,stat,command -p <pid>` and\n  `pgrep -P <pid>` reveal stuck children / orphans (`PPID 1` = reparented after\n  a parent crash). An interpreter that won't exit is usually blocked joining a\n  non-daemon child or an `mp.Queue` feeder thread (call `cancel_join_thread()`\n  on queues whose buffered data is worthless at teardown).\n","category":"root","tokens":4074}]}