{"owner":"microsoft","repo":"openvmm","hasSkills":true,"totalSkillsCount":5,"totalTokensCount":10660,"categories":["anthropic-skill","copilot-instructions"],"hasMcp":false,"mcpConfig":null,"found":[".github/skills/fuzzing/SKILL.md",".github/skills/guide-maintenance/SKILL.md",".github/skills/vmm-tests/SKILL.md",".github/copilot-instructions.md",".github/skills/openvmm-ci-investigation/SKILL.md"],"skills":{".github/skills/fuzzing/SKILL.md":"---\nname: fuzzing\ndescription: \"Run, optimize, and debug OpenVMM fuzzers. Covers cargo-fuzz targets, crash reproduction, lldb debugging, code coverage analysis, entropy optimization, and multi-target parallel campaigns.\"\n---\n\n# OpenVMM Fuzzer Guide\n\n## Prerequisites\n\n- **Linux only** — libfuzzer-sys doesn't support Windows.\n- **cargo-fuzz**: `cargo install cargo-fuzz`\n- **lldb** (for debugging): `sudo apt-get install -y lldb`\n- **aarch64 RUSTFLAGS**: On aarch64, set `RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\"` or builds fail with atomics errors. Not needed on x86_64.\n\n## Fuzz targets\n\nFuzz targets live in `/fuzz/` directories alongside the crate they test.\nEach has a `Cargo.toml` with `cargo-fuzz = true` metadata and a `fuzz_.rs`\nbinary. Find all targets:\n\n```bash\ncargo xtask fuzz list\n```\n\n## Running a fuzzer\n\n```bash\n# Continuous fuzzing (finds new crashes)\ncargo xtask fuzz run fuzz_ide\n\n# Reproduce a specific crash artifact\ncargo xtask fuzz run fuzz_ide path/to/crash-artifact\n```\n\nThe xtask wrapper sets `XTASK_FUZZ_REPRO=1` automatically when an artifact\npath is provided, which enables `init_tracing_if_repro()` in the fuzz target.\n\n## Build without running\n\n```bash\ncargo xtask fuzz build fuzz_ide\n```\n\nThe binary lands at `target//release/fuzz_ide` (e.g.,\n`target/aarch64-unknown-linux-gnu/release/fuzz_ide` or\n`target/x86_64-unknown-linux-gnu/release/fuzz_ide`).\n\n## Reproducing a crash (direct binary)\n\nRunning the binary directly is faster for iteration than going through xtask:\n\n```bash\n# Without tracing (fast, quiet)\n./target//release/fuzz_ide path/to/crash-artifact\n\n# With tracing (verbose — shows device state at each poll)\nXTASK_FUZZ_REPRO=1 ./target//release/fuzz_ide path/to/crash-artifact\n\n# With backtrace\nRUST_BACKTRACE=full ./target//release/fuzz_ide path/to/crash-artifact\n```\n\n## Debugging with lldb\n\n### Key lessons\n\n1. **Use batch mode with a command file** — lldb's interactive prompt doesn't\n work well from automated terminals. Write commands to a file and use\n `lldb -b -s `.\n2. **Set `auto-confirm true`** — prevents lldb from blocking on \"Do you really\n want to quit?\" prompts.\n3. **Breakpoint on `panic_fmt`** — `rust_begin_unwind` is often mangled and\n won't resolve. Use `-r panic_fmt` instead.\n4. **Drop `XTASK_FUZZ_REPRO`** when using lldb — the tracing output from\n repeated polls produces megabytes of spam that buries the debugger output.\n5. **Release builds have limited variable info** — `frame variable` may show\n nothing. For richer inspection, build with `--dev` (see below).\n\n### Command file template\n\nCreate a file (e.g., `scratch/lldb_fuzz.cmd`):\n\n```\nsettings set auto-confirm true\nbreakpoint set -r panic_fmt\nrun\nbt 20\nframe select 1\nsource list\nframe select 2\nsource list\nframe variable\nquit\n```\n\n### Run it\n\n```bash\nlldb -b -s scratch/lldb_fuzz.cmd \\\n -- ./target//release/fuzz_ide \\\n path/to/crash-artifact 2>&1 | tail -80\n```\n\nThe `| tail -80` trims libfuzzer startup noise. Adjust as needed.\n\n### Using rust-lldb\n\n`rust-lldb` adds Rust pretty-printers for `Vec`, `String`, etc. It requires\nlldb to be installed. Usage is the same but replace `lldb` with `rust-lldb`:\n\n```bash\nrust-lldb -b -s scratch/lldb_fuzz.cmd \\\n -- ./target//release/fuzz_ide \\\n path/to/crash-artifact 2>&1 | tail -80\n```\n\n### Debug builds for variable inspection\n\nRelease builds have limited `frame variable` output. For full variable\ninspection, build with `--dev`:\n\n```bash\ncargo xtask fuzz build fuzz_ide -- --dev\n```\n\nThe debug binary lands at `target//debug/fuzz_ide`. It's slower but\nlldb can inspect all locals and struct fields.\n\n### Navigating Rust wrapper types in lldb\n\nReaching the actual data through Arc, Mutex, UnsafeCell etc. requires\nknowing the field path. Use `type lookup` to discover struct layouts:\n\n```\n(lldb) type lookup closeable_mutex::CloseableMutex\n```\n\nCommon wrapper-type traversal patterns:\n\n| Rust type | lldb field path |\n|-----------|----------------|\n| `Arc` | `var.ptr.pointer->data` (through `ArcInner`) |\n| `CloseableMutex` | `.value.value` (UnsafeCell inside Mutex) |\n| `UnsafeCell` | `.value` |\n| `Option` | `.$variants$.$variant$.value.__0` (check `$discr$` — 0 = None for most types) |\n| Rust enum | `.$variants$.$variant$N.value.__0` (N = variant index) |\n\nFull path example for IDE device state through `Arc>`:\n\n```\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].state\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].enlightened_write\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].bus_master_state\n```\n\nFor drive registers inside `Option` containing an enum variant:\n\n```\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].drives[0].$variants$.$variant$.value.__0.$variants$.$variant$.value.__0.state.regs\n```\n\n### Setting breakpoint ignore counts\n\nTo skip the first N hits of a breakpoint (useful when a function is called\nhundreds of times during polling):\n\n```\n(lldb) breakpoint set -f lib.rs -l 1253\n(lldb) breakpoint modify -i 500 1\n(lldb) run\n```\n\nThis skips the first 500 hits, then stops on hit 501.\n\n### Reading Rust bitfield status registers\n\nStatus register values appear as `(__0 = 'X')` where `'X'` is the char at\nthat byte value. Decode manually: `0x58` = bits 3,4,6 set = `drq`, `dsc`,\n`drdy`.\n\n## Minimizing crash inputs\n\n```bash\ncargo xtask fuzz tmin fuzz_ide path/to/crash-artifact\n```\n\n## Corpus management\n\n```bash\n# Minimize the corpus (remove redundant inputs)\ncargo xtask fuzz cmin fuzz_ide\n```\n\nCorpus files live in `/fuzz/corpus//`.\nCrash artifacts land in `/fuzz/artifacts//`.\n\n## Fuzzer optimization principles\n\n### Entropy efficiency\nEvery byte of fuzzer input should drive meaningful behavior. Common wastes:\n- **Arbitrary setup parameters that don't affect coverage** — e.g., guest memory size, buffer counts. Use fixed values.\n- **`u.ratio()` calls for branching** — consumes entropy for each decision. Use `Arbitrary` enums instead (1 byte per decision vs 8+ for ratio).\n- **Redundant `arbitrary_data()` calls** — generate composite values with one call instead of field-by-field. E.g., a Guid from `[u8; 16]` instead of 11 separate calls.\n\n### Simplification\n- **Merge duplicated functions** that only differ in a small parameter. Use an enum variant to select behavior.\n- **Remove dead protocol negotiation** — if a device rejects all interesting operations without init, always negotiate. Raw packet paths can still exercise init state machines through mutation.\n- **Remove wrapper functions** that add no value over calling the underlying API directly.\n\n### Coverage gaps\n- **0-item/empty configurations** — fuzzers that always attach 1+ disks/devices miss the empty-controller code paths. Use `0..=N` ranges.\n- **Hardcoded setup parameters** can hide code paths. E.g., fixed namespace IDs mean only one NVMe identify-namespace path is tested. Make setup params fuzzer-driven where they affect behavior.\n- **TODOs mark legitimate gaps** — don't remove TODOs unless you've verified the marked code path is either (a) implemented or (b) genuinely doesn't affect coverage.\n\n### Performance\n- **Smaller backing stores** — RAM disks used for fuzzing don't need to be large. 32KB is sufficient for most device fuzzers (was 4MB).\n- **Fixed guest memory** — varying guest memory size wastes entropy without affecting device logic.\n- **Always negotiate protocols** — skipping negotiation wastes 100% of remaining fuzz actions on a single error branch.\n- **`-fork=N`** for parallel fuzzing across CPUs. Allocate cores proportionally to fuzzer complexity.\n\n## Multi-target parallel fuzzing\n\nRun multiple fuzzers across available CPUs for extended campaigns:\n\n```bash\n# 6-hour campaign across 112 cores, 7 fuzzers\n# Each runs with -fork=N, survives crashes/timeouts/OOMs\nnohup bash -c 'RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\" \\\n cargo xtask fuzz run fuzz_storvsp -- -- \\\n -fork=20 -max_total_time=21600 \\\n -ignore_crashes=1 -ignore_timeouts=1 -ignore_ooms=1 \\\n -print_final_stats=1' > /tmp/fuzz_storvsp.log 2>&1 &\n```\n\nAllocate cores based on fuzzer complexity:\n- Heavy fuzzers (storvsp, ide, nvme, nvram): 20 cores each\n- Medium fuzzers (cmos_rtc): 15 cores\n- Light fuzzers (diagnostics, scsi_buffers): 7–10 cores\n\nMonitor progress: `tail -1 /tmp/fuzz_*.log`\n\nAfter the run, collect coverage on each fuzzer for gap analysis.\n\n## Code coverage\n\n### Prerequisites\n\nEverything from the main prerequisites, plus:\n\n- **llvm-tools**: `rustup component add llvm-tools`\n- **lcov** (for HTML reports): `sudo apt-get install -y lcov`\n\n### Collecting coverage\n\nThe xtask has built-in coverage support. It runs each corpus entry through\na coverage-instrumented build and merges the results:\n\n```bash\n# Collect coverage data only\ncargo xtask fuzz coverage fuzz_ide\n\n# Collect + generate HTML report (requires lcov + genhtml)\ncargo xtask fuzz coverage fuzz_ide --with-html-report\n\n# Skip rebuild, just regenerate report from existing profdata\ncargo xtask fuzz coverage fuzz_ide --with-html-report --only-report\n```\n\nOn aarch64, set `RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\"` as usual.\n\nCoverage artifacts land in:\n\n| Artifact | Path |\n|----------|------|\n| profdata | `/fuzz/coverage//coverage.profdata` |\n| HTML report | `target//coverage//release/lcov_html_/index.html` |\n\n### Filtering coverage to relevant crates\n\nThe raw HTML report covers *all* compiled code including dependencies — the\noverall percentage is meaningless (typically 4–6%). What matters is coverage\nof the **target crate itself** and its immediate domain dependencies.\n\nUse `llvm-cov report` with source path filtering:\n\n```bash\n# Find the llvm-cov binary from the toolchain\nLLVM_COV=$(find $(rustc --print sysroot) -name \"llvm-cov\" -type f | head -1)\n\n# Full per-file report, excluding third-party code\n$LLVM_COV report \\\n -instr-profile /fuzz/coverage//coverage.profdata \\\n -object target//coverage//release/ \\\n --ignore-filename-regex='(rustc|\\.cargo|registry)' 2>&1 \\\n | grep 'vm/devices/storage/ide/src/'\n```\n\nFor lcov-based per-file analysis, export to lcov and extract per-file stats:\n\n```bash\n$LLVM_COV export \\\n -instr-profile /fuzz/coverage//coverage.profdata \\\n -object target//coverage//release/ \\\n --ignore-filename-regex='(rustc|\\.cargo|registry)' \\\n -format=lcov > /tmp/coverage.lcov\n\n# Per-file function/line summary from lcov\nawk '/SF:.*ide\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep -E '^(FNF|FNH|LF|LH):'\n# FNF/FNH = functions found/hit, LF/LH = lines found/hit\n\n# Find uncovered line numbers\nawk '/SF:.*ide\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep '^DA:' | grep ',0$' | cut -d: -f2 | cut -d, -f1\n```\n\n## Coverage scope rules\n\nEach fuzzer has a **primary scope** (the crate it exists to test) and\n**secondary scope** (dependencies it exercises incidentally). Coverage\nimprovements should focus on primary scope first.\n\n### What coverage matters per fuzzer\n\n| Fuzzer | Primary scope | Secondary scope | Out of scope |\n|--------|--------------|-----------------|--------------|\n| fuzz_ide | `ide/src/` | `scsidisk/src/scsidvd/`, `scsidisk/src/atapi_scsi.rs`, `disk_backend/`, `disklayer_ram/` | `pci_core/`, `vmcore/`, `guestmem/` (infrastructure) |\n| fuzz_storvsp | `storvsp/src/` | `scsidisk/src/lib.rs`, `vmbus_ring/`, `vmbus_async/`, `disk_backend/` | `vmbus_channel/`, `vmbus_core/` (infrastructure) |\n| fuzz_nvme_driver | `nvme_driver/src/` | `nvme/src/`, `nvme_spec/`, `user_driver/` | `page_pool_alloc/`, `vmcore/` |\n| fuzz_firmware_uefi_nvram | `firmware_uefi/src/service/nvram/` | `uefi_nvram_specvars/` | `crypto`, `openssl` (infrastructure) |\n| fuzz_firmware_uefi_diagnostics | `firmware_uefi/src/service/diagnostics/` | — | `guestmem/` |\n| fuzz_chipset_cmos_rtc | `chipset/src/cmos_rtc/` | `chipset_device_fuzz/` | `pal_async/`, `vmcore/` |\n| fuzz_scsi_buffers | `scsi_buffers/src/` | — | `guestmem/` |\n| fuzz_chipset_battery | `chipset/src/battery/` | `chipset_device_fuzz/` | — |\n\n**Rules:**\n\n1. **Primary scope is the goal.** Line coverage of the target crate is the\n metric to optimize. The fuzzer exists to find bugs in this code.\n\n2. **Secondary scope matters when it's a trust boundary.** `scsidisk` parsing\n guest-supplied SCSI commands is a real attack surface — coverage there has\n value. But `disklayer_ram` just storing bytes is infrastructure.\n\n3. **Out-of-scope code is noise.** `pci_core`, `vmcore`, `guestmem` get\n exercised incidentally. Don't optimize the fuzzer to improve their coverage\n — those crates deserve their own fuzzers if coverage matters.\n\n4. **Save/restore and resolver code is structurally unreachable.** Functions\n like `save_restore.rs`, `resolver.rs` won't be hit by fuzzing because\n the fuzzer doesn't exercise serialization or resource resolution paths.\n Don't count them against coverage targets.\n\n5. **Inspect (`InspectMut`) implementations are low-value.** These debug/\n diagnostics paths aren't part of the attack surface. Low coverage is\n acceptable.\n\n## Coverage-guided fuzzer improvement process\n\n### Step 1: Collect and filter\n\n```bash\n# Run coverage\ncargo xtask fuzz coverage --with-html-report\n\n# Open the HTML report for visual inspection\n# target//coverage//release/lcov_html_/index.html\n```\n\n### Step 2: Identify gaps in primary scope\n\nLook at the HTML report or extract uncovered lines:\n\n```bash\nawk '/SF:.*\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep '^DA:' | grep ',0$' | cut -d: -f2 | cut -d, -f1\n```\n\nRead the source at those lines. Classify each uncovered region:\n\n| Classification | Action |\n|---------------|--------|\n| **Reachable via fuzzer input** — the fuzzer *could* reach this code but hasn't found the right input sequence | Investigate why. Is the path gated by specific byte patterns the fuzzer struggles to generate? |\n| **Structurally unreachable** — the fuzzer harness doesn't wire up the code path (e.g., save/restore, PCI config reads the harness skips) | Extend the harness: add new `FuzzAction` variants, expose more device interfaces to the fuzzer. |\n| **Error-handling / defensive code** — `unreachable!()`, error paths for invalid hardware states | Low priority. These are worth testing but don't indicate a fuzzer deficiency. |\n| **Inspect / debug only** — `InspectMut`, `Display`, `Debug` impls | Skip entirely. Not attack surface. |\n\n### Step 3: Improve the fuzzer\n\nImprovements fall into two categories, in priority order:\n\n**A. Extend the fuzz harness**\n\nIf an entire code path is structurally unreachable, the harness needs changes.\nExamples:\n\n- fuzz_ide doesn't exercise PCI config space reads → add PCI config read/write\n to the `FuzzAction` enum in `chipset_device_fuzz`\n- fuzz_storvsp doesn't test sub-channel operations → add corresponding VMBus\n channel actions\n\n**B. Improve `Arbitrary` implementations**\n\nIf the fuzzer generates mostly one type of operation, check the `Arbitrary`\nimpl. Common issues:\n\n- Uniform distribution across action types wastes entropy on less interesting\n paths. Weight toward rarer operations.\n- Overly constrained generation (e.g., only valid SCSI commands) misses\n malformed-input bugs.\n- Overly unconstrained generation (random bytes everywhere) wastes time on\n inputs that fail early validation and never reach deep code.\n\n### Step 4: Measure improvement\n\nAfter making changes:\n\n1. Run the fuzzer for a fixed duration (e.g., 1 hour)\n2. Minimize the corpus: `cargo xtask fuzz cmin `\n3. Collect coverage again\n4. Compare line counts against prior coverage runs\n\nTrack coverage changes over time. Ratchet up: if a change reduces primary\nscope coverage, investigate why.\n\n### Step 5: Iterate\n\nCoverage-guided improvement is iterative. Each cycle:\n\n1. Collect → 2. Identify gaps → 3. Fix → 4. Fuzz → 5. Measure → repeat\n\nTarget **≥85% line coverage** in primary scope as a practical goal. The\nremaining ~15% is typically error-handling code, debug impls, and paths\nrequiring specific hardware/timing conditions the fuzzer can't easily produce.\n",".github/skills/guide-maintenance/SKILL.md":"---\nname: guide-maintenance\ndescription: >\n Maintain the OpenVMM Guide. Load when: (1) adding, removing, or moving Guide\n pages, (2) adding new device crates or CLI args that need doc coverage, or\n (3) auditing Guide freshness against code changes.\n---\n\n# Guide Maintenance Skill\n\nProcedures for keeping the OpenVMM Guide (`Guide/src/`) in sync with the codebase.\n\n`.github/instructions/doc-code-sync.instructions.md` contains the instructions\nand heuristics that are automatically loaded during Copilot code review on\n`*.rs` and `Cargo.toml` files.\n\n---\n\n## Adding a New Guide Page\n\n### 1. Choose the right location\n\n| Content type | Path pattern |\n|-------------|-------------|\n| Emulated device reference | `reference/emulated//.md` |\n| VMBus device reference | `reference/devices/vmbus/.md` |\n| Virtio device reference | `reference/devices/virtio/.md` |\n| Device backend reference | `reference/backends/.md` |\n| Architecture deep-dive | `reference/architecture/openvmm/` or `reference/architecture/openhcl/` |\n| CLI / management | `reference/openvmm/management/` |\n| Developer tool | `dev_guide/dev_tools/.md` |\n| Test framework | `dev_guide/tests/` |\n\n### 2. Create the page\n\n- Follow the style in `.github/instructions/guide-docs.instructions.md`\n- Include a brief overview, key concepts, and links to relevant rustdoc\n- Reference crate names in backticks\n- Use relative paths from repo root when pointing at code\n\n### 3. Update SUMMARY.md\n\nAdd the page to `Guide/src/SUMMARY.md` in the correct section. Pages with\ncontent get a path; placeholder topics for future work get an empty link `()`.\n\n```markdown\n- [Page Title](./reference/path/to/page.md)\n```\n\n---\n\n## Removing or Moving a Guide Page\n\n1. Update or remove the entry in `Guide/src/SUMMARY.md`\n2. Check for cross-references from other Guide pages (grep for the old path)\n3. If moving: ensure the SUMMARY.md link points to the new location\n\n---\n\n## Adding a New Device Crate\n\nWhen a new crate is added under `vm/devices/`:\n\n1. **Does it need a Guide page?** If it's a user-visible device or backend,\n yes. Internal plumbing crates (e.g., `*_resources`, `*_protocol`) usually\n don't need their own page but should be mentioned in the parent device page.\n\n2. **Create the page** following the \"Adding a New Guide Page\" procedure above.\n\n3. **Update the \"What to Flag\" list** if the new crate introduces a new\n category of reviewable change.\n\n---\n\n## Adding or Changing CLI Arguments\n\nWhen `openvmm/openvmm_entry/src/cli_args.rs` changes:\n\n1. Update `Guide/src/reference/openvmm/management/cli.md`\n2. The CLI page has a disclaimer that it may be out of date — keep it honest,\n but still update the page when making changes\n3. New arguments should include: flag syntax, description, default value,\n and any interactions with other flags\n\n---\n\n## Auditing Guide Freshness\n\nTo check whether the Guide is in sync with the code:\n\n1. **Scan the Guide pages** in `Guide/src/`\n2. Check whether the Guide page content still matches the code:\n - Crate names still correct?\n - Struct/enum names still exist?\n - Behavioral descriptions still accurate?\n - Links to rustdoc still resolve?\n3. Check `Guide/src/SUMMARY.md` for placeholder links `()` — these are\n topics that need content. Prioritize ones whose code areas have been\n actively developed.\n\n### Quick audit commands\n\n```bash\n# Find Guide pages that reference a crate name\ngrep -r \"crate_name\" Guide/src/\n\n# Find placeholder topics (empty links) in SUMMARY.md\ngrep '()\\s*$' Guide/src/SUMMARY.md\n```\n",".github/skills/vmm-tests/SKILL.md":"---\nname: vmm-tests\ndescription: \"Run VMM tests locally with cargo xflowey vmm-tests-run. Load when running or debugging VMM tests, or when you need to understand the petri test framework, artifact handling, or cross-compilation for VMM tests.\"\n---\n\n# Running VMM Tests\n\nVMM tests boot full virtual machines and validate behavior. They live in\n`vmm_tests/vmm_tests/tests/tests/` and use the `petri` test framework.\n\n**Always use `cargo xflowey vmm-tests-run`** — never raw `cargo nextest run -p\nvmm_tests`. The xflowey command handles artifact discovery, dependency\nbuilding, and test execution automatically.\n\n## Quick Start\n\n```bash\n# Run a specific test\ncargo xflowey vmm-tests-run --filter \"test(my_test_name)\"\n\n# Run all tests matching a prefix\ncargo xflowey vmm-tests-run --filter \"test(/^boot_/)\"\n\n# Run all tests (rarely needed locally)\ncargo xflowey vmm-tests-run --filter \"all()\"\n```\n\nUsually you don't need `--dir` — it defaults to `target/vmm_tests`. When\ncross-compiling for Windows from WSL2, `--dir` on the Windows filesystem (e.g.,\n`--dir /mnt/d/vmm_tests`) is required **only** when the selected tests use disk\nimages that need a Windows filesystem: **Hyper-V tests**, or tests using **VHDX\n/ dynamic VHD1** images. For everything else (streamed disks, or plain fixed\nVHD1 / VMGS / ISO files), the default WSL-side directory works. `vmm-tests-run`\ndetects this and errors with a clear message if `--dir` is required.\n\n## Filter Syntax\n\nFilters use [nextest filter expressions](https://nexte.st/docs/filtersets/):\n\n| Expression | Matches |\n|-----------|---------|\n| `test(foo)` | Tests with `foo` in the name |\n| `test(/^boot_/)` | Tests starting with `boot_` (regex) |\n| `test(foo) & !test(hyperv)` | `foo` tests excluding Hyper-V variants |\n| `all()` | Everything |\n\n## Platform Targeting\n\nBy default, tests build for the current host. Use `--target` for\ncross-compilation:\n\n```bash\n# Cross-compile and run Windows tests from WSL2\ncargo xflowey vmm-tests-run --target windows-x64\n```\n\n| Target | Description |\n|--------|-------------|\n| `windows-x64` | Windows x86_64 (Hyper-V / WHP) |\n| `windows-aarch64` | Windows ARM64 (Hyper-V / WHP) |\n| `linux-x64` | Linux x86_64 |\n\n**Windows from WSL2**: The default output directory works for most tests. Add\n`--dir` on the Windows filesystem (e.g., `--dir /mnt/d/vmm_tests`) only when the\nselected tests use disk images requiring a Windows filesystem (Hyper-V tests,\nor VHDX / dynamic VHD1 images); `vmm-tests-run` errors with a clear message when\nit's needed. Cross-compilation setup is required first — see\n`Guide/src/dev_guide/getting_started/cross_compile.md`.\n\n## Artifact Handling (Lazy Fetch)\n\nBy default, disk images (VHDs/ISOs) are streamed on demand via HTTP with local\nSQLite caching. This avoids multi-GB upfront downloads.\n\n- `--no-lazy-fetch` — download all images upfront instead of streaming\n- Lazy fetch is automatically disabled for Hyper-V tests (they need local files)\n- `--skip-vhd-prompt` — skip interactive VHD download prompts (useful for\n automation)\n\n## Viewing Logs\n\nTest output (petri logs, guest serial, etc.) is shown by default on failure.\nFor full OpenVMM tracing, set the `OPENVMM_LOG` environment variable:\n\n```bash\nOPENVMM_LOG=trace cargo xflowey vmm-tests-run --filter \"test(foo)\"\n```\n\n## Other Useful Flags\n\n| Flag | Purpose |\n|------|---------|\n| `--release` | Release build (default: debug) |\n| `--build-only` | Build without running |\n| `--verbose` | Verbose cargo output |\n| `--install-missing-deps` | Auto-install missing system dependencies |\n| `--custom-uefi-firmware ` | Use a custom UEFI firmware (MSVM.fd) |\n| `--custom-kernel ` | Use a custom kernel image |\n\nRun `cargo xflowey vmm-tests-run --help` for the full option list.\n\n## Common Pitfalls\n\n- **Don't use `cargo nextest run -p vmm_tests` directly** — artifacts won't\n be present and tests will fail with missing-artifact errors.\n- **Windows output dir from WSL** — when the selected tests need a Windows\n filesystem (Hyper-V tests, or VHDX / dynamic VHD1 images), `--dir` must be on\n `/mnt/c/` or `/mnt/d/`, not in the WSL filesystem. Other tests use the\n default.\n- **Hyper-V tests** — require Hyper-V Administrators group membership and\n disable lazy fetch automatically.\n- **CI failures** — use the `openvmm-ci-investigation` skill to diagnose\n failing VMM tests in CI, not this workflow.\n",".github/copilot-instructions.md":"# OpenVMM Repository\n\nOpenVMM is a modular, cross-platform Virtual Machine Monitor (VMM) written in Rust.\nThis repository is home to both OpenVMM and OpenHCL (a paravisor).\nDocumentation lives in `Guide/` and is published at https://openvmm.dev.\n\n## Build & Setup\n\nRestore required dependencies before building for the first time:\n```bash\ncargo xflowey restore-packages\n```\n\nThe project supports cross-compilation for `x86_64` and `aarch64`.\nOpenHCL can only be built from Linux (WSL2 counts).\nFor cross-compilation from WSL2 to Windows, see\n`Guide/src/dev_guide/getting_started/cross_compile.md` and source\n`. ./build_support/setup_windows_cross.sh`.\n\n## Git Commit Rules\n\n- **Never amend commits that have already been pushed without express\n permission from the user.** Make new commits instead. PRs are\n squash-merged, so a clean history is unnecessary.\n- Rebasing onto `main` to resolve conflicts is fine, but do not use\n `git commit --amend`, `git rebase -i`, or `git push --force` to clean\n up history on already-pushed commits.\n\n## Pre-Commit Checklist (MANDATORY)\n\n**You MUST run these commands before every `git commit` in this repo.\nDo NOT commit without completing all three steps.**\n\n1. `cargo clippy --all-targets -p ` — for each modified package.\n2. `cargo doc --no-deps -p ` — for each modified package.\n3. `cargo xtask fmt --fix` — fix formatting, headers, naming conventions.\n Run this **last** because fixing clippy/doc issues may introduce\n formatting changes that need to be cleaned up.\n\nIf `cargo xtask fmt --fix` still fails after auto-fixes, fix the remaining\nreported issues manually and re-run until it succeeds. Do not run individual\n`--pass` commands afterward.\n\n## Trust Boundaries & Safety\n\nBoth OpenVMM and OpenHCL process data from untrusted sources. Code must\n**never panic** on untrusted input.\n\n- **OpenVMM does not trust the guest** — code must not panic on any guest input\n- **OpenHCL does not trust the root** — code must not panic on any root input\n- **OpenHCL does not trust the VTL0 guest** — the attack surface is subtle and needs human review\n\n**Error handling across trust boundaries:**\n- Use `thiserror` for typed error enums at library/API boundaries and\n protocol-facing code\n- Use `anyhow` with `.context(\"...\")` for application-level plumbing and\n context propagation\n- Never `.unwrap()` or `.expect()` on data that crosses a trust boundary\n- For protocol/hardware enums where unknown values must round-trip without\n panicking, use the `open_enum!` macro instead of a normal Rust `enum`\n- Rate-limit trace events that can be triggered repeatedly by guest\n interactions — use `tracelimit::warn_ratelimited!` (or `error_ratelimited!`,\n `info_ratelimited!`) instead of bare `tracing::warn!` etc.\n\n**Other safety rules:**\n1. Avoid `unsafe` code\n2. Avoid taking new external dependencies, especially those that significantly increase binary size\n3. Several OpenHCL crates (e.g., `minimal_rt`, `openhcl_boot`, `sidecar`,\n `host_fdt_parser`) must support `no_std` builds. In particular,\n `minimal_rt` and `host_fdt_parser` are unconditionally `no_std`, and\n `openhcl_boot` and `sidecar` use `cfg_attr(minimal_rt, no_std, no_main)`.\n Do not introduce `std`-only dependencies or APIs into code that is\n compiled for the `minimal_rt` configuration in these crates\n4. Prefer `assert!` over `debug_assert!` for internal invariants — the\n performance cost is negligible in nearly all code, and catching invariant\n violations in release builds is more valuable. The project follows a\n \"fail fast\" philosophy: crash immediately on broken invariants rather\n than letting the process continue in an undefined state where bugs are\n harder to diagnose. (This does not apply to untrusted input — use error\n handling at trust boundaries, not assertions.)\n\n## Testing\n\nRun tests with cargo-nextest using the `agent` profile, which suppresses\noutput for passing tests and only shows slow/failing tests:\n```bash\ncargo nextest run --profile agent -p \n```\n\n**Do NOT pipe test output to `grep`, `tail`, or other filters.** The `agent`\nprofile already minimizes output. Piping hides failures and makes hangs\ninvisible.\n\nFor VMM test validation during development, use `cargo xflowey vmm-tests-run`:\n```bash\ncargo xflowey vmm-tests-run --filter \"test(my_test_name)\"\n```\nDo not pass `--dir` in most cases. When cross-compiling for Windows from WSL2,\n`--dir` on the Windows filesystem (e.g., `--dir /mnt/d/vmm_tests`) is required\nonly when the selected tests use disk images that need a Windows filesystem\n(Hyper-V tests, or VHDX / dynamic VHD1 images); `vmm-tests-run` detects this and\nerrors with a clear message when it's needed. For native host tests, and for\nWindows-from-WSL2 tests that stream disks or use fixed VHD1 / VMGS / ISO images,\nit is unnecessary.\n\nThis automatically discovers artifacts, builds dependencies, and runs tests.\nSee `Guide/src/dev_guide/tests/vmm.md` for details.\n\n- **Unit tests** — spread throughout crates in `#[cfg(test)]` blocks.\n Should be fast, isolated, and not require root/administrator access.\n Add `use test_with_tracing::test;` in test modules so that `tracing`\n is initialized and traces appear in test output.\n- **VMM tests** — integration tests in `vmm_tests/` using the petri\n framework (requires additional setup).\n- Mark tests requiring special setup with `#[ignore]`.\n- Update `Guide/` docs when adding features or changing behavior\n (see `.github/instructions/doc-code-sync.instructions.md` for the mapping)\n- **CI failures** — to investigate failing CI checks on a PR, load the\n `openvmm-ci-investigation` skill.\n\n## Rust Edition\n\nThis project uses the **Rust 2024 edition** (`edition = \"2024\"` in root\n`Cargo.toml`).\n\n## Common Pitfalls\n\n- **`guest_arch` not `target_arch`**: For guest-architecture-specific code,\n use `cfg(guest_arch = \"x86_64\")` — **not** `cfg(target_arch = \"x86_64\")`.\n The VMM can run guests of a different architecture than the host. Using\n `target_arch` will fail CI.\n- **Workspace dependencies**: All dependency versions are centralized in\n the root `Cargo.toml`. In crate `Cargo.toml` files, use\n `dep_name.workspace = true` — not inline version specifiers. Add new\n dependencies to `[workspace.dependencies]` in the root first.\n- **Pipeline YAMLs are auto-generated**: Files under `ci-flowey/` (e.g.,\n `openvmm-pr.yaml`) are generated by the flowey framework — do not\n hand-edit them. Run `cargo xflowey regen` to regenerate.\n- **flowey nodes**: Use `flowey::shell_cmd!` and `rt.sh` inside flowey\n nodes — not `xshell::cmd!` or `xshell::Shell::new`.\n\n## Autonomous Agent Inner Loop\n\nWhen running as a coding agent (GitHub Copilot coding agent or similar),\nfollow this validation loop **before pushing each commit**. This covers\nthe common early CI failures (including the fmt + clippy checks from job0)\nlocally, avoiding slow push-and-wait cycles.\n\n1. **Identify modified packages.** For each file you changed, find the\n crate's `Cargo.toml` and note the package name.\n2. **Check compilation:** `cargo check -p ` — fast type-check.\n3. **Clippy:** `cargo clippy --all-targets -p ` — lint.\n4. **Doc:** `cargo doc --no-deps -p ` — catch doc errors.\n5. **Unit tests:** `cargo nextest run --profile agent -p ` — run the crate's\n tests. If nextest is not installed, use `cargo test -p `.\n6. **Formatting:** `cargo xtask fmt --fix` — run last, since earlier\n fixes may introduce formatting changes.\n\nIf any step fails, fix the issue and re-run from that step. Do not push\nuntil all six steps pass.\n\n**Cost notes:** Steps 2–5 are scoped to the modified package (`-p`),\nso they are fast even in this large workspace. Step 6 runs workspace-wide\nbut is also fast. The full cycle typically takes under 2 minutes for\na single-crate change.\n",".github/skills/openvmm-ci-investigation/SKILL.md":"---\nname: openvmm-ci-investigation\ndescription: Investigate CI failures on OpenVMM PRs. Load when a PR has failing CI checks, you need to download and analyze test artifacts, or you need to diagnose build, fmt, clippy, or VMM test failures.\n---\n\n# Investigating CI Failures\n\nWhen a PR has failing CI checks, **always start by running the investigation\nscript**. Do not manually query the GitHub API or download artifacts by hand\n— the script handles all of that automatically.\n\n## Step 1: Run the Script\n\n```bash\n# Investigate the current branch's PR (no argument needed)\npython3 repo_support/investigate_ci.py\n\n# Investigate a specific PR by number\npython3 repo_support/investigate_ci.py 2946\n\n# Or by run ID directly\npython3 repo_support/investigate_ci.py 23017249697\n```\n\nThe script automatically:\n1. Resolves the PR to **every** failed CI run for its head commit. A commit\n can have multiple workflows (e.g. \"OpenVMM PR\" and \"OpenVMM Docs PR\"),\n and each failing one is investigated in turn — so don't assume a single\n run covers everything. Only the newest run of each workflow is used, so\n superseded runs are ignored; a `NOTE: is still in_progress`\n line means results are incomplete and more failures may appear later.\n2. Identifies failed jobs\n3. Downloads `*-unit-tests-junit-xml` artifacts and parses JUnit XML for\n unit test failures\n4. Downloads `*-vmm-tests-logs` artifacts if they exist\n5. Finds tests with `petri.failed` markers and extracts ERROR/WARN lines\n6. If no test or JUnit artifacts exist (build/fmt/clippy failure), shows the\n tail of the failed job's log\n\n## Step 2: Analyze the Script Output\n\nRead the script's output to identify:\n- Which tests failed (unit tests and/or VMM tests)\n- Error messages and root causes\n- Whether it's a build/fmt/clippy failure vs. a test failure\n\nWhen more than one run failed, the output is split by `# RUN n of m` banners.\nCheck **all** of them — a docs failure and a build failure are reported in\nseparate runs.\n\nThen use the information to diagnose the issue and suggest fixes.\n\n## Step 3: Diagnose from Logs and Code\n\n**Always try to diagnose the failure from CI logs, test code, and error\nmessages first.** Most failures can be understood without local reproduction.\nRead the relevant test source, trace the error through the code, and form\na hypothesis before considering local repro.\n\n## Step 4: Reproduce Locally (Only If Needed)\n\nIf you cannot diagnose the failure from logs alone, **ask the user** whether\nthey want to attempt local reproduction before proceeding. Do not\nautomatically start building or running tests on the user's machine.\n\nIf the user agrees, check whether the failing platform matches the host\narchitecture — you can only reproduce tests locally on the same arch. If\nit doesn't match, explain this to the user and continue diagnosing from\nCI logs and test code.\n\nTo reproduce locally, load the `vmm-tests` skill for instructions on\nrunning with `cargo xflowey vmm-tests-run`. Do **not** use\n`cargo nextest run -p vmm_tests` directly — it won't have the required\nartifacts.\n\n## Reference: Manual Commands\n\n> **Only use these if the script fails or you need to dig deeper into a\n> specific artifact.** In normal usage, the script above is sufficient.\n\nIf the script isn't available or you need more control, follow these steps:\n\n### 1. Find the failing run\n\n```bash\n# Get the run ID for a PR\ngh pr checks -R microsoft/openvmm\n# Or list runs for a specific commit\ngh run list -R microsoft/openvmm --commit \n```\n\n### 2. Identify the failing job\n\n```bash\ngh run view -R microsoft/openvmm --json jobs \\\n -q '[.jobs[] | select(.conclusion == \"failure\") | {name, databaseId}]'\n```\n\n### 3. Download test artifacts\n\n**Unit test results** are stored in JUnit XML artifacts named\n`{platform}-unit-tests-junit-xml`. Known platforms include:\n- `x64-linux`\n- `aarch64-linux`\n- `aarch64-linux-musl`\n\n```bash\n# Download unit test JUnit XML for a platform\ngh run download -R microsoft/openvmm \\\n -n aarch64-linux-unit-tests-junit-xml -D /tmp/junit-xml\n# Parse failures from the XML\npython3 -c \"\nimport xml.etree.ElementTree as ET, sys\nfor f in __import__('pathlib').Path(sys.argv[1]).rglob('*.xml'):\n for tc in ET.parse(f).iter('testcase'):\n fail = tc.find('failure')\n if fail is None:\n fail = tc.find('error')\n if fail is not None:\n print(f'FAIL: {tc.get(\\\"classname\\\",\\\"\\\")}::{tc.get(\\\"name\\\",\\\"\\\")}')\n print(f' {fail.get(\\\"message\\\",\\\"\\\")[:200]}')\n\" /tmp/junit-xml\n```\n\n**VMM test results** are stored in artifacts named `{platform}-vmm-tests-logs`.\nThe known platforms are:\n- `x64-windows-intel`\n- `x64-windows-intel-tdx`\n- `x64-windows-amd`\n- `x64-windows-amd-snp`\n- `x64-linux`\n- `aarch64-windows`\n\n```bash\n# Download a specific platform's test logs\ngh run download -R microsoft/openvmm \\\n -n x64-windows-amd-snp-vmm-tests-logs -D /tmp/test-logs\n```\n\n### 4. Find failed tests\n\nEach test gets its own directory inside the artifact. Look for `petri.failed`\nmarker files (passing tests have `petri.passed` instead):\n\n```bash\nfind /tmp/test-logs -name \"petri.failed\"\n```\n\nThe `petri.failed` file contains the error petri recorded for the test. The\ntest's real name is in `petri.test`, which petri writes before the test body\nruns — so a directory with `petri.test` but *no* result marker is a test that\nwas killed or crashed before it could report.\n\n### 5. Extract errors from petri.jsonl\n\nThe `petri.jsonl` file in each test directory is the primary structured log.\nEach line is a JSON object with fields: `timestamp`, `source`, `severity`,\n`message`. Filter for `ERROR` and `WARN` severity for a quick diagnosis:\n\n```bash\npython3 -c \"\nimport json, sys\nfor line in open(sys.argv[1]):\n try:\n e = json.loads(line.strip())\n if e.get('severity') in ('ERROR', 'WARN'):\n print(f'[{e[\\\"severity\\\"]}] {e.get(\\\"source\\\",\\\"?\\\")}: {e.get(\\\"message\\\",\\\"\\\").strip()[:200]}')\n except: pass\n\" /tmp/test-logs//petri.jsonl\n```\n\n## Artifact Contents\n\n### Unit test JUnit XML\n\nArtifacts named `{platform}-unit-tests-junit-xml` contain JUnit XML files\nwith `` elements. Failed tests have `` or ``\nchildren with `message` attributes describing the failure. These are the\nprimary artifacts for diagnosing unit test / cargo-nextest failures.\n\n### VMM test logs (petri)\n\nEach test directory contains:\n- `petri.jsonl` — Structured JSON Lines log **(primary file for investigation)**\n- `petri.log` — Plain text version of the test log\n- `petri.test` — The test's name, written before the test body runs\n- `petri.passed` or `petri.failed` — Pass/fail marker; a failure marker holds the error\n- `openhcl.log` — OpenHCL serial console output, if the test exercised OpenHCL\n- `hyperv.log` — Hyper-V event log, if the test exercises the Hyper-V backend\n- `openvmm.log` — OpenVMM serial console output, if the test exercises the OpenVMM backend\n- `guest.log`, `uefi.log` — Guest OS serial output\n- Sometimes: `screenshot_*.png` — periodic screenshots of the guest\n- Sometimes: `dumpfile.dmp`\n\n## Viewing Results in Browser\n\nTest results are uploaded to Azure Blob Storage and viewable at:\n`https://openvmm.dev/test-results/#/runs/_`\n\nLogs are uploaded per re-run attempt, so the run key includes the attempt\nnumber (`1` for the first attempt). `investigate_ci.py` prints the correct\nURL in its summary.\n\n## Common Failure Patterns\n\n- **Unit test failure**: A `unit tests` job failed. The script downloads\n JUnit XML artifacts and shows the failing test names and messages. Common\n causes: new test code that relies on OS capabilities not available in CI\n (e.g. TAP devices, elevated permissions).\n- **Formatting / house-rules**: The `quick check [fmt, clippy x64-linux]` job\n failed. No test artifacts will exist. Run `cargo xtask fmt --fix` locally\n and check the job log for the specific rule that failed.\n- **Docs failure**: A job in the separate \"OpenVMM Docs PR\" workflow failed —\n either `build mdbook guide` (broken Guide markdown/links; mdbook logs\n `[ERROR] (mdbook::...)`) or `build and check docs` (rustdoc warnings, which\n are denied in CI). Reproduce with `cargo doc --no-deps -p `.\n No test artifacts will exist.\n- **TripleFault**: VM hit a fatal error during boot. Check `petri.jsonl` for\n Hyper-V Worker/Chipset errors. Often infrastructure-related, not caused by\n the PR's code changes.\n- **Timeout**: Test exceeded its time limit. Check if the VM booted at all.\n- **Guest assertion failure**: Guest-side test failed. Check `guest.log`.\n- **Build failure**: No test artifacts will exist. Check the job log directly\n with `gh run view --job --log`.\n\n## Important API Note\n\nThe `gh run view --json artifacts` flag does **not** exist. To list artifacts\nfor a run, use the GitHub API directly:\n\n```bash\ngh api repos/microsoft/openvmm/actions/runs//artifacts\n```\n"},"files":{".github/skills/fuzzing/SKILL.md":"---\nname: fuzzing\ndescription: \"Run, optimize, and debug OpenVMM fuzzers. Covers cargo-fuzz targets, crash reproduction, lldb debugging, code coverage analysis, entropy optimization, and multi-target parallel campaigns.\"\n---\n\n# OpenVMM Fuzzer Guide\n\n## Prerequisites\n\n- **Linux only** — libfuzzer-sys doesn't support Windows.\n- **cargo-fuzz**: `cargo install cargo-fuzz`\n- **lldb** (for debugging): `sudo apt-get install -y lldb`\n- **aarch64 RUSTFLAGS**: On aarch64, set `RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\"` or builds fail with atomics errors. Not needed on x86_64.\n\n## Fuzz targets\n\nFuzz targets live in `/fuzz/` directories alongside the crate they test.\nEach has a `Cargo.toml` with `cargo-fuzz = true` metadata and a `fuzz_.rs`\nbinary. Find all targets:\n\n```bash\ncargo xtask fuzz list\n```\n\n## Running a fuzzer\n\n```bash\n# Continuous fuzzing (finds new crashes)\ncargo xtask fuzz run fuzz_ide\n\n# Reproduce a specific crash artifact\ncargo xtask fuzz run fuzz_ide path/to/crash-artifact\n```\n\nThe xtask wrapper sets `XTASK_FUZZ_REPRO=1` automatically when an artifact\npath is provided, which enables `init_tracing_if_repro()` in the fuzz target.\n\n## Build without running\n\n```bash\ncargo xtask fuzz build fuzz_ide\n```\n\nThe binary lands at `target//release/fuzz_ide` (e.g.,\n`target/aarch64-unknown-linux-gnu/release/fuzz_ide` or\n`target/x86_64-unknown-linux-gnu/release/fuzz_ide`).\n\n## Reproducing a crash (direct binary)\n\nRunning the binary directly is faster for iteration than going through xtask:\n\n```bash\n# Without tracing (fast, quiet)\n./target//release/fuzz_ide path/to/crash-artifact\n\n# With tracing (verbose — shows device state at each poll)\nXTASK_FUZZ_REPRO=1 ./target//release/fuzz_ide path/to/crash-artifact\n\n# With backtrace\nRUST_BACKTRACE=full ./target//release/fuzz_ide path/to/crash-artifact\n```\n\n## Debugging with lldb\n\n### Key lessons\n\n1. **Use batch mode with a command file** — lldb's interactive prompt doesn't\n work well from automated terminals. Write commands to a file and use\n `lldb -b -s `.\n2. **Set `auto-confirm true`** — prevents lldb from blocking on \"Do you really\n want to quit?\" prompts.\n3. **Breakpoint on `panic_fmt`** — `rust_begin_unwind` is often mangled and\n won't resolve. Use `-r panic_fmt` instead.\n4. **Drop `XTASK_FUZZ_REPRO`** when using lldb — the tracing output from\n repeated polls produces megabytes of spam that buries the debugger output.\n5. **Release builds have limited variable info** — `frame variable` may show\n nothing. For richer inspection, build with `--dev` (see below).\n\n### Command file template\n\nCreate a file (e.g., `scratch/lldb_fuzz.cmd`):\n\n```\nsettings set auto-confirm true\nbreakpoint set -r panic_fmt\nrun\nbt 20\nframe select 1\nsource list\nframe select 2\nsource list\nframe variable\nquit\n```\n\n### Run it\n\n```bash\nlldb -b -s scratch/lldb_fuzz.cmd \\\n -- ./target//release/fuzz_ide \\\n path/to/crash-artifact 2>&1 | tail -80\n```\n\nThe `| tail -80` trims libfuzzer startup noise. Adjust as needed.\n\n### Using rust-lldb\n\n`rust-lldb` adds Rust pretty-printers for `Vec`, `String`, etc. It requires\nlldb to be installed. Usage is the same but replace `lldb` with `rust-lldb`:\n\n```bash\nrust-lldb -b -s scratch/lldb_fuzz.cmd \\\n -- ./target//release/fuzz_ide \\\n path/to/crash-artifact 2>&1 | tail -80\n```\n\n### Debug builds for variable inspection\n\nRelease builds have limited `frame variable` output. For full variable\ninspection, build with `--dev`:\n\n```bash\ncargo xtask fuzz build fuzz_ide -- --dev\n```\n\nThe debug binary lands at `target//debug/fuzz_ide`. It's slower but\nlldb can inspect all locals and struct fields.\n\n### Navigating Rust wrapper types in lldb\n\nReaching the actual data through Arc, Mutex, UnsafeCell etc. requires\nknowing the field path. Use `type lookup` to discover struct layouts:\n\n```\n(lldb) type lookup closeable_mutex::CloseableMutex\n```\n\nCommon wrapper-type traversal patterns:\n\n| Rust type | lldb field path |\n|-----------|----------------|\n| `Arc` | `var.ptr.pointer->data` (through `ArcInner`) |\n| `CloseableMutex` | `.value.value` (UnsafeCell inside Mutex) |\n| `UnsafeCell` | `.value` |\n| `Option` | `.$variants$.$variant$.value.__0` (check `$discr$` — 0 = None for most types) |\n| Rust enum | `.$variants$.$variant$N.value.__0` (N = variant index) |\n\nFull path example for IDE device state through `Arc>`:\n\n```\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].state\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].enlightened_write\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].bus_master_state\n```\n\nFor drive registers inside `Option` containing an enum variant:\n\n```\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].drives[0].$variants$.$variant$.value.__0.$variants$.$variant$.value.__0.state.regs\n```\n\n### Setting breakpoint ignore counts\n\nTo skip the first N hits of a breakpoint (useful when a function is called\nhundreds of times during polling):\n\n```\n(lldb) breakpoint set -f lib.rs -l 1253\n(lldb) breakpoint modify -i 500 1\n(lldb) run\n```\n\nThis skips the first 500 hits, then stops on hit 501.\n\n### Reading Rust bitfield status registers\n\nStatus register values appear as `(__0 = 'X')` where `'X'` is the char at\nthat byte value. Decode manually: `0x58` = bits 3,4,6 set = `drq`, `dsc`,\n`drdy`.\n\n## Minimizing crash inputs\n\n```bash\ncargo xtask fuzz tmin fuzz_ide path/to/crash-artifact\n```\n\n## Corpus management\n\n```bash\n# Minimize the corpus (remove redundant inputs)\ncargo xtask fuzz cmin fuzz_ide\n```\n\nCorpus files live in `/fuzz/corpus//`.\nCrash artifacts land in `/fuzz/artifacts//`.\n\n## Fuzzer optimization principles\n\n### Entropy efficiency\nEvery byte of fuzzer input should drive meaningful behavior. Common wastes:\n- **Arbitrary setup parameters that don't affect coverage** — e.g., guest memory size, buffer counts. Use fixed values.\n- **`u.ratio()` calls for branching** — consumes entropy for each decision. Use `Arbitrary` enums instead (1 byte per decision vs 8+ for ratio).\n- **Redundant `arbitrary_data()` calls** — generate composite values with one call instead of field-by-field. E.g., a Guid from `[u8; 16]` instead of 11 separate calls.\n\n### Simplification\n- **Merge duplicated functions** that only differ in a small parameter. Use an enum variant to select behavior.\n- **Remove dead protocol negotiation** — if a device rejects all interesting operations without init, always negotiate. Raw packet paths can still exercise init state machines through mutation.\n- **Remove wrapper functions** that add no value over calling the underlying API directly.\n\n### Coverage gaps\n- **0-item/empty configurations** — fuzzers that always attach 1+ disks/devices miss the empty-controller code paths. Use `0..=N` ranges.\n- **Hardcoded setup parameters** can hide code paths. E.g., fixed namespace IDs mean only one NVMe identify-namespace path is tested. Make setup params fuzzer-driven where they affect behavior.\n- **TODOs mark legitimate gaps** — don't remove TODOs unless you've verified the marked code path is either (a) implemented or (b) genuinely doesn't affect coverage.\n\n### Performance\n- **Smaller backing stores** — RAM disks used for fuzzing don't need to be large. 32KB is sufficient for most device fuzzers (was 4MB).\n- **Fixed guest memory** — varying guest memory size wastes entropy without affecting device logic.\n- **Always negotiate protocols** — skipping negotiation wastes 100% of remaining fuzz actions on a single error branch.\n- **`-fork=N`** for parallel fuzzing across CPUs. Allocate cores proportionally to fuzzer complexity.\n\n## Multi-target parallel fuzzing\n\nRun multiple fuzzers across available CPUs for extended campaigns:\n\n```bash\n# 6-hour campaign across 112 cores, 7 fuzzers\n# Each runs with -fork=N, survives crashes/timeouts/OOMs\nnohup bash -c 'RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\" \\\n cargo xtask fuzz run fuzz_storvsp -- -- \\\n -fork=20 -max_total_time=21600 \\\n -ignore_crashes=1 -ignore_timeouts=1 -ignore_ooms=1 \\\n -print_final_stats=1' > /tmp/fuzz_storvsp.log 2>&1 &\n```\n\nAllocate cores based on fuzzer complexity:\n- Heavy fuzzers (storvsp, ide, nvme, nvram): 20 cores each\n- Medium fuzzers (cmos_rtc): 15 cores\n- Light fuzzers (diagnostics, scsi_buffers): 7–10 cores\n\nMonitor progress: `tail -1 /tmp/fuzz_*.log`\n\nAfter the run, collect coverage on each fuzzer for gap analysis.\n\n## Code coverage\n\n### Prerequisites\n\nEverything from the main prerequisites, plus:\n\n- **llvm-tools**: `rustup component add llvm-tools`\n- **lcov** (for HTML reports): `sudo apt-get install -y lcov`\n\n### Collecting coverage\n\nThe xtask has built-in coverage support. It runs each corpus entry through\na coverage-instrumented build and merges the results:\n\n```bash\n# Collect coverage data only\ncargo xtask fuzz coverage fuzz_ide\n\n# Collect + generate HTML report (requires lcov + genhtml)\ncargo xtask fuzz coverage fuzz_ide --with-html-report\n\n# Skip rebuild, just regenerate report from existing profdata\ncargo xtask fuzz coverage fuzz_ide --with-html-report --only-report\n```\n\nOn aarch64, set `RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\"` as usual.\n\nCoverage artifacts land in:\n\n| Artifact | Path |\n|----------|------|\n| profdata | `/fuzz/coverage//coverage.profdata` |\n| HTML report | `target//coverage//release/lcov_html_/index.html` |\n\n### Filtering coverage to relevant crates\n\nThe raw HTML report covers *all* compiled code including dependencies — the\noverall percentage is meaningless (typically 4–6%). What matters is coverage\nof the **target crate itself** and its immediate domain dependencies.\n\nUse `llvm-cov report` with source path filtering:\n\n```bash\n# Find the llvm-cov binary from the toolchain\nLLVM_COV=$(find $(rustc --print sysroot) -name \"llvm-cov\" -type f | head -1)\n\n# Full per-file report, excluding third-party code\n$LLVM_COV report \\\n -instr-profile /fuzz/coverage//coverage.profdata \\\n -object target//coverage//release/ \\\n --ignore-filename-regex='(rustc|\\.cargo|registry)' 2>&1 \\\n | grep 'vm/devices/storage/ide/src/'\n```\n\nFor lcov-based per-file analysis, export to lcov and extract per-file stats:\n\n```bash\n$LLVM_COV export \\\n -instr-profile /fuzz/coverage//coverage.profdata \\\n -object target//coverage//release/ \\\n --ignore-filename-regex='(rustc|\\.cargo|registry)' \\\n -format=lcov > /tmp/coverage.lcov\n\n# Per-file function/line summary from lcov\nawk '/SF:.*ide\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep -E '^(FNF|FNH|LF|LH):'\n# FNF/FNH = functions found/hit, LF/LH = lines found/hit\n\n# Find uncovered line numbers\nawk '/SF:.*ide\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep '^DA:' | grep ',0$' | cut -d: -f2 | cut -d, -f1\n```\n\n## Coverage scope rules\n\nEach fuzzer has a **primary scope** (the crate it exists to test) and\n**secondary scope** (dependencies it exercises incidentally). Coverage\nimprovements should focus on primary scope first.\n\n### What coverage matters per fuzzer\n\n| Fuzzer | Primary scope | Secondary scope | Out of scope |\n|--------|--------------|-----------------|--------------|\n| fuzz_ide | `ide/src/` | `scsidisk/src/scsidvd/`, `scsidisk/src/atapi_scsi.rs`, `disk_backend/`, `disklayer_ram/` | `pci_core/`, `vmcore/`, `guestmem/` (infrastructure) |\n| fuzz_storvsp | `storvsp/src/` | `scsidisk/src/lib.rs`, `vmbus_ring/`, `vmbus_async/`, `disk_backend/` | `vmbus_channel/`, `vmbus_core/` (infrastructure) |\n| fuzz_nvme_driver | `nvme_driver/src/` | `nvme/src/`, `nvme_spec/`, `user_driver/` | `page_pool_alloc/`, `vmcore/` |\n| fuzz_firmware_uefi_nvram | `firmware_uefi/src/service/nvram/` | `uefi_nvram_specvars/` | `crypto`, `openssl` (infrastructure) |\n| fuzz_firmware_uefi_diagnostics | `firmware_uefi/src/service/diagnostics/` | — | `guestmem/` |\n| fuzz_chipset_cmos_rtc | `chipset/src/cmos_rtc/` | `chipset_device_fuzz/` | `pal_async/`, `vmcore/` |\n| fuzz_scsi_buffers | `scsi_buffers/src/` | — | `guestmem/` |\n| fuzz_chipset_battery | `chipset/src/battery/` | `chipset_device_fuzz/` | — |\n\n**Rules:**\n\n1. **Primary scope is the goal.** Line coverage of the target crate is the\n metric to optimize. The fuzzer exists to find bugs in this code.\n\n2. **Secondary scope matters when it's a trust boundary.** `scsidisk` parsing\n guest-supplied SCSI commands is a real attack surface — coverage there has\n value. But `disklayer_ram` just storing bytes is infrastructure.\n\n3. **Out-of-scope code is noise.** `pci_core`, `vmcore`, `guestmem` get\n exercised incidentally. Don't optimize the fuzzer to improve their coverage\n — those crates deserve their own fuzzers if coverage matters.\n\n4. **Save/restore and resolver code is structurally unreachable.** Functions\n like `save_restore.rs`, `resolver.rs` won't be hit by fuzzing because\n the fuzzer doesn't exercise serialization or resource resolution paths.\n Don't count them against coverage targets.\n\n5. **Inspect (`InspectMut`) implementations are low-value.** These debug/\n diagnostics paths aren't part of the attack surface. Low coverage is\n acceptable.\n\n## Coverage-guided fuzzer improvement process\n\n### Step 1: Collect and filter\n\n```bash\n# Run coverage\ncargo xtask fuzz coverage --with-html-report\n\n# Open the HTML report for visual inspection\n# target//coverage//release/lcov_html_/index.html\n```\n\n### Step 2: Identify gaps in primary scope\n\nLook at the HTML report or extract uncovered lines:\n\n```bash\nawk '/SF:.*\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep '^DA:' | grep ',0$' | cut -d: -f2 | cut -d, -f1\n```\n\nRead the source at those lines. Classify each uncovered region:\n\n| Classification | Action |\n|---------------|--------|\n| **Reachable via fuzzer input** — the fuzzer *could* reach this code but hasn't found the right input sequence | Investigate why. Is the path gated by specific byte patterns the fuzzer struggles to generate? |\n| **Structurally unreachable** — the fuzzer harness doesn't wire up the code path (e.g., save/restore, PCI config reads the harness skips) | Extend the harness: add new `FuzzAction` variants, expose more device interfaces to the fuzzer. |\n| **Error-handling / defensive code** — `unreachable!()`, error paths for invalid hardware states | Low priority. These are worth testing but don't indicate a fuzzer deficiency. |\n| **Inspect / debug only** — `InspectMut`, `Display`, `Debug` impls | Skip entirely. Not attack surface. |\n\n### Step 3: Improve the fuzzer\n\nImprovements fall into two categories, in priority order:\n\n**A. Extend the fuzz harness**\n\nIf an entire code path is structurally unreachable, the harness needs changes.\nExamples:\n\n- fuzz_ide doesn't exercise PCI config space reads → add PCI config read/write\n to the `FuzzAction` enum in `chipset_device_fuzz`\n- fuzz_storvsp doesn't test sub-channel operations → add corresponding VMBus\n channel actions\n\n**B. Improve `Arbitrary` implementations**\n\nIf the fuzzer generates mostly one type of operation, check the `Arbitrary`\nimpl. Common issues:\n\n- Uniform distribution across action types wastes entropy on less interesting\n paths. Weight toward rarer operations.\n- Overly constrained generation (e.g., only valid SCSI commands) misses\n malformed-input bugs.\n- Overly unconstrained generation (random bytes everywhere) wastes time on\n inputs that fail early validation and never reach deep code.\n\n### Step 4: Measure improvement\n\nAfter making changes:\n\n1. Run the fuzzer for a fixed duration (e.g., 1 hour)\n2. Minimize the corpus: `cargo xtask fuzz cmin `\n3. Collect coverage again\n4. Compare line counts against prior coverage runs\n\nTrack coverage changes over time. Ratchet up: if a change reduces primary\nscope coverage, investigate why.\n\n### Step 5: Iterate\n\nCoverage-guided improvement is iterative. Each cycle:\n\n1. Collect → 2. Identify gaps → 3. Fix → 4. Fuzz → 5. Measure → repeat\n\nTarget **≥85% line coverage** in primary scope as a practical goal. The\nremaining ~15% is typically error-handling code, debug impls, and paths\nrequiring specific hardware/timing conditions the fuzzer can't easily produce.\n",".github/skills/guide-maintenance/SKILL.md":"---\nname: guide-maintenance\ndescription: >\n Maintain the OpenVMM Guide. Load when: (1) adding, removing, or moving Guide\n pages, (2) adding new device crates or CLI args that need doc coverage, or\n (3) auditing Guide freshness against code changes.\n---\n\n# Guide Maintenance Skill\n\nProcedures for keeping the OpenVMM Guide (`Guide/src/`) in sync with the codebase.\n\n`.github/instructions/doc-code-sync.instructions.md` contains the instructions\nand heuristics that are automatically loaded during Copilot code review on\n`*.rs` and `Cargo.toml` files.\n\n---\n\n## Adding a New Guide Page\n\n### 1. Choose the right location\n\n| Content type | Path pattern |\n|-------------|-------------|\n| Emulated device reference | `reference/emulated//.md` |\n| VMBus device reference | `reference/devices/vmbus/.md` |\n| Virtio device reference | `reference/devices/virtio/.md` |\n| Device backend reference | `reference/backends/.md` |\n| Architecture deep-dive | `reference/architecture/openvmm/` or `reference/architecture/openhcl/` |\n| CLI / management | `reference/openvmm/management/` |\n| Developer tool | `dev_guide/dev_tools/.md` |\n| Test framework | `dev_guide/tests/` |\n\n### 2. Create the page\n\n- Follow the style in `.github/instructions/guide-docs.instructions.md`\n- Include a brief overview, key concepts, and links to relevant rustdoc\n- Reference crate names in backticks\n- Use relative paths from repo root when pointing at code\n\n### 3. Update SUMMARY.md\n\nAdd the page to `Guide/src/SUMMARY.md` in the correct section. Pages with\ncontent get a path; placeholder topics for future work get an empty link `()`.\n\n```markdown\n- [Page Title](./reference/path/to/page.md)\n```\n\n---\n\n## Removing or Moving a Guide Page\n\n1. Update or remove the entry in `Guide/src/SUMMARY.md`\n2. Check for cross-references from other Guide pages (grep for the old path)\n3. If moving: ensure the SUMMARY.md link points to the new location\n\n---\n\n## Adding a New Device Crate\n\nWhen a new crate is added under `vm/devices/`:\n\n1. **Does it need a Guide page?** If it's a user-visible device or backend,\n yes. Internal plumbing crates (e.g., `*_resources`, `*_protocol`) usually\n don't need their own page but should be mentioned in the parent device page.\n\n2. **Create the page** following the \"Adding a New Guide Page\" procedure above.\n\n3. **Update the \"What to Flag\" list** if the new crate introduces a new\n category of reviewable change.\n\n---\n\n## Adding or Changing CLI Arguments\n\nWhen `openvmm/openvmm_entry/src/cli_args.rs` changes:\n\n1. Update `Guide/src/reference/openvmm/management/cli.md`\n2. The CLI page has a disclaimer that it may be out of date — keep it honest,\n but still update the page when making changes\n3. New arguments should include: flag syntax, description, default value,\n and any interactions with other flags\n\n---\n\n## Auditing Guide Freshness\n\nTo check whether the Guide is in sync with the code:\n\n1. **Scan the Guide pages** in `Guide/src/`\n2. Check whether the Guide page content still matches the code:\n - Crate names still correct?\n - Struct/enum names still exist?\n - Behavioral descriptions still accurate?\n - Links to rustdoc still resolve?\n3. Check `Guide/src/SUMMARY.md` for placeholder links `()` — these are\n topics that need content. Prioritize ones whose code areas have been\n actively developed.\n\n### Quick audit commands\n\n```bash\n# Find Guide pages that reference a crate name\ngrep -r \"crate_name\" Guide/src/\n\n# Find placeholder topics (empty links) in SUMMARY.md\ngrep '()\\s*$' Guide/src/SUMMARY.md\n```\n",".github/skills/vmm-tests/SKILL.md":"---\nname: vmm-tests\ndescription: \"Run VMM tests locally with cargo xflowey vmm-tests-run. Load when running or debugging VMM tests, or when you need to understand the petri test framework, artifact handling, or cross-compilation for VMM tests.\"\n---\n\n# Running VMM Tests\n\nVMM tests boot full virtual machines and validate behavior. They live in\n`vmm_tests/vmm_tests/tests/tests/` and use the `petri` test framework.\n\n**Always use `cargo xflowey vmm-tests-run`** — never raw `cargo nextest run -p\nvmm_tests`. The xflowey command handles artifact discovery, dependency\nbuilding, and test execution automatically.\n\n## Quick Start\n\n```bash\n# Run a specific test\ncargo xflowey vmm-tests-run --filter \"test(my_test_name)\"\n\n# Run all tests matching a prefix\ncargo xflowey vmm-tests-run --filter \"test(/^boot_/)\"\n\n# Run all tests (rarely needed locally)\ncargo xflowey vmm-tests-run --filter \"all()\"\n```\n\nUsually you don't need `--dir` — it defaults to `target/vmm_tests`. When\ncross-compiling for Windows from WSL2, `--dir` on the Windows filesystem (e.g.,\n`--dir /mnt/d/vmm_tests`) is required **only** when the selected tests use disk\nimages that need a Windows filesystem: **Hyper-V tests**, or tests using **VHDX\n/ dynamic VHD1** images. For everything else (streamed disks, or plain fixed\nVHD1 / VMGS / ISO files), the default WSL-side directory works. `vmm-tests-run`\ndetects this and errors with a clear message if `--dir` is required.\n\n## Filter Syntax\n\nFilters use [nextest filter expressions](https://nexte.st/docs/filtersets/):\n\n| Expression | Matches |\n|-----------|---------|\n| `test(foo)` | Tests with `foo` in the name |\n| `test(/^boot_/)` | Tests starting with `boot_` (regex) |\n| `test(foo) & !test(hyperv)` | `foo` tests excluding Hyper-V variants |\n| `all()` | Everything |\n\n## Platform Targeting\n\nBy default, tests build for the current host. Use `--target` for\ncross-compilation:\n\n```bash\n# Cross-compile and run Windows tests from WSL2\ncargo xflowey vmm-tests-run --target windows-x64\n```\n\n| Target | Description |\n|--------|-------------|\n| `windows-x64` | Windows x86_64 (Hyper-V / WHP) |\n| `windows-aarch64` | Windows ARM64 (Hyper-V / WHP) |\n| `linux-x64` | Linux x86_64 |\n\n**Windows from WSL2**: The default output directory works for most tests. Add\n`--dir` on the Windows filesystem (e.g., `--dir /mnt/d/vmm_tests`) only when the\nselected tests use disk images requiring a Windows filesystem (Hyper-V tests,\nor VHDX / dynamic VHD1 images); `vmm-tests-run` errors with a clear message when\nit's needed. Cross-compilation setup is required first — see\n`Guide/src/dev_guide/getting_started/cross_compile.md`.\n\n## Artifact Handling (Lazy Fetch)\n\nBy default, disk images (VHDs/ISOs) are streamed on demand via HTTP with local\nSQLite caching. This avoids multi-GB upfront downloads.\n\n- `--no-lazy-fetch` — download all images upfront instead of streaming\n- Lazy fetch is automatically disabled for Hyper-V tests (they need local files)\n- `--skip-vhd-prompt` — skip interactive VHD download prompts (useful for\n automation)\n\n## Viewing Logs\n\nTest output (petri logs, guest serial, etc.) is shown by default on failure.\nFor full OpenVMM tracing, set the `OPENVMM_LOG` environment variable:\n\n```bash\nOPENVMM_LOG=trace cargo xflowey vmm-tests-run --filter \"test(foo)\"\n```\n\n## Other Useful Flags\n\n| Flag | Purpose |\n|------|---------|\n| `--release` | Release build (default: debug) |\n| `--build-only` | Build without running |\n| `--verbose` | Verbose cargo output |\n| `--install-missing-deps` | Auto-install missing system dependencies |\n| `--custom-uefi-firmware ` | Use a custom UEFI firmware (MSVM.fd) |\n| `--custom-kernel ` | Use a custom kernel image |\n\nRun `cargo xflowey vmm-tests-run --help` for the full option list.\n\n## Common Pitfalls\n\n- **Don't use `cargo nextest run -p vmm_tests` directly** — artifacts won't\n be present and tests will fail with missing-artifact errors.\n- **Windows output dir from WSL** — when the selected tests need a Windows\n filesystem (Hyper-V tests, or VHDX / dynamic VHD1 images), `--dir` must be on\n `/mnt/c/` or `/mnt/d/`, not in the WSL filesystem. Other tests use the\n default.\n- **Hyper-V tests** — require Hyper-V Administrators group membership and\n disable lazy fetch automatically.\n- **CI failures** — use the `openvmm-ci-investigation` skill to diagnose\n failing VMM tests in CI, not this workflow.\n",".github/copilot-instructions.md":"# OpenVMM Repository\n\nOpenVMM is a modular, cross-platform Virtual Machine Monitor (VMM) written in Rust.\nThis repository is home to both OpenVMM and OpenHCL (a paravisor).\nDocumentation lives in `Guide/` and is published at https://openvmm.dev.\n\n## Build & Setup\n\nRestore required dependencies before building for the first time:\n```bash\ncargo xflowey restore-packages\n```\n\nThe project supports cross-compilation for `x86_64` and `aarch64`.\nOpenHCL can only be built from Linux (WSL2 counts).\nFor cross-compilation from WSL2 to Windows, see\n`Guide/src/dev_guide/getting_started/cross_compile.md` and source\n`. ./build_support/setup_windows_cross.sh`.\n\n## Git Commit Rules\n\n- **Never amend commits that have already been pushed without express\n permission from the user.** Make new commits instead. PRs are\n squash-merged, so a clean history is unnecessary.\n- Rebasing onto `main` to resolve conflicts is fine, but do not use\n `git commit --amend`, `git rebase -i`, or `git push --force` to clean\n up history on already-pushed commits.\n\n## Pre-Commit Checklist (MANDATORY)\n\n**You MUST run these commands before every `git commit` in this repo.\nDo NOT commit without completing all three steps.**\n\n1. `cargo clippy --all-targets -p ` — for each modified package.\n2. `cargo doc --no-deps -p ` — for each modified package.\n3. `cargo xtask fmt --fix` — fix formatting, headers, naming conventions.\n Run this **last** because fixing clippy/doc issues may introduce\n formatting changes that need to be cleaned up.\n\nIf `cargo xtask fmt --fix` still fails after auto-fixes, fix the remaining\nreported issues manually and re-run until it succeeds. Do not run individual\n`--pass` commands afterward.\n\n## Trust Boundaries & Safety\n\nBoth OpenVMM and OpenHCL process data from untrusted sources. Code must\n**never panic** on untrusted input.\n\n- **OpenVMM does not trust the guest** — code must not panic on any guest input\n- **OpenHCL does not trust the root** — code must not panic on any root input\n- **OpenHCL does not trust the VTL0 guest** — the attack surface is subtle and needs human review\n\n**Error handling across trust boundaries:**\n- Use `thiserror` for typed error enums at library/API boundaries and\n protocol-facing code\n- Use `anyhow` with `.context(\"...\")` for application-level plumbing and\n context propagation\n- Never `.unwrap()` or `.expect()` on data that crosses a trust boundary\n- For protocol/hardware enums where unknown values must round-trip without\n panicking, use the `open_enum!` macro instead of a normal Rust `enum`\n- Rate-limit trace events that can be triggered repeatedly by guest\n interactions — use `tracelimit::warn_ratelimited!` (or `error_ratelimited!`,\n `info_ratelimited!`) instead of bare `tracing::warn!` etc.\n\n**Other safety rules:**\n1. Avoid `unsafe` code\n2. Avoid taking new external dependencies, especially those that significantly increase binary size\n3. Several OpenHCL crates (e.g., `minimal_rt`, `openhcl_boot`, `sidecar`,\n `host_fdt_parser`) must support `no_std` builds. In particular,\n `minimal_rt` and `host_fdt_parser` are unconditionally `no_std`, and\n `openhcl_boot` and `sidecar` use `cfg_attr(minimal_rt, no_std, no_main)`.\n Do not introduce `std`-only dependencies or APIs into code that is\n compiled for the `minimal_rt` configuration in these crates\n4. Prefer `assert!` over `debug_assert!` for internal invariants — the\n performance cost is negligible in nearly all code, and catching invariant\n violations in release builds is more valuable. The project follows a\n \"fail fast\" philosophy: crash immediately on broken invariants rather\n than letting the process continue in an undefined state where bugs are\n harder to diagnose. (This does not apply to untrusted input — use error\n handling at trust boundaries, not assertions.)\n\n## Testing\n\nRun tests with cargo-nextest using the `agent` profile, which suppresses\noutput for passing tests and only shows slow/failing tests:\n```bash\ncargo nextest run --profile agent -p \n```\n\n**Do NOT pipe test output to `grep`, `tail`, or other filters.** The `agent`\nprofile already minimizes output. Piping hides failures and makes hangs\ninvisible.\n\nFor VMM test validation during development, use `cargo xflowey vmm-tests-run`:\n```bash\ncargo xflowey vmm-tests-run --filter \"test(my_test_name)\"\n```\nDo not pass `--dir` in most cases. When cross-compiling for Windows from WSL2,\n`--dir` on the Windows filesystem (e.g., `--dir /mnt/d/vmm_tests`) is required\nonly when the selected tests use disk images that need a Windows filesystem\n(Hyper-V tests, or VHDX / dynamic VHD1 images); `vmm-tests-run` detects this and\nerrors with a clear message when it's needed. For native host tests, and for\nWindows-from-WSL2 tests that stream disks or use fixed VHD1 / VMGS / ISO images,\nit is unnecessary.\n\nThis automatically discovers artifacts, builds dependencies, and runs tests.\nSee `Guide/src/dev_guide/tests/vmm.md` for details.\n\n- **Unit tests** — spread throughout crates in `#[cfg(test)]` blocks.\n Should be fast, isolated, and not require root/administrator access.\n Add `use test_with_tracing::test;` in test modules so that `tracing`\n is initialized and traces appear in test output.\n- **VMM tests** — integration tests in `vmm_tests/` using the petri\n framework (requires additional setup).\n- Mark tests requiring special setup with `#[ignore]`.\n- Update `Guide/` docs when adding features or changing behavior\n (see `.github/instructions/doc-code-sync.instructions.md` for the mapping)\n- **CI failures** — to investigate failing CI checks on a PR, load the\n `openvmm-ci-investigation` skill.\n\n## Rust Edition\n\nThis project uses the **Rust 2024 edition** (`edition = \"2024\"` in root\n`Cargo.toml`).\n\n## Common Pitfalls\n\n- **`guest_arch` not `target_arch`**: For guest-architecture-specific code,\n use `cfg(guest_arch = \"x86_64\")` — **not** `cfg(target_arch = \"x86_64\")`.\n The VMM can run guests of a different architecture than the host. Using\n `target_arch` will fail CI.\n- **Workspace dependencies**: All dependency versions are centralized in\n the root `Cargo.toml`. In crate `Cargo.toml` files, use\n `dep_name.workspace = true` — not inline version specifiers. Add new\n dependencies to `[workspace.dependencies]` in the root first.\n- **Pipeline YAMLs are auto-generated**: Files under `ci-flowey/` (e.g.,\n `openvmm-pr.yaml`) are generated by the flowey framework — do not\n hand-edit them. Run `cargo xflowey regen` to regenerate.\n- **flowey nodes**: Use `flowey::shell_cmd!` and `rt.sh` inside flowey\n nodes — not `xshell::cmd!` or `xshell::Shell::new`.\n\n## Autonomous Agent Inner Loop\n\nWhen running as a coding agent (GitHub Copilot coding agent or similar),\nfollow this validation loop **before pushing each commit**. This covers\nthe common early CI failures (including the fmt + clippy checks from job0)\nlocally, avoiding slow push-and-wait cycles.\n\n1. **Identify modified packages.** For each file you changed, find the\n crate's `Cargo.toml` and note the package name.\n2. **Check compilation:** `cargo check -p ` — fast type-check.\n3. **Clippy:** `cargo clippy --all-targets -p ` — lint.\n4. **Doc:** `cargo doc --no-deps -p ` — catch doc errors.\n5. **Unit tests:** `cargo nextest run --profile agent -p ` — run the crate's\n tests. If nextest is not installed, use `cargo test -p `.\n6. **Formatting:** `cargo xtask fmt --fix` — run last, since earlier\n fixes may introduce formatting changes.\n\nIf any step fails, fix the issue and re-run from that step. Do not push\nuntil all six steps pass.\n\n**Cost notes:** Steps 2–5 are scoped to the modified package (`-p`),\nso they are fast even in this large workspace. Step 6 runs workspace-wide\nbut is also fast. The full cycle typically takes under 2 minutes for\na single-crate change.\n",".github/skills/openvmm-ci-investigation/SKILL.md":"---\nname: openvmm-ci-investigation\ndescription: Investigate CI failures on OpenVMM PRs. Load when a PR has failing CI checks, you need to download and analyze test artifacts, or you need to diagnose build, fmt, clippy, or VMM test failures.\n---\n\n# Investigating CI Failures\n\nWhen a PR has failing CI checks, **always start by running the investigation\nscript**. Do not manually query the GitHub API or download artifacts by hand\n— the script handles all of that automatically.\n\n## Step 1: Run the Script\n\n```bash\n# Investigate the current branch's PR (no argument needed)\npython3 repo_support/investigate_ci.py\n\n# Investigate a specific PR by number\npython3 repo_support/investigate_ci.py 2946\n\n# Or by run ID directly\npython3 repo_support/investigate_ci.py 23017249697\n```\n\nThe script automatically:\n1. Resolves the PR to **every** failed CI run for its head commit. A commit\n can have multiple workflows (e.g. \"OpenVMM PR\" and \"OpenVMM Docs PR\"),\n and each failing one is investigated in turn — so don't assume a single\n run covers everything. Only the newest run of each workflow is used, so\n superseded runs are ignored; a `NOTE: is still in_progress`\n line means results are incomplete and more failures may appear later.\n2. Identifies failed jobs\n3. Downloads `*-unit-tests-junit-xml` artifacts and parses JUnit XML for\n unit test failures\n4. Downloads `*-vmm-tests-logs` artifacts if they exist\n5. Finds tests with `petri.failed` markers and extracts ERROR/WARN lines\n6. If no test or JUnit artifacts exist (build/fmt/clippy failure), shows the\n tail of the failed job's log\n\n## Step 2: Analyze the Script Output\n\nRead the script's output to identify:\n- Which tests failed (unit tests and/or VMM tests)\n- Error messages and root causes\n- Whether it's a build/fmt/clippy failure vs. a test failure\n\nWhen more than one run failed, the output is split by `# RUN n of m` banners.\nCheck **all** of them — a docs failure and a build failure are reported in\nseparate runs.\n\nThen use the information to diagnose the issue and suggest fixes.\n\n## Step 3: Diagnose from Logs and Code\n\n**Always try to diagnose the failure from CI logs, test code, and error\nmessages first.** Most failures can be understood without local reproduction.\nRead the relevant test source, trace the error through the code, and form\na hypothesis before considering local repro.\n\n## Step 4: Reproduce Locally (Only If Needed)\n\nIf you cannot diagnose the failure from logs alone, **ask the user** whether\nthey want to attempt local reproduction before proceeding. Do not\nautomatically start building or running tests on the user's machine.\n\nIf the user agrees, check whether the failing platform matches the host\narchitecture — you can only reproduce tests locally on the same arch. If\nit doesn't match, explain this to the user and continue diagnosing from\nCI logs and test code.\n\nTo reproduce locally, load the `vmm-tests` skill for instructions on\nrunning with `cargo xflowey vmm-tests-run`. Do **not** use\n`cargo nextest run -p vmm_tests` directly — it won't have the required\nartifacts.\n\n## Reference: Manual Commands\n\n> **Only use these if the script fails or you need to dig deeper into a\n> specific artifact.** In normal usage, the script above is sufficient.\n\nIf the script isn't available or you need more control, follow these steps:\n\n### 1. Find the failing run\n\n```bash\n# Get the run ID for a PR\ngh pr checks -R microsoft/openvmm\n# Or list runs for a specific commit\ngh run list -R microsoft/openvmm --commit \n```\n\n### 2. Identify the failing job\n\n```bash\ngh run view -R microsoft/openvmm --json jobs \\\n -q '[.jobs[] | select(.conclusion == \"failure\") | {name, databaseId}]'\n```\n\n### 3. Download test artifacts\n\n**Unit test results** are stored in JUnit XML artifacts named\n`{platform}-unit-tests-junit-xml`. Known platforms include:\n- `x64-linux`\n- `aarch64-linux`\n- `aarch64-linux-musl`\n\n```bash\n# Download unit test JUnit XML for a platform\ngh run download -R microsoft/openvmm \\\n -n aarch64-linux-unit-tests-junit-xml -D /tmp/junit-xml\n# Parse failures from the XML\npython3 -c \"\nimport xml.etree.ElementTree as ET, sys\nfor f in __import__('pathlib').Path(sys.argv[1]).rglob('*.xml'):\n for tc in ET.parse(f).iter('testcase'):\n fail = tc.find('failure')\n if fail is None:\n fail = tc.find('error')\n if fail is not None:\n print(f'FAIL: {tc.get(\\\"classname\\\",\\\"\\\")}::{tc.get(\\\"name\\\",\\\"\\\")}')\n print(f' {fail.get(\\\"message\\\",\\\"\\\")[:200]}')\n\" /tmp/junit-xml\n```\n\n**VMM test results** are stored in artifacts named `{platform}-vmm-tests-logs`.\nThe known platforms are:\n- `x64-windows-intel`\n- `x64-windows-intel-tdx`\n- `x64-windows-amd`\n- `x64-windows-amd-snp`\n- `x64-linux`\n- `aarch64-windows`\n\n```bash\n# Download a specific platform's test logs\ngh run download -R microsoft/openvmm \\\n -n x64-windows-amd-snp-vmm-tests-logs -D /tmp/test-logs\n```\n\n### 4. Find failed tests\n\nEach test gets its own directory inside the artifact. Look for `petri.failed`\nmarker files (passing tests have `petri.passed` instead):\n\n```bash\nfind /tmp/test-logs -name \"petri.failed\"\n```\n\nThe `petri.failed` file contains the error petri recorded for the test. The\ntest's real name is in `petri.test`, which petri writes before the test body\nruns — so a directory with `petri.test` but *no* result marker is a test that\nwas killed or crashed before it could report.\n\n### 5. Extract errors from petri.jsonl\n\nThe `petri.jsonl` file in each test directory is the primary structured log.\nEach line is a JSON object with fields: `timestamp`, `source`, `severity`,\n`message`. Filter for `ERROR` and `WARN` severity for a quick diagnosis:\n\n```bash\npython3 -c \"\nimport json, sys\nfor line in open(sys.argv[1]):\n try:\n e = json.loads(line.strip())\n if e.get('severity') in ('ERROR', 'WARN'):\n print(f'[{e[\\\"severity\\\"]}] {e.get(\\\"source\\\",\\\"?\\\")}: {e.get(\\\"message\\\",\\\"\\\").strip()[:200]}')\n except: pass\n\" /tmp/test-logs//petri.jsonl\n```\n\n## Artifact Contents\n\n### Unit test JUnit XML\n\nArtifacts named `{platform}-unit-tests-junit-xml` contain JUnit XML files\nwith `` elements. Failed tests have `` or ``\nchildren with `message` attributes describing the failure. These are the\nprimary artifacts for diagnosing unit test / cargo-nextest failures.\n\n### VMM test logs (petri)\n\nEach test directory contains:\n- `petri.jsonl` — Structured JSON Lines log **(primary file for investigation)**\n- `petri.log` — Plain text version of the test log\n- `petri.test` — The test's name, written before the test body runs\n- `petri.passed` or `petri.failed` — Pass/fail marker; a failure marker holds the error\n- `openhcl.log` — OpenHCL serial console output, if the test exercised OpenHCL\n- `hyperv.log` — Hyper-V event log, if the test exercises the Hyper-V backend\n- `openvmm.log` — OpenVMM serial console output, if the test exercises the OpenVMM backend\n- `guest.log`, `uefi.log` — Guest OS serial output\n- Sometimes: `screenshot_*.png` — periodic screenshots of the guest\n- Sometimes: `dumpfile.dmp`\n\n## Viewing Results in Browser\n\nTest results are uploaded to Azure Blob Storage and viewable at:\n`https://openvmm.dev/test-results/#/runs/_`\n\nLogs are uploaded per re-run attempt, so the run key includes the attempt\nnumber (`1` for the first attempt). `investigate_ci.py` prints the correct\nURL in its summary.\n\n## Common Failure Patterns\n\n- **Unit test failure**: A `unit tests` job failed. The script downloads\n JUnit XML artifacts and shows the failing test names and messages. Common\n causes: new test code that relies on OS capabilities not available in CI\n (e.g. TAP devices, elevated permissions).\n- **Formatting / house-rules**: The `quick check [fmt, clippy x64-linux]` job\n failed. No test artifacts will exist. Run `cargo xtask fmt --fix` locally\n and check the job log for the specific rule that failed.\n- **Docs failure**: A job in the separate \"OpenVMM Docs PR\" workflow failed —\n either `build mdbook guide` (broken Guide markdown/links; mdbook logs\n `[ERROR] (mdbook::...)`) or `build and check docs` (rustdoc warnings, which\n are denied in CI). Reproduce with `cargo doc --no-deps -p `.\n No test artifacts will exist.\n- **TripleFault**: VM hit a fatal error during boot. Check `petri.jsonl` for\n Hyper-V Worker/Chipset errors. Often infrastructure-related, not caused by\n the PR's code changes.\n- **Timeout**: Test exceeded its time limit. Check if the VM booted at all.\n- **Guest assertion failure**: Guest-side test failed. Check `guest.log`.\n- **Build failure**: No test artifacts will exist. Check the job log directly\n with `gh run view --job --log`.\n\n## Important API Note\n\nThe `gh run view --json artifacts` flag does **not** exist. To list artifacts\nfor a run, use the GitHub API directly:\n\n```bash\ngh api repos/microsoft/openvmm/actions/runs//artifacts\n```\n"},"items":[{"name":"SKILL.md","path":".github/skills/fuzzing/SKILL.md","rawUrl":"https://raw.githubusercontent.com/microsoft/openvmm/HEAD/.github/skills/fuzzing/SKILL.md","title":"fuzzing","category":"anthropic-skill","format":"markdown","content":"---\nname: fuzzing\ndescription: \"Run, optimize, and debug OpenVMM fuzzers. Covers cargo-fuzz targets, crash reproduction, lldb debugging, code coverage analysis, entropy optimization, and multi-target parallel campaigns.\"\n---\n\n# OpenVMM Fuzzer Guide\n\n## Prerequisites\n\n- **Linux only** — libfuzzer-sys doesn't support Windows.\n- **cargo-fuzz**: `cargo install cargo-fuzz`\n- **lldb** (for debugging): `sudo apt-get install -y lldb`\n- **aarch64 RUSTFLAGS**: On aarch64, set `RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\"` or builds fail with atomics errors. Not needed on x86_64.\n\n## Fuzz targets\n\nFuzz targets live in `/fuzz/` directories alongside the crate they test.\nEach has a `Cargo.toml` with `cargo-fuzz = true` metadata and a `fuzz_.rs`\nbinary. Find all targets:\n\n```bash\ncargo xtask fuzz list\n```\n\n## Running a fuzzer\n\n```bash\n# Continuous fuzzing (finds new crashes)\ncargo xtask fuzz run fuzz_ide\n\n# Reproduce a specific crash artifact\ncargo xtask fuzz run fuzz_ide path/to/crash-artifact\n```\n\nThe xtask wrapper sets `XTASK_FUZZ_REPRO=1` automatically when an artifact\npath is provided, which enables `init_tracing_if_repro()` in the fuzz target.\n\n## Build without running\n\n```bash\ncargo xtask fuzz build fuzz_ide\n```\n\nThe binary lands at `target//release/fuzz_ide` (e.g.,\n`target/aarch64-unknown-linux-gnu/release/fuzz_ide` or\n`target/x86_64-unknown-linux-gnu/release/fuzz_ide`).\n\n## Reproducing a crash (direct binary)\n\nRunning the binary directly is faster for iteration than going through xtask:\n\n```bash\n# Without tracing (fast, quiet)\n./target//release/fuzz_ide path/to/crash-artifact\n\n# With tracing (verbose — shows device state at each poll)\nXTASK_FUZZ_REPRO=1 ./target//release/fuzz_ide path/to/crash-artifact\n\n# With backtrace\nRUST_BACKTRACE=full ./target//release/fuzz_ide path/to/crash-artifact\n```\n\n## Debugging with lldb\n\n### Key lessons\n\n1. **Use batch mode with a command file** — lldb's interactive prompt doesn't\n work well from automated terminals. Write commands to a file and use\n `lldb -b -s `.\n2. **Set `auto-confirm true`** — prevents lldb from blocking on \"Do you really\n want to quit?\" prompts.\n3. **Breakpoint on `panic_fmt`** — `rust_begin_unwind` is often mangled and\n won't resolve. Use `-r panic_fmt` instead.\n4. **Drop `XTASK_FUZZ_REPRO`** when using lldb — the tracing output from\n repeated polls produces megabytes of spam that buries the debugger output.\n5. **Release builds have limited variable info** — `frame variable` may show\n nothing. For richer inspection, build with `--dev` (see below).\n\n### Command file template\n\nCreate a file (e.g., `scratch/lldb_fuzz.cmd`):\n\n```\nsettings set auto-confirm true\nbreakpoint set -r panic_fmt\nrun\nbt 20\nframe select 1\nsource list\nframe select 2\nsource list\nframe variable\nquit\n```\n\n### Run it\n\n```bash\nlldb -b -s scratch/lldb_fuzz.cmd \\\n -- ./target//release/fuzz_ide \\\n path/to/crash-artifact 2>&1 | tail -80\n```\n\nThe `| tail -80` trims libfuzzer startup noise. Adjust as needed.\n\n### Using rust-lldb\n\n`rust-lldb` adds Rust pretty-printers for `Vec`, `String`, etc. It requires\nlldb to be installed. Usage is the same but replace `lldb` with `rust-lldb`:\n\n```bash\nrust-lldb -b -s scratch/lldb_fuzz.cmd \\\n -- ./target//release/fuzz_ide \\\n path/to/crash-artifact 2>&1 | tail -80\n```\n\n### Debug builds for variable inspection\n\nRelease builds have limited `frame variable` output. For full variable\ninspection, build with `--dev`:\n\n```bash\ncargo xtask fuzz build fuzz_ide -- --dev\n```\n\nThe debug binary lands at `target//debug/fuzz_ide`. It's slower but\nlldb can inspect all locals and struct fields.\n\n### Navigating Rust wrapper types in lldb\n\nReaching the actual data through Arc, Mutex, UnsafeCell etc. requires\nknowing the field path. Use `type lookup` to discover struct layouts:\n\n```\n(lldb) type lookup closeable_mutex::CloseableMutex\n```\n\nCommon wrapper-type traversal patterns:\n\n| Rust type | lldb field path |\n|-----------|----------------|\n| `Arc` | `var.ptr.pointer->data` (through `ArcInner`) |\n| `CloseableMutex` | `.value.value` (UnsafeCell inside Mutex) |\n| `UnsafeCell` | `.value` |\n| `Option` | `.$variants$.$variant$.value.__0` (check `$discr$` — 0 = None for most types) |\n| Rust enum | `.$variants$.$variant$N.value.__0` (N = variant index) |\n\nFull path example for IDE device state through `Arc>`:\n\n```\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].state\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].enlightened_write\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].bus_master_state\n```\n\nFor drive registers inside `Option` containing an enum variant:\n\n```\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].drives[0].$variants$.$variant$.value.__0.$variants$.$variant$.value.__0.state.regs\n```\n\n### Setting breakpoint ignore counts\n\nTo skip the first N hits of a breakpoint (useful when a function is called\nhundreds of times during polling):\n\n```\n(lldb) breakpoint set -f lib.rs -l 1253\n(lldb) breakpoint modify -i 500 1\n(lldb) run\n```\n\nThis skips the first 500 hits, then stops on hit 501.\n\n### Reading Rust bitfield status registers\n\nStatus register values appear as `(__0 = 'X')` where `'X'` is the char at\nthat byte value. Decode manually: `0x58` = bits 3,4,6 set = `drq`, `dsc`,\n`drdy`.\n\n## Minimizing crash inputs\n\n```bash\ncargo xtask fuzz tmin fuzz_ide path/to/crash-artifact\n```\n\n## Corpus management\n\n```bash\n# Minimize the corpus (remove redundant inputs)\ncargo xtask fuzz cmin fuzz_ide\n```\n\nCorpus files live in `/fuzz/corpus//`.\nCrash artifacts land in `/fuzz/artifacts//`.\n\n## Fuzzer optimization principles\n\n### Entropy efficiency\nEvery byte of fuzzer input should drive meaningful behavior. Common wastes:\n- **Arbitrary setup parameters that don't affect coverage** — e.g., guest memory size, buffer counts. Use fixed values.\n- **`u.ratio()` calls for branching** — consumes entropy for each decision. Use `Arbitrary` enums instead (1 byte per decision vs 8+ for ratio).\n- **Redundant `arbitrary_data()` calls** — generate composite values with one call instead of field-by-field. E.g., a Guid from `[u8; 16]` instead of 11 separate calls.\n\n### Simplification\n- **Merge duplicated functions** that only differ in a small parameter. Use an enum variant to select behavior.\n- **Remove dead protocol negotiation** — if a device rejects all interesting operations without init, always negotiate. Raw packet paths can still exercise init state machines through mutation.\n- **Remove wrapper functions** that add no value over calling the underlying API directly.\n\n### Coverage gaps\n- **0-item/empty configurations** — fuzzers that always attach 1+ disks/devices miss the empty-controller code paths. Use `0..=N` ranges.\n- **Hardcoded setup parameters** can hide code paths. E.g., fixed namespace IDs mean only one NVMe identify-namespace path is tested. Make setup params fuzzer-driven where they affect behavior.\n- **TODOs mark legitimate gaps** — don't remove TODOs unless you've verified the marked code path is either (a) implemented or (b) genuinely doesn't affect coverage.\n\n### Performance\n- **Smaller backing stores** — RAM disks used for fuzzing don't need to be large. 32KB is sufficient for most device fuzzers (was 4MB).\n- **Fixed guest memory** — varying guest memory size wastes entropy without affecting device logic.\n- **Always negotiate protocols** — skipping negotiation wastes 100% of remaining fuzz actions on a single error branch.\n- **`-fork=N`** for parallel fuzzing across CPUs. Allocate cores proportionally to fuzzer complexity.\n\n## Multi-target parallel fuzzing\n\nRun multiple fuzzers across available CPUs for extended campaigns:\n\n```bash\n# 6-hour campaign across 112 cores, 7 fuzzers\n# Each runs with -fork=N, survives crashes/timeouts/OOMs\nnohup bash -c 'RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\" \\\n cargo xtask fuzz run fuzz_storvsp -- -- \\\n -fork=20 -max_total_time=21600 \\\n -ignore_crashes=1 -ignore_timeouts=1 -ignore_ooms=1 \\\n -print_final_stats=1' > /tmp/fuzz_storvsp.log 2>&1 &\n```\n\nAllocate cores based on fuzzer complexity:\n- Heavy fuzzers (storvsp, ide, nvme, nvram): 20 cores each\n- Medium fuzzers (cmos_rtc): 15 cores\n- Light fuzzers (diagnostics, scsi_buffers): 7–10 cores\n\nMonitor progress: `tail -1 /tmp/fuzz_*.log`\n\nAfter the run, collect coverage on each fuzzer for gap analysis.\n\n## Code coverage\n\n### Prerequisites\n\nEverything from the main prerequisites, plus:\n\n- **llvm-tools**: `rustup component add llvm-tools`\n- **lcov** (for HTML reports): `sudo apt-get install -y lcov`\n\n### Collecting coverage\n\nThe xtask has built-in coverage support. It runs each corpus entry through\na coverage-instrumented build and merges the results:\n\n```bash\n# Collect coverage data only\ncargo xtask fuzz coverage fuzz_ide\n\n# Collect + generate HTML report (requires lcov + genhtml)\ncargo xtask fuzz coverage fuzz_ide --with-html-report\n\n# Skip rebuild, just regenerate report from existing profdata\ncargo xtask fuzz coverage fuzz_ide --with-html-report --only-report\n```\n\nOn aarch64, set `RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\"` as usual.\n\nCoverage artifacts land in:\n\n| Artifact | Path |\n|----------|------|\n| profdata | `/fuzz/coverage//coverage.profdata` |\n| HTML report | `target//coverage//release/lcov_html_/index.html` |\n\n### Filtering coverage to relevant crates\n\nThe raw HTML report covers *all* compiled code including dependencies — the\noverall percentage is meaningless (typically 4–6%). What matters is coverage\nof the **target crate itself** and its immediate domain dependencies.\n\nUse `llvm-cov report` with source path filtering:\n\n```bash\n# Find the llvm-cov binary from the toolchain\nLLVM_COV=$(find $(rustc --print sysroot) -name \"llvm-cov\" -type f | head -1)\n\n# Full per-file report, excluding third-party code\n$LLVM_COV report \\\n -instr-profile /fuzz/coverage//coverage.profdata \\\n -object target//coverage//release/ \\\n --ignore-filename-regex='(rustc|\\.cargo|registry)' 2>&1 \\\n | grep 'vm/devices/storage/ide/src/'\n```\n\nFor lcov-based per-file analysis, export to lcov and extract per-file stats:\n\n```bash\n$LLVM_COV export \\\n -instr-profile /fuzz/coverage//coverage.profdata \\\n -object target//coverage//release/ \\\n --ignore-filename-regex='(rustc|\\.cargo|registry)' \\\n -format=lcov > /tmp/coverage.lcov\n\n# Per-file function/line summary from lcov\nawk '/SF:.*ide\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep -E '^(FNF|FNH|LF|LH):'\n# FNF/FNH = functions found/hit, LF/LH = lines found/hit\n\n# Find uncovered line numbers\nawk '/SF:.*ide\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep '^DA:' | grep ',0$' | cut -d: -f2 | cut -d, -f1\n```\n\n## Coverage scope rules\n\nEach fuzzer has a **primary scope** (the crate it exists to test) and\n**secondary scope** (dependencies it exercises incidentally). Coverage\nimprovements should focus on primary scope first.\n\n### What coverage matters per fuzzer\n\n| Fuzzer | Primary scope | Secondary scope | Out of scope |\n|--------|--------------|-----------------|--------------|\n| fuzz_ide | `ide/src/` | `scsidisk/src/scsidvd/`, `scsidisk/src/atapi_scsi.rs`, `disk_backend/`, `disklayer_ram/` | `pci_core/`, `vmcore/`, `guestmem/` (infrastructure) |\n| fuzz_storvsp | `storvsp/src/` | `scsidisk/src/lib.rs`, `vmbus_ring/`, `vmbus_async/`, `disk_backend/` | `vmbus_channel/`, `vmbus_core/` (infrastructure) |\n| fuzz_nvme_driver | `nvme_driver/src/` | `nvme/src/`, `nvme_spec/`, `user_driver/` | `page_pool_alloc/`, `vmcore/` |\n| fuzz_firmware_uefi_nvram | `firmware_uefi/src/service/nvram/` | `uefi_nvram_specvars/` | `crypto`, `openssl` (infrastructure) |\n| fuzz_firmware_uefi_diagnostics | `firmware_uefi/src/service/diagnostics/` | — | `guestmem/` |\n| fuzz_chipset_cmos_rtc | `chipset/src/cmos_rtc/` | `chipset_device_fuzz/` | `pal_async/`, `vmcore/` |\n| fuzz_scsi_buffers | `scsi_buffers/src/` | — | `guestmem/` |\n| fuzz_chipset_battery | `chipset/src/battery/` | `chipset_device_fuzz/` | — |\n\n**Rules:**\n\n1. **Primary scope is the goal.** Line coverage of the target crate is the\n metric to optimize. The fuzzer exists to find bugs in this code.\n\n2. **Secondary scope matters when it's a trust boundary.** `scsidisk` parsing\n guest-supplied SCSI commands is a real attack surface — coverage there has\n value. But `disklayer_ram` just storing bytes is infrastructure.\n\n3. **Out-of-scope code is noise.** `pci_core`, `vmcore`, `guestmem` get\n exercised incidentally. Don't optimize the fuzzer to improve their coverage\n — those crates deserve their own fuzzers if coverage matters.\n\n4. **Save/restore and resolver code is structurally unreachable.** Functions\n like `save_restore.rs`, `resolver.rs` won't be hit by fuzzing because\n the fuzzer doesn't exercise serialization or resource resolution paths.\n Don't count them against coverage targets.\n\n5. **Inspect (`InspectMut`) implementations are low-value.** These debug/\n diagnostics paths aren't part of the attack surface. Low coverage is\n acceptable.\n\n## Coverage-guided fuzzer improvement process\n\n### Step 1: Collect and filter\n\n```bash\n# Run coverage\ncargo xtask fuzz coverage --with-html-report\n\n# Open the HTML report for visual inspection\n# target//coverage//release/lcov_html_/index.html\n```\n\n### Step 2: Identify gaps in primary scope\n\nLook at the HTML report or extract uncovered lines:\n\n```bash\nawk '/SF:.*\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep '^DA:' | grep ',0$' | cut -d: -f2 | cut -d, -f1\n```\n\nRead the source at those lines. Classify each uncovered region:\n\n| Classification | Action |\n|---------------|--------|\n| **Reachable via fuzzer input** — the fuzzer *could* reach this code but hasn't found the right input sequence | Investigate why. Is the path gated by specific byte patterns the fuzzer struggles to generate? |\n| **Structurally unreachable** — the fuzzer harness doesn't wire up the code path (e.g., save/restore, PCI config reads the harness skips) | Extend the harness: add new `FuzzAction` variants, expose more device interfaces to the fuzzer. |\n| **Error-handling / defensive code** — `unreachable!()`, error paths for invalid hardware states | Low priority. These are worth testing but don't indicate a fuzzer deficiency. |\n| **Inspect / debug only** — `InspectMut`, `Display`, `Debug` impls | Skip entirely. Not attack surface. |\n\n### Step 3: Improve the fuzzer\n\nImprovements fall into two categories, in priority order:\n\n**A. Extend the fuzz harness**\n\nIf an entire code path is structurally unreachable, the harness needs changes.\nExamples:\n\n- fuzz_ide doesn't exercise PCI config space reads → add PCI config read/write\n to the `FuzzAction` enum in `chipset_device_fuzz`\n- fuzz_storvsp doesn't test sub-channel operations → add corresponding VMBus\n channel actions\n\n**B. Improve `Arbitrary` implementations**\n\nIf the fuzzer generates mostly one type of operation, check the `Arbitrary`\nimpl. Common issues:\n\n- Uniform distribution across action types wastes entropy on less interesting\n paths. Weight toward rarer operations.\n- Overly constrained generation (e.g., only valid SCSI commands) misses\n malformed-input bugs.\n- Overly unconstrained generation (random bytes everywhere) wastes time on\n inputs that fail early validation and never reach deep code.\n\n### Step 4: Measure improvement\n\nAfter making changes:\n\n1. Run the fuzzer for a fixed duration (e.g., 1 hour)\n2. Minimize the corpus: `cargo xtask fuzz cmin `\n3. Collect coverage again\n4. Compare line counts against prior coverage runs\n\nTrack coverage changes over time. Ratchet up: if a change reduces primary\nscope coverage, investigate why.\n\n### Step 5: Iterate\n\nCoverage-guided improvement is iterative. Each cycle:\n\n1. Collect → 2. Identify gaps → 3. Fix → 4. Fuzz → 5. Measure → repeat\n\nTarget **≥85% line coverage** in primary scope as a practical goal. The\nremaining ~15% is typically error-handling code, debug impls, and paths\nrequiring specific hardware/timing conditions the fuzzer can't easily produce.\n","frontmatter":{"name":"fuzzing","description":"Run, optimize, and debug OpenVMM fuzzers. Covers cargo-fuzz targets, crash reproduction, lldb debugging, code coverage analysis, entropy optimization, and multi-target parallel campaigns."},"isInternal":false,"tokens":4325,"sizeBytes":16351},{"name":"SKILL.md","path":".github/skills/guide-maintenance/SKILL.md","rawUrl":"https://raw.githubusercontent.com/microsoft/openvmm/HEAD/.github/skills/guide-maintenance/SKILL.md","title":"guide-maintenance","category":"anthropic-skill","format":"markdown","content":"---\nname: guide-maintenance\ndescription: >\n Maintain the OpenVMM Guide. Load when: (1) adding, removing, or moving Guide\n pages, (2) adding new device crates or CLI args that need doc coverage, or\n (3) auditing Guide freshness against code changes.\n---\n\n# Guide Maintenance Skill\n\nProcedures for keeping the OpenVMM Guide (`Guide/src/`) in sync with the codebase.\n\n`.github/instructions/doc-code-sync.instructions.md` contains the instructions\nand heuristics that are automatically loaded during Copilot code review on\n`*.rs` and `Cargo.toml` files.\n\n---\n\n## Adding a New Guide Page\n\n### 1. Choose the right location\n\n| Content type | Path pattern |\n|-------------|-------------|\n| Emulated device reference | `reference/emulated//.md` |\n| VMBus device reference | `reference/devices/vmbus/.md` |\n| Virtio device reference | `reference/devices/virtio/.md` |\n| Device backend reference | `reference/backends/.md` |\n| Architecture deep-dive | `reference/architecture/openvmm/` or `reference/architecture/openhcl/` |\n| CLI / management | `reference/openvmm/management/` |\n| Developer tool | `dev_guide/dev_tools/.md` |\n| Test framework | `dev_guide/tests/` |\n\n### 2. Create the page\n\n- Follow the style in `.github/instructions/guide-docs.instructions.md`\n- Include a brief overview, key concepts, and links to relevant rustdoc\n- Reference crate names in backticks\n- Use relative paths from repo root when pointing at code\n\n### 3. Update SUMMARY.md\n\nAdd the page to `Guide/src/SUMMARY.md` in the correct section. Pages with\ncontent get a path; placeholder topics for future work get an empty link `()`.\n\n```markdown\n- [Page Title](./reference/path/to/page.md)\n```\n\n---\n\n## Removing or Moving a Guide Page\n\n1. Update or remove the entry in `Guide/src/SUMMARY.md`\n2. Check for cross-references from other Guide pages (grep for the old path)\n3. If moving: ensure the SUMMARY.md link points to the new location\n\n---\n\n## Adding a New Device Crate\n\nWhen a new crate is added under `vm/devices/`:\n\n1. **Does it need a Guide page?** If it's a user-visible device or backend,\n yes. Internal plumbing crates (e.g., `*_resources`, `*_protocol`) usually\n don't need their own page but should be mentioned in the parent device page.\n\n2. **Create the page** following the \"Adding a New Guide Page\" procedure above.\n\n3. **Update the \"What to Flag\" list** if the new crate introduces a new\n category of reviewable change.\n\n---\n\n## Adding or Changing CLI Arguments\n\nWhen `openvmm/openvmm_entry/src/cli_args.rs` changes:\n\n1. Update `Guide/src/reference/openvmm/management/cli.md`\n2. The CLI page has a disclaimer that it may be out of date — keep it honest,\n but still update the page when making changes\n3. New arguments should include: flag syntax, description, default value,\n and any interactions with other flags\n\n---\n\n## Auditing Guide Freshness\n\nTo check whether the Guide is in sync with the code:\n\n1. **Scan the Guide pages** in `Guide/src/`\n2. Check whether the Guide page content still matches the code:\n - Crate names still correct?\n - Struct/enum names still exist?\n - Behavioral descriptions still accurate?\n - Links to rustdoc still resolve?\n3. Check `Guide/src/SUMMARY.md` for placeholder links `()` — these are\n topics that need content. Prioritize ones whose code areas have been\n actively developed.\n\n### Quick audit commands\n\n```bash\n# Find Guide pages that reference a crate name\ngrep -r \"crate_name\" Guide/src/\n\n# Find placeholder topics (empty links) in SUMMARY.md\ngrep '()\\s*$' Guide/src/SUMMARY.md\n```\n","frontmatter":{"name":"guide-maintenance","description":"Maintain the OpenVMM Guide. Load when: (1) adding, removing, or moving Guide pages, (2) adding new device crates or CLI args that need doc coverage, or (3) auditing Guide freshness against code changes.\n"},"isInternal":false,"tokens":869,"sizeBytes":3591},{"name":"SKILL.md","path":".github/skills/openvmm-ci-investigation/SKILL.md","rawUrl":"https://raw.githubusercontent.com/microsoft/openvmm/HEAD/.github/skills/openvmm-ci-investigation/SKILL.md","title":"openvmm-ci-investigation","category":"anthropic-skill","format":"markdown","content":"---\nname: openvmm-ci-investigation\ndescription: Investigate CI failures on OpenVMM PRs. Load when a PR has failing CI checks, you need to download and analyze test artifacts, or you need to diagnose build, fmt, clippy, or VMM test failures.\n---\n\n# Investigating CI Failures\n\nWhen a PR has failing CI checks, **always start by running the investigation\nscript**. Do not manually query the GitHub API or download artifacts by hand\n— the script handles all of that automatically.\n\n## Step 1: Run the Script\n\n```bash\n# Investigate the current branch's PR (no argument needed)\npython3 repo_support/investigate_ci.py\n\n# Investigate a specific PR by number\npython3 repo_support/investigate_ci.py 2946\n\n# Or by run ID directly\npython3 repo_support/investigate_ci.py 23017249697\n```\n\nThe script automatically:\n1. Resolves the PR to **every** failed CI run for its head commit. A commit\n can have multiple workflows (e.g. \"OpenVMM PR\" and \"OpenVMM Docs PR\"),\n and each failing one is investigated in turn — so don't assume a single\n run covers everything. Only the newest run of each workflow is used, so\n superseded runs are ignored; a `NOTE: is still in_progress`\n line means results are incomplete and more failures may appear later.\n2. Identifies failed jobs\n3. Downloads `*-unit-tests-junit-xml` artifacts and parses JUnit XML for\n unit test failures\n4. Downloads `*-vmm-tests-logs` artifacts if they exist\n5. Finds tests with `petri.failed` markers and extracts ERROR/WARN lines\n6. If no test or JUnit artifacts exist (build/fmt/clippy failure), shows the\n tail of the failed job's log\n\n## Step 2: Analyze the Script Output\n\nRead the script's output to identify:\n- Which tests failed (unit tests and/or VMM tests)\n- Error messages and root causes\n- Whether it's a build/fmt/clippy failure vs. a test failure\n\nWhen more than one run failed, the output is split by `# RUN n of m` banners.\nCheck **all** of them — a docs failure and a build failure are reported in\nseparate runs.\n\nThen use the information to diagnose the issue and suggest fixes.\n\n## Step 3: Diagnose from Logs and Code\n\n**Always try to diagnose the failure from CI logs, test code, and error\nmessages first.** Most failures can be understood without local reproduction.\nRead the relevant test source, trace the error through the code, and form\na hypothesis before considering local repro.\n\n## Step 4: Reproduce Locally (Only If Needed)\n\nIf you cannot diagnose the failure from logs alone, **ask the user** whether\nthey want to attempt local reproduction before proceeding. Do not\nautomatically start building or running tests on the user's machine.\n\nIf the user agrees, check whether the failing platform matches the host\narchitecture — you can only reproduce tests locally on the same arch. If\nit doesn't match, explain this to the user and continue diagnosing from\nCI logs and test code.\n\nTo reproduce locally, load the `vmm-tests` skill for instructions on\nrunning with `cargo xflowey vmm-tests-run`. Do **not** use\n`cargo nextest run -p vmm_tests` directly — it won't have the required\nartifacts.\n\n## Reference: Manual Commands\n\n> **Only use these if the script fails or you need to dig deeper into a\n> specific artifact.** In normal usage, the script above is sufficient.\n\nIf the script isn't available or you need more control, follow these steps:\n\n### 1. Find the failing run\n\n```bash\n# Get the run ID for a PR\ngh pr checks -R microsoft/openvmm\n# Or list runs for a specific commit\ngh run list -R microsoft/openvmm --commit \n```\n\n### 2. Identify the failing job\n\n```bash\ngh run view -R microsoft/openvmm --json jobs \\\n -q '[.jobs[] | select(.conclusion == \"failure\") | {name, databaseId}]'\n```\n\n### 3. Download test artifacts\n\n**Unit test results** are stored in JUnit XML artifacts named\n`{platform}-unit-tests-junit-xml`. Known platforms include:\n- `x64-linux`\n- `aarch64-linux`\n- `aarch64-linux-musl`\n\n```bash\n# Download unit test JUnit XML for a platform\ngh run download -R microsoft/openvmm \\\n -n aarch64-linux-unit-tests-junit-xml -D /tmp/junit-xml\n# Parse failures from the XML\npython3 -c \"\nimport xml.etree.ElementTree as ET, sys\nfor f in __import__('pathlib').Path(sys.argv[1]).rglob('*.xml'):\n for tc in ET.parse(f).iter('testcase'):\n fail = tc.find('failure')\n if fail is None:\n fail = tc.find('error')\n if fail is not None:\n print(f'FAIL: {tc.get(\\\"classname\\\",\\\"\\\")}::{tc.get(\\\"name\\\",\\\"\\\")}')\n print(f' {fail.get(\\\"message\\\",\\\"\\\")[:200]}')\n\" /tmp/junit-xml\n```\n\n**VMM test results** are stored in artifacts named `{platform}-vmm-tests-logs`.\nThe known platforms are:\n- `x64-windows-intel`\n- `x64-windows-intel-tdx`\n- `x64-windows-amd`\n- `x64-windows-amd-snp`\n- `x64-linux`\n- `aarch64-windows`\n\n```bash\n# Download a specific platform's test logs\ngh run download -R microsoft/openvmm \\\n -n x64-windows-amd-snp-vmm-tests-logs -D /tmp/test-logs\n```\n\n### 4. Find failed tests\n\nEach test gets its own directory inside the artifact. Look for `petri.failed`\nmarker files (passing tests have `petri.passed` instead):\n\n```bash\nfind /tmp/test-logs -name \"petri.failed\"\n```\n\nThe `petri.failed` file contains the error petri recorded for the test. The\ntest's real name is in `petri.test`, which petri writes before the test body\nruns — so a directory with `petri.test` but *no* result marker is a test that\nwas killed or crashed before it could report.\n\n### 5. Extract errors from petri.jsonl\n\nThe `petri.jsonl` file in each test directory is the primary structured log.\nEach line is a JSON object with fields: `timestamp`, `source`, `severity`,\n`message`. Filter for `ERROR` and `WARN` severity for a quick diagnosis:\n\n```bash\npython3 -c \"\nimport json, sys\nfor line in open(sys.argv[1]):\n try:\n e = json.loads(line.strip())\n if e.get('severity') in ('ERROR', 'WARN'):\n print(f'[{e[\\\"severity\\\"]}] {e.get(\\\"source\\\",\\\"?\\\")}: {e.get(\\\"message\\\",\\\"\\\").strip()[:200]}')\n except: pass\n\" /tmp/test-logs//petri.jsonl\n```\n\n## Artifact Contents\n\n### Unit test JUnit XML\n\nArtifacts named `{platform}-unit-tests-junit-xml` contain JUnit XML files\nwith `` elements. Failed tests have `` or ``\nchildren with `message` attributes describing the failure. These are the\nprimary artifacts for diagnosing unit test / cargo-nextest failures.\n\n### VMM test logs (petri)\n\nEach test directory contains:\n- `petri.jsonl` — Structured JSON Lines log **(primary file for investigation)**\n- `petri.log` — Plain text version of the test log\n- `petri.test` — The test's name, written before the test body runs\n- `petri.passed` or `petri.failed` — Pass/fail marker; a failure marker holds the error\n- `openhcl.log` — OpenHCL serial console output, if the test exercised OpenHCL\n- `hyperv.log` — Hyper-V event log, if the test exercises the Hyper-V backend\n- `openvmm.log` — OpenVMM serial console output, if the test exercises the OpenVMM backend\n- `guest.log`, `uefi.log` — Guest OS serial output\n- Sometimes: `screenshot_*.png` — periodic screenshots of the guest\n- Sometimes: `dumpfile.dmp`\n\n## Viewing Results in Browser\n\nTest results are uploaded to Azure Blob Storage and viewable at:\n`https://openvmm.dev/test-results/#/runs/_`\n\nLogs are uploaded per re-run attempt, so the run key includes the attempt\nnumber (`1` for the first attempt). `investigate_ci.py` prints the correct\nURL in its summary.\n\n## Common Failure Patterns\n\n- **Unit test failure**: A `unit tests` job failed. The script downloads\n JUnit XML artifacts and shows the failing test names and messages. Common\n causes: new test code that relies on OS capabilities not available in CI\n (e.g. TAP devices, elevated permissions).\n- **Formatting / house-rules**: The `quick check [fmt, clippy x64-linux]` job\n failed. No test artifacts will exist. Run `cargo xtask fmt --fix` locally\n and check the job log for the specific rule that failed.\n- **Docs failure**: A job in the separate \"OpenVMM Docs PR\" workflow failed —\n either `build mdbook guide` (broken Guide markdown/links; mdbook logs\n `[ERROR] (mdbook::...)`) or `build and check docs` (rustdoc warnings, which\n are denied in CI). Reproduce with `cargo doc --no-deps -p `.\n No test artifacts will exist.\n- **TripleFault**: VM hit a fatal error during boot. Check `petri.jsonl` for\n Hyper-V Worker/Chipset errors. Often infrastructure-related, not caused by\n the PR's code changes.\n- **Timeout**: Test exceeded its time limit. Check if the VM booted at all.\n- **Guest assertion failure**: Guest-side test failed. Check `guest.log`.\n- **Build failure**: No test artifacts will exist. Check the job log directly\n with `gh run view --job --log`.\n\n## Important API Note\n\nThe `gh run view --json artifacts` flag does **not** exist. To list artifacts\nfor a run, use the GitHub API directly:\n\n```bash\ngh api repos/microsoft/openvmm/actions/runs//artifacts\n```\n","frontmatter":{"name":"openvmm-ci-investigation","description":"Investigate CI failures on OpenVMM PRs. Load when a PR has failing CI checks, you need to download and analyze test artifacts, or you need to diagnose build, fmt, clippy, or VMM test failures."},"isInternal":false,"tokens":2333,"sizeBytes":9014},{"name":"SKILL.md","path":".github/skills/vmm-tests/SKILL.md","rawUrl":"https://raw.githubusercontent.com/microsoft/openvmm/HEAD/.github/skills/vmm-tests/SKILL.md","title":"vmm-tests","category":"anthropic-skill","format":"markdown","content":"---\nname: vmm-tests\ndescription: \"Run VMM tests locally with cargo xflowey vmm-tests-run. Load when running or debugging VMM tests, or when you need to understand the petri test framework, artifact handling, or cross-compilation for VMM tests.\"\n---\n\n# Running VMM Tests\n\nVMM tests boot full virtual machines and validate behavior. They live in\n`vmm_tests/vmm_tests/tests/tests/` and use the `petri` test framework.\n\n**Always use `cargo xflowey vmm-tests-run`** — never raw `cargo nextest run -p\nvmm_tests`. The xflowey command handles artifact discovery, dependency\nbuilding, and test execution automatically.\n\n## Quick Start\n\n```bash\n# Run a specific test\ncargo xflowey vmm-tests-run --filter \"test(my_test_name)\"\n\n# Run all tests matching a prefix\ncargo xflowey vmm-tests-run --filter \"test(/^boot_/)\"\n\n# Run all tests (rarely needed locally)\ncargo xflowey vmm-tests-run --filter \"all()\"\n```\n\nUsually you don't need `--dir` — it defaults to `target/vmm_tests`. When\ncross-compiling for Windows from WSL2, `--dir` on the Windows filesystem (e.g.,\n`--dir /mnt/d/vmm_tests`) is required **only** when the selected tests use disk\nimages that need a Windows filesystem: **Hyper-V tests**, or tests using **VHDX\n/ dynamic VHD1** images. For everything else (streamed disks, or plain fixed\nVHD1 / VMGS / ISO files), the default WSL-side directory works. `vmm-tests-run`\ndetects this and errors with a clear message if `--dir` is required.\n\n## Filter Syntax\n\nFilters use [nextest filter expressions](https://nexte.st/docs/filtersets/):\n\n| Expression | Matches |\n|-----------|---------|\n| `test(foo)` | Tests with `foo` in the name |\n| `test(/^boot_/)` | Tests starting with `boot_` (regex) |\n| `test(foo) & !test(hyperv)` | `foo` tests excluding Hyper-V variants |\n| `all()` | Everything |\n\n## Platform Targeting\n\nBy default, tests build for the current host. Use `--target` for\ncross-compilation:\n\n```bash\n# Cross-compile and run Windows tests from WSL2\ncargo xflowey vmm-tests-run --target windows-x64\n```\n\n| Target | Description |\n|--------|-------------|\n| `windows-x64` | Windows x86_64 (Hyper-V / WHP) |\n| `windows-aarch64` | Windows ARM64 (Hyper-V / WHP) |\n| `linux-x64` | Linux x86_64 |\n\n**Windows from WSL2**: The default output directory works for most tests. Add\n`--dir` on the Windows filesystem (e.g., `--dir /mnt/d/vmm_tests`) only when the\nselected tests use disk images requiring a Windows filesystem (Hyper-V tests,\nor VHDX / dynamic VHD1 images); `vmm-tests-run` errors with a clear message when\nit's needed. Cross-compilation setup is required first — see\n`Guide/src/dev_guide/getting_started/cross_compile.md`.\n\n## Artifact Handling (Lazy Fetch)\n\nBy default, disk images (VHDs/ISOs) are streamed on demand via HTTP with local\nSQLite caching. This avoids multi-GB upfront downloads.\n\n- `--no-lazy-fetch` — download all images upfront instead of streaming\n- Lazy fetch is automatically disabled for Hyper-V tests (they need local files)\n- `--skip-vhd-prompt` — skip interactive VHD download prompts (useful for\n automation)\n\n## Viewing Logs\n\nTest output (petri logs, guest serial, etc.) is shown by default on failure.\nFor full OpenVMM tracing, set the `OPENVMM_LOG` environment variable:\n\n```bash\nOPENVMM_LOG=trace cargo xflowey vmm-tests-run --filter \"test(foo)\"\n```\n\n## Other Useful Flags\n\n| Flag | Purpose |\n|------|---------|\n| `--release` | Release build (default: debug) |\n| `--build-only` | Build without running |\n| `--verbose` | Verbose cargo output |\n| `--install-missing-deps` | Auto-install missing system dependencies |\n| `--custom-uefi-firmware ` | Use a custom UEFI firmware (MSVM.fd) |\n| `--custom-kernel ` | Use a custom kernel image |\n\nRun `cargo xflowey vmm-tests-run --help` for the full option list.\n\n## Common Pitfalls\n\n- **Don't use `cargo nextest run -p vmm_tests` directly** — artifacts won't\n be present and tests will fail with missing-artifact errors.\n- **Windows output dir from WSL** — when the selected tests need a Windows\n filesystem (Hyper-V tests, or VHDX / dynamic VHD1 images), `--dir` must be on\n `/mnt/c/` or `/mnt/d/`, not in the WSL filesystem. Other tests use the\n default.\n- **Hyper-V tests** — require Hyper-V Administrators group membership and\n disable lazy fetch automatically.\n- **CI failures** — use the `openvmm-ci-investigation` skill to diagnose\n failing VMM tests in CI, not this workflow.\n","frontmatter":{"name":"vmm-tests","description":"Run VMM tests locally with cargo xflowey vmm-tests-run. Load when running or debugging VMM tests, or when you need to understand the petri test framework, artifact handling, or cross-compilation for VMM tests."},"isInternal":false,"tokens":1153,"sizeBytes":4393},{"name":"copilot-instructions.md","path":".github/copilot-instructions.md","rawUrl":"https://raw.githubusercontent.com/microsoft/openvmm/HEAD/.github/copilot-instructions.md","title":"GitHub Copilot Instructions","category":"copilot-instructions","format":"markdown","content":"# OpenVMM Repository\n\nOpenVMM is a modular, cross-platform Virtual Machine Monitor (VMM) written in Rust.\nThis repository is home to both OpenVMM and OpenHCL (a paravisor).\nDocumentation lives in `Guide/` and is published at https://openvmm.dev.\n\n## Build & Setup\n\nRestore required dependencies before building for the first time:\n```bash\ncargo xflowey restore-packages\n```\n\nThe project supports cross-compilation for `x86_64` and `aarch64`.\nOpenHCL can only be built from Linux (WSL2 counts).\nFor cross-compilation from WSL2 to Windows, see\n`Guide/src/dev_guide/getting_started/cross_compile.md` and source\n`. ./build_support/setup_windows_cross.sh`.\n\n## Git Commit Rules\n\n- **Never amend commits that have already been pushed without express\n permission from the user.** Make new commits instead. PRs are\n squash-merged, so a clean history is unnecessary.\n- Rebasing onto `main` to resolve conflicts is fine, but do not use\n `git commit --amend`, `git rebase -i`, or `git push --force` to clean\n up history on already-pushed commits.\n\n## Pre-Commit Checklist (MANDATORY)\n\n**You MUST run these commands before every `git commit` in this repo.\nDo NOT commit without completing all three steps.**\n\n1. `cargo clippy --all-targets -p ` — for each modified package.\n2. `cargo doc --no-deps -p ` — for each modified package.\n3. `cargo xtask fmt --fix` — fix formatting, headers, naming conventions.\n Run this **last** because fixing clippy/doc issues may introduce\n formatting changes that need to be cleaned up.\n\nIf `cargo xtask fmt --fix` still fails after auto-fixes, fix the remaining\nreported issues manually and re-run until it succeeds. Do not run individual\n`--pass` commands afterward.\n\n## Trust Boundaries & Safety\n\nBoth OpenVMM and OpenHCL process data from untrusted sources. Code must\n**never panic** on untrusted input.\n\n- **OpenVMM does not trust the guest** — code must not panic on any guest input\n- **OpenHCL does not trust the root** — code must not panic on any root input\n- **OpenHCL does not trust the VTL0 guest** — the attack surface is subtle and needs human review\n\n**Error handling across trust boundaries:**\n- Use `thiserror` for typed error enums at library/API boundaries and\n protocol-facing code\n- Use `anyhow` with `.context(\"...\")` for application-level plumbing and\n context propagation\n- Never `.unwrap()` or `.expect()` on data that crosses a trust boundary\n- For protocol/hardware enums where unknown values must round-trip without\n panicking, use the `open_enum!` macro instead of a normal Rust `enum`\n- Rate-limit trace events that can be triggered repeatedly by guest\n interactions — use `tracelimit::warn_ratelimited!` (or `error_ratelimited!`,\n `info_ratelimited!`) instead of bare `tracing::warn!` etc.\n\n**Other safety rules:**\n1. Avoid `unsafe` code\n2. Avoid taking new external dependencies, especially those that significantly increase binary size\n3. Several OpenHCL crates (e.g., `minimal_rt`, `openhcl_boot`, `sidecar`,\n `host_fdt_parser`) must support `no_std` builds. In particular,\n `minimal_rt` and `host_fdt_parser` are unconditionally `no_std`, and\n `openhcl_boot` and `sidecar` use `cfg_attr(minimal_rt, no_std, no_main)`.\n Do not introduce `std`-only dependencies or APIs into code that is\n compiled for the `minimal_rt` configuration in these crates\n4. Prefer `assert!` over `debug_assert!` for internal invariants — the\n performance cost is negligible in nearly all code, and catching invariant\n violations in release builds is more valuable. The project follows a\n \"fail fast\" philosophy: crash immediately on broken invariants rather\n than letting the process continue in an undefined state where bugs are\n harder to diagnose. (This does not apply to untrusted input — use error\n handling at trust boundaries, not assertions.)\n\n## Testing\n\nRun tests with cargo-nextest using the `agent` profile, which suppresses\noutput for passing tests and only shows slow/failing tests:\n```bash\ncargo nextest run --profile agent -p \n```\n\n**Do NOT pipe test output to `grep`, `tail`, or other filters.** The `agent`\nprofile already minimizes output. Piping hides failures and makes hangs\ninvisible.\n\nFor VMM test validation during development, use `cargo xflowey vmm-tests-run`:\n```bash\ncargo xflowey vmm-tests-run --filter \"test(my_test_name)\"\n```\nDo not pass `--dir` in most cases. When cross-compiling for Windows from WSL2,\n`--dir` on the Windows filesystem (e.g., `--dir /mnt/d/vmm_tests`) is required\nonly when the selected tests use disk images that need a Windows filesystem\n(Hyper-V tests, or VHDX / dynamic VHD1 images); `vmm-tests-run` detects this and\nerrors with a clear message when it's needed. For native host tests, and for\nWindows-from-WSL2 tests that stream disks or use fixed VHD1 / VMGS / ISO images,\nit is unnecessary.\n\nThis automatically discovers artifacts, builds dependencies, and runs tests.\nSee `Guide/src/dev_guide/tests/vmm.md` for details.\n\n- **Unit tests** — spread throughout crates in `#[cfg(test)]` blocks.\n Should be fast, isolated, and not require root/administrator access.\n Add `use test_with_tracing::test;` in test modules so that `tracing`\n is initialized and traces appear in test output.\n- **VMM tests** — integration tests in `vmm_tests/` using the petri\n framework (requires additional setup).\n- Mark tests requiring special setup with `#[ignore]`.\n- Update `Guide/` docs when adding features or changing behavior\n (see `.github/instructions/doc-code-sync.instructions.md` for the mapping)\n- **CI failures** — to investigate failing CI checks on a PR, load the\n `openvmm-ci-investigation` skill.\n\n## Rust Edition\n\nThis project uses the **Rust 2024 edition** (`edition = \"2024\"` in root\n`Cargo.toml`).\n\n## Common Pitfalls\n\n- **`guest_arch` not `target_arch`**: For guest-architecture-specific code,\n use `cfg(guest_arch = \"x86_64\")` — **not** `cfg(target_arch = \"x86_64\")`.\n The VMM can run guests of a different architecture than the host. Using\n `target_arch` will fail CI.\n- **Workspace dependencies**: All dependency versions are centralized in\n the root `Cargo.toml`. In crate `Cargo.toml` files, use\n `dep_name.workspace = true` — not inline version specifiers. Add new\n dependencies to `[workspace.dependencies]` in the root first.\n- **Pipeline YAMLs are auto-generated**: Files under `ci-flowey/` (e.g.,\n `openvmm-pr.yaml`) are generated by the flowey framework — do not\n hand-edit them. Run `cargo xflowey regen` to regenerate.\n- **flowey nodes**: Use `flowey::shell_cmd!` and `rt.sh` inside flowey\n nodes — not `xshell::cmd!` or `xshell::Shell::new`.\n\n## Autonomous Agent Inner Loop\n\nWhen running as a coding agent (GitHub Copilot coding agent or similar),\nfollow this validation loop **before pushing each commit**. This covers\nthe common early CI failures (including the fmt + clippy checks from job0)\nlocally, avoiding slow push-and-wait cycles.\n\n1. **Identify modified packages.** For each file you changed, find the\n crate's `Cargo.toml` and note the package name.\n2. **Check compilation:** `cargo check -p ` — fast type-check.\n3. **Clippy:** `cargo clippy --all-targets -p ` — lint.\n4. **Doc:** `cargo doc --no-deps -p ` — catch doc errors.\n5. **Unit tests:** `cargo nextest run --profile agent -p ` — run the crate's\n tests. If nextest is not installed, use `cargo test -p `.\n6. **Formatting:** `cargo xtask fmt --fix` — run last, since earlier\n fixes may introduce formatting changes.\n\nIf any step fails, fix the issue and re-run from that step. Do not push\nuntil all six steps pass.\n\n**Cost notes:** Steps 2–5 are scoped to the modified package (`-p`),\nso they are fast even in this large workspace. Step 6 runs workspace-wide\nbut is also fast. The full cycle typically takes under 2 minutes for\na single-crate change.\n","isInternal":false,"tokens":1980,"sizeBytes":7911}],"systemPromptSnippet":"\n\n---\nname: fuzzing\ndescription: \"Run, optimize, and debug OpenVMM fuzzers. Covers cargo-fuzz targets, crash reproduction, lldb debugging, code coverage analysis, entropy optimization, and multi-target parallel campaigns.\"\n---\n\n# OpenVMM Fuzzer Guide\n\n## Prerequisites\n\n- **Linux only** — libfuzzer-sys doesn't support Windows.\n- **cargo-fuzz**: `cargo install cargo-fuzz`\n- **lldb** (for debugging): `sudo apt-get install -y lldb`\n- **aarch64 RUSTFLAGS**: On aarch64, set `RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\"` or builds fail with atomics errors. Not needed on x86_64.\n\n## Fuzz targets\n\nFuzz targets live in `/fuzz/` directories alongside the crate they test.\nEach has a `Cargo.toml` with `cargo-fuzz = true` metadata and a `fuzz_.rs`\nbinary. Find all targets:\n\n```bash\ncargo xtask fuzz list\n```\n\n## Running a fuzzer\n\n```bash\n# Continuous fuzzing (finds new crashes)\ncargo xtask fuzz run fuzz_ide\n\n# Reproduce a specific crash artifact\ncargo xtask fuzz run fuzz_ide path/to/crash-artifact\n```\n\nThe xtask wrapper sets `XTASK_FUZZ_REPRO=1` automatically when an artifact\npath is provided, which enables `init_tracing_if_repro()` in the fuzz target.\n\n## Build without running\n\n```bash\ncargo xtask fuzz build fuzz_ide\n```\n\nThe binary lands at `target//release/fuzz_ide` (e.g.,\n`target/aarch64-unknown-linux-gnu/release/fuzz_ide` or\n`target/x86_64-unknown-linux-gnu/release/fuzz_ide`).\n\n## Reproducing a crash (direct binary)\n\nRunning the binary directly is faster for iteration than going through xtask:\n\n```bash\n# Without tracing (fast, quiet)\n./target//release/fuzz_ide path/to/crash-artifact\n\n# With tracing (verbose — shows device state at each poll)\nXTASK_FUZZ_REPRO=1 ./target//release/fuzz_ide path/to/crash-artifact\n\n# With backtrace\nRUST_BACKTRACE=full ./target//release/fuzz_ide path/to/crash-artifact\n```\n\n## Debugging with lldb\n\n### Key lessons\n\n1. **Use batch mode with a command file** — lldb's interactive prompt doesn't\n work well from automated terminals. Write commands to a file and use\n `lldb -b -s `.\n2. **Set `auto-confirm true`** — prevents lldb from blocking on \"Do you really\n want to quit?\" prompts.\n3. **Breakpoint on `panic_fmt`** — `rust_begin_unwind` is often mangled and\n won't resolve. Use `-r panic_fmt` instead.\n4. **Drop `XTASK_FUZZ_REPRO`** when using lldb — the tracing output from\n repeated polls produces megabytes of spam that buries the debugger output.\n5. **Release builds have limited variable info** — `frame variable` may show\n nothing. For richer inspection, build with `--dev` (see below).\n\n### Command file template\n\nCreate a file (e.g., `scratch/lldb_fuzz.cmd`):\n\n```\nsettings set auto-confirm true\nbreakpoint set -r panic_fmt\nrun\nbt 20\nframe select 1\nsource list\nframe select 2\nsource list\nframe variable\nquit\n```\n\n### Run it\n\n```bash\nlldb -b -s scratch/lldb_fuzz.cmd \\\n -- ./target//release/fuzz_ide \\\n path/to/crash-artifact 2>&1 | tail -80\n```\n\nThe `| tail -80` trims libfuzzer startup noise. Adjust as needed.\n\n### Using rust-lldb\n\n`rust-lldb` adds Rust pretty-printers for `Vec`, `String`, etc. It requires\nlldb to be installed. Usage is the same but replace `lldb` with `rust-lldb`:\n\n```bash\nrust-lldb -b -s scratch/lldb_fuzz.cmd \\\n -- ./target//release/fuzz_ide \\\n path/to/crash-artifact 2>&1 | tail -80\n```\n\n### Debug builds for variable inspection\n\nRelease builds have limited `frame variable` output. For full variable\ninspection, build with `--dev`:\n\n```bash\ncargo xtask fuzz build fuzz_ide -- --dev\n```\n\nThe debug binary lands at `target//debug/fuzz_ide`. It's slower but\nlldb can inspect all locals and struct fields.\n\n### Navigating Rust wrapper types in lldb\n\nReaching the actual data through Arc, Mutex, UnsafeCell etc. requires\nknowing the field path. Use `type lookup` to discover struct layouts:\n\n```\n(lldb) type lookup closeable_mutex::CloseableMutex\n```\n\nCommon wrapper-type traversal patterns:\n\n| Rust type | lldb field path |\n|-----------|----------------|\n| `Arc` | `var.ptr.pointer->data` (through `ArcInner`) |\n| `CloseableMutex` | `.value.value` (UnsafeCell inside Mutex) |\n| `UnsafeCell` | `.value` |\n| `Option` | `.$variants$.$variant$.value.__0` (check `$discr$` — 0 = None for most types) |\n| Rust enum | `.$variants$.$variant$N.value.__0` (N = variant index) |\n\nFull path example for IDE device state through `Arc>`:\n\n```\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].state\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].enlightened_write\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].bus_master_state\n```\n\nFor drive registers inside `Option` containing an enum variant:\n\n```\n(lldb) expr ide_device.ptr.pointer->data.value.value.channels[0].drives[0].$variants$.$variant$.value.__0.$variants$.$variant$.value.__0.state.regs\n```\n\n### Setting breakpoint ignore counts\n\nTo skip the first N hits of a breakpoint (useful when a function is called\nhundreds of times during polling):\n\n```\n(lldb) breakpoint set -f lib.rs -l 1253\n(lldb) breakpoint modify -i 500 1\n(lldb) run\n```\n\nThis skips the first 500 hits, then stops on hit 501.\n\n### Reading Rust bitfield status registers\n\nStatus register values appear as `(__0 = 'X')` where `'X'` is the char at\nthat byte value. Decode manually: `0x58` = bits 3,4,6 set = `drq`, `dsc`,\n`drdy`.\n\n## Minimizing crash inputs\n\n```bash\ncargo xtask fuzz tmin fuzz_ide path/to/crash-artifact\n```\n\n## Corpus management\n\n```bash\n# Minimize the corpus (remove redundant inputs)\ncargo xtask fuzz cmin fuzz_ide\n```\n\nCorpus files live in `/fuzz/corpus//`.\nCrash artifacts land in `/fuzz/artifacts//`.\n\n## Fuzzer optimization principles\n\n### Entropy efficiency\nEvery byte of fuzzer input should drive meaningful behavior. Common wastes:\n- **Arbitrary setup parameters that don't affect coverage** — e.g., guest memory size, buffer counts. Use fixed values.\n- **`u.ratio()` calls for branching** — consumes entropy for each decision. Use `Arbitrary` enums instead (1 byte per decision vs 8+ for ratio).\n- **Redundant `arbitrary_data()` calls** — generate composite values with one call instead of field-by-field. E.g., a Guid from `[u8; 16]` instead of 11 separate calls.\n\n### Simplification\n- **Merge duplicated functions** that only differ in a small parameter. Use an enum variant to select behavior.\n- **Remove dead protocol negotiation** — if a device rejects all interesting operations without init, always negotiate. Raw packet paths can still exercise init state machines through mutation.\n- **Remove wrapper functions** that add no value over calling the underlying API directly.\n\n### Coverage gaps\n- **0-item/empty configurations** — fuzzers that always attach 1+ disks/devices miss the empty-controller code paths. Use `0..=N` ranges.\n- **Hardcoded setup parameters** can hide code paths. E.g., fixed namespace IDs mean only one NVMe identify-namespace path is tested. Make setup params fuzzer-driven where they affect behavior.\n- **TODOs mark legitimate gaps** — don't remove TODOs unless you've verified the marked code path is either (a) implemented or (b) genuinely doesn't affect coverage.\n\n### Performance\n- **Smaller backing stores** — RAM disks used for fuzzing don't need to be large. 32KB is sufficient for most device fuzzers (was 4MB).\n- **Fixed guest memory** — varying guest memory size wastes entropy without affecting device logic.\n- **Always negotiate protocols** — skipping negotiation wastes 100% of remaining fuzz actions on a single error branch.\n- **`-fork=N`** for parallel fuzzing across CPUs. Allocate cores proportionally to fuzzer complexity.\n\n## Multi-target parallel fuzzing\n\nRun multiple fuzzers across available CPUs for extended campaigns:\n\n```bash\n# 6-hour campaign across 112 cores, 7 fuzzers\n# Each runs with -fork=N, survives crashes/timeouts/OOMs\nnohup bash -c 'RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\" \\\n cargo xtask fuzz run fuzz_storvsp -- -- \\\n -fork=20 -max_total_time=21600 \\\n -ignore_crashes=1 -ignore_timeouts=1 -ignore_ooms=1 \\\n -print_final_stats=1' > /tmp/fuzz_storvsp.log 2>&1 &\n```\n\nAllocate cores based on fuzzer complexity:\n- Heavy fuzzers (storvsp, ide, nvme, nvram): 20 cores each\n- Medium fuzzers (cmos_rtc): 15 cores\n- Light fuzzers (diagnostics, scsi_buffers): 7–10 cores\n\nMonitor progress: `tail -1 /tmp/fuzz_*.log`\n\nAfter the run, collect coverage on each fuzzer for gap analysis.\n\n## Code coverage\n\n### Prerequisites\n\nEverything from the main prerequisites, plus:\n\n- **llvm-tools**: `rustup component add llvm-tools`\n- **lcov** (for HTML reports): `sudo apt-get install -y lcov`\n\n### Collecting coverage\n\nThe xtask has built-in coverage support. It runs each corpus entry through\na coverage-instrumented build and merges the results:\n\n```bash\n# Collect coverage data only\ncargo xtask fuzz coverage fuzz_ide\n\n# Collect + generate HTML report (requires lcov + genhtml)\ncargo xtask fuzz coverage fuzz_ide --with-html-report\n\n# Skip rebuild, just regenerate report from existing profdata\ncargo xtask fuzz coverage fuzz_ide --with-html-report --only-report\n```\n\nOn aarch64, set `RUSTFLAGS=\"-Ctarget-feature=+lse,+neon\"` as usual.\n\nCoverage artifacts land in:\n\n| Artifact | Path |\n|----------|------|\n| profdata | `/fuzz/coverage//coverage.profdata` |\n| HTML report | `target//coverage//release/lcov_html_/index.html` |\n\n### Filtering coverage to relevant crates\n\nThe raw HTML report covers *all* compiled code including dependencies — the\noverall percentage is meaningless (typically 4–6%). What matters is coverage\nof the **target crate itself** and its immediate domain dependencies.\n\nUse `llvm-cov report` with source path filtering:\n\n```bash\n# Find the llvm-cov binary from the toolchain\nLLVM_COV=$(find $(rustc --print sysroot) -name \"llvm-cov\" -type f | head -1)\n\n# Full per-file report, excluding third-party code\n$LLVM_COV report \\\n -instr-profile /fuzz/coverage//coverage.profdata \\\n -object target//coverage//release/ \\\n --ignore-filename-regex='(rustc|\\.cargo|registry)' 2>&1 \\\n | grep 'vm/devices/storage/ide/src/'\n```\n\nFor lcov-based per-file analysis, export to lcov and extract per-file stats:\n\n```bash\n$LLVM_COV export \\\n -instr-profile /fuzz/coverage//coverage.profdata \\\n -object target//coverage//release/ \\\n --ignore-filename-regex='(rustc|\\.cargo|registry)' \\\n -format=lcov > /tmp/coverage.lcov\n\n# Per-file function/line summary from lcov\nawk '/SF:.*ide\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep -E '^(FNF|FNH|LF|LH):'\n# FNF/FNH = functions found/hit, LF/LH = lines found/hit\n\n# Find uncovered line numbers\nawk '/SF:.*ide\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep '^DA:' | grep ',0$' | cut -d: -f2 | cut -d, -f1\n```\n\n## Coverage scope rules\n\nEach fuzzer has a **primary scope** (the crate it exists to test) and\n**secondary scope** (dependencies it exercises incidentally). Coverage\nimprovements should focus on primary scope first.\n\n### What coverage matters per fuzzer\n\n| Fuzzer | Primary scope | Secondary scope | Out of scope |\n|--------|--------------|-----------------|--------------|\n| fuzz_ide | `ide/src/` | `scsidisk/src/scsidvd/`, `scsidisk/src/atapi_scsi.rs`, `disk_backend/`, `disklayer_ram/` | `pci_core/`, `vmcore/`, `guestmem/` (infrastructure) |\n| fuzz_storvsp | `storvsp/src/` | `scsidisk/src/lib.rs`, `vmbus_ring/`, `vmbus_async/`, `disk_backend/` | `vmbus_channel/`, `vmbus_core/` (infrastructure) |\n| fuzz_nvme_driver | `nvme_driver/src/` | `nvme/src/`, `nvme_spec/`, `user_driver/` | `page_pool_alloc/`, `vmcore/` |\n| fuzz_firmware_uefi_nvram | `firmware_uefi/src/service/nvram/` | `uefi_nvram_specvars/` | `crypto`, `openssl` (infrastructure) |\n| fuzz_firmware_uefi_diagnostics | `firmware_uefi/src/service/diagnostics/` | — | `guestmem/` |\n| fuzz_chipset_cmos_rtc | `chipset/src/cmos_rtc/` | `chipset_device_fuzz/` | `pal_async/`, `vmcore/` |\n| fuzz_scsi_buffers | `scsi_buffers/src/` | — | `guestmem/` |\n| fuzz_chipset_battery | `chipset/src/battery/` | `chipset_device_fuzz/` | — |\n\n**Rules:**\n\n1. **Primary scope is the goal.** Line coverage of the target crate is the\n metric to optimize. The fuzzer exists to find bugs in this code.\n\n2. **Secondary scope matters when it's a trust boundary.** `scsidisk` parsing\n guest-supplied SCSI commands is a real attack surface — coverage there has\n value. But `disklayer_ram` just storing bytes is infrastructure.\n\n3. **Out-of-scope code is noise.** `pci_core`, `vmcore`, `guestmem` get\n exercised incidentally. Don't optimize the fuzzer to improve their coverage\n — those crates deserve their own fuzzers if coverage matters.\n\n4. **Save/restore and resolver code is structurally unreachable.** Functions\n like `save_restore.rs`, `resolver.rs` won't be hit by fuzzing because\n the fuzzer doesn't exercise serialization or resource resolution paths.\n Don't count them against coverage targets.\n\n5. **Inspect (`InspectMut`) implementations are low-value.** These debug/\n diagnostics paths aren't part of the attack surface. Low coverage is\n acceptable.\n\n## Coverage-guided fuzzer improvement process\n\n### Step 1: Collect and filter\n\n```bash\n# Run coverage\ncargo xtask fuzz coverage --with-html-report\n\n# Open the HTML report for visual inspection\n# target//coverage//release/lcov_html_/index.html\n```\n\n### Step 2: Identify gaps in primary scope\n\nLook at the HTML report or extract uncovered lines:\n\n```bash\nawk '/SF:.*\\/src\\/lib\\.rs$/,/end_of_record/' /tmp/coverage.lcov \\\n | grep '^DA:' | grep ',0$' | cut -d: -f2 | cut -d, -f1\n```\n\nRead the source at those lines. Classify each uncovered region:\n\n| Classification | Action |\n|---------------|--------|\n| **Reachable via fuzzer input** — the fuzzer *could* reach this code but hasn't found the right input sequence | Investigate why. Is the path gated by specific byte patterns the fuzzer struggles to generate? |\n| **Structurally unreachable** — the fuzzer harness doesn't wire up the code path (e.g., save/restore, PCI config reads the harness skips) | Extend the harness: add new `FuzzAction` variants, expose more device interfaces to the fuzzer. |\n| **Error-handling / defensive code** — `unreachable!()`, error paths for invalid hardware states | Low priority. These are worth testing but don't indicate a fuzzer deficiency. |\n| **Inspect / debug only** — `InspectMut`, `Display`, `Debug` impls | Skip entirely. Not attack surface. |\n\n### Step 3: Improve the fuzzer\n\nImprovements fall into two categories, in priority order:\n\n**A. Extend the fuzz harness**\n\nIf an entire code path is structurally unreachable, the harness needs changes.\nExamples:\n\n- fuzz_ide doesn't exercise PCI config space reads → add PCI config read/write\n to the `FuzzAction` enum in `chipset_device_fuzz`\n- fuzz_storvsp doesn't test sub-channel operations → add corresponding VMBus\n channel actions\n\n**B. Improve `Arbitrary` implementations**\n\nIf the fuzzer generates mostly one type of operation, check the `Arbitrary`\nimpl. Common issues:\n\n- Uniform distribution across action types wastes entropy on less interesting\n paths. Weight toward rarer operations.\n- Overly constrained generation (e.g., only valid SCSI commands) misses\n malformed-input bugs.\n- Overly unconstrained generation (random bytes everywhere) wastes time on\n inputs that fail early validation and never reach deep code.\n\n### Step 4: Measure improvement\n\nAfter making changes:\n\n1. Run the fuzzer for a fixed duration (e.g., 1 hour)\n2. Minimize the corpus: `cargo xtask fuzz cmin `\n3. Collect coverage again\n4. Compare line counts against prior coverage runs\n\nTrack coverage changes over time. Ratchet up: if a change reduces primary\nscope coverage, investigate why.\n\n### Step 5: Iterate\n\nCoverage-guided improvement is iterative. Each cycle:\n\n1. Collect → 2. Identify gaps → 3. Fix → 4. Fuzz → 5. Measure → repeat\n\nTarget **≥85% line coverage** in primary scope as a practical goal. The\nremaining ~15% is typically error-handling code, debug impls, and paths\nrequiring specific hardware/timing conditions the fuzzer can't easily produce.\n\n\n---\nname: guide-maintenance\ndescription: >\n Maintain the OpenVMM Guide. Load when: (1) adding, removing, or moving Guide\n pages, (2) adding new device crates or CLI args that need doc coverage, or\n (3) auditing Guide freshness against code changes.\n---\n\n# Guide Maintenance Skill\n\nProcedures for keeping the OpenVMM Guide (`Guide/src/`) in sync with the codebase.\n\n`.github/instructions/doc-code-sync.instructions.md` contains the instructions\nand heuristics that are automatically loaded during Copilot code review on\n`*.rs` and `Cargo.toml` files.\n\n---\n\n## Adding a New Guide Page\n\n### 1. Choose the right location\n\n| Content type | Path pattern |\n|-------------|-------------|\n| Emulated device reference | `reference/emulated//.md` |\n| VMBus device reference | `reference/devices/vmbus/.md` |\n| Virtio device reference | `reference/devices/virtio/.md` |\n| Device backend reference | `reference/backends/.md` |\n| Architecture deep-dive | `reference/architecture/openvmm/` or `reference/architecture/openhcl/` |\n| CLI / management | `reference/openvmm/management/` |\n| Developer tool | `dev_guide/dev_tools/.md` |\n| Test framework | `dev_guide/tests/` |\n\n### 2. Create the page\n\n- Follow the style in `.github/instructions/guide-docs.instructions.md`\n- Include a brief overview, key concepts, and links to relevant rustdoc\n- Reference crate names in backticks\n- Use relative paths from repo root when pointing at code\n\n### 3. Update SUMMARY.md\n\nAdd the page to `Guide/src/SUMMARY.md` in the correct section. Pages with\ncontent get a path; placeholder topics for future work get an empty link `()`.\n\n```markdown\n- [Page Title](./reference/path/to/page.md)\n```\n\n---\n\n## Removing or Moving a Guide Page\n\n1. Update or remove the entry in `Guide/src/SUMMARY.md`\n2. Check for cross-references from other Guide pages (grep for the old path)\n3. If moving: ensure the SUMMARY.md link points to the new location\n\n---\n\n## Adding a New Device Crate\n\nWhen a new crate is added under `vm/devices/`:\n\n1. **Does it need a Guide page?** If it's a user-visible device or backend,\n yes. Internal plumbing crates (e.g., `*_resources`, `*_protocol`) usually\n don't need their own page but should be mentioned in the parent device page.\n\n2. **Create the page** following the \"Adding a New Guide Page\" procedure above.\n\n3. **Update the \"What to Flag\" list** if the new crate introduces a new\n category of reviewable change.\n\n---\n\n## Adding or Changing CLI Arguments\n\nWhen `openvmm/openvmm_entry/src/cli_args.rs` changes:\n\n1. Update `Guide/src/reference/openvmm/management/cli.md`\n2. The CLI page has a disclaimer that it may be out of date — keep it honest,\n but still update the page when making changes\n3. New arguments should include: flag syntax, description, default value,\n and any interactions with other flags\n\n---\n\n## Auditing Guide Freshness\n\nTo check whether the Guide is in sync with the code:\n\n1. **Scan the Guide pages** in `Guide/src/`\n2. Check whether the Guide page content still matches the code:\n - Crate names still correct?\n - Struct/enum names still exist?\n - Behavioral descriptions still accurate?\n - Links to rustdoc still resolve?\n3. Check `Guide/src/SUMMARY.md` for placeholder links `()` — these are\n topics that need content. Prioritize ones whose code areas have been\n actively developed.\n\n### Quick audit commands\n\n```bash\n# Find Guide pages that reference a crate name\ngrep -r \"crate_name\" Guide/src/\n\n# Find placeholder topics (empty links) in SUMMARY.md\ngrep '()\\s*$' Guide/src/SUMMARY.md\n```\n\n\n---\nname: openvmm-ci-investigation\ndescription: Investigate CI failures on OpenVMM PRs. Load when a PR has failing CI checks, you need to download and analyze test artifacts, or you need to diagnose build, fmt, clippy, or VMM test failures.\n---\n\n# Investigating CI Failures\n\nWhen a PR has failing CI checks, **always start by running the investigation\nscript**. Do not manually query the GitHub API or download artifacts by hand\n— the script handles all of that automatically.\n\n## Step 1: Run the Script\n\n```bash\n# Investigate the current branch's PR (no argument needed)\npython3 repo_support/investigate_ci.py\n\n# Investigate a specific PR by number\npython3 repo_support/investigate_ci.py 2946\n\n# Or by run ID directly\npython3 repo_support/investigate_ci.py 23017249697\n```\n\nThe script automatically:\n1. Resolves the PR to **every** failed CI run for its head commit. A commit\n can have multiple workflows (e.g. \"OpenVMM PR\" and \"OpenVMM Docs PR\"),\n and each failing one is investigated in turn — so don't assume a single\n run covers everything. Only the newest run of each workflow is used, so\n superseded runs are ignored; a `NOTE: is still in_progress`\n line means results are incomplete and more failures may appear later.\n2. Identifies failed jobs\n3. Downloads `*-unit-tests-junit-xml` artifacts and parses JUnit XML for\n unit test failures\n4. Downloads `*-vmm-tests-logs` artifacts if they exist\n5. Finds tests with `petri.failed` markers and extracts ERROR/WARN lines\n6. If no test or JUnit artifacts exist (build/fmt/clippy failure), shows the\n tail of the failed job's log\n\n## Step 2: Analyze the Script Output\n\nRead the script's output to identify:\n- Which tests failed (unit tests and/or VMM tests)\n- Error messages and root causes\n- Whether it's a build/fmt/clippy failure vs. a test failure\n\nWhen more than one run failed, the output is split by `# RUN n of m` banners.\nCheck **all** of them — a docs failure and a build failure are reported in\nseparate runs.\n\nThen use the information to diagnose the issue and suggest fixes.\n\n## Step 3: Diagnose from Logs and Code\n\n**Always try to diagnose the failure from CI logs, test code, and error\nmessages first.** Most failures can be understood without local reproduction.\nRead the relevant test source, trace the error through the code, and form\na hypothesis before considering local repro.\n\n## Step 4: Reproduce Locally (Only If Needed)\n\nIf you cannot diagnose the failure from logs alone, **ask the user** whether\nthey want to attempt local reproduction before proceeding. Do not\nautomatically start building or running tests on the user's machine.\n\nIf the user agrees, check whether the failing platform matches the host\narchitecture — you can only reproduce tests locally on the same arch. If\nit doesn't match, explain this to the user and continue diagnosing from\nCI logs and test code.\n\nTo reproduce locally, load the `vmm-tests` skill for instructions on\nrunning with `cargo xflowey vmm-tests-run`. Do **not** use\n`cargo nextest run -p vmm_tests` directly — it won't have the required\nartifacts.\n\n## Reference: Manual Commands\n\n> **Only use these if the script fails or you need to dig deeper into a\n> specific artifact.** In normal usage, the script above is sufficient.\n\nIf the script isn't available or you need more control, follow these steps:\n\n### 1. Find the failing run\n\n```bash\n# Get the run ID for a PR\ngh pr checks -R microsoft/openvmm\n# Or list runs for a specific commit\ngh run list -R microsoft/openvmm --commit \n```\n\n### 2. Identify the failing job\n\n```bash\ngh run view -R microsoft/openvmm --json jobs \\\n -q '[.jobs[] | select(.conclusion == \"failure\") | {name, databaseId}]'\n```\n\n### 3. Download test artifacts\n\n**Unit test results** are stored in JUnit XML artifacts named\n`{platform}-unit-tests-junit-xml`. Known platforms include:\n- `x64-linux`\n- `aarch64-linux`\n- `aarch64-linux-musl`\n\n```bash\n# Download unit test JUnit XML for a platform\ngh run download -R microsoft/openvmm \\\n -n aarch64-linux-unit-tests-junit-xml -D /tmp/junit-xml\n# Parse failures from the XML\npython3 -c \"\nimport xml.etree.ElementTree as ET, sys\nfor f in __import__('pathlib').Path(sys.argv[1]).rglob('*.xml'):\n for tc in ET.parse(f).iter('testcase'):\n fail = tc.find('failure')\n if fail is None:\n fail = tc.find('error')\n if fail is not None:\n print(f'FAIL: {tc.get(\\\"classname\\\",\\\"\\\")}::{tc.get(\\\"name\\\",\\\"\\\")}')\n print(f' {fail.get(\\\"message\\\",\\\"\\\")[:200]}')\n\" /tmp/junit-xml\n```\n\n**VMM test results** are stored in artifacts named `{platform}-vmm-tests-logs`.\nThe known platforms are:\n- `x64-windows-intel`\n- `x64-windows-intel-tdx`\n- `x64-windows-amd`\n- `x64-windows-amd-snp`\n- `x64-linux`\n- `aarch64-windows`\n\n```bash\n# Download a specific platform's test logs\ngh run download -R microsoft/openvmm \\\n -n x64-windows-amd-snp-vmm-tests-logs -D /tmp/test-logs\n```\n\n### 4. Find failed tests\n\nEach test gets its own directory inside the artifact. Look for `petri.failed`\nmarker files (passing tests have `petri.passed` instead):\n\n```bash\nfind /tmp/test-logs -name \"petri.failed\"\n```\n\nThe `petri.failed` file contains the error petri recorded for the test. The\ntest's real name is in `petri.test`, which petri writes before the test body\nruns — so a directory with `petri.test` but *no* result marker is a test that\nwas killed or crashed before it could report.\n\n### 5. Extract errors from petri.jsonl\n\nThe `petri.jsonl` file in each test directory is the primary structured log.\nEach line is a JSON object with fields: `timestamp`, `source`, `severity`,\n`message`. Filter for `ERROR` and `WARN` severity for a quick diagnosis:\n\n```bash\npython3 -c \"\nimport json, sys\nfor line in open(sys.argv[1]):\n try:\n e = json.loads(line.strip())\n if e.get('severity') in ('ERROR', 'WARN'):\n print(f'[{e[\\\"severity\\\"]}] {e.get(\\\"source\\\",\\\"?\\\")}: {e.get(\\\"message\\\",\\\"\\\").strip()[:200]}')\n except: pass\n\" /tmp/test-logs//petri.jsonl\n```\n\n## Artifact Contents\n\n### Unit test JUnit XML\n\nArtifacts named `{platform}-unit-tests-junit-xml` contain JUnit XML files\nwith `` elements. Failed tests have `` or ``\nchildren with `message` attributes describing the failure. These are the\nprimary artifacts for diagnosing unit test / cargo-nextest failures.\n\n### VMM test logs (petri)\n\nEach test directory contains:\n- `petri.jsonl` — Structured JSON Lines log **(primary file for investigation)**\n- `petri.log` — Plain text version of the test log\n- `petri.test` — The test's name, written before the test body runs\n- `petri.passed` or `petri.failed` — Pass/fail marker; a failure marker holds the error\n- `openhcl.log` — OpenHCL serial console output, if the test exercised OpenHCL\n- `hyperv.log` — Hyper-V event log, if the test exercises the Hyper-V backend\n- `openvmm.log` — OpenVMM serial console output, if the test exercises the OpenVMM backend\n- `guest.log`, `uefi.log` — Guest OS serial output\n- Sometimes: `screenshot_*.png` — periodic screenshots of the guest\n- Sometimes: `dumpfile.dmp`\n\n## Viewing Results in Browser\n\nTest results are uploaded to Azure Blob Storage and viewable at:\n`https://openvmm.dev/test-results/#/runs/_`\n\nLogs are uploaded per re-run attempt, so the run key includes the attempt\nnumber (`1` for the first attempt). `investigate_ci.py` prints the correct\nURL in its summary.\n\n## Common Failure Patterns\n\n- **Unit test failure**: A `unit tests` job failed. The script downloads\n JUnit XML artifacts and shows the failing test names and messages. Common\n causes: new test code that relies on OS capabilities not available in CI\n (e.g. TAP devices, elevated permissions).\n- **Formatting / house-rules**: The `quick check [fmt, clippy x64-linux]` job\n failed. No test artifacts will exist. Run `cargo xtask fmt --fix` locally\n and check the job log for the specific rule that failed.\n- **Docs failure**: A job in the separate \"OpenVMM Docs PR\" workflow failed —\n either `build mdbook guide` (broken Guide markdown/links; mdbook logs\n `[ERROR] (mdbook::...)`) or `build and check docs` (rustdoc warnings, which\n are denied in CI). Reproduce with `cargo doc --no-deps -p `.\n No test artifacts will exist.\n- **TripleFault**: VM hit a fatal error during boot. Check `petri.jsonl` for\n Hyper-V Worker/Chipset errors. Often infrastructure-related, not caused by\n the PR's code changes.\n- **Timeout**: Test exceeded its time limit. Check if the VM booted at all.\n- **Guest assertion failure**: Guest-side test failed. Check `guest.log`.\n- **Build failure**: No test artifacts will exist. Check the job log directly\n with `gh run view --job --log`.\n\n## Important API Note\n\nThe `gh run view --json artifacts` flag does **not** exist. To list artifacts\nfor a run, use the GitHub API directly:\n\n```bash\ngh api repos/microsoft/openvmm/actions/runs//artifacts\n```\n\n\n---\nname: vmm-tests\ndescription: \"Run VMM tests locally with cargo xflowey vmm-tests-run. Load when running or debugging VMM tests, or when you need to understand the petri test framework, artifact handling, or cross-compilation for VMM tests.\"\n---\n\n# Running VMM Tests\n\nVMM tests boot full virtual machines and validate behavior. They live in\n`vmm_tests/vmm_tests/tests/tests/` and use the `petri` test framework.\n\n**Always use `cargo xflowey vmm-tests-run`** — never raw `cargo nextest run -p\nvmm_tests`. The xflowey command handles artifact discovery, dependency\nbuilding, and test execution automatically.\n\n## Quick Start\n\n```bash\n# Run a specific test\ncargo xflowey vmm-tests-run --filter \"test(my_test_name)\"\n\n# Run all tests matching a prefix\ncargo xflowey vmm-tests-run --filter \"test(/^boot_/)\"\n\n# Run all tests (rarely needed locally)\ncargo xflowey vmm-tests-run --filter \"all()\"\n```\n\nUsually you don't need `--dir` — it defaults to `target/vmm_tests`. When\ncross-compiling for Windows from WSL2, `--dir` on the Windows filesystem (e.g.,\n`--dir /mnt/d/vmm_tests`) is required **only** when the selected tests use disk\nimages that need a Windows filesystem: **Hyper-V tests**, or tests using **VHDX\n/ dynamic VHD1** images. For everything else (streamed disks, or plain fixed\nVHD1 / VMGS / ISO files), the default WSL-side directory works. `vmm-tests-run`\ndetects this and errors with a clear message if `--dir` is required.\n\n## Filter Syntax\n\nFilters use [nextest filter expressions](https://nexte.st/docs/filtersets/):\n\n| Expression | Matches |\n|-----------|---------|\n| `test(foo)` | Tests with `foo` in the name |\n| `test(/^boot_/)` | Tests starting with `boot_` (regex) |\n| `test(foo) & !test(hyperv)` | `foo` tests excluding Hyper-V variants |\n| `all()` | Everything |\n\n## Platform Targeting\n\nBy default, tests build for the current host. Use `--target` for\ncross-compilation:\n\n```bash\n# Cross-compile and run Windows tests from WSL2\ncargo xflowey vmm-tests-run --target windows-x64\n```\n\n| Target | Description |\n|--------|-------------|\n| `windows-x64` | Windows x86_64 (Hyper-V / WHP) |\n| `windows-aarch64` | Windows ARM64 (Hyper-V / WHP) |\n| `linux-x64` | Linux x86_64 |\n\n**Windows from WSL2**: The default output directory works for most tests. Add\n`--dir` on the Windows filesystem (e.g., `--dir /mnt/d/vmm_tests`) only when the\nselected tests use disk images requiring a Windows filesystem (Hyper-V tests,\nor VHDX / dynamic VHD1 images); `vmm-tests-run` errors with a clear message when\nit's needed. Cross-compilation setup is required first — see\n`Guide/src/dev_guide/getting_started/cross_compile.md`.\n\n## Artifact Handling (Lazy Fetch)\n\nBy default, disk images (VHDs/ISOs) are streamed on demand via HTTP with local\nSQLite caching. This avoids multi-GB upfront downloads.\n\n- `--no-lazy-fetch` — download all images upfront instead of streaming\n- Lazy fetch is automatically disabled for Hyper-V tests (they need local files)\n- `--skip-vhd-prompt` — skip interactive VHD download prompts (useful for\n automation)\n\n## Viewing Logs\n\nTest output (petri logs, guest serial, etc.) is shown by default on failure.\nFor full OpenVMM tracing, set the `OPENVMM_LOG` environment variable:\n\n```bash\nOPENVMM_LOG=trace cargo xflowey vmm-tests-run --filter \"test(foo)\"\n```\n\n## Other Useful Flags\n\n| Flag | Purpose |\n|------|---------|\n| `--release` | Release build (default: debug) |\n| `--build-only` | Build without running |\n| `--verbose` | Verbose cargo output |\n| `--install-missing-deps` | Auto-install missing system dependencies |\n| `--custom-uefi-firmware ` | Use a custom UEFI firmware (MSVM.fd) |\n| `--custom-kernel ` | Use a custom kernel image |\n\nRun `cargo xflowey vmm-tests-run --help` for the full option list.\n\n## Common Pitfalls\n\n- **Don't use `cargo nextest run -p vmm_tests` directly** — artifacts won't\n be present and tests will fail with missing-artifact errors.\n- **Windows output dir from WSL** — when the selected tests need a Windows\n filesystem (Hyper-V tests, or VHDX / dynamic VHD1 images), `--dir` must be on\n `/mnt/c/` or `/mnt/d/`, not in the WSL filesystem. Other tests use the\n default.\n- **Hyper-V tests** — require Hyper-V Administrators group membership and\n disable lazy fetch automatically.\n- **CI failures** — use the `openvmm-ci-investigation` skill to diagnose\n failing VMM tests in CI, not this workflow.\n\n\n# OpenVMM Repository\n\nOpenVMM is a modular, cross-platform Virtual Machine Monitor (VMM) written in Rust.\nThis repository is home to both OpenVMM and OpenHCL (a paravisor).\nDocumentation lives in `Guide/` and is published at https://openvmm.dev.\n\n## Build & Setup\n\nRestore required dependencies before building for the first time:\n```bash\ncargo xflowey restore-packages\n```\n\nThe project supports cross-compilation for `x86_64` and `aarch64`.\nOpenHCL can only be built from Linux (WSL2 counts).\nFor cross-compilation from WSL2 to Windows, see\n`Guide/src/dev_guide/getting_started/cross_compile.md` and source\n`. ./build_support/setup_windows_cross.sh`.\n\n## Git Commit Rules\n\n- **Never amend commits that have already been pushed without express\n permission from the user.** Make new commits instead. PRs are\n squash-merged, so a clean history is unnecessary.\n- Rebasing onto `main` to resolve conflicts is fine, but do not use\n `git commit --amend`, `git rebase -i`, or `git push --force` to clean\n up history on already-pushed commits.\n\n## Pre-Commit Checklist (MANDATORY)\n\n**You MUST run these commands before every `git commit` in this repo.\nDo NOT commit without completing all three steps.**\n\n1. `cargo clippy --all-targets -p ` — for each modified package.\n2. `cargo doc --no-deps -p ` — for each modified package.\n3. `cargo xtask fmt --fix` — fix formatting, headers, naming conventions.\n Run this **last** because fixing clippy/doc issues may introduce\n formatting changes that need to be cleaned up.\n\nIf `cargo xtask fmt --fix` still fails after auto-fixes, fix the remaining\nreported issues manually and re-run until it succeeds. Do not run individual\n`--pass` commands afterward.\n\n## Trust Boundaries & Safety\n\nBoth OpenVMM and OpenHCL process data from untrusted sources. Code must\n**never panic** on untrusted input.\n\n- **OpenVMM does not trust the guest** — code must not panic on any guest input\n- **OpenHCL does not trust the root** — code must not panic on any root input\n- **OpenHCL does not trust the VTL0 guest** — the attack surface is subtle and needs human review\n\n**Error handling across trust boundaries:**\n- Use `thiserror` for typed error enums at library/API boundaries and\n protocol-facing code\n- Use `anyhow` with `.context(\"...\")` for application-level plumbing and\n context propagation\n- Never `.unwrap()` or `.expect()` on data that crosses a trust boundary\n- For protocol/hardware enums where unknown values must round-trip without\n panicking, use the `open_enum!` macro instead of a normal Rust `enum`\n- Rate-limit trace events that can be triggered repeatedly by guest\n interactions — use `tracelimit::warn_ratelimited!` (or `error_ratelimited!`,\n `info_ratelimited!`) instead of bare `tracing::warn!` etc.\n\n**Other safety rules:**\n1. Avoid `unsafe` code\n2. Avoid taking new external dependencies, especially those that significantly increase binary size\n3. Several OpenHCL crates (e.g., `minimal_rt`, `openhcl_boot`, `sidecar`,\n `host_fdt_parser`) must support `no_std` builds. In particular,\n `minimal_rt` and `host_fdt_parser` are unconditionally `no_std`, and\n `openhcl_boot` and `sidecar` use `cfg_attr(minimal_rt, no_std, no_main)`.\n Do not introduce `std`-only dependencies or APIs into code that is\n compiled for the `minimal_rt` configuration in these crates\n4. Prefer `assert!` over `debug_assert!` for internal invariants — the\n performance cost is negligible in nearly all code, and catching invariant\n violations in release builds is more valuable. The project follows a\n \"fail fast\" philosophy: crash immediately on broken invariants rather\n than letting the process continue in an undefined state where bugs are\n harder to diagnose. (This does not apply to untrusted input — use error\n handling at trust boundaries, not assertions.)\n\n## Testing\n\nRun tests with cargo-nextest using the `agent` profile, which suppresses\noutput for passing tests and only shows slow/failing tests:\n```bash\ncargo nextest run --profile agent -p \n```\n\n**Do NOT pipe test output to `grep`, `tail`, or other filters.** The `agent`\nprofile already minimizes output. Piping hides failures and makes hangs\ninvisible.\n\nFor VMM test validation during development, use `cargo xflowey vmm-tests-run`:\n```bash\ncargo xflowey vmm-tests-run --filter \"test(my_test_name)\"\n```\nDo not pass `--dir` in most cases. When cross-compiling for Windows from WSL2,\n`--dir` on the Windows filesystem (e.g., `--dir /mnt/d/vmm_tests`) is required\nonly when the selected tests use disk images that need a Windows filesystem\n(Hyper-V tests, or VHDX / dynamic VHD1 images); `vmm-tests-run` detects this and\nerrors with a clear message when it's needed. For native host tests, and for\nWindows-from-WSL2 tests that stream disks or use fixed VHD1 / VMGS / ISO images,\nit is unnecessary.\n\nThis automatically discovers artifacts, builds dependencies, and runs tests.\nSee `Guide/src/dev_guide/tests/vmm.md` for details.\n\n- **Unit tests** — spread throughout crates in `#[cfg(test)]` blocks.\n Should be fast, isolated, and not require root/administrator access.\n Add `use test_with_tracing::test;` in test modules so that `tracing`\n is initialized and traces appear in test output.\n- **VMM tests** — integration tests in `vmm_tests/` using the petri\n framework (requires additional setup).\n- Mark tests requiring special setup with `#[ignore]`.\n- Update `Guide/` docs when adding features or changing behavior\n (see `.github/instructions/doc-code-sync.instructions.md` for the mapping)\n- **CI failures** — to investigate failing CI checks on a PR, load the\n `openvmm-ci-investigation` skill.\n\n## Rust Edition\n\nThis project uses the **Rust 2024 edition** (`edition = \"2024\"` in root\n`Cargo.toml`).\n\n## Common Pitfalls\n\n- **`guest_arch` not `target_arch`**: For guest-architecture-specific code,\n use `cfg(guest_arch = \"x86_64\")` — **not** `cfg(target_arch = \"x86_64\")`.\n The VMM can run guests of a different architecture than the host. Using\n `target_arch` will fail CI.\n- **Workspace dependencies**: All dependency versions are centralized in\n the root `Cargo.toml`. In crate `Cargo.toml` files, use\n `dep_name.workspace = true` — not inline version specifiers. Add new\n dependencies to `[workspace.dependencies]` in the root first.\n- **Pipeline YAMLs are auto-generated**: Files under `ci-flowey/` (e.g.,\n `openvmm-pr.yaml`) are generated by the flowey framework — do not\n hand-edit them. Run `cargo xflowey regen` to regenerate.\n- **flowey nodes**: Use `flowey::shell_cmd!` and `rt.sh` inside flowey\n nodes — not `xshell::cmd!` or `xshell::Shell::new`.\n\n## Autonomous Agent Inner Loop\n\nWhen running as a coding agent (GitHub Copilot coding agent or similar),\nfollow this validation loop **before pushing each commit**. This covers\nthe common early CI failures (including the fmt + clippy checks from job0)\nlocally, avoiding slow push-and-wait cycles.\n\n1. **Identify modified packages.** For each file you changed, find the\n crate's `Cargo.toml` and note the package name.\n2. **Check compilation:** `cargo check -p ` — fast type-check.\n3. **Clippy:** `cargo clippy --all-targets -p ` — lint.\n4. **Doc:** `cargo doc --no-deps -p ` — catch doc errors.\n5. **Unit tests:** `cargo nextest run --profile agent -p ` — run the crate's\n tests. If nextest is not installed, use `cargo test -p `.\n6. **Formatting:** `cargo xtask fmt --fix` — run last, since earlier\n fixes may introduce formatting changes.\n\nIf any step fails, fix the issue and re-run from that step. Do not push\nuntil all six steps pass.\n\n**Cost notes:** Steps 2–5 are scoped to the modified package (`-p`),\nso they are fast even in this large workspace. Step 6 runs workspace-wide\nbut is also fast. The full cycle typically takes under 2 minutes for\na single-crate change.\n"}