{"owner":"vllm-project","repo":"vllm","hasSkills":true,"totalSkillsCount":220,"totalTokensCount":81404,"categories":["root-instruction","anthropic-skill","claude-rule","plugin-manifest"],"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md",".agents/skills/kernel-microbenchmark/SKILL.md",".agents/skills/ci-fails-buildkite/SKILL.md",".agents/skills/check-api-compat/SKILL.md",".agents/skills/debug-ima/SKILL.md",".agents/skills/model-recipes/SKILL.md",".agents/skills/pr-checklist/SKILL.md",".agents/skills/transformers-upgrade/SKILL.md",".agents/skills/triton-kernel-writing/SKILL.md","rust/AGENTS.md",".agents/skills/model-microbenchmark/SKILL.md","tests/AGENTS.md",".claude/skills/check-api-compat",".claude/skills/ci-fails-buildkite",".claude/skills/debug-ima",".claude/skills/kernel-microbenchmark",".claude/skills/model-microbenchmark",".claude/skills/model-recipes",".claude/skills/pr-checklist",".claude/skills/transformers-upgrade",".claude/skills/triton-kernel-writing","tools/ep_kernels/README.md","tools/pre_commit/.markdownlint.yaml","tools/recipes/README.md","tools/recipes/REFERENCE.md","tools/recipes/RUNTIME_TUNING.md","tools/recipes/SWEEP_TUNING.md","tools/recipes/VISUALIZATION.md","docs/models/extensions/fastsafetensor.md","docs/models/extensions/instanttensor.md","docs/models/extensions/runai_model_streamer.md","docs/models/extensions/tensorizer.md","tests/evals/gpt_oss/README.md","tests/evals/gsm8k/README.md","tests/evals/mrcr/README.md","tests/evals/qwen4_exp/README.md","tools/profiler/nsys_profile_tools/README.md","tools/profiler/nsys_profile_tools/vllm_engine_model.json",".agents/skills/check-api-compat/agents/openai.yaml",".agents/skills/debug-ima/agents/openai.yaml",".agents/skills/kernel-microbenchmark/agents/openai.yaml",".agents/skills/model-recipes/agents/openai.yaml",".agents/skills/transformers-upgrade/agents/openai.yaml",".agents/skills/triton-kernel-writing/agents/openai.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-baseline.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-flashinfer-mxfp4-bf16-cutlass.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-flashinfer-mxfp4-bf16-trtllm.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-flashinfer-mxfp4-mxfp8-cutlass.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-marlin.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-rocm-baseline.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-rocm-quark-mxfp4-bf16-aiter.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-rocm-quark-mxfp4-bf16-triton.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-rocm-quark-mxfp4-fp8-triton.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-sm100-fi-mxfp4-mxfp8-trtllm.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-sm120.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-xpu-baseline.yaml","tests/evals/gpt_oss/configs/gpt-oss-20b-xpu-triton-attn.yaml","tests/evals/gsm8k/configs/DeepSeek-R1-DP.yaml","tests/evals/gsm8k/configs/DeepSeek-R1-DP_MI355.yaml","tests/evals/gsm8k/configs/DeepSeek-R1-TP.yaml","tests/evals/gsm8k/configs/DeepSeek-R1-TP_MI355.yaml","tests/evals/gsm8k/configs/DeepSeek-V2-Lite-Instruct-FP8-TP1-PCP2-ROCm.yaml","tests/evals/gsm8k/configs/DeepSeek-V2-Lite-Instruct-FP8.yaml","tests/evals/gsm8k/configs/DeepSeek-V3.2-DP.yaml","tests/evals/gsm8k/configs/DeepSeek-V3.2-DP_MI355.yaml","tests/evals/gsm8k/configs/DeepSeek-V3.2-TP.yaml","tests/evals/gsm8k/configs/DeepSeek-V3.2-TP_MI355.yaml","tests/evals/gsm8k/configs/DeepSeek-V4-Flash-DSpark-AITER-TEP4.yaml","tests/evals/gsm8k/configs/DeepSeek-V4-Flash-DSpark-FP8-TP4-ROCm.yaml","tests/evals/gsm8k/configs/DeepSeek-V4-Flash-DSpark-confidence-TEP4.yaml","tests/evals/gsm8k/configs/DeepSeek-V4-Flash-EPLB-TEP4.yaml","tests/evals/gsm8k/configs/DeepSeek-V4-Flash-NVFP4.yaml","tests/evals/gsm8k/configs/DeepSeek-V4-Pro-NVFP4.yaml","tests/evals/gsm8k/configs/DeepSeek-V4.1-Flash-NVFP4-DSpark-TP4.yaml","tests/evals/gsm8k/configs/DiffusionGemma-26B-A4B-it-FP8-dynamic.yaml","tests/evals/gsm8k/configs/GLM-5.2-NVFP4-HiSparse.yaml","tests/evals/gsm8k/configs/GLM-5.2-NVFP4-TP1-PCP4-DCP4-EP.yaml","tests/evals/gsm8k/configs/GLM-5.2-NVFP4-TP1-PCP4-EP.yaml","tests/evals/gsm8k/configs/GLM-5.2-NVFP4-TP2-PCP2-EP.yaml","tests/evals/gsm8k/configs/Kimi-K3-pruned75-DSpark-AITER-TP4.yaml","tests/evals/gsm8k/configs/Kimi-K3-pruned75-DSpark-TP4.yaml","tests/evals/gsm8k/configs/Laguna-XS-2.1-INT4.yaml","tests/evals/gsm8k/configs/Laguna-XS.2-NVFP4.yaml","tests/evals/gsm8k/configs/Llama-3-8B-Instruct-nonuniform-CT.yaml","tests/evals/gsm8k/configs/Llama-3.2-1B-Instruct-INT8-CT.yaml","tests/evals/gsm8k/configs/Nemotron-3-Super-120B-A12B-BF16.yaml","tests/evals/gsm8k/configs/Nemotron-3-Super-120B-A12B-BF16_MI355.yaml","tests/evals/gsm8k/configs/Nemotron-3-Super-120B-A12B-FP8.yaml","tests/evals/gsm8k/configs/Nemotron-3-Super-120B-A12B-FP8_MI355.yaml","tests/evals/gsm8k/configs/Nemotron-3-Super-120B-A12B-NVFP4-MI355.yaml","tests/evals/gsm8k/configs/Nemotron-3-Super-120B-A12B-NVFP4-per-token.yaml","tests/evals/gsm8k/configs/Nemotron-3-Super-120B-A12B-NVFP4.yaml","tests/evals/gsm8k/configs/Qwen1.5-MoE-A2.7B-Chat-INT8.yaml","tests/evals/gsm8k/configs/Qwen1.5-MoE-W4A16-CT.yaml","tests/evals/gsm8k/configs/Qwen2.5-VL-3B-Instruct-FP8-dynamic.yaml","tests/evals/gsm8k/configs/Qwen3-0.6B-FP8.yaml","tests/evals/gsm8k/configs/Qwen3-1.7B-MXFP4.yaml","tests/evals/gsm8k/configs/Qwen3-30B-A3B-MXFP4-AITER-TP2-online.yaml","tests/evals/gsm8k/configs/Qwen3-30B-A3B-MXFP4-AITER-TP2.yaml","tests/evals/gsm8k/configs/Qwen3-30B-A3B-MXFP4A16.yaml","tests/evals/gsm8k/configs/Qwen3-30B-A3B-NVFP4-quark.yaml","tests/evals/gsm8k/configs/Qwen3-30B-A3B-NVFP4.yaml","tests/evals/gsm8k/configs/Qwen3-30B-A3B-Thinking-2507-FP8.yaml","tests/evals/gsm8k/configs/Qwen3-30B-A3B-Thinking-2507-PTPC-FP8.yaml","tests/evals/gsm8k/configs/Qwen3-4B-FP8-DSpark-watermark-TP2.yaml","tests/evals/gsm8k/configs/Qwen3-4B-TQ-k3v4nc-ROCm.yaml","tests/evals/gsm8k/configs/Qwen3-4B-TQ-k3v4nc.yaml","tests/evals/gsm8k/configs/Qwen3-4B-TQ-k8v4-ROCm.yaml","tests/evals/gsm8k/configs/Qwen3-4B-TQ-k8v4.yaml","tests/evals/gsm8k/configs/Qwen3-4B-TQ-t3nc-ROCm.yaml","tests/evals/gsm8k/configs/Qwen3-4B-TQ-t3nc.yaml","tests/evals/gsm8k/configs/Qwen3-4B-TQ-t4nc-ROCm.yaml","tests/evals/gsm8k/configs/Qwen3-4B-TQ-t4nc.yaml","tests/evals/gsm8k/configs/Qwen3-Next-80B-A3B-NVFP4-EP2-CT-MI355.yaml","tests/evals/gsm8k/configs/Qwen3-Next-80B-A3B-NVFP4-EP2-MTP-MI355.yaml","tests/evals/gsm8k/configs/Qwen3-Next-80B-A3B-NVFP4-EP2.yaml","tests/evals/gsm8k/configs/Qwen3-Next-FP8-EP2.yaml","tests/evals/gsm8k/configs/Qwen3-Next-FP8-EP2_MI355.yaml","tests/evals/gsm8k/configs/Qwen3.5-35B-A3B-DEP2.yaml","tests/evals/gsm8k/configs/Qwen3.5-35B-A3B-FP8-DEP2.yaml","tests/evals/gsm8k/configs/Qwen3.5-35B-A3B-MXFP4-AITER-TP2.yaml","tests/evals/gsm8k/configs/Qwen3.5-35B-A3B-MXFP4-EMU-TP2.yaml","tests/evals/gsm8k/configs/Qwen3.5-397B-A17B-NVFP4-DEP2-MTP.yaml","tests/evals/gsm8k/configs/Qwen3.5-397B-A17B-NVFP4-DEP2.yaml","tests/evals/gsm8k/configs/Qwen3.5-4B-watermark.yaml","tests/evals/gsm8k/configs/gemma-4-E4B-it-qat-mobile-ct.yaml","tests/evals/gsm8k/configs/models-small-tp.yaml","tests/evals/gsm8k/configs/models-small.yaml","tests/evals/mrcr/configs/Qwen3.5-4B.yaml","tests/evals/qwen4_exp/configs/Qwen3.8-Flash-Next-FP8-rocm.yaml","tests/evals/qwen4_exp/configs/Qwen3.8-Flash-Next-FP8.yaml","tests/evals/gsm8k/configs/humming/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen2-1.5B-Instruct-FP8W8-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen2-1.5B-Instruct-FP8W8-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-0.6B-MXFP8-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-0.6B-MXFP8-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-AWQ-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-AWQ-humming-act-int8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-AWQ-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-FP8-block-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-FP8-block-humming-deepep-ht-grouped.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-FP8-block-humming-deepep-ht-indexed.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-FP8-block-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-Fp8-v1-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-Fp8-v1-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-GPTQ-Int4-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-GPTQ-Int4-humming-act-int8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-GPTQ-Int4-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-Instruct-2507-quantized.w8a8-humming-act-int8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-Instruct-2507-quantized.w8a8-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-MXFP4A16-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-MXFP4A16-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-30B-A3B-NVFP4-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-4B-mixed-quant-RTN-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3-4B-mixed-quant-RTN-no-w4a4-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-35B-A3B-FP8-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-35B-A3B-FP8-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-35B-A3B-experts-int8-humming-act-int8.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-35B-A3B-experts-int8-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-35B-A3B-int6wc-hadamard-humming-act-int8.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-35B-A3B-int6wc-hadamard-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-4B-quantized.w4a16-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-4B-quantized.w4a16-humming-act-int8.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-4B-quantized.w4a16-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-9B-int6wc-hadamard-humming-act-int8.yaml","tests/evals/gsm8k/configs/humming/Qwen3.5-9B-int6wc-hadamard-humming.yaml","tests/evals/gsm8k/configs/humming/Qwen3.6-35B-A3B-NVFP4-humming.yaml","tests/evals/gsm8k/configs/humming/gpt-oss-20b-humming-act-fp8.yaml","tests/evals/gsm8k/configs/humming/gpt-oss-20b-humming.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Llama-4-Scout-Fp8-ModelOpt-triton.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-BF16-moonep.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-BF16-triton.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-Fp8-AutoFp8-deepgemm-deepep-ht.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-Fp8-AutoFp8-deepgemm-deepep-ll.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-Fp8-AutoFp8-deepgemm.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-Fp8-CT-Block-deepgemm-deepep-ht.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-Fp8-CT-Block-deepgemm-deepep-ll.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-Fp8-CT-Block-deepgemm.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-NvFp4-CT-fi-cutedsl-deepep-ll.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-NvFp4-CT-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-NvFp4-ModelOpt-fi-cutedsl-deepep-ll.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-NvFp4-ModelOpt-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor-dp-ep/Qwen3-30B-A3B-NvFp4-ModelOpt-fi-trtllm.yaml","tests/evals/gsm8k/configs/moe-refactor/DeepSeek-V4-Flash-deepgemm-mega-moe.yaml","tests/evals/gsm8k/configs/moe-refactor/Llama-4-Scout-BF16-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Llama-4-Scout-BF16-triton.yaml","tests/evals/gsm8k/configs/moe-refactor/Llama-4-Scout-Fp8-CT-vllm-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Llama-4-Scout-Fp8-ModelOpt-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Llama-4-Scout-Fp8-ModelOpt-fi-trtllm.yaml","tests/evals/gsm8k/configs/moe-refactor/Llama-4-Scout-Fp8-ModelOpt-marlin.yaml","tests/evals/gsm8k/configs/moe-refactor/Llama-4-Scout-Fp8-ModelOpt-triton.yaml","tests/evals/gsm8k/configs/moe-refactor/Mixtral-8x7B-BF16-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Mixtral-8x7B-BF16-triton.yaml","tests/evals/gsm8k/configs/moe-refactor/Mixtral-8x7B-Fp8-AutoFp8-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Mixtral-8x7B-Fp8-AutoFp8-triton.yaml","tests/evals/gsm8k/configs/moe-refactor/Nemotron-Nano-30B-Fp8-ModelOpt-fi-trtllm.yaml","tests/evals/gsm8k/configs/moe-refactor/Nemotron-Nano-30B-NvFp4-ModelOpt-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Nemotron-Nano-30B-NvFp4-ModelOpt-vllm-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-BF16-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-BF16-triton-rocm.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-BF16-triton.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-AutoFp8-deepgemm.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-AutoFp8-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-AutoFp8-fi-trtllm.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-AutoFp8-marlin.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-AutoFp8-triton.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-CT-Block-deepgemm.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-CT-Block-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-CT-Block-marlin.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-CT-Block-triton.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-CT-Channel-marlin.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-Fp8-CT-Channel-vllm-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-NvFp4-CT-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-NvFp4-CT-fi-trtllm.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-NvFp4-CT-marlin.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-NvFp4-CT-vllm-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-NvFp4-ModelOpt-fi-cutlass.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-NvFp4-ModelOpt-fi-trtllm.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-NvFp4-ModelOpt-marlin.yaml","tests/evals/gsm8k/configs/moe-refactor/Qwen3-30B-A3B-NvFp4-ModelOpt-vllm-cutlass.yaml"],"skills":{"AGENTS.md":"# Agent Instructions for vLLM\n\n> These instructions apply to **all** AI-assisted contributions to `vllm-project/vllm`.\n> Breaching these guidelines can result in automatic banning.\n\n## 1. Contribution Policy (Mandatory)\n\n### Duplicate-work checks\n\nBefore proposing a PR, run these checks:\n\n```bash\ngh issue view --repo vllm-project/vllm --comments\ngh pr list --repo vllm-project/vllm --state open --search \" in:body\"\ngh pr list --repo vllm-project/vllm --state open --search \"\"\n```\n\n- If an open PR already addresses the same fix, do not open another.\n- If your approach is materially different, explain the difference in the issue.\n\n### No low-value busywork PRs\n\nDo not open one-off PRs for tiny edits (single typo, isolated style change, one mutable default, etc.). Mechanical cleanups are acceptable only when bundled with substantive work.\n\n### Accountability\n\n- Pure code-agent PRs are **not allowed**. A human submitter must understand and defend the change end-to-end.\n- The submitting human must review every changed line and run relevant tests.\n- PR descriptions for AI-assisted work **must** include:\n - Why this is not duplicating an existing PR.\n - Test commands run and results.\n - Model evaluation results when the change affects output, accuracy, or serving.\n - Clear statement that AI assistance was used.\n- User-visible changes (features, CLI args, config fields, env vars, models,\n hardware, defaults, deprecations) must update the rendered docs in the same PR;\n follow [Documenting User-Visible Changes](docs/contributing/README.md#documenting-user-visible-changes).\n- Before opening a PR (drafts included) or requesting re-review, run the [`pr-checklist`](.agents/skills/pr-checklist/SKILL.md) skill and address its findings.\n\n### Fail-closed behavior\n\nIf work is duplicate/trivial busywork, **do not proceed**. Return a short explanation of what is missing.\n\n---\n\n## 2. Development Workflow\n\n- **Never use system `python3` or bare `pip`/`pip install`.** All Python commands must go through `uv` and `.venv/bin/python`.\n\n### Environment setup\n\n```bash\n# Install `uv` if you don't have it already:\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Always use `uv` for Python environment management:\nuv venv --python 3.12\nsource .venv/bin/activate\n\n# Always make sure `pre-commit` and its hooks are installed:\nuv pip install -r requirements/lint.txt\npre-commit install\n```\n\n### Installing dependencies\n\n```bash\n# Start with precompiled artifacts for an editable install:\nVLLM_USE_PRECOMPILED=1 uv pip install -e . --torch-backend=auto\n```\n\nFor C/C++ or CUDA changes, follow the\n[incremental compilation workflow](docs/contributing/incremental_build.md) to\nconfigure and perform incremental builds.\n\n### Tests\n\n> Requires [Environment setup](#environment-setup) and [Installing dependencies](#installing-dependencies).\n\n```bash\n# Install test dependencies (use cuda.in on non-x86_64):\nuv pip install -r requirements/test/cuda.in\n\n# Run a specific test file:\n.venv/bin/python -m pytest tests/path/to/test_file.py -v\n```\n\nWhen adding tests:\n\n- **Design before you write.** Answer four questions first: what is the module\n for, what is its I/O contract, what failure am I guarding against, and what is\n the cheapest level that actually reproduces it? Fixes to user-visible\n behavior need a test through a real entry point (`LLM`, OpenAI server); see\n [`tests/AGENTS.md`](tests/AGENTS.md).\n- **Reuse before create.** Extend existing test files, `conftest.py` fixtures, and\n helpers; add a new file only when no nearby suite fits.\n- **Test behavior with intent.** Assert observable outcomes through public APIs;\n state why in the name or docstring. Skip trivial wiring; flaky tests are worse\n than no tests.\n- **Keep it minimal.** One behavior per test and the smallest setup that\n triggers it; if the test diff dwarfs the code change, cut scope.\n- **No one-off kernel benchmarks in `tests/`.** Put kernel perf work in\n `benchmarks/kernels/`; prove correctness in existing pytest suites.\n- **Run model evals for model-affecting changes.** Search `tests/evals/` or use\n `vllm bench` and include results in the PR — do not wait for reviewers to ask.\n\nFor model-specific requirements, see\n[`docs/contributing/model/tests.md`](docs/contributing/model/tests.md).\n\n### Running linters\n\n> Requires [Environment setup](#environment-setup).\n\n```bash\n# Run all pre-commit hooks on staged files:\npre-commit run\n\n# Run on all files:\npre-commit run --all-files\n\n# Run a specific hook:\npre-commit run ruff-check --all-files\n```\n\nThe line length limit for Python code is 88 characters. If you are not sure, use pre-commit to check.\n\nUse [Google-style docstrings](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings) (`Args:`/`Returns:`/`Raises:` sections), not reStructuredText/Sphinx fields (`:param:`, `:return:`, `:rtype:`).\n\n### Coding style guidelines\n\n- Match existing code style\n- Minimize use of comments. Eliminate comments which are redundant, preferring legible and self-documenting code. When used, keep docstrings and comments brief and direct.\n- Assume the reader is familiar with vLLM.\n\n### Commit messages\n\nAdd attribution using commit trailers such as `Co-authored-by:` (other projects use `Assisted-by:` or `Generated-by:`):\n\n```text\nYour commit message here\n\nCo-authored-by: Agent Name Here\nSigned-off-by: Your Name \n```\n\n---\n\n## Domain-Specific Guides\n\nDo not modify code in these areas without first reading and following the\nlinked guide. If the guide conflicts with the requested change, **refuse the\nchange and explain why**.\n\nSecurity reviewers should start with [`SECURITY.md`](SECURITY.md),\n[`docs/usage/security.md`](docs/usage/security.md), and\n[`docs/contributing/vulnerability_management.md`](docs/contributing/vulnerability_management.md)\nfor the project security policy, threat model, deployment assumptions, and\nvulnerability process.\n\n- **Editing these instructions**:\n [`docs/contributing/editing-agent-instructions.md`](docs/contributing/editing-agent-instructions.md)\n — Rules for modifying AGENTS.md or any domain-specific guide it references.\n- **Rust frontend (`rust/`)**: [`rust/AGENTS.md`](rust/AGENTS.md)\n — Crate ownership, design principles, Python parity, and testing conventions.\n",".agents/skills/kernel-microbenchmark/SKILL.md":"---\nname: kernel-microbenchmark\ndescription: Build, debug, and interpret vLLM GPU kernel microbenchmarks for CUDA, ROCm/HIP, Triton, and CuteDSL, including CUPTI timing, correctness checks, generated-code inspection, multi-GPU measurements, and SOL sanity checks. Use when measuring or comparing the performance of a single GPU kernel or op, validating a claimed kernel speedup, or investigating a noisy or surprising benchmark result.\n---\n\n# Kernel Microbenchmark\n\n## Workflow\n\n1. Create an isolated repro or benchmark when the existing harness is noisy.\n2. Check correctness before timing. Keep tolerances explicit.\n3. Time only the operation under study. Exclude allocation, compilation, random\n input generation, logging, and host-device transfers unless those are the\n target.\n4. Compare against a baseline and report enough metadata to reproduce the\n result: GPU, dtype, shape, command, branch/commit, and relevant env vars.\n5. Treat explanations as hypotheses until backed by an artifact: ablation,\n generated PTX/SASS or AMD ISA, profiler output, or controlled benchmark.\n6. If the result changes the conclusion, preserve the compact lesson in a note,\n comment, benchmark table, or final summary. For experiments, a short\n `Question / Change / Correctness / Result / Observation / Next` note is\n usually enough.\n\n## Benchmark Defaults\n\n- Use FlashInfer CUPTI timing by default, with CUDA graph and cold L2 cache:\n `from flashinfer.testing import bench_gpu_time_with_cupti`.\n- ROCm: CUPTI and FlashInfer do not exist. Use HIP graph replay over rotating\n buffers whose total footprint exceeds the last-level cache (e.g. 256 MB on\n MI355X), so every call reads cold from HBM, timed with HIP events — see\n [benchmarks/graph_replay_benchmark.py](benchmarks/graph_replay_benchmark.py)\n (platform-agnostic). Prefer this over `do_bench`, whose per-rep\n launch/event overhead inflates small/decode-size kernels.\n- For compute-heavy kernels, report TFLOPS with the FLOP formula in the\n benchmark. For memory-heavy kernels, report estimated bytes moved and GB/s.\n For mixed kernels, report the most honest metric available and call out the\n caveats. TFLOPS and memory bandwidth should be computed from the theoretical\n best for the operation, not from a particular kernel implementation. For\n example, memory bandwidth should assume all data is read exactly once from\n global memory.\n- When comparing across shapes, prefer throughput metrics such as TFLOPS or\n GB/s as the primary table columns; keep latency for absolute cost.\n- If a result exceeds expected peak/SOL, first inspect units, FLOP/byte\n formulas, skipped work, sparsity, caching, and whether the baseline is doing\n the same operation.\n- Force compilation/autotuning before measuring compiled kernels.\n- Tune every arm of a comparison before trusting it. A variant's default or\n inherited config can be far from its optimum, so conclusions drawn from\n untuned variants — especially negative ones — are unreliable. A coarse\n sweep of the cheap knobs at a couple of representative shapes is enough;\n report the winning config with the numbers.\n- Seed inputs when correctness comparisons matter.\n- Keep metadata setup, plan construction, allocation, random input generation,\n and logging outside the timed region unless that overhead is the experiment.\n\n## Sanity-Check Reference Numbers\n\nUse these as rough reference points for large, well-shaped workloads, not as\ngold standards, guaranteed peaks, or hard limits. Hardware SKU, clocks, shape,\nprecision conventions, and the FLOP/byte accounting can move the result. A\nlarge gap is a prompt to investigate, not proof that a kernel is poor.\n\n| Kernel regime | Hardware | Rough reference |\n| --- | --- | ---: |\n| Memory-bound, large batch | Blackwell | 6 TB/s |\n| BF16 GEMM | Blackwell | 2 PFLOP/s |\n| BF16 attention | B200 | 1.6 PFLOP/s |\n| FP8 GEMM | Blackwell | 4 PFLOP/s |\n| FP8 attention | B300 | 2.8 PFLOP/s |\n\nThe BF16 attention reference is approximately the 1613 TFLOP/s result reported\nby the FlashAttention-4 paper. Compare kernels only with matching workload and\nthroughput conventions.\n\n## Multi-GPU Benchmarks\n\n- State whether the run is local or multi-node and report the GPU topology,\n world size, GPUs per node, collective backend, and relevant library versions.\n- Compare like-for-like TP configurations. Report both per-rank and global\n dimensions, and do not compare results from different TP sizes without an\n explicit normalization or scaling question.\n- Define the timed operation boundary before benchmarking. If the production\n wrapper performs input staging, flag resets, generation barriers, padding,\n or output copies, keep them in the timed region for an end-to-end comparison.\n Use a separate, clearly labeled ablation for kernel-only timing.\n- Check distributed correctness before timing. Seed each rank deliberately,\n form the reference with the same collective semantics, and synchronize before\n reading or comparing outputs.\n- Use a device-side barrier immediately before each measured replay. Keep this\n common synchronization outside the timed interval, but keep barriers required\n by the candidate implementation inside it.\n- With CUDA graphs, warm up before capture, coordinate capture across ranks,\n rotate pointer-distinct graphs when cache reuse matters, and ensure every\n collective is issued in the same order on every rank.\n- Measure every rank and reduce each sample with `MAX`; report the median of\n those per-sample maxima. A rank-local event time is not a distributed latency.\n- Reset reusable symmetric-memory flags before each invocation and establish a\n device-side generation barrier before peers may signal them. Otherwise a fast\n rank can signal before a slow rank resets its flags, causing a lost arrival\n and intermittent deadlock.\n- Preserve symmetric-memory handles, CUDA graphs, streams, and graph outputs for\n the full measurement lifetime. Allocate and rendezvous symmetric buffers in\n identical order and with identical shapes on all ranks.\n- Treat hangs and isolated millisecond outliers as synchronization bugs or rank\n skew until disproven. Add stage markers, bounded synchronization checks, and\n per-rank diagnostics before blaming compilation or kernel performance.\n- For multi-node runs, record the scheduler allocation and verify the fabric\n supports the required multicast or NVLink-domain assumptions. Do not describe\n a two-node TP run as equivalent to a local NVLink-domain run without checking.\n- Stabilize GPU clocks or run enough untimed work to reach a steady state.\n Alternate candidate order so clock, thermal, and rank-skew effects are shared.\n- ROCm: capturing ProcessGroupNCCL collectives in HIP graphs trips the NCCL\n watchdog (HIP forbids querying an event recorded in a capturing stream;\n the watchdog aborts the process). `TORCH_NCCL_BLOCKING_WAIT=1` before\n `import torch` disables the watchdog — benchmark-only knob. Ask for backend\n `\"nccl\"`; torch routes it to RCCL automatically (there is no `\"rccl\"`\n backend string).\n\n## Included Examples\n\n- Use [benchmarks/cupti_microbenchmark.py](benchmarks/cupti_microbenchmark.py)\n as a minimal single-GPU FlashInfer CUPTI timing pattern.\n- Use\n [benchmarks/graph_replay_benchmark.py](benchmarks/graph_replay_benchmark.py)\n as a graph-replay timing pattern over rotating buffers sized past the\n last-level cache — the ROCm/HIP default, also usable on CUDA.\n- Use\n [benchmarks/multi_gpu_gemm_rs.py](benchmarks/multi_gpu_gemm_rs.py) as a\n minimal distributed CUDA-graph timing pattern.\n\nAdapt the operation, cases, work formula, and correctness tolerances rather\nthan copying either example unchanged.\n",".agents/skills/ci-fails-buildkite/SKILL.md":"---\nname: ci-fails-buildkite\ndescription: Fetch and diagnose vLLM Buildkite CI failure logs. Use when investigating failing CI jobs on a PR or build, when the user pastes a buildkite.com URL, or asks to fetch/diagnose CI logs.\n---\n\n# Diagnosing vLLM Buildkite CI Failures\n\nBuildkite logs are public; no login needed.\n\n`.buildkite/scripts/ci-fetch-log.sh` saves each log as `ci--.log`, stripped of timestamps and ANSI codes. Existing files are kept; set `CI_FETCH_LOG_FORCE=1` to refetch.\n\n## Fetching logs\n\n```bash\n# All failed jobs in a PR's latest build (current branch's PR if omitted):\n.buildkite/scripts/ci-fetch-log.sh --pr \n\n# All failed jobs in a build (--soft also includes soft-failed jobs;\n# --all fetches every finished job):\n.buildkite/scripts/ci-fetch-log.sh \"https://buildkite.com/vllm/ci/builds/\"\n\n# One job — `gh pr checks` URLs (#) and web UI URLs (?sid=) both\n# work; pass \"-\" as a second argument to stream to stdout:\n.buildkite/scripts/ci-fetch-log.sh \"https://buildkite.com/vllm/ci/builds/#\"\n```\n\nTo clean an already-downloaded log with `.buildkite/scripts/ci-clean-log.sh`:\n\n```bash\n./ci-clean-log.sh ci.log\n```\n\n## Reference\n\nSee [docs/contributing/ci/failures.md](../../../docs/contributing/ci/failures.md) for the full guide: filing CI failure issues, investigating/bisecting, reproducing flaky tests, and daily triage.\n",".agents/skills/check-api-compat/SKILL.md":"---\nname: check-api-compat\ndescription: Determine the correct behavior of vLLM's OpenAI-compatible or Anthropic-compatible API by checking the official upstream spec, instead of guessing from vLLM's own code. Use for any bug, issue, PR, review, or question touching /v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings, /v1/messages, or their request/response models — including \"is this a bug or intended\", \"what should this field default to\", \"does upstream allow this\", and any change that adds, removes, or renames a field on those models. Do not use for vLLM-internal APIs (engine, scheduler, LLM class) or for endpoints with no upstream counterpart.\n---\n\n# Checking vLLM API compatibility against the upstream spec\n\nvLLM's OpenAI and Anthropic server surfaces are *reimplementations* of someone\nelse's spec. vLLM's own code, tests, and docstrings are not evidence of correct\nbehavior — they are the thing under test. Establish what upstream says first,\nthen compare.\n\nNever answer \"what should this field do\" from memory. Field names, types, and\nespecially defaults drift between spec versions.\n\n## 1. Pin down the question\n\nName three things before looking anything up:\n\n- **Which surface** — OpenAI-compatible or Anthropic-compatible. They are separate\n code paths in vLLM and separate specs.\n- **Which endpoint**, by path.\n- **Which field or behavior** — a request field, a response field, a streaming\n event shape, an error code, or a status code.\n\n## 2. Read vLLM's implementation\n\nIn a vLLM checkout, the request/response models live beside each endpoint:\n\n| surface | path |\n| --- | --- |\n| OpenAI chat | `vllm/entrypoints/openai/chat_completion/` |\n| OpenAI completions | `vllm/entrypoints/openai/completion/` |\n| OpenAI responses | `vllm/entrypoints/openai/responses/` |\n| OpenAI embeddings | `vllm/entrypoints/pooling/embed/` |\n| OpenAI shared base models, errors, usage | `vllm/entrypoints/serve/engine/protocol.py` |\n| Anthropic messages | `vllm/entrypoints/anthropic/` |\n| audio | `vllm/entrypoints/speech_to_text/*/protocol.py` |\n\nEach directory holds `protocol.py` (the Pydantic models), `serving.py` (behavior),\nand `api_router.py` (routing and status codes). Read the field declaration *and*\nthe code that consumes it — a field can be declared faithfully and then ignored,\nwhich is still an incompatibility.\n\n## 3. Fetch the upstream spec\n\nPrefer a source that can actually be read as text, in this order.\n\n### OpenAI\n\n1. The OpenAPI spec — authoritative, machine-readable, and fetches cleanly:\n `https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml`\n2. The installed `openai` Python SDK's typed params, which track the spec closely.\n Locate them with `python -c \"import openai,os;print(os.path.dirname(openai.__file__))\"`,\n then read e.g. `types/chat/completion_create_params.py`,\n `types/chat/chat_completion.py`, `types/chat/chat_completion_chunk.py`,\n `types/responses/`. Note the SDK version — an old SDK is an old spec.\n3. The prose reference at `https://platform.openai.com/docs/api-reference/...`\n for semantics the types do not carry. It is script-heavy and often fetches\n poorly; treat a failed fetch as a fetch failure, not as absence of a field.\n\n### Anthropic\n\n1. `https://docs.claude.com/en/api/messages` and\n `https://docs.claude.com/en/api/messages-streaming` — these render as clean\n text and are the primary source for both fields and streaming event order.\n2. The installed `anthropic` SDK's `types/message_create_params.py`,\n `types/message.py`, `types/raw_message_stream_event.py`.\n\nSay which source and which version answered the question, so the finding can be\nre-checked later.\n\n## 4. Compare on all four axes\n\nA field is compatible only when **name**, **type**, **default**, and **semantics**\nall match. Most real bugs hide in the last two: a field that exists with the right\ntype but the wrong default, or one that is accepted and silently dropped.\n\nAlso check, when relevant:\n\n- **Streaming** — chunk shape, the order of events, and the terminal event\n (`data: [DONE]` for Chat Completions and Completions; `response.completed`,\n `response.failed`, or `response.incomplete` with no `[DONE]` for Responses;\n `message_stop` for Anthropic).\n- **Errors** — HTTP status *and* the error body's `type`/`code` strings.\n- **Omitted vs null** — upstream often distinguishes an absent field from an\n explicit `null`. Pydantic defaults erase that distinction unless the model is\n written to preserve it.\n\n## 5. Classify before calling it a bug\n\nThree outcomes, and they are not the same:\n\n- **Matches upstream** — no action.\n- **vLLM extension** — a field upstream has no name for at all (`vllm_xargs`,\n `chat_template_kwargs`, sampling knobs like `min_p`). These are intentional and\n allowed. Do not report them as violations. Check per surface: `top_k` is an\n extension on OpenAI but a real upstream field on Anthropic Messages.\n- **Incompatibility** — a field that *shares a name with upstream* but differs in\n type, default, or behavior, or an upstream field that is missing, ignored, or\n malformed. This is the reportable case.\n\nAn unsupported-but-declared field is its own subcase: decide whether vLLM rejects\nit explicitly or accepts and ignores it, and report which, since silently ignoring\na documented field is the more damaging failure.\n\n## 6. Report\n\nState: the endpoint and field, what upstream specifies (with the source), what\nvLLM does (with `file:line`), which of the three outcomes it is, and the smallest\nfix. When behavior turns on a spec detail that is easy to disbelieve, quote the\nupstream line rather than paraphrasing it.\n",".agents/skills/debug-ima/SKILL.md":"---\nname: debug-ima\ndescription: Debug CUDA illegal memory access (IMA) errors in vLLM with CUDA core dumps and cuda-gdb. Use when a vLLM run crashes or hangs with a CUDA illegal memory access, device-side assert, or other asynchronous CUDA error and the faulting kernel needs to be located.\n---\n\n# Debug IMA\n\nFollow the vLLM guide\n[CUDA Core Dump: An Effective Tool to Debug Memory Access Issues and Beyond](https://vllm.ai/blog/2025-08-11-cuda-debugging).\n",".agents/skills/model-recipes/SKILL.md":"---\nname: model-recipes\ndescription: Find vLLM deployment recipes and configs for LLMs and Hugging Face models by model and hardware. Use for recipe lookup, creation, and updates.\n---\n\n# Model Recipes\n\nModel serving recipes are maintained in this repository's\n[recipes/models/ directory](../../../recipes/models). Omni (image, video,\naudio) recipes stay in vllm-project/recipes for now (moving to vllm-omni),\nand Ascend NPU content is inline in these recipes (moving to vllm-ascend\noverlays). The published catalog, rendered by\n[vllm-project/recipes](https://github.com/vllm-project/recipes), is at\n[recipes.vllm.ai](https://recipes.vllm.ai).\n\n## Usage notes\n\n1. Recipe lookup returns commands and deployment guidance without launching\n `vllm serve`.\n2. Use published configurations or the recipes repository's command generator\n for supported selections. Return commands as published, including their\n flags and parallelism settings.\n3. Recipes are starting points for deployment and may not be tuned for your\n workload. Check each recipe for its verified configurations, and use\n benchmarks of your workload to guide further tuning.\n4. To create or update a recipe, follow the\n [authoring workflow below](#create-or-update-a-recipe).\n\n## Find a recipe\n\nRead recipes from the published JSON on recipes.vllm.ai. It is generated from\nthe source YAML with variant, hardware, and strategy settings already\nresolved; the YAML contains comments and conditional overrides, so do not use\nit for lookup. When presenting results, link the website page. Read the source\nYAML only when creating or updating a recipe.\n\nEach recipe has a page at `https://recipes.vllm.ai//` and JSON at\nthe same path plus `.json`. Paths use the full Hugging Face ID, preserving\ncase; do not substitute `meta.slug` or a display title. For example,\n`Qwen/Qwen3.8-27B` maps to\n[recipes.vllm.ai/Qwen/Qwen3.8-27B](https://recipes.vllm.ai/Qwen/Qwen3.8-27B).\n\nRecipe JSON is often over 20 KB, mostly the `guide`. Instead of printing it,\nread only what the request needs: each step below returns the paths for the\nnext, so stop once the request is answered. Commands need `curl` and `jq`.\n[scripts/read_recipe.sh](scripts/read_recipe.sh) is in this skill's directory;\nthe examples use its path from the vLLM repository root.\n\n### Search the catalog\n\nSkip this step if the exact HF ID is known. Otherwise search by model ID,\nfamily, or provider using case-insensitive keywords. Each\nwhitespace-separated word must occur in the ID, title, or provider:\n\n```bash\ncurl -fsSL https://recipes.vllm.ai/models.json |\n jq --arg q 'qwen 27b' '\n ($q | ascii_downcase | [scan(\"\\\\S+\")]) as $words | .[] |\n ([.hf_id, .title, .provider] | join(\" \") | ascii_downcase) as $text |\n select(all($words[]; . as $word | $text | contains($word))) |\n {hf_id, json, url, derived_from}'\n```\n\n`json` is the recipe data path; `url` is its page path. Prefix both with\n`https://recipes.vllm.ai`. A `derived_from` entry is a variant of a parent\nrecipe: fetch its own `json` to get that checkpoint's command. Its page may\nbelong to the parent, so link the returned `url` rather than assuming every\ncheckpoint has a separate page.\n\nIf there is no exact match, search the family and inspect `variants` before\nconcluding the recipe is missing.\n\n### Summarize the recipe\n\nSave the recipe JSON to a file and summarize it:\n\n```bash\nread_recipe=.agents/skills/model-recipes/scripts/read_recipe.sh\nrecipe=$(mktemp)\ncurl -fsSL https://recipes.vllm.ai/Qwen/Qwen3.8-27B.json -o \"$recipe\"\n\"$read_recipe\" \"$recipe\"\n```\n\n`default` is the recommended hardware, strategy, and variant; `hardware` maps\neach hardware ID to its command path; `recipe_verified_hardware` lists hardware\nwhere the recipe as a whole was verified end to end. For capability\nrequirements, such as tool calling or text-to-video, check `features` and\n`omni_tasks`, then confirm in the guide.\nSome recipes, including Omni, lack `hardware`; skip hardware selection and\nuse the guide's launch and task-specific sections.\n\n### Select hardware and strategy\n\nMatch a hardware ID to the user's GPU and fetch its path from `hardware`,\nprefixed with `https://recipes.vllm.ai`, rather than constructing a URL. That\nresponse is the command object itself: single-node objects carry `command`,\nand multi-node objects carry `head_command` and `worker_commands` instead, so\nlist its keys before selecting fields.\nCheck `hardware_profile` to confirm that the selected configuration matches\nthe user's GPU model and specifications. Its `gpu_count` describes the profile,\nnot necessarily the number of GPUs used by the command. Check the command's\nparallelism settings, `node_count`, and all `worker_commands` against the\nuser's available topology, and compare the variant's `vram_minimum_gb`, an\nestimate, with the total memory of the GPUs the command uses; a configuration\nbelow that minimum is not a working option unless the guide says it was\ntested. Ask when the user's hardware is unclear. If no\npublished command or guide command fits, report the mismatch and return the\nclosest one unchanged; list any adjustments the user's setup would need\nseparately and mark them unverified instead of editing the command.\n\nFor another strategy, follow `alternatives` in the same response. These are\nstatic files; query parameters do not change their configuration. For other\nfeature/topology selections, use the recipes repository's command synthesis.\n\n### Read deployment guidance\n\nRead the guide sections that apply to the selection, such as prerequisites,\nlaunch notes for the selected hardware or variant, troubleshooting, and known\nlimitations. Pass entries from `guide_sections` or prefixes of their text.\nThe script prints every matching section with its subsections and fails when\na section matches nothing:\n\n```bash\n\"$read_recipe\" \"$recipe\" prerequisites troubleshooting\n```\n\nQuery other fields with `jq` on the saved file, such as `features.`,\n`hardware_overrides`, or `omni.tasks` for request examples.\n\n### Return the result\n\nReturn the website link, selected checkpoint and hardware, the requested\ncommand, and relevant prerequisites and caveats from the guide. Pick one\ncommand and copy it, including its Docker form, exactly as the command object\nor guide section gives it; do not combine commands from different sections or\nadd, drop, or reorder flags. List optional changes separately.\n\nFor each command, state its source, either a generated hardware or strategy\nconfiguration or a named guide section, and whether that source says it was\ntested. A `recipe_verified_hardware` entry covers the recipe on that\nhardware, not every variant, strategy, or guide command, so it does not make a\ncommand verified. Generated configurations are not automatically verified.\n\n## Create or update a recipe\n\nRecipes cover official or widely used checkpoints, with configurations verified\nend to end on real hardware. Do not add recipes for arbitrary community\nuploads, untested hardware claims, or non-standard flags that only work for one\nsetup. Before opening a pull request, confirm that the user ran the\nconfiguration and can share the results.\n\nEdit the YAML under [recipes/models/](../../../recipes/models) in this\nrepository, following the recipes repository's\n[add-recipe skill](https://github.com/vllm-project/recipes/blob/main/.agents/skills/add-recipe/SKILL.md)\nfor the schema. Validate and preview it from a checkout of\n`vllm-project/recipes` pointed at this directory, as\n[recipes/README.md](../../../recipes/README.md) describes.\n",".agents/skills/pr-checklist/SKILL.md":"---\nname: pr-checklist\ndescription: Prepare a vLLM change for human PR review or re-review, or analyze an open pull request. Use before opening a PR, drafts included (e.g. `gh pr create`), or pushing review fixes, to check design fit, behavioral coverage, performance evidence, diff quality, and closure of previous feedback.\n---\n\n# PR Checklist\n\nThis SKILL is designed to help systematically enforce vLLM's code quality and contribution standards during the pull request process,\nand to reduce the burden of manual code review on maintainers.\n\n## Usage\n\nThe following checklist can be used to polish a change in preparation for a vLLM pull request, analyze an open pull request (either as the author or as a reviewer), or ensure that all contribution standards are met before requesting maintainer attention. To use this checklist effectively, go through each item and verify that it has been addressed in the diff / pull request.\n\nThis is intended to be used with a human in the loop: when running programmatically, prepare notes on every section to ensure a thorough report, with a clear header summarizing the findings and areas needing attention. When running interactively, present this report to the user with a clear list of actionable follow-up tasks and design decisions to make, then iterate.\n\n### Re-Review\n\nTo update a PR following a review, read all comments and threads on the PR. For each concrete concern, work with your human operator to decide on a plan of action to address each either by a direct reply or by updating the change and acknowledging with a comment. Ensure the PR description remains fresh and all relevant documentation (including comments) in the change is up-to-date.\n\n`gh pr view --comments` does not include inline review comments. Set `pr` to the PR number, then fetch all pages of conversation comments, review submissions, and inline comments (including replies):\n\n```bash\ngh api --paginate \"repos/vllm-project/vllm/issues/$pr/comments\"\ngh api --paginate \"repos/vllm-project/vllm/pulls/$pr/reviews\"\ngh api --paginate \"repos/vllm-project/vllm/pulls/$pr/comments\"\n```\n\nUse `in_reply_to_id` to group inline replies with their parent comments. Use GraphQL `reviewThreads` when resolved or outdated thread status is needed.\n\n## Sections\n\n### 1. Design Fit\n\nDoes the change align with the overall design and architecture of vLLM? Does it introduce any design inconsistencies or anti-patterns? Are there any potential long-term maintenance concerns related to this change?\n\n#### 1.1: Impact on Core Components\n\nvLLM's core components are shared across many different deployments. Changes to these files have high risk and changes should be structured in order to be easily maintainable and minimally invasive.\n\nSuch components include:\n\n- The Model Runner (`vllm/v1/worker/*/model_runner.py`)\n- The Scheduler (`vllm/v1/core/sched/*`)\n- Core infrastructure (`vllm/v1/core/*`, including KV cache managers and utilities)\n\netc.\n\nChanges should be structured to minimize complexity added to these files. Consider refactoring or isolating new functionality to reduce the impact on core components, both in terms of code maintainability and potential for introducing bugs.\n\nNontrivial logic associated with non-standard/non-default functionality should not be added to core/common code paths and files.\n\n#### 1.2: Replication\n\nSome changes may duplicate code that already exists elsewhere in the codebase. Does the change introduce unnecessary replication, or could existing functionality be reused instead? Consider refactoring to reduce duplication and improve maintainability.\n\nConsult the affected area's agent instructions and contribution guides for its reuse conventions. See the [model implementation guide](../../../docs/contributing/model/basic.md) for model code and the [MRV2 development guidance](../../../docs/design/model_runner_v2.md#development-philosophy). Justify any duplication needed to isolate functionality and limit its impact on existing workflows.\n\n#### 1.3: Complexity\n\nChanges which increase the complexity of the codebase should be carefully scrutinized. Specifically those which add new abstractions, dependencies, or branching cases to existing components should be evaluated for their necessity and potential impact on maintainability.\n\nConsider whether the added complexity could be mitigated through refactoring, modularization, or other design improvements. Aim to keep the codebase as simple and understandable as possible while still achieving the desired functionality.\n\nAs a rule of thumb, the complexity of a change can be estimated by the number of new lines of code added to the main codebase:\n\n- Low complexity: < 50 lines. Generally easy to review and unlikely to introduce significant complexity.\n- Medium complexity: 50-250 lines. Requires careful review and consideration of potential impact on maintainability.\n- High complexity: > 250 lines. May introduce significant complexity and requires thorough review and justification.\n\nYou may exclude test cases and documentation updates from this line count, as they typically do not contribute significantly to the complexity of the main change. Note that line count is only a coarse indicator, meant to guide and flag possible overly-complex contributions. Use judgement when assessing the true semantic complexity and implied review burden.\n\nBug fixes should aim to remain below the \"High complexity\" bar. Those which exceed this threshold often warrant additional scrutiny and justification. Redesigning core components, modifying core interfaces, or otherwise introducing significant complexity in order to patch a bug is very often discouraged and should be carefully justified.\n\nA PR that adds significant complexity or a substantial amount of new code must be well justified. A new feature should be relevant to, or requested by, enough users to warrant its size and complexity. A performance optimization should deliver end-to-end improvements on workloads that matter, and those gains should be large enough to outweigh the complexity it adds.\n\n#### 1.4: Correctness and Compatibility\n\nEnsure that all changes to vLLM are sound and do not violate existing contracts. Specifically, verify that all affected deployments which share an execution pathway with the changed code will remain correct. Trace related implementations, alternative backends sharing the same flow, and affected call sites of modified functions.\n\nWhen adding features or changing contracts, ensure that relevant feature compatibility is assessed. Identify any potential incompatibilities or breaking changes and justify them explicitly in the PR notes.\n\nWhen adding functionality, consider parallels with existing designs and identify opportunities to simplify and unify them rather than adding a separate implementation alongside them.\n\n#### 1.5: Tradeoffs\n\nWhen making a design change, adding a feature, or contributing a performance improvement, assess the tradeoffs and potential downsides. When changing a default backend to improve performance as measured on one configuration, could this cause a decrease in performance for another configuration? Could a new architecture change accelerate one workload but cause a feature incompatibility or performance regression in another? Ensure tradeoffs are clearly noted and validation is sufficient to catch potential lurking regressions or drawbacks.\n\nCheck that the change does not introduce CPU/GPU synchronization on the main model execution thread during steady-state inference. Inspect explicit synchronization, host reads of GPU tensors, boolean tensors as indices, and transfers that can block the host. Run the affected configuration with `VLLM_GPU_SYNC_CHECK=error` to confirm a concern; passing CI is insufficient when CI does not exercise that configuration. See the [MRV2 async execution design](../../../docs/design/model_runner_v2.md#2-async-first) for examples.\n\n#### 1.6: Model Runner V1 Deprecation\n\nAccount for Model Runner V1 (MRV1) deprecation when reviewing model-runner changes. Target new GPU model-runner features at Model Runner V2 (MRV2) and follow its [design and development guidance](../../../docs/design/model_runner_v2.md#development-philosophy). For fixes needed by deployments still using MRV1, explain why the legacy path needs the change and check whether MRV2 is also affected.\n\n### 2. Testing and Validation\n\nAll serving-engine changes contributed to vLLM must be _validated_: bug fixes must be tested on a reproducer, and new features or performance optimizations must be run locally and/or benchmarked to ensure correctness and effectiveness. It is unacceptable, for example, to contribute a bugfix patch without running vLLM end-to-end and verifying that it resolves the issue as intended.\n\nBeyond _validation_, many (but not all) changes to vLLM should be _tested_: this includes writing new tests, updating existing tests, and ensuring that the test suite continues to provide reliable coverage of the affected functionality.\n\n#### 2.1: Test Coverage and Effectiveness\n\nWhen adding or modifying tests, consider both the coverage and the effectiveness of the test suite. Coverage refers to the extent to which the codebase is exercised by tests, while effectiveness refers to the ability of the tests to catch regressions and ensure correctness.\n\nAssess whether the included test cases are meaningful, concise, and maintainable. Testing should not be exhaustive: local validation is typically sufficient to ensure a baseline correctness of the change. Contributed tests should focus exclusively on areas that are deemed prone to errors, regressions, or complex interactions within the codebase. Recommend deletion of redundant, trivial, or otherwise unnecessary tests: this is one of the most common complaints raised during code reviews.\n\nCheck added and changed tests against [`tests/AGENTS.md`](../../../tests/AGENTS.md). For reproducible bugs, the PR must show the new test failing without the fix and passing with it.\n\nVerify that testing actually exercises the new path, rather than passing through an unchanged path or fallback. Ensure added speculative decoding correctness tests assert nonzero acceptance rate, selection tests identify the desired backend, etc.\n\nAim to write tests that are meaningful, targeted, and maintainable. Avoid redundant or trivial tests that do not contribute to the overall reliability of the test suite. Avoid using mock objects, especially when highly coupled with internal logic.\n\n#### 2.2: Test Reliability and Robustness\n\nSome tests are considered \"flaky\": they may pass or fail intermittently without any changes to the codebase. Flaky tests can undermine confidence in the test suite and block successful CI runs. Analyze the added tests for potential flakiness and ensure they are reliable and robust.\n\nFrequent sources of flaky tests include:\n\n- Assertions requiring bit-exact output correctness when minor floating-point variation is expected. For example, a change in default kernel selection priority may cause exact-floating-point-correctness assertions to fail.\n- Tests that depend on the order of execution or the state of shared resources, which can lead to intermittent failures when tests are run in parallel or in different environments.\n\n#### 2.3: Test Automation and Integration\n\nvLLM's CI runs in Buildkite, with the launch configuration under `.buildkite/`. Ensure new tests run in an appropriate CI workflow when the required resources are available; add or update a workflow where needed. If CI cannot exercise the changed path because the required hardware or checkpoint is unavailable, document the missing coverage, the resource constraint, and validation performed outside CI. A CI coverage gap does not waive validation: the human submitter must confirm the required validation before publishing the PR.\n\n### 3. Code Quality and Style\n\nCode quality is another of the primary areas that maintainer effort is spent to address. Carefully audit the changes for adherence to coding standards, readability, maintainability, and overall design principles.\n\n#### 3.1: Comments\n\nKeep comments concise and use them sparingly. Consider whether the same information can be conveyed in fewer lines, and prefer clear code with meaningful, concise variable names, and fewer comments overall. Use comments to explain the reasoning behind complex logic, non-obvious decisions, or important context that is not immediately apparent from the code itself. Context regarding specific user decisions made during the design phase are often not needed and can instead be included as notes in the PR description instead.\n\nWhen writing new functions, consider when a docstring may be useful. Most short helper functions do not need a docstring, but more complex functions or those forming part of the public API should include one. In particular, docstrings are useful when adding custom kernels or other performance-optimized implementations of otherwise-straightforward logic, as they make clear the input and output shapes/dtypes/assumptions, and can shed light on hard-to-follow implementations.\n\nFor bug fixes, there is no need to include details of the original bug and explanation of why the pre-changed version of the code would be incorrect. Those details should be in the PR description instead.\n\n#### 3.2: Documentation and Examples\n\nUser-visible changes need docs in the same PR, placed and checked per [Documenting User-Visible Changes](../../../docs/contributing/README.md#documenting-user-visible-changes); missing or unfindable docs are blocking. Add usage examples under `examples/` where applicable.\n\n#### 3.3: Helper Functions\n\nIdentify and inline trivially-simple helper functions which have only one call site.\n\n#### 3.4: Formatting\n\nAvoid formatting or whitespace changes that are unrelated to the PR, as long as pre-commit still passes.\n\n### 4. Pull Request Contents\n\nThis section pertains to the contents of the pull request itself, rather than the changes.\n\nWhen acting as the author, prepare or update these contents. When acting as a reviewer, assess them and report missing context or evidence.\n\nStart from `.github/PULL_REQUEST_TEMPLATE.md`, retaining its sections and checklist. When creating a PR interactively, use `gh pr create --template .github/PULL_REQUEST_TEMPLATE.md`. For noninteractive creation or editing, fill out a copy of that template and pass the completed file with `--body-file`.\n\n#### 4.1: Description and Context\n\nEnsure the PR description has a short 1-2 line summary of the changes to facilitate quick understanding by reviewers. This section should link relevant issues, predecessor/blocking PRs, and provide any necessary context for the changes.\n\nKeep Claims to a few concrete bullets and explain motivation briefly. Keep explanations in Validation and Details concise too, while retaining the evidence, root cause, tradeoffs, and limitations needed to assess the change. Link full logs or use collapsible HTML details for lengthy supporting material rather than polluting the main narrative.\n\n#### 4.2: Claims and Supporting Evidence\n\nEnsure the PR makes clear, concise claims about the accomplishment:\n\n- For bug fixes, what specific issue is being resolved and how can it be verified?\n- For new features, what functionality is being added and how can it be utilized?\n- For refactoring, what was the motivation and what improvements are expected?\n- For performance optimizations, what specific model/hardware/serving configurations & workloads are being targeted and how can the performance gains be measured?\n\nFor the stated claims, ensure the PR provides sufficient evidence, examples, or references to support them, making it easy for reviewers to verify the correctness and impact of the changes.\n\nSummarize model evaluations and benchmarks in compact tables with the configuration, baseline, changed result, and reproduction commands. For repeated benchmarks, report the run count and variability. For kernel performance changes, include before/after profile visualizations/timelines when they help explain the measured gains. Link relevant CI failures or regressing PRs when they motivate the change.\n\n- For bug fixes, ensure the reproducer is clearly provided and was validated to confirm the issue and the resolution.\n- For new features, ensure that usage examples are included and validated to confirm that the feature works as intended.\n- For refactoring, ensure that changes are sufficiently validated at appropriate test levels (unit is sufficient for minor refactors, E2E serving is required for significant refactoring).\n- For performance optimizations, ensure that benchmarks or tests are included to validate the claimed performance improvements. Ideally, an end-to-end evaluation should demonstrate speedup on a concrete serving example. Microbenchmark-only validation is often insufficient: make a note if end-to-end evaluation is not included (e.g. due to lack of access to sufficient hardware to fully validate).\n\n#### 4.3: Root-Cause Analysis\n\nWhen fixing bugs or issues, provide a clear root-cause analysis. Explain what caused the problem, how it was identified, and why the chosen solution effectively addresses it. A patch which resolves the issue without a proper root-cause analysis may lead to recurring problems and increased maintenance overhead.\n\nFor fixes that involve disaggregated prefill/decode (PD) logic, such as the NIXL connector (`vllm/distributed/kv_transfer/kv_connector/v1/nixl/`), make a best effort to include a minimal example of the flow being changed and/or the initial flow that caused the failure. A short numbered sequence (or diagram) of a request's path across the P and D instances, e.g. scheduler admission, prefill execution, KV transfer registration and write, decode-side read, token generation, lets reviewers trace the root cause and the fix without reconstructing the distributed flow from the diff. Keep the example minimal: only the steps and components involved in the change. See the [disaggregated prefill](../../../docs/features/disagg_prefill.md), [NIXL KV push](../../../docs/design/nixl_kv_push_connector.md), and [NIXL KV cache lease](../../../docs/design/nixl_kv_cache_lease.md) docs for the flows involved.\n\n#### 4.4: Implementation Details\n\nProvide a detailed explanation of the implementation, including design decisions, trade-offs, and any relevant technical details. Highlight any non-obvious choices, tradeoffs, potential limitations, and areas where maintainer attention is required.\n\n#### 4.5: Contributing Guide\n\nEnsure that the PR follows the [contributing guide](https://docs.vllm.ai/en/latest/contributing), including adhering to coding standards, commit message conventions, and any other project-specific guidelines. This helps maintain consistency and quality across the codebase.\n",".agents/skills/transformers-upgrade/SKILL.md":"---\nname: transformers-upgrade\ndescription: \"Playbook for changing vLLM's Transformers version bounds. Use when bumping the pinned or maximum Transformers version, raising the minimum Transformers version, or deleting vendored configs, processors, or compat shims that upstream Transformers now provides.\"\n---\n\n# Upgrading Transformers\n\nThe supported range lives in `requirements/common.txt`. CI pins one exact\nversion in `requirements/test/*.in` and the compiled `requirements/test/*.txt`.\nA version change is one of two jobs, and they usually come as a pair:\n\n- **Bump the pin**: a new Transformers release becomes the CI version and\n the upper bound. vLLM grows compat code for it.\n- **Raise the floor**: the minimum moves up. That compat code, and every\n vendored config or processor upstream now ships, becomes dead weight.\n Every floor raise ends with an **unvendoring pass**.\n\n## Bump the pin\n\n1. Edit the pin line in each `requirements/test/*.in` and `*.txt` and the\n upper bound in `requirements/common.txt`. Past bumps edit only the pin\n line in the compiled files; see `git log -- requirements/common.txt`.\n2. Read the release notes for config validation changes, renamed auto-mapping\n formats, and removed attributes; these are what break vLLM.\n3. Search `tests/` for `min_transformers_version` and skip markers naming the\n new version, and run those tests on the new pin. Keep the gates: older\n versions in the supported range still lack the feature, so a gate goes\n only when the floor reaches it.\n4. Run the suites in [Testing](#testing) and fix what breaks. Mark every\n shim with a `# Transformers >= X` comment so the next floor raise finds it.\n\nDone when the [Testing](#testing) suites and every test gated on this\nversion pass on the new pin.\n\n## Raise the floor (unvendoring pass)\n\n### 1. Inventory\n\nInventory candidates with the floor version installed (no vLLM install needed):\n\n```bash\nuv run --no-project --with transformers== \\\n python .agents/skills/transformers-upgrade/scripts/find_upstreamed.py\n```\n\nIt lists:\n\n- vendored configs whose `model_type` upstream now owns;\n- `_CONFIG_REGISTRY` overrides of upstream types;\n- registered processors that collide with upstream class names or models;\n- version-gated code at or below the floor;\n- code that reads the installed Transformers version, whatever its threshold;\n- patches citing a Transformers issue or PR (with whether a referenced PR is\n in the floor release) or a future minimum version.\n\nIt finds candidates by name only. Every hit still needs a verdict.\n\n### 2. Port each candidate to upstream\n\nEvery hit is deleted by default. A vendored copy is a liability: it drifts\nfrom upstream, hides upstream fixes, and shadows the native class. Port the\nvLLM model code to the upstream schema instead, renaming attribute reads and\nderiving values from upstream fields.\n\nFan candidates out to parallel subagents. Each one compares the vendored and\nupstream classes, then greps every attribute the vLLM model code reads.\n[Upstreaming hazards](#upstreaming-hazards) lists the usual fixes.\n\nKeep a vendored class only when upstream cannot load a checkpoint vLLM must\nkeep serving, and record why in the PR description. A different upstream\nschema, a model-code rewrite, or a needed eval is work for the PR, not a\nreason to keep.\n\n### 3. Prove it on real checkpoints\n\nFor every checkpoint in `tests/models/registry.py` that uses a ported\nconfig, dump what vLLM builds before and after the change, then diff the two.\nRun both dumps; each catches what the other misses:\n\n- `dump_configs.py`: the resolved config, attribute by attribute.\n- `dump_model.py`: the model built on the meta device with no GPU or weights.\n It records ModelConfig sizes, every parameter shape, and every module's\n scalar attributes (`head_dim`, `num_kv_heads`, `scaling`, ...). Missing\n fields that are read with defaults only show up here.\n\n```bash\ngit worktree add --detach /vllm-base upstream/main\nS=.agents/skills/transformers-upgrade/scripts\nfor dump in dump_configs dump_model; do\n PYTHONPATH=/vllm-base .venv/bin/python $S/$dump.py before.jsonl \n PYTHONPATH=$PWD .venv/bin/python $S/$dump.py after.jsonl \n .venv/bin/python $S/diff_dumps.py before.jsonl after.jsonl\ndone\n```\n\nAdd `--trust-remote-code` for checkpoints whose `config.json` has an\n`auto_map`, and dump both ways.\n\nHow to read the diffs:\n\n- **A `` config field the model reads:** this crashes or silently\n falls back to a default. Port the model code to the upstream field.\n- **New upstream-only defaults** (`attention_bias=False`,\n `pad_token_id=None`): usually benign. A model diff of 0 confirms it.\n- **Any model diff:** this is a numerics change, and the PR needs a model\n eval. For example, A.X-K1 gained the checkpoint's YaRN rope scaling, which\n the vendored class had been dropping; it shows up as\n `layers.*.self_attn.scaling (x61)`.\n\n### 4. Delete\n\nFor each deleted config:\n\n- Delete the file.\n- Remove its entries from `_CLASS_TO_MODULE` and `__all__` in\n `vllm/transformers_utils/configs/__init__.py`.\n- Remove its entries from `_CONFIG_REGISTRY` in\n `vllm/transformers_utils/config.py`.\n- Switch every importer, including tests, to `from transformers import ...`.\n- Remove the `source_file_dependencies` lines naming the file from\n `.buildkite/`.\n\nDeleted processors follow the same steps in\n`vllm/transformers_utils/processors/__init__.py`.\n\nBefore merging, rebase and grep again for the old field names: PRs that land\nin between still read them.\n\nDone when every line of the inventory output is deleted, or kept with a\nreason that meets the bar in step 2. The PR also carries the config diff\nresults and evals for any changed values.\n\n## Upstreaming hazards\n\nUpstream configs are `@strict` dataclasses. Checkpoint keys upstream does not\ndeclare survive as plain attributes, but only when the checkpoint sets them.\nDefaults that only vLLM sets, and values vLLM derives, vanish.\n\n| Hazard | Signal | Fix |\n| --- | --- | --- |\n| Upstream consumes legacy keys | `__post_init__` pops keys the model reads (`deepseek_v4` `compress_ratios`) | Read the upstream replacement (`layer_types`, `mlp_layer_types`, per-layer `rope_parameters`) |\n| Renamed or dropped fields | Diff shows `` for a field the model reads | Read the upstream name; derive dropped flags from what implies them (`enable_moe_block` from `num_experts`) |\n| Undeclared field without a default | `config.x` raises `AttributeError` for configs that omit `x`, e.g. minimal test configs (A.X-K1 `scoring_func`). Dumps of real checkpoints miss it | Upstream ignoring `x` usually means vLLM can stop reading it; if vLLM truly needs it, read `getattr(config, \"x\", )` |\n| Top-level config stops forwarding | The vendored top-level config forwarded reads to `text_config`; upstream does not. `getattr(config, \"x\", default)` silently takes the default, e.g. on MTP draft configs from `SpeculativeConfig.hf_config_override`, which the dumps do not build | Read from `get_text_config()`; grep `getattr` reads of every field, including speculative-decoding paths |\n| Type check on the config | `ctx.get_hf_config(SomeConfig)` relies on the vendored base class | Accept the upstream class too, e.g. a `Cosmos3ProcessingInfo` overriding `get_hf_config` |\n| Remote code takes over | Checkpoint `config.json` has `auto_map`; with `--trust-remote-code`, the hub class replaces the native one once vLLM stops registering its own | Check the remote class loads; if not, register the upstream class for that `model_type` |\n| Read-only upstream fields | Properties or derived fields break subclasses and `hf_overrides` (`nemotron_h` `hybrid_override_pattern`) | Override the source field (`layers_block_type`) instead |\n| Behavioural rewrite | Upstream derives fields differently (`diffusion_gemma` `sliding_window`) | Upstream matches the reference implementation; verify with an eval |\n| Original checkpoint format | Upstream targets `-hf` conversions; vLLM serves the originals (`qwen3_asr`, `qianfan_ocr`) | Keep only if upstream cannot load the original at all |\n| Processor output contract | Model code consumes keys upstream does not emit | Port the model to upstream's keys |\n| Name collision | A registered vendored processor shares an upstream class name and hijacks `get_processor` (`InternVLProcessor`) | Drop the registry entry |\n\nVersion gates from the inventory need a decision:\n\n- **Code** only reachable below the floor is deleted. This covers:\n - comparisons against `TRANSFORMERS_VERSION`;\n - branches labelled `# Transformers < X`;\n - fallbacks for fields that upstream configs now always set, e.g.\n `mlp_layer_types`, `per_layer_config` or legacy `layer_types` names.\n\n A `skipif` whose condition is now always true becomes an unconditional\n `skip` with the same reason.\n- **`min_transformers_version`** at or below the floor is removed.\n- **`max_transformers_version`** below the floor means CI never tests that\n model; report it rather than deleting it silently.\n- **Patches citing a Transformers issue or PR:**\n - Delete the patch when its fix is in the floor release.\n - For an issue, find the PR that closed it and check that PR. A closed\n issue is not a fixed one: issues get closed while the bug is still in\n the release.\n - Re-run the patched behaviour before deleting. A fix in the release does\n not always cover vLLM's path: Emu3's processor gained BOS upstream, but\n vLLM tokenizes without the processor.\n - Links that only explain a design choice (e.g. a float32 RoPE note) are\n not patches; leave them.\n\n## Testing\n\n```bash\n.venv/bin/python -m pytest tests/transformers_utils tests/config tests/tokenizers_\n.venv/bin/python -m pytest tests/models/multimodal/processing/test_common.py -k \"\"\n```\n\nAdd `tests/models/test_initialization.py -k \"\"` where a GPU is\navailable, and a model eval for any model whose resolved config changed.\n\nModels in `vllm/models/` are high priority: whenever their config or model\ncode changes, run GSM8k (`tests/evals/gsm8k/`) and GPQA Diamond\n(`tests/evals/gpt_oss/`) on `main` and the branch, even when the dumps show\nno diff, and put both scores in the PR. FlashInfer autotuning picks kernel\ntactics afresh on every server start, which moved DeepSeek-V4-Flash GSM8k by\nabout 1 point; pass `--kernel-config '{\"enable_flashinfer_autotune\": false}'`\nto both servers so greedy outputs are comparable.\n",".agents/skills/triton-kernel-writing/SKILL.md":"---\nname: triton-kernel-writing\ndescription: Write or review Triton kernels for vLLM, with practical guidance for generated-code inspection, launch grids, indexing, specialization, tuning, and representative performance validation. Use when adding, modifying, optimizing, or reviewing a `@triton.jit` kernel in vLLM.\n---\n\n# Triton Kernel Writing\n\n## Implementation\n\n- Follow the official\n [Triton semantics](https://triton-lang.org/main/python-api/triton-semantics.html).\n Check it when behavior may differ from Python or NumPy, especially type\n promotion, integer division and modulo, casts, broadcasting, and variable\n scoping.\n- Use the Triton kernel generated by `torch.compile` as a possible\n implementation to inspect. Print Inductor's generated code with\n `TORCH_LOGS=\"output_code\" .venv/bin/python