{"owner":"huggingface","repo":"diffusers","hasSkills":true,"totalSkillsCount":12,"totalTokensCount":10942,"categories":["plugin-manifest","root-instruction","anthropic-skill","marketplace","claude-rule"],"hasMcp":false,"mcpConfig":null,"found":[".ai/.claude-plugin/plugin.json",".ai/.codex-plugin/plugin.json",".ai/AGENTS.md",".ai/plugin.json",".ai/skills/custom-blocks/SKILL.md",".ai/skills/diffusers-cli/SKILL.md",".ai/skills/diffusers-cli/run.md",".ai/skills/model-integration/SKILL.md",".ai/skills/self-review/SKILL.md",".claude-plugin/marketplace.json","AGENTS.md","CLAUDE.md"],"skills":{".ai/.claude-plugin/plugin.json":"{\n \"name\": \"diffusers\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"author\": {\n \"name\": \"The Hugging Face team\"\n },\n \"homepage\": \"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents\",\n \"repository\": \"https://github.com/huggingface/diffusers\",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"diffusers\",\n \"diffusion-models\",\n \"contributing\"\n ]\n}\n",".ai/.codex-plugin/plugin.json":"{\n \"name\": \"diffusers\",\n \"version\": \"0.1.0\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"skills\": \"./skills/\"\n}\n",".ai/AGENTS.md":"# Diffusers — Agent Guide\n\n## Setup\n\nWe recommend developing in a virtual environment managed by [uv](https://docs.astral.sh/uv/):\n\n```bash\nuv venv && source .venv/bin/activate\nuv pip install -e . # provides diffusers-cli\n```\n\nList the available skills, and what each one is for, with:\n\n```bash\ndiffusers-cli skills list\n```\n\nInstall them with:\n\n```bash\ndiffusers-cli skills add # install one, or --all for every skill\n # --claude / --codex / --cursor to pick one target\n```\n\n`diffusers-cli skills update` refreshes what you installed.\n\nSkills used to be installed through the `Makefile`, which symlinked the project-level `.claude/skills` and\n`.agents/skills` at `.ai/skills` in this repo. Remove those links if they exist — otherwise the install writes through\nthem into `.ai/` itself:\n\n```bash\nrm -f .claude/skills .agents/skills\n```\n\nAt the start of a session, confirm the setup and tell the user what you find:\n\n```bash\npython utils/check_ai.py # guides and skills are consistent\ndiffusers-cli skills list # which skills are installed\n```\n\nClaude Code and Codex can also install via plugins\n\n```bash\nclaude plugin marketplace add huggingface/diffusers\nclaude plugin install diffusers@diffusers-skills --scope project\n\ncodex plugin marketplace add huggingface/diffusers # then install from the Plugins Directory\n```\n\n\n## Human in the loop\n\nEverything a PR reviewer sees must come from a human, or be approved by one for the exact wording. That includes commit messages, code comments and docstrings, PR titles and descriptions, and any PR or issue comment, review, or reply. Ask whether the user wants you to draft that text or will write it themselves. Keep drafts short and easy to understand. When you hand one over, remind them to read it for real and check that it reads well for other humans.\n\n- **Don't commit unless the user approved the exact commit message, and never push or open a PR on\n your own.** The user decides when anything is published, each time.\n- **Don't post to GitHub directly** — no comments, reviews, or replies. Draft when asked and hand the text to the user.\n\n## Code formatting\n\n- `make style` and `make fix-copies` should be run before opening a PR\n\n## Reference guides\n\n- **Coding style** — see [code_style.md](references/code_style.md) for how code should read, and the `# Copied from` rules.\n- **Models** — see [models.md](references/models.md) for model conventions, attention pattern, implementation rules, dependencies, and gotchas. For adding or converting a model, use the [model-integration](skills/model-integration/SKILL.md) skill.\n- **Pipelines** — see [pipelines.md](references/pipelines.md) for pipeline conventions, patterns, and gotchas.\n- **Modular pipelines** — see [modular.md](references/modular.md) for modular pipeline conventions, patterns, and gotchas.\n- **Tests** — see [testing.md](references/testing.md) for test conventions: required test layers, tester mixins, and dummy-component rules.\n- **Reporting** — see [reporting.md](references/reporting.md) for how to write bug reports (what a reproduction is) and performance claims (end-to-end numbers first).\n\n## Skills\n\nTask-specific guides live in `.ai/skills/` and are loaded on demand by AI agents. Available skills include:\n\n- [model-integration](skills/model-integration/SKILL.md) (adding/converting pipelines)\n- [custom-blocks](skills/custom-blocks/SKILL.md) (packaging a `ModularPipelineBlocks` subclass for the Hub)\n- [diffusers-cli](skills/diffusers-cli/SKILL.md) (running pipelines, inspecting schemas, and using the Diffusers CLI)\n- [self-review](skills/self-review/SKILL.md) (pre-PR self-review against the project rules)\n\n## Self-review before a PR\n\nBefore opening a PR, run self-review against [review-rules.md](references/review-rules.md). The [self-review skill](skills/self-review/SKILL.md) runs this as the same pass the `@claude` CI reviewer uses. Share the final report on the PR (description or comment) — see the skill for details.\n",".ai/plugin.json":"{\n \"$schema\": \"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json\",\n \"name\": \"diffusers\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"author\": {\n \"name\": \"The Hugging Face team\"\n },\n \"homepage\": \"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents\",\n \"repository\": \"https://github.com/huggingface/diffusers\",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"diffusers\",\n \"diffusion-models\",\n \"contributing\"\n ]\n}\n",".ai/skills/custom-blocks/SKILL.md":"---\nname: custom-blocks\ndescription: >\n Use when the user has written (or wants to write) a `ModularPipelineBlocks`\n subclass in a local Python file and needs to package it into a Hub-uploadable\n directory. Covers the workflow from a single `block.py` file to a published\n custom-block repo that consumers can load via\n `ModularPipeline.from_pretrained(, trust_remote_code=True)`.\n---\n\n## What this skill is for\n\nA `ModularPipelineBlocks` subclass is a unit of pipeline logic — input/output spec plus a `__call__` — that\nslots into diffusers' modular pipeline composition. Once you have one defined locally, you almost always want to\npublish it as a small Hub repo so others can `from_pretrained` it. `diffusers-cli custom_blocks` automates the\npackaging step: it parses your Python file, instantiates the chosen block class, and writes a\n`save_pretrained`-style directory in your cwd that's ready to push to the Hub.\n\nUse this skill when:\n\n- The user is writing a custom modular block and asks \"how do I publish this?\" or \"package this for the Hub\".\n- The user has a `block.py` (or similar) file with one or more `ModularPipelineBlocks` subclasses.\n- You're scaffolding a new modular pipeline repo and need the on-disk layout that `ModularPipelineBlocks.from_pretrained`\n expects.\n\nDon't use this skill for: running an existing modular pipeline (`diffusers-cli run`), introspecting one\n(`diffusers-cli schema`), or writing the block class itself — this skill packages an *already-written* block.\n\n## The end-to-end workflow\n\n```\n[you: write block.py] → diffusers-cli custom_blocks → [packaged dir in cwd]\n ↓\n hf upload .\n ↓\n consumers: ModularPipeline.from_pretrained(, trust_remote_code=True)\n diffusers-cli schema --model --trust-remote-code\n diffusers-cli run --model --trust-remote-code ...\n```\n\nThe skill covers the middle box. The bookends (writing the block and uploading) are out of scope.\n\n## Command surface\n\n```bash\ndiffusers-cli custom_blocks [--block_module_name ] [--block_class_name ]\n```\n\n### Flags\n\n- `--block_module_name ` — Python file containing the block class. Defaults to `block.py` in the cwd.\n- `--block_class_name ` — Which class in the file to package. Optional: if omitted, the CLI parses the\n file with `ast`, finds every class that inherits from `ModularPipelineBlocks`, and uses the first one (with\n an info log naming the others). Specify explicitly when the file defines more than one block and you want a\n specific one.\n\n### What it does\n\n1. **AST scan**: parses `` without executing it, walks top-level `ClassDef` nodes, and collects every\n class whose `bases` include `ModularPipelineBlocks` or one of its composite subclasses\n (`SequentialPipelineBlocks`, `AutoPipelineBlocks`, `ConditionalPipelineBlocks`, `LoopSequentialPipelineBlocks`).\n2. **Pick a class**: uses `--block_class_name` if given, else the first found. Errors with the list of available\n classes if your name doesn't match.\n3. **Load and save**: imports the file via `importlib.util.spec_from_file_location` (this does execute the\n module — make sure your block.py is something you trust to run), instantiates the chosen class with no\n constructor args, and calls `.save_pretrained(os.getcwd())`.\n\nThe result is a Hub-uploadable directory laid out the way `ModularPipelineBlocks.from_pretrained` expects:\nyour block source, an `auto_map` in the config so consumers know to load it with `trust_remote_code=True`,\nand any artifacts `save_pretrained` writes for that block class.\n\n## End-to-end example\n\nGiven a `block.py` like:\n\n```python\nfrom diffusers.modular_pipelines import ModularPipelineBlocks, InputParam, OutputParam\n\nclass MyDenoiseBlock(ModularPipelineBlocks):\n model_name = \"my-denoise\"\n\n @property\n def inputs(self):\n return [\n InputParam(\"latents\", type_hint=\"torch.Tensor\", required=True, description=\"Noisy latents.\"),\n InputParam(\"guidance_scale\", type_hint=\"float\", default=7.5),\n ]\n\n @property\n def intermediate_outputs(self):\n return [OutputParam(\"latents\", type_hint=\"torch.Tensor\")]\n\n def __call__(self, components, state):\n # ... denoising logic ...\n return components, state\n```\n\nPackage it:\n\n```bash\ndiffusers-cli custom_blocks --block_module_name block.py\n```\n\nOutput in cwd:\n\n```\n./\n├── block.py\n├── modular_config.json # contains auto_map → MyDenoiseBlock\n└── (any state files MyDenoiseBlock.save_pretrained writes)\n```\n\nUpload to the Hub:\n\n```bash\nhf upload my-user/my-denoise-block .\n```\n\nConsumers can now use it:\n\n```python\nfrom diffusers import ModularPipeline\npipe = ModularPipeline.from_pretrained(\"my-user/my-denoise-block\", trust_remote_code=True)\n```\n\nOr via CLI:\n\n```bash\ndiffusers-cli schema --model my-user/my-denoise-block --trust-remote-code\ndiffusers-cli run --model my-user/my-denoise-block --trust-remote-code \\\n --pipeline-kwargs '{\"latents\": \"...\", \"guidance_scale\": 7.5}'\n```\n\n## Common errors\n\n- **`Could not parse '': SyntaxError`** — the file isn't valid Python. Fix the syntax; the AST step runs\n before any execution.\n- **`block_class_name could not be retrieved. Available classes from : [ClassA, ClassB]`** — your\n `--block_class_name` doesn't match any `ModularPipelineBlocks` subclass found. Pick from the list shown.\n- **No classes found**: silent — the command will try to use the first entry in an empty list and raise\n `IndexError`. If you hit that, double-check your class actually inherits from `ModularPipelineBlocks` or one\n of the composite classes listed above (the AST scan looks for those literal class names; aliased imports like\n `from diffusers import ... as MPB` won't be picked up).\n- **Block requires constructor args**: the command calls `()` with no args. If your block needs\n `__init__` parameters, refactor to take them from `state`/`components` at `__call__` time instead, or\n hardcode defaults in `__init__`.\n\n## Verifying the install\n\nIf `diffusers-cli` isn't on PATH after `pip install -e .`, reinstall with\n`pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the binary is missing recent\nfeatures (e.g. `unrecognized arguments: --lora`), reinstall. See the\n[`diffusers-cli` skill](https://github.com/huggingface/diffusers/blob/main/.ai/skills/diffusers-cli/SKILL.md#verifying-the-cli-is-installed) for more.\n\n## Related\n\n- the [`diffusers-cli` docs](https://github.com/huggingface/diffusers/blob/main/docs/source/en/using-diffusers/cli.md) — once your block\n is uploaded, `schema` and `run` call it from the terminal without writing Python.\n- diffusers' [modular pipelines docs](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) — for writing the block\n class itself.\n",".ai/skills/diffusers-cli/SKILL.md":"---\nname: diffusers-cli\ndescription: >\n Use when the user wants to run a diffusers pipeline from a terminal (one-off\n generation, batch jobs, smoke-testing a new model), run on HF Sandbox\n hardware via `--remote`, introspect a pipeline's input schema before\n calling it, or attach a LoRA at inference time. Prefer this over writing\n ad-hoc Python scripts for generation tasks.\n---\n\n## Overview\n\n`diffusers-cli` is the shipped CLI in `src/diffusers/commands/`. Subcommands relevant to agentic use:\n\n| Command | Purpose |\n| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `run` | Run any `DiffusionPipeline` or `ModularPipeline`. Forwards `--pipeline-kwargs` verbatim, saves output by detecting its runtime type, optionally runs on HF Jobs via `--remote`. |\n| `schema` | Print the input schema for a pipeline repo (kwarg names, types, defaults, descriptions). **No weights downloaded** — only the small index file. |\n| `custom_blocks` | Package a local `ModularPipelineBlocks` subclass for the Hub. |\n| `env` | Print versions of diffusers + torch + transformers + accelerate + safetensors + CUDA + GPU info. Use when investigating environment issues, dtype/precision support, or building bug reports. |\n\n## When to read which file\n\nMost agentic work goes through `run`. Read the matching reference file before constructing a command:\n\n- **[`run.md`](run.md)** — full reference for `diffusers-cli run`. Covers `--pipeline-kwargs`\n semantics and the shell-quoting gotcha, LoRA via `--lora`, optimization flags (`--dtype`, `--cpu-offload`,\n `--attention-backend`, `--vae-tiling/slicing`), output handling and `--push-to` bucket uploads, the full\n `--remote` HF Jobs flow (image, container command, log streaming, timing payload, artifact download), and\n context parallel (`--context-parallel`) for both local-torchrun and `--remote` paths.\n\nThe other commands are small enough that `diffusers-cli --help` is the canonical reference:\n\n```bash\ndiffusers-cli schema --help\ndiffusers-cli custom_blocks --help\ndiffusers-cli env --help\n```\n\n## When NOT to use this skill\n\n- Multi-stage workflows where you need intermediate tensor manipulation between pipelines → write Python.\n- Training or fine-tuning → CLI only covers inference.\n- Anything requiring `quantization_config` or other low-level loader knobs not exposed by the CLI flags → write\n Python. (`device_map` is exposed as `--device-map`; see [run.md](run.md#optimization-flags).)\n\n## Verifying the CLI is installed\n\nThe console entry point is registered in `pyproject.toml` (`diffusers-cli =\n\"diffusers.commands.diffusers_cli:main\"`). If `diffusers-cli` is not on PATH after `pip install -e .`, reinstall\nwith `pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the installed binary is\nmissing recent features (e.g. you see `unrecognized arguments: --lora`), reinstall.\n\n## Output formats\n\n`--format {auto, human, agent, json}` (top-level flag, must appear before the subcommand):\n\n- **`human`** — plain-text indented output for terminals (default when not running under an agent harness). No ANSI color.\n- **`agent`** — TSV tables and `key=value` lines. Auto-selected when an agent env var is present\n (`CLAUDECODE`, `CLAUDE_CODE`, `CODEX_SANDBOX`, `CURSOR_AI`, `AIDER_AI_CONTEXT`, `GH_COPILOT_AGENT`,\n `AI_AGENT`). Token-cheap for LLM agents to read.\n- **`json`** — compact JSON. Use for programmatic parsing (scripts, services) where type fidelity and nested\n structures matter.\n\n`stdout` carries data; `stderr` carries hints/warnings/progress — parseable output is never polluted.\n\nRule of thumb: `--format json` for scripts that will `json.loads()` the output, otherwise leave it on\nauto-detect (`agent` for LLMs, `human` for terminals).\n",".ai/skills/diffusers-cli/run.md":"# `diffusers-cli run` — reference\n\nFull surface for `diffusers-cli run`. Use this file as the source of truth when constructing a `run`\ninvocation. The top-level [`SKILL.md`](SKILL.md) covers when to use the CLI; this file covers how.\n\n## The schema → run flow\n\nFor any model you haven't called before, run `schema` first to learn its input contract, then `run` with\nthe right `--pipeline-kwargs`:\n\n```bash\n# 1. Discover what kwargs the pipeline takes (no weight download)\ndiffusers-cli --format json schema --model black-forest-labs/FLUX.2-klein-9B\n\n# 2. Run it\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --dtype bf16\n```\n\n`schema --format json` emits a `{task, model, pipeline_class, inputs[]}` payload where each input is\n`{name, type_hint, default, required, description}`.\n\n## Standard vs modular detection\n\n`run` auto-detects which kind of pipeline it's calling:\n\n1. If `model_index.json` exists on the repo → `DiffusionPipeline.from_pretrained` path.\n2. Otherwise → `ModularPipeline.from_pretrained` path.\n\nYou don't need to tell it which. Modular repos must pass `--trust-remote-code` if they ship custom block code.\n\n## `--pipeline-kwargs` semantics\n\nA JSON object passed straight through to `pipeline(**kwargs)`. String values at known media-input keys are\nauto-loaded before the pipeline is called:\n\n- **Images** (`image`, `mask_image`, `control_image`, `ip_adapter_image`, `image_2`) → `PIL.Image.Image`\n via `diffusers.utils.load_image`. Accepts URLs or local paths.\n- **Videos** (`video`, `control_video`) → `list[PIL.Image.Image]` via `diffusers.utils.load_video`.\n- **Audio** (`initial_audio_waveforms`, `reference_audio`, `src_audio`) → `torch.Tensor` via `torchaudio.load`.\n For `initial_audio_waveforms`, the file's native sample rate is auto-written to\n `initial_audio_sampling_rate` if you didn't pass it explicitly.\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\", \"prompt\": \"make the fur grey\", \"strength\": 0.6}'\n```\n\nMedia resolution runs **before** the pipeline weights load, so a dead URL or missing file fails within\nseconds instead of after a multi-minute model download.\n\n**Batched inputs**: any media key (and `prompt`) accepts a JSON array — each string in the list is loaded\nindividually, and the pipeline processes the whole list in one forward pass on the GPU. Use this when you\nhave several inputs that share flavor/model/dtype and want them batched:\n\n```bash\ndiffusers-cli run --model black-forest-labs/FLUX.1-Kontext-dev \\\n --pipeline-kwargs '{\"prompt\": [\"make it grey\", \"make it pink\", \"make it blue\"],\n \"image\": [\"https://.../cat1.png\", \"https://.../cat2.png\", \"https://.../cat3.png\"]}'\n```\n\n**Shell-quoting gotcha**: the JSON must be on one line (or use `\\` to line-continue). A literal newline inside the\nsingle-quoted argument lands as a raw control char inside the string and breaks `json.loads`.\n\n## LoRA adapters (`--lora`)\n\nAttach one or more LoRAs after the pipeline loads via a JSON spec. `--lora` accepts a single object or a list.\n\nSingle adapter:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"a tiny grey cat\"}' \\\n --lora '{\"lora_id\": \"alvdansen/littletinies\", \"lora_scale\": 0.8}'\n```\n\nMultiple stacked adapters:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"a tiny grey cat\"}' \\\n --lora '[\n {\"lora_id\": \"alvdansen/littletinies\", \"lora_scale\": 0.5, \"adapter_name\": \"style\"},\n {\"lora_id\": \"author/detail-boost\", \"lora_scale\": 0.3}\n ]'\n```\n\nPer-entry fields: `lora_id` (required), `lora_scale` (optional, default `1.0`), `adapter_name` (optional; when\nomitted while stacking, auto-generated as `lora_0`, `lora_1`, …; single-entry defaults to `\"default\"`).\n\nEach entry calls `pipeline.load_lora_weights(, adapter_name=)`. After all adapters load, one\n`pipeline.set_adapters(names, adapter_weights=scales)` activates them together.\n\n## Optimization flags\n\n- `--dtype {auto, bf16, fp16, fp32, …}` — pipeline weight dtype. `bf16` is the right default for modern DiTs on\n A100/H100.\n- `--device-map ` — component placement, forwarded to `from_pretrained(device_map=...)`. Accepts a plain\n torch device (`cuda`, `cuda:0`, `cpu`, `mps`), the string `balanced` (auto-splits pipeline components across\n visible GPUs), or a JSON dict `{\"transformer\": \"cuda:0\", \"vae\": \"cuda:1\"}` for explicit per-component\n placement. Auto-detects if omitted (pinned to `cuda:$LOCAL_RANK` under torchrun). `balanced` and dict values\n are incompatible with `--cpu-offload`.\n- `--cpu-offload {model, group}` — `model` uses `enable_model_cpu_offload`, `group` uses\n `enable_group_offload(offload_type=\"leaf_level\", use_stream=True)`. Use `group` to fit a 9B+ model on a single\n A100. Onload target device comes from `--device-map` (must be a plain device string in this case).\n- `--attention-backend {default, flash_hub, flash_varlen_hub, flash_4_hub, sage_hub, sage_blackwell_hub}` — hub-hosted kernels,\n auto-downloaded on first use. Failures (kernel not available, CUDA arch mismatch, network) raise a clear\n `SystemExit` listing the alternatives instead of silently reverting to the default. Only supported on\n transformer-based pipelines; UNet pipelines get a `logger.warning` and the flag is ignored.\n- `--vae-tiling` / `--vae-slicing` — lower peak VAE decode VRAM.\n- `--compile [JSON]` — `torch.compile` every denoiser submodule. See [Compile](#compile) below.\n- `--context-parallel` — Ulysses-style context parallelism on a DiT. See [Context parallel](#context-parallel) below.\n\n## Output handling\n\n`run` detects the pipeline return type and saves accordingly:\n\n- `PIL.Image` / list of them → `.png` (zero-padded, e.g. `0000.png`)\n- Frame sequence (≥2 PILs or ndarrays) → `0000.mp4` (uses `--fps`, default 8)\n- Numpy audio array → `0000.wav` (uses `--sampling-rate`)\n- Anything else → JSON dump\n\n**Default output directory** is `~/.diffusers/cli/run/outputs/diffusers-run--/`.\nEach run gets its own subdirectory so consecutive invocations don't overwrite each other. The same run id\nis used for the local dir, the sandbox's `DIFFUSERS_CLI_RUN_ID` env var, and the sandbox input/output paths\nunder `--remote`, so a run is traceable end-to-end.\n\nOverride the destination with `--output ` (file or directory). Explicit `--output` bypasses the\n`diffusers-run-*` namespace — files land flat in the path you gave.\n\n### `--push-to`\n\nUpload outputs to an HF bucket. Objects land under `//`. The bucket is created\nif it doesn't exist. Behavior interacts with `--output` and `--remote`:\n\n| flags | download to local? | bucket write? |\n|---|---|---|\n| (neither) | ✅ `~/.diffusers/cli/run/outputs//` | — |\n| `--output /path` | ✅ `/path/` | — |\n| `--push-to my/bucket` | ❌ | ✅ (bucket only) |\n| `--push-to my/bucket --output /path` | ✅ | ✅ (both) |\n\nThe rule: explicit `--push-to` means \"the bucket is my destination\" — skip the local download unless the\nuser also explicitly asked for a local target via `--output`.\n\n## Remote execution (`--remote`)\n\nAdd `--remote` to run the same call inside a [Hugging Face Sandbox](https://huggingface.co/docs/huggingface_hub/en/guides/sandbox)\n— an isolated cloud VM (built on HF Jobs) the CLI drives over HTTP: it uploads inputs, runs the pipeline,\ndownloads outputs, then terminates the sandbox.\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --remote --flavor a100-large \\\n --dtype bf16 \\\n --cpu-offload group\n```\n\nWhat happens:\n\n1. Your HF token is picked up (from `--token` or your login) and forwarded into the sandbox as `HF_TOKEN`.\n2. `--pipeline-kwargs` are parsed locally so JSON errors fail fast (no wasted sandbox time).\n3. A dedicated sandbox is created on `--flavor` from `diffusers/diffusers-cli-cuda:latest` by default — a\n prebuilt image (`docker/diffusers-cli-cuda/` in this repo, rebuilt nightly) that already ships torch,\n CUDA, `diffusers`, and the rest of the CLI's deps. Any local file paths in `--pipeline-kwargs` are\n uploaded into the sandbox under `/tmp/diffusers-cli/inputs//` via native file transfer (no\n bucket), and the JSON paths are rewritten to point at them.\n4. No dependency install runs on the default image — that step is skipped unless you passed `--dependencies`\n (only the extras are installed then) or pointed `--image` somewhere else (the full set is installed).\n Output streams live to your terminal.\n5. The sandbox CLI writes outputs to `/tmp/diffusers-cli/outputs/`; the CLI downloads every artifact back into\n the local target (see [`--push-to`](#-push-to) for when the download is skipped).\n6. The sandbox is terminated (unless `--keep-alive`/`--sandbox-id`), and the wallclock `run_seconds` for the\n pipeline command is printed and included in the JSON payload.\n\nFlags:\n\n- `--flavor ` — sandbox hardware (e.g. `a10g-small`, `a100-large`, `4xa100-large`).\n- `--timeout ` — max wallclock for the run command inside the sandbox (e.g. `30m`, `2h`). Defaults to `10m`.\n- `--dependencies ` — extra pip deps (repeatable), installed on top of whatever the image ships.\n- `--namespace ` — create the sandbox under a different account.\n- `--push-to ` — see [`--push-to`](#-push-to) above. The upload runs inside the sandbox; an explicit\n value with no `--output` makes the bucket the sole destination and skips the local download.\n- `--image ` — override the sandbox image. Must ship torch + CUDA; the CLI then installs the small\n Python deps on top via `uv pip install --system` on every cold sandbox, which the default image avoids.\n Useful for pinning a specific torch or bundling extra system libs.\n- `--volume [:]` — mount an HF storage bucket into the sandbox as a read-write directory.\n Repeatable. Default mount is `/mnt/buckets/`. Reference mounted files from `--pipeline-kwargs`\n like any other local path — no upload happens, the container reads straight from the FUSE mount. Applied\n only on new sandbox creation — ignored when reconnecting via `--sandbox-id`. Enables batch workflows:\n loop over bucket contents on the host, one `--sandbox-id` call per input.\n- `--keep-alive` — don't terminate the sandbox after the run; its id is printed so a later run can reconnect.\n- `--sandbox-id ` — reconnect to a kept-alive sandbox instead of creating a new one. Implies `--keep-alive`.\n- `--idle-timeout ` — auto-shutdown after this much inactivity (default 10m). The billing backstop\n for kept-alive sandboxes. Only applied on new sandbox creation — ignored when reconnecting via `--sandbox-id`.\n\n### Reusing a warm sandbox (`--keep-alive` / `--sandbox-id`)\n\nBy default each `--remote` run is ephemeral: create → run → download → kill. The cold cost (dep install +\nmodel weight download + any compile) is paid every time. For iterative work, keep one sandbox warm and reuse\nit — deps, the HF weight cache, and the `torch.compile` cache all survive on its disk:\n\n```bash\n# First run: create + keep alive. Prints sandbox_id=.\ndiffusers-cli run -m black-forest-labs/FLUX.2-klein-9B --dtype bf16 \\\n --pipeline-kwargs '{\"prompt\": \"a cat\"}' --remote --flavor a100-large --keep-alive\n\n# Subsequent runs: reconnect — model already cached, only inference runs.\ndiffusers-cli run -m black-forest-labs/FLUX.2-klein-9B --dtype bf16 \\\n --pipeline-kwargs '{\"prompt\": \"a dog\"}' --remote --flavor a100-large --sandbox-id \n\n# Stop it when done (or let --idle-timeout reap it).\nhf sandbox kill \n```\n\nNotes on `--remote` argv forwarding: flags that control how the run is dispatched (`--flavor`, `--timeout`,\n`--namespace`, `--dependencies`, `--image`, `--keep-alive`, `--sandbox-id`, `--idle-timeout`, `--format`) are\nstripped before the argv is rebuilt for the sandbox CLI. Everything else — model, dtype, pipeline-kwargs,\noptimizations, output flags — is forwarded verbatim (`--output` is repointed at the sandbox output dir).\n\n## Compile\n\n`--compile` runs `torch.compile` over every `transformer*` / `unet*` submodule on the pipeline. Prefers\nregional compilation via `module.compile_repeated_blocks(**kwargs)` when the model exposes `_repeated_blocks`\n— this only compiles the repeated inner blocks (the bulk of the compute) rather than the whole module, so\nfirst-step latency is much lower. Falls back to full `torch.compile(module, **kwargs)` when no regional\nmetadata is declared.\n\n```bash\n# Bare — uses fullgraph=true\ndiffusers-cli run --model --dtype bf16 --compile --pipeline-kwargs '...'\n\n# With kwargs forwarded to torch.compile\ndiffusers-cli run --model --dtype bf16 \\\n --compile '{\"mode\": \"max-autotune\", \"fullgraph\": true}' \\\n --pipeline-kwargs '...'\n```\n\n**When it's worth it**: multi-step generation (~50+ denoising steps). You pay a one-time compilation cost on\nthe first step, then every subsequent step is faster.\n\n**Under `--remote`**: on an ephemeral sandbox the compile cache doesn't survive, so the compilation cost is\npaid on every run — `--compile` only breaks even on very long generations. On a `--keep-alive`/`--sandbox-id`\nsandbox the compile cache persists on disk, so the cost is paid once and reused: keep a warm sandbox and\n`--compile` pays off across runs.\n\n`--compile` is **currently not supported with `--context-parallel`** — CP shards attention across ranks while\nregional compile assumes a stable single-device graph. If both are set, the CLI logs a warning and skips the\ncompile step; CP still runs.\n\n## Context parallel\n\n`--context-parallel` enables Ulysses CP on a DiT-based pipeline. **Locally** the user must launch via torchrun:\n\n```bash\ntorchrun --nproc-per-node=2 -m diffusers.commands.diffusers_cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\"}' \\\n --dtype bf16 \\\n --context-parallel\n```\n\n**Remotely** the CLI handles the torchrun wrapping — just pass `--context-parallel` to a `--remote` invocation on\na multi-GPU flavor:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --remote --flavor 4xa100-large \\\n --dtype bf16 \\\n --context-parallel\n```\n\nInside the sandbox, CP swaps the entrypoint to `torchrun --nproc-per-node=gpu -m\ndiffusers.commands.diffusers_cli`, initializes a hybrid process group (`cpu:gloo,cuda:nccl` — NCCL for the\nattention all-to-all, Gloo for `ulysses_anything`'s per-rank size coordination), pins each rank to\n`cuda:{LOCAL_RANK}`, and gates output saving/printing to rank 0 only.\n\n**Memory note**: CP shards the sequence, **not the weights**. Every rank still holds the full transformer. Wins\nare wall-clock attention speedup and headroom for very long sequences, not \"fit a model that doesn't fit.\" For\nweight sharding you'd want TP or FSDP — not exposed in the CLI yet.\n\nCP is DiT-only. UNet pipelines raise a clear error directing you to a DiT pipeline (FLUX, SD3, HunyuanDiT,\nAuraFlow, …).\n\n## Output mode (`--format`)\n\n`--format` controls the shape of **stdout metadata** (which paths were written, timing, sandbox id, pushed\nbucket URLs) — **not** the media file format. Written images are always PNG, videos MP4, audio WAV; only the\nsummary printed alongside them changes shape.\n\nThe CLI auto-detects when running under an AI coding agent (Claude Code, Cursor, Aider, GH Copilot Agent — via\n`CLAUDECODE`, `CLAUDE_CODE`, `CURSOR_AI`, `AIDER_AI_CONTEXT`, `GH_COPILOT_AGENT`) and switches to **agent\nmode** automatically — TSV tables, `key=value` results, compact JSON dicts, no progress bars.\n\nOverride explicitly with `--format {auto, human, agent, json, quiet}` placed **before** the subcommand:\n\n```bash\ndiffusers-cli --format json run --model --pipeline-kwargs '...'\n```\n",".ai/skills/model-integration/SKILL.md":"---\nname: model-integration\ndescription: >\n Use when adding a new model or pipeline to diffusers, setting up file\n structure for a new model, converting a pipeline to modular format, or\n converting weights for a new version of an already-supported model.\n---\n\n## Goal\n\nIntegrate a new model into diffusers end-to-end, to full numerical parity with the reference implementation — one workflow at a time.\n\n## Setup — gather before starting\n\nBefore writing any code, gather info in this order:\n\n1. **Reference repo** — ask for the github link. If they've already set it up locally, ask for the path. Otherwise, ask what setup steps are needed (install deps, download checkpoints, set env vars, etc.) and run through them before proceeding.\n2. **Inference script** — ask for a runnable end-to-end script for a basic workflow first (e.g. T2V). Then ask what other workflows they want to support (I2V, V2V, etc.) and agree on the full implementation order together.\n3. **Standard vs modular** — **default to modular.** [Modular Diffusers](references/modular.md) is the preferred implementation for new pipelines; the standard `DiffusionPipeline` is still supported but no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (modality baked into the checkpoint) or that are actively evolving.\n\nAsk step 3 as an `AskUserQuestion`, with modular marked as the recommended default.\n\nOnce you have everything, **confirm the plan** with the user before implementing — state exactly what you'll do, e.g. \"I'll integrate model X with pipeline Y based on your script, and verify the model matches the reference before considering it done.\"\n\nThen work through the **Integration checklist** below\n\n## Integration checklist\n\nA pipeline in Diffusers (be it standard or modular) will have multiple components. These components can be models, schedulers, processors, etc.\n\n- [ ] **Transformer model**\n - [ ] Implement the model with `from_pretrained` support (conventions: [models.md](references/models.md))\n - [ ] Convert weights (see **Weight / Checkpoint Conversion**)\n - [ ] Parity test against the reference (internal, not shipped — see **Model parity test**)\n - [ ] Register in the relevant `__init__.py` files (lazy imports)\n - [ ] Model-level tests (see **Testing**)\n- [ ] **VAE** (if applicable) — reuse an existing `AutoencoderKL*` if possible; if a new one is needed, follow the same sub-steps as the transformer\n- [ ] **Scheduler** — reuse an existing scheduler, or add a custom one\n- [ ] **Pipeline**\n - [ ] Implement the pipeline — see [modular.md](references/modular.md) for modular pipeline, or [pipelines.md](references/pipelines.md) for standard pipeline\n - [ ] Add a LoRA mixin if applicable\n - [ ] Register in the relevant `__init__.py` files (lazy imports)\n - [ ] Pipeline-level tests (see **Testing**)\n- [ ] **Docs** — see **File structure**\n- [ ] **Style** — `make style` and `make quality`\n\n## File structure\n\nA new model PR roughly lands these files (the contents of `pipelines//` and `modular_pipelines//` live in their guides):\n\n```\nsrc/diffusers/\n models/transformers/transformer_.py # the model (or models/autoencoders/, models/unets/)\n schedulers/scheduling_.py # only if a custom scheduler is needed\n loaders/lora_pipeline.py # LoRA mixin — add to the existing file\n pipelines// # standard pipeline — see pipelines.md\n modular_pipelines// # modular pipeline — see modular.md\ntests/\n models/transformers/test_models_transformer_.py\n pipelines//test_.py\ndocs/source/en/\n _toctree.yml # register the new pages in the docs index\n api/models/.md\n api/pipelines/.md\n```\n\n## Model integration specific rules\n\n**Match the reference's numerical logic.** Restructuring code to fit diffusers APIs (`ModelMixin`, `ConfigMixin`, blocks for modular, etc.) is expected, and required diffusers conventions (e.g. the attention pattern in [models.md](references/models.md)) take precedence. Beyond those, keep the actual computation as close to the reference as possible — don't reorder operations, change the math, or rename internals for aesthetics, even if it looks unclean. Small deviations make output mismatches very hard to track down.\n\n## Weight / Checkpoint Conversion\n\nConvert the original checkpoint into diffusers format with a standalone script under `scripts/` (e.g. `scripts/convert__to_diffusers.py`). The flow:\n\n1. Map the original state-dict keys to the diffusers module names (renames + any tensor surgery — see patterns below).\n2. Instantiate the diffusers model from its config and load the converted state dict.\n3. `save_pretrained(...)` to a local path, then load it back with `from_pretrained` to confirm it round-trips.\n\nAll weights load through the standard paths — `from_pretrained`, or `from_single_file` (add `FromSingleFileMixin` + a weight-mapping) for an original-format single checkpoint. No custom `from_pretrained`, no manual runtime loading. See the loading rule in [models.md](references/models.md).\n\nCommon conversion patterns to watch for model-level components:\n- Fused QKV weights that need splitting into separate Q, K, V\n- Scale/shift ordering differences (reference stores `[shift, scale]`, diffusers expects `[scale, shift]`)\n- Weight transpositions (linear stored as transposed conv, or vice versa)\n- Interleaved head dimensions that need reshaping\n- Bias terms absorbed into different layers\n\n## Testing\n\nTwo test layers must be added for any new pipeline: pipeline-level tests, and (if a new model is introduced) model-level tests. Conventions for both layers — file locations, tester mixins, dummy-component rules — live in [testing.md](references/testing.md); follow it when writing the tests.\n\n## Model parity test\n\nConfirm the diffusers implementation matches the reference. Test each component on **CPU/float32** with a strict tolerance (`max_diff < 1e-3`), comparing the **freshly converted** weights against the reference in a single script — both sides side by side, nothing saved to disk in between. See [pitfalls.md](references/pitfalls.md) for the common sources of numerical discrepancy.\n\nThis is an **internal verification tool for integration — it should not be shipped in the PR** (it imports the reference repo). The tests that ship with the PR are the model-level and pipeline-level tests in [testing.md](references/testing.md).\n\nThe example below is schematic (placeholder names). `ReferenceModel` is the component **imported from the original repo**, and `convert_my_component` is **the same conversion function you wrote for the conversion script for the component**. You should make sure both load the *same* checkpoint weights and run the *same* input, so any difference is a conversion or implementation bug — not a difference in inputs.\n\n```python\n@torch.inference_mode()\ndef test_my_component():\n # deterministic input — use the same shape & dtype the real model receives at this stage\n gen = torch.Generator().manual_seed(42)\n x = torch.randn(1, 16, 32, 32, generator=gen, dtype=torch.float32) # adjust to the real input shape\n\n original_state_dict = load_original_weights(...) # the original checkpoint — both sides load these same weights\n\n # reference: the original repo's implementation (load one model at a time to fit in CPU RAM)\n ref_model = ReferenceModel(config) # ReferenceModel: imported from the original repo\n ref_model.load_state_dict(original_state_dict, strict=True)\n ref_model = ref_model.float().eval()\n ref_out = ref_model(x).clone() # clone before freeing the model\n del ref_model\n\n # diffusers: convert those same weights with your conversion-script function, then run\n diff_model = convert_my_component(original_state_dict) # convert_my_component: the fn from convert__to_diffusers.py\n diff_model = diff_model.float().eval()\n diff_out = diff_model(x)\n\n max_diff = (ref_out - diff_out).abs().max().item()\n assert max_diff < 1e-3, f\"FAIL: max_diff={max_diff:.2e}\"\n```\n",".ai/skills/self-review/SKILL.md":"---\nname: self-review\ndescription: >\n Use before opening a PR, or whenever asked to self-review a diffusers\n contribution. Applies the same rubric as the `@claude` CI (checks the diff\n against references/review-rules.md, traces call paths for dead code). Reports findings grouped by\n severity, flagging what to fix before submitting (blocking issues + dead code)\n vs what to leave for the actual review. Report-only — does not edit files.\n---\n\n# Self-review\n\nRuns the same rubric as the `@claude` CI reviewer, so you catch issues before a\nmaintainer does — but over your **whole** PR diff. (The CI scopes itself to\n`src/diffusers/`, `tests/`, and `.ai/`; for your own PR, also review your docs\nand scripts.) You're already on the branch with the conventions loaded, so: get\nthe\ndiff → review it against the rubric → report → iterate with the contributor\nuntil it's ready, then remind them to share the final notes on the PR.\n\n## 1. Get the diff\n\n```bash\ngit diff main...HEAD # use your target branch if not main\n```\n\nIf the branch trails `main` and the diff looks polluted with unrelated merged\nfiles, scope to your own commits: `git log main..HEAD --oneline`, then\n`git show `.\n\n## 2. Read the rubric\n\n`references/review-rules.md` is the canonical rubric (the CI pins it from `main`) — read\nit and review against it; don't rely on a remembered copy. For the areas you\ntouched, also read `references/code_style.md`, `references/models.md`, `references/pipelines.md`,\n`references/modular.md`, `references/testing.md`, or `references/pitfalls.md`.\n\n## 3. Report\n\n- **Blocking issues** — numbered. Each: title → explanation → `file.py:line` →\n impact. Cite the rule, e.g. *Per `references/models.md`: \"…only keep the inference path.\"*\n- **Non-blocking issues** — same format, lower severity.\n- **Dead code (advisory)** — a table: `path:line` · Likely-dead / Used · reason.\n- **Summary** — short synthesis and a verdict (**READY** / **NEEDS CHANGES**),\n spelling out:\n - **Fix before submitting** — all blocking issues, and remove the flagged dead code.\n - **Leave for the actual review** — non-blocking issues that aren't obviously\n correct; raise these with the reviewer rather than guessing at them now.\n\nReport only — do not edit files. Be concrete, cite the rule, review the whole\ndiff, and don't invent issues or flag pure style.\n\n## 4. Iterate until ready, then share\n\nExpect several rounds: the contributor addresses findings, you review again.\nKeep working with them to fix as much as possible until the verdict is\n**READY** — the **Leave for the actual review** items are the only ones that\nshould reach the reviewer unresolved. End the final round's report by\nreminding the contributor to share it on the PR (description or a comment) —\nit saves the reviewer a few rounds of back-and-forth. Never commit the notes as\npart of the diff.\n",".claude-plugin/marketplace.json":"{\n \"name\": \"diffusers-skills\",\n \"description\": \"Agent plugins published from the diffusers repository\",\n \"owner\": {\n \"name\": \"The Hugging Face team\"\n },\n \"plugins\": [\n {\n \"name\": \"diffusers\",\n \"source\": \"./.ai\",\n \"description\": \"Conventions and task skills for contributing to diffusers\"\n }\n ]\n}\n","AGENTS.md":".ai/AGENTS.md","CLAUDE.md":".ai/AGENTS.md"},"files":{".ai/.claude-plugin/plugin.json":"{\n \"name\": \"diffusers\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"author\": {\n \"name\": \"The Hugging Face team\"\n },\n \"homepage\": \"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents\",\n \"repository\": \"https://github.com/huggingface/diffusers\",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"diffusers\",\n \"diffusion-models\",\n \"contributing\"\n ]\n}\n",".ai/.codex-plugin/plugin.json":"{\n \"name\": \"diffusers\",\n \"version\": \"0.1.0\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"skills\": \"./skills/\"\n}\n",".ai/AGENTS.md":"# Diffusers — Agent Guide\n\n## Setup\n\nWe recommend developing in a virtual environment managed by [uv](https://docs.astral.sh/uv/):\n\n```bash\nuv venv && source .venv/bin/activate\nuv pip install -e . # provides diffusers-cli\n```\n\nList the available skills, and what each one is for, with:\n\n```bash\ndiffusers-cli skills list\n```\n\nInstall them with:\n\n```bash\ndiffusers-cli skills add # install one, or --all for every skill\n # --claude / --codex / --cursor to pick one target\n```\n\n`diffusers-cli skills update` refreshes what you installed.\n\nSkills used to be installed through the `Makefile`, which symlinked the project-level `.claude/skills` and\n`.agents/skills` at `.ai/skills` in this repo. Remove those links if they exist — otherwise the install writes through\nthem into `.ai/` itself:\n\n```bash\nrm -f .claude/skills .agents/skills\n```\n\nAt the start of a session, confirm the setup and tell the user what you find:\n\n```bash\npython utils/check_ai.py # guides and skills are consistent\ndiffusers-cli skills list # which skills are installed\n```\n\nClaude Code and Codex can also install via plugins\n\n```bash\nclaude plugin marketplace add huggingface/diffusers\nclaude plugin install diffusers@diffusers-skills --scope project\n\ncodex plugin marketplace add huggingface/diffusers # then install from the Plugins Directory\n```\n\n\n## Human in the loop\n\nEverything a PR reviewer sees must come from a human, or be approved by one for the exact wording. That includes commit messages, code comments and docstrings, PR titles and descriptions, and any PR or issue comment, review, or reply. Ask whether the user wants you to draft that text or will write it themselves. Keep drafts short and easy to understand. When you hand one over, remind them to read it for real and check that it reads well for other humans.\n\n- **Don't commit unless the user approved the exact commit message, and never push or open a PR on\n your own.** The user decides when anything is published, each time.\n- **Don't post to GitHub directly** — no comments, reviews, or replies. Draft when asked and hand the text to the user.\n\n## Code formatting\n\n- `make style` and `make fix-copies` should be run before opening a PR\n\n## Reference guides\n\n- **Coding style** — see [code_style.md](references/code_style.md) for how code should read, and the `# Copied from` rules.\n- **Models** — see [models.md](references/models.md) for model conventions, attention pattern, implementation rules, dependencies, and gotchas. For adding or converting a model, use the [model-integration](skills/model-integration/SKILL.md) skill.\n- **Pipelines** — see [pipelines.md](references/pipelines.md) for pipeline conventions, patterns, and gotchas.\n- **Modular pipelines** — see [modular.md](references/modular.md) for modular pipeline conventions, patterns, and gotchas.\n- **Tests** — see [testing.md](references/testing.md) for test conventions: required test layers, tester mixins, and dummy-component rules.\n- **Reporting** — see [reporting.md](references/reporting.md) for how to write bug reports (what a reproduction is) and performance claims (end-to-end numbers first).\n\n## Skills\n\nTask-specific guides live in `.ai/skills/` and are loaded on demand by AI agents. Available skills include:\n\n- [model-integration](skills/model-integration/SKILL.md) (adding/converting pipelines)\n- [custom-blocks](skills/custom-blocks/SKILL.md) (packaging a `ModularPipelineBlocks` subclass for the Hub)\n- [diffusers-cli](skills/diffusers-cli/SKILL.md) (running pipelines, inspecting schemas, and using the Diffusers CLI)\n- [self-review](skills/self-review/SKILL.md) (pre-PR self-review against the project rules)\n\n## Self-review before a PR\n\nBefore opening a PR, run self-review against [review-rules.md](references/review-rules.md). The [self-review skill](skills/self-review/SKILL.md) runs this as the same pass the `@claude` CI reviewer uses. Share the final report on the PR (description or comment) — see the skill for details.\n",".ai/plugin.json":"{\n \"$schema\": \"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json\",\n \"name\": \"diffusers\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"author\": {\n \"name\": \"The Hugging Face team\"\n },\n \"homepage\": \"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents\",\n \"repository\": \"https://github.com/huggingface/diffusers\",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"diffusers\",\n \"diffusion-models\",\n \"contributing\"\n ]\n}\n",".ai/skills/custom-blocks/SKILL.md":"---\nname: custom-blocks\ndescription: >\n Use when the user has written (or wants to write) a `ModularPipelineBlocks`\n subclass in a local Python file and needs to package it into a Hub-uploadable\n directory. Covers the workflow from a single `block.py` file to a published\n custom-block repo that consumers can load via\n `ModularPipeline.from_pretrained(, trust_remote_code=True)`.\n---\n\n## What this skill is for\n\nA `ModularPipelineBlocks` subclass is a unit of pipeline logic — input/output spec plus a `__call__` — that\nslots into diffusers' modular pipeline composition. Once you have one defined locally, you almost always want to\npublish it as a small Hub repo so others can `from_pretrained` it. `diffusers-cli custom_blocks` automates the\npackaging step: it parses your Python file, instantiates the chosen block class, and writes a\n`save_pretrained`-style directory in your cwd that's ready to push to the Hub.\n\nUse this skill when:\n\n- The user is writing a custom modular block and asks \"how do I publish this?\" or \"package this for the Hub\".\n- The user has a `block.py` (or similar) file with one or more `ModularPipelineBlocks` subclasses.\n- You're scaffolding a new modular pipeline repo and need the on-disk layout that `ModularPipelineBlocks.from_pretrained`\n expects.\n\nDon't use this skill for: running an existing modular pipeline (`diffusers-cli run`), introspecting one\n(`diffusers-cli schema`), or writing the block class itself — this skill packages an *already-written* block.\n\n## The end-to-end workflow\n\n```\n[you: write block.py] → diffusers-cli custom_blocks → [packaged dir in cwd]\n ↓\n hf upload .\n ↓\n consumers: ModularPipeline.from_pretrained(, trust_remote_code=True)\n diffusers-cli schema --model --trust-remote-code\n diffusers-cli run --model --trust-remote-code ...\n```\n\nThe skill covers the middle box. The bookends (writing the block and uploading) are out of scope.\n\n## Command surface\n\n```bash\ndiffusers-cli custom_blocks [--block_module_name ] [--block_class_name ]\n```\n\n### Flags\n\n- `--block_module_name ` — Python file containing the block class. Defaults to `block.py` in the cwd.\n- `--block_class_name ` — Which class in the file to package. Optional: if omitted, the CLI parses the\n file with `ast`, finds every class that inherits from `ModularPipelineBlocks`, and uses the first one (with\n an info log naming the others). Specify explicitly when the file defines more than one block and you want a\n specific one.\n\n### What it does\n\n1. **AST scan**: parses `` without executing it, walks top-level `ClassDef` nodes, and collects every\n class whose `bases` include `ModularPipelineBlocks` or one of its composite subclasses\n (`SequentialPipelineBlocks`, `AutoPipelineBlocks`, `ConditionalPipelineBlocks`, `LoopSequentialPipelineBlocks`).\n2. **Pick a class**: uses `--block_class_name` if given, else the first found. Errors with the list of available\n classes if your name doesn't match.\n3. **Load and save**: imports the file via `importlib.util.spec_from_file_location` (this does execute the\n module — make sure your block.py is something you trust to run), instantiates the chosen class with no\n constructor args, and calls `.save_pretrained(os.getcwd())`.\n\nThe result is a Hub-uploadable directory laid out the way `ModularPipelineBlocks.from_pretrained` expects:\nyour block source, an `auto_map` in the config so consumers know to load it with `trust_remote_code=True`,\nand any artifacts `save_pretrained` writes for that block class.\n\n## End-to-end example\n\nGiven a `block.py` like:\n\n```python\nfrom diffusers.modular_pipelines import ModularPipelineBlocks, InputParam, OutputParam\n\nclass MyDenoiseBlock(ModularPipelineBlocks):\n model_name = \"my-denoise\"\n\n @property\n def inputs(self):\n return [\n InputParam(\"latents\", type_hint=\"torch.Tensor\", required=True, description=\"Noisy latents.\"),\n InputParam(\"guidance_scale\", type_hint=\"float\", default=7.5),\n ]\n\n @property\n def intermediate_outputs(self):\n return [OutputParam(\"latents\", type_hint=\"torch.Tensor\")]\n\n def __call__(self, components, state):\n # ... denoising logic ...\n return components, state\n```\n\nPackage it:\n\n```bash\ndiffusers-cli custom_blocks --block_module_name block.py\n```\n\nOutput in cwd:\n\n```\n./\n├── block.py\n├── modular_config.json # contains auto_map → MyDenoiseBlock\n└── (any state files MyDenoiseBlock.save_pretrained writes)\n```\n\nUpload to the Hub:\n\n```bash\nhf upload my-user/my-denoise-block .\n```\n\nConsumers can now use it:\n\n```python\nfrom diffusers import ModularPipeline\npipe = ModularPipeline.from_pretrained(\"my-user/my-denoise-block\", trust_remote_code=True)\n```\n\nOr via CLI:\n\n```bash\ndiffusers-cli schema --model my-user/my-denoise-block --trust-remote-code\ndiffusers-cli run --model my-user/my-denoise-block --trust-remote-code \\\n --pipeline-kwargs '{\"latents\": \"...\", \"guidance_scale\": 7.5}'\n```\n\n## Common errors\n\n- **`Could not parse '': SyntaxError`** — the file isn't valid Python. Fix the syntax; the AST step runs\n before any execution.\n- **`block_class_name could not be retrieved. Available classes from : [ClassA, ClassB]`** — your\n `--block_class_name` doesn't match any `ModularPipelineBlocks` subclass found. Pick from the list shown.\n- **No classes found**: silent — the command will try to use the first entry in an empty list and raise\n `IndexError`. If you hit that, double-check your class actually inherits from `ModularPipelineBlocks` or one\n of the composite classes listed above (the AST scan looks for those literal class names; aliased imports like\n `from diffusers import ... as MPB` won't be picked up).\n- **Block requires constructor args**: the command calls `()` with no args. If your block needs\n `__init__` parameters, refactor to take them from `state`/`components` at `__call__` time instead, or\n hardcode defaults in `__init__`.\n\n## Verifying the install\n\nIf `diffusers-cli` isn't on PATH after `pip install -e .`, reinstall with\n`pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the binary is missing recent\nfeatures (e.g. `unrecognized arguments: --lora`), reinstall. See the\n[`diffusers-cli` skill](https://github.com/huggingface/diffusers/blob/main/.ai/skills/diffusers-cli/SKILL.md#verifying-the-cli-is-installed) for more.\n\n## Related\n\n- the [`diffusers-cli` docs](https://github.com/huggingface/diffusers/blob/main/docs/source/en/using-diffusers/cli.md) — once your block\n is uploaded, `schema` and `run` call it from the terminal without writing Python.\n- diffusers' [modular pipelines docs](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) — for writing the block\n class itself.\n",".ai/skills/diffusers-cli/SKILL.md":"---\nname: diffusers-cli\ndescription: >\n Use when the user wants to run a diffusers pipeline from a terminal (one-off\n generation, batch jobs, smoke-testing a new model), run on HF Sandbox\n hardware via `--remote`, introspect a pipeline's input schema before\n calling it, or attach a LoRA at inference time. Prefer this over writing\n ad-hoc Python scripts for generation tasks.\n---\n\n## Overview\n\n`diffusers-cli` is the shipped CLI in `src/diffusers/commands/`. Subcommands relevant to agentic use:\n\n| Command | Purpose |\n| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `run` | Run any `DiffusionPipeline` or `ModularPipeline`. Forwards `--pipeline-kwargs` verbatim, saves output by detecting its runtime type, optionally runs on HF Jobs via `--remote`. |\n| `schema` | Print the input schema for a pipeline repo (kwarg names, types, defaults, descriptions). **No weights downloaded** — only the small index file. |\n| `custom_blocks` | Package a local `ModularPipelineBlocks` subclass for the Hub. |\n| `env` | Print versions of diffusers + torch + transformers + accelerate + safetensors + CUDA + GPU info. Use when investigating environment issues, dtype/precision support, or building bug reports. |\n\n## When to read which file\n\nMost agentic work goes through `run`. Read the matching reference file before constructing a command:\n\n- **[`run.md`](run.md)** — full reference for `diffusers-cli run`. Covers `--pipeline-kwargs`\n semantics and the shell-quoting gotcha, LoRA via `--lora`, optimization flags (`--dtype`, `--cpu-offload`,\n `--attention-backend`, `--vae-tiling/slicing`), output handling and `--push-to` bucket uploads, the full\n `--remote` HF Jobs flow (image, container command, log streaming, timing payload, artifact download), and\n context parallel (`--context-parallel`) for both local-torchrun and `--remote` paths.\n\nThe other commands are small enough that `diffusers-cli --help` is the canonical reference:\n\n```bash\ndiffusers-cli schema --help\ndiffusers-cli custom_blocks --help\ndiffusers-cli env --help\n```\n\n## When NOT to use this skill\n\n- Multi-stage workflows where you need intermediate tensor manipulation between pipelines → write Python.\n- Training or fine-tuning → CLI only covers inference.\n- Anything requiring `quantization_config` or other low-level loader knobs not exposed by the CLI flags → write\n Python. (`device_map` is exposed as `--device-map`; see [run.md](run.md#optimization-flags).)\n\n## Verifying the CLI is installed\n\nThe console entry point is registered in `pyproject.toml` (`diffusers-cli =\n\"diffusers.commands.diffusers_cli:main\"`). If `diffusers-cli` is not on PATH after `pip install -e .`, reinstall\nwith `pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the installed binary is\nmissing recent features (e.g. you see `unrecognized arguments: --lora`), reinstall.\n\n## Output formats\n\n`--format {auto, human, agent, json}` (top-level flag, must appear before the subcommand):\n\n- **`human`** — plain-text indented output for terminals (default when not running under an agent harness). No ANSI color.\n- **`agent`** — TSV tables and `key=value` lines. Auto-selected when an agent env var is present\n (`CLAUDECODE`, `CLAUDE_CODE`, `CODEX_SANDBOX`, `CURSOR_AI`, `AIDER_AI_CONTEXT`, `GH_COPILOT_AGENT`,\n `AI_AGENT`). Token-cheap for LLM agents to read.\n- **`json`** — compact JSON. Use for programmatic parsing (scripts, services) where type fidelity and nested\n structures matter.\n\n`stdout` carries data; `stderr` carries hints/warnings/progress — parseable output is never polluted.\n\nRule of thumb: `--format json` for scripts that will `json.loads()` the output, otherwise leave it on\nauto-detect (`agent` for LLMs, `human` for terminals).\n",".ai/skills/diffusers-cli/run.md":"# `diffusers-cli run` — reference\n\nFull surface for `diffusers-cli run`. Use this file as the source of truth when constructing a `run`\ninvocation. The top-level [`SKILL.md`](SKILL.md) covers when to use the CLI; this file covers how.\n\n## The schema → run flow\n\nFor any model you haven't called before, run `schema` first to learn its input contract, then `run` with\nthe right `--pipeline-kwargs`:\n\n```bash\n# 1. Discover what kwargs the pipeline takes (no weight download)\ndiffusers-cli --format json schema --model black-forest-labs/FLUX.2-klein-9B\n\n# 2. Run it\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --dtype bf16\n```\n\n`schema --format json` emits a `{task, model, pipeline_class, inputs[]}` payload where each input is\n`{name, type_hint, default, required, description}`.\n\n## Standard vs modular detection\n\n`run` auto-detects which kind of pipeline it's calling:\n\n1. If `model_index.json` exists on the repo → `DiffusionPipeline.from_pretrained` path.\n2. Otherwise → `ModularPipeline.from_pretrained` path.\n\nYou don't need to tell it which. Modular repos must pass `--trust-remote-code` if they ship custom block code.\n\n## `--pipeline-kwargs` semantics\n\nA JSON object passed straight through to `pipeline(**kwargs)`. String values at known media-input keys are\nauto-loaded before the pipeline is called:\n\n- **Images** (`image`, `mask_image`, `control_image`, `ip_adapter_image`, `image_2`) → `PIL.Image.Image`\n via `diffusers.utils.load_image`. Accepts URLs or local paths.\n- **Videos** (`video`, `control_video`) → `list[PIL.Image.Image]` via `diffusers.utils.load_video`.\n- **Audio** (`initial_audio_waveforms`, `reference_audio`, `src_audio`) → `torch.Tensor` via `torchaudio.load`.\n For `initial_audio_waveforms`, the file's native sample rate is auto-written to\n `initial_audio_sampling_rate` if you didn't pass it explicitly.\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\", \"prompt\": \"make the fur grey\", \"strength\": 0.6}'\n```\n\nMedia resolution runs **before** the pipeline weights load, so a dead URL or missing file fails within\nseconds instead of after a multi-minute model download.\n\n**Batched inputs**: any media key (and `prompt`) accepts a JSON array — each string in the list is loaded\nindividually, and the pipeline processes the whole list in one forward pass on the GPU. Use this when you\nhave several inputs that share flavor/model/dtype and want them batched:\n\n```bash\ndiffusers-cli run --model black-forest-labs/FLUX.1-Kontext-dev \\\n --pipeline-kwargs '{\"prompt\": [\"make it grey\", \"make it pink\", \"make it blue\"],\n \"image\": [\"https://.../cat1.png\", \"https://.../cat2.png\", \"https://.../cat3.png\"]}'\n```\n\n**Shell-quoting gotcha**: the JSON must be on one line (or use `\\` to line-continue). A literal newline inside the\nsingle-quoted argument lands as a raw control char inside the string and breaks `json.loads`.\n\n## LoRA adapters (`--lora`)\n\nAttach one or more LoRAs after the pipeline loads via a JSON spec. `--lora` accepts a single object or a list.\n\nSingle adapter:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"a tiny grey cat\"}' \\\n --lora '{\"lora_id\": \"alvdansen/littletinies\", \"lora_scale\": 0.8}'\n```\n\nMultiple stacked adapters:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"a tiny grey cat\"}' \\\n --lora '[\n {\"lora_id\": \"alvdansen/littletinies\", \"lora_scale\": 0.5, \"adapter_name\": \"style\"},\n {\"lora_id\": \"author/detail-boost\", \"lora_scale\": 0.3}\n ]'\n```\n\nPer-entry fields: `lora_id` (required), `lora_scale` (optional, default `1.0`), `adapter_name` (optional; when\nomitted while stacking, auto-generated as `lora_0`, `lora_1`, …; single-entry defaults to `\"default\"`).\n\nEach entry calls `pipeline.load_lora_weights(, adapter_name=)`. After all adapters load, one\n`pipeline.set_adapters(names, adapter_weights=scales)` activates them together.\n\n## Optimization flags\n\n- `--dtype {auto, bf16, fp16, fp32, …}` — pipeline weight dtype. `bf16` is the right default for modern DiTs on\n A100/H100.\n- `--device-map ` — component placement, forwarded to `from_pretrained(device_map=...)`. Accepts a plain\n torch device (`cuda`, `cuda:0`, `cpu`, `mps`), the string `balanced` (auto-splits pipeline components across\n visible GPUs), or a JSON dict `{\"transformer\": \"cuda:0\", \"vae\": \"cuda:1\"}` for explicit per-component\n placement. Auto-detects if omitted (pinned to `cuda:$LOCAL_RANK` under torchrun). `balanced` and dict values\n are incompatible with `--cpu-offload`.\n- `--cpu-offload {model, group}` — `model` uses `enable_model_cpu_offload`, `group` uses\n `enable_group_offload(offload_type=\"leaf_level\", use_stream=True)`. Use `group` to fit a 9B+ model on a single\n A100. Onload target device comes from `--device-map` (must be a plain device string in this case).\n- `--attention-backend {default, flash_hub, flash_varlen_hub, flash_4_hub, sage_hub, sage_blackwell_hub}` — hub-hosted kernels,\n auto-downloaded on first use. Failures (kernel not available, CUDA arch mismatch, network) raise a clear\n `SystemExit` listing the alternatives instead of silently reverting to the default. Only supported on\n transformer-based pipelines; UNet pipelines get a `logger.warning` and the flag is ignored.\n- `--vae-tiling` / `--vae-slicing` — lower peak VAE decode VRAM.\n- `--compile [JSON]` — `torch.compile` every denoiser submodule. See [Compile](#compile) below.\n- `--context-parallel` — Ulysses-style context parallelism on a DiT. See [Context parallel](#context-parallel) below.\n\n## Output handling\n\n`run` detects the pipeline return type and saves accordingly:\n\n- `PIL.Image` / list of them → `.png` (zero-padded, e.g. `0000.png`)\n- Frame sequence (≥2 PILs or ndarrays) → `0000.mp4` (uses `--fps`, default 8)\n- Numpy audio array → `0000.wav` (uses `--sampling-rate`)\n- Anything else → JSON dump\n\n**Default output directory** is `~/.diffusers/cli/run/outputs/diffusers-run--/`.\nEach run gets its own subdirectory so consecutive invocations don't overwrite each other. The same run id\nis used for the local dir, the sandbox's `DIFFUSERS_CLI_RUN_ID` env var, and the sandbox input/output paths\nunder `--remote`, so a run is traceable end-to-end.\n\nOverride the destination with `--output ` (file or directory). Explicit `--output` bypasses the\n`diffusers-run-*` namespace — files land flat in the path you gave.\n\n### `--push-to`\n\nUpload outputs to an HF bucket. Objects land under `//`. The bucket is created\nif it doesn't exist. Behavior interacts with `--output` and `--remote`:\n\n| flags | download to local? | bucket write? |\n|---|---|---|\n| (neither) | ✅ `~/.diffusers/cli/run/outputs//` | — |\n| `--output /path` | ✅ `/path/` | — |\n| `--push-to my/bucket` | ❌ | ✅ (bucket only) |\n| `--push-to my/bucket --output /path` | ✅ | ✅ (both) |\n\nThe rule: explicit `--push-to` means \"the bucket is my destination\" — skip the local download unless the\nuser also explicitly asked for a local target via `--output`.\n\n## Remote execution (`--remote`)\n\nAdd `--remote` to run the same call inside a [Hugging Face Sandbox](https://huggingface.co/docs/huggingface_hub/en/guides/sandbox)\n— an isolated cloud VM (built on HF Jobs) the CLI drives over HTTP: it uploads inputs, runs the pipeline,\ndownloads outputs, then terminates the sandbox.\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --remote --flavor a100-large \\\n --dtype bf16 \\\n --cpu-offload group\n```\n\nWhat happens:\n\n1. Your HF token is picked up (from `--token` or your login) and forwarded into the sandbox as `HF_TOKEN`.\n2. `--pipeline-kwargs` are parsed locally so JSON errors fail fast (no wasted sandbox time).\n3. A dedicated sandbox is created on `--flavor` from `diffusers/diffusers-cli-cuda:latest` by default — a\n prebuilt image (`docker/diffusers-cli-cuda/` in this repo, rebuilt nightly) that already ships torch,\n CUDA, `diffusers`, and the rest of the CLI's deps. Any local file paths in `--pipeline-kwargs` are\n uploaded into the sandbox under `/tmp/diffusers-cli/inputs//` via native file transfer (no\n bucket), and the JSON paths are rewritten to point at them.\n4. No dependency install runs on the default image — that step is skipped unless you passed `--dependencies`\n (only the extras are installed then) or pointed `--image` somewhere else (the full set is installed).\n Output streams live to your terminal.\n5. The sandbox CLI writes outputs to `/tmp/diffusers-cli/outputs/`; the CLI downloads every artifact back into\n the local target (see [`--push-to`](#-push-to) for when the download is skipped).\n6. The sandbox is terminated (unless `--keep-alive`/`--sandbox-id`), and the wallclock `run_seconds` for the\n pipeline command is printed and included in the JSON payload.\n\nFlags:\n\n- `--flavor ` — sandbox hardware (e.g. `a10g-small`, `a100-large`, `4xa100-large`).\n- `--timeout ` — max wallclock for the run command inside the sandbox (e.g. `30m`, `2h`). Defaults to `10m`.\n- `--dependencies ` — extra pip deps (repeatable), installed on top of whatever the image ships.\n- `--namespace ` — create the sandbox under a different account.\n- `--push-to ` — see [`--push-to`](#-push-to) above. The upload runs inside the sandbox; an explicit\n value with no `--output` makes the bucket the sole destination and skips the local download.\n- `--image ` — override the sandbox image. Must ship torch + CUDA; the CLI then installs the small\n Python deps on top via `uv pip install --system` on every cold sandbox, which the default image avoids.\n Useful for pinning a specific torch or bundling extra system libs.\n- `--volume [:]` — mount an HF storage bucket into the sandbox as a read-write directory.\n Repeatable. Default mount is `/mnt/buckets/`. Reference mounted files from `--pipeline-kwargs`\n like any other local path — no upload happens, the container reads straight from the FUSE mount. Applied\n only on new sandbox creation — ignored when reconnecting via `--sandbox-id`. Enables batch workflows:\n loop over bucket contents on the host, one `--sandbox-id` call per input.\n- `--keep-alive` — don't terminate the sandbox after the run; its id is printed so a later run can reconnect.\n- `--sandbox-id ` — reconnect to a kept-alive sandbox instead of creating a new one. Implies `--keep-alive`.\n- `--idle-timeout ` — auto-shutdown after this much inactivity (default 10m). The billing backstop\n for kept-alive sandboxes. Only applied on new sandbox creation — ignored when reconnecting via `--sandbox-id`.\n\n### Reusing a warm sandbox (`--keep-alive` / `--sandbox-id`)\n\nBy default each `--remote` run is ephemeral: create → run → download → kill. The cold cost (dep install +\nmodel weight download + any compile) is paid every time. For iterative work, keep one sandbox warm and reuse\nit — deps, the HF weight cache, and the `torch.compile` cache all survive on its disk:\n\n```bash\n# First run: create + keep alive. Prints sandbox_id=.\ndiffusers-cli run -m black-forest-labs/FLUX.2-klein-9B --dtype bf16 \\\n --pipeline-kwargs '{\"prompt\": \"a cat\"}' --remote --flavor a100-large --keep-alive\n\n# Subsequent runs: reconnect — model already cached, only inference runs.\ndiffusers-cli run -m black-forest-labs/FLUX.2-klein-9B --dtype bf16 \\\n --pipeline-kwargs '{\"prompt\": \"a dog\"}' --remote --flavor a100-large --sandbox-id \n\n# Stop it when done (or let --idle-timeout reap it).\nhf sandbox kill \n```\n\nNotes on `--remote` argv forwarding: flags that control how the run is dispatched (`--flavor`, `--timeout`,\n`--namespace`, `--dependencies`, `--image`, `--keep-alive`, `--sandbox-id`, `--idle-timeout`, `--format`) are\nstripped before the argv is rebuilt for the sandbox CLI. Everything else — model, dtype, pipeline-kwargs,\noptimizations, output flags — is forwarded verbatim (`--output` is repointed at the sandbox output dir).\n\n## Compile\n\n`--compile` runs `torch.compile` over every `transformer*` / `unet*` submodule on the pipeline. Prefers\nregional compilation via `module.compile_repeated_blocks(**kwargs)` when the model exposes `_repeated_blocks`\n— this only compiles the repeated inner blocks (the bulk of the compute) rather than the whole module, so\nfirst-step latency is much lower. Falls back to full `torch.compile(module, **kwargs)` when no regional\nmetadata is declared.\n\n```bash\n# Bare — uses fullgraph=true\ndiffusers-cli run --model --dtype bf16 --compile --pipeline-kwargs '...'\n\n# With kwargs forwarded to torch.compile\ndiffusers-cli run --model --dtype bf16 \\\n --compile '{\"mode\": \"max-autotune\", \"fullgraph\": true}' \\\n --pipeline-kwargs '...'\n```\n\n**When it's worth it**: multi-step generation (~50+ denoising steps). You pay a one-time compilation cost on\nthe first step, then every subsequent step is faster.\n\n**Under `--remote`**: on an ephemeral sandbox the compile cache doesn't survive, so the compilation cost is\npaid on every run — `--compile` only breaks even on very long generations. On a `--keep-alive`/`--sandbox-id`\nsandbox the compile cache persists on disk, so the cost is paid once and reused: keep a warm sandbox and\n`--compile` pays off across runs.\n\n`--compile` is **currently not supported with `--context-parallel`** — CP shards attention across ranks while\nregional compile assumes a stable single-device graph. If both are set, the CLI logs a warning and skips the\ncompile step; CP still runs.\n\n## Context parallel\n\n`--context-parallel` enables Ulysses CP on a DiT-based pipeline. **Locally** the user must launch via torchrun:\n\n```bash\ntorchrun --nproc-per-node=2 -m diffusers.commands.diffusers_cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\"}' \\\n --dtype bf16 \\\n --context-parallel\n```\n\n**Remotely** the CLI handles the torchrun wrapping — just pass `--context-parallel` to a `--remote` invocation on\na multi-GPU flavor:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --remote --flavor 4xa100-large \\\n --dtype bf16 \\\n --context-parallel\n```\n\nInside the sandbox, CP swaps the entrypoint to `torchrun --nproc-per-node=gpu -m\ndiffusers.commands.diffusers_cli`, initializes a hybrid process group (`cpu:gloo,cuda:nccl` — NCCL for the\nattention all-to-all, Gloo for `ulysses_anything`'s per-rank size coordination), pins each rank to\n`cuda:{LOCAL_RANK}`, and gates output saving/printing to rank 0 only.\n\n**Memory note**: CP shards the sequence, **not the weights**. Every rank still holds the full transformer. Wins\nare wall-clock attention speedup and headroom for very long sequences, not \"fit a model that doesn't fit.\" For\nweight sharding you'd want TP or FSDP — not exposed in the CLI yet.\n\nCP is DiT-only. UNet pipelines raise a clear error directing you to a DiT pipeline (FLUX, SD3, HunyuanDiT,\nAuraFlow, …).\n\n## Output mode (`--format`)\n\n`--format` controls the shape of **stdout metadata** (which paths were written, timing, sandbox id, pushed\nbucket URLs) — **not** the media file format. Written images are always PNG, videos MP4, audio WAV; only the\nsummary printed alongside them changes shape.\n\nThe CLI auto-detects when running under an AI coding agent (Claude Code, Cursor, Aider, GH Copilot Agent — via\n`CLAUDECODE`, `CLAUDE_CODE`, `CURSOR_AI`, `AIDER_AI_CONTEXT`, `GH_COPILOT_AGENT`) and switches to **agent\nmode** automatically — TSV tables, `key=value` results, compact JSON dicts, no progress bars.\n\nOverride explicitly with `--format {auto, human, agent, json, quiet}` placed **before** the subcommand:\n\n```bash\ndiffusers-cli --format json run --model --pipeline-kwargs '...'\n```\n",".ai/skills/model-integration/SKILL.md":"---\nname: model-integration\ndescription: >\n Use when adding a new model or pipeline to diffusers, setting up file\n structure for a new model, converting a pipeline to modular format, or\n converting weights for a new version of an already-supported model.\n---\n\n## Goal\n\nIntegrate a new model into diffusers end-to-end, to full numerical parity with the reference implementation — one workflow at a time.\n\n## Setup — gather before starting\n\nBefore writing any code, gather info in this order:\n\n1. **Reference repo** — ask for the github link. If they've already set it up locally, ask for the path. Otherwise, ask what setup steps are needed (install deps, download checkpoints, set env vars, etc.) and run through them before proceeding.\n2. **Inference script** — ask for a runnable end-to-end script for a basic workflow first (e.g. T2V). Then ask what other workflows they want to support (I2V, V2V, etc.) and agree on the full implementation order together.\n3. **Standard vs modular** — **default to modular.** [Modular Diffusers](references/modular.md) is the preferred implementation for new pipelines; the standard `DiffusionPipeline` is still supported but no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (modality baked into the checkpoint) or that are actively evolving.\n\nAsk step 3 as an `AskUserQuestion`, with modular marked as the recommended default.\n\nOnce you have everything, **confirm the plan** with the user before implementing — state exactly what you'll do, e.g. \"I'll integrate model X with pipeline Y based on your script, and verify the model matches the reference before considering it done.\"\n\nThen work through the **Integration checklist** below\n\n## Integration checklist\n\nA pipeline in Diffusers (be it standard or modular) will have multiple components. These components can be models, schedulers, processors, etc.\n\n- [ ] **Transformer model**\n - [ ] Implement the model with `from_pretrained` support (conventions: [models.md](references/models.md))\n - [ ] Convert weights (see **Weight / Checkpoint Conversion**)\n - [ ] Parity test against the reference (internal, not shipped — see **Model parity test**)\n - [ ] Register in the relevant `__init__.py` files (lazy imports)\n - [ ] Model-level tests (see **Testing**)\n- [ ] **VAE** (if applicable) — reuse an existing `AutoencoderKL*` if possible; if a new one is needed, follow the same sub-steps as the transformer\n- [ ] **Scheduler** — reuse an existing scheduler, or add a custom one\n- [ ] **Pipeline**\n - [ ] Implement the pipeline — see [modular.md](references/modular.md) for modular pipeline, or [pipelines.md](references/pipelines.md) for standard pipeline\n - [ ] Add a LoRA mixin if applicable\n - [ ] Register in the relevant `__init__.py` files (lazy imports)\n - [ ] Pipeline-level tests (see **Testing**)\n- [ ] **Docs** — see **File structure**\n- [ ] **Style** — `make style` and `make quality`\n\n## File structure\n\nA new model PR roughly lands these files (the contents of `pipelines//` and `modular_pipelines//` live in their guides):\n\n```\nsrc/diffusers/\n models/transformers/transformer_.py # the model (or models/autoencoders/, models/unets/)\n schedulers/scheduling_.py # only if a custom scheduler is needed\n loaders/lora_pipeline.py # LoRA mixin — add to the existing file\n pipelines// # standard pipeline — see pipelines.md\n modular_pipelines// # modular pipeline — see modular.md\ntests/\n models/transformers/test_models_transformer_.py\n pipelines//test_.py\ndocs/source/en/\n _toctree.yml # register the new pages in the docs index\n api/models/.md\n api/pipelines/.md\n```\n\n## Model integration specific rules\n\n**Match the reference's numerical logic.** Restructuring code to fit diffusers APIs (`ModelMixin`, `ConfigMixin`, blocks for modular, etc.) is expected, and required diffusers conventions (e.g. the attention pattern in [models.md](references/models.md)) take precedence. Beyond those, keep the actual computation as close to the reference as possible — don't reorder operations, change the math, or rename internals for aesthetics, even if it looks unclean. Small deviations make output mismatches very hard to track down.\n\n## Weight / Checkpoint Conversion\n\nConvert the original checkpoint into diffusers format with a standalone script under `scripts/` (e.g. `scripts/convert__to_diffusers.py`). The flow:\n\n1. Map the original state-dict keys to the diffusers module names (renames + any tensor surgery — see patterns below).\n2. Instantiate the diffusers model from its config and load the converted state dict.\n3. `save_pretrained(...)` to a local path, then load it back with `from_pretrained` to confirm it round-trips.\n\nAll weights load through the standard paths — `from_pretrained`, or `from_single_file` (add `FromSingleFileMixin` + a weight-mapping) for an original-format single checkpoint. No custom `from_pretrained`, no manual runtime loading. See the loading rule in [models.md](references/models.md).\n\nCommon conversion patterns to watch for model-level components:\n- Fused QKV weights that need splitting into separate Q, K, V\n- Scale/shift ordering differences (reference stores `[shift, scale]`, diffusers expects `[scale, shift]`)\n- Weight transpositions (linear stored as transposed conv, or vice versa)\n- Interleaved head dimensions that need reshaping\n- Bias terms absorbed into different layers\n\n## Testing\n\nTwo test layers must be added for any new pipeline: pipeline-level tests, and (if a new model is introduced) model-level tests. Conventions for both layers — file locations, tester mixins, dummy-component rules — live in [testing.md](references/testing.md); follow it when writing the tests.\n\n## Model parity test\n\nConfirm the diffusers implementation matches the reference. Test each component on **CPU/float32** with a strict tolerance (`max_diff < 1e-3`), comparing the **freshly converted** weights against the reference in a single script — both sides side by side, nothing saved to disk in between. See [pitfalls.md](references/pitfalls.md) for the common sources of numerical discrepancy.\n\nThis is an **internal verification tool for integration — it should not be shipped in the PR** (it imports the reference repo). The tests that ship with the PR are the model-level and pipeline-level tests in [testing.md](references/testing.md).\n\nThe example below is schematic (placeholder names). `ReferenceModel` is the component **imported from the original repo**, and `convert_my_component` is **the same conversion function you wrote for the conversion script for the component**. You should make sure both load the *same* checkpoint weights and run the *same* input, so any difference is a conversion or implementation bug — not a difference in inputs.\n\n```python\n@torch.inference_mode()\ndef test_my_component():\n # deterministic input — use the same shape & dtype the real model receives at this stage\n gen = torch.Generator().manual_seed(42)\n x = torch.randn(1, 16, 32, 32, generator=gen, dtype=torch.float32) # adjust to the real input shape\n\n original_state_dict = load_original_weights(...) # the original checkpoint — both sides load these same weights\n\n # reference: the original repo's implementation (load one model at a time to fit in CPU RAM)\n ref_model = ReferenceModel(config) # ReferenceModel: imported from the original repo\n ref_model.load_state_dict(original_state_dict, strict=True)\n ref_model = ref_model.float().eval()\n ref_out = ref_model(x).clone() # clone before freeing the model\n del ref_model\n\n # diffusers: convert those same weights with your conversion-script function, then run\n diff_model = convert_my_component(original_state_dict) # convert_my_component: the fn from convert__to_diffusers.py\n diff_model = diff_model.float().eval()\n diff_out = diff_model(x)\n\n max_diff = (ref_out - diff_out).abs().max().item()\n assert max_diff < 1e-3, f\"FAIL: max_diff={max_diff:.2e}\"\n```\n",".ai/skills/self-review/SKILL.md":"---\nname: self-review\ndescription: >\n Use before opening a PR, or whenever asked to self-review a diffusers\n contribution. Applies the same rubric as the `@claude` CI (checks the diff\n against references/review-rules.md, traces call paths for dead code). Reports findings grouped by\n severity, flagging what to fix before submitting (blocking issues + dead code)\n vs what to leave for the actual review. Report-only — does not edit files.\n---\n\n# Self-review\n\nRuns the same rubric as the `@claude` CI reviewer, so you catch issues before a\nmaintainer does — but over your **whole** PR diff. (The CI scopes itself to\n`src/diffusers/`, `tests/`, and `.ai/`; for your own PR, also review your docs\nand scripts.) You're already on the branch with the conventions loaded, so: get\nthe\ndiff → review it against the rubric → report → iterate with the contributor\nuntil it's ready, then remind them to share the final notes on the PR.\n\n## 1. Get the diff\n\n```bash\ngit diff main...HEAD # use your target branch if not main\n```\n\nIf the branch trails `main` and the diff looks polluted with unrelated merged\nfiles, scope to your own commits: `git log main..HEAD --oneline`, then\n`git show `.\n\n## 2. Read the rubric\n\n`references/review-rules.md` is the canonical rubric (the CI pins it from `main`) — read\nit and review against it; don't rely on a remembered copy. For the areas you\ntouched, also read `references/code_style.md`, `references/models.md`, `references/pipelines.md`,\n`references/modular.md`, `references/testing.md`, or `references/pitfalls.md`.\n\n## 3. Report\n\n- **Blocking issues** — numbered. Each: title → explanation → `file.py:line` →\n impact. Cite the rule, e.g. *Per `references/models.md`: \"…only keep the inference path.\"*\n- **Non-blocking issues** — same format, lower severity.\n- **Dead code (advisory)** — a table: `path:line` · Likely-dead / Used · reason.\n- **Summary** — short synthesis and a verdict (**READY** / **NEEDS CHANGES**),\n spelling out:\n - **Fix before submitting** — all blocking issues, and remove the flagged dead code.\n - **Leave for the actual review** — non-blocking issues that aren't obviously\n correct; raise these with the reviewer rather than guessing at them now.\n\nReport only — do not edit files. Be concrete, cite the rule, review the whole\ndiff, and don't invent issues or flag pure style.\n\n## 4. Iterate until ready, then share\n\nExpect several rounds: the contributor addresses findings, you review again.\nKeep working with them to fix as much as possible until the verdict is\n**READY** — the **Leave for the actual review** items are the only ones that\nshould reach the reviewer unresolved. End the final round's report by\nreminding the contributor to share it on the PR (description or a comment) —\nit saves the reviewer a few rounds of back-and-forth. Never commit the notes as\npart of the diff.\n",".claude-plugin/marketplace.json":"{\n \"name\": \"diffusers-skills\",\n \"description\": \"Agent plugins published from the diffusers repository\",\n \"owner\": {\n \"name\": \"The Hugging Face team\"\n },\n \"plugins\": [\n {\n \"name\": \"diffusers\",\n \"source\": \"./.ai\",\n \"description\": \"Conventions and task skills for contributing to diffusers\"\n }\n ]\n}\n","AGENTS.md":".ai/AGENTS.md","CLAUDE.md":".ai/AGENTS.md"},"items":[{"name":"AGENTS.md","path":".ai/AGENTS.md","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/AGENTS.md","title":".ai - AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"# Diffusers — Agent Guide\n\n## Setup\n\nWe recommend developing in a virtual environment managed by [uv](https://docs.astral.sh/uv/):\n\n```bash\nuv venv && source .venv/bin/activate\nuv pip install -e . # provides diffusers-cli\n```\n\nList the available skills, and what each one is for, with:\n\n```bash\ndiffusers-cli skills list\n```\n\nInstall them with:\n\n```bash\ndiffusers-cli skills add # install one, or --all for every skill\n # --claude / --codex / --cursor to pick one target\n```\n\n`diffusers-cli skills update` refreshes what you installed.\n\nSkills used to be installed through the `Makefile`, which symlinked the project-level `.claude/skills` and\n`.agents/skills` at `.ai/skills` in this repo. Remove those links if they exist — otherwise the install writes through\nthem into `.ai/` itself:\n\n```bash\nrm -f .claude/skills .agents/skills\n```\n\nAt the start of a session, confirm the setup and tell the user what you find:\n\n```bash\npython utils/check_ai.py # guides and skills are consistent\ndiffusers-cli skills list # which skills are installed\n```\n\nClaude Code and Codex can also install via plugins\n\n```bash\nclaude plugin marketplace add huggingface/diffusers\nclaude plugin install diffusers@diffusers-skills --scope project\n\ncodex plugin marketplace add huggingface/diffusers # then install from the Plugins Directory\n```\n\n\n## Human in the loop\n\nEverything a PR reviewer sees must come from a human, or be approved by one for the exact wording. That includes commit messages, code comments and docstrings, PR titles and descriptions, and any PR or issue comment, review, or reply. Ask whether the user wants you to draft that text or will write it themselves. Keep drafts short and easy to understand. When you hand one over, remind them to read it for real and check that it reads well for other humans.\n\n- **Don't commit unless the user approved the exact commit message, and never push or open a PR on\n your own.** The user decides when anything is published, each time.\n- **Don't post to GitHub directly** — no comments, reviews, or replies. Draft when asked and hand the text to the user.\n\n## Code formatting\n\n- `make style` and `make fix-copies` should be run before opening a PR\n\n## Reference guides\n\n- **Coding style** — see [code_style.md](references/code_style.md) for how code should read, and the `# Copied from` rules.\n- **Models** — see [models.md](references/models.md) for model conventions, attention pattern, implementation rules, dependencies, and gotchas. For adding or converting a model, use the [model-integration](skills/model-integration/SKILL.md) skill.\n- **Pipelines** — see [pipelines.md](references/pipelines.md) for pipeline conventions, patterns, and gotchas.\n- **Modular pipelines** — see [modular.md](references/modular.md) for modular pipeline conventions, patterns, and gotchas.\n- **Tests** — see [testing.md](references/testing.md) for test conventions: required test layers, tester mixins, and dummy-component rules.\n- **Reporting** — see [reporting.md](references/reporting.md) for how to write bug reports (what a reproduction is) and performance claims (end-to-end numbers first).\n\n## Skills\n\nTask-specific guides live in `.ai/skills/` and are loaded on demand by AI agents. Available skills include:\n\n- [model-integration](skills/model-integration/SKILL.md) (adding/converting pipelines)\n- [custom-blocks](skills/custom-blocks/SKILL.md) (packaging a `ModularPipelineBlocks` subclass for the Hub)\n- [diffusers-cli](skills/diffusers-cli/SKILL.md) (running pipelines, inspecting schemas, and using the Diffusers CLI)\n- [self-review](skills/self-review/SKILL.md) (pre-PR self-review against the project rules)\n\n## Self-review before a PR\n\nBefore opening a PR, run self-review against [review-rules.md](references/review-rules.md). The [self-review skill](skills/self-review/SKILL.md) runs this as the same pass the `@claude` CI reviewer uses. Share the final report on the PR (description or comment) — see the skill for details.\n","isInternal":false,"tokens":959,"sizeBytes":4103},{"name":"AGENTS.md","path":"AGENTS.md","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/AGENTS.md","title":"AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":".ai/AGENTS.md","isInternal":false,"tokens":5,"sizeBytes":13},{"name":"CLAUDE.md","path":"CLAUDE.md","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/CLAUDE.md","title":"Claude Agent Guidelines & System Prompt","category":"claude-rule","format":"markdown","content":".ai/AGENTS.md","isInternal":false,"tokens":5,"sizeBytes":13},{"name":"SKILL.md","path":".ai/skills/custom-blocks/SKILL.md","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/skills/custom-blocks/SKILL.md","title":"custom-blocks","category":"anthropic-skill","format":"markdown","content":"---\nname: custom-blocks\ndescription: >\n Use when the user has written (or wants to write) a `ModularPipelineBlocks`\n subclass in a local Python file and needs to package it into a Hub-uploadable\n directory. Covers the workflow from a single `block.py` file to a published\n custom-block repo that consumers can load via\n `ModularPipeline.from_pretrained(, trust_remote_code=True)`.\n---\n\n## What this skill is for\n\nA `ModularPipelineBlocks` subclass is a unit of pipeline logic — input/output spec plus a `__call__` — that\nslots into diffusers' modular pipeline composition. Once you have one defined locally, you almost always want to\npublish it as a small Hub repo so others can `from_pretrained` it. `diffusers-cli custom_blocks` automates the\npackaging step: it parses your Python file, instantiates the chosen block class, and writes a\n`save_pretrained`-style directory in your cwd that's ready to push to the Hub.\n\nUse this skill when:\n\n- The user is writing a custom modular block and asks \"how do I publish this?\" or \"package this for the Hub\".\n- The user has a `block.py` (or similar) file with one or more `ModularPipelineBlocks` subclasses.\n- You're scaffolding a new modular pipeline repo and need the on-disk layout that `ModularPipelineBlocks.from_pretrained`\n expects.\n\nDon't use this skill for: running an existing modular pipeline (`diffusers-cli run`), introspecting one\n(`diffusers-cli schema`), or writing the block class itself — this skill packages an *already-written* block.\n\n## The end-to-end workflow\n\n```\n[you: write block.py] → diffusers-cli custom_blocks → [packaged dir in cwd]\n ↓\n hf upload .\n ↓\n consumers: ModularPipeline.from_pretrained(, trust_remote_code=True)\n diffusers-cli schema --model --trust-remote-code\n diffusers-cli run --model --trust-remote-code ...\n```\n\nThe skill covers the middle box. The bookends (writing the block and uploading) are out of scope.\n\n## Command surface\n\n```bash\ndiffusers-cli custom_blocks [--block_module_name ] [--block_class_name ]\n```\n\n### Flags\n\n- `--block_module_name ` — Python file containing the block class. Defaults to `block.py` in the cwd.\n- `--block_class_name ` — Which class in the file to package. Optional: if omitted, the CLI parses the\n file with `ast`, finds every class that inherits from `ModularPipelineBlocks`, and uses the first one (with\n an info log naming the others). Specify explicitly when the file defines more than one block and you want a\n specific one.\n\n### What it does\n\n1. **AST scan**: parses `` without executing it, walks top-level `ClassDef` nodes, and collects every\n class whose `bases` include `ModularPipelineBlocks` or one of its composite subclasses\n (`SequentialPipelineBlocks`, `AutoPipelineBlocks`, `ConditionalPipelineBlocks`, `LoopSequentialPipelineBlocks`).\n2. **Pick a class**: uses `--block_class_name` if given, else the first found. Errors with the list of available\n classes if your name doesn't match.\n3. **Load and save**: imports the file via `importlib.util.spec_from_file_location` (this does execute the\n module — make sure your block.py is something you trust to run), instantiates the chosen class with no\n constructor args, and calls `.save_pretrained(os.getcwd())`.\n\nThe result is a Hub-uploadable directory laid out the way `ModularPipelineBlocks.from_pretrained` expects:\nyour block source, an `auto_map` in the config so consumers know to load it with `trust_remote_code=True`,\nand any artifacts `save_pretrained` writes for that block class.\n\n## End-to-end example\n\nGiven a `block.py` like:\n\n```python\nfrom diffusers.modular_pipelines import ModularPipelineBlocks, InputParam, OutputParam\n\nclass MyDenoiseBlock(ModularPipelineBlocks):\n model_name = \"my-denoise\"\n\n @property\n def inputs(self):\n return [\n InputParam(\"latents\", type_hint=\"torch.Tensor\", required=True, description=\"Noisy latents.\"),\n InputParam(\"guidance_scale\", type_hint=\"float\", default=7.5),\n ]\n\n @property\n def intermediate_outputs(self):\n return [OutputParam(\"latents\", type_hint=\"torch.Tensor\")]\n\n def __call__(self, components, state):\n # ... denoising logic ...\n return components, state\n```\n\nPackage it:\n\n```bash\ndiffusers-cli custom_blocks --block_module_name block.py\n```\n\nOutput in cwd:\n\n```\n./\n├── block.py\n├── modular_config.json # contains auto_map → MyDenoiseBlock\n└── (any state files MyDenoiseBlock.save_pretrained writes)\n```\n\nUpload to the Hub:\n\n```bash\nhf upload my-user/my-denoise-block .\n```\n\nConsumers can now use it:\n\n```python\nfrom diffusers import ModularPipeline\npipe = ModularPipeline.from_pretrained(\"my-user/my-denoise-block\", trust_remote_code=True)\n```\n\nOr via CLI:\n\n```bash\ndiffusers-cli schema --model my-user/my-denoise-block --trust-remote-code\ndiffusers-cli run --model my-user/my-denoise-block --trust-remote-code \\\n --pipeline-kwargs '{\"latents\": \"...\", \"guidance_scale\": 7.5}'\n```\n\n## Common errors\n\n- **`Could not parse '': SyntaxError`** — the file isn't valid Python. Fix the syntax; the AST step runs\n before any execution.\n- **`block_class_name could not be retrieved. Available classes from : [ClassA, ClassB]`** — your\n `--block_class_name` doesn't match any `ModularPipelineBlocks` subclass found. Pick from the list shown.\n- **No classes found**: silent — the command will try to use the first entry in an empty list and raise\n `IndexError`. If you hit that, double-check your class actually inherits from `ModularPipelineBlocks` or one\n of the composite classes listed above (the AST scan looks for those literal class names; aliased imports like\n `from diffusers import ... as MPB` won't be picked up).\n- **Block requires constructor args**: the command calls `()` with no args. If your block needs\n `__init__` parameters, refactor to take them from `state`/`components` at `__call__` time instead, or\n hardcode defaults in `__init__`.\n\n## Verifying the install\n\nIf `diffusers-cli` isn't on PATH after `pip install -e .`, reinstall with\n`pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the binary is missing recent\nfeatures (e.g. `unrecognized arguments: --lora`), reinstall. See the\n[`diffusers-cli` skill](https://github.com/huggingface/diffusers/blob/main/.ai/skills/diffusers-cli/SKILL.md#verifying-the-cli-is-installed) for more.\n\n## Related\n\n- the [`diffusers-cli` docs](https://github.com/huggingface/diffusers/blob/main/docs/source/en/using-diffusers/cli.md) — once your block\n is uploaded, `schema` and `run` call it from the terminal without writing Python.\n- diffusers' [modular pipelines docs](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) — for writing the block\n class itself.\n","frontmatter":{"name":"custom-blocks","description":"Use when the user has written (or wants to write) a `ModularPipelineBlocks` subclass in a local Python file and needs to package it into a Hub-uploadable directory. Covers the workflow from a single `block.py` file to a published custom-block repo that consumers can load via `ModularPipeline.from_pretrained(, trust_remote_code=True)`.\n"},"isInternal":false,"tokens":1679,"sizeBytes":7137},{"name":"run.md","path":".ai/skills/diffusers-cli/run.md","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/skills/diffusers-cli/run.md","title":"Diffusers-cli Skill","category":"anthropic-skill","format":"markdown","content":"# `diffusers-cli run` — reference\n\nFull surface for `diffusers-cli run`. Use this file as the source of truth when constructing a `run`\ninvocation. The top-level [`SKILL.md`](SKILL.md) covers when to use the CLI; this file covers how.\n\n## The schema → run flow\n\nFor any model you haven't called before, run `schema` first to learn its input contract, then `run` with\nthe right `--pipeline-kwargs`:\n\n```bash\n# 1. Discover what kwargs the pipeline takes (no weight download)\ndiffusers-cli --format json schema --model black-forest-labs/FLUX.2-klein-9B\n\n# 2. Run it\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --dtype bf16\n```\n\n`schema --format json` emits a `{task, model, pipeline_class, inputs[]}` payload where each input is\n`{name, type_hint, default, required, description}`.\n\n## Standard vs modular detection\n\n`run` auto-detects which kind of pipeline it's calling:\n\n1. If `model_index.json` exists on the repo → `DiffusionPipeline.from_pretrained` path.\n2. Otherwise → `ModularPipeline.from_pretrained` path.\n\nYou don't need to tell it which. Modular repos must pass `--trust-remote-code` if they ship custom block code.\n\n## `--pipeline-kwargs` semantics\n\nA JSON object passed straight through to `pipeline(**kwargs)`. String values at known media-input keys are\nauto-loaded before the pipeline is called:\n\n- **Images** (`image`, `mask_image`, `control_image`, `ip_adapter_image`, `image_2`) → `PIL.Image.Image`\n via `diffusers.utils.load_image`. Accepts URLs or local paths.\n- **Videos** (`video`, `control_video`) → `list[PIL.Image.Image]` via `diffusers.utils.load_video`.\n- **Audio** (`initial_audio_waveforms`, `reference_audio`, `src_audio`) → `torch.Tensor` via `torchaudio.load`.\n For `initial_audio_waveforms`, the file's native sample rate is auto-written to\n `initial_audio_sampling_rate` if you didn't pass it explicitly.\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\", \"prompt\": \"make the fur grey\", \"strength\": 0.6}'\n```\n\nMedia resolution runs **before** the pipeline weights load, so a dead URL or missing file fails within\nseconds instead of after a multi-minute model download.\n\n**Batched inputs**: any media key (and `prompt`) accepts a JSON array — each string in the list is loaded\nindividually, and the pipeline processes the whole list in one forward pass on the GPU. Use this when you\nhave several inputs that share flavor/model/dtype and want them batched:\n\n```bash\ndiffusers-cli run --model black-forest-labs/FLUX.1-Kontext-dev \\\n --pipeline-kwargs '{\"prompt\": [\"make it grey\", \"make it pink\", \"make it blue\"],\n \"image\": [\"https://.../cat1.png\", \"https://.../cat2.png\", \"https://.../cat3.png\"]}'\n```\n\n**Shell-quoting gotcha**: the JSON must be on one line (or use `\\` to line-continue). A literal newline inside the\nsingle-quoted argument lands as a raw control char inside the string and breaks `json.loads`.\n\n## LoRA adapters (`--lora`)\n\nAttach one or more LoRAs after the pipeline loads via a JSON spec. `--lora` accepts a single object or a list.\n\nSingle adapter:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"a tiny grey cat\"}' \\\n --lora '{\"lora_id\": \"alvdansen/littletinies\", \"lora_scale\": 0.8}'\n```\n\nMultiple stacked adapters:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"a tiny grey cat\"}' \\\n --lora '[\n {\"lora_id\": \"alvdansen/littletinies\", \"lora_scale\": 0.5, \"adapter_name\": \"style\"},\n {\"lora_id\": \"author/detail-boost\", \"lora_scale\": 0.3}\n ]'\n```\n\nPer-entry fields: `lora_id` (required), `lora_scale` (optional, default `1.0`), `adapter_name` (optional; when\nomitted while stacking, auto-generated as `lora_0`, `lora_1`, …; single-entry defaults to `\"default\"`).\n\nEach entry calls `pipeline.load_lora_weights(, adapter_name=)`. After all adapters load, one\n`pipeline.set_adapters(names, adapter_weights=scales)` activates them together.\n\n## Optimization flags\n\n- `--dtype {auto, bf16, fp16, fp32, …}` — pipeline weight dtype. `bf16` is the right default for modern DiTs on\n A100/H100.\n- `--device-map ` — component placement, forwarded to `from_pretrained(device_map=...)`. Accepts a plain\n torch device (`cuda`, `cuda:0`, `cpu`, `mps`), the string `balanced` (auto-splits pipeline components across\n visible GPUs), or a JSON dict `{\"transformer\": \"cuda:0\", \"vae\": \"cuda:1\"}` for explicit per-component\n placement. Auto-detects if omitted (pinned to `cuda:$LOCAL_RANK` under torchrun). `balanced` and dict values\n are incompatible with `--cpu-offload`.\n- `--cpu-offload {model, group}` — `model` uses `enable_model_cpu_offload`, `group` uses\n `enable_group_offload(offload_type=\"leaf_level\", use_stream=True)`. Use `group` to fit a 9B+ model on a single\n A100. Onload target device comes from `--device-map` (must be a plain device string in this case).\n- `--attention-backend {default, flash_hub, flash_varlen_hub, flash_4_hub, sage_hub, sage_blackwell_hub}` — hub-hosted kernels,\n auto-downloaded on first use. Failures (kernel not available, CUDA arch mismatch, network) raise a clear\n `SystemExit` listing the alternatives instead of silently reverting to the default. Only supported on\n transformer-based pipelines; UNet pipelines get a `logger.warning` and the flag is ignored.\n- `--vae-tiling` / `--vae-slicing` — lower peak VAE decode VRAM.\n- `--compile [JSON]` — `torch.compile` every denoiser submodule. See [Compile](#compile) below.\n- `--context-parallel` — Ulysses-style context parallelism on a DiT. See [Context parallel](#context-parallel) below.\n\n## Output handling\n\n`run` detects the pipeline return type and saves accordingly:\n\n- `PIL.Image` / list of them → `.png` (zero-padded, e.g. `0000.png`)\n- Frame sequence (≥2 PILs or ndarrays) → `0000.mp4` (uses `--fps`, default 8)\n- Numpy audio array → `0000.wav` (uses `--sampling-rate`)\n- Anything else → JSON dump\n\n**Default output directory** is `~/.diffusers/cli/run/outputs/diffusers-run--/`.\nEach run gets its own subdirectory so consecutive invocations don't overwrite each other. The same run id\nis used for the local dir, the sandbox's `DIFFUSERS_CLI_RUN_ID` env var, and the sandbox input/output paths\nunder `--remote`, so a run is traceable end-to-end.\n\nOverride the destination with `--output ` (file or directory). Explicit `--output` bypasses the\n`diffusers-run-*` namespace — files land flat in the path you gave.\n\n### `--push-to`\n\nUpload outputs to an HF bucket. Objects land under `//`. The bucket is created\nif it doesn't exist. Behavior interacts with `--output` and `--remote`:\n\n| flags | download to local? | bucket write? |\n|---|---|---|\n| (neither) | ✅ `~/.diffusers/cli/run/outputs//` | — |\n| `--output /path` | ✅ `/path/` | — |\n| `--push-to my/bucket` | ❌ | ✅ (bucket only) |\n| `--push-to my/bucket --output /path` | ✅ | ✅ (both) |\n\nThe rule: explicit `--push-to` means \"the bucket is my destination\" — skip the local download unless the\nuser also explicitly asked for a local target via `--output`.\n\n## Remote execution (`--remote`)\n\nAdd `--remote` to run the same call inside a [Hugging Face Sandbox](https://huggingface.co/docs/huggingface_hub/en/guides/sandbox)\n— an isolated cloud VM (built on HF Jobs) the CLI drives over HTTP: it uploads inputs, runs the pipeline,\ndownloads outputs, then terminates the sandbox.\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --remote --flavor a100-large \\\n --dtype bf16 \\\n --cpu-offload group\n```\n\nWhat happens:\n\n1. Your HF token is picked up (from `--token` or your login) and forwarded into the sandbox as `HF_TOKEN`.\n2. `--pipeline-kwargs` are parsed locally so JSON errors fail fast (no wasted sandbox time).\n3. A dedicated sandbox is created on `--flavor` from `diffusers/diffusers-cli-cuda:latest` by default — a\n prebuilt image (`docker/diffusers-cli-cuda/` in this repo, rebuilt nightly) that already ships torch,\n CUDA, `diffusers`, and the rest of the CLI's deps. Any local file paths in `--pipeline-kwargs` are\n uploaded into the sandbox under `/tmp/diffusers-cli/inputs//` via native file transfer (no\n bucket), and the JSON paths are rewritten to point at them.\n4. No dependency install runs on the default image — that step is skipped unless you passed `--dependencies`\n (only the extras are installed then) or pointed `--image` somewhere else (the full set is installed).\n Output streams live to your terminal.\n5. The sandbox CLI writes outputs to `/tmp/diffusers-cli/outputs/`; the CLI downloads every artifact back into\n the local target (see [`--push-to`](#-push-to) for when the download is skipped).\n6. The sandbox is terminated (unless `--keep-alive`/`--sandbox-id`), and the wallclock `run_seconds` for the\n pipeline command is printed and included in the JSON payload.\n\nFlags:\n\n- `--flavor ` — sandbox hardware (e.g. `a10g-small`, `a100-large`, `4xa100-large`).\n- `--timeout ` — max wallclock for the run command inside the sandbox (e.g. `30m`, `2h`). Defaults to `10m`.\n- `--dependencies ` — extra pip deps (repeatable), installed on top of whatever the image ships.\n- `--namespace ` — create the sandbox under a different account.\n- `--push-to ` — see [`--push-to`](#-push-to) above. The upload runs inside the sandbox; an explicit\n value with no `--output` makes the bucket the sole destination and skips the local download.\n- `--image ` — override the sandbox image. Must ship torch + CUDA; the CLI then installs the small\n Python deps on top via `uv pip install --system` on every cold sandbox, which the default image avoids.\n Useful for pinning a specific torch or bundling extra system libs.\n- `--volume [:]` — mount an HF storage bucket into the sandbox as a read-write directory.\n Repeatable. Default mount is `/mnt/buckets/`. Reference mounted files from `--pipeline-kwargs`\n like any other local path — no upload happens, the container reads straight from the FUSE mount. Applied\n only on new sandbox creation — ignored when reconnecting via `--sandbox-id`. Enables batch workflows:\n loop over bucket contents on the host, one `--sandbox-id` call per input.\n- `--keep-alive` — don't terminate the sandbox after the run; its id is printed so a later run can reconnect.\n- `--sandbox-id ` — reconnect to a kept-alive sandbox instead of creating a new one. Implies `--keep-alive`.\n- `--idle-timeout ` — auto-shutdown after this much inactivity (default 10m). The billing backstop\n for kept-alive sandboxes. Only applied on new sandbox creation — ignored when reconnecting via `--sandbox-id`.\n\n### Reusing a warm sandbox (`--keep-alive` / `--sandbox-id`)\n\nBy default each `--remote` run is ephemeral: create → run → download → kill. The cold cost (dep install +\nmodel weight download + any compile) is paid every time. For iterative work, keep one sandbox warm and reuse\nit — deps, the HF weight cache, and the `torch.compile` cache all survive on its disk:\n\n```bash\n# First run: create + keep alive. Prints sandbox_id=.\ndiffusers-cli run -m black-forest-labs/FLUX.2-klein-9B --dtype bf16 \\\n --pipeline-kwargs '{\"prompt\": \"a cat\"}' --remote --flavor a100-large --keep-alive\n\n# Subsequent runs: reconnect — model already cached, only inference runs.\ndiffusers-cli run -m black-forest-labs/FLUX.2-klein-9B --dtype bf16 \\\n --pipeline-kwargs '{\"prompt\": \"a dog\"}' --remote --flavor a100-large --sandbox-id \n\n# Stop it when done (or let --idle-timeout reap it).\nhf sandbox kill \n```\n\nNotes on `--remote` argv forwarding: flags that control how the run is dispatched (`--flavor`, `--timeout`,\n`--namespace`, `--dependencies`, `--image`, `--keep-alive`, `--sandbox-id`, `--idle-timeout`, `--format`) are\nstripped before the argv is rebuilt for the sandbox CLI. Everything else — model, dtype, pipeline-kwargs,\noptimizations, output flags — is forwarded verbatim (`--output` is repointed at the sandbox output dir).\n\n## Compile\n\n`--compile` runs `torch.compile` over every `transformer*` / `unet*` submodule on the pipeline. Prefers\nregional compilation via `module.compile_repeated_blocks(**kwargs)` when the model exposes `_repeated_blocks`\n— this only compiles the repeated inner blocks (the bulk of the compute) rather than the whole module, so\nfirst-step latency is much lower. Falls back to full `torch.compile(module, **kwargs)` when no regional\nmetadata is declared.\n\n```bash\n# Bare — uses fullgraph=true\ndiffusers-cli run --model --dtype bf16 --compile --pipeline-kwargs '...'\n\n# With kwargs forwarded to torch.compile\ndiffusers-cli run --model --dtype bf16 \\\n --compile '{\"mode\": \"max-autotune\", \"fullgraph\": true}' \\\n --pipeline-kwargs '...'\n```\n\n**When it's worth it**: multi-step generation (~50+ denoising steps). You pay a one-time compilation cost on\nthe first step, then every subsequent step is faster.\n\n**Under `--remote`**: on an ephemeral sandbox the compile cache doesn't survive, so the compilation cost is\npaid on every run — `--compile` only breaks even on very long generations. On a `--keep-alive`/`--sandbox-id`\nsandbox the compile cache persists on disk, so the cost is paid once and reused: keep a warm sandbox and\n`--compile` pays off across runs.\n\n`--compile` is **currently not supported with `--context-parallel`** — CP shards attention across ranks while\nregional compile assumes a stable single-device graph. If both are set, the CLI logs a warning and skips the\ncompile step; CP still runs.\n\n## Context parallel\n\n`--context-parallel` enables Ulysses CP on a DiT-based pipeline. **Locally** the user must launch via torchrun:\n\n```bash\ntorchrun --nproc-per-node=2 -m diffusers.commands.diffusers_cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\"}' \\\n --dtype bf16 \\\n --context-parallel\n```\n\n**Remotely** the CLI handles the torchrun wrapping — just pass `--context-parallel` to a `--remote` invocation on\na multi-GPU flavor:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --remote --flavor 4xa100-large \\\n --dtype bf16 \\\n --context-parallel\n```\n\nInside the sandbox, CP swaps the entrypoint to `torchrun --nproc-per-node=gpu -m\ndiffusers.commands.diffusers_cli`, initializes a hybrid process group (`cpu:gloo,cuda:nccl` — NCCL for the\nattention all-to-all, Gloo for `ulysses_anything`'s per-rank size coordination), pins each rank to\n`cuda:{LOCAL_RANK}`, and gates output saving/printing to rank 0 only.\n\n**Memory note**: CP shards the sequence, **not the weights**. Every rank still holds the full transformer. Wins\nare wall-clock attention speedup and headroom for very long sequences, not \"fit a model that doesn't fit.\" For\nweight sharding you'd want TP or FSDP — not exposed in the CLI yet.\n\nCP is DiT-only. UNet pipelines raise a clear error directing you to a DiT pipeline (FLUX, SD3, HunyuanDiT,\nAuraFlow, …).\n\n## Output mode (`--format`)\n\n`--format` controls the shape of **stdout metadata** (which paths were written, timing, sandbox id, pushed\nbucket URLs) — **not** the media file format. Written images are always PNG, videos MP4, audio WAV; only the\nsummary printed alongside them changes shape.\n\nThe CLI auto-detects when running under an AI coding agent (Claude Code, Cursor, Aider, GH Copilot Agent — via\n`CLAUDECODE`, `CLAUDE_CODE`, `CURSOR_AI`, `AIDER_AI_CONTEXT`, `GH_COPILOT_AGENT`) and switches to **agent\nmode** automatically — TSV tables, `key=value` results, compact JSON dicts, no progress bars.\n\nOverride explicitly with `--format {auto, human, agent, json, quiet}` placed **before** the subcommand:\n\n```bash\ndiffusers-cli --format json run --model --pipeline-kwargs '...'\n```\n","isInternal":false,"tokens":4391,"sizeBytes":16666},{"name":"SKILL.md","path":".ai/skills/diffusers-cli/SKILL.md","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/skills/diffusers-cli/SKILL.md","title":"diffusers-cli","category":"anthropic-skill","format":"markdown","content":"---\nname: diffusers-cli\ndescription: >\n Use when the user wants to run a diffusers pipeline from a terminal (one-off\n generation, batch jobs, smoke-testing a new model), run on HF Sandbox\n hardware via `--remote`, introspect a pipeline's input schema before\n calling it, or attach a LoRA at inference time. Prefer this over writing\n ad-hoc Python scripts for generation tasks.\n---\n\n## Overview\n\n`diffusers-cli` is the shipped CLI in `src/diffusers/commands/`. Subcommands relevant to agentic use:\n\n| Command | Purpose |\n| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `run` | Run any `DiffusionPipeline` or `ModularPipeline`. Forwards `--pipeline-kwargs` verbatim, saves output by detecting its runtime type, optionally runs on HF Jobs via `--remote`. |\n| `schema` | Print the input schema for a pipeline repo (kwarg names, types, defaults, descriptions). **No weights downloaded** — only the small index file. |\n| `custom_blocks` | Package a local `ModularPipelineBlocks` subclass for the Hub. |\n| `env` | Print versions of diffusers + torch + transformers + accelerate + safetensors + CUDA + GPU info. Use when investigating environment issues, dtype/precision support, or building bug reports. |\n\n## When to read which file\n\nMost agentic work goes through `run`. Read the matching reference file before constructing a command:\n\n- **[`run.md`](run.md)** — full reference for `diffusers-cli run`. Covers `--pipeline-kwargs`\n semantics and the shell-quoting gotcha, LoRA via `--lora`, optimization flags (`--dtype`, `--cpu-offload`,\n `--attention-backend`, `--vae-tiling/slicing`), output handling and `--push-to` bucket uploads, the full\n `--remote` HF Jobs flow (image, container command, log streaming, timing payload, artifact download), and\n context parallel (`--context-parallel`) for both local-torchrun and `--remote` paths.\n\nThe other commands are small enough that `diffusers-cli --help` is the canonical reference:\n\n```bash\ndiffusers-cli schema --help\ndiffusers-cli custom_blocks --help\ndiffusers-cli env --help\n```\n\n## When NOT to use this skill\n\n- Multi-stage workflows where you need intermediate tensor manipulation between pipelines → write Python.\n- Training or fine-tuning → CLI only covers inference.\n- Anything requiring `quantization_config` or other low-level loader knobs not exposed by the CLI flags → write\n Python. (`device_map` is exposed as `--device-map`; see [run.md](run.md#optimization-flags).)\n\n## Verifying the CLI is installed\n\nThe console entry point is registered in `pyproject.toml` (`diffusers-cli =\n\"diffusers.commands.diffusers_cli:main\"`). If `diffusers-cli` is not on PATH after `pip install -e .`, reinstall\nwith `pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the installed binary is\nmissing recent features (e.g. you see `unrecognized arguments: --lora`), reinstall.\n\n## Output formats\n\n`--format {auto, human, agent, json}` (top-level flag, must appear before the subcommand):\n\n- **`human`** — plain-text indented output for terminals (default when not running under an agent harness). No ANSI color.\n- **`agent`** — TSV tables and `key=value` lines. Auto-selected when an agent env var is present\n (`CLAUDECODE`, `CLAUDE_CODE`, `CODEX_SANDBOX`, `CURSOR_AI`, `AIDER_AI_CONTEXT`, `GH_COPILOT_AGENT`,\n `AI_AGENT`). Token-cheap for LLM agents to read.\n- **`json`** — compact JSON. Use for programmatic parsing (scripts, services) where type fidelity and nested\n structures matter.\n\n`stdout` carries data; `stderr` carries hints/warnings/progress — parseable output is never polluted.\n\nRule of thumb: `--format json` for scripts that will `json.loads()` the output, otherwise leave it on\nauto-detect (`agent` for LLMs, `human` for terminals).\n","frontmatter":{"name":"diffusers-cli","description":"Use when the user wants to run a diffusers pipeline from a terminal (one-off generation, batch jobs, smoke-testing a new model), run on HF Sandbox hardware via `--remote`, introspect a pipeline's input schema before calling it, or attach a LoRA at inference time. Prefer this over writing ad-hoc Python scripts for generation tasks.\n"},"isInternal":false,"tokens":928,"sizeBytes":4327},{"name":"SKILL.md","path":".ai/skills/model-integration/SKILL.md","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/skills/model-integration/SKILL.md","title":"model-integration","category":"anthropic-skill","format":"markdown","content":"---\nname: model-integration\ndescription: >\n Use when adding a new model or pipeline to diffusers, setting up file\n structure for a new model, converting a pipeline to modular format, or\n converting weights for a new version of an already-supported model.\n---\n\n## Goal\n\nIntegrate a new model into diffusers end-to-end, to full numerical parity with the reference implementation — one workflow at a time.\n\n## Setup — gather before starting\n\nBefore writing any code, gather info in this order:\n\n1. **Reference repo** — ask for the github link. If they've already set it up locally, ask for the path. Otherwise, ask what setup steps are needed (install deps, download checkpoints, set env vars, etc.) and run through them before proceeding.\n2. **Inference script** — ask for a runnable end-to-end script for a basic workflow first (e.g. T2V). Then ask what other workflows they want to support (I2V, V2V, etc.) and agree on the full implementation order together.\n3. **Standard vs modular** — **default to modular.** [Modular Diffusers](references/modular.md) is the preferred implementation for new pipelines; the standard `DiffusionPipeline` is still supported but no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (modality baked into the checkpoint) or that are actively evolving.\n\nAsk step 3 as an `AskUserQuestion`, with modular marked as the recommended default.\n\nOnce you have everything, **confirm the plan** with the user before implementing — state exactly what you'll do, e.g. \"I'll integrate model X with pipeline Y based on your script, and verify the model matches the reference before considering it done.\"\n\nThen work through the **Integration checklist** below\n\n## Integration checklist\n\nA pipeline in Diffusers (be it standard or modular) will have multiple components. These components can be models, schedulers, processors, etc.\n\n- [ ] **Transformer model**\n - [ ] Implement the model with `from_pretrained` support (conventions: [models.md](references/models.md))\n - [ ] Convert weights (see **Weight / Checkpoint Conversion**)\n - [ ] Parity test against the reference (internal, not shipped — see **Model parity test**)\n - [ ] Register in the relevant `__init__.py` files (lazy imports)\n - [ ] Model-level tests (see **Testing**)\n- [ ] **VAE** (if applicable) — reuse an existing `AutoencoderKL*` if possible; if a new one is needed, follow the same sub-steps as the transformer\n- [ ] **Scheduler** — reuse an existing scheduler, or add a custom one\n- [ ] **Pipeline**\n - [ ] Implement the pipeline — see [modular.md](references/modular.md) for modular pipeline, or [pipelines.md](references/pipelines.md) for standard pipeline\n - [ ] Add a LoRA mixin if applicable\n - [ ] Register in the relevant `__init__.py` files (lazy imports)\n - [ ] Pipeline-level tests (see **Testing**)\n- [ ] **Docs** — see **File structure**\n- [ ] **Style** — `make style` and `make quality`\n\n## File structure\n\nA new model PR roughly lands these files (the contents of `pipelines//` and `modular_pipelines//` live in their guides):\n\n```\nsrc/diffusers/\n models/transformers/transformer_.py # the model (or models/autoencoders/, models/unets/)\n schedulers/scheduling_.py # only if a custom scheduler is needed\n loaders/lora_pipeline.py # LoRA mixin — add to the existing file\n pipelines// # standard pipeline — see pipelines.md\n modular_pipelines// # modular pipeline — see modular.md\ntests/\n models/transformers/test_models_transformer_.py\n pipelines//test_.py\ndocs/source/en/\n _toctree.yml # register the new pages in the docs index\n api/models/.md\n api/pipelines/.md\n```\n\n## Model integration specific rules\n\n**Match the reference's numerical logic.** Restructuring code to fit diffusers APIs (`ModelMixin`, `ConfigMixin`, blocks for modular, etc.) is expected, and required diffusers conventions (e.g. the attention pattern in [models.md](references/models.md)) take precedence. Beyond those, keep the actual computation as close to the reference as possible — don't reorder operations, change the math, or rename internals for aesthetics, even if it looks unclean. Small deviations make output mismatches very hard to track down.\n\n## Weight / Checkpoint Conversion\n\nConvert the original checkpoint into diffusers format with a standalone script under `scripts/` (e.g. `scripts/convert__to_diffusers.py`). The flow:\n\n1. Map the original state-dict keys to the diffusers module names (renames + any tensor surgery — see patterns below).\n2. Instantiate the diffusers model from its config and load the converted state dict.\n3. `save_pretrained(...)` to a local path, then load it back with `from_pretrained` to confirm it round-trips.\n\nAll weights load through the standard paths — `from_pretrained`, or `from_single_file` (add `FromSingleFileMixin` + a weight-mapping) for an original-format single checkpoint. No custom `from_pretrained`, no manual runtime loading. See the loading rule in [models.md](references/models.md).\n\nCommon conversion patterns to watch for model-level components:\n- Fused QKV weights that need splitting into separate Q, K, V\n- Scale/shift ordering differences (reference stores `[shift, scale]`, diffusers expects `[scale, shift]`)\n- Weight transpositions (linear stored as transposed conv, or vice versa)\n- Interleaved head dimensions that need reshaping\n- Bias terms absorbed into different layers\n\n## Testing\n\nTwo test layers must be added for any new pipeline: pipeline-level tests, and (if a new model is introduced) model-level tests. Conventions for both layers — file locations, tester mixins, dummy-component rules — live in [testing.md](references/testing.md); follow it when writing the tests.\n\n## Model parity test\n\nConfirm the diffusers implementation matches the reference. Test each component on **CPU/float32** with a strict tolerance (`max_diff < 1e-3`), comparing the **freshly converted** weights against the reference in a single script — both sides side by side, nothing saved to disk in between. See [pitfalls.md](references/pitfalls.md) for the common sources of numerical discrepancy.\n\nThis is an **internal verification tool for integration — it should not be shipped in the PR** (it imports the reference repo). The tests that ship with the PR are the model-level and pipeline-level tests in [testing.md](references/testing.md).\n\nThe example below is schematic (placeholder names). `ReferenceModel` is the component **imported from the original repo**, and `convert_my_component` is **the same conversion function you wrote for the conversion script for the component**. You should make sure both load the *same* checkpoint weights and run the *same* input, so any difference is a conversion or implementation bug — not a difference in inputs.\n\n```python\n@torch.inference_mode()\ndef test_my_component():\n # deterministic input — use the same shape & dtype the real model receives at this stage\n gen = torch.Generator().manual_seed(42)\n x = torch.randn(1, 16, 32, 32, generator=gen, dtype=torch.float32) # adjust to the real input shape\n\n original_state_dict = load_original_weights(...) # the original checkpoint — both sides load these same weights\n\n # reference: the original repo's implementation (load one model at a time to fit in CPU RAM)\n ref_model = ReferenceModel(config) # ReferenceModel: imported from the original repo\n ref_model.load_state_dict(original_state_dict, strict=True)\n ref_model = ref_model.float().eval()\n ref_out = ref_model(x).clone() # clone before freeing the model\n del ref_model\n\n # diffusers: convert those same weights with your conversion-script function, then run\n diff_model = convert_my_component(original_state_dict) # convert_my_component: the fn from convert__to_diffusers.py\n diff_model = diff_model.float().eval()\n diff_out = diff_model(x)\n\n max_diff = (ref_out - diff_out).abs().max().item()\n assert max_diff < 1e-3, f\"FAIL: max_diff={max_diff:.2e}\"\n```\n","frontmatter":{"name":"model-integration","description":"Use when adding a new model or pipeline to diffusers, setting up file structure for a new model, converting a pipeline to modular format, or converting weights for a new version of an already-supported model.\n"},"isInternal":false,"tokens":1860,"sizeBytes":8268},{"name":"SKILL.md","path":".ai/skills/self-review/SKILL.md","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/skills/self-review/SKILL.md","title":"self-review","category":"anthropic-skill","format":"markdown","content":"---\nname: self-review\ndescription: >\n Use before opening a PR, or whenever asked to self-review a diffusers\n contribution. Applies the same rubric as the `@claude` CI (checks the diff\n against references/review-rules.md, traces call paths for dead code). Reports findings grouped by\n severity, flagging what to fix before submitting (blocking issues + dead code)\n vs what to leave for the actual review. Report-only — does not edit files.\n---\n\n# Self-review\n\nRuns the same rubric as the `@claude` CI reviewer, so you catch issues before a\nmaintainer does — but over your **whole** PR diff. (The CI scopes itself to\n`src/diffusers/`, `tests/`, and `.ai/`; for your own PR, also review your docs\nand scripts.) You're already on the branch with the conventions loaded, so: get\nthe\ndiff → review it against the rubric → report → iterate with the contributor\nuntil it's ready, then remind them to share the final notes on the PR.\n\n## 1. Get the diff\n\n```bash\ngit diff main...HEAD # use your target branch if not main\n```\n\nIf the branch trails `main` and the diff looks polluted with unrelated merged\nfiles, scope to your own commits: `git log main..HEAD --oneline`, then\n`git show `.\n\n## 2. Read the rubric\n\n`references/review-rules.md` is the canonical rubric (the CI pins it from `main`) — read\nit and review against it; don't rely on a remembered copy. For the areas you\ntouched, also read `references/code_style.md`, `references/models.md`, `references/pipelines.md`,\n`references/modular.md`, `references/testing.md`, or `references/pitfalls.md`.\n\n## 3. Report\n\n- **Blocking issues** — numbered. Each: title → explanation → `file.py:line` →\n impact. Cite the rule, e.g. *Per `references/models.md`: \"…only keep the inference path.\"*\n- **Non-blocking issues** — same format, lower severity.\n- **Dead code (advisory)** — a table: `path:line` · Likely-dead / Used · reason.\n- **Summary** — short synthesis and a verdict (**READY** / **NEEDS CHANGES**),\n spelling out:\n - **Fix before submitting** — all blocking issues, and remove the flagged dead code.\n - **Leave for the actual review** — non-blocking issues that aren't obviously\n correct; raise these with the reviewer rather than guessing at them now.\n\nReport only — do not edit files. Be concrete, cite the rule, review the whole\ndiff, and don't invent issues or flag pure style.\n\n## 4. Iterate until ready, then share\n\nExpect several rounds: the contributor addresses findings, you review again.\nKeep working with them to fix as much as possible until the verdict is\n**READY** — the **Leave for the actual review** items are the only ones that\nshould reach the reviewer unresolved. End the final round's report by\nreminding the contributor to share it on the PR (description or a comment) —\nit saves the reviewer a few rounds of back-and-forth. Never commit the notes as\npart of the diff.\n","frontmatter":{"name":"self-review","description":"Use before opening a PR, or whenever asked to self-review a diffusers contribution. Applies the same rubric as the `@claude` CI (checks the diff against references/review-rules.md, traces call paths for dead code). Reports findings grouped by severity, flagging what to fix before submitting (blocking issues + dead code) vs what to leave for the actual review. Report-only — does not edit files.\n"},"isInternal":false,"tokens":712,"sizeBytes":2911},{"name":"marketplace.json","path":".claude-plugin/marketplace.json","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.claude-plugin/marketplace.json","title":"diffusers-skills","category":"marketplace","format":"json","content":"{\n \"name\": \"diffusers-skills\",\n \"description\": \"Agent plugins published from the diffusers repository\",\n \"owner\": {\n \"name\": \"The Hugging Face team\"\n },\n \"plugins\": [\n {\n \"name\": \"diffusers\",\n \"source\": \"./.ai\",\n \"description\": \"Conventions and task skills for contributing to diffusers\"\n }\n ]\n}\n","frontmatter":{"name":"diffusers-skills","description":"Agent plugins published from the diffusers repository","owner":{"name":"The Hugging Face team"},"plugins":[{"name":"diffusers","source":"./.ai","description":"Conventions and task skills for contributing to diffusers"}]},"isInternal":false,"tokens":87,"sizeBytes":326},{"name":"plugin.json","path":".ai/.claude-plugin/plugin.json","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/.claude-plugin/plugin.json","title":"diffusers","category":"plugin-manifest","format":"json","content":"{\n \"name\": \"diffusers\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"author\": {\n \"name\": \"The Hugging Face team\"\n },\n \"homepage\": \"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents\",\n \"repository\": \"https://github.com/huggingface/diffusers\",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"diffusers\",\n \"diffusion-models\",\n \"contributing\"\n ]\n}\n","frontmatter":{"name":"diffusers","description":"Conventions and task skills for contributing to diffusers","author":{"name":"The Hugging Face team"},"homepage":"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents","repository":"https://github.com/huggingface/diffusers","license":"Apache-2.0","keywords":["diffusers","diffusion-models","contributing"]},"isInternal":false,"tokens":125,"sizeBytes":436},{"name":"plugin.json","path":".ai/.codex-plugin/plugin.json","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/.codex-plugin/plugin.json","title":"diffusers","category":"plugin-manifest","format":"json","content":"{\n \"name\": \"diffusers\",\n \"version\": \"0.1.0\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"skills\": \"./skills/\"\n}\n","frontmatter":{"name":"diffusers","version":"0.1.0","description":"Conventions and task skills for contributing to diffusers","skills":"./skills/"},"isInternal":false,"tokens":44,"sizeBytes":151},{"name":"plugin.json","path":".ai/plugin.json","rawUrl":"https://raw.githubusercontent.com/huggingface/diffusers/HEAD/.ai/plugin.json","title":"diffusers","category":"plugin-manifest","format":"json","content":"{\n \"$schema\": \"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json\",\n \"name\": \"diffusers\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"author\": {\n \"name\": \"The Hugging Face team\"\n },\n \"homepage\": \"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents\",\n \"repository\": \"https://github.com/huggingface/diffusers\",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"diffusers\",\n \"diffusion-models\",\n \"contributing\"\n ]\n}\n","frontmatter":{"$schema":"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json","name":"diffusers","description":"Conventions and task skills for contributing to diffusers","author":{"name":"The Hugging Face team"},"homepage":"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents","repository":"https://github.com/huggingface/diffusers","license":"Apache-2.0","keywords":["diffusers","diffusion-models","contributing"]},"isInternal":false,"tokens":147,"sizeBytes":511}],"systemPromptSnippet":"\n\n# Diffusers — Agent Guide\n\n## Setup\n\nWe recommend developing in a virtual environment managed by [uv](https://docs.astral.sh/uv/):\n\n```bash\nuv venv && source .venv/bin/activate\nuv pip install -e . # provides diffusers-cli\n```\n\nList the available skills, and what each one is for, with:\n\n```bash\ndiffusers-cli skills list\n```\n\nInstall them with:\n\n```bash\ndiffusers-cli skills add # install one, or --all for every skill\n # --claude / --codex / --cursor to pick one target\n```\n\n`diffusers-cli skills update` refreshes what you installed.\n\nSkills used to be installed through the `Makefile`, which symlinked the project-level `.claude/skills` and\n`.agents/skills` at `.ai/skills` in this repo. Remove those links if they exist — otherwise the install writes through\nthem into `.ai/` itself:\n\n```bash\nrm -f .claude/skills .agents/skills\n```\n\nAt the start of a session, confirm the setup and tell the user what you find:\n\n```bash\npython utils/check_ai.py # guides and skills are consistent\ndiffusers-cli skills list # which skills are installed\n```\n\nClaude Code and Codex can also install via plugins\n\n```bash\nclaude plugin marketplace add huggingface/diffusers\nclaude plugin install diffusers@diffusers-skills --scope project\n\ncodex plugin marketplace add huggingface/diffusers # then install from the Plugins Directory\n```\n\n\n## Human in the loop\n\nEverything a PR reviewer sees must come from a human, or be approved by one for the exact wording. That includes commit messages, code comments and docstrings, PR titles and descriptions, and any PR or issue comment, review, or reply. Ask whether the user wants you to draft that text or will write it themselves. Keep drafts short and easy to understand. When you hand one over, remind them to read it for real and check that it reads well for other humans.\n\n- **Don't commit unless the user approved the exact commit message, and never push or open a PR on\n your own.** The user decides when anything is published, each time.\n- **Don't post to GitHub directly** — no comments, reviews, or replies. Draft when asked and hand the text to the user.\n\n## Code formatting\n\n- `make style` and `make fix-copies` should be run before opening a PR\n\n## Reference guides\n\n- **Coding style** — see [code_style.md](references/code_style.md) for how code should read, and the `# Copied from` rules.\n- **Models** — see [models.md](references/models.md) for model conventions, attention pattern, implementation rules, dependencies, and gotchas. For adding or converting a model, use the [model-integration](skills/model-integration/SKILL.md) skill.\n- **Pipelines** — see [pipelines.md](references/pipelines.md) for pipeline conventions, patterns, and gotchas.\n- **Modular pipelines** — see [modular.md](references/modular.md) for modular pipeline conventions, patterns, and gotchas.\n- **Tests** — see [testing.md](references/testing.md) for test conventions: required test layers, tester mixins, and dummy-component rules.\n- **Reporting** — see [reporting.md](references/reporting.md) for how to write bug reports (what a reproduction is) and performance claims (end-to-end numbers first).\n\n## Skills\n\nTask-specific guides live in `.ai/skills/` and are loaded on demand by AI agents. Available skills include:\n\n- [model-integration](skills/model-integration/SKILL.md) (adding/converting pipelines)\n- [custom-blocks](skills/custom-blocks/SKILL.md) (packaging a `ModularPipelineBlocks` subclass for the Hub)\n- [diffusers-cli](skills/diffusers-cli/SKILL.md) (running pipelines, inspecting schemas, and using the Diffusers CLI)\n- [self-review](skills/self-review/SKILL.md) (pre-PR self-review against the project rules)\n\n## Self-review before a PR\n\nBefore opening a PR, run self-review against [review-rules.md](references/review-rules.md). The [self-review skill](skills/self-review/SKILL.md) runs this as the same pass the `@claude` CI reviewer uses. Share the final report on the PR (description or comment) — see the skill for details.\n\n\n.ai/AGENTS.md\n\n\n.ai/AGENTS.md\n\n\n---\nname: custom-blocks\ndescription: >\n Use when the user has written (or wants to write) a `ModularPipelineBlocks`\n subclass in a local Python file and needs to package it into a Hub-uploadable\n directory. Covers the workflow from a single `block.py` file to a published\n custom-block repo that consumers can load via\n `ModularPipeline.from_pretrained(, trust_remote_code=True)`.\n---\n\n## What this skill is for\n\nA `ModularPipelineBlocks` subclass is a unit of pipeline logic — input/output spec plus a `__call__` — that\nslots into diffusers' modular pipeline composition. Once you have one defined locally, you almost always want to\npublish it as a small Hub repo so others can `from_pretrained` it. `diffusers-cli custom_blocks` automates the\npackaging step: it parses your Python file, instantiates the chosen block class, and writes a\n`save_pretrained`-style directory in your cwd that's ready to push to the Hub.\n\nUse this skill when:\n\n- The user is writing a custom modular block and asks \"how do I publish this?\" or \"package this for the Hub\".\n- The user has a `block.py` (or similar) file with one or more `ModularPipelineBlocks` subclasses.\n- You're scaffolding a new modular pipeline repo and need the on-disk layout that `ModularPipelineBlocks.from_pretrained`\n expects.\n\nDon't use this skill for: running an existing modular pipeline (`diffusers-cli run`), introspecting one\n(`diffusers-cli schema`), or writing the block class itself — this skill packages an *already-written* block.\n\n## The end-to-end workflow\n\n```\n[you: write block.py] → diffusers-cli custom_blocks → [packaged dir in cwd]\n ↓\n hf upload .\n ↓\n consumers: ModularPipeline.from_pretrained(, trust_remote_code=True)\n diffusers-cli schema --model --trust-remote-code\n diffusers-cli run --model --trust-remote-code ...\n```\n\nThe skill covers the middle box. The bookends (writing the block and uploading) are out of scope.\n\n## Command surface\n\n```bash\ndiffusers-cli custom_blocks [--block_module_name ] [--block_class_name ]\n```\n\n### Flags\n\n- `--block_module_name ` — Python file containing the block class. Defaults to `block.py` in the cwd.\n- `--block_class_name ` — Which class in the file to package. Optional: if omitted, the CLI parses the\n file with `ast`, finds every class that inherits from `ModularPipelineBlocks`, and uses the first one (with\n an info log naming the others). Specify explicitly when the file defines more than one block and you want a\n specific one.\n\n### What it does\n\n1. **AST scan**: parses `` without executing it, walks top-level `ClassDef` nodes, and collects every\n class whose `bases` include `ModularPipelineBlocks` or one of its composite subclasses\n (`SequentialPipelineBlocks`, `AutoPipelineBlocks`, `ConditionalPipelineBlocks`, `LoopSequentialPipelineBlocks`).\n2. **Pick a class**: uses `--block_class_name` if given, else the first found. Errors with the list of available\n classes if your name doesn't match.\n3. **Load and save**: imports the file via `importlib.util.spec_from_file_location` (this does execute the\n module — make sure your block.py is something you trust to run), instantiates the chosen class with no\n constructor args, and calls `.save_pretrained(os.getcwd())`.\n\nThe result is a Hub-uploadable directory laid out the way `ModularPipelineBlocks.from_pretrained` expects:\nyour block source, an `auto_map` in the config so consumers know to load it with `trust_remote_code=True`,\nand any artifacts `save_pretrained` writes for that block class.\n\n## End-to-end example\n\nGiven a `block.py` like:\n\n```python\nfrom diffusers.modular_pipelines import ModularPipelineBlocks, InputParam, OutputParam\n\nclass MyDenoiseBlock(ModularPipelineBlocks):\n model_name = \"my-denoise\"\n\n @property\n def inputs(self):\n return [\n InputParam(\"latents\", type_hint=\"torch.Tensor\", required=True, description=\"Noisy latents.\"),\n InputParam(\"guidance_scale\", type_hint=\"float\", default=7.5),\n ]\n\n @property\n def intermediate_outputs(self):\n return [OutputParam(\"latents\", type_hint=\"torch.Tensor\")]\n\n def __call__(self, components, state):\n # ... denoising logic ...\n return components, state\n```\n\nPackage it:\n\n```bash\ndiffusers-cli custom_blocks --block_module_name block.py\n```\n\nOutput in cwd:\n\n```\n./\n├── block.py\n├── modular_config.json # contains auto_map → MyDenoiseBlock\n└── (any state files MyDenoiseBlock.save_pretrained writes)\n```\n\nUpload to the Hub:\n\n```bash\nhf upload my-user/my-denoise-block .\n```\n\nConsumers can now use it:\n\n```python\nfrom diffusers import ModularPipeline\npipe = ModularPipeline.from_pretrained(\"my-user/my-denoise-block\", trust_remote_code=True)\n```\n\nOr via CLI:\n\n```bash\ndiffusers-cli schema --model my-user/my-denoise-block --trust-remote-code\ndiffusers-cli run --model my-user/my-denoise-block --trust-remote-code \\\n --pipeline-kwargs '{\"latents\": \"...\", \"guidance_scale\": 7.5}'\n```\n\n## Common errors\n\n- **`Could not parse '': SyntaxError`** — the file isn't valid Python. Fix the syntax; the AST step runs\n before any execution.\n- **`block_class_name could not be retrieved. Available classes from : [ClassA, ClassB]`** — your\n `--block_class_name` doesn't match any `ModularPipelineBlocks` subclass found. Pick from the list shown.\n- **No classes found**: silent — the command will try to use the first entry in an empty list and raise\n `IndexError`. If you hit that, double-check your class actually inherits from `ModularPipelineBlocks` or one\n of the composite classes listed above (the AST scan looks for those literal class names; aliased imports like\n `from diffusers import ... as MPB` won't be picked up).\n- **Block requires constructor args**: the command calls `()` with no args. If your block needs\n `__init__` parameters, refactor to take them from `state`/`components` at `__call__` time instead, or\n hardcode defaults in `__init__`.\n\n## Verifying the install\n\nIf `diffusers-cli` isn't on PATH after `pip install -e .`, reinstall with\n`pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the binary is missing recent\nfeatures (e.g. `unrecognized arguments: --lora`), reinstall. See the\n[`diffusers-cli` skill](https://github.com/huggingface/diffusers/blob/main/.ai/skills/diffusers-cli/SKILL.md#verifying-the-cli-is-installed) for more.\n\n## Related\n\n- the [`diffusers-cli` docs](https://github.com/huggingface/diffusers/blob/main/docs/source/en/using-diffusers/cli.md) — once your block\n is uploaded, `schema` and `run` call it from the terminal without writing Python.\n- diffusers' [modular pipelines docs](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) — for writing the block\n class itself.\n\n\n# `diffusers-cli run` — reference\n\nFull surface for `diffusers-cli run`. Use this file as the source of truth when constructing a `run`\ninvocation. The top-level [`SKILL.md`](SKILL.md) covers when to use the CLI; this file covers how.\n\n## The schema → run flow\n\nFor any model you haven't called before, run `schema` first to learn its input contract, then `run` with\nthe right `--pipeline-kwargs`:\n\n```bash\n# 1. Discover what kwargs the pipeline takes (no weight download)\ndiffusers-cli --format json schema --model black-forest-labs/FLUX.2-klein-9B\n\n# 2. Run it\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --dtype bf16\n```\n\n`schema --format json` emits a `{task, model, pipeline_class, inputs[]}` payload where each input is\n`{name, type_hint, default, required, description}`.\n\n## Standard vs modular detection\n\n`run` auto-detects which kind of pipeline it's calling:\n\n1. If `model_index.json` exists on the repo → `DiffusionPipeline.from_pretrained` path.\n2. Otherwise → `ModularPipeline.from_pretrained` path.\n\nYou don't need to tell it which. Modular repos must pass `--trust-remote-code` if they ship custom block code.\n\n## `--pipeline-kwargs` semantics\n\nA JSON object passed straight through to `pipeline(**kwargs)`. String values at known media-input keys are\nauto-loaded before the pipeline is called:\n\n- **Images** (`image`, `mask_image`, `control_image`, `ip_adapter_image`, `image_2`) → `PIL.Image.Image`\n via `diffusers.utils.load_image`. Accepts URLs or local paths.\n- **Videos** (`video`, `control_video`) → `list[PIL.Image.Image]` via `diffusers.utils.load_video`.\n- **Audio** (`initial_audio_waveforms`, `reference_audio`, `src_audio`) → `torch.Tensor` via `torchaudio.load`.\n For `initial_audio_waveforms`, the file's native sample rate is auto-written to\n `initial_audio_sampling_rate` if you didn't pass it explicitly.\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\", \"prompt\": \"make the fur grey\", \"strength\": 0.6}'\n```\n\nMedia resolution runs **before** the pipeline weights load, so a dead URL or missing file fails within\nseconds instead of after a multi-minute model download.\n\n**Batched inputs**: any media key (and `prompt`) accepts a JSON array — each string in the list is loaded\nindividually, and the pipeline processes the whole list in one forward pass on the GPU. Use this when you\nhave several inputs that share flavor/model/dtype and want them batched:\n\n```bash\ndiffusers-cli run --model black-forest-labs/FLUX.1-Kontext-dev \\\n --pipeline-kwargs '{\"prompt\": [\"make it grey\", \"make it pink\", \"make it blue\"],\n \"image\": [\"https://.../cat1.png\", \"https://.../cat2.png\", \"https://.../cat3.png\"]}'\n```\n\n**Shell-quoting gotcha**: the JSON must be on one line (or use `\\` to line-continue). A literal newline inside the\nsingle-quoted argument lands as a raw control char inside the string and breaks `json.loads`.\n\n## LoRA adapters (`--lora`)\n\nAttach one or more LoRAs after the pipeline loads via a JSON spec. `--lora` accepts a single object or a list.\n\nSingle adapter:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"a tiny grey cat\"}' \\\n --lora '{\"lora_id\": \"alvdansen/littletinies\", \"lora_scale\": 0.8}'\n```\n\nMultiple stacked adapters:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"a tiny grey cat\"}' \\\n --lora '[\n {\"lora_id\": \"alvdansen/littletinies\", \"lora_scale\": 0.5, \"adapter_name\": \"style\"},\n {\"lora_id\": \"author/detail-boost\", \"lora_scale\": 0.3}\n ]'\n```\n\nPer-entry fields: `lora_id` (required), `lora_scale` (optional, default `1.0`), `adapter_name` (optional; when\nomitted while stacking, auto-generated as `lora_0`, `lora_1`, …; single-entry defaults to `\"default\"`).\n\nEach entry calls `pipeline.load_lora_weights(, adapter_name=)`. After all adapters load, one\n`pipeline.set_adapters(names, adapter_weights=scales)` activates them together.\n\n## Optimization flags\n\n- `--dtype {auto, bf16, fp16, fp32, …}` — pipeline weight dtype. `bf16` is the right default for modern DiTs on\n A100/H100.\n- `--device-map ` — component placement, forwarded to `from_pretrained(device_map=...)`. Accepts a plain\n torch device (`cuda`, `cuda:0`, `cpu`, `mps`), the string `balanced` (auto-splits pipeline components across\n visible GPUs), or a JSON dict `{\"transformer\": \"cuda:0\", \"vae\": \"cuda:1\"}` for explicit per-component\n placement. Auto-detects if omitted (pinned to `cuda:$LOCAL_RANK` under torchrun). `balanced` and dict values\n are incompatible with `--cpu-offload`.\n- `--cpu-offload {model, group}` — `model` uses `enable_model_cpu_offload`, `group` uses\n `enable_group_offload(offload_type=\"leaf_level\", use_stream=True)`. Use `group` to fit a 9B+ model on a single\n A100. Onload target device comes from `--device-map` (must be a plain device string in this case).\n- `--attention-backend {default, flash_hub, flash_varlen_hub, flash_4_hub, sage_hub, sage_blackwell_hub}` — hub-hosted kernels,\n auto-downloaded on first use. Failures (kernel not available, CUDA arch mismatch, network) raise a clear\n `SystemExit` listing the alternatives instead of silently reverting to the default. Only supported on\n transformer-based pipelines; UNet pipelines get a `logger.warning` and the flag is ignored.\n- `--vae-tiling` / `--vae-slicing` — lower peak VAE decode VRAM.\n- `--compile [JSON]` — `torch.compile` every denoiser submodule. See [Compile](#compile) below.\n- `--context-parallel` — Ulysses-style context parallelism on a DiT. See [Context parallel](#context-parallel) below.\n\n## Output handling\n\n`run` detects the pipeline return type and saves accordingly:\n\n- `PIL.Image` / list of them → `.png` (zero-padded, e.g. `0000.png`)\n- Frame sequence (≥2 PILs or ndarrays) → `0000.mp4` (uses `--fps`, default 8)\n- Numpy audio array → `0000.wav` (uses `--sampling-rate`)\n- Anything else → JSON dump\n\n**Default output directory** is `~/.diffusers/cli/run/outputs/diffusers-run--/`.\nEach run gets its own subdirectory so consecutive invocations don't overwrite each other. The same run id\nis used for the local dir, the sandbox's `DIFFUSERS_CLI_RUN_ID` env var, and the sandbox input/output paths\nunder `--remote`, so a run is traceable end-to-end.\n\nOverride the destination with `--output ` (file or directory). Explicit `--output` bypasses the\n`diffusers-run-*` namespace — files land flat in the path you gave.\n\n### `--push-to`\n\nUpload outputs to an HF bucket. Objects land under `//`. The bucket is created\nif it doesn't exist. Behavior interacts with `--output` and `--remote`:\n\n| flags | download to local? | bucket write? |\n|---|---|---|\n| (neither) | ✅ `~/.diffusers/cli/run/outputs//` | — |\n| `--output /path` | ✅ `/path/` | — |\n| `--push-to my/bucket` | ❌ | ✅ (bucket only) |\n| `--push-to my/bucket --output /path` | ✅ | ✅ (both) |\n\nThe rule: explicit `--push-to` means \"the bucket is my destination\" — skip the local download unless the\nuser also explicitly asked for a local target via `--output`.\n\n## Remote execution (`--remote`)\n\nAdd `--remote` to run the same call inside a [Hugging Face Sandbox](https://huggingface.co/docs/huggingface_hub/en/guides/sandbox)\n— an isolated cloud VM (built on HF Jobs) the CLI drives over HTTP: it uploads inputs, runs the pipeline,\ndownloads outputs, then terminates the sandbox.\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --remote --flavor a100-large \\\n --dtype bf16 \\\n --cpu-offload group\n```\n\nWhat happens:\n\n1. Your HF token is picked up (from `--token` or your login) and forwarded into the sandbox as `HF_TOKEN`.\n2. `--pipeline-kwargs` are parsed locally so JSON errors fail fast (no wasted sandbox time).\n3. A dedicated sandbox is created on `--flavor` from `diffusers/diffusers-cli-cuda:latest` by default — a\n prebuilt image (`docker/diffusers-cli-cuda/` in this repo, rebuilt nightly) that already ships torch,\n CUDA, `diffusers`, and the rest of the CLI's deps. Any local file paths in `--pipeline-kwargs` are\n uploaded into the sandbox under `/tmp/diffusers-cli/inputs//` via native file transfer (no\n bucket), and the JSON paths are rewritten to point at them.\n4. No dependency install runs on the default image — that step is skipped unless you passed `--dependencies`\n (only the extras are installed then) or pointed `--image` somewhere else (the full set is installed).\n Output streams live to your terminal.\n5. The sandbox CLI writes outputs to `/tmp/diffusers-cli/outputs/`; the CLI downloads every artifact back into\n the local target (see [`--push-to`](#-push-to) for when the download is skipped).\n6. The sandbox is terminated (unless `--keep-alive`/`--sandbox-id`), and the wallclock `run_seconds` for the\n pipeline command is printed and included in the JSON payload.\n\nFlags:\n\n- `--flavor ` — sandbox hardware (e.g. `a10g-small`, `a100-large`, `4xa100-large`).\n- `--timeout ` — max wallclock for the run command inside the sandbox (e.g. `30m`, `2h`). Defaults to `10m`.\n- `--dependencies ` — extra pip deps (repeatable), installed on top of whatever the image ships.\n- `--namespace ` — create the sandbox under a different account.\n- `--push-to ` — see [`--push-to`](#-push-to) above. The upload runs inside the sandbox; an explicit\n value with no `--output` makes the bucket the sole destination and skips the local download.\n- `--image ` — override the sandbox image. Must ship torch + CUDA; the CLI then installs the small\n Python deps on top via `uv pip install --system` on every cold sandbox, which the default image avoids.\n Useful for pinning a specific torch or bundling extra system libs.\n- `--volume [:]` — mount an HF storage bucket into the sandbox as a read-write directory.\n Repeatable. Default mount is `/mnt/buckets/`. Reference mounted files from `--pipeline-kwargs`\n like any other local path — no upload happens, the container reads straight from the FUSE mount. Applied\n only on new sandbox creation — ignored when reconnecting via `--sandbox-id`. Enables batch workflows:\n loop over bucket contents on the host, one `--sandbox-id` call per input.\n- `--keep-alive` — don't terminate the sandbox after the run; its id is printed so a later run can reconnect.\n- `--sandbox-id ` — reconnect to a kept-alive sandbox instead of creating a new one. Implies `--keep-alive`.\n- `--idle-timeout ` — auto-shutdown after this much inactivity (default 10m). The billing backstop\n for kept-alive sandboxes. Only applied on new sandbox creation — ignored when reconnecting via `--sandbox-id`.\n\n### Reusing a warm sandbox (`--keep-alive` / `--sandbox-id`)\n\nBy default each `--remote` run is ephemeral: create → run → download → kill. The cold cost (dep install +\nmodel weight download + any compile) is paid every time. For iterative work, keep one sandbox warm and reuse\nit — deps, the HF weight cache, and the `torch.compile` cache all survive on its disk:\n\n```bash\n# First run: create + keep alive. Prints sandbox_id=.\ndiffusers-cli run -m black-forest-labs/FLUX.2-klein-9B --dtype bf16 \\\n --pipeline-kwargs '{\"prompt\": \"a cat\"}' --remote --flavor a100-large --keep-alive\n\n# Subsequent runs: reconnect — model already cached, only inference runs.\ndiffusers-cli run -m black-forest-labs/FLUX.2-klein-9B --dtype bf16 \\\n --pipeline-kwargs '{\"prompt\": \"a dog\"}' --remote --flavor a100-large --sandbox-id \n\n# Stop it when done (or let --idle-timeout reap it).\nhf sandbox kill \n```\n\nNotes on `--remote` argv forwarding: flags that control how the run is dispatched (`--flavor`, `--timeout`,\n`--namespace`, `--dependencies`, `--image`, `--keep-alive`, `--sandbox-id`, `--idle-timeout`, `--format`) are\nstripped before the argv is rebuilt for the sandbox CLI. Everything else — model, dtype, pipeline-kwargs,\noptimizations, output flags — is forwarded verbatim (`--output` is repointed at the sandbox output dir).\n\n## Compile\n\n`--compile` runs `torch.compile` over every `transformer*` / `unet*` submodule on the pipeline. Prefers\nregional compilation via `module.compile_repeated_blocks(**kwargs)` when the model exposes `_repeated_blocks`\n— this only compiles the repeated inner blocks (the bulk of the compute) rather than the whole module, so\nfirst-step latency is much lower. Falls back to full `torch.compile(module, **kwargs)` when no regional\nmetadata is declared.\n\n```bash\n# Bare — uses fullgraph=true\ndiffusers-cli run --model --dtype bf16 --compile --pipeline-kwargs '...'\n\n# With kwargs forwarded to torch.compile\ndiffusers-cli run --model --dtype bf16 \\\n --compile '{\"mode\": \"max-autotune\", \"fullgraph\": true}' \\\n --pipeline-kwargs '...'\n```\n\n**When it's worth it**: multi-step generation (~50+ denoising steps). You pay a one-time compilation cost on\nthe first step, then every subsequent step is faster.\n\n**Under `--remote`**: on an ephemeral sandbox the compile cache doesn't survive, so the compilation cost is\npaid on every run — `--compile` only breaks even on very long generations. On a `--keep-alive`/`--sandbox-id`\nsandbox the compile cache persists on disk, so the cost is paid once and reused: keep a warm sandbox and\n`--compile` pays off across runs.\n\n`--compile` is **currently not supported with `--context-parallel`** — CP shards attention across ranks while\nregional compile assumes a stable single-device graph. If both are set, the CLI logs a warning and skips the\ncompile step; CP still runs.\n\n## Context parallel\n\n`--context-parallel` enables Ulysses CP on a DiT-based pipeline. **Locally** the user must launch via torchrun:\n\n```bash\ntorchrun --nproc-per-node=2 -m diffusers.commands.diffusers_cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\"}' \\\n --dtype bf16 \\\n --context-parallel\n```\n\n**Remotely** the CLI handles the torchrun wrapping — just pass `--context-parallel` to a `--remote` invocation on\na multi-GPU flavor:\n\n```bash\ndiffusers-cli run \\\n --model black-forest-labs/FLUX.2-klein-9B \\\n --pipeline-kwargs '{\"prompt\": \"Make the cats fur grey\", \"image\": \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\"}' \\\n --remote --flavor 4xa100-large \\\n --dtype bf16 \\\n --context-parallel\n```\n\nInside the sandbox, CP swaps the entrypoint to `torchrun --nproc-per-node=gpu -m\ndiffusers.commands.diffusers_cli`, initializes a hybrid process group (`cpu:gloo,cuda:nccl` — NCCL for the\nattention all-to-all, Gloo for `ulysses_anything`'s per-rank size coordination), pins each rank to\n`cuda:{LOCAL_RANK}`, and gates output saving/printing to rank 0 only.\n\n**Memory note**: CP shards the sequence, **not the weights**. Every rank still holds the full transformer. Wins\nare wall-clock attention speedup and headroom for very long sequences, not \"fit a model that doesn't fit.\" For\nweight sharding you'd want TP or FSDP — not exposed in the CLI yet.\n\nCP is DiT-only. UNet pipelines raise a clear error directing you to a DiT pipeline (FLUX, SD3, HunyuanDiT,\nAuraFlow, …).\n\n## Output mode (`--format`)\n\n`--format` controls the shape of **stdout metadata** (which paths were written, timing, sandbox id, pushed\nbucket URLs) — **not** the media file format. Written images are always PNG, videos MP4, audio WAV; only the\nsummary printed alongside them changes shape.\n\nThe CLI auto-detects when running under an AI coding agent (Claude Code, Cursor, Aider, GH Copilot Agent — via\n`CLAUDECODE`, `CLAUDE_CODE`, `CURSOR_AI`, `AIDER_AI_CONTEXT`, `GH_COPILOT_AGENT`) and switches to **agent\nmode** automatically — TSV tables, `key=value` results, compact JSON dicts, no progress bars.\n\nOverride explicitly with `--format {auto, human, agent, json, quiet}` placed **before** the subcommand:\n\n```bash\ndiffusers-cli --format json run --model --pipeline-kwargs '...'\n```\n\n\n---\nname: diffusers-cli\ndescription: >\n Use when the user wants to run a diffusers pipeline from a terminal (one-off\n generation, batch jobs, smoke-testing a new model), run on HF Sandbox\n hardware via `--remote`, introspect a pipeline's input schema before\n calling it, or attach a LoRA at inference time. Prefer this over writing\n ad-hoc Python scripts for generation tasks.\n---\n\n## Overview\n\n`diffusers-cli` is the shipped CLI in `src/diffusers/commands/`. Subcommands relevant to agentic use:\n\n| Command | Purpose |\n| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `run` | Run any `DiffusionPipeline` or `ModularPipeline`. Forwards `--pipeline-kwargs` verbatim, saves output by detecting its runtime type, optionally runs on HF Jobs via `--remote`. |\n| `schema` | Print the input schema for a pipeline repo (kwarg names, types, defaults, descriptions). **No weights downloaded** — only the small index file. |\n| `custom_blocks` | Package a local `ModularPipelineBlocks` subclass for the Hub. |\n| `env` | Print versions of diffusers + torch + transformers + accelerate + safetensors + CUDA + GPU info. Use when investigating environment issues, dtype/precision support, or building bug reports. |\n\n## When to read which file\n\nMost agentic work goes through `run`. Read the matching reference file before constructing a command:\n\n- **[`run.md`](run.md)** — full reference for `diffusers-cli run`. Covers `--pipeline-kwargs`\n semantics and the shell-quoting gotcha, LoRA via `--lora`, optimization flags (`--dtype`, `--cpu-offload`,\n `--attention-backend`, `--vae-tiling/slicing`), output handling and `--push-to` bucket uploads, the full\n `--remote` HF Jobs flow (image, container command, log streaming, timing payload, artifact download), and\n context parallel (`--context-parallel`) for both local-torchrun and `--remote` paths.\n\nThe other commands are small enough that `diffusers-cli --help` is the canonical reference:\n\n```bash\ndiffusers-cli schema --help\ndiffusers-cli custom_blocks --help\ndiffusers-cli env --help\n```\n\n## When NOT to use this skill\n\n- Multi-stage workflows where you need intermediate tensor manipulation between pipelines → write Python.\n- Training or fine-tuning → CLI only covers inference.\n- Anything requiring `quantization_config` or other low-level loader knobs not exposed by the CLI flags → write\n Python. (`device_map` is exposed as `--device-map`; see [run.md](run.md#optimization-flags).)\n\n## Verifying the CLI is installed\n\nThe console entry point is registered in `pyproject.toml` (`diffusers-cli =\n\"diffusers.commands.diffusers_cli:main\"`). If `diffusers-cli` is not on PATH after `pip install -e .`, reinstall\nwith `pip install -e . --force-reinstall --no-deps` and check `which diffusers-cli`. If the installed binary is\nmissing recent features (e.g. you see `unrecognized arguments: --lora`), reinstall.\n\n## Output formats\n\n`--format {auto, human, agent, json}` (top-level flag, must appear before the subcommand):\n\n- **`human`** — plain-text indented output for terminals (default when not running under an agent harness). No ANSI color.\n- **`agent`** — TSV tables and `key=value` lines. Auto-selected when an agent env var is present\n (`CLAUDECODE`, `CLAUDE_CODE`, `CODEX_SANDBOX`, `CURSOR_AI`, `AIDER_AI_CONTEXT`, `GH_COPILOT_AGENT`,\n `AI_AGENT`). Token-cheap for LLM agents to read.\n- **`json`** — compact JSON. Use for programmatic parsing (scripts, services) where type fidelity and nested\n structures matter.\n\n`stdout` carries data; `stderr` carries hints/warnings/progress — parseable output is never polluted.\n\nRule of thumb: `--format json` for scripts that will `json.loads()` the output, otherwise leave it on\nauto-detect (`agent` for LLMs, `human` for terminals).\n\n\n---\nname: model-integration\ndescription: >\n Use when adding a new model or pipeline to diffusers, setting up file\n structure for a new model, converting a pipeline to modular format, or\n converting weights for a new version of an already-supported model.\n---\n\n## Goal\n\nIntegrate a new model into diffusers end-to-end, to full numerical parity with the reference implementation — one workflow at a time.\n\n## Setup — gather before starting\n\nBefore writing any code, gather info in this order:\n\n1. **Reference repo** — ask for the github link. If they've already set it up locally, ask for the path. Otherwise, ask what setup steps are needed (install deps, download checkpoints, set env vars, etc.) and run through them before proceeding.\n2. **Inference script** — ask for a runnable end-to-end script for a basic workflow first (e.g. T2V). Then ask what other workflows they want to support (I2V, V2V, etc.) and agree on the full implementation order together.\n3. **Standard vs modular** — **default to modular.** [Modular Diffusers](references/modular.md) is the preferred implementation for new pipelines; the standard `DiffusionPipeline` is still supported but no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (modality baked into the checkpoint) or that are actively evolving.\n\nAsk step 3 as an `AskUserQuestion`, with modular marked as the recommended default.\n\nOnce you have everything, **confirm the plan** with the user before implementing — state exactly what you'll do, e.g. \"I'll integrate model X with pipeline Y based on your script, and verify the model matches the reference before considering it done.\"\n\nThen work through the **Integration checklist** below\n\n## Integration checklist\n\nA pipeline in Diffusers (be it standard or modular) will have multiple components. These components can be models, schedulers, processors, etc.\n\n- [ ] **Transformer model**\n - [ ] Implement the model with `from_pretrained` support (conventions: [models.md](references/models.md))\n - [ ] Convert weights (see **Weight / Checkpoint Conversion**)\n - [ ] Parity test against the reference (internal, not shipped — see **Model parity test**)\n - [ ] Register in the relevant `__init__.py` files (lazy imports)\n - [ ] Model-level tests (see **Testing**)\n- [ ] **VAE** (if applicable) — reuse an existing `AutoencoderKL*` if possible; if a new one is needed, follow the same sub-steps as the transformer\n- [ ] **Scheduler** — reuse an existing scheduler, or add a custom one\n- [ ] **Pipeline**\n - [ ] Implement the pipeline — see [modular.md](references/modular.md) for modular pipeline, or [pipelines.md](references/pipelines.md) for standard pipeline\n - [ ] Add a LoRA mixin if applicable\n - [ ] Register in the relevant `__init__.py` files (lazy imports)\n - [ ] Pipeline-level tests (see **Testing**)\n- [ ] **Docs** — see **File structure**\n- [ ] **Style** — `make style` and `make quality`\n\n## File structure\n\nA new model PR roughly lands these files (the contents of `pipelines//` and `modular_pipelines//` live in their guides):\n\n```\nsrc/diffusers/\n models/transformers/transformer_.py # the model (or models/autoencoders/, models/unets/)\n schedulers/scheduling_.py # only if a custom scheduler is needed\n loaders/lora_pipeline.py # LoRA mixin — add to the existing file\n pipelines// # standard pipeline — see pipelines.md\n modular_pipelines// # modular pipeline — see modular.md\ntests/\n models/transformers/test_models_transformer_.py\n pipelines//test_.py\ndocs/source/en/\n _toctree.yml # register the new pages in the docs index\n api/models/.md\n api/pipelines/.md\n```\n\n## Model integration specific rules\n\n**Match the reference's numerical logic.** Restructuring code to fit diffusers APIs (`ModelMixin`, `ConfigMixin`, blocks for modular, etc.) is expected, and required diffusers conventions (e.g. the attention pattern in [models.md](references/models.md)) take precedence. Beyond those, keep the actual computation as close to the reference as possible — don't reorder operations, change the math, or rename internals for aesthetics, even if it looks unclean. Small deviations make output mismatches very hard to track down.\n\n## Weight / Checkpoint Conversion\n\nConvert the original checkpoint into diffusers format with a standalone script under `scripts/` (e.g. `scripts/convert__to_diffusers.py`). The flow:\n\n1. Map the original state-dict keys to the diffusers module names (renames + any tensor surgery — see patterns below).\n2. Instantiate the diffusers model from its config and load the converted state dict.\n3. `save_pretrained(...)` to a local path, then load it back with `from_pretrained` to confirm it round-trips.\n\nAll weights load through the standard paths — `from_pretrained`, or `from_single_file` (add `FromSingleFileMixin` + a weight-mapping) for an original-format single checkpoint. No custom `from_pretrained`, no manual runtime loading. See the loading rule in [models.md](references/models.md).\n\nCommon conversion patterns to watch for model-level components:\n- Fused QKV weights that need splitting into separate Q, K, V\n- Scale/shift ordering differences (reference stores `[shift, scale]`, diffusers expects `[scale, shift]`)\n- Weight transpositions (linear stored as transposed conv, or vice versa)\n- Interleaved head dimensions that need reshaping\n- Bias terms absorbed into different layers\n\n## Testing\n\nTwo test layers must be added for any new pipeline: pipeline-level tests, and (if a new model is introduced) model-level tests. Conventions for both layers — file locations, tester mixins, dummy-component rules — live in [testing.md](references/testing.md); follow it when writing the tests.\n\n## Model parity test\n\nConfirm the diffusers implementation matches the reference. Test each component on **CPU/float32** with a strict tolerance (`max_diff < 1e-3`), comparing the **freshly converted** weights against the reference in a single script — both sides side by side, nothing saved to disk in between. See [pitfalls.md](references/pitfalls.md) for the common sources of numerical discrepancy.\n\nThis is an **internal verification tool for integration — it should not be shipped in the PR** (it imports the reference repo). The tests that ship with the PR are the model-level and pipeline-level tests in [testing.md](references/testing.md).\n\nThe example below is schematic (placeholder names). `ReferenceModel` is the component **imported from the original repo**, and `convert_my_component` is **the same conversion function you wrote for the conversion script for the component**. You should make sure both load the *same* checkpoint weights and run the *same* input, so any difference is a conversion or implementation bug — not a difference in inputs.\n\n```python\n@torch.inference_mode()\ndef test_my_component():\n # deterministic input — use the same shape & dtype the real model receives at this stage\n gen = torch.Generator().manual_seed(42)\n x = torch.randn(1, 16, 32, 32, generator=gen, dtype=torch.float32) # adjust to the real input shape\n\n original_state_dict = load_original_weights(...) # the original checkpoint — both sides load these same weights\n\n # reference: the original repo's implementation (load one model at a time to fit in CPU RAM)\n ref_model = ReferenceModel(config) # ReferenceModel: imported from the original repo\n ref_model.load_state_dict(original_state_dict, strict=True)\n ref_model = ref_model.float().eval()\n ref_out = ref_model(x).clone() # clone before freeing the model\n del ref_model\n\n # diffusers: convert those same weights with your conversion-script function, then run\n diff_model = convert_my_component(original_state_dict) # convert_my_component: the fn from convert__to_diffusers.py\n diff_model = diff_model.float().eval()\n diff_out = diff_model(x)\n\n max_diff = (ref_out - diff_out).abs().max().item()\n assert max_diff < 1e-3, f\"FAIL: max_diff={max_diff:.2e}\"\n```\n\n\n---\nname: self-review\ndescription: >\n Use before opening a PR, or whenever asked to self-review a diffusers\n contribution. Applies the same rubric as the `@claude` CI (checks the diff\n against references/review-rules.md, traces call paths for dead code). Reports findings grouped by\n severity, flagging what to fix before submitting (blocking issues + dead code)\n vs what to leave for the actual review. Report-only — does not edit files.\n---\n\n# Self-review\n\nRuns the same rubric as the `@claude` CI reviewer, so you catch issues before a\nmaintainer does — but over your **whole** PR diff. (The CI scopes itself to\n`src/diffusers/`, `tests/`, and `.ai/`; for your own PR, also review your docs\nand scripts.) You're already on the branch with the conventions loaded, so: get\nthe\ndiff → review it against the rubric → report → iterate with the contributor\nuntil it's ready, then remind them to share the final notes on the PR.\n\n## 1. Get the diff\n\n```bash\ngit diff main...HEAD # use your target branch if not main\n```\n\nIf the branch trails `main` and the diff looks polluted with unrelated merged\nfiles, scope to your own commits: `git log main..HEAD --oneline`, then\n`git show `.\n\n## 2. Read the rubric\n\n`references/review-rules.md` is the canonical rubric (the CI pins it from `main`) — read\nit and review against it; don't rely on a remembered copy. For the areas you\ntouched, also read `references/code_style.md`, `references/models.md`, `references/pipelines.md`,\n`references/modular.md`, `references/testing.md`, or `references/pitfalls.md`.\n\n## 3. Report\n\n- **Blocking issues** — numbered. Each: title → explanation → `file.py:line` →\n impact. Cite the rule, e.g. *Per `references/models.md`: \"…only keep the inference path.\"*\n- **Non-blocking issues** — same format, lower severity.\n- **Dead code (advisory)** — a table: `path:line` · Likely-dead / Used · reason.\n- **Summary** — short synthesis and a verdict (**READY** / **NEEDS CHANGES**),\n spelling out:\n - **Fix before submitting** — all blocking issues, and remove the flagged dead code.\n - **Leave for the actual review** — non-blocking issues that aren't obviously\n correct; raise these with the reviewer rather than guessing at them now.\n\nReport only — do not edit files. Be concrete, cite the rule, review the whole\ndiff, and don't invent issues or flag pure style.\n\n## 4. Iterate until ready, then share\n\nExpect several rounds: the contributor addresses findings, you review again.\nKeep working with them to fix as much as possible until the verdict is\n**READY** — the **Leave for the actual review** items are the only ones that\nshould reach the reviewer unresolved. End the final round's report by\nreminding the contributor to share it on the PR (description or a comment) —\nit saves the reviewer a few rounds of back-and-forth. Never commit the notes as\npart of the diff.\n\n\n{\n \"name\": \"diffusers-skills\",\n \"description\": \"Agent plugins published from the diffusers repository\",\n \"owner\": {\n \"name\": \"The Hugging Face team\"\n },\n \"plugins\": [\n {\n \"name\": \"diffusers\",\n \"source\": \"./.ai\",\n \"description\": \"Conventions and task skills for contributing to diffusers\"\n }\n ]\n}\n\n\n{\n \"name\": \"diffusers\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"author\": {\n \"name\": \"The Hugging Face team\"\n },\n \"homepage\": \"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents\",\n \"repository\": \"https://github.com/huggingface/diffusers\",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"diffusers\",\n \"diffusion-models\",\n \"contributing\"\n ]\n}\n\n\n{\n \"name\": \"diffusers\",\n \"version\": \"0.1.0\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"skills\": \"./skills/\"\n}\n\n\n{\n \"$schema\": \"https://agent-plugins.org/schemas/1.0.0/plugin.schema.json\",\n \"name\": \"diffusers\",\n \"description\": \"Conventions and task skills for contributing to diffusers\",\n \"author\": {\n \"name\": \"The Hugging Face team\"\n },\n \"homepage\": \"https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents\",\n \"repository\": \"https://github.com/huggingface/diffusers\",\n \"license\": \"Apache-2.0\",\n \"keywords\": [\n \"diffusers\",\n \"diffusion-models\",\n \"contributing\"\n ]\n}\n"}