{"owner":"huggingface","repo":"transformers","hasSkills":true,"totalSkillsCount":5,"totalTokensCount":5647,"categories":["root-instruction","copilot-instructions","claude-rule","anthropic-skill"],"hasMcp":false,"mcpConfig":null,"found":[".ai/AGENTS.md",".github/copilot-instructions.md","AGENTS.md","CLAUDE.md",".ai/skills/add-or-fix-type-checking/SKILL.md"],"skills":{".ai/AGENTS.md":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n",".github/copilot-instructions.md":"# copilot-instructions.md Guide for Hugging Face Transformers\n\nThis copilot-instructions.md file provides guidance for code agents working with this codebase.\n\n## Core Project Structure\n\n- `/src/transformers`: This contains the core source code for the library\n - `/models`: Code for individual models. Models inherit from base classes in the root `/src/transformers` directory.\n- `/tests`: This contains the core test classes for the library. These are usually inherited rather than directly run.\n - `/models`: Tests for individual models. Model tests inherit from common tests in the root `/tests` directory.\n- `/docs`: This contains the documentation for the library, including guides, tutorials, and API references.\n\n## Coding Conventions for Hugging Face Transformers\n\n- PRs should be as brief as possible. Bugfix PRs in particular can often be only one or two lines long, and do not need large comments, docstrings or new functions in this case. Aim to minimize the size of the diff.\n- When writing tests, they should be added to an existing file. The only exception is for PRs to add a new model, when a new test directory should be created for that model.\n- Code style is enforced in the CI. You can install the style tools with `pip install -e .[quality]`. You can then run `make fixup` to apply style and consistency fixes to your code.\n\n## Copying and inheritance\n\nMany models in the codebase have similar code, but it is not shared by inheritance because we want each model file to be self-contained.\nWe use two mechanisms to keep this code in sync:\n\n- \"Copied from\" syntax. Functions or entire classes can have a comment at the top like this: `# Copied from transformers.models.llama.modeling_llama.rotate_half` or `# Copied from transformers.models.t5.modeling_t5.T5LayerNorm with T5->MT5`\n These comments are actively checked by the style tools, and copies will automatically be updated when the base code is updated. If you need to update a copied function, you should\n either update the base function and use `make fixup` to propagate the change to all copies, or simply remove the `# Copied from` comment if that is inappropriate.\n- \"Modular\" files. These files briefly define models by composing them using inheritance from other models. They are not meant to be used directly. Instead, the style tools\n automatically generate a complete modeling file, like `modeling_bert.py`, from the modular file like `modular_bert.py`. If a model has a modular file, the modeling file\n should never be edited directly! Instead, changes should be made in the modular file, and then you should run `make fixup` to update the modeling file automatically.\n\nWhen adding new models, you should prefer `modular` style and inherit as many classes as possible from existing models.\n\n## Testing\n\nAfter making changes, you should usually run `make fixup` to ensure any copies and modular files are updated, and then test all affected models. This includes both\nthe model you made the changes in and any other models that were updated by `make fixup`. Tests can be run with `pytest tests/models/[name]/test_modeling_[name].py`\nIf your changes affect code in other classes like tokenizers or processors, you should run those tests instead, like `test_processing_[name].py` or `test_tokenization_[name].py`.\n\nIn order to run tests, you may need to install dependencies. You can do this with `pip install -e .[testing]`. You will probably also need to `pip install torch accelerate` if your environment does not already have them.\n","AGENTS.md":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n","CLAUDE.md":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n",".ai/skills/add-or-fix-type-checking/SKILL.md":"---\nname: add-or-fix-type-checking\ndescription: Fixes broken typing checks detected by ty, make typing, or make check-repo. Use when typing errors appear in local runs, CI, or PR logs.\n---\n\n# Add Or Fix Type Checking\n\n## Input\n\n- ``: module or directory to type-check (if known).\n- Optional `make typing` or CI output showing typing failures.\n\n## Workflow\n\n1. **Identify scope from the failing run**:\n - If you already have `make typing` or CI output, extract the failing file/module paths.\n - If not, run:\n ```bash\n make typing\n ```\n - Choose the narrowest target that covers the failures.\n\n2. **Run `ty check` for the target** to get a focused baseline:\n ```bash\n ty check --respect-ignore-files --exclude '**/*_pb*' \n ```\n\n3. **Triage errors by category** before fixing anything:\n - Wrong/missing type annotations on signatures\n - Attribute access on union types (for example `X | None`)\n - Functions returning broad unions (for example `str | list | BatchEncoding`)\n - Mixin/protocol self-type issues\n - Dynamic attributes on objects or modules\n - Third-party stub gaps (missing kwargs, missing `__version__`, etc.)\n\n4. **Apply fixes using this priority order** (simplest first):\n\n a. **Narrow unions with `isinstance()` / `if x is None` / `hasattr()`**.\n This is the primary tool for resolving union-type errors. `ty` narrows\n through all of these patterns, including the negative forms:\n ```python\n # Narrow X | None — use `if ...: raise`, never `assert`\n if x is None:\n raise ValueError(\"x must not be None\")\n x.method() # ty knows x is X here\n\n # Narrow str | UploadFile\n if isinstance(field, str):\n raise TypeError(\"Expected file upload, got string\")\n await field.read() # ty knows field is UploadFile here\n\n # Narrow broad union parameters early in a function body\n # (common for methods accepting e.g. list | dict | BatchEncoding)\n if isinstance(encoded_inputs, (list, tuple)):\n raise TypeError(\"Expected a mapping, got sequence\")\n encoded_inputs.keys() # ty sees only the dict/mapping types now\n ```\n\n b. **Use local variables to help ty track narrowing across closures**.\n When `self.x` is `X | None` and you need to pass it to nested functions\n or closures, `ty` cannot track that `self.x` stays non-None. Copy to a\n local variable and narrow the local:\n ```python\n manager = self.batching_manager\n if manager is None:\n raise RuntimeError(\"Manager not initialized\")\n # Use `manager` (not `self.batching_manager`) in nested functions\n ```\n\n c. **Split chained calls when the intermediate type is a broad union**.\n If `func().method()` fails because `func()` returns a union, split it:\n ```python\n # BAD: ty can't narrow through chained calls\n result = func(return_dict=True).to(device)[\"input_ids\"]\n\n # GOOD: split, narrow, then chain\n result = func(return_dict=True)\n if not hasattr(result, \"to\"):\n raise TypeError(\"Expected dict-like result\")\n inputs = result.to(device)[\"input_ids\"]\n ```\n\n d. **Fix incorrect type hints at the source**. If a parameter is typed `X | None`\n but can never be `None` when actually called, remove `None` from the hint.\n\n e. **Annotate untyped attributes**. Add type annotations to instance variables\n set in `__init__` or elsewhere (for example `self.foo: list[int] = []`).\n Declare class-level attributes that are set dynamically later\n (for example `_cache: Cache`, `_token_tensor: torch.Tensor | None`).\n\n f. **Use `@overload` for methods with input-dependent return types**.\n When a method returns different types based on the input type (e.g.\n `__getitem__` with str vs int keys), use `@overload` to declare each\n signature separately:\n ```python\n from typing import overload\n\n @overload\n def __getitem__(self, item: str) -> ValueType: ...\n @overload\n def __getitem__(self, item: int) -> EncodingType: ...\n @overload\n def __getitem__(self, item: slice) -> dict[str, ValueType]: ...\n\n def __getitem__(self, item: int | str | slice) -> ValueType | EncodingType | dict[str, ValueType]:\n ... # actual implementation\n ```\n This eliminates `cast()` calls at usage sites by giving the checker\n precise return types for each call pattern.\n\n g. **Make container classes generic to propagate value types**.\n When a class like `UserDict` holds values whose type changes after\n transformation (e.g. lists → tensors after `.to()`), make the class\n generic so methods can return narrowed types:\n ```python\n from typing import Generic, overload\n from typing_extensions import TypeVar\n\n _V = TypeVar(\"_V\", default=Any) # default=Any keeps existing code working\n\n class MyDict(UserDict, Generic[_V]):\n @overload\n def __getitem__(self, item: str) -> _V: ...\n # ...\n\n def to(self, device) -> MyDict[torch.Tensor]:\n # after .to(), values are tensors\n ...\n return self # type: ignore[return-value]\n ```\n The `default=Any` (from `typing_extensions`) means unparameterized usage\n like `MyDict()` stays `MyDict[Any]` — no existing code needs to change.\n Only methods that narrow the value type (like `.to()`) declare a specific\n return type. This eliminates `cast()` at all call sites.\n\n h. **Use `self: \"ProtocolType\"` for mixins**. When a mixin accesses attributes\n from its host class, define a Protocol in `src/transformers/_typing.py` and\n annotate `self` on methods that need it. Apply this consistently to all methods\n in the mixin. Import under `TYPE_CHECKING` to avoid circular imports.\n\n i. **Use `TypeGuard` functions for dynamic module attributes** (for example\n `torch.npu`, `torch.xpu`, `torch.compiler`). Instead of `getattr(torch, \"npu\")`\n or `hasattr(torch, \"npu\") and torch.npu.is_available()`, define a type guard\n function in `src/transformers/_typing.py`:\n ```python\n def has_torch_npu(mod: ModuleType) -> TypeGuard[Any]:\n return hasattr(mod, \"npu\") and mod.npu.is_available()\n ```\n Then use it as a narrowing check: `if has_torch_npu(torch): torch.npu.device_count()`.\n After the guard, `ty` treats the module as `Any`, allowing attribute access without\n `getattr()` or `cast()`. See existing guards in `_typing.py` for all device backends.\n\n **Key rules for type guards**:\n - Use `TypeGuard[Any]` (not a Protocol) — this is the simplest form that works\n with `ty` and avoids losing the original module's known attributes.\n - The guard function must be called directly in an `if` condition for narrowing\n to work. `ty` does NOT narrow through `and` conditions or `if not guard: return`.\n - Import guards with `from .._typing import has_torch_xxx` (not via module\n attribute `_typing.has_torch_xxx`) — `ty` only resolves `TypeGuard` from\n direct imports.\n\n j. **Use `getattr()` / `setattr()` for dynamic model/config attributes**.\n For runtime-injected fields (for example config/model flags), use\n `getattr(obj, \"field\", default)` for reads and `setattr(obj, \"field\", value)`\n for writes. Also use `getattr()` for third-party packages missing type stubs\n (for example `getattr(safetensors, \"__version__\", \"unknown\")`).\n Avoid `getattr(torch, \"npu\")` style — use type guards instead (see above).\n\n k. **Use `cast()` as a last resort before `# type: ignore`**.\n Use when you've structurally validated the type but the checker can't see it:\n pattern-matched AST nodes, known-typed dict values, or validated API responses.\n ```python\n # After structural validation confirms the type:\n stmt = cast(cst.Assign, node.body[0])\n annotations = cast(list[Annotation], [])\n ```\n Do not use `cast()` for module attribute narrowing — use type guards.\n Do not use `cast()` when `@overload` or generics can solve it at the source.\n\n l. **Use `# type: ignore` only for third-party stub defects**. This means\n cases where the third-party package's type stubs are wrong or incomplete\n and there is no way to narrow or cast around it. Examples:\n - A kwarg that exists at runtime but is missing from the stubs\n - A method that exists but isn't declared in the stubs\n Always add the specific error code: `# type: ignore[call-arg]`, not bare\n `# type: ignore`.\n\n5. **Things to never do**:\n - **Never use `assert` for type narrowing.** Asserts are stripped by `python -O`\n and must not be relied on for correctness. Use `if ...: raise` instead.\n - **Never use `# type: ignore` as a first resort.** Exhaust all approaches above first.\n - Do not use `getattr(torch, \"backend\")` to access dynamic device backends\n (`npu`, `xpu`, `hpu`, `musa`, `mlu`, `neuron`, `compiler`) — use type guards\n - Do not use `cast()` for module attribute narrowing — use type guards\n - Do not use `cast()` when `@overload` or generics can eliminate it at the source\n - Do not add helper methods or abstractions just to satisfy the type checker\n (especially for only 1-2 occurrences)\n - Do not pollute base classes with domain-specific fields; use Protocols\n - Do not add `if x is not None` guards for values guaranteed non-None\n by the call chain; fix the annotation instead\n - Do not use conditional inheritance patterns; annotate `self` instead\n\n6. **Organization**:\n - Keep shared Protocols and type aliases in `src/transformers/_typing.py`\n - Import type-only symbols under `if TYPE_CHECKING:` to avoid circular deps\n - Use `from __future__ import annotations` for PEP 604 syntax (`X | Y`)\n\n7. **Verify and close the PR loop**:\n - Re-run `ty check` on the same ``\n - Re-run `make typing` to confirm the type/model-rules step passes\n - If working toward merge readiness, run `make check-repo`\n - Ensure runtime behavior did not change and run relevant tests\n\n8. **Update CI coverage when adding new typed areas**:\n - Update `ty_check_dirs` in `Makefile` to include newly type-checked directories.\n"},"files":{".ai/AGENTS.md":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n",".github/copilot-instructions.md":"# copilot-instructions.md Guide for Hugging Face Transformers\n\nThis copilot-instructions.md file provides guidance for code agents working with this codebase.\n\n## Core Project Structure\n\n- `/src/transformers`: This contains the core source code for the library\n - `/models`: Code for individual models. Models inherit from base classes in the root `/src/transformers` directory.\n- `/tests`: This contains the core test classes for the library. These are usually inherited rather than directly run.\n - `/models`: Tests for individual models. Model tests inherit from common tests in the root `/tests` directory.\n- `/docs`: This contains the documentation for the library, including guides, tutorials, and API references.\n\n## Coding Conventions for Hugging Face Transformers\n\n- PRs should be as brief as possible. Bugfix PRs in particular can often be only one or two lines long, and do not need large comments, docstrings or new functions in this case. Aim to minimize the size of the diff.\n- When writing tests, they should be added to an existing file. The only exception is for PRs to add a new model, when a new test directory should be created for that model.\n- Code style is enforced in the CI. You can install the style tools with `pip install -e .[quality]`. You can then run `make fixup` to apply style and consistency fixes to your code.\n\n## Copying and inheritance\n\nMany models in the codebase have similar code, but it is not shared by inheritance because we want each model file to be self-contained.\nWe use two mechanisms to keep this code in sync:\n\n- \"Copied from\" syntax. Functions or entire classes can have a comment at the top like this: `# Copied from transformers.models.llama.modeling_llama.rotate_half` or `# Copied from transformers.models.t5.modeling_t5.T5LayerNorm with T5->MT5`\n These comments are actively checked by the style tools, and copies will automatically be updated when the base code is updated. If you need to update a copied function, you should\n either update the base function and use `make fixup` to propagate the change to all copies, or simply remove the `# Copied from` comment if that is inappropriate.\n- \"Modular\" files. These files briefly define models by composing them using inheritance from other models. They are not meant to be used directly. Instead, the style tools\n automatically generate a complete modeling file, like `modeling_bert.py`, from the modular file like `modular_bert.py`. If a model has a modular file, the modeling file\n should never be edited directly! Instead, changes should be made in the modular file, and then you should run `make fixup` to update the modeling file automatically.\n\nWhen adding new models, you should prefer `modular` style and inherit as many classes as possible from existing models.\n\n## Testing\n\nAfter making changes, you should usually run `make fixup` to ensure any copies and modular files are updated, and then test all affected models. This includes both\nthe model you made the changes in and any other models that were updated by `make fixup`. Tests can be run with `pytest tests/models/[name]/test_modeling_[name].py`\nIf your changes affect code in other classes like tokenizers or processors, you should run those tests instead, like `test_processing_[name].py` or `test_tokenization_[name].py`.\n\nIn order to run tests, you may need to install dependencies. You can do this with `pip install -e .[testing]`. You will probably also need to `pip install torch accelerate` if your environment does not already have them.\n","AGENTS.md":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n","CLAUDE.md":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n",".ai/skills/add-or-fix-type-checking/SKILL.md":"---\nname: add-or-fix-type-checking\ndescription: Fixes broken typing checks detected by ty, make typing, or make check-repo. Use when typing errors appear in local runs, CI, or PR logs.\n---\n\n# Add Or Fix Type Checking\n\n## Input\n\n- ``: module or directory to type-check (if known).\n- Optional `make typing` or CI output showing typing failures.\n\n## Workflow\n\n1. **Identify scope from the failing run**:\n - If you already have `make typing` or CI output, extract the failing file/module paths.\n - If not, run:\n ```bash\n make typing\n ```\n - Choose the narrowest target that covers the failures.\n\n2. **Run `ty check` for the target** to get a focused baseline:\n ```bash\n ty check --respect-ignore-files --exclude '**/*_pb*' \n ```\n\n3. **Triage errors by category** before fixing anything:\n - Wrong/missing type annotations on signatures\n - Attribute access on union types (for example `X | None`)\n - Functions returning broad unions (for example `str | list | BatchEncoding`)\n - Mixin/protocol self-type issues\n - Dynamic attributes on objects or modules\n - Third-party stub gaps (missing kwargs, missing `__version__`, etc.)\n\n4. **Apply fixes using this priority order** (simplest first):\n\n a. **Narrow unions with `isinstance()` / `if x is None` / `hasattr()`**.\n This is the primary tool for resolving union-type errors. `ty` narrows\n through all of these patterns, including the negative forms:\n ```python\n # Narrow X | None — use `if ...: raise`, never `assert`\n if x is None:\n raise ValueError(\"x must not be None\")\n x.method() # ty knows x is X here\n\n # Narrow str | UploadFile\n if isinstance(field, str):\n raise TypeError(\"Expected file upload, got string\")\n await field.read() # ty knows field is UploadFile here\n\n # Narrow broad union parameters early in a function body\n # (common for methods accepting e.g. list | dict | BatchEncoding)\n if isinstance(encoded_inputs, (list, tuple)):\n raise TypeError(\"Expected a mapping, got sequence\")\n encoded_inputs.keys() # ty sees only the dict/mapping types now\n ```\n\n b. **Use local variables to help ty track narrowing across closures**.\n When `self.x` is `X | None` and you need to pass it to nested functions\n or closures, `ty` cannot track that `self.x` stays non-None. Copy to a\n local variable and narrow the local:\n ```python\n manager = self.batching_manager\n if manager is None:\n raise RuntimeError(\"Manager not initialized\")\n # Use `manager` (not `self.batching_manager`) in nested functions\n ```\n\n c. **Split chained calls when the intermediate type is a broad union**.\n If `func().method()` fails because `func()` returns a union, split it:\n ```python\n # BAD: ty can't narrow through chained calls\n result = func(return_dict=True).to(device)[\"input_ids\"]\n\n # GOOD: split, narrow, then chain\n result = func(return_dict=True)\n if not hasattr(result, \"to\"):\n raise TypeError(\"Expected dict-like result\")\n inputs = result.to(device)[\"input_ids\"]\n ```\n\n d. **Fix incorrect type hints at the source**. If a parameter is typed `X | None`\n but can never be `None` when actually called, remove `None` from the hint.\n\n e. **Annotate untyped attributes**. Add type annotations to instance variables\n set in `__init__` or elsewhere (for example `self.foo: list[int] = []`).\n Declare class-level attributes that are set dynamically later\n (for example `_cache: Cache`, `_token_tensor: torch.Tensor | None`).\n\n f. **Use `@overload` for methods with input-dependent return types**.\n When a method returns different types based on the input type (e.g.\n `__getitem__` with str vs int keys), use `@overload` to declare each\n signature separately:\n ```python\n from typing import overload\n\n @overload\n def __getitem__(self, item: str) -> ValueType: ...\n @overload\n def __getitem__(self, item: int) -> EncodingType: ...\n @overload\n def __getitem__(self, item: slice) -> dict[str, ValueType]: ...\n\n def __getitem__(self, item: int | str | slice) -> ValueType | EncodingType | dict[str, ValueType]:\n ... # actual implementation\n ```\n This eliminates `cast()` calls at usage sites by giving the checker\n precise return types for each call pattern.\n\n g. **Make container classes generic to propagate value types**.\n When a class like `UserDict` holds values whose type changes after\n transformation (e.g. lists → tensors after `.to()`), make the class\n generic so methods can return narrowed types:\n ```python\n from typing import Generic, overload\n from typing_extensions import TypeVar\n\n _V = TypeVar(\"_V\", default=Any) # default=Any keeps existing code working\n\n class MyDict(UserDict, Generic[_V]):\n @overload\n def __getitem__(self, item: str) -> _V: ...\n # ...\n\n def to(self, device) -> MyDict[torch.Tensor]:\n # after .to(), values are tensors\n ...\n return self # type: ignore[return-value]\n ```\n The `default=Any` (from `typing_extensions`) means unparameterized usage\n like `MyDict()` stays `MyDict[Any]` — no existing code needs to change.\n Only methods that narrow the value type (like `.to()`) declare a specific\n return type. This eliminates `cast()` at all call sites.\n\n h. **Use `self: \"ProtocolType\"` for mixins**. When a mixin accesses attributes\n from its host class, define a Protocol in `src/transformers/_typing.py` and\n annotate `self` on methods that need it. Apply this consistently to all methods\n in the mixin. Import under `TYPE_CHECKING` to avoid circular imports.\n\n i. **Use `TypeGuard` functions for dynamic module attributes** (for example\n `torch.npu`, `torch.xpu`, `torch.compiler`). Instead of `getattr(torch, \"npu\")`\n or `hasattr(torch, \"npu\") and torch.npu.is_available()`, define a type guard\n function in `src/transformers/_typing.py`:\n ```python\n def has_torch_npu(mod: ModuleType) -> TypeGuard[Any]:\n return hasattr(mod, \"npu\") and mod.npu.is_available()\n ```\n Then use it as a narrowing check: `if has_torch_npu(torch): torch.npu.device_count()`.\n After the guard, `ty` treats the module as `Any`, allowing attribute access without\n `getattr()` or `cast()`. See existing guards in `_typing.py` for all device backends.\n\n **Key rules for type guards**:\n - Use `TypeGuard[Any]` (not a Protocol) — this is the simplest form that works\n with `ty` and avoids losing the original module's known attributes.\n - The guard function must be called directly in an `if` condition for narrowing\n to work. `ty` does NOT narrow through `and` conditions or `if not guard: return`.\n - Import guards with `from .._typing import has_torch_xxx` (not via module\n attribute `_typing.has_torch_xxx`) — `ty` only resolves `TypeGuard` from\n direct imports.\n\n j. **Use `getattr()` / `setattr()` for dynamic model/config attributes**.\n For runtime-injected fields (for example config/model flags), use\n `getattr(obj, \"field\", default)` for reads and `setattr(obj, \"field\", value)`\n for writes. Also use `getattr()` for third-party packages missing type stubs\n (for example `getattr(safetensors, \"__version__\", \"unknown\")`).\n Avoid `getattr(torch, \"npu\")` style — use type guards instead (see above).\n\n k. **Use `cast()` as a last resort before `# type: ignore`**.\n Use when you've structurally validated the type but the checker can't see it:\n pattern-matched AST nodes, known-typed dict values, or validated API responses.\n ```python\n # After structural validation confirms the type:\n stmt = cast(cst.Assign, node.body[0])\n annotations = cast(list[Annotation], [])\n ```\n Do not use `cast()` for module attribute narrowing — use type guards.\n Do not use `cast()` when `@overload` or generics can solve it at the source.\n\n l. **Use `# type: ignore` only for third-party stub defects**. This means\n cases where the third-party package's type stubs are wrong or incomplete\n and there is no way to narrow or cast around it. Examples:\n - A kwarg that exists at runtime but is missing from the stubs\n - A method that exists but isn't declared in the stubs\n Always add the specific error code: `# type: ignore[call-arg]`, not bare\n `# type: ignore`.\n\n5. **Things to never do**:\n - **Never use `assert` for type narrowing.** Asserts are stripped by `python -O`\n and must not be relied on for correctness. Use `if ...: raise` instead.\n - **Never use `# type: ignore` as a first resort.** Exhaust all approaches above first.\n - Do not use `getattr(torch, \"backend\")` to access dynamic device backends\n (`npu`, `xpu`, `hpu`, `musa`, `mlu`, `neuron`, `compiler`) — use type guards\n - Do not use `cast()` for module attribute narrowing — use type guards\n - Do not use `cast()` when `@overload` or generics can eliminate it at the source\n - Do not add helper methods or abstractions just to satisfy the type checker\n (especially for only 1-2 occurrences)\n - Do not pollute base classes with domain-specific fields; use Protocols\n - Do not add `if x is not None` guards for values guaranteed non-None\n by the call chain; fix the annotation instead\n - Do not use conditional inheritance patterns; annotate `self` instead\n\n6. **Organization**:\n - Keep shared Protocols and type aliases in `src/transformers/_typing.py`\n - Import type-only symbols under `if TYPE_CHECKING:` to avoid circular deps\n - Use `from __future__ import annotations` for PEP 604 syntax (`X | Y`)\n\n7. **Verify and close the PR loop**:\n - Re-run `ty check` on the same ``\n - Re-run `make typing` to confirm the type/model-rules step passes\n - If working toward merge readiness, run `make check-repo`\n - Ensure runtime behavior did not change and run relevant tests\n\n8. **Update CI coverage when adding new typed areas**:\n - Update `ty_check_dirs` in `Makefile` to include newly type-checked directories.\n"},"items":[{"name":"AGENTS.md","path":".ai/AGENTS.md","rawUrl":"https://raw.githubusercontent.com/huggingface/transformers/HEAD/.ai/AGENTS.md","title":".ai - AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n","isInternal":false,"tokens":792,"sizeBytes":3287},{"name":"AGENTS.md","path":"AGENTS.md","rawUrl":"https://raw.githubusercontent.com/huggingface/transformers/HEAD/AGENTS.md","title":"AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n","isInternal":false,"tokens":792,"sizeBytes":3287},{"name":"SKILL.md","path":".ai/skills/add-or-fix-type-checking/SKILL.md","rawUrl":"https://raw.githubusercontent.com/huggingface/transformers/HEAD/.ai/skills/add-or-fix-type-checking/SKILL.md","title":"add-or-fix-type-checking","category":"anthropic-skill","format":"markdown","content":"---\nname: add-or-fix-type-checking\ndescription: Fixes broken typing checks detected by ty, make typing, or make check-repo. Use when typing errors appear in local runs, CI, or PR logs.\n---\n\n# Add Or Fix Type Checking\n\n## Input\n\n- ``: module or directory to type-check (if known).\n- Optional `make typing` or CI output showing typing failures.\n\n## Workflow\n\n1. **Identify scope from the failing run**:\n - If you already have `make typing` or CI output, extract the failing file/module paths.\n - If not, run:\n ```bash\n make typing\n ```\n - Choose the narrowest target that covers the failures.\n\n2. **Run `ty check` for the target** to get a focused baseline:\n ```bash\n ty check --respect-ignore-files --exclude '**/*_pb*' \n ```\n\n3. **Triage errors by category** before fixing anything:\n - Wrong/missing type annotations on signatures\n - Attribute access on union types (for example `X | None`)\n - Functions returning broad unions (for example `str | list | BatchEncoding`)\n - Mixin/protocol self-type issues\n - Dynamic attributes on objects or modules\n - Third-party stub gaps (missing kwargs, missing `__version__`, etc.)\n\n4. **Apply fixes using this priority order** (simplest first):\n\n a. **Narrow unions with `isinstance()` / `if x is None` / `hasattr()`**.\n This is the primary tool for resolving union-type errors. `ty` narrows\n through all of these patterns, including the negative forms:\n ```python\n # Narrow X | None — use `if ...: raise`, never `assert`\n if x is None:\n raise ValueError(\"x must not be None\")\n x.method() # ty knows x is X here\n\n # Narrow str | UploadFile\n if isinstance(field, str):\n raise TypeError(\"Expected file upload, got string\")\n await field.read() # ty knows field is UploadFile here\n\n # Narrow broad union parameters early in a function body\n # (common for methods accepting e.g. list | dict | BatchEncoding)\n if isinstance(encoded_inputs, (list, tuple)):\n raise TypeError(\"Expected a mapping, got sequence\")\n encoded_inputs.keys() # ty sees only the dict/mapping types now\n ```\n\n b. **Use local variables to help ty track narrowing across closures**.\n When `self.x` is `X | None` and you need to pass it to nested functions\n or closures, `ty` cannot track that `self.x` stays non-None. Copy to a\n local variable and narrow the local:\n ```python\n manager = self.batching_manager\n if manager is None:\n raise RuntimeError(\"Manager not initialized\")\n # Use `manager` (not `self.batching_manager`) in nested functions\n ```\n\n c. **Split chained calls when the intermediate type is a broad union**.\n If `func().method()` fails because `func()` returns a union, split it:\n ```python\n # BAD: ty can't narrow through chained calls\n result = func(return_dict=True).to(device)[\"input_ids\"]\n\n # GOOD: split, narrow, then chain\n result = func(return_dict=True)\n if not hasattr(result, \"to\"):\n raise TypeError(\"Expected dict-like result\")\n inputs = result.to(device)[\"input_ids\"]\n ```\n\n d. **Fix incorrect type hints at the source**. If a parameter is typed `X | None`\n but can never be `None` when actually called, remove `None` from the hint.\n\n e. **Annotate untyped attributes**. Add type annotations to instance variables\n set in `__init__` or elsewhere (for example `self.foo: list[int] = []`).\n Declare class-level attributes that are set dynamically later\n (for example `_cache: Cache`, `_token_tensor: torch.Tensor | None`).\n\n f. **Use `@overload` for methods with input-dependent return types**.\n When a method returns different types based on the input type (e.g.\n `__getitem__` with str vs int keys), use `@overload` to declare each\n signature separately:\n ```python\n from typing import overload\n\n @overload\n def __getitem__(self, item: str) -> ValueType: ...\n @overload\n def __getitem__(self, item: int) -> EncodingType: ...\n @overload\n def __getitem__(self, item: slice) -> dict[str, ValueType]: ...\n\n def __getitem__(self, item: int | str | slice) -> ValueType | EncodingType | dict[str, ValueType]:\n ... # actual implementation\n ```\n This eliminates `cast()` calls at usage sites by giving the checker\n precise return types for each call pattern.\n\n g. **Make container classes generic to propagate value types**.\n When a class like `UserDict` holds values whose type changes after\n transformation (e.g. lists → tensors after `.to()`), make the class\n generic so methods can return narrowed types:\n ```python\n from typing import Generic, overload\n from typing_extensions import TypeVar\n\n _V = TypeVar(\"_V\", default=Any) # default=Any keeps existing code working\n\n class MyDict(UserDict, Generic[_V]):\n @overload\n def __getitem__(self, item: str) -> _V: ...\n # ...\n\n def to(self, device) -> MyDict[torch.Tensor]:\n # after .to(), values are tensors\n ...\n return self # type: ignore[return-value]\n ```\n The `default=Any` (from `typing_extensions`) means unparameterized usage\n like `MyDict()` stays `MyDict[Any]` — no existing code needs to change.\n Only methods that narrow the value type (like `.to()`) declare a specific\n return type. This eliminates `cast()` at all call sites.\n\n h. **Use `self: \"ProtocolType\"` for mixins**. When a mixin accesses attributes\n from its host class, define a Protocol in `src/transformers/_typing.py` and\n annotate `self` on methods that need it. Apply this consistently to all methods\n in the mixin. Import under `TYPE_CHECKING` to avoid circular imports.\n\n i. **Use `TypeGuard` functions for dynamic module attributes** (for example\n `torch.npu`, `torch.xpu`, `torch.compiler`). Instead of `getattr(torch, \"npu\")`\n or `hasattr(torch, \"npu\") and torch.npu.is_available()`, define a type guard\n function in `src/transformers/_typing.py`:\n ```python\n def has_torch_npu(mod: ModuleType) -> TypeGuard[Any]:\n return hasattr(mod, \"npu\") and mod.npu.is_available()\n ```\n Then use it as a narrowing check: `if has_torch_npu(torch): torch.npu.device_count()`.\n After the guard, `ty` treats the module as `Any`, allowing attribute access without\n `getattr()` or `cast()`. See existing guards in `_typing.py` for all device backends.\n\n **Key rules for type guards**:\n - Use `TypeGuard[Any]` (not a Protocol) — this is the simplest form that works\n with `ty` and avoids losing the original module's known attributes.\n - The guard function must be called directly in an `if` condition for narrowing\n to work. `ty` does NOT narrow through `and` conditions or `if not guard: return`.\n - Import guards with `from .._typing import has_torch_xxx` (not via module\n attribute `_typing.has_torch_xxx`) — `ty` only resolves `TypeGuard` from\n direct imports.\n\n j. **Use `getattr()` / `setattr()` for dynamic model/config attributes**.\n For runtime-injected fields (for example config/model flags), use\n `getattr(obj, \"field\", default)` for reads and `setattr(obj, \"field\", value)`\n for writes. Also use `getattr()` for third-party packages missing type stubs\n (for example `getattr(safetensors, \"__version__\", \"unknown\")`).\n Avoid `getattr(torch, \"npu\")` style — use type guards instead (see above).\n\n k. **Use `cast()` as a last resort before `# type: ignore`**.\n Use when you've structurally validated the type but the checker can't see it:\n pattern-matched AST nodes, known-typed dict values, or validated API responses.\n ```python\n # After structural validation confirms the type:\n stmt = cast(cst.Assign, node.body[0])\n annotations = cast(list[Annotation], [])\n ```\n Do not use `cast()` for module attribute narrowing — use type guards.\n Do not use `cast()` when `@overload` or generics can solve it at the source.\n\n l. **Use `# type: ignore` only for third-party stub defects**. This means\n cases where the third-party package's type stubs are wrong or incomplete\n and there is no way to narrow or cast around it. Examples:\n - A kwarg that exists at runtime but is missing from the stubs\n - A method that exists but isn't declared in the stubs\n Always add the specific error code: `# type: ignore[call-arg]`, not bare\n `# type: ignore`.\n\n5. **Things to never do**:\n - **Never use `assert` for type narrowing.** Asserts are stripped by `python -O`\n and must not be relied on for correctness. Use `if ...: raise` instead.\n - **Never use `# type: ignore` as a first resort.** Exhaust all approaches above first.\n - Do not use `getattr(torch, \"backend\")` to access dynamic device backends\n (`npu`, `xpu`, `hpu`, `musa`, `mlu`, `neuron`, `compiler`) — use type guards\n - Do not use `cast()` for module attribute narrowing — use type guards\n - Do not use `cast()` when `@overload` or generics can eliminate it at the source\n - Do not add helper methods or abstractions just to satisfy the type checker\n (especially for only 1-2 occurrences)\n - Do not pollute base classes with domain-specific fields; use Protocols\n - Do not add `if x is not None` guards for values guaranteed non-None\n by the call chain; fix the annotation instead\n - Do not use conditional inheritance patterns; annotate `self` instead\n\n6. **Organization**:\n - Keep shared Protocols and type aliases in `src/transformers/_typing.py`\n - Import type-only symbols under `if TYPE_CHECKING:` to avoid circular deps\n - Use `from __future__ import annotations` for PEP 604 syntax (`X | Y`)\n\n7. **Verify and close the PR loop**:\n - Re-run `ty check` on the same ``\n - Re-run `make typing` to confirm the type/model-rules step passes\n - If working toward merge readiness, run `make check-repo`\n - Ensure runtime behavior did not change and run relevant tests\n\n8. **Update CI coverage when adding new typed areas**:\n - Update `ty_check_dirs` in `Makefile` to include newly type-checked directories.\n","frontmatter":{"name":"add-or-fix-type-checking","description":"Fixes broken typing checks detected by ty, make typing, or make check-repo. Use when typing errors appear in local runs, CI, or PR logs."},"isInternal":false,"tokens":2497,"sizeBytes":10320},{"name":"CLAUDE.md","path":"CLAUDE.md","rawUrl":"https://raw.githubusercontent.com/huggingface/transformers/HEAD/CLAUDE.md","title":"Claude Agent Guidelines & System Prompt","category":"claude-rule","format":"markdown","content":"## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n","isInternal":false,"tokens":792,"sizeBytes":3287},{"name":"copilot-instructions.md","path":".github/copilot-instructions.md","rawUrl":"https://raw.githubusercontent.com/huggingface/transformers/HEAD/.github/copilot-instructions.md","title":"GitHub Copilot Instructions","category":"copilot-instructions","format":"markdown","content":"# copilot-instructions.md Guide for Hugging Face Transformers\n\nThis copilot-instructions.md file provides guidance for code agents working with this codebase.\n\n## Core Project Structure\n\n- `/src/transformers`: This contains the core source code for the library\n - `/models`: Code for individual models. Models inherit from base classes in the root `/src/transformers` directory.\n- `/tests`: This contains the core test classes for the library. These are usually inherited rather than directly run.\n - `/models`: Tests for individual models. Model tests inherit from common tests in the root `/tests` directory.\n- `/docs`: This contains the documentation for the library, including guides, tutorials, and API references.\n\n## Coding Conventions for Hugging Face Transformers\n\n- PRs should be as brief as possible. Bugfix PRs in particular can often be only one or two lines long, and do not need large comments, docstrings or new functions in this case. Aim to minimize the size of the diff.\n- When writing tests, they should be added to an existing file. The only exception is for PRs to add a new model, when a new test directory should be created for that model.\n- Code style is enforced in the CI. You can install the style tools with `pip install -e .[quality]`. You can then run `make fixup` to apply style and consistency fixes to your code.\n\n## Copying and inheritance\n\nMany models in the codebase have similar code, but it is not shared by inheritance because we want each model file to be self-contained.\nWe use two mechanisms to keep this code in sync:\n\n- \"Copied from\" syntax. Functions or entire classes can have a comment at the top like this: `# Copied from transformers.models.llama.modeling_llama.rotate_half` or `# Copied from transformers.models.t5.modeling_t5.T5LayerNorm with T5->MT5`\n These comments are actively checked by the style tools, and copies will automatically be updated when the base code is updated. If you need to update a copied function, you should\n either update the base function and use `make fixup` to propagate the change to all copies, or simply remove the `# Copied from` comment if that is inappropriate.\n- \"Modular\" files. These files briefly define models by composing them using inheritance from other models. They are not meant to be used directly. Instead, the style tools\n automatically generate a complete modeling file, like `modeling_bert.py`, from the modular file like `modular_bert.py`. If a model has a modular file, the modeling file\n should never be edited directly! Instead, changes should be made in the modular file, and then you should run `make fixup` to update the modeling file automatically.\n\nWhen adding new models, you should prefer `modular` style and inherit as many classes as possible from existing models.\n\n## Testing\n\nAfter making changes, you should usually run `make fixup` to ensure any copies and modular files are updated, and then test all affected models. This includes both\nthe model you made the changes in and any other models that were updated by `make fixup`. Tests can be run with `pytest tests/models/[name]/test_modeling_[name].py`\nIf your changes affect code in other classes like tokenizers or processors, you should run those tests instead, like `test_processing_[name].py` or `test_tokenization_[name].py`.\n\nIn order to run tests, you may need to install dependencies. You can do this with `pip install -e .[testing]`. You will probably also need to `pip install torch accelerate` if your environment does not already have them.\n","isInternal":false,"tokens":774,"sizeBytes":3529}],"systemPromptSnippet":"\n\n## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n\n\n## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n\n\n---\nname: add-or-fix-type-checking\ndescription: Fixes broken typing checks detected by ty, make typing, or make check-repo. Use when typing errors appear in local runs, CI, or PR logs.\n---\n\n# Add Or Fix Type Checking\n\n## Input\n\n- ``: module or directory to type-check (if known).\n- Optional `make typing` or CI output showing typing failures.\n\n## Workflow\n\n1. **Identify scope from the failing run**:\n - If you already have `make typing` or CI output, extract the failing file/module paths.\n - If not, run:\n ```bash\n make typing\n ```\n - Choose the narrowest target that covers the failures.\n\n2. **Run `ty check` for the target** to get a focused baseline:\n ```bash\n ty check --respect-ignore-files --exclude '**/*_pb*' \n ```\n\n3. **Triage errors by category** before fixing anything:\n - Wrong/missing type annotations on signatures\n - Attribute access on union types (for example `X | None`)\n - Functions returning broad unions (for example `str | list | BatchEncoding`)\n - Mixin/protocol self-type issues\n - Dynamic attributes on objects or modules\n - Third-party stub gaps (missing kwargs, missing `__version__`, etc.)\n\n4. **Apply fixes using this priority order** (simplest first):\n\n a. **Narrow unions with `isinstance()` / `if x is None` / `hasattr()`**.\n This is the primary tool for resolving union-type errors. `ty` narrows\n through all of these patterns, including the negative forms:\n ```python\n # Narrow X | None — use `if ...: raise`, never `assert`\n if x is None:\n raise ValueError(\"x must not be None\")\n x.method() # ty knows x is X here\n\n # Narrow str | UploadFile\n if isinstance(field, str):\n raise TypeError(\"Expected file upload, got string\")\n await field.read() # ty knows field is UploadFile here\n\n # Narrow broad union parameters early in a function body\n # (common for methods accepting e.g. list | dict | BatchEncoding)\n if isinstance(encoded_inputs, (list, tuple)):\n raise TypeError(\"Expected a mapping, got sequence\")\n encoded_inputs.keys() # ty sees only the dict/mapping types now\n ```\n\n b. **Use local variables to help ty track narrowing across closures**.\n When `self.x` is `X | None` and you need to pass it to nested functions\n or closures, `ty` cannot track that `self.x` stays non-None. Copy to a\n local variable and narrow the local:\n ```python\n manager = self.batching_manager\n if manager is None:\n raise RuntimeError(\"Manager not initialized\")\n # Use `manager` (not `self.batching_manager`) in nested functions\n ```\n\n c. **Split chained calls when the intermediate type is a broad union**.\n If `func().method()` fails because `func()` returns a union, split it:\n ```python\n # BAD: ty can't narrow through chained calls\n result = func(return_dict=True).to(device)[\"input_ids\"]\n\n # GOOD: split, narrow, then chain\n result = func(return_dict=True)\n if not hasattr(result, \"to\"):\n raise TypeError(\"Expected dict-like result\")\n inputs = result.to(device)[\"input_ids\"]\n ```\n\n d. **Fix incorrect type hints at the source**. If a parameter is typed `X | None`\n but can never be `None` when actually called, remove `None` from the hint.\n\n e. **Annotate untyped attributes**. Add type annotations to instance variables\n set in `__init__` or elsewhere (for example `self.foo: list[int] = []`).\n Declare class-level attributes that are set dynamically later\n (for example `_cache: Cache`, `_token_tensor: torch.Tensor | None`).\n\n f. **Use `@overload` for methods with input-dependent return types**.\n When a method returns different types based on the input type (e.g.\n `__getitem__` with str vs int keys), use `@overload` to declare each\n signature separately:\n ```python\n from typing import overload\n\n @overload\n def __getitem__(self, item: str) -> ValueType: ...\n @overload\n def __getitem__(self, item: int) -> EncodingType: ...\n @overload\n def __getitem__(self, item: slice) -> dict[str, ValueType]: ...\n\n def __getitem__(self, item: int | str | slice) -> ValueType | EncodingType | dict[str, ValueType]:\n ... # actual implementation\n ```\n This eliminates `cast()` calls at usage sites by giving the checker\n precise return types for each call pattern.\n\n g. **Make container classes generic to propagate value types**.\n When a class like `UserDict` holds values whose type changes after\n transformation (e.g. lists → tensors after `.to()`), make the class\n generic so methods can return narrowed types:\n ```python\n from typing import Generic, overload\n from typing_extensions import TypeVar\n\n _V = TypeVar(\"_V\", default=Any) # default=Any keeps existing code working\n\n class MyDict(UserDict, Generic[_V]):\n @overload\n def __getitem__(self, item: str) -> _V: ...\n # ...\n\n def to(self, device) -> MyDict[torch.Tensor]:\n # after .to(), values are tensors\n ...\n return self # type: ignore[return-value]\n ```\n The `default=Any` (from `typing_extensions`) means unparameterized usage\n like `MyDict()` stays `MyDict[Any]` — no existing code needs to change.\n Only methods that narrow the value type (like `.to()`) declare a specific\n return type. This eliminates `cast()` at all call sites.\n\n h. **Use `self: \"ProtocolType\"` for mixins**. When a mixin accesses attributes\n from its host class, define a Protocol in `src/transformers/_typing.py` and\n annotate `self` on methods that need it. Apply this consistently to all methods\n in the mixin. Import under `TYPE_CHECKING` to avoid circular imports.\n\n i. **Use `TypeGuard` functions for dynamic module attributes** (for example\n `torch.npu`, `torch.xpu`, `torch.compiler`). Instead of `getattr(torch, \"npu\")`\n or `hasattr(torch, \"npu\") and torch.npu.is_available()`, define a type guard\n function in `src/transformers/_typing.py`:\n ```python\n def has_torch_npu(mod: ModuleType) -> TypeGuard[Any]:\n return hasattr(mod, \"npu\") and mod.npu.is_available()\n ```\n Then use it as a narrowing check: `if has_torch_npu(torch): torch.npu.device_count()`.\n After the guard, `ty` treats the module as `Any`, allowing attribute access without\n `getattr()` or `cast()`. See existing guards in `_typing.py` for all device backends.\n\n **Key rules for type guards**:\n - Use `TypeGuard[Any]` (not a Protocol) — this is the simplest form that works\n with `ty` and avoids losing the original module's known attributes.\n - The guard function must be called directly in an `if` condition for narrowing\n to work. `ty` does NOT narrow through `and` conditions or `if not guard: return`.\n - Import guards with `from .._typing import has_torch_xxx` (not via module\n attribute `_typing.has_torch_xxx`) — `ty` only resolves `TypeGuard` from\n direct imports.\n\n j. **Use `getattr()` / `setattr()` for dynamic model/config attributes**.\n For runtime-injected fields (for example config/model flags), use\n `getattr(obj, \"field\", default)` for reads and `setattr(obj, \"field\", value)`\n for writes. Also use `getattr()` for third-party packages missing type stubs\n (for example `getattr(safetensors, \"__version__\", \"unknown\")`).\n Avoid `getattr(torch, \"npu\")` style — use type guards instead (see above).\n\n k. **Use `cast()` as a last resort before `# type: ignore`**.\n Use when you've structurally validated the type but the checker can't see it:\n pattern-matched AST nodes, known-typed dict values, or validated API responses.\n ```python\n # After structural validation confirms the type:\n stmt = cast(cst.Assign, node.body[0])\n annotations = cast(list[Annotation], [])\n ```\n Do not use `cast()` for module attribute narrowing — use type guards.\n Do not use `cast()` when `@overload` or generics can solve it at the source.\n\n l. **Use `# type: ignore` only for third-party stub defects**. This means\n cases where the third-party package's type stubs are wrong or incomplete\n and there is no way to narrow or cast around it. Examples:\n - A kwarg that exists at runtime but is missing from the stubs\n - A method that exists but isn't declared in the stubs\n Always add the specific error code: `# type: ignore[call-arg]`, not bare\n `# type: ignore`.\n\n5. **Things to never do**:\n - **Never use `assert` for type narrowing.** Asserts are stripped by `python -O`\n and must not be relied on for correctness. Use `if ...: raise` instead.\n - **Never use `# type: ignore` as a first resort.** Exhaust all approaches above first.\n - Do not use `getattr(torch, \"backend\")` to access dynamic device backends\n (`npu`, `xpu`, `hpu`, `musa`, `mlu`, `neuron`, `compiler`) — use type guards\n - Do not use `cast()` for module attribute narrowing — use type guards\n - Do not use `cast()` when `@overload` or generics can eliminate it at the source\n - Do not add helper methods or abstractions just to satisfy the type checker\n (especially for only 1-2 occurrences)\n - Do not pollute base classes with domain-specific fields; use Protocols\n - Do not add `if x is not None` guards for values guaranteed non-None\n by the call chain; fix the annotation instead\n - Do not use conditional inheritance patterns; annotate `self` instead\n\n6. **Organization**:\n - Keep shared Protocols and type aliases in `src/transformers/_typing.py`\n - Import type-only symbols under `if TYPE_CHECKING:` to avoid circular deps\n - Use `from __future__ import annotations` for PEP 604 syntax (`X | Y`)\n\n7. **Verify and close the PR loop**:\n - Re-run `ty check` on the same ``\n - Re-run `make typing` to confirm the type/model-rules step passes\n - If working toward merge readiness, run `make check-repo`\n - Ensure runtime behavior did not change and run relevant tests\n\n8. **Update CI coverage when adding new typed areas**:\n - Update `ty_check_dirs` in `Makefile` to include newly type-checked directories.\n\n\n## Commands\n\n- `make style` — ruff format + lint (required for code style checks).\n- `make typing` — ty type checker and model structure rules.\n- `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings.\n- `make check-repo` — `make typing` plus consistency checks.\n- `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI.\n\nRun `make style` (or `make fix-repo`) as the last step before opening a PR.\n\n## Local agent setup\n\nHosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`).\n\n## Before opening a PR\n\n- Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting.\n- Check for overlapping work, and do not open a second PR for a fix already covered:\n\n ```bash\n gh issue view --repo huggingface/transformers --comments\n gh pr list --repo huggingface/transformers --state open --search \" in:body\"\n gh pr list --repo huggingface/transformers --state open --search \"\"\n ```\n\n If your approach is materially different, say why a second PR is needed in the issue.\n- No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution.\n- First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked.\n\n## Copies and modular models\n\nModel files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways — modular is the current one, copies are legacy:\n\n1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. **Do not add new `# Copied from` statements**; write a modular file instead.\n2. A `modular_.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md).\n\nTwo modular traps:\n\n- **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red.\n- **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError(\"...\")` in a method body does the same for a method. Substituting a \"real\" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods).\n\n\n# copilot-instructions.md Guide for Hugging Face Transformers\n\nThis copilot-instructions.md file provides guidance for code agents working with this codebase.\n\n## Core Project Structure\n\n- `/src/transformers`: This contains the core source code for the library\n - `/models`: Code for individual models. Models inherit from base classes in the root `/src/transformers` directory.\n- `/tests`: This contains the core test classes for the library. These are usually inherited rather than directly run.\n - `/models`: Tests for individual models. Model tests inherit from common tests in the root `/tests` directory.\n- `/docs`: This contains the documentation for the library, including guides, tutorials, and API references.\n\n## Coding Conventions for Hugging Face Transformers\n\n- PRs should be as brief as possible. Bugfix PRs in particular can often be only one or two lines long, and do not need large comments, docstrings or new functions in this case. Aim to minimize the size of the diff.\n- When writing tests, they should be added to an existing file. The only exception is for PRs to add a new model, when a new test directory should be created for that model.\n- Code style is enforced in the CI. You can install the style tools with `pip install -e .[quality]`. You can then run `make fixup` to apply style and consistency fixes to your code.\n\n## Copying and inheritance\n\nMany models in the codebase have similar code, but it is not shared by inheritance because we want each model file to be self-contained.\nWe use two mechanisms to keep this code in sync:\n\n- \"Copied from\" syntax. Functions or entire classes can have a comment at the top like this: `# Copied from transformers.models.llama.modeling_llama.rotate_half` or `# Copied from transformers.models.t5.modeling_t5.T5LayerNorm with T5->MT5`\n These comments are actively checked by the style tools, and copies will automatically be updated when the base code is updated. If you need to update a copied function, you should\n either update the base function and use `make fixup` to propagate the change to all copies, or simply remove the `# Copied from` comment if that is inappropriate.\n- \"Modular\" files. These files briefly define models by composing them using inheritance from other models. They are not meant to be used directly. Instead, the style tools\n automatically generate a complete modeling file, like `modeling_bert.py`, from the modular file like `modular_bert.py`. If a model has a modular file, the modeling file\n should never be edited directly! Instead, changes should be made in the modular file, and then you should run `make fixup` to update the modeling file automatically.\n\nWhen adding new models, you should prefer `modular` style and inherit as many classes as possible from existing models.\n\n## Testing\n\nAfter making changes, you should usually run `make fixup` to ensure any copies and modular files are updated, and then test all affected models. This includes both\nthe model you made the changes in and any other models that were updated by `make fixup`. Tests can be run with `pytest tests/models/[name]/test_modeling_[name].py`\nIf your changes affect code in other classes like tokenizers or processors, you should run those tests instead, like `test_processing_[name].py` or `test_tokenization_[name].py`.\n\nIn order to run tests, you may need to install dependencies. You can do this with `pip install -e .[testing]`. You will probably also need to `pip install torch accelerate` if your environment does not already have them.\n"}