pytorch-image-models Documentation

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT,...

37,063 stars Python Python 1 file · ~21,202 tokens #augmix#convnext#distributed-training#efficientnet#image-classification#imagenet#maxvit#mixnet
RAW Doc

1. Project Overview & Quickstart (huggingface/pytorch-image-models)

File: README.md

PyTorch Image Models

What's New

August 11, 2026

  • Add model defs and pretrained weights for
  • Add per-batch image and batch size scheduling for non-NaFlex training, including progressive small-to-large resolution schedules.
  • Improve NaFlexViT with key-only attention masks, compile-friendly and memory-efficient position sampling, and cached variable-patch interpolation.
  • Improve model factory path and checkpoint handling; fix equal-token positional embedding resampling.

July 10, 2026

  • Improve optimizer torch.compile and tensor learning-rate support.
  • Extend NaFlexViT patch-layout (for NaFlex-CLAP), and forward_intermediates (NaFlex dict input) support.
  • Harden pickle loading and improve custom-label inference.
  • Release 1.0.28

May 27, 2026

May 8, 2026

  • Release 1.0.27

April 23, 2026

  • Add Gemma4 ViT encoders w/ NaFlex pipeline support (variable aspect/size per image). Thanks Yonghye Kwon
  • Support DINOv3 weights in NaFlexVit. Thanks Yonghye Kwon
  • Some improvements to Muon fallback (AdamW/NadamW) lr behavior

March 23, 2026

  • Improve pickle checkpoint handling security. Default all loading to weights_only=True, add safe_global for ArgParse.
  • Improve attention mask handling for core ViT/EVA models & layers. Resolve bool masks, pass is_causal through for SSL tasks.
  • Fix class & register token uses with ViT and no pos embed enabled.
  • Add Patch Representation Refinement (PRR) as a pooling option in ViT. Thanks Sina (https://github.com/sinahmr).
  • Improve consistency of output projection / MLP dimensions for attention pooling layers.
  • Hiera model F.SDPA optimization to allow Flash Attention kernel use.
  • Caution added to SGDP optimizer.
  • Release 1.0.26. First maintenance release since my departure from Hugging Face.

Feb 23, 2026

  • Add token distillation training support to distillation task wrappers
  • Remove some torch.jit usage in prep for official deprecation
  • Caution added to AdamP optimizer
  • Call reset_parameters() even if meta-device init so that buffers get init w/ hacks like init_empty_weights
  • Tweak Muon optimizer to work with DTensor/FSDP2 (clamp_ instead of clamp_min_, alternate NS branch for DTensor)
  • Release 1.0.25

Jan 21, 2026

  • Compat Break: Fix oversight w/ QKV vs MLP bias in ParallelScalingBlock (& DiffParallelScalingBlock)
    • Does not impact any trained timm models but could impact downstream use.

Jan 5 & 6, 2026

  • Release 1.0.24
  • Add new benchmark result csv files for inference timing on all models w/ RTX Pro 6000, 5090, and 4090 cards w/ PyTorch 2.9.1
  • Fix moved module error in deprecated timm.models.layers import path that impacts legacy imports
  • Release 1.0.23

Dec 30, 2025

Dec 12, 2025

Dec 1, 2025

  • Add lightweight task abstraction, add logits and feature distillation support to train script via new tasks.
  • Remove old APEX AMP support

Nov 4, 2025

Oct 31, 2025 🎃

  • Update imagenet & OOD variant result csv files to include a few new models and verify correctness over several torch & timm versions
  • EfficientNet-X and EfficientNet-H B5 model weights added as part of a hparam search for AdamW vs Muon (still iterating on Muon runs)

Oct 16-20, 2025

  • Add an impl of the Muon optimizer (based on https://github.com/KellerJordan/Muon) with customizations
    • extra flexibility and improved handling for conv weights and fallbacks for weight shapes not suited for orthogonalization
    • small speedup for NS iterations by reducing allocs and using fused (b)add(b)mm ops
    • by default uses AdamW (or NAdamW if nesterov=True) updates if muon not suitable for parameter shape (or excluded via param group flag)
    • like torch impl, select from several LR scale adjustment fns via adjust_lr_fn
    • select from several NS coefficient presets or specify your own via ns_coefficients
  • First 2 steps of 'meta' device model initialization supported
    • Fix several ops that were breaking creation under 'meta' device context
    • Add device & dtype factory kwarg support to all models and modules (anything inherting from nn.Module) in timm
  • License fields added to pretrained cfgs in code
  • Release 1.0.21

Sept 21, 2025

  • Remap DINOv3 ViT weight tags from lvd_1689m -> lvd1689m to match (same for sat_493m -> sat493m)
  • Release 1.0.20

Sept 17, 2025

July 23, 2025

  • Add set_input_size() method to EVA models, used by OpenCLIP 3.0.0 to allow resizing for timm based encoder models.
  • Release 1.0.18, needed for PE-Core S & T models in OpenCLIP 3.0.0
  • Fix small typing issue that broke Python 3.9 compat. 1.0.19 patch release.

July 21, 2025

  • ROPE support added to NaFlexViT. All models covered by the EVA base (eva.py) including EVA, EVA02, Meta PE ViT, timm SBB ViT w/ ROPE, and Naver ROPE-ViT can be now loaded in NaFlexViT when use_naflex=True passed at model creation time
  • More Meta PE ViT encoders added, including small/tiny variants, lang variants w/ tiling, and more spatial variants.
  • PatchDropout fixed with NaFlexViT and also w/ EVA models (regression after adding Naver ROPE-ViT)
  • Fix XY order with grid_indexing='xy', impacted non-square image use in 'xy' mode (only ROPE-ViT and PE impacted).

July 7, 2025

  • MobileNet-v5 backbone tweaks for improved Google Gemma 3n behaviour (to pair with updated official weights)
    • Add stem bias (zero'd in updated weights, compat break with old weights)
    • GELU -> GELU (tanh approx). A minor change to be closer to JAX
  • Add two arguments to layer-decay support, a min scale clamp and 'no optimization' scale threshold
  • Add 'Fp32' LayerNorm, RMSNorm, SimpleNorm variants that can be enabled to force computation of norm in float32
  • Some typing, argument cleanup for norm, norm+act layers done with above
  • Support Naver ROPE-ViT (https://github.com/naver-ai/rope-vit) in eva.py, add RotaryEmbeddingMixed module for mixed mode, weights on HuggingFace Hub
model img_size top1 top5 param_count
vit_large_patch16_rope_mixed_ape_224.naver_in1k 224 84.84 97.122 304.4
vit_large_patch16_rope_mixed_224.naver_in1k 224 84.828 97.116 304.2
vit_large_patch16_rope_ape_224.naver_in1k 224 84.65 97.154 304.37
vit_large_patch16_rope_224.naver_in1k 224 84.648 97.122 304.17
vit_base_patch16_rope_mixed_ape_224.naver_in1k 224 83.894 96.754 86.59
vit_base_patch16_rope_mixed_224.naver_in1k 224 83.804 96.712 86.44
vit_base_patch16_rope_ape_224.naver_in1k 224 83.782 96.61 86.59
vit_base_patch16_rope_224.naver_in1k 224 83.718 96.672 86.43
vit_small_patch16_rope_224.naver_in1k 224 81.23 95.022 21.98
vit_small_patch16_rope_mixed_224.naver_in1k 224 81.216 95.022 21.99
vit_small_patch16_rope_ape_224.naver_in1k 224 81.004 95.016 22.06
vit_small_patch16_rope_mixed_ape_224.naver_in1k 224 80.986 94.976 22.06
  • Some cleanup of ROPE modules, helpers, and FX tracing leaf registration
  • Preparing version 1.0.17 release

June 26, 2025

  • MobileNetV5 backbone (w/ encoder only variant) for Gemma 3n image encoder
  • Version 1.0.16 released

June 23, 2025

  • Add F.grid_sample based 2D and factorized pos embed resize to NaFlexViT. Faster when lots of different sizes (based on example by https://github.com/stas-sl).
  • Further speed up patch embed resample by replacing vmap with matmul (based on snippet by https://github.com/stas-sl).
  • Add 3 initial native aspect NaFlexViT checkpoints created while testing, ImageNet-1k and 3 different pos embed configs w/ same hparams.
Model Top-1 Acc Top-5 Acc Params (M) Eval Seq Len
naflexvit_base_patch16_par_gap.e300_s576_in1k 83.67 96.45 86.63 576
naflexvit_base_patch16_parfac_gap.e300_s576_in1k 83.63 96.41 86.46 576
naflexvit_base_patch16_gap.e300_s576_in1k 83.50 96.46 86.63 576
  • Support gradient checkpointing for forward_intermediates and fix some checkpointing bugs. Thanks https://github.com/brianhou0208
  • Add 'corrected weight decay' (https://arxiv.org/abs/2506.02285) as option to AdamW (legacy), Adopt, Kron, Adafactor (BV), Lamb, LaProp, Lion, NadamW, RmsPropTF, SGDW optimizers
  • Switch PE (perception encoder) ViT models to use native timm weights instead of remapping on the fly
  • Fix cuda stream bug in prefetch loader

June 5, 2025

  • Initial NaFlexVit model code. NaFlexVit is a Vision Transformer with:
    1. Encapsulated embedding and position encoding in a single module
    2. Support for nn.Linear patch embedding on pre-patchified (dictionary) inputs
    3. Support for NaFlex variable aspect, variable resolution (SigLip-2: https://arxiv.org/abs/2502.14786)
    4. Support for FlexiViT variable patch size (https://arxiv.org/abs/2212.08013)
    5. Support for NaViT fractional/factorized position embedding (https://arxiv.org/abs/2307.06304)
  • Existing vit models in vision_transformer.py can be loaded into the NaFlexVit model by adding the use_naflex=True flag to create_model
    • Some native weights coming soon
  • A full NaFlex data pipeline is available that allows training / fine-tuning / evaluating with variable aspect / size images
    • To enable in train.py and validate.py add the --naflex-loader arg, must be used with a NaFlexVit
  • To evaluate an existing (classic) ViT loaded in NaFlexVit model w/ NaFlex data pipe:
    • python validate.py /imagenet --amp -j 8 --model vit_base_patch16_224 --model-kwargs use_naflex=True --naflex-loader --naflex-max-seq-len 256
  • The training has some extra args features worth noting
    • The --naflex-train-seq-lens' argument specifies which sequence lengths to randomly pick from per batch during training
    • The --naflex-max-seq-len argument sets the target sequence length for validation
    • Adding --model-kwargs enable_patch_interpolator=True --naflex-patch-sizes 12 16 24 will enable random patch size selection per-batch w/ interpolation
    • The --naflex-loss-scale arg changes loss scaling mode per batch relative to the batch size, timm NaFlex loading changes the batch size for each seq len

May 28, 2025

Feb 21, 2025

  • SigLIP 2 ViT image encoders added (https://huggingface.co/collections/timm/siglip-2-67b8e72ba08b09dd97aecaf9)
    • Variable resolution / aspect NaFlex versions are a WIP
  • Add 'SO150M2' ViT weights trained with SBB recipes, great results, better for ImageNet than previous attempt w/ less training.
    • vit_so150m2_patch16_reg1_gap_448.sbb_e200_in12k_ft_in1k - 88.1% top-1
    • vit_so150m2_patch16_reg1_gap_384.sbb_e200_in12k_ft_in1k - 87.9% top-1
    • vit_so150m2_patch16_reg1_gap_256.sbb_e200_in12k_ft_in1k - 87.3% top-1
    • vit_so150m2_patch16_reg4_gap_256.sbb_e200_in12k
  • Updated InternViT-300M '2.5' weights
  • Release 1.0.15

Feb 1, 2025

  • FYI PyTorch 2.6 & Python 3.13 are tested and working w/ current main and released version of timm

Jan 27, 2025

Jan 19, 2025

  • Fix loading of LeViT safetensor weights, remove conversion code which should have been deactivated
  • Add 'SO150M' ViT weights trained with SBB recipes, decent results, but not optimal shape for ImageNet-12k/1k pretrain/ft
    • vit_so150m_patch16_reg4_gap_256.sbb_e250_in12k_ft_in1k - 86.7% top-1
    • vit_so150m_patch16_reg4_gap_384.sbb_e250_in12k_ft_in1k - 87.4% top-1
    • vit_so150m_patch16_reg4_gap_256.sbb_e250_in12k
  • Misc typing, typo, etc. cleanup
  • 1.0.14 release to get above LeViT fix out

Jan 9, 2025

  • Add support to train and validate in pure bfloat16 or float16
  • wandb project name arg added by https://github.com/caojiaolong, use arg.experiment for name
  • Fix old issue w/ checkpoint saving not working on filesystem w/o hard-link support (e.g. FUSE fs mounts)
  • 1.0.13 release

Jan 6, 2025

  • Add torch.utils.checkpoint.checkpoint() wrapper in timm.models that defaults use_reentrant=False, unless TIMM_REENTRANT_CKPT=1 is set in env.

Dec 31, 2024

Introduction

PyTorch Image Models (timm) is a collection of image models, layers, utilities, optimizers, schedulers, data-loaders / augmentations, and reference training / validation scripts that aim to pull together a wide variety of SOTA models with ability to reproduce ImageNet training results.

The work of many others is present here. I've tried to make sure all source material is acknowledged via links to github, arxiv papers, etc in the README, documentation, and code docstrings. Please let me know if I missed anything.

Features

Models

All model architecture families include variants with pretrained weights. There are specific model variants without any weights, it is NOT a bug. Help training new or better weights is always appreciated.

Optimizers

To see full list of optimizers w/ descriptions: timm.optim.list_optimizers(with_description=True)

Included optimizers available via timm.optim.create_optimizer_v2 factory method:

Augmentations

Regularization

Other

Several (less common) features that I often utilize in my projects are included. Many of their additions are the reason why I maintain my own set of models, instead of using others' via PIP:

Results

Model validation results can be found in the results tables

Getting Started (Documentation)

The official documentation can be found at https://huggingface.co/docs/hub/timm. Documentation contributions are welcome.

Getting Started with PyTorch Image Models (timm): A Practitioner’s Guide by Chris Hughes is an extensive blog post covering many aspects of timm in detail.

timmdocs is an alternate set of documentation for timm. A big thanks to Aman Arora for his efforts creating timmdocs.

paperswithcode is a good resource for browsing the models within timm.

Train, Validation, Inference Scripts

The root folder of the repository contains reference train, validation, and inference scripts that work with the included models and other features of this repository. They are adaptable for other datasets and use cases with a little hacking. See documentation.

Awesome PyTorch Resources

One of the greatest assets of PyTorch is the community and their contributions. A few of my favourite resources that pair well with the models and components here are listed below.

Object Detection, Instance and Semantic Segmentation

Computer Vision / Image Augmentation

Knowledge Distillation

Metric Learning

Training / Frameworks

Deployment

Licenses

Code

The code here is licensed Apache 2.0. I've taken care to make sure any third party code included or adapted has compatible (permissive) licenses such as MIT, BSD, etc. I've made an effort to avoid any GPL / LGPL conflicts. That said, it is your responsibility to ensure you comply with licenses here and conditions of any dependent licenses. Where applicable, I've linked the sources/references for various components in docstrings. If you think I've missed anything please create an issue.

Pretrained Weights

So far all of the pretrained weights available here are pretrained on ImageNet with a select few that have some additional pretraining (see extra note below). ImageNet was released for non-commercial research purposes only (https://image-net.org/download). It's not clear what the implications of that are for the use of pretrained weights from that dataset. Any models I have trained with ImageNet are done for research purposes and one should assume that the original dataset license applies to the weights. It's best to seek legal advice if you intend to use the pretrained weights in a commercial product.

Pretrained on more than ImageNet

Several weights included or references here were pretrained with proprietary datasets that I do not have access to. These include the Facebook WSL, SSL, SWSL ResNe(Xt) and the Google Noisy Student EfficientNet models. The Facebook models have an explicit non-commercial license (CC-BY-NC 4.0, https://github.com/facebookresearch/semi-supervised-ImageNet1K-models, https://github.com/facebookresearch/WSL-Images). The Google models do not appear to have any restriction beyond the Apache 2.0 license (and ImageNet concerns). In either case, you should contact Facebook or Google with any questions.

Citing

BibTeX

bibtex
@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/rwightman/pytorch-image-models}}
}

Latest DOI

File: hfdocs/README.md

Hugging Face Timm Docs

Getting Started

text
pip install git+https://github.com/huggingface/doc-builder.git@main#egg=hf-doc-builder
pip install watchdog black

Preview the Docs Locally

text
doc-builder preview timm hfdocs/source

File: results/README.md

Validation and Benchmark Results

This folder contains validation and benchmark results for the models in this collection. Validation scores are currently only run for models with pretrained weights and ImageNet-1k heads, benchmark numbers are run for all.

Datasets

There are currently results for the ImageNet validation set and 5 additional test / label sets.

The test set results include rank and top-1/top-5 differences from clean validation. For the "Real Labels", ImageNetV2, and Sketch test sets, the differences were calculated against the full 1000 class ImageNet-1k validation set. For both the Adversarial and Rendition sets, the differences were calculated against 'clean' runs on the ImageNet-1k validation set with the same 200 classes used in each test set respectively.

ImageNet Validation - [`results-imagenet.csv`](results-imagenet.csv)

The standard 50,000 image ImageNet-1k validation set. Model selection during training utilizes this validation set, so it is not a true test set. Question: Does anyone have the official ImageNet-1k test set classification labels now that challenges are done?

ImageNet-"Real Labels" - [`results-imagenet-real.csv`](results-imagenet-real.csv)

The usual ImageNet-1k validation set with a fresh new set of labels intended to improve on mistakes in the original annotation process.

ImageNetV2 Matched Frequency - [`results-imagenetv2-matched-frequency.csv`](results-imagenetv2-matched-frequency.csv)

An ImageNet test set of 10,000 images sampled from new images roughly 10 years after the original. Care was taken to replicate the original ImageNet curation/sampling process.

ImageNet-Sketch - [`results-sketch.csv`](results-sketch.csv)

50,000 non photographic (or photos of such) images (sketches, doodles, mostly monochromatic) covering all 1000 ImageNet classes.

ImageNet-Adversarial - [`results-imagenet-a.csv`](results-imagenet-a.csv)

A collection of 7500 images covering 200 of the 1000 ImageNet classes. Images are naturally occurring adversarial examples that confuse typical ImageNet classifiers. This is a challenging dataset, your typical ResNet-50 will score 0% top-1.

For clean validation with same 200 classes, see results-imagenet-a-clean.csv

ImageNet-Rendition - [`results-imagenet-r.csv`](results-imagenet-r.csv)

Renditions of 200 ImageNet classes resulting in 30,000 images for testing robustness.

For clean validation with same 200 classes, see results-imagenet-r-clean.csv

TODO

Benchmark

CSV files with a model_benchmark prefix include benchmark numbers for models on various accelerators with different precision. Currently only run on RTX 3090 w/ AMP for inference, I intend to add more in the future.

Metadata

CSV files with model_metadata prefix contain extra information about the source training, currently the pretraining dataset and technique (ie distillation, SSL, WSL, etc). Eventually I'd like to have metadata about augmentation, regularization, etc. but that will be a challenge to source consistently.

2. Official Technical Reference & Guides (huggingface/doc-builder)

doc-builder

This is the package we use to build the documentation of our Hugging Face repos.

Table of Contents

Installation

You can install from PyPi with

bash
pip install hf-doc-builder

To install from source, clone this repository then

bash
cd doc-builder
pip install -e .

Previewing

To preview the docs, use the following command:

bash
doc-builder preview {package_name} {path_to_docs}

For example:

bash
doc-builder preview datasets ~/Desktop/datasets/docs/source/

**preview command only works with existing doc files. When you add a completely new file, you need to update _toctree.yml & restart preview command (ctrl-c to stop it & call doc-builder preview ... again).

Doc building

To build the documentation of a given package, use the following command:

bash
#Add --not_python_module if not building doc for a python lib
doc-builder build {package_name} {path_to_docs} --build_dir {build_dir}

For instance, here is how you can build the Datasets documentation (requires pip install datasets[dev]) if you have cloned the repo in ~/git/datasets:

bash
doc-builder build datasets ~/git/datasets/docs/source --build_dir ~/tmp/test-build

This will generate MDX files that you can preview like any Markdown file in your favorite editor. To have a look at the documentation in HTML, you need to install node version 14 or higher. Then you can run (still with the example on Datasets)

bash
doc-builder build datasets ~/git/datasets/docs/source --build_dir ~/tmp/test-build --html

which will build HTML files in ~/tmp/test-build. You can then inspect those files in your browser.

Specifying a version

By default, doc-builder automatically determines the version:

  • For Python modules: uses the package's __version__ attribute (or main for dev versions)
  • For non-Python modules (with --not_python_module): uses the default branch name (typically main)

You can override this behavior with the --version flag:

bash
# Build documentation for a specific semantic version (must start with "v")
doc-builder build transformers ~/git/transformers/docs/source --build_dir ~/tmp/test-build --version v4.30.0

# Build documentation for the default branch (non-Python modules)
doc-builder build hub ~/git/hub-docs/docs/source --build_dir ~/tmp/test-build --not_python_module --version main

Important: When specifying a semantic version with --version, it must start with the letter "v" (e.g., v1.0.0, v2.3.1). For branch names like main, master, or other default branches, the "v" prefix is not required.

Page-level HTML build cache

Building the HTML docs of a large library (e.g. transformers) prerenders hundreds of pages, even when a commit only changed one of them. --html_page_cache enables a page-level cache so that only pages whose generated content changed are prerendered again:

bash
# cache in a Hugging Face storage bucket (https://huggingface.co/docs/hub/storage-buckets) ...
doc-builder build datasets ~/git/datasets/docs/source --build_dir ~/tmp/test-build --html \
    --html_page_cache hf://buckets/hf-doc-build/doc-build-cache

# ... or in a local directory
doc-builder build datasets ~/git/datasets/docs/source --build_dir ~/tmp/test-build --html \
    --html_page_cache ~/tmp/doc-page-cache --html_page_cache_write

A page is reused when its generated MDX (post autodoc and internal-link resolution — so docstring changes invalidate exactly the affected pages), the kit folder, and the doc-builder version are unchanged. Keys are version-normalized: pages that only differ by the built version (e.g. main vs pr_123 in resolved internal links) share a cache entry, and on a cross-version reuse the page URLs are rewritten to the target version while asset URLs stay pinned to the source version, whose content-hashed assets remain hosted.

--html_page_cache_write also stores freshly built pages. Only enable it on trusted builds (the shared GitHub workflows enable it on main-branch builds and keep PR builds read-only), so that untrusted code cannot poison the cache. Cache failures are never fatal: any error degrades to building the affected pages.

Building without installing heavy dependencies

doc-builder imports the documented library to read docstrings with inspect — necessary because libraries like transformers construct docstrings dynamically. Historically that meant installing the library's full dependency tree (torch, GPU wheels, custom containers) just to build documentation.

doc-builder mocks heavy dependencies instead, so only the documented library itself (and its light dependencies) needs to be installed. Each library's registry file src/doc_builder/mock_deps/<library>.txt records both sides of that split:

  • bare lines name the heavy dependencies to mock at build time (applied automatically by doc-builder build/preview);
  • real: lines name the light dependencies a doc build must install for real (things the library imports unguarded, version-pinned requirements, packages used through isinstance checks, ...);
  • nodeps: lines are also installed for real but with --no-deps, because their own dependency trees pull in heavy packages (e.g. accelerate, whose install requires torch).

doc-builder light-install <library> installs the real:/nodeps: dependencies (via uv), so a full light setup is:

bash
uv pip install ./accelerate --no-deps
doc-builder light-install accelerate
doc-builder build accelerate ~/git/accelerate/docs/source --build_dir ~/tmp/test-build
# torch and deepspeed are mocked automatically (src/doc_builder/mock_deps/accelerate.txt)

Install the library before running light-install: bare registry names are pinned to the library's own declared version ranges (e.g. transformers' tokenizers>=0.22,<=0.23), so registry files never have to chase upstream pins. light-install exits with code 3 when the library has no registry entry — light-install <library> --check tests this without installing anything, which is how the shared GitHub workflows decide between the light path and a full [dev] install. For a library without a registry entry (or to experiment), --mock_deps torch,... adds extra mocks on top of the registry.

The registry key is the doc-builder library name as passed to doc-builder build — for namespace subpackages that is the dotted module name (e.g. mock_deps/optimum.intel.txt), and libraries that build under a shared name share a file (optimum-onnx builds as optimum). A registered name may itself be a dotted submodule (e.g. optimum.onnxruntime), which mocks a missing sibling inside an installed namespace package.

The mocked packages are importable, pass the importlib.util.find_spec + importlib.metadata availability checks HF libraries use, are subclassable without affecting the real subclass's signature or docstring, behave as pass-through decorators, and render in signatures like the real objects (repr(torch.float32) == "torch.float32"). Packages that are actually installed are never mocked, and doc-builder preview supports the flag too.

Fidelity, verified on accelerate built without torch installed: 46/48 pages byte-identical to a real-torch build; the two remaining pages differ only in typing paths rendered from the public import path instead of the internal one (e.g. torch.nn.Module instead of torch.nn.modules.module.Module).

Notebook conversion

doc-builder can also automatically convert some of the documentation guides or tutorials into notebooks. This requires two steps:

  • add [[open-in-colab]] in the tutorial for which you want to build a notebook
  • add --notebook_dir {path_to_notebook_folder} to the build command.

Runnable code blocks

doc-builder recognizes runnable Python fences tagged with runnable or runnable:<label>:

md
```py runnable:quickstart
from transformers import pipeline
pipe = pipeline("sentiment-analysis")
print(pipe("I love this!"))
```

You can use either py or python fences:

md
```python runnable:my_example
print("hello")
```

During conversion:

  • the runnable annotation is removed from the fence in rendered docs
  • the code content is preserved unless lines are hidden with # doc-builder: hide
  • # doc-builder: hide can hide a single line or a full indented block from rendered docs

The label is used in warning messages (for example, runnable:quickstart) to identify the block.

doc-builder tags and transforms runnable blocks, but it does not execute them by itself.

Use the reusable helper from hf-doc-builder:

python
from pathlib import Path

from doc_builder.testing import DocIntegrationTest


class MyPageDocIntegrationTest(DocIntegrationTest):
    doc_path = Path(__file__).resolve().parents[2] / "docs" / "source" / "en" / "my_page.md"

DocIntegrationTest finds runnable py/python fences in the target markdown file, creates one test per block, and executes each block with contextual failure output.

Run locally with:

bash
pytest -q tests/docs/test_my_page_docs.py

This executes trusted documentation code with exec, so keep it limited to repo-controlled docs and CI.

For continuation blocks, # pytest-decorator, bare-assert warnings, and GitHub Actions wiring, see docs/runnable-code-blocks.md.

Writing in notebooks

You can write your docs in jupyter notebooks & use doc-builder to: turn jupyter notebooks into mdx files.

In some situations, such as course & tutorials, it makes more sense to write in jupyter notebooks (& use doc-builder converter) rather than writing in mdx files directly.

The process is:

  1. In your build_main_documentation.yml & build_pr_documentation.yml enable the flag convert_notebooks: true.
  2. After this flag is enabled, doc-builder will convert all .ipynb files in path_to_docs to mdx files.

Moreover, you can locally convert .ipynb files into mdx files.

bash
doc-builder notebook-to-mdx {path to notebook file or folder containing notebook files}

Templates for GitHub Actions

doc-builder provides templates for GitHub Actions, so you can build your documentation with every pull request, push to some branch etc. To use them in your project, simply create the following three files in the .github/workflows/ directory:

  • build_main_documentation.yml: responsible for building the docs for the main branch, releases etc.
  • build_pr_documentation.yml: responsible for building the docs on each PR.
  • upload_pr_documentation.yml: responsible for uploading the PR artifacts to the Hugging Face Hub.
  • delete_doc_comment_trigger.yml: responsible for removing the comments from the HuggingFaceDocBuilder bot that provides a URL to the PR docs.

Within each workflow, the main thing to include is a pointer from the uses field to the corresponding workflow in doc-builder. For example, this is what the PR workflow looks like in the datasets library:

yaml
name: Build PR Documentation

on:
  pull_request:

concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

jobs:
  build:
    uses: huggingface/doc-builder/.github/workflows/build_pr_documentation.yml@main # Runs this doc-builder workflow
    with:
      commit_sha: ${{ github.event.pull_request.head.sha }}
      pr_number: ${{ github.event.number }}
      package: datasets # Replace this with your package name

Note the use of special arguments like pr_number and package under the with field. You can find the various options by inspecting each of the doc-builder workflow files.

Enabling multilingual documentation

doc-builder can also convert documentation that's been translated from the English source into one or more languages. To enable the conversion, the documentation directories should be structured as follows:

text
doc_folder
├── en
│   ├── _toctree.yml
│   ├── _redirects.yml
│   ...
└── es
    ├── _toctree.yml
    ├── _redirects.yml
    ...

Note that each language directory has its own table of contents file _toctree.yml and that all languages are arranged under a single doc_folder directory - see the course repo for an example. You can then build the individual language subsets as follows:

bash
doc-builder build {package_name} {path_to_docs} --build_dir {build_dir} --language {lang_id}

To automatically build the documentation for all languages via the GitHub Actions templates, simply provide the languages argument to your workflow, with a space-separated list of the languages you wish to build, e.g. languages: en es.

Redirects

You can optionally provide _redirects.yml for "old links". The yml file should look like:

yml
how_to: getting_started
package_reference/classes: package_reference/main_classes
# old_local: new_local

Fixing and testing doc-builder

If you are working on a fix or an update of the doc-builder tool itself, you will eventually want to test it in the CI of another repository (transformers, diffusers, courses, etc.). To do so you should set the doc_builder_revision argument in your workflow file to point to your branch. Here is an example of what it would look like in the transformers.js project:

yml
jobs:
  build:
    uses: huggingface/doc-builder/.github/workflows/build_pr_documentation.yml@my-test-branch
    with:
      repo_owner: xenova
      commit_sha: ${{ github.sha }}
      pr_number: ${{ github.event.number }}
      package: transformers.js
      path_to_docs: transformers.js/docs/source
      pre_command: cd transformers.js && npm install && npm run docs-api
      additional_args: --not_python_module
      doc_builder_revision: my-test-branch # <- add this line

Once the docs build is complete in your project, you can drop that change.

Development docs

To test doc-builder changes locally, use the dummy/ folder. It contains a minimal setup for manual testing. Don't commit your changes in dummy/ since it's only for local development.

Run a preview with:

bash
doc-builder preview dummy dummy --not_python_module

Documentation page should be available at http://localhost:5173/index.

Development Commands

Since we've modernized the tooling to use ruff and uv, here are the commands for development:

Code Quality & Formatting

Check code quality (linting)

bash
uv run ruff check .

Fix auto-fixable linting issues

bash
uv run ruff check --fix .

Check code formatting

bash
uv run ruff format --check .

Apply code formatting

bash
uv run ruff format .

Run both linting and formatting checks (equivalent to old `make quality`)

bash
uv run ruff check . && uv run ruff format --check .

Apply both linting fixes and formatting (equivalent to old `make style`)

bash
uv run ruff check --fix . && uv run ruff format .

Check for broken links in the docs (includes line, link text and url)

bash
uv run doc-builder check-links /path/to/docs
bash
uv run doc-builder check-links /path/to/docs --format link

Testing

Run all tests (equivalent to old `make test`)

bash
uv run python -m pytest -n 1 --dist=loadfile -s -v ./tests/

Run specific test file

bash
uv run python -m pytest tests/test_utils.py -v

Run tests with coverage

bash
uv run python -m pytest --cov=src/doc_builder tests/

Development Workflow

Install dependencies for development

bash
uv sync --extra dev

Install only testing dependencies

bash
uv sync --extra testing

Install only quality/linting dependencies

bash
uv sync --extra quality

Run the full quality check (for CI)

bash
uv run ruff check .
uv run ruff format --check .

Run the full test suite (for CI)

bash
uv run python -m pytest -n 1 --dist=loadfile -s -v ./tests/

Writing documentation for Hugging Face libraries

doc-builder expects Markdown so you should write any new documentation in ".mdx" files for tutorials, guides, API documentations. For docstrings, we follow the Google format with the main difference that you should use Markdown instead of restructured text (hopefully, that will be easier!)

Values that should be put in code should either be surrounded by backticks: `like so`. Note that argument names
and objects like True, None or any strings should usually be put in code.

Multi-line code blocks can be useful for displaying examples. They are done between two lines of three backticks as usual in Markdown:

text
```
# first line of code
# second line
# etc
```

We follow the doctest syntax for the examples to automatically test
the results stay consistent with the library.

Syntax:

html
[`XXXClass`] or [`~XXXClass`] // for class
[`XXXClass.method`] or [`~XXXClass.method`] // for method

Example: here & here (as used inside docstring).

When mentioning a class, function or method, it is recommended to use the following syntax for internal links so that our tool
automatically adds a link to its documentation: [`XXXClass`] or [`function`]. This requires the class or
function to be in the main package.

If you want to create a link to some internal class or function, you need to
provide its path. For instance, in the Transformers documentation [`file_utils.ModelOutput`] will create a link to the documentation of ModelOutput. This link will have file_utils.ModelOutput in the description. To get rid of the path and only keep the name of the object you are
linking to in the description, add a ~: [`~file_utils.ModelOutput`] will generate a link with ModelOutput in the description.

The same works for methods, so you can either use [`XXXClass.method`] or [`~XXXClass.method`].

Syntax:

html
[`XXXLibrary.XXXClass`] or [`~XXXLibrary.XXXClass`] // for class
[`XXXLibrary.XXXClass.method`] or [`~XXXLibrary.XXXClass.method`] // for method

Example: here linking object from accelerate inside transformers.

Tip

To write a block that you'd like to see highlighted as a note or warning, use the blockquote syntax. This is the
preferred approach for new content.

Syntax:

md
> [!TIP]
> Here is a tip. Go to this url [website](www.tip.com)
> 
> Second line

Example: here

Legacy docs may still use the <Tip> component, but we recommend migrating to blockquotes when updating files.

For warnings, change the introduction to:

Syntax:

md
> [!WARNING]

Example: here

Framework Content

If your documentation has a block that is framework-dependent (PyTorch vs TensorFlow vs Flax), you can use the
following syntax:

Syntax:

html
<frameworkcontent>

PyTorch content goes here

<tf>
TensorFlow content goes here
</tf>
<flax>
Flax content goes here
</flax>
</frameworkcontent>

Example: here

Note: all frameworks are optional (you can write a PyTorch-only block for instance) and the order does not matter.

Options

Show alternatives (let's say code blocks for different version of a library etc.) in a way where a user can select an option and see the selected option content:

Syntax:

html
<hfoptions id="some id">
<hfoption id="id for option 1">
{YOUR MARKDOWN}
</hfoption>
<hfoption id="id for option 2">
{YOUR MARKDOWN}
</hfoption>
... however many <hfoption> tags
</hfoptions>

Example: here

Note: for multiple <hfoptions id="some id"> in a same page, you may consider using same id so that when a user selects one option it affects all other hfoptions blocks. If you don't want this behaviour, use different ids.

Anchor links for markdown headings are generated automatically (with the following rule: 1. lowercase, 2. replace space with dash -, 3. strip [^a-z0-9-]):

Syntax:

text
## My awesome section
// the anchor link is: `my-awesome-section`

Example: here

Moreover, there is a way to customize the anchor link.

Syntax:

text
## My awesome section[[some-section]]
// the anchor link is: `some-section`

Example: here

LaTeX

Latex display mode. $$...$$

Syntax:

text
$$Y = X * \textbf{dequantize}(W); \text{quantize}(W)$$

Example: here

Latex inline mode. some prose \\( ... \\) more prose

Syntax:

text
A paragraph about \\( Y = X * \textbf{dequantize}(W); \text{quantize}(W) \\) the equation.

Example: here

Code Blocks

Code blocks are written using a regular markdown syntax ```. However, there is a special flag you can put in your mdx files to change the wrapping style of the resulting html from overflow/scrollbar to wrap.

Syntax:

text

Example: here

Stretch Tables

By default, tables in the documentation are sized to fit their content. If you want tables to stretch to the full width of the content area, you can add a special flag at the top of your mdx file.

Syntax:

text

This will make all tables in that page expand to 100% width of the content container.

Inference Snippet

The InferenceSnippet component is used to render an interactive interface for AI model inference. It uses huggingface/huggingface.js under the hood to get the snippets.

Props

Below is a description of the props that can be passed to this component:

  • pipeline (string, required):
    Specifies the type of pipeline to be used for inference. Common values include "text-generation", "text-classification", etc.

  • providersMapping (mapping of {modelId: string, providerModelId: string}, required):
    A mapping which keys are provider names and values are objects with modelId and providerModelId.
    Example: {"fireworks-ai": {modelId: "deepseek-ai/DeepSeek-R1", providerModelId: "accounts/fireworks/models/deepseek-r1", novita: {modelId: "deepseek-ai/DeepSeek-V3-0324", providerModelId: "deepseek/deepseek-v3-0324"}}

  • conversational (boolean, optional):
    If set to true, the component will enable conversational mode, allowing for multi-turn interactions for text-generation models.

Example Usage

svelte
svelte
svelte

Adding new inference provider

Step 1: get latest huggingface/huggingface.js by running the command below:

text
cd kit
npm run update-inference-providers

Step 2: add an icon for the new provider in kit/src/lib/InferenceSnippet/InferenceSnippet.svelte.

Writing API documentation (Python)

Autodoc

To show the full documentation of any object of the python library you are documenting, use the [[autodoc]] marker.

Syntax:

text
[[autodoc]] SomeObject

Example: here

If the object is a class, this will include every public method of it that is documented. If for some reason you wish for a method
not to be displayed in the documentation, you can do so by specifying which methods should be in the docs, here is an example:

Syntax:

text
[[autodoc]] XXXTokenizer
    - build_inputs_with_special_tokens
    - get_special_tokens_mask
    - create_token_type_ids_from_sequences
    - save_vocabulary

Example: here

If you just want to add a method that is not documented (for instance magic method like __call__ are not documented
by default) you can put the list of methods to add in a list that contains all:

Syntax:

text
## XXXTokenizer

[[autodoc]] XXXTokenizer
    - all
    - __call__

Example: here

Code Blocks from file references

You can create a code-block by referencing a file excerpt with <literalinclude> (sphinx-inspired) syntax.
There should be json between <literalinclude> open & close tags.

Syntax:

text
<literalinclude>
{"path": "./data/convert_literalinclude_dummy.txt", # relative path
"language": "python", # defaults to " (empty str)
"start-after": "START python_import",  # defaults to start of file
"end-before": "END python_import",  # defaults to end of file
"dedent": 7 # defaults to 0
}
</literalinclude>

Writing source documentation

Description

For a class or function description string, use markdown with all the custom syntax of doc-builder.

Example: here

Arguments

Arguments of a function/class/method should be defined with the Args: (or Arguments: or Parameters:) prefix, followed by a line return and
an indentation. The argument should be followed by its type, with its shape if it is a tensor, a colon, and its
Syntax:

text
Args:
        n_layers (`int`): The number of layers of the model.

Example: here

If the description is too long to fit in one line, another indentation is necessary before writing the description
after the argument.

Syntax:

text
Args:
        input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
            Indices of input sequence tokens in the vocabulary.

            Indices can be obtained using [`AlbertTokenizer`]. See [`~PreTrainedTokenizer.encode`] and
            [`~PreTrainedTokenizer.__call__`] for details.

            [What are input IDs?](../glossary#input-ids)

Example: here

You can check the full example it comes from here

Attributes

If a class is similar to that of a dataclass but the parameters do not align to the available attributes of the class, such as in the below example, Attributes instance should be rewritten as **Attributes** in order to have the documentation properly render these. Otherwise it will assume that Attributes is synonymous to Parameters.

Syntax:

diff
class SomeClass:
      """
      Docstring
-     Attributes:
+     **Attributes**:
          - **attr_a** (`type_a`) -- Doc a
          - **attr_b** (`type_b`) -- Doc b
      """
      def __init__(self, param_a, param_b):
          ...

Parameter typing and default value

For optional arguments or arguments with defaults we follow the following syntax. Imagine we have a function with the
following signature:

text
def my_function(x: str = None, a: float = 1):

then its documentation should look like this:

Syntax:

text
Args:
        x (`str`, *optional*):
            This argument controls ...
        a (`float`, *optional*, defaults to 1):
            This argument is used to ...

Example: here

Note that we always omit the "defaults to `None`" when None is the default for any argument. Also note that even
if the first line describing your argument type and its default gets long, you can't break it on several lines. You can
however write as many lines as you want in the indented description (see the example above with input_ids).

If your argument has for type a class defined in the package, you can use the syntax we saw earlier to link to its
documentation:

text
Args:
         config ([`BertConfig`]):
            Model configuration class with all the parameters of the model.

            Initializing with a config file does not load the weights associated with the model, only the
            configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights.

Returns

The return block should be introduced with the Returns: prefix, followed by a line return and an indentation.
The first line should be the type of the return, followed by a line return. No need to indent further for the elements
building the return.

Here's an example for a single value return:

Syntax:

text
Returns:
        `List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.

Example: here

Here's an example for tuple return, comprising several objects:

Syntax:

text
Returns:
        `tuple(torch.FloatTensor)` comprising various elements depending on the configuration ([`BertConfig`]) and inputs:
        - ** loss** (*optional*, returned when `masked_lm_labels` is provided) `torch.FloatTensor` of shape `(1,)` --
          Total loss as the sum of the masked language modeling loss and the next sequence prediction (classification) loss.
        - **prediction_scores** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`) --
          Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).

Example: here

Yields

Similarly, Yields is also supported.

Syntax:

text
Yields:
    `tuple[str, io.BufferedReader]`:
        2-tuple (path_within_archive, file_object).
        File object is opened in binary mode.

Example: here

Raises

You can also document Raises.

Syntax:

text
Args:
         config ([`BertConfig`]):
            Model configuration class with all the parameters of the model.

            Initializing with a config file does not load the weights associated with the model, only the
            configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights.

    Raises:
        `pa.ArrowInvalidError`: if the arrow data casting fails
        TypeError: if the target type is not supported according, e.g.
            - point1
            - point2
        [`HTTPError`](https://2.python-requests.org/en/master/api/#requests.HTTPError) if credentials are invalid
        [`HTTPError`](https://2.python-requests.org/en/master/api/#requests.HTTPError) if connection got lost

    Returns:
        `List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.

Example: here

Directives for Added, Changed, Deprecated

There are directives for Added, Changed, & Deprecated.
Syntax:

text
Args:
        cache_dir (`str`, *optional*): Directory to cache data.
        config_name (`str`, *optional*): Name of the dataset configuration.
            It affects the data generated on disk: different configurations will have their own subdirectories and
            versions.
            If not provided, the default configuration is used (if it exists).

            <Added version="2.3.0">

            `name` was renamed to `config_name`.

            </Added>
        name (`str`): Configuration name for the dataset.

            <Deprecated version="2.3.0">

            Use `config_name` instead.

            </Deprecated>

Example: here

Developing svelte locally

We use svelte components for doc UI (Tip component, Docstring component, etc.).

Follow these steps to develop svelte locally:

  1. Create this file if it doesn't already exist: doc-builder/kit/src/routes/_toctree.yml. Contents should be:
text
- sections: 
  - local: index
    title: Index page
  title: Index page
  1. Create this file if it doesn't already exist: doc-builder/kit/src/routes/index.mdx. Contents should be whatever you'd like to test. For example:
text
<script lang="ts">
import Tip from "$lib/Tip.svelte";
import Youtube from "$lib/Youtube.svelte";
import Docstring from "$lib/Docstring.svelte";
import CodeBlock from "$lib/CodeBlock.svelte";
import CodeBlockFw from "$lib/CodeBlockFw.svelte";
</script>

<Tip>

  [Here](https://myurl.com)

</Tip>

## Some heading
And some text [Here](https://myurl.com)

Physics is the natural science that studies matter,[a] its fundamental constituents, its motion and behavior through space and time, and the related entities of energy and force.[2] Physics is one of the most fundamental scientific disciplines, with its main goal being to understand how the universe behaves.[b][3][4][5] A scientist who specializes in the field of physics is called a physicist.
  1. Install dependencies & run dev mode
bash
cd doc-builder/kit
npm ci
npm run dev -- --open
  1. Start developing. See svelte files in doc-builder/kit/src/lib for reference. The flow should be:
    1. Create a svelte component in doc-builder/kit/src/lib
    2. Import it & test it in doc-builder/kit/src/routes/index.mdx

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#augmix Browse augmix catalog → #convnext Browse convnext catalog → #distributed-training Browse distributed-training catalog → #efficientnet Browse efficientnet catalog → #image-classification Browse image-classification catalog → #imagenet Browse imagenet catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →