dinov2 Documentation

Technical Documentation: facebookresearch/dinov2

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`)

:new: [2025-12-18] Added support for loading XRay-DINO backbone following Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learning, more details are here

:new: [2025-12-16] Added Channel-Adaptive DINO code following Scaling Channel-Adaptive Self-Supervised Learning, more details are here

:new: [2025-12-16] Added Cell-DINO code following [Cell-DINO: Self-Supervised Image-based Embeddings for Cell Fluorescent Microscopy](to appear in Plos One Computational Biology), more details are here

[2025-08-14] Please check out the more recent DINOv3 effort continuing this line of work.

[2025-06-11] Added dino.txt inference code, following DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment.

[2023-10-26] Added DINOv2 backbones with registers, following Vision Transformers Need Registers.

DINOv2: Learning Robust Visual Features without Supervision

Meta AI Research, FAIR

Maxime Oquab,
Timothée Darcet,
Théo Moutakanni,
Huy V. Vo,
Marc Szafraniec,
Vasil Khalidov,
Patrick Labatut,
Armand Joulin,
Piotr Bojanowski

[Paper #1] Paper #2] [Blog] [Demo] [BibTeX]

PyTorch implementation and pretrained models for DINOv2. For details, see the papers: DINOv2: Learning Robust Visual Features without Supervision and Vision Transformers Need Registers.

DINOv2 models produce high-performance visual features that can be directly employed with classifiers as simple as linear layers on a variety of computer vision tasks; these visual features are robust and perform well across domains without any requirement for fine-tuning. The models were pretrained on a dataset of 142 M images without using any labels or annotations.

https://github.com/facebookresearch/dinov2/assets/60359573/f168823e-7922-415a-b429-578badf5c356

Pretrained models

model	# ofparams	withregisters	ImageNetk-NN	ImageNetlinear	download
ViT-S/14 distilled	21 M	:x:	79.0%	81.1%	backbone only
ViT-S/14 distilled	21 M	:white_check_mark:	79.1%	80.9%	backbone only
ViT-B/14 distilled	86 M	:x:	82.1%	84.5%	backbone only
ViT-B/14 distilled	86 M	:white_check_mark:	82.0%	84.6%	backbone only
ViT-L/14 distilled	300 M	:x:	83.5%	86.3%	backbone only
ViT-L/14 distilled	300 M	:white_check_mark:	83.8%	86.7%	backbone only
ViT-g/14	1,100 M	:x:	83.5%	86.5%	backbone only
ViT-g/14	1,100 M	:white_check_mark:	83.7%	87.1%	backbone only

Pretrained backbones (via PyTorch Hub)

Please follow the instructions here to install PyTorch (the only required dependency for loading the model). Installing PyTorch with CUDA support is strongly recommended.

A corresponding model card is included in the repository.

python

import torch

# DINOv2
dinov2_vits14 = torch.hub.load('facebookresearch/dinov2', 'dinov2_vits14')
dinov2_vitb14 = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitb14')
dinov2_vitl14 = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitl14')
dinov2_vitg14 = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitg14')

# DINOv2 with registers
dinov2_vits14_reg = torch.hub.load('facebookresearch/dinov2', 'dinov2_vits14_reg')
dinov2_vitb14_reg = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitb14_reg')
dinov2_vitl14_reg = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitl14_reg')
dinov2_vitg14_reg = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitg14_reg')

Pretrained backbone: XRay-DINO

Request for downloading the model is here:

https://ai.meta.com/resources/models-and-libraries/raydino-downloads/

After filling the form, you will get an email with a temporary link. You can either download it using wget and indicate the checkpoint path in your local filesystem, or you can directly use the URL from the email in the following code:

python

import torch

REPO_DIR = <PATH/TO/A/LOCAL/DIRECTORY/WHERE/THE/DINOV2/REPO/WAS/CLONED>

xray_dino_vitl16 = torch.hub.load(REPO_DIR, 'xray_dino_vitl16', source='local', weights=<CHECKPOINT/URL/OR/PATH>)

License
Model weights are released under the FAIR Noncommercial Research License. See LICENSE_XRAY_DINO_MODEL for additional details.

Pretrained heads - Image classification

backbone	withregisters	download
ImageNet
ViT-S/14 distilled	:x:	linear head (1 layer,

text

[4 layers](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_linear4_head.pth)) |

The (full) classifier models can be loaded via PyTorch Hub:

python

import torch

# DINOv2
dinov2_vits14_lc = torch.hub.load('facebookresearch/dinov2', 'dinov2_vits14_lc')
dinov2_vitb14_lc = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitb14_lc')
dinov2_vitl14_lc = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitl14_lc')
dinov2_vitg14_lc = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitg14_lc')

# DINOv2 with registers
dinov2_vits14_reg_lc = torch.hub.load('facebookresearch/dinov2', 'dinov2_vits14_reg_lc')
dinov2_vitb14_reg_lc = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitb14_reg_lc')
dinov2_vitl14_reg_lc = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitl14_reg_lc')
dinov2_vitg14_reg_lc = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitg14_reg_lc')

Pretrained heads - Depth estimation

backbone	download head
NYUd	KITTI
ViT-S/14 distilled	linear (1 layer,

text

[4 layers](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_nyu_linear4_head.pth)),
    [DPT](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_nyu_dpt_head.pth) | linear ([1 layer](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_kitti_linear_head.pth),
    [4 layers](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_kitti_linear4_head.pth)),
    [DPT](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_kitti_dpt_head.pth) |

Pretrained heads - Semantic segmentation

backbone	download model	download head
ADE20K	ADE20K	VOC2012
ViT-S/14 distilled		linear,

text

[multi-scale](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_ade20k_ms_head.pth) | [linear](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_voc2012_linear_head.pth),
    [multi-scale](https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_voc2012_ms_head.pth) |

Pretrained heads - Zero-shot tasks with dino.txt

backbone	withregisters	download
ViT-L/14 distilled	:white_check_mark:	vision head,

text

[text model](https://dl.fbaipublicfiles.com/dinov2/dinov2_vitl14/dinov2_vitl14_reg4_dinotxt_tet1280d20h24l_text_encoder.pth),
    [vocabulary](https://dl.fbaipublicfiles.com/dinov2/thirdparty/bpe_simple_vocab_16e6.txt.gz),
    [vocabulary license](https://dl.fbaipublicfiles.com/dinov2/thirdparty/LICENSE) |

The (full) dino.txt model can be loaded via PyTorch Hub:

python

import torch

# DINOv2
dinov2_vitl14_reg4_dinotxt_tet1280d20h24l = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitl14_reg4_dinotxt_tet1280d20h24l')

Installation

The training and evaluation code requires PyTorch 2.0 and xFormers 0.0.18 as well as a number of other 3rd party packages. Note that the code has only been tested with the specified versions and also expects a Linux environment. To setup all the required dependencies for training and evaluation, please follow the instructions below:

conda (Recommended) - Clone the repository and then create and activate a dinov2 conda environment using the provided environment definition:

shell

conda env create -f conda.yaml
conda activate dinov2

pip - Clone the repository and then use the provided requirements.txt to install the dependencies:

shell

pip install -r requirements.txt

For dense tasks (depth estimation and semantic segmentation), there are additional dependencies (specific versions of mmcv and mmsegmentation) which are captured in the extras dependency specifications:

conda (Recommended):

shell

conda env create -f conda-extras.yaml
conda activate dinov2-extras

pip:

shell

pip install -r requirements.txt -r requirements-extras.txt

Data preparation

ImageNet-1k

The root directory of the dataset should hold the following contents:

<ROOT>/test/ILSVRC2012_test_00000001.JPEG
<ROOT>/test/[..]
<ROOT>/test/ILSVRC2012_test_00100000.JPEG
<ROOT>/train/n01440764/n01440764_10026.JPEG
<ROOT>/train/[...]
<ROOT>/train/n15075141/n15075141_9993.JPEG
<ROOT>/val/n01440764/ILSVRC2012_val_00000293.JPEG
<ROOT>/val/[...]
<ROOT>/val/n15075141/ILSVRC2012_val_00049174.JPEG
<ROOT>/labels.txt

The provided dataset implementation expects a few additional metadata files to be present under the extra directory:

<EXTRA>/class-ids-TRAIN.npy
<EXTRA>/class-ids-VAL.npy
<EXTRA>/class-names-TRAIN.npy
<EXTRA>/class-names-VAL.npy
<EXTRA>/entries-TEST.npy
<EXTRA>/entries-TRAIN.npy
<EXTRA>/entries-VAL.npy

These metadata files can be generated (once) with the following lines of Python code:

python

from dinov2.data.datasets import ImageNet

for split in ImageNet.Split:
    dataset = ImageNet(split=split, root="<ROOT>", extra="<EXTRA>")
    dataset.dump_extra()

Note that the root and extra directories do not have to be distinct directories.

ImageNet-22k

Please adapt the dataset class to match your local setup.

:warning: To execute the commands provided in the next sections for training and evaluation, the dinov2 package should be included in the Python module search path, i.e. simply prefix the command to run with PYTHONPATH=..

Training

Fast setup: training DINOv2 ViT-L/16 on ImageNet-1k

Run DINOv2 training on 4 A100-80GB nodes (32 GPUs) in a SLURM cluster environment with submitit:

shell

python dinov2/run/train/train.py \
    --nodes 4 \
    --config-file dinov2/configs/train/vitl16_short.yaml \
    --output-dir <PATH/TO/OUTPUT/DIR> \
    train.dataset_path=ImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>

Training time is approximately 1 day and the resulting checkpoint should reach 81.6% on k-NN eval and 82.9% on linear eval.

The training code saves the weights of the teacher in the eval folder every 12500 iterations for evaluation.

Long setup: training DINOv2 ViT-L/14 on ImageNet-22k

Run DINOv2 training on 12 A100-80GB nodes (96 GPUs) in a SLURM cluster environment with submitit:

shell

python dinov2/run/train/train.py \
    --nodes 12 \
    --config-file dinov2/configs/train/vitl14.yaml \
    --output-dir <PATH/TO/OUTPUT/DIR> \
    train.dataset_path=ImageNet22k:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>

Training time is approximately 3.3 days and the resulting checkpoint should reach 82.0% on k-NN eval and 84.5% on linear eval.

The training code saves the weights of the teacher in the eval folder every 12500 iterations for evaluation.

Evaluation

The training code regularly saves the teacher weights. In order to evaluate the model, run the following evaluation on a single node:

k-NN classification on ImageNet-1k

shell

python dinov2/run/eval/knn.py \
    --config-file <PATH/TO/OUTPUT/DIR>/config.yaml \
    --pretrained-weights <PATH/TO/OUTPUT/DIR>/eval/training_24999/teacher_checkpoint.pth \
    --output-dir <PATH/TO/OUTPUT/DIR>/eval/training_24999/knn \
    --train-dataset ImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET> \
    --val-dataset ImageNet:split=VAL:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>

Logistic regression classification on ImageNet-1k

shell

python dinov2/run/eval/log_regression.py \
    --config-file <PATH/TO/OUTPUT/DIR>/config.yaml \
    --pretrained-weights <PATH/TO/OUTPUT/DIR>/eval/training_24999/teacher_checkpoint.pth \
    --output-dir <PATH/TO/OUTPUT/DIR>/eval/training_24999/logreg \
    --train-dataset ImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET> \
    --val-dataset ImageNet:split=VAL:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>

Linear classification with data augmentation on ImageNet-1k

shell

python dinov2/run/eval/linear.py \
    --config-file <PATH/TO/OUTPUT/DIR>/config.yaml \
    --pretrained-weights <PATH/TO/OUTPUT/DIR>/eval/training_24999/teacher_checkpoint.pth \
    --output-dir <PATH/TO/OUTPUT/DIR>/eval/training_24999/linear \
    --train-dataset ImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET> \
    --val-dataset ImageNet:split=VAL:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>

We release the weights from evaluating the different models:

model	withregisters	ImageNettop-1	linear evaluation
ViT-S/14 distilled	:x:	81.1%	linear head weights
ViT-S/14 distilled	:white_check_mark:	80.8%	linear head weights
ViT-B/14 distilled	:x:	84.5%	linear head weights
ViT-B/14 distilled	:white_check_mark:	84.4%	linear head weights
ViT-L/14 distilled	:x:	86.3%	linear head weights
ViT-L/14 distilled	:white_check_mark:	86.5%	linear head weights
ViT-g/14	:x:	86.5%	linear head weights
ViT-g/14	:white_check_mark:	87.0%	linear head weights

The performance of the provided pretrained model weights can be evaluated as follows on ImageNet-1k:

shell

python dinov2/run/eval/linear.py \
    --config-file dinov2/configs/eval/vitg14_pretrain.yaml \
    --pretrained-weights https://dl.fbaipublicfiles.com/dinov2/dinov2_vitg14/dinov2_vitg14_pretrain.pth \
    --train-dataset ImageNet:split=TRAIN:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET> \
    --val-dataset ImageNet:split=VAL:root=<PATH/TO/DATASET>:extra=<PATH/TO/DATASET>

Notebooks

A few notebooks are provided to help the community leverage the models and code:

[Depth estimation](https://github.com/facebookresearch/dinov2/blob/main/notebooks/depth_estimation.ipynb) - How to load and use the depth heads in combination with a matching backbone via mmcv

[Semantic segmentation](https://github.com/facebookresearch/dinov2/blob/main/notebooks/semantic_segmentation.ipynb) - How to load and use the segmentation heads in combination with a matching backbone via mmcv, and also how to load and use the Mask2Former-based segmentation model trained on ADE20K

License

DINOv2 code and model weights are released under the Apache License 2.0. See LICENSE for additional details.

Contributing

See contributing and the code of conduct.

Citing DINOv2

If you find this repository useful, please consider giving a star :star: and citation :t-rex::

text

@misc{oquab2023dinov2,
  title={DINOv2: Learning Robust Visual Features without Supervision},
  author={Oquab, Maxime and Darcet, Timothée and Moutakanni, Theo and Vo, Huy V. and Szafraniec, Marc and Khalidov, Vasil and Fernandez, Pierre and Haziza, Daniel and Massa, Francisco and El-Nouby, Alaaeldin and Howes, Russell and Huang, Po-Yao and Xu, Hu and Sharma, Vasu and Li, Shang-Wen and Galuba, Wojciech and Rabbat, Mike and Assran, Mido and Ballas, Nicolas and Synnaeve, Gabriel and Misra, Ishan and Jegou, Herve and Mairal, Julien and Labatut, Patrick and Joulin, Armand and Bojanowski, Piotr},
  journal={arXiv:2304.07193},
  year={2023}
}

text

@misc{darcet2023vitneedreg,
  title={Vision Transformers Need Registers},
  author={Darcet, Timothée and Oquab, Maxime and Mairal, Julien and Bojanowski, Piotr},
  journal={arXiv:2309.16588},
  year={2023}
}

text

@misc{jose2024dinov2meetstextunified,
  title={DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment},
  author={Cijo Jose and Théo Moutakanni and Dahyun Kang and Federico Baldassarre and Timothée Darcet and Hu Xu and Daniel Li and Marc Szafraniec and Michaël Ramamonjisoa and Maxime Oquab and Oriane Siméoni and Huy V. Vo and Patrick Labatut and Piotr Bojanowski},
  journal={arXiv:2412.16334},
  year={2024}
}

DINOv2 for Biology

The contents of the source code contained in the cell_dino folders, including the code and model weights, are intended for research use only. It is not for use in medical procedures, including any diagnostics, treatment, or curative applications. Do not use this model for any clinical purpose or as a substitute for professional medical judgement.

Scaling Channel-Adaptive Self-Supervised Learning (Channel-Adaptive DINO)

[Paper )] [BibTeX]

Alice V. De Lorenci, Seungeun Yi, Théo Moutakanni, Piotr Bojanowski, Camille Couprie, Juan C. Caicedo, Wolfgang M. Pernice,

with special thanks to Elouan Gardes for his contributions to the codebase.

README

Cell-DINO: Self-Supervised Image-based Embeddings for Cell Fluorescent Microscopy (Cell-DINO)

Théo Moutakanni, Camille Couprie, Seungeun Yi, Elouan Gardes, Piotr Bojanowski, Hugo Touvron, Michael Doron, Zitong S. Chen, Nikita Moshkov, Mathilde Caron, Armand Joulin, Wolfgang M. Pernice, Juan C. Caicedo

to appear soon.

README

Pretrained models

ℹ️ Please follow the link provided below to get access to all the model weights: once accepted, an e-mail will be sent with the complete list of URLs pointing to all the available model weights. These URLs can then be used to either:

download the model or adapter weights to a local filesystem and point torch.hub.load() to these local weights via the pretrained_path parameters, or
directly invoke torch.hub.load() to download and load a backbone from its URL via also the pretrained_url parameter.

⚠️ Please use wget instead of a web browser to download the weights.

Download link:
https://ai.meta.com/resources/models-and-libraries/cell-dino-downloads/

python

import torch

REPO_DIR = <PATH/TO/A/LOCAL/DIRECTORY/WHERE/THE/DINOV2/REPO/WAS/CLONED>

# You can either download the URL link first, then load:
cell_dino_vits8 = torch.hub.load(REPO_DIR, 'cell_dino_cp_vits8', source='local', pretrained_path=<CHECKPOINT/PATH>)
# Or directly download the URL while using `torch.hub.load`:
cell_dino_vits8 = torch.hub.load(REPO_DIR, 'cell_dino_cp_vits8', source='local', pretrained_url=<CHECKPOINT/URL>)

# Similarily for other models:
cell_dino_vitl16_hpa_sc = torch.hub.load(REPO_DIR, 'cell_dino_hpa_vitl16', source='local', pretrained_path=<CHECKPOINT/PATH>)
cell_dino_vitl16_hpa_fov = torch.hub.load(REPO_DIR, 'cell_dino_hpa_vitl16', source='local', pretrained_path=<CHECKPOINT/PATH>)
channel_adaptive_dino_vitl16 = torch.hub.load(REPO_DIR, 'channel_adaptive_dino_vitl16', source='local', pretrained_path=<CHECKPOINT/PATH>)
cell_dino_vitl14 = torch.hub.load(REPO_DIR, 'cell_dino_hpa_vitl14', source='local', pretrained_path=<CHECKPOINT/PATH>)

Licenses

Code is released under the CC BY NC License. See LICENSE_CELL_DINO_CODE for additional details.
Model weights are released under the FAIR Noncommercial Research License. See LICENSE_CELL_DINO_CODE_WEIGHTS for additional details.

README CELL DINO (`docs/README_CELL_DINO.md`)

Cell-DINO: Self-Supervised Image-based Embeddings for Cell Fluorescent Microscopy

Théo Moutakanni*, Camille Couprie*, Seungeun Yi*, Elouan Gardes*, Piotr Bojanowski*, Hugo Touvron*, Michael Doron, Zitong S. Chen, Nikita Moshkov, Mathilde Caron, Armand Joulin, Wolfgang M. Pernice, Juan C. Caicedo

[BibTeX]

*Meta AI Research, FAIR

PyTorch implementation and pretrained models for Cell-DINO.

The contents of this repo, including the code and model weights, are intended for research use only. It is not for use in medical procedures, including any diagnostics, treatment, or curative applications. Do not use this model for any clinical purpose or as a substitute for professional medical judgement.

Pretrained models

One model pretrained on HPA single cell, one model pretrained on HPA Field of View, one model pretrained on HPA Field of View at a larger resolution and one model pretrained on the combined cell painting dataset are released, see dowload instructions at the end of README.md, section "DINO for Biology".

Installation

Follow instructions in the DINOv2 README or build the following environment:

shell

conda create -n py310 python=3.10
conda activate py310
pip install -r requirements.txt
pip install -U scikit-learn

Data preparation

The HPA-FoV and HPA single cell (HPAone) datasets are available here
The folder also contains three sample images ("sample_images") that can be dowloaded to test quickly inference in the provided notebook in notebooks/cell_dino

THe required files for HPA single cell (dataloader : HPAone.py) are:

fixed_size_masked_single_cells_HPA
varied_size_masked_single_cells_HPA
varied_size_masked_single_cells_pretrain_20240507.csv
fixed_size_masked_single_cells_evaluation_20240507.csv
fixed_size_masked_single_cells_pretrain_20240507.csv

Training

Fast setup: training DINOv2 ViT-L/16 on HPA single cell dataset

Run CellDINO training on 4 A100-80GB nodes (32 GPUs) in a SLURM cluster environment with submitit:

shell

python dinov2/run/train/train.py \
    --nodes 4 \
    --config-file dinov2/configs/train/cell_dino/vitl16_hpaone.yaml \
    --output-dir <PATH/TO/OUTPUT/DIR> \
    train.dataset_path=HPAone:split=ALL:root=<PATH/TO/DATASET>

Training time is approximately 2 days on 4 A100 GPU nodes and the resulting checkpoint should reach 78.5 F1 accuracy for protein localization with a linear evaluation.

The training code saves the weights of the teacher in the eval folder every 9000 iterations for evaluation.

Evaluation

The training code regularly saves the teacher weights. In order to evaluate the model, run the following evaluation on a single node:

Linear classification with data augmentation on HPAone or HPAFov (replace 'HPAone' by 'HPAFoV'):

shell

PYTHONPATH=.:dinov2/data python dinov2/run/eval/cell_dino/linear.py \
  --config-file dinov2/configs/eval/cell_dino/vitl16_pretrain.yaml \
  --pretrained-weights <PATH/TO/OUTPUT/DIR>/eval/training_44999/teacher_checkpoint.pth \
  --output-dir <PATH/TO/OUTPUT/DIR>/eval/training_44999/linear \
  --train-dataset HPAone:split=TRAIN:mode=PROTEIN_LOCALIZATION:root=<PATH/TO/DATASET> \
  --val-dataset HPAone:split=VAL:mode=PROTEIN_LOCALIZATION:root<PATH/TO/DATASET> \
  --val-metric-type  mean_per_class_multilabel_f1 \
  --loss-type binary_cross_entropy \
  --avgpool \

We release the weights from evaluating the different models:

The performance of the provided pretrained model weights can be evaluated as follows on HPAone and HPAFoV (replace 'HPAone' by 'HPAFoV', and make sure to load the correct model weights) for the protein localization task:

shell

PYTHONPATH=.:dinov2/data python dinov2/run/eval/linear_celldino.py \
  --config-file dinov2/configs/eval/cell_dino/vitl16_pretrain.yaml \
  --pretrained-weights <CHECKPOINT/PATH> \
  --output-dir <PATH/TO/OUTPUT/DIR> \
  --train-dataset HPAone:split=TRAIN:mode=PROTEIN_LOCALIZATION:root=<PATH/TO/DATASET> \
  --val-dataset HPAone:split=VAL:mode=PROTEIN_LOCALIZATION:root=root<PATH/TO/DATASET> \
  --val-metric-type  mean_per_class_multilabel_f1 \
  --loss-type binary_cross_entropy \
  --avgpool \

and

shell

PYTHONPATH=.:dinov2/data python dinov2/run/eval/cell_dino/linear.py \
  --config-file dinov2/configs/eval/cell_dino/vitl16_pretrain.yaml \
  --pretrained-weights <CHECKPOINT/PATH> \
  --output-dir <PATH/TO/OUTPUT/DIR> \
  --train-dataset HPAone:split=TRAIN:mode=CELL_TYPE:root=<PATH/TO/DATASET> \
  --val-dataset HPAone:split=VAL:mode=CELL_TYPE:root=root<PATH/TO/DATASET> \
  --val-metric-type  mean_per_class_multiclass_f1 \
  --avgpool \

for the cell line classification task.

knn evaluation on HPAone:

shell

PYTHONPATH=.:dinov2/data python dinov2/run/eval/cell_dino/knn.py \
--config-file dinov2/configs/eval/cell_dino/vitl16_pretrain.yaml \
--pretrained-weights <CHECKPOINT/PATH>\
--output-dir <PATH/TO/OUTPUT/DIR>\
--train-dataset HPAone:split=TRAIN:mode=CELL_TYPE:root=<PATH/TO/DATASET> \
--val-dataset HPAone:split=VAL:mode=CELL_TYPE:root=<PATH/TO/DATASET> \
--metric-type mean_per_class_multiclass_f1 \
--crop-size 384 \
--batch-size 256 \
--resize-size 0 \
--nb_knn 1 \

for the cell line classification task.
For the knn evaluation on HPAFoV, replace 'HPAone' by 'HPAFoV' in the command above.
For protein localization, don't forget to change the metric type to mean_per_class_multilabel_f1.

License

Cell-DINO code is released under the CC by NC licence See LICENSE_CELL_DINO_CODE for additional details.
Model weights will be released under the FAIR Non-Commercial Research License. See LICENSE_CELL_DINO_MODELS for additional details.

Contributing

See contributing and the code of conduct.

Citing Cell-DINO

If you find this repository useful, please consider giving a star :star: and citation :t-rex::

text

@misc{,
  title={Cell-DINO: Self-Supervised Image-based Embeddings for Cell Fluorescent Microscopy},
  author={Moutakanni, Th\'eo and Couprie, Camille and Yi, Seungeun and Gardes, Elouan Gardes and Bojanowski, Piotr and Touvron, Hugo and Doron, Michael and Chen, Zitong S. and Moshkov, Nikita and Caron, Mathilde and Joulin, Armand and Pernice, Wolfgang M. and Caicedo, Juan C.},
  journal={in review to PloS One on Computational Biology},
  year={2025}
}

README CHANNEL ADAPTIVE DINO (`docs/README_CHANNEL_ADAPTIVE_DINO.md`)

Scaling Channel-Adaptive Self-Supervised Learning

[Paper )] [BibTeX]

Meta AI Research, FAIR

Alice V. De Lorenci, Seungeun Yi, Théo Moutakanni, Piotr Bojanowski, Camille Couprie, Juan C. Caicedo, Wolfgang M. Pernice,

with special thanks to Elouan Gardes for his contributions to the codebase.

PyTorch implementation and pretrained model for ChannelAdaptive-DINO.

Pretrained model

You can download the model weights trained on the Extended CHAMMI dataset (combination of five cell microscopy datasets with variable numbers of channels) on torchhub.

Installation

Follow instructions in the DINOv2 README. There are two additionnal dependencies to pandas and tifffile.

What is included / not included

This repository includes the Bag of Channel implementation, not the Hierarchical attention approach.

Data preparation

The CHAMMI dataset is available here.

The HPA-FoV dataset is available here

Content: a directory new_512_whole_images and two csv files:

"whole_images_512_test.csv"

"whole_images_512_train.csv"

Training

Fast setup: training Channel-Adaptive DINO ViT-L/16 on HPA single cell dataset

Run Channel-Adaptive DINO training on 4 A100-80GB nodes (32 GPUs) in a SLURM cluster environment with submitit:

shell

python dinov2/run/train/train.py \
    --nodes 4 \
    --config-file dinov2/configs/train/cell_dino/vitl16_boc_hpafov.yaml \
    --output-dir <PATH/TO/OUTPUT/DIR> \
    train.dataset_path=HPAFoV:split=TRAIN:root=<PATH/TO/DATASET>:wildcard=SEPARATE_CHANNELS"

Training time is approximately 2 days.
The training code saves the weights of the teacher in the eval folder every 12500 iterations for evaluation.
This example only performs pretraining on the HPA-FoV dataset.

Evaluation

The training code regularly saves the teacher weights. In order to evaluate the model, run the following evaluation on a single node:

Linear Evaluation on HPAFoV

shell

PYTHONPATH=.:dinov2/data python dinov2/run/eval/cell_dino/linear.py \
  --config-file dinov2/configs/eval/cell_dino/vitl16_channel_adaptive_pretrain.yaml \
  --pretrained-weights <PATH/TO/OUTPUT/DIR>/eval/training_359999/teacher_checkpoint.pth \
  --output-dir <PATH/TO/OUTPUT/DIR>/eval/training_359999/linear \
  --train-dataset HPAFoV:split=TRAIN:mode=PROTEIN_LOCALIZATION:root=<PATH/TO/DATASET> \
  --val-dataset HPAFoV:split=VAL:mode=PROTEIN_LOCALIZATION:root=<PATH/TO/DATASET> \
  --val-metric-type mean_per_class_multilabel_f1 \
  --loss-type binary_cross_entropy \
  --bag-of-channels \
  --crop-size 384 \
  --n-last-blocks 4 \
  --batch-size 32 \
  --epoch-length 145 \
  --epochs 30 \
  --avgpool \

KNN classification on CHAMMI

Go to the docs directory, modifify some paths in launcher_knn_eval_on_chammi.sh and run

shell

./launcher_knn_eval_on_chammi.sh WTC TASK_ONE ;
./launcher_knn_eval_on_chammi.sh WTC TASK_TWO ;
./launcher_knn_eval_on_chammi.sh HPA TASK_ONE ;
./launcher_knn_eval_on_chammi.sh HPA TASK_TWO ;
./launcher_knn_eval_on_chammi.sh HPA TASK_THREE ;
./launcher_knn_eval_on_chammi.sh CP TASK_ONE ;
./launcher_knn_eval_on_chammi.sh CP TASK_TWO ;
./launcher_knn_eval_on_chammi.sh CP TASK_THREE ;
./launcher_knn_eval_on_chammi.sh CP TASK_FOUR ;

Linear classification on CHAMMI

Go to the docs directory, modifify some paths in launcher_CHAMMI_eval.sh and run

shell

./launcher_CHAMMI_eval.sh WTC TASK_ONE ;
./launcher_CHAMMI_eval.sh WTC TASK_TWO ;
./launcher_CHAMMI_eval.sh HPA TASK_ONE ;
./launcher_CHAMMI_eval.sh HPA TASK_TWO ;
./launcher_CHAMMI_eval.sh HPA TASK_THREE ;
./launcher_CHAMMI_eval.sh CP TASK_ONE ;
./launcher_CHAMMI_eval.sh CP TASK_TWO ;
./launcher_CHAMMI_eval.sh CP TASK_THREE ;
./launcher_CHAMMI_eval.sh CP TASK_FOUR ;

	WTC - Task 1	WTC - Task 2	HPA - Task 1	HPA - Task 2	HPA - Task 3	CP - Task 1	CP - Task 2	CP - Task 3	CP - Task 4
knn reproduced	80.3	79.3	91.6	61.4	29.0	89.8	57.6	23.4	18.4
knn paper	79.4	79.0	86.6	59.3	29.6	92.6	57.6	22.1	18.5
Linear reproduced	89.9	87.9	92.7	87.2	66.2	89.9	59.8	26.6	32.5
Linear paper	90.5	89.2	88.3	84.7	65.0	90.5	60.5	25.8	32.7

License

Contributing

See contributing and the code of conduct.

Citing ChannelAdaptiveDINO and DINOv2

If you find this repository useful, please consider giving a star :star: and citation :t-rex::

text

@misc{Delorenci2025scaling,
  title={Scaling Channel-Adaptive Self-Supervised Learning},
  author={V. De Lorenci, Alice and Yi, Seungeun and Moutakanni, Theo and Bojanowski, Piotr and Couprie, Camille and Caicedo, Juan C. and  Pernice, Wolfgang M.},
  journal={TMLR},
  year={2025}
}

text

@misc{oquab2023dinov2,
  title={DINOv2: Learning Robust Visual Features without Supervision},
  author={Oquab, Maxime and Darcet, Timothée and Moutakanni, Theo and Vo, Huy V. and Szafraniec, Marc and Khalidov, Vasil and Fernandez, Pierre and Haziza, Daniel and Massa, Francisco and El-Nouby, Alaaeldin and Howes, Russell and Huang, Po-Yao and Xu, Hu and Sharma, Vasu and Li, Shang-Wen and Galuba, Wojciech and Rabbat, Mike and Assran, Mido and Ballas, Nicolas and Synnaeve, Gabriel and Misra, Ishan and Jegou, Herve and Mairal, Julien and Labatut, Patrick and Joulin, Armand and Bojanowski, Piotr},
  journal={arXiv:2304.07193},
  year={2023}
}

--- METRICS ---

Files Extracted: 3
Estimated Token Budget: ~11571 tokens
Canonical Reference: https://codewiki.google/github.com/facebookresearch/dinov2

Technical Documentation: facebookresearch/dinov2

Overview (`README.md`) Copy Section

DINOv2: Learning Robust Visual Features without Supervision

Pretrained models Copy Chapter

Pretrained backbones (via PyTorch Hub) Copy Section

Pretrained backbone: XRay-DINO Copy Section

Pretrained heads - Image classification Copy Section

Pretrained heads - Depth estimation Copy Section

Pretrained heads - Semantic segmentation Copy Section

Pretrained heads - Zero-shot tasks with dino.txt Copy Section

Installation Copy Chapter

Data preparation Copy Chapter

ImageNet-1k Copy Section

ImageNet-22k Copy Section

Training Copy Chapter

Fast setup: training DINOv2 ViT-L/16 on ImageNet-1k Copy Section

Long setup: training DINOv2 ViT-L/14 on ImageNet-22k Copy Section

Evaluation Copy Chapter

k-NN classification on ImageNet-1k Copy Section

Logistic regression classification on ImageNet-1k Copy Section

Linear classification with data augmentation on ImageNet-1k Copy Section

Notebooks Copy Chapter

License Copy Chapter

Contributing Copy Chapter

Citing DINOv2 Copy Chapter

DINOv2 for Biology

Scaling Channel-Adaptive Self-Supervised Learning (Channel-Adaptive DINO) Copy Chapter

Cell-DINO: Self-Supervised Image-based Embeddings for Cell Fluorescent Microscopy (Cell-DINO) Copy Chapter

Pretrained models Copy Chapter

Licenses Copy Chapter

README CELL DINO (`docs/README_CELL_DINO.md`) Copy Section

Cell-DINO: Self-Supervised Image-based Embeddings for Cell Fluorescent Microscopy

Pretrained models Copy Chapter

Installation Copy Chapter

Data preparation Copy Chapter

Training Copy Chapter

Fast setup: training DINOv2 ViT-L/16 on HPA single cell dataset Copy Section

Evaluation Copy Chapter

Linear classification with data augmentation on HPAone or HPAFov (replace 'HPAone' by 'HPAFoV'): Copy Section

knn evaluation on HPAone: Copy Section

License Copy Chapter

Contributing Copy Chapter

Citing Cell-DINO Copy Chapter

README CHANNEL ADAPTIVE DINO (`docs/README_CHANNEL_ADAPTIVE_DINO.md`) Copy Section

Scaling Channel-Adaptive Self-Supervised Learning

Pretrained model Copy Chapter

Installation Copy Chapter

What is included / not included Copy Chapter

Data preparation Copy Chapter

Training Copy Chapter

Fast setup: training Channel-Adaptive DINO ViT-L/16 on HPA single cell dataset Copy Section

Evaluation Copy Chapter

Linear Evaluation on HPAFoV Copy Section

KNN classification on CHAMMI Copy Section

Linear classification on CHAMMI Copy Section

License Copy Chapter

Contributing Copy Chapter

Citing ChannelAdaptiveDINO and DINOv2 Copy Chapter

Explore Related Technologies & Topics

Overview (`README.md`)

Pretrained models

Pretrained backbones (via PyTorch Hub)

Pretrained backbone: XRay-DINO

Pretrained heads - Image classification

Pretrained heads - Depth estimation

Pretrained heads - Semantic segmentation

Pretrained heads - Zero-shot tasks with dino.txt

Installation

Data preparation

ImageNet-1k

ImageNet-22k

Training

Fast setup: training DINOv2 ViT-L/16 on ImageNet-1k

Long setup: training DINOv2 ViT-L/14 on ImageNet-22k

Evaluation

k-NN classification on ImageNet-1k

Logistic regression classification on ImageNet-1k

Linear classification with data augmentation on ImageNet-1k

Notebooks

License

Contributing

Citing DINOv2

Scaling Channel-Adaptive Self-Supervised Learning (Channel-Adaptive DINO)

Cell-DINO: Self-Supervised Image-based Embeddings for Cell Fluorescent Microscopy (Cell-DINO)

Pretrained models

Licenses

README CELL DINO (`docs/README_CELL_DINO.md`)

Pretrained models

Installation

Data preparation

Training

Fast setup: training DINOv2 ViT-L/16 on HPA single cell dataset

Evaluation

Linear classification with data augmentation on HPAone or HPAFov (replace 'HPAone' by 'HPAFoV'):

knn evaluation on HPAone:

License

Contributing

Citing Cell-DINO

README CHANNEL ADAPTIVE DINO (`docs/README_CHANNEL_ADAPTIVE_DINO.md`)

Pretrained model

Installation

What is included / not included

Data preparation

Training

Fast setup: training Channel-Adaptive DINO ViT-L/16 on HPA single cell dataset

Evaluation

Linear Evaluation on HPAFoV

KNN classification on CHAMMI

Linear classification on CHAMMI

License

Contributing

Citing ChannelAdaptiveDINO and DINOv2