README (README.md)

This work presents Depth Anything V2. It significantly outperforms V1 in fine-grained details and robustness. Compared with SD-based models, it enjoys faster inference speed, fewer parameters, and higher depth accuracy.

News

2025-01-22: Video Depth Anything has been released. It generates consistent depth maps for super-long videos (e.g., over 5 minutes).
2024-12-22: Prompt Depth Anything has been released. It supports 4K resolution metric depth estimation when low-res LiDAR is used to prompt the DA models.
2024-07-06: Depth Anything V2 is supported in Transformers. See the instructions for convenient usage.
2024-06-25: Depth Anything is integrated into Apple Core ML Models. See the instructions (V1, V2) for usage.
2024-06-22: We release smaller metric depth models based on Depth-Anything-V2-Small and Base.
2024-06-20: Our repository and project page are flagged by GitHub and removed from the public for 6 days. Sorry for the inconvenience.
2024-06-14: Paper, project page, code, models, demo, and benchmark are all released.

Pre-trained Models

We provide four models of varying scales for robust relative depth estimation:

Model	Params	Checkpoint
Depth-Anything-V2-Small

Usage

Prepraration

bash

git clone https://github.com/DepthAnything/Depth-Anything-V2
cd Depth-Anything-V2
pip install -r requirements.txt

Download the checkpoints listed here and put them under the checkpoints directory.

Use our models

python

import cv2
import torch

from depth_anything_v2.dpt import DepthAnythingV2

DEVICE = 'cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu'

model_configs = {
    'vits': {'encoder': 'vits', 'features': 64, 'out_channels': [48, 96, 192, 384]},
    'vitb': {'encoder': 'vitb', 'features': 128, 'out_channels': [96, 192, 384, 768]},
    'vitl': {'encoder': 'vitl', 'features': 256, 'out_channels': [256, 512, 1024, 1024]},
    'vitg': {'encoder': 'vitg', 'features': 384, 'out_channels': [1536, 1536, 1536, 1536]}
}

encoder = 'vitl' # or 'vits', 'vitb', 'vitg'

model = DepthAnythingV2(**model_configs[encoder])
model.load_state_dict(torch.load(f'checkpoints/depth_anything_v2_{encoder}.pth', map_location='cpu'))
model = model.to(DEVICE).eval()

raw_img = cv2.imread('your/image/path')
depth = model.infer_image(raw_img) # HxW raw depth map in numpy

If you do not want to clone this repository, you can also load our models through Transformers. Below is a simple code snippet. Please refer to the official page for more details.

Note 1: Make sure you can connect to Hugging Face and have installed the latest Transformers.
Note 2: Due to the upsampling difference between OpenCV (we used) and Pillow (HF used), predictions may differ slightly. So you are more recommended to use our models through the way introduced above.

python

from transformers import pipeline
from PIL import Image

pipe = pipeline(task="depth-estimation", model="depth-anything/Depth-Anything-V2-Small-hf")
image = Image.open('your/image/path')
depth = pipe(image)["depth"]

Running script on images

bash

python run.py \
  --encoder <vits | vitb | vitl | vitg> \
  --img-path  --outdir <outdir> \
  [--input-size <size>] [--pred-only] [--grayscale]

Options:

--img-path: You can either 1) point it to an image directory storing all interested images, 2) point it to a single image, or 3) point it to a text file storing all image paths.
--input-size (optional): By default, we use input size 518 for model inference. You can increase the size for even more fine-grained results.
--pred-only (optional): Only save the predicted depth map, without raw image.
--grayscale (optional): Save the grayscale depth map, without applying color palette.

For example:

bash

python run.py --encoder vitl --img-path assets/examples --outdir depth_vis

Running script on videos

bash

python run_video.py \
  --encoder <vits | vitb | vitl | vitg> \
  --video-path assets/examples_video --outdir video_depth_vis \
  [--input-size <size>] [--pred-only] [--grayscale]

Our larger model has better temporal consistency on videos.

Gradio demo

To use our gradio demo locally:

bash

python app.py

You can also try our online demo.

Note: Compared to V1, we have made a minor modification to the DINOv2-DPT architecture (originating from this issue). In V1, we unintentionally used features from the last four layers of DINOv2 for decoding. In V2, we use intermediate features instead. Although this modification did not improve details or accuracy, we decided to follow this common practice.

Fine-tuned to Metric Depth Estimation

Please refer to metric depth estimation.

DA-2K Evaluation Benchmark

Please refer to DA-2K benchmark.

Community Support

We sincerely appreciate all the community support for our Depth Anything series. Thank you a lot!

Apple Core ML:

- https://developer.apple.com/machine-learning/models
- https://huggingface.co/apple/coreml-depth-anything-v2-small
- https://huggingface.co/apple/coreml-depth-anything-small

Transformers:

- https://huggingface.co/docs/transformers/main/en/model_doc/depth_anything_v2
- https://huggingface.co/docs/transformers/main/en/model_doc/depth_anything

TensorRT:

- https://github.com/spacewalk01/depth-anything-tensorrt
- https://github.com/zhujiajian98/Depth-Anythingv2-TensorRT-python

ONNX: https://github.com/fabio-sim/Depth-Anything-ONNX
ComfyUI: https://github.com/kijai/ComfyUI-DepthAnythingV2
Transformers.js (real-time depth in web): https://huggingface.co/spaces/Xenova/webgpu-realtime-depth-estimation
Android:

- https://github.com/shubham0204/Depth-Anything-Android
- https://github.com/FeiGeChuanShu/ncnn-android-depth_anything

Acknowledgement

We are sincerely grateful to the awesome Hugging Face team (@Pedro Cuenca, @Niels Rogge, @Merve Noyan, @Amy Roberts, et al.) for their huge efforts in supporting our models in Transformers and Apple Core ML.

We also thank the DINOv2 team for contributing such impressive models to our community.

LICENSE

Depth-Anything-V2-Small model is under the Apache-2.0 license. Depth-Anything-V2-Base/Large/Giant models are under the CC-BY-NC-4.0 license.

Citation

If you find this project useful, please consider citing:

bibtex

@article{depth_anything_v2,
  title={Depth Anything V2},
  author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  journal={arXiv:2406.09414},
  year={2024}
}

@inproceedings{depth_anything_v1,
  title={Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data}, 
  author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  booktitle={CVPR},
  year={2024}
}

DA 2K (DA-2K.md)

DA-2K Evaluation Benchmark

Introduction

DA-2K is proposed in Depth Anything V2 to evaluate the relative depth estimation capability. It encompasses eight representative scenarios of indoor, outdoor, non_real, transparent_reflective, adverse_style, aerial, underwater, and object. It consists of 1K diverse high-quality images and 2K precise pair-wise relative depth annotations.

Please refer to our paper for details in constructing this benchmark.

Usage

Please first download the benchmark.

All annotations are stored in annotations.json. The annotation file is a JSON object where each key is the path to an image file, and the value is a list of annotations associated with that image. Each annotation describes two points and identifies which point is closer to the camera. The structure is detailed below:

text

{
  "image_path": [
    {
      "point1": [h1, w1], # (vertical position, horizontal position)
      "point2": [h2, w2], # (vertical position, horizontal position)
      "closer_point": "point1" # we always set "point1" as the closer one
    },
    ...
  ],
  ...
}

To visualize the annotations:

bash

python visualize.py [--scene-type <type>]

Options

--scene-type <type> (optional): Specify the scene type (indoor, outdoor, non_real, transparent_reflective, adverse_style, aerial, underwater, and object). Skip this argument or set <type> as "" to include all scene types.

Citation

If you find this benchmark useful, please consider citing:

bibtex

@article{depth_anything_v2,
  title={Depth Anything V2},
  author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  journal={arXiv:2406.09414},
  year={2024}
}

Requirements (requirements.txt)

gradio_imageslider
gradio
matplotlib
opencv-python
torch
torchvision

Metric Depth/README (metric_depth/README.md)

Depth Anything V2 for Metric Depth Estimation

We here provide a simple codebase to fine-tune our Depth Anything V2 pre-trained encoder for metric depth estimation. Built on our powerful encoder, we use a simple DPT head to regress the depth. We fine-tune our pre-trained encoder on synthetic Hypersim / Virtual KITTI datasets for indoor / outdoor metric depth estimation, respectively.

Pre-trained Models

We provide six metric depth models of three scales for indoor and outdoor scenes, respectively.

Base Model	Params	Indoor (Hypersim)	Outdoor (Virtual KITTI 2)
Depth-Anything-V2-Small

We recommend to first try our larger models (if computational cost is affordable) and the indoor version.

Usage

Prepraration

bash

git clone https://github.com/DepthAnything/Depth-Anything-V2
cd Depth-Anything-V2/metric_depth
pip install -r requirements.txt

Download the checkpoints listed here and put them under the checkpoints directory.

Use our models

python

import cv2
import torch

from depth_anything_v2.dpt import DepthAnythingV2

model_configs = {
    'vits': {'encoder': 'vits', 'features': 64, 'out_channels': [48, 96, 192, 384]},
    'vitb': {'encoder': 'vitb', 'features': 128, 'out_channels': [96, 192, 384, 768]},
    'vitl': {'encoder': 'vitl', 'features': 256, 'out_channels': [256, 512, 1024, 1024]}
}

encoder = 'vitl' # or 'vits', 'vitb'
dataset = 'hypersim' # 'hypersim' for indoor model, 'vkitti' for outdoor model
max_depth = 20 # 20 for indoor model, 80 for outdoor model

model = DepthAnythingV2(**{**model_configs[encoder], 'max_depth': max_depth})
model.load_state_dict(torch.load(f'checkpoints/depth_anything_v2_metric_{dataset}_{encoder}.pth', map_location='cpu'))
model.eval()

raw_img = cv2.imread('your/image/path')
depth = model.infer_image(raw_img) # HxW depth map in meters in numpy

Running script on images

Here, we take the vitl encoder as an example. You can also use vitb or vits encoders.

bash

# indoor scenes
python run.py \
  --encoder vitl \
  --load-from checkpoints/depth_anything_v2_metric_hypersim_vitl.pth \
  --max-depth 20 \
  --img-path  --outdir <outdir> [--input-size <size>] [--save-numpy]

# outdoor scenes
python run.py \
  --encoder vitl \
  --load-from checkpoints/depth_anything_v2_metric_vkitti_vitl.pth \
  --max-depth 80 \
  --img-path  --outdir <outdir> [--input-size <size>] [--save-numpy]

Project 2D images to point clouds:

bash

python depth_to_pointcloud.py \
  --encoder vitl \
  --load-from checkpoints/depth_anything_v2_metric_hypersim_vitl.pth \
  --max-depth 20 \
  --img-path  --outdir <outdir>

Reproduce training

Please first prepare the Hypersim and Virtual KITTI 2 datasets. Then:

bash

bash dist_train.sh

Citation

If you find this project useful, please consider citing:

bibtex

@article{depth_anything_v2,
  title={Depth Anything V2},
  author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  journal={arXiv:2406.09414},
  year={2024}
}

@inproceedings{depth_anything_v1,
  title={Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data}, 
  author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  booktitle={CVPR},
  year={2024}
}

README (README.md) Copy Section

News Copy Chapter

Pre-trained Models Copy Chapter

Usage Copy Chapter

Prepraration Copy Section

Use our models Copy Section

Running script on *images* Copy Section

Running script on *videos* Copy Section

Gradio demo Copy Section

Fine-tuned to Metric Depth Estimation Copy Chapter

DA-2K Evaluation Benchmark Copy Chapter

Community Support Copy Chapter

Acknowledgement Copy Chapter

LICENSE Copy Chapter

Citation Copy Chapter

DA 2K (DA-2K.md) Copy Section

DA-2K Evaluation Benchmark

Introduction Copy Chapter

Usage Copy Chapter

Citation Copy Chapter

Requirements (requirements.txt) Copy Section

Metric Depth/README (metric_depth/README.md) Copy Section

Depth Anything V2 for Metric Depth Estimation

Pre-trained Models

Usage Copy Chapter

Prepraration Copy Section

Use our models Copy Section

Running script on images Copy Section

Project 2D images to point clouds: Copy Section

Reproduce training Copy Section

Citation Copy Chapter

Explore Related Technologies & Topics

README (README.md)

News

Pre-trained Models

Usage

Prepraration

Use our models

Running script on images

Running script on videos

Gradio demo

Fine-tuned to Metric Depth Estimation

DA-2K Evaluation Benchmark

Community Support

Acknowledgement

LICENSE

Citation

DA 2K (DA-2K.md)

Introduction

Usage

Citation

Requirements (requirements.txt)

Metric Depth/README (metric_depth/README.md)

Usage

Prepraration

Use our models

Running script on images

Project 2D images to point clouds:

Reproduce training

Citation