detectron2 Documentation

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.

34,625 stars Python Python 1 file · ~3,135 tokens
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy detectron2 on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run) 
docker run -d --name detectron2 --restart always facebookresearch/detectron2:latest
Configured Services & Ports
⚙️ detectron2
image: build locally
docker-compose.yml
yaml (docker-compose.yml) 
version: "2.3"
services:
  detectron2:
    build:
      context: .
      dockerfile: Dockerfile
      args:
        USER_ID: ${USER_ID:-1000}
    deploy:
      resources:
        reservations:
          devices:
            - capabilities:
              - gpu
    shm_size: "8gb"
    ulimits:
      memlock: -1
      stack: 67108864
    volumes:
      - /tmp/.X11-unix:/tmp/.X11-unix:ro
    environment:
      - DISPLAY=$DISPLAY
      - NVIDIA_VISIBLE_DEVICES=all
    # Uncomment with proper source to access webcam from docker
    # devices: 
    #   - /dev/video0:/dev/video0

File: README.md

Detectron2 is Facebook AI Research's next generation library
that provides state-of-the-art detection and segmentation algorithms.
It is the successor of
Detectron
and maskrcnn-benchmark.
It supports a number of computer vision research projects and production applications in Facebook.

Learn More about Detectron2

  • Includes new capabilities such as panoptic segmentation, Densepose, Cascade R-CNN, rotated bounding boxes, PointRend,
    DeepLab, ViTDet, MViTv2 etc.
  • Used as a library to support building research projects on top of it.
  • Models can be exported to TorchScript format or Caffe2 format for deployment.
  • It trains much faster.

See our blog post
to see more demos.
See this interview to learn more about the stories behind detectron2.

Installation

See installation instructions.

Getting Started

See Getting Started with Detectron2,
and the Colab Notebook
to learn about basic usage.

Learn more at our documentation.
And see projects/ for some projects that are built on top of detectron2.

Model Zoo and Baselines

We provide a large set of baseline results and trained models available for download in the Detectron2 Model Zoo.

License

Detectron2 is released under the Apache 2.0 license.

Citing Detectron2

If you use Detectron2 in your research or wish to refer to the baseline results published in the Model Zoo, please use the following BibTeX entry.

bibtex
@misc{wu2019detectron2,
  author =       {Yuxin Wu and Alexander Kirillov and Francisco Massa and
                  Wan-Yen Lo and Ross Girshick},
  title =        {Detectron2},
  howpublished = {\url{https://github.com/facebookresearch/detectron2}},
  year =         {2019}
}

File: configs/common/README.md

This directory provides definitions for a few common models, dataloaders, scheduler,
and optimizers that are often used in training.
The definition of these objects are provided in the form of lazy instantiation:
their arguments can be edited by users before constructing the objects.

They can be imported, or loaded by model_zoo.get_config API in users' own configs.

File: configs/Detectron1-Comparisons/README.md

Detectron2 model zoo's experimental settings and a few implementation details are different from Detectron.

The differences in implementation details are shared in
Compatibility with Other Libraries.

The differences in model zoo's experimental settings include:

  • Use scale augmentation during training. This improves AP with lower training cost.
  • Use L1 loss instead of smooth L1 loss for simplicity. This sometimes improves box AP but may
    affect other AP.
  • Use POOLER_SAMPLING_RATIO=0 instead of 2. This does not significantly affect AP.
  • Use ROIAlignV2. This does not significantly affect AP.

In this directory, we provide a few configs that do not have the above changes.
They mimic Detectron's behavior as close as possible,
and provide a fair comparison of accuracy and speed against Detectron.

Faster R-CNN 1x 0.219 0.038 3.1 36.9 137781054 model   metrics
Keypoint R-CNN 1x 0.313 0.071 5.0 53.1 64.2 137781195 model   metrics
Mask R-CNN 1x 0.273 0.043 3.4 37.8 34.9 137781281 model   metrics

Comparisons:

  • Faster R-CNN: Detectron's AP is 36.7, similar to ours.
  • Keypoint R-CNN: Detectron's AP is box 53.6, keypoint 64.2. Fixing a Detectron's
    bug lead to a drop in box AP, and can be
    compensated back by some parameter tuning.
  • Mask R-CNN: Detectron's AP is box 37.7, mask 33.9. We're 1 AP better in mask AP, due to more correct implementation.
    See this article for details.

For speed comparison, see benchmarks.

File: configs/quick_schedules/README.md

These are quick configs for performance or accuracy regression tracking purposes.

  • *instance_test.yaml: can train on 2 GPUs. They are used to test whether the training can
    successfully finish. They are not expected to produce reasonable training results.
  • *inference_acc_test.yaml: They should be run using --eval-only. They run inference using pre-trained models and verify
    the results are as expected.
  • *training_acc_test.yaml: They should be trained on 8 GPUs. They finish in about an hour and verify the training accuracy
    is within the normal range.

File: datasets/README.md

Use Builtin Datasets

A dataset can be used by accessing DatasetCatalog
for its data, or MetadataCatalog for its metadata (class names, etc).
This document explains how to setup the builtin datasets so they can be used by the above APIs.
Use Custom Datasets gives a deeper dive on how to use DatasetCatalog and MetadataCatalog,
and how to add new datasets to them.

Detectron2 has builtin support for a few datasets.
The datasets are assumed to exist in a directory specified by the environment variable
DETECTRON2_DATASETS.
Under this directory, detectron2 will look for datasets in the structure described below, if needed.

text
$DETECTRON2_DATASETS/
  coco/
  lvis/
  cityscapes/
  VOC20{07,12}/

You can set the location for builtin datasets by export DETECTRON2_DATASETS=/path/to/datasets.
If left unset, the default is ./datasets relative to your current working directory.

The model zoo
contains configs and models that use these builtin datasets.

Expected dataset structure for [COCO instance/keypoint detection](https://cocodataset.org/#download):

text
coco/
  annotations/
    instances_{train,val}2017.json
    person_keypoints_{train,val}2017.json
  {train,val}2017/
    # image files that are mentioned in the corresponding json

You can use the 2014 version of the dataset as well.

Some of the builtin tests (dev/run_*_tests.sh) uses a tiny version of the COCO dataset,
which you can download with ./datasets/prepare_for_tests.sh.

Expected dataset structure for PanopticFPN:

Extract panoptic annotations from COCO website
into the following structure:

text
coco/
  annotations/
    panoptic_{train,val}2017.json
  panoptic_{train,val}2017/  # png annotations
  panoptic_stuff_{train,val}2017/  # generated by the script mentioned below

Install panopticapi by:

text
pip install git+https://github.com/cocodataset/panopticapi.git

Then, run python datasets/prepare_panoptic_fpn.py, to extract semantic annotations from panoptic annotations.

Expected dataset structure for [LVIS instance segmentation](https://www.lvisdataset.org/dataset):

text
coco/
  {train,val,test}2017/
lvis/
  lvis_v0.5_{train,val}.json
  lvis_v0.5_image_info_test.json
  lvis_v1_{train,val}.json
  lvis_v1_image_info_test{,_challenge}.json

Install lvis-api by:

text
pip install git+https://github.com/lvis-dataset/lvis-api.git

To evaluate models trained on the COCO dataset using LVIS annotations,
run python datasets/prepare_cocofied_lvis.py to prepare "cocofied" LVIS annotations.

Expected dataset structure for [cityscapes](https://www.cityscapes-dataset.com/downloads/):

text
cityscapes/
  gtFine/
    train/
      aachen/
        color.png, instanceIds.png, labelIds.png, polygons.json,
        labelTrainIds.png
      ...
    val/
    test/
    # below are generated Cityscapes panoptic annotation
    cityscapes_panoptic_train.json
    cityscapes_panoptic_train/
    cityscapes_panoptic_val.json
    cityscapes_panoptic_val/
    cityscapes_panoptic_test.json
    cityscapes_panoptic_test/
  leftImg8bit/
    train/
    val/
    test/

Install cityscapes scripts by:

text
pip install git+https://github.com/mcordts/cityscapesScripts.git

Note: to create labelTrainIds.png, first prepare the above structure, then run cityscapesescript with:

text
CITYSCAPES_DATASET=/path/to/abovementioned/cityscapes python cityscapesscripts/preparation/createTrainIdLabelImgs.py

These files are not needed for instance segmentation.

Note: to generate Cityscapes panoptic dataset, run cityscapesescript with:

text
CITYSCAPES_DATASET=/path/to/abovementioned/cityscapes python cityscapesscripts/preparation/createPanopticImgs.py

These files are not needed for semantic and instance segmentation.

Expected dataset structure for [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/index.html):

text
VOC20{07,12}/
  Annotations/
  ImageSets/
    Main/
      trainval.txt
      test.txt
      # train.txt or val.txt, if you use these splits
  JPEGImages/

Expected dataset structure for [ADE20k Scene Parsing](http://sceneparsing.csail.mit.edu/):

text
ADEChallengeData2016/
  annotations/
  annotations_detectron2/
  images/
  objectInfo150.txt

The directory annotations_detectron2 is generated by running python datasets/prepare_ade20k_sem_seg.py.

File: demo/README.md

Detectron2 Demo

We provide a command line tool to run a simple demo of builtin configs.
The usage is explained in GETTING_STARTED.md.

See our blog post
for a high-quality demo generated with this tool.

File: detectron2/data/datasets/README.md

Common Datasets

The dataset implemented here do not need to load the data into the final format.
It should provide the minimal data structure needed to use the dataset, so it can be very efficient.

For example, for an image dataset, just provide the file names and labels, but don't read the images.
Let the downstream decide how to read.

File: detectron2/export/README.md

This directory contains code to prepare a detectron2 model for deployment.
Currently it supports exporting a detectron2 model to TorchScript, ONNX, or (deprecated) Caffe2 format.

Please see documentation for its usage.

Acknowledgements

Thanks to Mobile Vision team at Facebook for developing the Caffe2 conversion tools.

Thanks to Computing Platform Department - PAI team at Alibaba Group (@bddpqq, @chenbohua3) who
help export Detectron2 models to TorchScript.

Thanks to ONNX Converter team at Microsoft who help export Detectron2 models to ONNX.

File: detectron2/layers/csrc/README.md

To add a new Op:

  1. Create a new directory
  2. Implement new ops there
  3. Delcare its Python interface in vision.cpp.

File: detectron2/projects/README.md

Projects live in the projects directory under the root of this repository, but not here.

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →