PaddleDetection Documentation

Technical Documentation: PaddlePaddle/PaddleDetection

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Contribution README (`docs/contribution/README.md`)

Contributing to PaddleDetection

PaddleDetection非常欢迎你加入到飞桨社区的开源建设中，你可以通过以下方式参与贡献：

新建一个 ISSUE 来反馈 bug
新建一个 ISSUE 来提出新功能需求、建议、疑问
提 PR 来修复一个 bug
提 PR 来实现一个新功能

同时我们也会组织专项活动，引导大家参与到PaddleDetection的开发中：

Yes, PP-YOLOE! 基于PP-YOLOE的算法开发

贡献指南

提ISSUE、PR的步骤请参考飞桨官网-贡献指南-代码贡献流程

开发者

我们非常欢迎你可以为PaddleDetection提供代码，也十分感谢你的反馈。

感谢Mandroide清理代码并且统一部分函数接口。
感谢FL77N贡献Sparse-RCNN模型。
感谢Chen-Song贡献Swin Faster-RCNN模型。
感谢yangyudong, hchhtc123 开发PP-Tracking GUI界面
感谢Shigure19 开发PP-TinyPose健身APP
感谢manangoel99贡献Wandb可视化方式
感谢百度ACG政务产品部统管通办研发组视觉研发团队贡献PP-YOLOE蒸馏方案

非常感谢大家为飞桨贡献！共建飞桨繁荣社区！

Paddlex Overview (`docs/paddlex/overview.md`)

1. 低代码全流程开发简介

飞桨低代码开发工具PaddleX，依托于PaddleDetection的先进技术，支持了目标检测领域的低代码全流程开发能力。通过低代码全流程开发，可实现简单且高效的模型使用、组合与定制。这将显著减少模型开发的时间消耗，降低其开发难度，大大加快模型在行业中的应用和推广速度。特色如下：

🎨 模型丰富一键调用：将通用目标检测、小目标检测和实例分割涉及的55个模型整合为3条模型产线，通过极简的Python API一键调用，快速体验模型效果。此外，同一套API，也支持图像分类、图像分割、文本图像智能分析、通用OCR、时序预测等共计200+模型，形成20+单功能模块，方便开发者进行模型组合使用。
🚀 提高效率降低门槛：提供基于统一命令和图形界面两种方式，实现模型简洁高效的使用、组合与定制。支持高性能部署、服务化部署和端侧部署等多种部署方式。此外，对于各种主流硬件如英伟达GPU、昆仑芯、昇腾、寒武纪和海光等，进行模型开发时，都可以无缝切换。

说明：PaddleX 致力于实现产线级别的模型训练、推理与部署。模型产线是指一系列预定义好的、针对特定AI任务的开发流程，其中包含能够独立完成某类任务的单模型（单功能模块）组合。

2. 目标检测相关能力支持

PaddleX中目标检测领域相关的3条产线均支持本地快速推理，部分产线支持在线体验，您可以快速体验各个产线的预训练模型效果，如果您对产线的预训练模型效果满意，可以直接对产线进行高性能部署/服务化部署/端侧部署，如果不满意，您也可以使用产线的二次开发能力，提升效果。完整的产线开发流程请参考PaddleX产线使用概览或各产线使用教程。

此外，PaddleX为开发者提供了基于云端图形化开发界面的全流程开发工具, 详细请参考教程《零门槛开发产业级AI模型》

	在线体验	快速推理	高性能部署	服务化部署	端侧部署	二次开发	星河零代码产线
通用目标检测	链接	✅	✅	✅	✅	✅	✅
通用实例分割	链接	✅	✅	✅	🚧	✅	✅
小目标检测	🚧	✅	✅	✅	🚧	✅	🚧

❗注：以上功能均基于GPU/CPU实现。PaddleX还可在昆仑、昇腾、寒武纪和海光等主流硬件上进行快速推理和二次开发。下表详细列出了模型产线的支持情况，具体支持的模型列表请参阅模型列表(NPU) // 模型列表(XPU) // 模型列表(MLU) // 模型列表DCU。同时我们也在适配更多的模型，并在主流硬件上推动高性能和服务化部署的实施。

🚀 国产化硬件能力支持

产线名称	昇腾 910B	昆仑 R200/R300	寒武纪 MLU370X8	海光 Z100
通用目标检测	✅	✅	✅	🚧
通用实例分割	✅	🚧	✅	🚧

3. 目标检测相关模型产线列表和教程

通用目标检测产线: 使用教程
通用实例分割产线: 使用教程
小目标检测产线: 使用教程

4. 目标检测相关单功能模块列表和教程

目标检测模块: 使用教程
实例分割模块: 使用教程
小目标检测模块: 使用教程

Advanced Tutorials Openvino Inference README (`docs/advanced_tutorials/openvino_inference/README.md`)

Using OpenVINO for Inference

Introduction

PaddleDetection has been a vibrant open-source project and has a large amout of contributors and maintainers around it. It is an AI framework which enables developers to quickly integrate AI capacities into their own projects and applications.

Intel OpenVINO is a widely used free toolkit. It facilitates the optimization of a deep learning model from a framework and deployment using an inference engine onto Intel hardware.

Apparently, the upstream(Paddle) and the downstream(Intel OpenVINO) can work together to streamline and simplify the process of developing an AI model and deploying the model onto hardware, which, in turn, makes our lives easier.

This article will show you how to use a PaddleDetection model FairMOT from the Model Zoo in PaddleDetection and use it with OpenVINO to do the inference.

Prerequisites

This article is not an entry level introduction to help you set up everything, in order to focus on its main purpose, the instruction of setting up environment will be kept at the minmum level and respective instructions will be provided by their official website links.

Before we can do anything, please make sure you have PaddlePaddle environment set up.

text

conda install paddlepaddle==2.2.2 --channel https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/Paddle/

Please also download the converted ONNX format of FairMOT

Export the PaddleDetection Model to ONNX format

Download the FairMOT Inference Model
Using Paddle2ONNX to convert the model

Make sure you have the Paddle2ONNX installed

text

paddle2onnx --model_dir . --model_filename model.pdmodel \
--params_filename model.pdiparams \
--input_shape_dict "{'image': [1, 3, 320, 576], 'scale_factor': [1, 2], 'im_shape': [1, 2]}" \
--save_file fairmot_576_320_v2.onnx \
--opset_version 12 \
--enable_onnx_checker True

For more details about how to convert Paddle models to ONNX, please see Export ONNX Model.

Use the ONNX model for inference

Once the Paddle model has been converted to ONNX format, we can then use it with OpenVINO inference engine to do the prediction.

_{Please make sure you have the OpenVINO installed, here is the instruction for installation.}

Get the execution network

So the 1st thing to do here is to get an execution network which can be used later to do the inference.

Here is the code.

text

def get_net():
    ie = IECore()
    model_path = root_path / "PaddleDetection/FairMot/fairmot_576_320_v3.onnx"
    net = ie.read_network(model= str(model_path))
    exec_net = ie.load_network(network=net, device_name="CPU")
    return net, exec_net

Preprocessing

Every AI model has its own steps of preprocessing, let's have a look how to do it for the FairMOT model:

text

def prepare_input():
    transforms = [
        T.Resize(target_size=(target_width, target_height)),
        T.Normalize(mean=(0,0,0), std=(1,1,1))
    ]
    img_file = root_path / "images/street.jpeg"
    img = cv2.imread(str(img_file))
    normalized_img, _ = T.Compose(transforms)(img)
    # add an new axis in front
    img_input = normalized_img[np.newaxis, :]
    # scale_factor is calculated as: im_shape / original_im_shape
    h_scale = target_height / img.shape[0]
    w_scale = target_width / img.shape[1]
    input = {"image": img_input, "im_shape": [target_height, target_width], "scale_factor": [h_scale, w_scale]}
    return input, img

Prediction

After we have done all the load network and preprocessing, it finally comes to the stage of prediction.

text

def predict(exec_net, input):
    result = exec_net.infer(input)
    return result

You might be surprised to see the very exciting stage this small. Hang on there, the next stage is actually big again.

Post-processing

MOT(Multi-Object Tracking) is special, not like other AI models which require a few steps of post-processing. Instead, FairMOT requires a special object called tracker, to handle the prediction results. The prediction results are prediction detections and prediction embeddings.

Luckily, PaddleDetection has made this procesure easy for us, it has exported the JDETracker from ppdet, so that we do not need to write much code to handle it.

text

def postprocess(pred_dets, pred_embs, threshold = 0.5):
    tracker = JDETracker()
    online_targets_dict = tracker.update(pred_dets, pred_embs)
    online_tlwhs = defaultdict(list)
    online_scores = defaultdict(list)
    online_ids = defaultdict(list)
    for cls_id in range(1):
        online_targets = online_targets_dict[cls_id]
        for t in online_targets:
            tlwh = t.tlwh
            tid = t.track_id
            tscore = t.score
            # make sure the tscore is no less then the threshold.
            if tscore < threshold: continue
            # make sure the target area is not less than the min_box_area.
            if tlwh[2] * tlwh[3] <= tracker.min_box_area:
                continue
            # make sure the vertical ratio of a found target is within the range (1.6 as default ratio).
            if tracker.vertical_ratio > 0 and tlwh[2] / tlwh[3] > tracker.vertical_ratio:
                continue
            online_tlwhs[cls_id].append(tlwh)
            online_ids[cls_id].append(tid)
            online_scores[cls_id].append(tscore)
    online_im = plot_tracking_dict(
        img,
        1,
        online_tlwhs,
        online_ids,
        online_scores,
        frame_id=0)
    return online_im

Plot the detections (Optional)

This step is optional. For demo purpose, I just use plot_tracking_dict() method to draw all boundary boxes on the image. But you do not need to do this if you don't have the same requirement.

text

online_im = plot_tracking_dict(
    img,
    1,
    online_tlwhs,
    online_ids,
    online_scores,
    frame_id=0)

So these are the all steps which you need to follow in order to run FairMOT on your machine.

A companion article which explains in details of this procedure will be released soon and a link to that article will be updated here soon.

To see the full code, please take a look at Paddle OpenVINO Prediction.

Tutorials FAQ README (`docs/tutorials/FAQ/README.md`)

FAQ/常见问题

PaddleDetection非常感谢各位开发者提出任何使用问题或需求，我们根据大家的提问，总结FAQ/常见问题合集，并在每周一进行更新，以下是往期的FAQ，欢迎大家进行查阅。

Tutorials Data README (`docs/tutorials/data/README.md`)

数据准备

数据对于深度学习开发起到了至关重要的作用，数据采集和标注的质量是提升业务模型效果的重要因素。本文档主要介绍PaddleDetection中如何进行数据准备，包括采集高质量数据方法，覆盖多场景类型，提升模型泛化能力；以及各类任务数据标注工具和方法，并在PaddleDetection下使用

数据采集

在深度学习任务的实际落地中，数据采集往往决定了最终模型的效果，对于数据采集的几点建议如下：

确定方向

任务类型、数据的类别和目标场景这些因素决定了要收集什么数据，首先需要根据这些因素来确定整体数据收集的工作方向。

开源数据集

在实际场景中数据采集成本其实十分高昂，完全靠自己收集在时间和金钱上都有很高的成本，开源数据集是帮助增加训练数据量的重要手段，所以很多时候会考虑加入一些相似任务的开源数据。在使用中请遵守各个开源数据集的license规定的使用条件。

增加场景数据

开源数据一般不会覆盖实际使用的的目标场景，用户需要评估开源数据集中已包含的场景和目标场景间的差异，有针对性地补充目标场景数据，尽量让训练和部署数据的场景一致。

类别均衡

在采集阶段，也需要尽量保持类别均衡，帮助模型正确学习到目标特征。

数据标注及格式说明

任务类型	数据标注	数据格式说明
目标检测	文档链接	文档链接
关键点检测	文档链接	文档链接
多目标跟踪	文档链接	文档链接

Advanced Tutorials Customization Action Recognotion README (`docs/advanced_tutorials/customization/action_recognotion/README.md`)

简体中文 | English

行为识别任务二次开发

在产业落地过程中应用行为识别算法，不可避免地会出现希望自定义类型的行为识别的需求，或是对已有行为识别模型的优化，以提升在特定场景下模型的效果。鉴于行为的多样性，PP-Human支持抽烟、打电话、摔倒、打架、人员闯入五种异常行为识别，并根据行为的不同，集成了基于视频分类、基于检测、基于图像分类、基于跟踪以及基于骨骼点的五种行为识别技术方案，可覆盖90%+动作类型的识别，满足各类开发需求。我们在本文档通过案例来介绍如何根据期望识别的行为来进行行为识别方案的选择，以及使用PaddleDetection进行行为识别算法二次开发工作，包括：方案选择、数据准备、模型优化思路和新增行为的开发流程。

方案选择

在PaddleDetection的PP-Human中，我们为行为识别提供了多种方案：基于视频分类、基于图像分类、基于检测、基于跟踪以及基于骨骼点的行为识别方案，以期望满足不同场景、不同目标行为的需求。对于二次开发，首先我们需要确定要采用何种方案来实现行为识别的需求，其核心是要通过对场景和具体行为的分析、并考虑数据采集成本等因素，综合选择一个合适的识别方案。我们在这里简要列举了当前PaddleDetection中所支持的方案的优劣势和适用场景，供大家参考。

下面以PaddleDetection目前已经支持的几个具体动作为例，介绍每个动作方案的选型依据：

吸烟

方案选择：基于人体id检测的行为识别

原因：吸烟动作中具有香烟这个明显特征目标，因此我们可以认为当在某个人物的对应图像中检测到香烟时，该人物即在吸烟动作中。相比于基于视频或基于骨骼点的识别方案，训练检测模型需要采集的是图片级别而非视频级别的数据，可以明显减轻数据收集与标注的难度。此外，目标检测任务具有丰富的预训练模型资源，整体模型的效果会更有保障，

打电话

方案选择：基于人体id分类的行为识别

原因：打电话动作中虽然有手机这个特征目标，但为了区分看手机等动作，以及考虑到在安防场景下打电话动作中会出现较多对手机的遮挡（如手对手机的遮挡、人头对手机的遮挡等等），不利于检测模型正确检测到目标。同时打电话通常持续的时间较长，且人物本身的动作不会发生太大变化，因此可以因此采用帧级别图像分类的策略。
此外，打电话这个动作主要可以通过上半身判别，可以采用半身图片，去除冗余信息以降低模型训练的难度。

摔倒

方案选择：基于人体骨骼点的行为识别

原因：摔倒是一个明显的时序行为的动作，可由一个人物本身进行区分，具有场景无关的特性。由于PP-Human的场景定位偏向安防监控场景，背景变化较为复杂，且部署上需要考虑到实时性，因此采用了基于骨骼点的行为识别方案，以获得更好的泛化性及运行速度。

闯入

方案选择：基于人体id跟踪的行为识别

原因：闯入识别判断行人的路径或所在位置是否在某区域内即可，与人体自身动作无关，因此只需要跟踪人体跟踪结果分析是否存在闯入行为。

打架

方案选择：基于视频分类的行为识别

原因：与上面的动作不同，打架是一个典型的多人组成的行为。因此不再通过检测与跟踪模型来提取行人及其ID，而对整体视频片段进行处理。此外，打架场景下各个目标间的互相遮挡极为严重，关键点识别的准确性不高，采用基于骨骼点的方案难以保证精度。

下面详细展开五大类方案的数据准备、模型优化和新增行为识别方法

CHANGELOG (`docs/CHANGELOG.md`)

简体中文 | English

版本更新信息

历史版本信息

2.0-rc(02.23/2021)

动态图模型丰富度提升：
- 优化RCNN模型组网及训练方式，RCNN系列模型精度提升(依赖Paddle develop或2.0.1版本)
- 新增支持SSDLite，FCOS，TTFNet，SOLOv2系列模型
- 新增行人和车辆垂类目标检测模型
新增动态图基础模块：
- 新增MobileNetV3，HRNet骨干网络
- 优化RoIAlign计算逻辑，RCNN系列模型精度提升(依赖Paddle develop或2.0.1版本)
- 新增支持Synchronized Batch Norm
- 新增支持Modulated Deformable Convolution
预测部署：
- 发布动态图python、C++、Serving部署解决方案及文档，支持Faster RCNN，Mask RCNN，YOLOv3，PP-YOLO，SSD，TTFNet，FCOS，SOLOv2等系列模型预测部署
- 动态图预测部署支持TensorRT模式FP32，FP16推理加速
检测模型压缩：
- 裁剪：新增动态图裁剪支持，并发布YOLOv3-MobileNetV1裁剪模型
- 量化：新增动态图量化支持，并发布YOLOv3-MobileNetV1和YOLOv3-MobileNetV3量化模型
文档：
- 新增动态图入门教程文档：包含安装说明，快速开始，准备数据，训练/评估/预测流程文档
- 新增动态图进阶教程文档：包含模型压缩、推理部署文档
- 新增动态图模型库文档

v2.0-beta(12.20/2020)

动态图支持:
- 支持Faster-RCNN, Mask-RCNN, FPN, Cascade Faster/Mask RCNN, YOLOv3和SSD模型，试用版本。
模型提升：
- 更新PP-YOLO MobileNetv3 large和small模型，精度提升，并新增裁剪和蒸馏后的模型。
新功能：
- 支持VisualDL可视化数据预处理图片。
Bug修复:
- 修复BlazeFace人脸关键点预测bug。

v0.5.0(11/2020)

模型丰富度提升：
- 发布SOLOv2系列模型，其中SOLOv2-Light-R50-VD-DCN-FPN 模型在单卡V100上达到 38.6 FPS，加速24% ，COCO验证集精度达到38.8%, 提升2.4绝对百分点。
- 新增Android移动端检测demo，包括SSD、YOLO系列模型，可直接扫码安装体验。
移动端模型优化：
- 新增PACT新量化策略，YOLOv3-Mobilenetv3在COCO数据集上比普通量化相比提升0.7%。
易用性提升及功能组件：
- 增强generate_proposal_labels算子功能，规避模型出nan风险。
- 修复deploy下python与C++预测若干问题。
- 统一COCO与VOC数据集下评估流程，支持输出单类AP和P-R曲线。
- PP-YOLO支持矩形输入图像。
文档：
- 新增目标检测全流程教程，新增Jetson平台部署教程。

v0.4.0(07/2020)

模型丰富度提升：
- 发布PPYOLO模型，COCO数据集精度达到45.2%，单卡V100预测速度达到72.9 FPS，精度和预测速度优于YOLOv4模型。
- 新增TTFNet模型，base版本对齐竞品，COCO数据集精度达到32.9%。
- 新增HTC模型，base版本对齐竞品，COCO数据集精度达到42.2%。
- 新增BlazeFace人脸关键点检测模型，在Wider-Face数据集的Easy-Set精度达到85.2%。
- 新增ACFPN模型， COCO数据集精度达到39.6%。
- 发布服务器端通用目标检测模型（包含676类），相同策略在COCO数据集上，V100为19.5FPS时，COCO mAP可以达到49.4%。
移动端模型优化：
- 新增SSDLite系列优化模型，包括新增GhostNet的Backbone，新增FPN组件等，精度提升0.5%-1.5%。
易用性提升及功能组件：
- 新增GridMask, RandomErasing数据增强方法。
- 新增Matrix NMS支持。
- 新增EMA(Exponential Moving Average)训练支持。
- 新增多机训练方法，两机相对于单机平均加速比80%，多机训练支持待进一步验证。

v0.3.0(05/2020)

模型丰富度提升：
- 添加Efficientdet-D0模型，速度与精度优于竞品。
- 新增YOLOv4预测模型，精度对齐竞品；新增YOLOv4在Pascal VOC数据集上微调训练，精度达到85.5%。
- YOLOv3新增MobileNetV3骨干网络，COCO数据集精度达到31.6%。
- 添加Anchor-free模型FCOS，精度优于竞品。
- 添加Anchor-free模型CornernetSqueeze，精度优于竞品，优化模型的COCO数据集精度38.2%, +3.7%，速度较YOLOv3-Darknet53快5%。
- 添加服务器端实用目标检测模型CascadeRCNN-ResNet50vd模型，速度与精度优于竞品EfficientDet。
移动端推出3种模型：
- SSDLite系列模型：SSDLite-Mobilenetv3 small/large模型，精度优于竞品。
- YOLOv3移动端方案: YOLOv3-MobileNetv3模型压缩后加速3.5倍，速度和精度均领先于竞品的SSDLite模型。
- RCNN移动端方案：CascadeRCNN-MobileNetv3经过系列优化, 推出输入图像分别为320x320和640x640的模型，速度与精度具有较高性价比。
预测部署重构：
- 新增Python预测部署流程，支持RCNN，YOLO，SSD，RetinaNet，人脸系列模型，支持视频预测。
- 重构C++预测部署，提高易用性。
易用性提升及功能组件：
- 增加AutoAugment数据增强。
- 升级检测库文档结构。
- 支持迁移学习自动进行shape匹配。
- 优化mask分支评估阶段内存占用。

v0.2.0(02/2020)

新增模型:
- 新增基于CBResNet模型。
- 新增LibraRCNN模型。
- 进一步提升YOLOv3模型精度，基于COCO数据精度达到43.2%，相比上个版本提升1.4%。
新增基础模块:
- 主干网络: 新增CBResNet。
- loss模块: YOLOv3的loss支持细粒度op组合。
- 正则模块: 新增DropBlock模块。
功能优化和改进:
- 加速YOLOv3数据预处理，整体训练提速40%。
- 优化数据预处理逻辑，提升易用性。
- 增加人脸检测预测benchmark数据。
- 增加C++预测引擎Python API预测示例。
检测模型压缩 :
- 裁剪: 发布MobileNet-YOLOv3裁剪方案和模型，基于VOC数据FLOPs - 69.6%, mAP + 1.4%，基于COCO数据FLOPS-28.8%, mAP + 0.9%; 发布ResNet50vd-dcn-YOLOv3裁剪方案和模型，基于COCO数据集FLOPS - 18.4%, mAP + 0.8%。
- 蒸馏: 发布MobileNet-YOLOv3蒸馏方案和模型，基于VOC数据mAP + 2.8%，基于COCO数据mAP + 2.1%。
- 量化: 发布YOLOv3-MobileNet和BlazeFace的量化模型。
- 裁剪+蒸馏: 发布MobileNet-YOLOv3裁剪+蒸馏方案和模型，基于COCO数据FLOPS - 69.6%，基于TensorRT预测加速64.5%，mAP - 0.3 %; 发布ResNet50vd-dcn-YOLOv3裁剪+蒸馏方案和模型，基于COCO数据FLOPS - 43.7%，基于TensorRT预测加速24.0%，mAP + 0.6 %。
- 搜索: 开源BlazeFace-Nas的完成搜索方案。
预测部署:
- 集成 TensorRT，支持FP16、FP32、INT8量化推理加速。
文档:
- 增加详细的数据预处理模块介绍文档以及实现自定义数据Reader文档。
- 增加如何新增算法模型的文档。
- 文档部署到网站: https://paddledetection.readthedocs.io

12/2019

增加Res2Net模型。
增加HRNet模型。
增加GIOU loss和DIOU loss。

21/11/2019

增加CascadeClsAware RCNN模型。
增加CBNet，ResNet200和Non-local模型。
增加SoftNMS。
增加Open Image V5数据集和Objects365数据集模型。

10/2019

增加增强版YOLOv3模型，精度高达41.4%。
增加人脸检测模型BlazeFace、Faceboxes。
丰富基于COCO的模型，精度高达51.9%。
增加Objects365 2019 Challenge上夺冠的最佳单模型之一CACascade-RCNN。
增加行人检测和车辆检测预训练模型。
支持FP16训练。
增加跨平台的C++推理部署方案。
增加模型压缩示例。

2/9/2019

增加GroupNorm模型。
增加CascadeRCNN+Mask模型。

5/8/2019

增加Modulated Deformable Convolution系列模型。

29/7/2019

增加检测库中文文档
修复R-CNN系列模型训练同时进行评估的问题
新增ResNext101-vd + Mask R-CNN + FPN模型
新增基于VOC数据集的YOLOv3模型

3/7/2019

首次发布PaddleDetection检测库和检测模型库
模型包括：Faster R-CNN, Mask R-CNN, Faster R-CNN+FPN, Mask
R-CNN+FPN, Cascade-Faster-RCNN+FPN, RetinaNet, YOLOv3, 和SSD.

CHANGELOG En (`docs/CHANGELOG_en.md`)

English | 简体中文

Version Update Information

Last Version Information

2.6(02.15/2023)

Featured model
- Release rotated object detector PP-YOLOE-R：SOTA Anchor-free rotated object detection model with high accuracy and efficiency. It has a series of models, named s/m/l/x, for cloud and edge devices and avoids using special operators to be deployed friendly with TensorRT.
  - Release small object detector PP-YOLOE-SOD: End-to-end detection pipeline based on sliced images and SOTA model on VisDrone based on original images.
  - Release crowded object detector: Crowded object detection model with top accuracy on SKU dataset.
Functions in different scenarios
- Release real-time object detection model on edge device in PP-Human v2. The model reaches 45.7mAP and 80FPS on Jetson AGX
- Release real-time object detection model on edge device in PP-Vehicle. The model reaches 53.5mAP and 80FPS on Jetson AGX
- Support multi-stream deployment in PP-Human v2 and PP-Vehicle. Achieved 20FPS in 4-stream deployment on Jetson AGX
- Support retrograde and press line detection in PP-Vehicle
Cutting-edge algorithms
- Release YOLOv8 and YOLOv6 3.0 in YOLO Family
- Release object detection algorithm DINO, YOLOF
- Rich ViTDet series including PP-YOLOE+ViT_base, Mask RCNN + ViT_base, Mask RCNN + ViT_large
- Release MOT algorithm CenterTrack
- Release oriented object detection algorithm FCOSR
- Release instance segmentation algorithm QueryInst
- Release 3D keypoint detection algorithm Metro3d
- Release distillation algorithm FGD，LD，CWD and PP-YOLOE+ distillation with improvement of 1.1+ mAP
- Release SSOD algorithm DenseTeacher and adapt for PP-YOLOE+
- Release few shot finetuning algorithm, including Co-tuning and Contrastive learning
Framework capabilities
- New functions
  - Release Grad-CAM for heatmap visualization. Support Faster RCNN, Mask RCNN, PP-YOLOE, BlazeFace, SSD, RetinaNet.
- Improvement and fixes
  - Support python 3.10
  - Fix EMA for no-grad parameters
  - Simplify PP-YOLOE architecture
  - Support AdamW for Paddle 2.4.1

2.5(08.26/2022)

Featured model
- PP-YOLOE+：
  - Released PP-YOLOE+ model, with a 0.7%-2.4% mAP improvement on COCO test2017. 3.75 times faster model training convergence rate and 1.73-2.3 times faster end-to-end inference speed
  - Released pre-trained models for smart agriculture, night security detection, and industrial quality inspection with 1.3%-8.1% mAP accuracy improvement
  - supports 10 high-performance training deployment capabilities, including distributed training, online quantization, and serving deployment. We also provide more than five new deployment demos, such as C++/Python Serving, TRT native inference, and ONNX Runtime
- PP-PicoDet：
  - Release the PicoDet-NPU model to support full quantization of model deployment
  - Add PicoDet layout analysis model with 0.5% mAP accuracy improvement due to FGD distillation algorithm
- PP-TinyPose
  - Release PP-TinyPose Plus with 9.1% end-to-end AP improvement for business data sets such as physical exercises, dance, and other scenarios
  - Covers unconventional movements such as turning to one side, lying down, jumping, high lift
  - Add stabilization module (via filter) to significantly improve the stability at key points
Functions in different scenarios
- PP-Human v2
  - Release PP-Human v2, which supports four industrial features: behavioral recognition case zoo for multiple solutions, human attribute recognition, human traffic detection and trajectory retention, as well as high precision multi-camera tracking
  - Upgraded underlying algorithm capabilities: 1.5% mAP improvement in pedestrian detection accuracy; 10.2% MOTA improvement in pedestrian tracking accuracy, 34% speed improvement in the lightweight model; 0.6% ma improvement in attribute recognition accuracy, 62.5% speed improvement in the lightweight model
  - Provides comprehensive tutorials covering data collection and annotation, model training optimization and prediction deployment, and post-processing code modification in the pipeline
  - Supports online video streaming input
  - Become more user-friendly with a one-line code execution function that automates the process determination and model download
- PP-Vehicle
  - Launch PP-Vehicle, which supports four core functions for traffic application: license plate recognition, attribute recognition, traffic flow statistics, and violation detection
  - License plate recognition supports a lightweight model based on PP-OCR v3
  - Vehicle attribute recognition supports a multi-label classification model based on PP-LCNet
  - Compatible with various data input formats such as pictures, videos and online video streaming
  - Become more user-friendly with a one-line code execution function that automates the process determination and model download
Cutting-edge algorithms
- YOLO Family
  - Release the full range of YOLO family models covering the cutting-edge detection algorithms YOLOv5, YOLOv6 and YOLOv7
  - Based on the ConvNext backbone network, YOLO's algorithm training periods are reduced by 5-8 times with accuracy generally improving by 1%-5% mAP; Thanks to the model compression strategy, its speed increased by over 30% with no loss of precision.
- Newly add high precision detection model based on ViT backbone network, with a 55.7% mAP accuracy on the COCO dataset
- Newly add multi-object tracking model OC-SORT
- Newly add ConvNeXt backbone network.
Industrial application
- Intelligent physical exercise recognition based on PP-TinyPose Plus
- Fighting recognition based on PP-Human
- Business hall visitor analysis based on PP-Human
- Vehicle structuring analysis based on PP-Vehicle
- PCB board defect detection based on PP-YOLOE+
Framework capabilities
- New functions
  - Release auto-compression tools and demos, 0.3% mAP accuracy loss for PP-YOLOE l version, while 13% speed increase for V100
  - Release PaddleServing python/C++ and ONNXRuntime deployment demos
  - Release PP-YOLOE end-to-end TensorRT deployment demo
  - Release FGC distillation algorithm with RetinaNet accuracy improved by 3.3%
  - Release distributed training documentation
- Improvement and fixes
  - Fix compilation problem with Windows c++ deployment
  - Fix problems when saving results of inference data in VOC format
  - Fix the detection box output of FairMOT c++ deployment
  - Rotating frame detection model S2ANet supports batch size>1 deployment

2.4(03.24/2022)

PP-YOLOE：
- Release PP-YOLOE object detection models, achieve mAP as 51.6% on COCO test dataset and 78.1 FPS on Nvidia V100 by PP-YOLOE-l, reach SOTA performance for object detection on GPU``
- Release series models: s/m/l/x, and support deployment base on TensorRT & ONNX
- Spport AMP training and training speed is 33% faster than PP-YOLOv2
PP-PicoDet:
- Release enhanced models of PP-PicoDet, mAP promoted ~2% on COCO and inference speed accelerated 63% on CPU
- Release PP-PicoDet-XS model with 0.7M parameters
- Post-processing integrated into the network to optimize deployment pipeline
PP-Human：
- Release PP-Human human analysis pipeline，including pedestrian detection, attribute recognition, human tracking, multi-camera tracking, human statistics, action recognition. Supporting deployment with TensorRT
- Release StrongBaseline model for attribute recognition
- Release Centroid model for ReID
- Release ST-GCN model for falldown action recognition
Model richness:
- Publish YOLOX object detection model, release series models: nano/tiny/s/m/l/x, and YOLOX-x achieves mAP as 51.8% on COCO val2017 dataset
Function Optimize：
- Optimize 20% training speed when training with EMA, improve saving method of EMA weights
- Support saving inference results in COCO format
Deployment Optimize：
- Support export ONNX model by Paddle2ONNX for all RCNN models
- Supoort export model with fused decode OP for SSD models to enhance inference speed in edge side
- Support export NMS to TensorRT model, optmize inference speed on TensorRT

2.3(11.03/2021)

Feature models:
- Object detection: The lightweight object detection model PP-PicoDet, performace and inference speed reaches SOTA on mobile side
- Keypoint detection: The lightweight keypoint detection model PP-TinyPose for mobile side
Model richness:
- Object detection:
  - Publish Swin-Transformer object detection model
  - Publish TOOD(Task-aligned One-stage Object Detection) model
  - Publish GFL(Generalized Focal Loss) object detection model
  - Publish Sniper optimization method for tiny object detection, supporting Faster RCNN and PP-YOLO series models
  - Publish PP-YOLO optimized model PP-YOLO-EB for EdgeBoard
- Multi-object tracking:
  - Publish Real-time tracking system PP-Tracking
  - Publish high-precision, small-scale and lightweight model based on FairMot
  - Publish real-time tracking model zoo for pedestrian, head and vehicle tracking, including scenarios such as aerial surveillance, autonomous driving, dense crowds, and tiny object tracking
  - DeepSort support PP-YOLO, PP-PicoDet as object detector
- Keypoint detection:
  - Publish Lite HRNet model
Inference deployment:
- Support NPU deployment for YOLOv3 series
- Support C++ deployment for FairMot
- Support C++ and PaddleLite deployment for keypoint detection series model
Documents:
- Add series English documents

2.2(08.10/2021)

Model richness:
- Publish the Transformer test model: DETR, Deformable DETR, Sparse RCNN
- Key point test new Dark model, release Dark HRNet model
- Publish the MPII dataset HRNet keypoint detection model
- Release head and vehicle tracking vertical model
Model optimization:
- AlignConv optimization model was released by S2ANet, and DOTA dataset mAP was optimized to 74.0
Inference deployment
- Mainstream models support batch size>1 predictive deployment, including YOLOv3, PP-YOLO, Faster RCNN, SSD, TTFNet, FCOS
- New addition of target tracking models (JDE, Fair Mot, Deep Sort) Python side prediction deployment support, and support for TensorRT prediction
- FairMot joint key point detection model deployment Python side predictive deployment support
- Added support for key point detection model combined with PP-YOLO prediction deployment
Documents:
- New TensorRT version notes to Windows Predictive Deployment documentation
- FAQ documents are updated
Bug fixes:
- Fixed PP-YOLO series model training convergence problem
- Fixed the problem of no label data training when batch_size > 1

2.1(05.20/2021)

Model richness enhancement:
- Key point model: HRNet, HigherHRNet
- Publish the multi-target tracking model: DeepSort, FairMot, JDE
Basic framework Capabilities:
- Supports training without labels
Forecast deployment:
- Paddle Inference YOLOv3 series model support batch_size>1 prediction
- Rotating frame detection S2ANet model prediction deployment is open
- Incremental quantization model benchmark
- Add dynamic graph model and static graph model: Paddle-Lite demo
Detection model compression:
- Release PP-YOLO series model compression model
Documents:
- Update quick start, forecast deployment and other tutorial documentation
- Added ONNX model export tutorial
- Added the mobile deployment document

2.0(04.15/2021)

Description: Since version 2.0, dynamic graphs are used as the default version of Paddle Detection, the original dygraph directory is switched to the root directory, and the original static graph implementation is moved to the static directory.

Enhancement of dynamic graph model richness:
- PP-YOLOv2 and PP-YOLO tiny models were published. The accuracy of PP-YOLOv2 COCO Test dataset reached 49.5%, and the prediction speed of V100 reached 68.9 FPS
- Release the rotary frame detection model S2ANet
- Release the two-phase utility model PSS-Det
- Publish the face detection model Blazeface
New basic module:
- Added SENet, GhostNet, and Res2Net backbone networks
- Added VisualDL training visualization support
- Added single precision calculation and PR curve drawing function
- The YOLO models support THE NHWC data format
Forecast deployment:
- Publish forecast benchmark data for major models
- Adaptive to TensorRT6, support TensorRT dynamic size input, support TensorRT int8 quantitative prediction
- 7 types of models including PP-YOLO, YOLOv3, SSD, TTFNet, FCOS, Faster RCNN are deployed in Python/CPP/TRT prediction on Linux, Windows and NV Jetson platforms
Detection model compression:
- Distillation: Added dynamic map distillation support and released YOLOv3-MobileNetV1 distillation model
- Joint strategy: new dynamic graph prunning + distillation joint strategy compression scheme, and release YOLOv3-MobileNetV1 prunning + distillation compression model
- Problem fix: Fixed dynamic graph quantization model export problem
Documents:
- New English document of dynamic graph: including homepage document, getting started, quick start, model algorithm, new dataset, etc
- Added both English and Chinese installation documents of dynamic diagrams
- Added configuration file templates and description documents of dynamic graph RCNN series and YOLO series

Historical Version Information

2.0-rc(02.23/2021)

Enhancement of dynamic graph model richness:
- Optimize networking and training mode of RCNN models, and improve accuracy of RCNN series models (depending on Paddle Develop or version 2.0.1)
- Added support for SSDLite, FCOS, TTFNet, SOLOv2 series models
- Added pedestrian and vehicle vertical object detection models
New dynamic graph basic module:
- Added MobileNetV3 and HRNet backbone networks
- Improved roi-align calculation logic for RCNN series models (depending on Paddle Develop or version 2.0.1)
- Added support for Synchronized Batch Norm
- Added support for Modulated Deformable Convolution
Forecast deployment:
- Publish dynamic diagrams in python, C++, and Serving deployment solution and documentation. Support Faster RCNN, Mask RCNN, YOLOv3, PPYOLO, SSD, TTFNet, FCOS, SOLOv2 and other models to predict deployment
- Dynamic graph prediction deployment supports TensorRT mode FP32, FP16 inference acceleration
Detection model compression:
- Prunning: Added dynamic graph prunning support, and released YOLOv3-MobileNetV1 prunning model
- Quantization: Added quantization support of dynamic graph, and released quantization models of YOLOv3-MobileNetV1 and YOLOv3-MobileNetV3
Documents:
- New Dynamic Diagram tutorial documentation: includes installation instructions, quick start, data preparation, and training/evaluation/prediction process documentation
- New advanced tutorial documentation for dynamic diagrams: includes documentation for model compression and inference deployment
- Added dynamic graph model library documentation

v2.0-beta(12.20/2020)

Dynamic graph support:
- Support for Faster-RCNN, Mask-RCNN, FPN, Cascade Faster/Mask RCNN, YOLOv3 and SSD models, trial version.
Model upgrade:
- Updated PP-YOLO Mobile-Netv3 large and small models with improved accuracy, and added prunning and distillation models.
New features:
- Support VisualDL visual data preprocessing pictures.
Bug fix:
- Fix Blaze Face keypoint prediction bug.

v0.5.0(11/2020)

Model richness enhancement:
- SOLOv2 series models were released, in which the SOLOv2-Light-R50-VD-DCN-FPN model achieved 38.6 FPS on a single gpu V100, accelerating by 24%, and the accuracy of COCO verification set reached 38.8%, improving by 2.4 absolute percentage points.
- Added Android mobile terminal detection demo, including SSD, YOLO series model, can directly scan code installation experience.
Mobile terminal model optimization:
- Added to PACT's new quantization strategy, YOLOv3 Mobilenetv3 is 0.7% better than normal quantization on COCO datasets.
Ease of use and functional components:
- Enhance the function of generate_proposal_labels operator to avoid nan risk of the model.
- Fixed several problems with deploy python and C++ prediction.
- Unified COCO and VOC datasets under the evaluation process, support the output of a single class of AP and P-R curves.
- PP-YOLO supports rectangular input images.
Documents:
- Added object detection whole process tutorial, added Jetson platform deployment tutorial.

v0.4.0(07/2020)

Model richness enhancement:
- The PPYOLO model was released. The accuracy of COCO dataset reached 45.2%, and the prediction speed of single gpu V100 reached 72.9 FPS, which was better than that of YOL Ov4 model.
- New TTFNet model, base version aligned with competing products, COCO dataset accuracy up to 32.9%.
- New HTC model, base version aligned with competing products, COCO dataset accuracy up to 42.2%.
- BlazeFace key point detection model was added, with an accuracy of 85.2% in Wider-Face's Easy-Set.
- ACFPN model was added, and the accuracy of COCO dataset reached 39.6%.
- General object detection model (including 676 classes) on the publisher side. On the COCO dataset with the same strategy, when V100 is 19.5FPS, the COCO mAP can reach 49.4%.
Mobile terminal model optimization:
- Added SSD Lite series optimization models, including Ghost Net Backbone, FPN components, etc., with accuracy improved by 0.5% and 1.5%.
Ease of use and functional components:
- Add GridMask, Random Erasing data enhancement method.
- Added support for Matrix NMS.
- EMA(Exponential Moving Average) training support.
- The new multi-machine training method, the average acceleration ratio of two machines to single machine is 80%, multi-machine training support needs to be further verified.

v0.3.0(05/2020)

Model richness enhancement:
- Efficientdet-D0 model added, speed and accuracy is better than competing products.
- Added YOLOv4 prediction model, precision aligned with competing products; Added YOLOv4 fine tuning training on Pascal VOC datasets with accuracy of 85.5%.
- YOLOv3 added MobileNetV3 backbone network, COCO dataset accuracy reached 31.6%.
- Add Anchor-free model FCOS, the accuracy is better than competing products.
- Anchor-free model Cornernet Squeeze was added, the accuracy was better than competing products, and the accuracy of COCO dataset of optimized model was 38.2% and +3.7%, 5% faster than YOL Ov3 Darknet53.
- The CascadeRCNN-ResNet50vd model, which is a practical object detection model on the server side, is added, and its speed and accuracy are better than that of the competitive EfficientDet.
Mobile terminal launched three models:
- SSSDLite model: SSDLite-Mobilenetv3 small/large model, with better accuracy than competitors.
- YOLOv3 Mobile solution: The YOLOv3-MobileNetv3 model accelerates 3.5 times after compression, which is faster and more accurate than the SSD Lite model of competing products.
- RCNN Mobile terminal scheme: CascadeRCNN-MobileNetv3, after series optimization, launched models with input images of 320x320 and 640x640 respectively, with high cost performance for speed and accuracy.
Anticipate deployment refactoring:
- New Python prediction deployment process, support for RCNN, YOLO, SSD, Retina Net, face models, support for video prediction.
- Refactoring C++ predictive deployment to improve ease of use.
Ease of use and functional components:
- Added Auto Augment data enhancement.
- Upgrade the detection library document structure.
- Support shape matching automatically by transfer learning.
- Optimize memory footprint during mask branch evaluation.

v0.2.0(02/2020)

The new model:
- Added CBResNet model.
- Added LibraRCNN model.
- The accuracy of YOLOv3 model was further improved, and the accuracy based on COCO data reached 43.2%, 1.4% higher than the previous version.
New Basic module:
- Trunk network: CBResNet is added.
- Loss module: Loss of YOLOv3 supports fine-grained OP combinations.
- Regular module: Added the Drop Block module.
Function optimization and improvement:
- Accelerate YOLOv3 data preprocessing and increase the overall training speed by 40%.
- Optimize data preprocessing logic to improve ease of use.
- dd face detection prediction benchmark data.
- Added C++ prediction engine Python API prediction example.
Detection model compression:
- prunning: Release MobileNet-YOLOv3 prunning scheme and model, based on VOC data FLOPs 69.6%, mAP + 1.4%, based on COCO DATA FLOPS 28.8%, mAP + 0.9%; Release ResNet50vd-DCN-YOLOv3 clipped solution and model based on COCO datasets 18.4%, mAP + 0.8%.
- Distillation: Release MobileNet-YOLOv3 distillation scheme and model, based on VOC data mAP + 2.8%, COCO data mAP + 2.1%.
- Quantification: Release quantification models of YOLOv3 Mobile Net and Blaze Face.
- Prunning + distillation: release MobileNet-YOLOv3 prunning + distillation solution and model, 69.6% based on COCO DATA FLOPS, 64.5% based on TensorRT prediction acceleration, 0.3% mAP; Release ResNet50vd-DCN-YOLOv3 tailoring + distillation solution and model, 43.7% based on COCO Data FLOPS, 24.0% based on TensorRT prediction acceleration, mAP + 0.6%.
- Search: Open source Blaze Face Nas complete search solution.
Predict deployment:
- Integrated TensorRT, support FP16, FP32, INT8 quantitative inference acceleration.
Document:
- Add detailed data preprocessing module to introduce documents and implement custom data Reader documents.
- Added documentation on how to add algorithm models.
- Document deployment to the web site: https://paddledetection.readthedocs.io

12/2019

Add Res2Net model.
Add HRNet model.
Add GIOU loss and DIOU loss。

21/11/2019

Add CascadeClsAware RCNN model.
Add CBNet, ResNet200 and Non-local model.
Add SoftNMS.
Add Open Image V5 dataset and Objects365 dataset model

10/2019

Added enhanced YOLOv3 model with accuracy up to 41.4%.
Added Face detection models BlazeFace and Faceboxes.
Rich COCO based models, accuracy up to 51.9%.
Added CA-Cascade-RCNN, one of the best single models to win on Objects365 2019 Challenge.
Add pedestrian detection and vehicle detection pre-training models.
Support FP16 training.
Added cross-platform C++ inference deployment scheme.
Add model compression examples.

2/9/2019

Add GroupNorm model.
Add CascadeRCNN+Mask model.

5/8/2019

Add Modulated Deformable Convolution series model

29/7/2019

Add detection library Chinese document
Fixed an issue where R-CNN series model training was evaluated simultaneously
Add ResNext101-vd + Mask R-CNN + FPN models
Added YOLOv3 model based on VOC dataset

3/7/2019

First release of PaddleDetection Detection library and Detection model library
models：Faster R-CNN, Mask R-CNN, Faster R-CNN+FPN, Mask
R-CNN+FPN, Cascade-Faster-RCNN+FPN, RetinaNet, YOLOv3, 和SSD.

MODEL ZOO Cn (`docs/MODEL_ZOO_cn.md`)

模型库和基线

基础设置

测试环境

Python 3.7
PaddlePaddle 每日版本
CUDA 10.1
cuDNN 7.5
NCCL 2.4.8

通用设置

所有模型均在COCO17数据集中训练和测试。
YOLOv5、YOLOv6、YOLOv7和YOLOv8这几类模型的代码在PaddleYOLO中，PaddleYOLO库开源协议为GPL 3.0。
除非特殊说明，所有ResNet骨干网络采用ResNet-B结构。
推理时间(fps): 推理时间是在一张Tesla V100的GPU上通过'tools/eval.py'测试所有验证集得到，单位是fps(图片数/秒), cuDNN版本是7.5，包括数据加载、网络前向执行和后处理, batch size是1。

训练策略

我们采用和Detectron相同的训练策略。
1x 策略表示：在总batch size为8时，初始学习率为0.01，在8 epoch和11 epoch后学习率分别下降10倍，最终训练12 epoch。
2x 策略为1x策略的两倍，同时学习率调整的epoch数位置也为1x的两倍。

ImageNet预训练模型

Paddle提供基于ImageNet的骨架网络预训练模型。所有预训练模型均通过标准的Imagenet-1k数据集训练得到，ResNet和MobileNet等是采用余弦学习率调整策略或SSLD知识蒸馏训练得到的高精度预训练模型，可在PaddleClas查看模型细节。

基线

目标检测

Faster R-CNN

请参考Faster R-CNN

YOLOv3

请参考YOLOv3

PP-YOLOE/PP-YOLOE+

请参考PP-YOLOE

PP-YOLO/PP-YOLOv2

请参考PP-YOLO

PicoDet

请参考PicoDet

RetinaNet

请参考RetinaNet

Cascade R-CNN

请参考Cascade R-CNN

SSD/SSDLite

请参考SSD

FCOS

请参考FCOS

CenterNet

请参考CenterNet

TTFNet/PAFNet

请参考TTFNet

Group Normalization

请参考Group Normalization

Deformable ConvNets v2

请参考Deformable ConvNets v2

HRNets

请参考HRNets

Res2Net

请参考Res2Net

ConvNeXt

请参考ConvNeXt

GFL

请参考GFL

TOOD

请参考TOOD

PSS-DET(RCNN-Enhance)

请参考PSS-DET

DETR

请参考DETR

DINO

请参考DINO

YOLOX

请参考YOLOX

YOLOF

请参考YOLOF

实例分割

Mask R-CNN

请参考Mask R-CNN

Cascade R-CNN

请参考Cascade R-CNN

SOLOv2

请参考SOLOv2

QueryInst

请参考QueryInst

[PaddleYOLO](https://github.com/PaddlePaddle/PaddleYOLO)

请参考PaddleYOLO模型库

YOLOv5

请参考YOLOv5

YOLOv6(v3.0)

请参考YOLOv6

YOLOv7

请参考YOLOv7

YOLOv8

请参考YOLOv8

RTMDet

请参考RTMDet

人脸检测

请参考人脸检测模型库

BlazeFace

请参考BlazeFace

旋转框检测

请参考旋转框检测模型库

PP-YOLOE-R

请参考PP-YOLOE-R

FCOSR

请参考FCOSR

S2ANet

请参考S2ANet

关键点检测

请参考关键点检测模型库

PP-TinyPose

请参考PP-TinyPose

HRNet

请参考HRNet

Lite-HRNet

请参考Lite-HRNet

HigherHRNet

请参考HigherHRNet

多目标跟踪

请参考多目标跟踪模型库

DeepSORT

请参考DeepSORT

ByteTrack

请参考ByteTrack

OC-SORT

请参考OC-SORT

BoT-SORT

请参考BoT-SORT

CenterTrack

请参考CenterTrack

FairMOT/MC-FairMOT

请参考FairMOT

JDE

请参考JDE

MODEL ZOO En (`docs/MODEL_ZOO_en.md`)

Model Zoos and Baselines

Content

Basic Settings
Baseline

Basic Settings

Test Environment

Python 3.7
PaddlePaddle Daily version
CUDA 10.1
cuDNN 7.5
NCCL 2.4.8

General Settings

All models were trained and tested in the COCO17 dataset.
The codes of YOLOv5,YOLOv6,YOLOv7 and YOLOv8 can be found in PaddleYOLO. Note that the LICENSE of PaddleYOLO is GPL 3.0.
Unless special instructions, all the ResNet backbone network using ResNet-B structure.
Inference time (FPS): The reasoning time was calculated on a Tesla V100 GPU by tools/eval.py testing all validation sets in FPS (number of pictures/second). CuDNN version is 7.5, including data loading, network forward execution and post-processing, and Batch size is 1.

Training strategy

We adopt and Detectron in the same training strategy.
1x strategy indicates that when the total batch size is 8, the initial learning rate is 0.01, and the learning rate decreases by 10 times after 8 epoch and 11 epoch, respectively, and the final training is 12 epoch.
2x strategy is twice as much as strategy 1x, and the learning rate adjustment position of epochs is twice as much as strategy 1x.

ImageNet pretraining model

Paddle provides a skeleton network pretraining model based on ImageNet. All pre-training models were trained by standard Imagenet 1K dataset. ResNet and MobileNet are high-precision pre-training models obtained by cosine learning rate adjustment strategy or SSLD knowledge distillation training. Model details are available at PaddleClas.

Baseline

Object Detection

Faster R-CNN

Please refer to Faster R-CNN

YOLOv3

Please refer to YOLOv3

PP-YOLOE/PP-YOLOE+

Please refer to PP-YOLOE

PP-YOLO/PP-YOLOv2

Please refer to PP-YOLO

PicoDet

Please refer to PicoDet

RetinaNet

Please refer to RetinaNet

Cascade R-CNN

Please refer to Cascade R-CNN

SSD/SSDLite

Please refer to SSD

FCOS

Please refer to FCOS

CenterNet

Please refer to CenterNet

TTFNet/PAFNet

Please refer to TTFNet

Group Normalization

Please refer to Group Normalization

Deformable ConvNets v2

Please refer to Deformable ConvNets v2

HRNets

Please refer to HRNets

Res2Net

Please refer to Res2Net

ConvNeXt

Please refer to ConvNeXt

GFL

Please refer to GFL

TOOD

Please refer to TOOD

PSS-DET(RCNN-Enhance)

Please refer to PSS-DET

DETR

Please refer to DETR

Deformable DETR

Please refer to Deformable DETR

Sparse R-CNN

Please refer to Sparse R-CNN

Vision Transformer

Please refer to Vision Transformer

DINO

Please refer to DINO

YOLOX

Please refer to YOLOX

YOLOF

Please refer to YOLOF

Instance-Segmentation

Mask R-CNN

Please refer to Mask R-CNN

Cascade R-CNN

Please refer to Cascade R-CNN

SOLOv2

Please refer to SOLOv2

QueryInst

Please refer to QueryInst

[PaddleYOLO](https://github.com/PaddlePaddle/PaddleYOLO)

Please refer to Model Zoo for PaddleYOLO

YOLOv5

Please refer to YOLOv5

YOLOv6(v3.0)

Please refer to YOLOv6

YOLOv7

Please refer to YOLOv7

YOLOv8

Please refer to YOLOv7

RTMDet

Please refer to RTMDet

Face Detection

Please refer to Model Zoo for Face Detection

BlazeFace

Please refer to BlazeFace

Rotated Object detection

Please refer to Model Zoo for Rotated Object Detection

PP-YOLOE-R

Please refer to PP-YOLOE-R

FCOSR

Please refer to FCOSR

S2ANet

Please refer to S2ANet

KeyPoint Detection

Please refer to Model Zoo for KeyPoint Detection

PP-TinyPose

Please refer to PP-TinyPose

HRNet

Please refer to HRNet

Lite-HRNet

Please refer to Lite-HRNet

HigherHRNet

Please refer to HigherHRNet

Multi-Object Tracking

Please refer to Model Zoo for Multi-Object Tracking

DeepSORT

Please refer to DeepSORT

ByteTrack

Please refer to ByteTrack

OC-SORT

Please refer to OC-SORT

BoT-SORT

Please refer to BoT-SORT

CenterTrack

Please refer to CenterTrack

FairMOT/MC-FairMOT

Please refer to FairMOT

JDE

Please refer to JDE

PaddleDetection动转静支持列表 (`docs/PaddleDetection动转静支持列表.md`)

PaddleDetection动转静模型支持列表

模型名称	是否支持	运行命令
cascade_rcnn	✅	使用动态图训练: python tools/train.py -c configs/cascade_rcnn/cascade_rcnn_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/cascade_rcnn/cascade_rcnn_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
centernet	✅	使用动态图训练: python tools/train.py -c configs/centernet/centernet_r50_140e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/centernet/centernet_r50_140e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
clrnet	✅	使用动态图训练: python tools/train.py -c configs/clrnet/clrnet_resnet18_culane.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/clrnet/clrnet_resnet18_culane.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
convnext	✅	使用动态图训练: python tools/train.py -c configs/convnext/ppyoloe_convnext_tiny_36e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/convnext/ppyoloe_convnext_tiny_36e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
dcn	✅	使用动态图训练: python tools/train.py -c configs/dcn/faster_rcnn_dcn_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/dcn/faster_rcnn_dcn_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
deformable_detr	✅	使用动态图训练: python tools/train.py -c configs/deformable_detr/deformable_detr_r50_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/deformable_detr/deformable_detr_r50_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
detr	-	使用动态图训练: python tools/train.py -c configs/detr/detr_r50_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/detr/detr_r50_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
dino	-	使用动态图训练: python tools/train.py -c configs/dino/dino_r50_4scale_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/dino/dino_r50_4scale_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
face_detection	✅	使用动态图训练: python tools/train.py -c configs/face_detection/blazeface_1000e.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/face_detection/blazeface_1000e.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
faster_rcnn	✅	使用动态图训练: python tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
fcos	✅	使用动态图训练: python tools/train.py -c configs/fcos/fcos_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/fcos/fcos_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
few-shot	✅	使用动态图训练: python tools/train.py -c configs/few-shot/faster_rcnn_r50_vd_fpn_1x_coco_cotuning_roadsign.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/few-shot/faster_rcnn_r50_vd_fpn_1x_coco_cotuning_roadsign.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
focalnet	✅	使用动态图训练: python tools/train.py -c configs/focalnet/ppyoloe_plus_focalnet_tiny_36e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/focalnet/ppyoloe_plus_focalnet_tiny_36e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
gfl	-	使用动态图训练: python tools/train.py -c configs/gfl/gfl_r18vd_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/gfl/gfl_r18vd_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
gn	✅	使用动态图训练: python tools/train.py -c configs/gn/faster_rcnn_r50_fpn_gn_2x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/gn/faster_rcnn_r50_fpn_gn_2x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
group_detr	动态图、静态图运行均出现错误	使用动态图训练: python tools/train.py -c configs/group_detr/group_dino_r50_4scale_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/group_detr/group_dino_r50_4scale_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
hrnet	✅	使用动态图训练: python tools/train.py -c configs/hrnet/faster_rcnn_hrnetv2p_w18_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/hrnet/faster_rcnn_hrnetv2p_w18_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
mask_rcnn	✅	使用动态图训练: python tools/train.py -c configs/mask_rcnn/mask_rcnn_r50_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/mask_rcnn/mask_rcnn_r50_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
picodet	✅	使用动态图训练: python tools/train.py -c configs/picodet/picodet_l_320_coco_lcnet.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/picodet/picodet_l_320_coco_lcnet.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
pphuman	✅	使用动态图训练: python tools/train.py -c configs/pphuman/ppyoloe_crn_l_36e_crowdhuman.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/pphuman/ppyoloe_crn_l_36e_crowdhuman.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
ppvehicle	✅	使用动态图训练: python tools/train.py -c configs/ppvehicle/mot_ppyoloe_s_36e_ppvehicle.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/ppvehicle/mot_ppyoloe_s_36e_ppvehicle.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
ppyolo	✅	使用动态图训练: python tools/train.py -c configs/ppyolo/ppyolo_mbv3_small_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/ppyolo/ppyolo_mbv3_small_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
ppyoloe	✅	使用动态图训练: python tools/train.py -c configs/ppyoloe/ppyoloe_crn_m_300e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/ppyoloe/ppyoloe_crn_m_300e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
queryinst	-	使用动态图训练: python tools/train.py -c configs/queryinst/queryinst_r50_fpn_1x_pro100_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/queryinst/queryinst_r50_fpn_1x_pro100_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
rcnn_enhance	✅	使用动态图训练: python tools/train.py -c configs/rcnn_enhance/faster_rcnn_enhance_3x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/rcnn_enhance/faster_rcnn_enhance_3x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
res2net	✅	使用动态图训练: python tools/train.py -c configs/res2net/faster_rcnn_res2net50_vb_26w_4s_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/res2net/faster_rcnn_res2net50_vb_26w_4s_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
retinanet	✅	使用动态图训练: python tools/train.py -c configs/retinanet/retinanet_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/retinanet/retinanet_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
rtdetr	-	使用动态图训练: python tools/train.py -c configs/rtdetr/rtdetr_r18vd_6x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/rtdetr/rtdetr_r18vd_6x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
ppyoloe_sod	✅	使用动态图训练: python tools/train.py -c configs/smalldet/ppyoloe_plus_sod_crn_l_80e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/smalldet/ppyoloe_plus_sod_crn_l_80e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
solov2	✅	使用动态图训练: python tools/train.py -c configs/solov2/solov2_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/solov2/solov2_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
sparse_rcnn	-	使用动态图训练: python tools/train.py -c configs/sparse_rcnn/sparse_rcnn_r50_fpn_3x_pro100_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/sparse_rcnn/sparse_rcnn_r50_fpn_3x_pro100_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
sqr	✅	使用动态图训练: python tools/train.py -c configs/sqr/deformable_detr_sqr_r50_12e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/sqr/deformable_detr_sqr_r50_12e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
ssd	✅	使用动态图训练: python tools/train.py -c configs/ssd/ssd_r34_70e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/ssd/ssd_r34_70e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
swin	-	使用动态图训练: python tools/train.py -c configs/swin/faster_rcnn_swin_tiny_fpn_3x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/swin/faster_rcnn_swin_tiny_fpn_3x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
tood	✅	使用动态图训练: python tools/train.py -c configs/tood/tood_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/tood/tood_r50_fpn_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
ttfnet	✅	使用动态图训练: python tools/train.py -c configs/ttfnet/ttfnet_darknet53_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/ttfnet/ttfnet_darknet53_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
vitdet	✅	使用动态图训练: python tools/train.py -c configs/vitdet/ppyoloe_vit_base_csppan_cae_36e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/vitdet/ppyoloe_vit_base_csppan_cae_36e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
yolof	✅	使用动态图训练: python tools/train.py -c configs/yolof/yolof_r50_c5_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/yolof/yolof_r50_c5_1x_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
yolov3	✅	使用动态图训练: python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static
yolox	✅	使用动态图训练: python tools/train.py -c configs/yolox/yolox_l_300e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval使用静态图训练: python tools/train.py -c configs/yolox/yolox_l_300e_coco.yml -se_vdl=true --vdl_log_dir=vdl_dir/scalar --eval --to_static

Advanced Tutorials MODEL TECHNICAL (`docs/advanced_tutorials/MODEL_TECHNICAL.md`)

新增模型算法

为了让用户更好的使用PaddleDetection，本文档中，我们将介绍PaddleDetection的主要模型技术细节及应用

_BASE_: [
  '../datasets/coco_detection.yml', # 数据集配置文件，所有模型共用
  '../runtime.yml', # 运行时相关配置
  '_base_/optimizer_270e.yml', # 优化器相关配置
  '_base_/yolov3_darknet53.yml', # yolov3网络结构配置文件
  '_base_/yolov3_reader.yml', # yolov3 Reader模块配置
]

# 定义在此处的相关配置可以覆盖上述文件中的同名配置
snapshot_epoch: 5
weights: output/yolov3_darknet53_270e_coco/model_final

可以看到，配置文件中的模块进行了清晰的划分，除了公共的数据集配置以及运行时配置，其他配置被划分为优化器，网络结构以及Reader模块。PaddleDetection中支持丰富的优化器，学习率调整策略，预处理算子等，因此大多数情况下不需要编写优化器以及Reader相关的代码，而只需要在配置文件中配置即可。因此，新增一个模型的主要在于搭建网络结构。

PaddleDetection网络结构的代码在ppdet/modeling/中，所有网络结构以组件的形式进行定义与组合，网络结构的主要构成如下所示：

text

ppdet/modeling/
  ├── architectures
  │   ├── faster_rcnn.py # Faster Rcnn模型
  │   ├── ssd.py         # SSD模型
  │   ├── yolo.py      # YOLOv3模型
  │   │   ...
  ├── heads       # 检测头模块
  │   ├── xxx_head.py    # 定义各类检测头
  │   ├── roi_extractor.py #检测感兴趣区域提取
  ├── backbones          # 基干网络模块
  │   ├── resnet.py      # ResNet网络
  │   ├── mobilenet.py   # MobileNet网络
  │   │   ...
  ├── losses             # 损失函数模块
  │   ├── xxx_loss.py    # 定义注册各类loss函数
  ├── necks     # 特征融合模块
  │   ├── xxx_fpn.py  # 定义各种FPN模块
  ├── proposal_generator # anchor & proposal生成与匹配模块
  │   ├── anchor_generator.py   # anchor生成模块
  │   ├── proposal_generator.py # proposal生成模块
  │   ├── target.py   # anchor & proposal的匹配函数
  │   ├── target_layer.py   # anchor & proposal的匹配模块
  ├── tests  # 单元测试模块
  │   ├── test_xxx.py  # 对网络中的算子以及模块结构进行单元测试
  ├── ops.py  # 封装各类PaddlePaddle物体检测相关公共检测组件/算子
  ├── layers.py  # 封装及注册各类PaddlePaddle物体检测相关公共检测组件/算子
  ├── bbox_utils.py # 封装检测框相关的函数
  ├── post_process.py # 封装及注册后处理相关模块
  ├── shape_spec.py # 定义模块输出shape的类

2.新增模型

接下来，以单阶段检测器YOLOv3为例，对建立模型过程进行详细描述，按照此思路您可以快速搭建新的模型。

2.1新增网络结构

2.1.1新增Backbone

PaddleDetection中现有所有Backbone网络代码都放置在ppdet/modeling/backbones目录下，所以我们在其中新建darknet.py如下：

python

import paddle.nn as nn
from ppdet.core.workspace import register, serializable

@register
@serializable
class DarkNet(nn.Layer):

    __shared__ = ['norm_type']

    def __init__(self,
                 depth=53,
                 return_idx=[2, 3, 4],
                 norm_type='bn',
                 norm_decay=0.):
        super(DarkNet, self).__init__()
        # 省略内容

    def forward(self, inputs):
        # 省略处理逻辑
        pass

    @property
    def out_shape(self):
        # 省略内容
        pass

然后在backbones/__init__.py中加入引用：

python

from . import darknet
from .darknet import *

几点说明：

为了在yaml配置文件中灵活配置网络，所有Backbone需要利用ppdet.core.workspace里的register进行注册，形式请参考如上示例。此外，可以使用serializable以使backbone支持序列化；
所有的Backbone需继承paddle.nn.Layer类，并实现forward函数。此外，还需实现out_shape属性定义输出的feature map的channel信息，具体可参见源码；
__shared__为了实现一些参数的配置全局共享，这些参数可以被backbone, neck，head，loss等所有注册模块共享。

2.1.2新增Neck

特征融合模块放置在ppdet/modeling/necks目录下，我们在其中新建yolo_fpn.py如下：

python

import paddle.nn as nn
from ppdet.core.workspace import register, serializable

@register
@serializable
class YOLOv3FPN(nn.Layer):
    __shared__ = ['norm_type']

    def __init__(self,
                in_channels=[256, 512, 1024],
                norm_type='bn'):
        super(YOLOv3FPN, self).__init__()
        # 省略内容

    def forward(self, blocks):
        # 省略内容
        pass

    @classmethod
    def from_config(cls, cfg, input_shape):
        # 省略内容
        pass

    @property
    def out_shape(self):
        # 省略内容
        pass

然后在necks/__init__.py中加入引用：

python

from . import yolo_fpn
from .yolo_fpn import *

几点说明：

neck模块需要使用register进行注册，可以使用serializable进行序列化；
neck模块需要继承paddle.nn.Layer类，并实现forward函数。除此之外，还需要实现out_shape属性，用于定义输出的feature map的channel信息，还需要实现类函数from_config用于在配置文件中推理出输入channel，并用于YOLOv3FPN的初始化；
neck模块可以使用__shared__实现一些参数的配置全局共享。

2.1.3新增Head

Head模块全部存放在ppdet/modeling/heads目录下，我们在其中新建yolo_head.py如下

python

import paddle.nn as nn
from ppdet.core.workspace import register

@register
class YOLOv3Head(nn.Layer):
    __shared__ = ['num_classes']
    __inject__ = ['loss']

    def __init__(self,
                 anchors=[[10, 13], [16, 30], [33, 23],
                   [30, 61], [62, 45],[59, 119],
                   [116, 90], [156, 198], [373, 326]],
                 anchor_masks=[[6, 7, 8], [3, 4, 5], [0, 1, 2]],
                 num_classes=80,
                 loss='YOLOv3Loss',
                 iou_aware=False,
                 iou_aware_factor=0.4):
        super(YOLOv3Head, self).__init__()
        # 省略内容

    def forward(self, feats, targets=None):
        # 省略内容
        pass

然后在heads/__init__.py中加入引用：

python

from . import yolo_head
from .yolo_head import *

几点说明：

Head模块需要使用register进行注册；
Head模块需要继承paddle.nn.Layer类，并实现forward函数。
__inject__表示引入全局字典中已经封装好的模块。如loss等。

2.1.4新增Loss

Loss模块全部存放在ppdet/modeling/losses目录下，我们在其中新建yolo_loss.py下

python

import paddle.nn as nn
from ppdet.core.workspace import register

@register
class YOLOv3Loss(nn.Layer):

    __inject__ = ['iou_loss', 'iou_aware_loss']
    __shared__ = ['num_classes']

    def __init__(self,
                 num_classes=80,
                 ignore_thresh=0.7,
                 label_smooth=False,
                 downsample=[32, 16, 8],
                 scale_x_y=1.,
                 iou_loss=None,
                 iou_aware_loss=None):
        super(YOLOv3Loss, self).__init__()
        # 省略内容

    def forward(self, inputs, targets, anchors):
        # 省略内容
        pass

然后在losses/__init__.py中加入引用：

python

from . import yolo_loss
from .yolo_loss import *

几点说明：

loss模块需要使用register进行注册；
loss模块需要继承paddle.nn.Layer类，并实现forward函数。
可以使用__inject__表示引入全局字典中已经封装好的模块，使用__shared__可以实现一些参数的配置全局共享。

2.1.5新增后处理模块

后处理模块定义在ppdet/modeling/post_process.py中，其中定义了BBoxPostProcess类来进行后处理操作，如下所示：

python

from ppdet.core.workspace import register

@register
class BBoxPostProcess(object):
    __shared__ = ['num_classes']
    __inject__ = ['decode', 'nms']

    def __init__(self, num_classes=80, decode=None, nms=None):
        # 省略内容
        pass

    def __call__(self, head_out, rois, im_shape, scale_factor):
        # 省略内容
        pass

几点说明：

后处理模块需要使用register进行注册
__inject__注入了全局字典中封装好的模块，如decode和nms等。decode和nms定义在ppdet/modeling/layers.py中。

2.1.6新增Architecture

所有architecture网络代码都放置在ppdet/modeling/architectures目录下，meta_arch.py中定义了BaseArch类，代码如下：

python

import paddle.nn as nn
from ppdet.core.workspace import register

@register
class BaseArch(nn.Layer):
     def __init__(self):
        super(BaseArch, self).__init__()

    def forward(self, inputs):
        self.inputs = inputs
        self.model_arch()

        if self.training:
            out = self.get_loss()
        else:
            out = self.get_pred()
        return out

    def model_arch(self, ):
        pass

    def get_loss(self, ):
        raise NotImplementedError("Should implement get_loss method!")

    def get_pred(self, ):
        raise NotImplementedError("Should implement get_pred method!")

所有的architecture需要继承BaseArch类，如yolo.py中的YOLOv3定义如下：

python

@register
class YOLOv3(BaseArch):
    __category__ = 'architecture'
    __inject__ = ['post_process']

    def __init__(self,
                 backbone='DarkNet',
                 neck='YOLOv3FPN',
                 yolo_head='YOLOv3Head',
                 post_process='BBoxPostProcess'):
        super(YOLOv3, self).__init__()
        self.backbone = backbone
        self.neck = neck
        self.yolo_head = yolo_head
        self.post_process = post_process

    @classmethod
    def from_config(cls, cfg, *args, **kwargs):
        # 省略内容
        pass

    def get_loss(self):
        # 省略内容
        pass

    def get_pred(self):
        # 省略内容
        pass

几点说明：

所有的architecture需要使用register进行注册
在组建一个完整的网络时必须要设定__category__ = 'architecture'来表示一个完整的物体检测模型；
backbone, neck, yolo_head以及post_process等检测组件传入到architecture中组成最终的网络。像这样将检测模块化，提升了检测模型的复用性，可以通过组合不同的检测组件得到多个模型。
from_config类函数实现了模块间组合时channel的自动配置。

2.2新增配置文件

2.2.1网络结构配置文件

上面详细地介绍了如何新增一个architecture，接下来演示如何配置一个模型，yolov3关于网络结构的配置在configs/yolov3/_base_/文件夹中定义，如yolov3_darknet53.yml定义了yolov3_darknet的网络结构，其定义如下：

text

architecture: YOLOv3
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/DarkNet53_pretrained.pdparams
norm_type: sync_bn

YOLOv3:
  backbone: DarkNet
  neck: YOLOv3FPN
  yolo_head: YOLOv3Head
  post_process: BBoxPostProcess

DarkNet:
  depth: 53
  return_idx: [2, 3, 4]

# use default config
# YOLOv3FPN:

YOLOv3Head:
  anchors: [[10, 13], [16, 30], [33, 23],
            [30, 61], [62, 45], [59, 119],
            [116, 90], [156, 198], [373, 326]]
  anchor_masks: [[6, 7, 8], [3, 4, 5], [0, 1, 2]]
  loss: YOLOv3Loss

YOLOv3Loss:
  ignore_thresh: 0.7
  downsample: [32, 16, 8]
  label_smooth: false

BBoxPostProcess:
  decode:
    name: YOLOBox
    conf_thresh: 0.005
    downsample_ratio: 32
    clip_bbox: true
  nms:
    name: MultiClassNMS
    keep_top_k: 100
    score_threshold: 0.01
    nms_threshold: 0.45
    nms_top_k: 1000

可以看到在配置文件中，首先需要指定网络的architecture，pretrain_weights指定训练模型的url或者路径，norm_type等可以作为全局参数共享。模型的定义自上而下依次在文件中定义，与上节中的模型组件一一对应。对于一些模型组件，如果采用默认
的参数，可以不用配置，如上文中的yolo_fpn。通过改变相关配置，我们可以轻易地组合出另一个模型，比如configs/yolov3/_base_/yolov3_mobilenet_v1.yml将backbone从Darknet切换成MobileNet。

2.2.2优化器配置文件

优化器配置文件定义模型使用的优化器以及学习率的调度策略，目前PaddleDetection中已经集成了多种多样的优化器和学习率策略，具体可参见代码ppdet/optimizer.py。比如，yolov3的优化器配置文件定义在configs/yolov3/_base_/optimizer_270e.yml，其定义如下：

text

epoch: 270

LearningRate:
  base_lr: 0.001
  schedulers:
  - !PiecewiseDecay
    gamma: 0.1
    milestones:
    # epoch数目
    - 216
    - 243
  - !LinearWarmup
    start_factor: 0.
    steps: 4000

OptimizerBuilder:
  optimizer:
    momentum: 0.9
    type: Momentum
  regularizer:
    factor: 0.0005
    type: L2

几点说明：

可以通过OptimizerBuilder.optimizer指定优化器的类型及参数，目前支持的优化器可以参考PaddlePaddle官方文档
可以设置LearningRate.schedulers设置不同学习率调整策略的组合，PaddlePaddle目前支持多种学习率调整策略，具体也可参考PaddlePaddle官方文档。需要注意的是，你需要对于PaddlePaddle中的学习率调整策略进行简单的封装，具体可参考源码ppdet/optimizer.py。

2.2.3Reader配置文件

关于Reader的配置可以参考Reader配置文档。

看过此文档，您应该对PaddleDetection中模型搭建与配置有了一定经验，结合源码会理解的更加透彻。关于模型技术，如您有其他问题或建议，请给我们提issue，我们非常欢迎您的反馈。

Advanced Tutorials MODEL TECHNICAL En (`docs/advanced_tutorials/MODEL_TECHNICAL_en.md`)

How to Create Model Algorithm

In order to make better use of PaddleDetection, we will introduce the main model technical details and application of PaddleDetection in this document

数据处理模块

ppdet/data/
  ├── reader.py     # 基于Dataloader封装的Reader模块
  ├── source  # 数据源管理模块
  │   ├── dataset.py      # 定义数据源基类，各类数据集继承于此
  │   ├── coco.py         # COCO数据集解析与格式化数据
  │   ├── voc.py          # Pascal VOC数据集解析与格式化数据
  │   ├── widerface.py    # WIDER-FACE数据集解析与格式化数据
  │   ├── category.py    # 相关数据集的类别信息
  ├── transform  # 数据预处理模块
  │   ├── batch_operators.py  # 定义各类基于批量数据的预处理算子
  │   ├── op_helper.py    # 预处理算子的辅助函数
  │   ├── operators.py    # 定义各类基于单张图片的预处理算子
  │   ├── gridmask_utils.py    # GridMask数据增强函数
  │   ├── autoaugment_utils.py  # AutoAugment辅助函数
  ├── shm_utils.py     # 用于使用共享内存的辅助函数

2.数据集

数据集定义在source目录下，其中dataset.py中定义了数据集的基类DetDataSet, 所有的数据集均继承于基类，DetDataset基类里定义了如下等方法：

方法	输入	输出	备注
__len__	无	int, 数据集中样本的数量	过滤掉了无标注的样本
__getitem__	int, 样本的索引idx	dict, 索引idx对应的样本roidb	得到transform之后的样本roidb
check_or_download_dataset	无	无	检查数据集是否存在，如果不存在则下载，目前支持COCO, VOC，widerface等数据集
set_kwargs	可选参数，以键值对的形式给出	无	目前用于支持接收mixup, cutmix等参数的设置
set_transform	一系列的transform函数	无	设置数据集的transform函数
set_epoch	int, 当前的epoch	无	用于dataset与训练过程的交互
parse_dataset	无	无	用于从数据中读取所有的样本
get_anno	无	无	用于获取标注文件的路径

当一个数据集类继承自DetDataSet，那么它只需要实现parse_dataset函数即可。parse_dataset根据数据集设置的数据集根路径dataset_dir，图片文件夹image_dir，标注文件路径anno_path取出所有的样本，并将其保存在一个列表roidbs中，每一个列表中的元素为一个样本xxx_rec(比如coco_rec或者voc_rec)，用dict表示，dict中包含样本的image, gt_bbox, gt_class等字段。COCO和Pascal-VOC数据集中的xxx_rec的数据结构定义如下：

python

xxx_rec = {
    'im_file': im_fname,         # 一张图像的完整路径
    'im_id': np.array([img_id]), # 一张图像的ID序号
    'h': im_h,                   # 图像高度
    'w': im_w,                   # 图像宽度
    'is_crowd': is_crowd,        # 是否是群落对象, 默认为0 (VOC中无此字段)
    'gt_class': gt_class,        # 标注框标签名称的ID序号
    'gt_bbox': gt_bbox,          # 标注框坐标(xmin, ymin, xmax, ymax)
    'gt_poly': gt_poly,          # 分割掩码，此字段只在coco_rec中出现，默认为None
    'difficult': difficult       # 是否是困难样本，此字段只在voc_rec中出现，默认为0
}

xxx_rec中的内容也可以通过DetDataSet的data_fields参数来控制，即可以过滤掉一些不需要的字段，但大多数情况下不需要修改，按照configs/datasets中的默认配置即可。

此外，在parse_dataset函数中，保存了类别名到id的映射的一个字典cname2cid。在coco数据集中，会利用COCO API从标注文件中加载数据集的类别名，并设置此字典。在voc数据集中，如果设置use_default_label=False，将从label_list.txt中读取类别列表，反之将使用voc默认的类别列表。

2.1COCO数据集

COCO数据集目前分为COCO2014和COCO2017，主要由json文件和image文件组成，其组织结构如下所示：

text

dataset/coco/
├── annotations
│   ├── instances_train2014.json
│   ├── instances_train2017.json
│   ├── instances_val2014.json
│   ├── instances_val2017.json
│   │   ...
├── train2017
│   ├── 000000000009.jpg
│   ├── 000000580008.jpg
│   │   ...
├── val2017
│   ├── 000000000139.jpg
│   ├── 000000000285.jpg
│   │   ...

在source/coco.py中定义并注册了COCODataSet数据集类，其继承自DetDataSet，并实现了parse_dataset方法，调用COCO API加载并解析COCO格式数据源roidbs和cname2cid，具体可参见source/coco.py源码。将其他数据集转换成COCO格式可以参考用户数据转成COCO数据

2.2Pascal VOC数据集

该数据集目前分为VOC2007和VOC2012，主要由xml文件和image文件组成，其组织结构如下所示：

text

dataset/voc/
  ├── trainval.txt
  ├── test.txt
  ├── label_list.txt (optional)
  ├── VOCdevkit/VOC2007
  │   ├── Annotations
  │       ├── 001789.xml
  │       │   ...
  │   ├── JPEGImages
  │       ├── 001789.jpg
  │       │   ...
  │   ├── ImageSets
  │       |   ...
  ├── VOCdevkit/VOC2012
  │   ├── Annotations
  │       ├── 2011_003876.xml
  │       │   ...
  │   ├── JPEGImages
  │       ├── 2011_003876.jpg
  │       │   ...
  │   ├── ImageSets
  │       │   ...

在source/voc.py中定义并注册了VOCDataSet数据集，它继承自DetDataSet基类，并重写了parse_dataset方法，解析VOC数据集中xml格式标注文件，更新roidbs和cname2cid。将其他数据集转换成VOC格式可以参考用户数据转成VOC数据

2.3自定义数据集

如果COCODataSet和VOCDataSet不能满足你的需求，可以通过自定义数据集的方式来加载你的数据集。只需要以下两步即可实现自定义数据集

新建source/xxx.py，定义类XXXDataSet继承自DetDataSet基类，完成注册与序列化，并重写parse_dataset方法对roidbs与cname2cid更新：

python

from ppdet.core.workspace import register, serializable

#注册并序列化
@register
@serializable
class XXXDataSet(DetDataSet):
    def __init__(self,
                dataset_dir=None,
                image_dir=None,
                anno_path=None,
                ...
                ):
        self.roidbs = None
        self.cname2cid = None
        ...

    def parse_dataset(self):
        ...
        省略具体解析数据逻辑
        ...
        self.roidbs, self.cname2cid = records, cname2cid

在source/__init__.py中添加引用：

python

from . import xxx
from .xxx import *

完成以上两步就将新的数据源XXXDataSet添加好了，你可以参考配置及运行实现自定义数据集的使用。

3.数据预处理

3.1数据增强算子

PaddleDetection中支持了种类丰富的数据增强算子，有单图像数据增强算子与批数据增强算子两种方式，您可选取合适的算子组合使用。单图像数据增强算子定义在transform/operators.py中，已支持的单图像数据增强算子详见下表：

名称	作用
Decode	从图像文件或内存buffer中加载图像，格式为RGB格式
Permute	假如输入是HWC顺序变成CHW
RandomErasingImage	对图像进行随机擦除
NormalizeImage	对图像像素值进行归一化，如果设置is_scale=True，则先将像素值除以255.0, 再进行归一化。
GridMask	GridMask数据增广
RandomDistort	随机扰动图片亮度、对比度、饱和度和色相
AutoAugment	AutoAugment数据增广，包含一系列数据增强方法
RandomFlip	随机水平翻转图像
Resize	对于图像进行resize，并对标注进行相应的变换
MultiscaleTestResize	将图像重新缩放为多尺度list的每个尺寸
RandomResize	对于图像进行随机Resize，可以Resize到不同的尺寸以及使用不同的插值策略
RandomExpand	将原始图片放入用像素均值填充的扩张图中，对此图进行裁剪、缩放和翻转
CropWithSampling	根据缩放比例、长宽比例生成若干候选框，再依据这些候选框和标注框的面积交并比(IoU)挑选出符合要求的裁剪结果
CropImageWithDataAchorSampling	基于CropImage，在人脸检测中，随机将图片尺度变换到一定范围的尺度，大大增强人脸的尺度变化
RandomCrop	原理同CropImage，以随机比例与IoU阈值进行处理
RandomScaledCrop	根据长边对图像进行随机裁剪，并对标注做相应的变换
Cutmix	Cutmix数据增强，对两张图片做拼接
Mixup	Mixup数据增强，按比例叠加两张图像
NormalizeBox	对bounding box进行归一化
PadBox	如果bounding box的数量少于num_max_boxes，则将零填充到bbox
BboxXYXY2XYWH	将bounding box从(xmin,ymin,xmax,ymin)形式转换为(xmin,ymin,width,height)格式
Pad	将图片Pad某一个数的整数倍或者指定的size，并支持指定Pad的方式
Poly2Mask	Poly2Mask数据增强｜

批数据增强算子定义在transform/batch_operators.py中, 目前支持的算子列表如下：

名称	作用
PadBatch	随机对每个batch的数据图片进行Pad操作，使得batch中的图片具有相同的shape
BatchRandomResize	对一个batch的图片进行resize，使得batch中的图片随机缩放到相同的尺寸
Gt2YoloTarget	通过gt数据生成YOLO系列模型的目标
Gt2FCOSTarget	通过gt数据生成FCOS模型的目标
Gt2TTFTarget	通过gt数据生成TTFNet模型的目标
Gt2Solov2Target	通过gt数据生成SOLOv2模型的目标

几点说明：

数据增强算子的输入为sample或者samples，每一个sample对应上文所说的DetDataSet输出的roidbs中的一个样本，如coco_rec或者voc_rec
单图像数据增强算子(Mixup, Cutmix等除外)也可用于批数据处理中。但是，单图像处理算子和批图像处理算子仍有一些差异，以RandomResize和BatchRandomResize为例，RandomResize会将一个Batch中的每张图片进行随机缩放，但是每一张图像Resize之后的形状不尽相同，BatchRandomResize则会将一个Batch中的所有图片随机缩放到相同的形状。
除BatchRandomResize外，定义在transform/batch_operators.py的批数据增强算子接收的输入图像均为CHW形式，所以使用这些批数据增强算子前请先使用Permute进行处理。如果用到Gt2xxxTarget算子，需要将其放置在靠后的位置。NormalizeBox算子建议放置在Gt2xxxTarget之前。将这些限制条件总结下来，推荐的预处理算子的顺序为
text
```
- XXX: {}
  - ...
  - BatchRandomResize: {...} # 如果不需要，可以移除，如果需要，放置在Permute之前
  - Permute: {} # 必须项
  - NormalizeBox: {} # 如果需要，建议放在Gt2XXXTarget之前
  - PadBatch: {...} # 如果不需要可移除，如果需要，建议放置在Permute之后
  - Gt2XXXTarget: {...} # 建议与PadBatch放置在最后的位置
```

3.2自定义数据增强算子

如果需要自定义数据增强算子，那么您需要了解下数据增强算子的相关逻辑。数据增强算子基类为定义在transform/operators.py中的BaseOperator类，单图像数据增强算子与批数据增强算子均继承自这个基类。完整定义参考源码，以下代码显示了BaseOperator类的关键函数: apply和__call__方法

python

class BaseOperator(object):

  ...

  def apply(self, sample, context=None):
      return sample

  def __call__(self, sample, context=None):
      if isinstance(sample, Sequence):
          for i in range(len(sample)):
              sample[i] = self.apply(sample[i], context)
      else:
          sample = self.apply(sample, context)
      return sample

__call__方法为BaseOperator的调用入口，接收一个sample(单图像)或者多个sample(多图像)作为输入，并调用apply函数对一个或者多个sample进行处理。大多数情况下，你只需要继承BaseOperator重写apply方法或者重写__call__方法即可，如下所示，定义了一个XXXOp继承自BaseOperator，并注册：

python

@register_op
class XXXOp(BaseOperator):
  def __init__(self,...):

    super(XXXImage, self).__init__()
    ...

  # 大多数情况下只需要重写apply方法
  def apply(self, sample, context=None):
    ...
    省略对输入的sample具体操作
    ...
    return sample

  # 如果有需要，可以重写__call__方法，如Mixup, Gt2XXXTarget等
  # def __call__(self, sample, context=None):
  #   ...
  #   省略对输入的sample具体操作
  #   ...
  #   return sample

大多数情况下，只需要重写apply方法即可，如transform/operators.py中除Mixup和Cutmix外的预处理算子。对于批处理的情况一般需要重写__call__方法，如transform/batch_operators.py的预处理算子。

4.Reader

Reader相关的类定义在reader.py, 其中定义了BaseDataLoader类。BaseDataLoader在paddle.io.DataLoader的基础上封装了一层，其具备paddle.io.DataLoader的所有功能，并能够实现不同模型对于DetDataset的不同需求，如可以通过对Reader进行设置，以控制DetDataset支持Mixup, Cutmix等操作。除此之外，数据预处理算子通过Compose类和BatchCompose类组合起来分别传入DetDataset和paddle.io.DataLoader中。
所有的Reader类都继承自BaseDataLoader类，具体可参见源码。

5.配置及运行

5.1 配置

与数据预处理相关的模块的配置文件包含所有模型公用的Dataset的配置文件，以及不同模型专用的Reader的配置文件。

5.1.1 Dataset配置

关于Dataset的配置文件存在于configs/datasets文件夹。比如COCO数据集的配置文件如下：

text

metric: COCO # 目前支持COCO, VOC, OID， WiderFace等评估标准
num_classes: 80 # num_classes数据集的类别数，不包含背景类

TrainDataset:
  !COCODataSet
    image_dir: train2017 # 训练集的图片所在文件相对于dataset_dir的路径
    anno_path: annotations/instances_train2017.json # 训练集的标注文件相对于dataset_dir的路径
    dataset_dir: dataset/coco #数据集所在路径，相对于PaddleDetection路径
    data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd'] # 控制dataset输出的sample所包含的字段，注意此为TrainDataset独有的且必须配置的字段

EvalDataset:
  !COCODataSet
    image_dir: val2017 # 验证集的图片所在文件夹相对于dataset_dir的路径
    anno_path: annotations/instances_val2017.json # 验证集的标注文件相对于dataset_dir的路径
    dataset_dir: dataset/coco # 数据集所在路径，相对于PaddleDetection路径

TestDataset:
  !ImageFolder
    anno_path: annotations/instances_val2017.json # 标注文件所在路径，仅用于读取数据集的类别信息，支持json和txt格式
    dataset_dir: dataset/coco # 数据集所在路径，若添加了此行，则`anno_path`路径为`dataset_dir/anno_path`，若此行不设置或去掉此行，则`anno_path`路径即为`anno_path`

在PaddleDetection的yml配置文件中，使用!直接序列化模块实例(可以是函数，实例等)，上述的配置文件均使用Dataset进行了序列化。

注意：
请运行前自行仔细检查数据集的配置路径，在训练或验证时如果TrainDataset和EvalDataset的路径配置有误，会提示自动下载数据集。若使用自定义数据集，在推理时如果TestDataset路径配置有误，会提示使用默认COCO数据集的类别信息。

5.1.2 Reader配置

不同模型专用的Reader定义在每一个模型的文件夹下，如yolov3的Reader配置文件定义在configs/yolov3/_base_/yolov3_reader.yml。一个Reader的示例配置如下：

text

worker_num: 2
TrainReader:
  sample_transforms:
    - Decode: {}
    ...
  batch_transforms:
    ...
  batch_size: 8
  shuffle: true
  drop_last: true
  use_shared_memory: true

EvalReader:
  sample_transforms:
    - Decode: {}
    ...
  batch_size: 1

TestReader:
  inputs_def:
    image_shape: [3, 608, 608]
  sample_transforms:
    - Decode: {}
    ...
  batch_size: 1

你可以在Reader中定义不同的预处理算子，每张卡的batch_size以及DataLoader的worker_num等。

5.2运行

在PaddleDetection的训练、评估和测试运行程序中，都通过创建Reader迭代器。Reader在ppdet/engine/trainer.py中创建。下面的代码展示了如何创建训练时的Reader

python

from ppdet.core.workspace import create
# build data loader
self.dataset = cfg['TrainDataset']
self.loader = create('TrainReader')(selfdataset, cfg.worker_num)

相应的预测以及评估时的Reader与之类似，具体可参考ppdet/engine/trainer.py源码。

关于数据处理模块，如您有其他问题或建议，请给我们提issue，我们非常欢迎您的反馈。

Advanced Tutorials READER En (`docs/advanced_tutorials/READER_en.md`)

Data Processing Module

Method	Input	Output	Note
__len__	no	int, the number of samples in the dataset	Filter out the unlabeled samples
__getitem__	int, The index of the sample	dict, Index idx to sample ROIDB	Get the sample roidb after transform
check_or_download_dataset	no	no	Check whether the dataset exists, if not, download, currently support COCO, VOC, Widerface and other datasets
set_kwargs	Optional arguments, given as key-value pairs	no	Currently used to support receiving mixup, cutMix and other parameters
set_transform	A series of transform functions	no	Set the transform function of the dataset
set_epoch	int, current epoch	no	Interaction between dataset and training process
parse_dataset	no	no	Used to read all samples from the data
get_anno	no	no	Used to get the path to the annotation file

Name	Function
Decode	Loads an image from an image file or memory buffer in RGB format
Permute	If the input is HWC, the sequence changes to CHW
RandomErasingImage	Random erasure of the image
NormalizeImage	The pixel value of the image is normalized. If is scale= True is set, the pixel value is divided by 255.0 before normalization.
GridMask	GridMask data is augmented
RandomDistort	Random disturbance of image brightness, contrast, saturation and hue
AutoAugment	Auto Augment data, which contains a series of data augmentation methods
RandomFlip	Randomly flip the image horizontally
Resize	Resize the image and transform the annotation accordingly
MultiscaleTestResize	Rescale the image to each size of the multi-scale list
RandomResize	Random Resize of images can be resized to different sizes and different interpolation strategies can be used
RandomExpand	Place the original image into an expanded image filled with pixel mean, crop, scale, and flip the image
CropWithSampling	Several candidate frames are generated according to the scaling ratio and length-width ratio, and then the prunning results that meet the requirements are selected according to the area intersection ratio (IoU) between these candidate frames and the marking frames
CropImageWithDataAchorSampling	Based on Crop Image, in face detection, the Image scale is randomly transformed to a certain range of scale, which greatly enhances the scale change of face
RandomCrop	The principle is the same as CropImage, which is processed with random proportion and IoU threshold
RandomScaledCrop	According to the long edge, the image is randomly clipped and the corresponding transformation is made to the annotations
Cutmix	Cutmix data enhancement, Mosaic of two images
Mixup	Mixup data enhancement to scale up two images
NormalizeBox	Bounding box is normalized
PadBox	If the number of bounding boxes is less than num Max boxes, zero is populated into bboxes
BboxXYXY2XYWH	Bounding Box is converted from (xmin,ymin,xmax,ymin) form to (xmin,ymin, Width,height) form
Pad	The image Pad is an integer multiple of a certain number or the specified size, and supports the way of specifying Pad
Poly2Mask	Poly2Mask data enhancement ｜

Name	Function
PadBatch	Pad operation is performed on each batch of data images randomly to make the images in the batch have the same shape
BatchRandomResize	Resize a batch of images so that the images in the batch are randomly scaled to the same size
Gt2YoloTarget	Generate the objectives of YOLO series models from GT data
Gt2FCOSTarget	Generate the target of the FCOS model from GT data
Gt2TTFTarget	Generate TTF Net model targets from GT data
Gt2Solov2Target	Generate targets for SOL Ov2 models from GT data

[Contribute to PaddleDetection] Yes, PP-YOLOE! 基于PP-YOLOE的算法开发

本期活动联系人：thinkthinking

建设目标

PP-YOLOE+是百度飞桨团队开源的最新SOTA通用检测模型，COCO数据集精度达54.7mAP，其L版本相比YOLOv7精度提升1.9%，V100端到端(包含前后处理)推理速度达42.2FPS。

我们鼓励大家基于PP-YOLOE去做新的算法开发，比如：

改造PP-YOLOE适用于旋转框、小目标、关键点检测、实例分割等场景；
精调PP-YOLOE用于工业质检、火灾检测、垃圾检测等垂类场景；
将PP-YOLOE用于PP-Human、PP-Vehicle等Pipeline中，提升pipeline的检测效果。

相信通过这些活动，大家可以对PP-YOLOE的细节有更深刻的理解，对业务场景的应用也可以做更细节的适配。

参与方式

方式一：列表选题，见招募列表（提供了选题方向、题目、优秀的对标项目、文章和代码，以供学习）。
方式二：自选题目，对于非参考列表内的题目，可自主命题，需要与负责人 thinkthinking讨论后决定题目。

题目认领

为避免重复选题、知晓任务状态、方便统计管理，请根据如下操作认领您的题目。

在本issue提交题目:issue

方式一（列表选题）：在“招募列表”中选择题目，并在issue中，回复下列信息：

text

【列表选题】
编号：XX
题目：XXXX
认领人：XX

方式二（自选题目）：自主命题，直接在 issue 中，回复下列信息：

text

【自选题目】
题目：XXXX
认领人：XX

招募列表

序号	类型	题目	难度	参考	认领人
01	模型改造	PP-YOLOE用于旋转框检测	高	https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.5/configs/rotate	----
02	模型改造	PP-YOLOE用于小目标检测	高	https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.5/configs/smalldet	----
03	模型改造	PP-YOLOE用于关键点检测	高	https://github.com/WongKinYiu/yolov7/tree/pose	----
04	模型改造	PP-YOLOE用于实例分割	高	https://github.com/WongKinYiu/yolov7/tree/mask	----
05	垂类应用	基于PP-YOLOE的缺陷检测	中	https://aistudio.baidu.com/aistudio/projectdetail/2367089	----
06	垂类应用	基于PP-YOLOE的行为检测	中	https://aistudio.baidu.com/aistudio/projectdetail/2500639	----
07	垂类应用	基于PP-YOLOE的异物检测	中	https://aistudio.baidu.com/aistudio/projectdetail/3846170?channelType=0&channel=0	----
08	垂类应用	基于PP-YOLOE的安全监测	中	https://aistudio.baidu.com/aistudio/projectdetail/2503301?channelType=0&channel=0	----
09	Pipeline	PP-YOLOE-->PP-Human大升级	中	https://aistudio.baidu.com/aistudio/projectdetail/4606001	----
10	Pipeline	PP-YOLOE-->PP-Vehicle大升级	中	https://aistudio.baidu.com/aistudio/projectdetail/4512254	----

【注意】招募列表外的，欢迎开发者联系活动负责人thinkthinking提交贡献👏

贡献指南

提ISSUE、PR的步骤请参考飞桨官网-贡献指南-代码贡献流程
AI-Studio使用指南请参考AI-Studio新手指南

原则及注意事项

需使用PaddlePaddle框架, 建议复用PaddleDetection代码。
建议使用Paddle框架最新版本.
PR需提到PaddleDetection-develop分支。
模型改造类的任务建议以PR形式提交
垂类应用以及Pipeline类的任务建议以AI-Studio项目形式提交，项目会同步到产业范例页面

还有不清楚的问题

欢迎大家随时在本issue下提问，飞桨会有专门的管理员进行疑问解答。

有任何问题，请联系本期活动联系人 thinkthinking

非常感谢大家为飞桨贡献！共建飞桨繁荣社区！

Feature Models PaddleYOLO MODEL (`docs/feature_models/PaddleYOLO_MODEL.md`)

简体中文 | English

[PaddleYOLO](https://github.com/PaddlePaddle/PaddleYOLO)

内容

PaddleYOLO
- 内容
- 简介
- 更新日志
- 模型库
  - PP-YOLOE
  - YOLOX
  - YOLOv5
  - YOLOv6
  - YOLOv7
  - YOLOv8
  - RTMDet
  - 注意:
  - VOC
- 使用指南
  - 一键运行全流程
  - 自定义数据集

简介

PaddleYOLO是基于PaddleDetection的YOLO系列模型库，只包含YOLO系列模型的相关代码，支持YOLOv3,PP-YOLO,PP-YOLOv2,PP-YOLOE,PP-YOLOE+,YOLOX,YOLOv5,YOLOv6,YOLOv7,YOLOv8,RTMDet等模型，欢迎一起使用和建设！

更新日志

【2022/01/10】支持YOLOv8预测和部署；
【2022/09/29】支持RTMDet预测和部署；
【2022/09/26】发布PaddleYOLO模型套件；
【2022/09/19】支持YOLOv6新版，包括n/t/s/m/l模型；
【2022/08/23】发布YOLOSeries代码库: 支持YOLOv3,PP-YOLOE,PP-YOLOE+,YOLOX,YOLOv5,YOLOv6,YOLOv7等YOLO模型，支持ConvNeXt骨干网络高精度版PP-YOLOE,YOLOX和YOLOv5等模型，支持PaddleSlim无损加速量化训练PP-YOLOE,YOLOv5,YOLOv6和YOLOv7等模型，详情可阅读此文章；

注意:

PaddleYOLO代码库协议为GPL 3.0，YOLOv5,YOLOv6,YOLOv7和YOLOv8这几类模型代码不合入PaddleDetection，其余YOLO模型推荐在PaddleDetection中使用，会最先发布PP-YOLO系列特色检测模型的最新进展；；
PaddleYOLO代码库推荐使用paddlepaddle-2.3.2以上的版本，请参考官网下载对应适合版本，Windows平台请安装paddle develop版本；
PaddleYOLO 的Roadmap issue用于收集用户的需求，欢迎提出您的建议和需求。
训练自定义数据集请参照文档和issue。请首先确保加载了COCO权重作为预训练，YOLO检测模型建议总batch_size至少大于64去训练，如果资源不够请换小模型或减小模型的输入尺度，为了保障较高检测精度，尽量不要尝试单卡训和总batch_size小于32训；

模型库

[PP-YOLOE](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/ppyoloe)

基础模型

网络模型	输入尺寸	图片数/GPU	学习率策略	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	下载链接	配置文件
PP-YOLOE-s	640	32	400e	2.9	43.4	60.0	7.93	17.36	model	config
PP-YOLOE-s	640	32	300e	2.9	43.0	59.6	7.93	17.36	model	config
PP-YOLOE-m	640	28	300e	6.0	49.0	65.9	23.43	49.91	model	config
PP-YOLOE-l	640	20	300e	8.7	51.4	68.6	52.20	110.07	model	config
PP-YOLOE-x	640	16	300e	14.9	52.3	69.5	98.42	206.59	model	config
PP-YOLOE-tiny ConvNeXt	640	16	36e	-	44.6	63.3	33.04	13.87	model	config
PP-YOLOE+_s	640	8	80e	2.9	43.7	60.6	7.93	17.36	model	config
PP-YOLOE+_m	640	8	80e	6.0	49.8	67.1	23.43	49.91	model	config
PP-YOLOE+_l	640	8	80e	8.7	52.9	70.1	52.20	110.07	model	config
PP-YOLOE+_x	640	8	80e	14.9	54.7	72.0	98.42	206.59	model	config

部署模型

网络模型	输入尺寸	导出后的权重(w/o NMS)	ONNX(w/o NMS)
PP-YOLOE-s(400epoch)	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE+_s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE+_m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE+_l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE+_x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[YOLOX](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolox)

基础模型

网络模型	输入尺寸	图片数/GPU	学习率策略	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	下载链接	配置文件
YOLOX-nano	416	8	300e	2.3	26.1	42.0	0.91	1.08	model	config
YOLOX-tiny	416	8	300e	2.8	32.9	50.4	5.06	6.45	model	config
YOLOX-s	640	8	300e	3.0	40.4	59.6	9.0	26.8	model	config
YOLOX-m	640	8	300e	5.8	46.9	65.7	25.3	73.8	model	config
YOLOX-l	640	8	300e	9.3	50.1	68.8	54.2	155.6	model	config
YOLOX-x	640	8	300e	16.6	51.8	70.6	99.1	281.9	model	config
YOLOX-cdn-tiny	416	8	300e	1.9	32.4	50.2	5.03	6.33	model	config
YOLOX-crn-s	640	8	300e	3.0	40.4	59.6	7.7	24.69	model	config
YOLOX-s ConvNeXt	640	8	36e	-	44.6	65.3	36.2	27.52	model	config

部署模型

网络模型	输入尺寸	导出后的权重(w/o NMS)	ONNX(w/o NMS)
YOLOx-nano	416	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-tiny	416	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[YOLOv5](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov5)

基础模型

网络模型	输入尺寸	图片数/GPU	学习率策略	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	下载链接	配置文件
YOLOv5-n	640	16	300e	2.6	28.0	45.7	1.87	4.52	model	config
YOLOv5-s	640	16	300e	3.2	37.6	56.7	7.24	16.54	model	config
YOLOv5-m	640	16	300e	5.2	45.4	64.1	21.19	49.08	model	config
YOLOv5-l	640	16	300e	7.9	48.9	67.1	46.56	109.32	model	config
YOLOv5-x	640	16	300e	13.7	50.6	68.7	86.75	205.92	model	config
YOLOv5-s ConvNeXt	640	8	36e	-	42.4	65.3	34.54	17.96	model	config
*YOLOv5p6-n	1280	16	300e	-	35.9	54.2	3.25	9.23	model	config
*YOLOv5p6-s	1280	16	300e	-	44.5	63.3	12.63	33.81	model	config
*YOLOv5p6-m	1280	16	300e	-	51.1	69.0	35.73	100.21	model	config
*YOLOv5p6-l	1280	8	300e	-	53.4	71.0	76.77	223.09	model	config
*YOLOv5p6-x	1280	8	300e	-	54.7	72.4	140.80	420.03	model	config

部署模型

网络模型	输入尺寸	导出后的权重(w/o NMS)	ONNX(w/o NMS)
YOLOv5-n	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv5-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv5-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv5-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv5-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[YOLOv6](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov6)

基础模型

网络网络	输入尺寸	图片数/GPU	学习率策略	TRT-FP16-Latency(ms)	mAP	AP50	Params(M)	FLOPs(G)	下载链接	配置文件
*YOLOv6-n	640	16	300e(+300e)	2.0	37.5	53.1	5.07	12.49	model	config
*YOLOv6-s	640	32	300e(+300e)	2.7	44.8	61.7	20.18	49.36	model	config
*YOLOv6-m	640	32	300e(+300e)	-	49.5	66.9	37.74	92.47	model	config
*YOLOv6-l(silu)	640	32	300e(+300e)	-	52.2	70.2	59.66	149.4	model	config

部署模型

网络模型	输入尺寸	导出后的权重(w/o NMS)	ONNX(w/o NMS)
yolov6-n	640	(w/ nms) \| (w/o nms)	(w/ nms) \| (w/o nms)
yolov6-s	640	(w/ nms) \| (w/o nms)	(w/ nms) \| (w/o nms)
yolov6-m	640	(w/ nms) \| (w/o nms)	(w/ nms) \| (w/o nms)
yolov6-l(silu)	640	(w/ nms) \| (w/o nms)	(w/ nms) \| (w/o nms)

[YOLOv7](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov7)

基础模型

网络模型	输入尺寸	图片数/GPU	学习率策略	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	下载链接	配置文件
YOLOv7-L	640	32	300e	7.4	51.0	70.2	37.62	106.08	model	config
*YOLOv7-X	640	32	300e	12.2	53.0	70.8	71.34	190.08	model	config
*YOLOv7P6-W6	1280	16	300e	25.5	54.4	71.8	70.43	360.26	model	config
*YOLOv7P6-E6	1280	10	300e	31.1	55.7	73.0	97.25	515.4	model	config
*YOLOv7P6-D6	1280	8	300e	37.4	56.1	73.3	133.81	702.92	model	config
*YOLOv7P6-E6E	1280	6	300e	48.7	56.5	73.7	151.76	843.52	model	config
YOLOv7-tiny	640	32	300e	-	37.3	54.5	6.23	6.90	model	config
YOLOv7-tiny	416	32	300e	-	33.3	49.5	6.23	2.91	model	config
YOLOv7-tiny	320	32	300e	-	29.1	43.8	6.23	1.73	model	config

部署模型

网络模型	输入尺寸	导出后的权重(w/o NMS)	ONNX(w/o NMS)
YOLOv7-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7P6-W6	1280	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7P6-E6	1280	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7P6-D6	1280	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7P6-E6E	1280	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7-tiny	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7-tiny	416	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7-tiny	320	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[YOLOv8](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov8)

基础模型

网络网络	输入尺寸	图片数/GPU	学习率策略	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	下载链接	配置文件
*YOLOv8-n	640	16	500e	2.4	37.3	53.0	3.16	8.7	model	config
*YOLOv8-s	640	16	500e	3.4	44.9	61.8	11.17	28.6	model	config
*YOLOv8-m	640	16	500e	6.5	50.2	67.3	25.90	78.9	model	config
*YOLOv8-l	640	16	500e	10.0	52.8	69.6	43.69	165.2	model	config
*YOLOv8-x	640	16	500e	15.1	53.8	70.6	68.23	257.8	model	config
*YOLOv8-P6-x	1280	16	500e	55.0	-	-	97.42	522.93	model	config

部署模型

网络模型	输入尺寸	导出后的权重(w/o NMS)	ONNX(w/o NMS)
YOLOv8-n	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv8-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv8-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv8-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv8-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[RTMDet](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/rtmdet)

基础模型

网络网络	输入尺寸	图片数/GPU	学习率策略	TRT-FP16-Latency(ms)	mAP	AP50	Params(M)	FLOPs(G)	下载链接	配置文件
*RTMDet-t	640	32	300e	2.8	40.9	57.9	4.90	16.21	model	config
*RTMDet-s	640	32	300e	3.3	44.5	62.0	8.89	29.71	model	config
*RTMDet-m	640	32	300e	6.4	49.1	66.8	24.71	78.47	model	config
*RTMDet-l	640	32	300e	10.2	51.2	68.8	52.31	160.32	model	config
*RTMDet-x	640	32	300e	18.0	52.6	70.4	94.86	283.12	model	config

部署模型

网络模型	输入尺寸	导出后的权重(w/o NMS)	ONNX(w/o NMS)
RTMDet-t	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
RTMDet-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
RTMDet-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
RTMDet-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
RTMDet-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

注意:

所有模型均使用COCO train2017作为训练集，在COCO val2017上验证精度，模型前带*表示训练更新中。
具体精度和速度细节请查看PP-YOLOE,YOLOX,YOLOv5,YOLOv6,YOLOv7，其中YOLOv5,YOLOv6,YOLOv7评估并未采用multi_label形式。
模型推理耗时(ms)为TensorRT-FP16下测试的耗时，不包含数据预处理和模型输出后处理(NMS)的耗时。测试采用单卡Tesla T4 GPU，batch size=1，测试环境为paddlepaddle-2.3.2, CUDA 11.2, CUDNN 8.2, GCC-8.2, TensorRT 8.0.3.4，具体请参考各自模型主页。
统计FLOPs(G)和Params(M)，首先安装PaddleSlim, pip install paddleslim，然后设置runtime.yml里print_flops: True和print_params: True，并且注意确保是单尺度下如640x640，打印的是MACs，FLOPs=2*MACs。
各模型导出后的权重以及ONNX，分为带(w)和不带(wo)后处理NMS，都提供了下载链接，请参考各自模型主页下载。w_nms表示带NMS后处理，可以直接使用预测出最终检测框结果如python deploy/python/infer.py --model_dir=ppyoloe_crn_l_300e_coco_w_nms/ --image_file=demo/000000014439.jpg --device=GPU；wo_nms表示不带NMS后处理，是测速时使用，如需预测出检测框结果需要找到对应head中的后处理相关代码并修改为如下：

text

if self.exclude_nms:
           # `exclude_nms=True` just use in benchmark for speed test
           # return pred_bboxes.sum(), pred_scores.sum() # 原先是这行，现在注释
           return pred_bboxes, pred_scores # 新加这行，表示保留进NMS前的原始结果
       else:
           bbox_pred, bbox_num, _ = self.nms(pred_bboxes, pred_scores)
           return bbox_pred, bbox_num

并重新导出，使用时再另接自己写的NMS后处理。

基于PaddleSlim对YOLO系列模型进行量化训练，可以实现精度基本无损，速度普遍提升30%以上，具体请参照模型自动化压缩工具ACT。

[VOC](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/voc)

基础模型

网络模型	输入尺寸	图片数/GPU	学习率策略	TRT-FP16-Latency(ms)	mAP(0.50,11point)	Params(M)	FLOPs(G)	下载链接	配置文件
YOLOv5-s	640	16	60e	3.2	80.3	7.24	16.54	下载链接	配置文件
YOLOv7-tiny	640	32	60e	2.6	80.2	6.23	6.90	下载链接	配置文件
YOLOX-s	640	8	40e	3.0	82.9	9.0	26.8	下载链接	配置文件
PP-YOLOE+_s	640	8	30e	2.9	86.7	7.93	17.36	下载链接	配置文件

注意:

VOC数据集训练的mAP为mAP(IoU=0.5)的结果，且评估未使用multi_label等trick；
所有YOLO VOC模型均加载各自模型的COCO权重作为预训练，各个配置文件的配置均为默认使用8卡GPU，可作为自定义数据集设置参考，具体精度会因数据集而异；
YOLO检测模型建议总batch_size至少大于64去训练，如果资源不够请换小模型或减小模型的输入尺度，为了保障较高检测精度，尽量不要尝试单卡训和总batch_size小于64训；
Params(M)和FLOPs(G)均为训练时所测，YOLOv7没有s模型，故选用tiny模型；
TRT-FP16-Latency(ms)测速相关请查看各YOLO模型的config的主页；

使用指南

下载MS-COCO数据集，官网下载地址为: annotations, train2017, val2017, test2017。
PaddleDetection团队提供的下载链接为：coco(共约22G)和test2017，注意test2017可不下载，评估是使用的val2017。

一键运行全流程

将以下命令写在一个脚本文件里如run.sh，一键运行命令为：sh run.sh，也可命令行一句句去运行。

text

/* Detailed source-code truncated for AI context efficiency. */

如果想切换模型，只要修改开头两行即可，如:
text
```
model_name=yolov7
job_name=yolov7_l_300e_coco
```
导出onnx，首先安装Paddle2ONNX，pip install paddle2onnx；
统计FLOPs(G)和Params(M)，首先安装PaddleSlim，pip install paddleslim，然后设置runtime.yml里print_flops: True和print_params: True，并且注意确保是单尺度下如640x640，打印的是MACs，FLOPs=2*MACs。

自定义数据集

数据集准备：

1.自定义数据集的标注制作，请参考DetAnnoTools;

2.自定义数据集的训练准备，请参考PrepareDataSet。

fintune训练：

除了更改数据集的路径外，训练一般推荐加载对应模型的COCO预训练权重去fintune，会更快收敛和达到更高精度，如：

base

# 单卡fintune训练：
# CUDA_VISIBLE_DEVICES=0 python tools/train.py -c ${config} --eval --amp -o pretrain_weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

# 多卡fintune训练：
python -m paddle.distributed.launch --log_dir=./log_dir --gpus 0,1,2,3,4,5,6,7 tools/train.py -c ${config} --eval --amp -o pretrain_weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

注意:

fintune训练一般会提示head分类分支最后一层卷积的通道数没对应上，属于正常情况，是由于自定义数据集一般和COCO数据集种类数不一致；
fintune训练一般epoch数可以设置更少，lr设置也更小点如1/10，最高精度可能出现在中间某个epoch；

预测和导出：

使用自定义数据集预测和导出模型时，如果TestDataset数据集路径设置不正确会默认使用COCO 80类。
除了TestDataset数据集路径设置正确外，也可以自行修改和添加对应的label_list.txt文件(一行记录一个对应种类)，TestDataset中的anno_path也可设置为绝对路径，如：

text

TestDataset:
  !ImageFolder
    anno_path: label_list.txt # 如不使用dataset_dir，则anno_path即为相对于PaddleDetection主目录的相对路径
    # dataset_dir: dataset/my_coco # 如使用dataset_dir，则dataset_dir/anno_path作为新的anno_path

label_list.txt里的一行记录一个对应种类，如下所示：

text

person
vehicle

Feature Models PaddleYOLO MODEL En (`docs/feature_models/PaddleYOLO_MODEL_en.md`)

简体中文 | English

[PaddleYOLO](https://github.com/PaddlePaddle/PaddleYOLO)

Introduction

PaddleYOLO
- Introduction
- Introduction
- Updates
- ModelZoo
  - PP-YOLOE
  - YOLOX
  - YOLOv5
  - YOLOv6
  - YOLOv7
  - YOLOv8
  - RTMDet
  - Notes：
  - VOC
- UserGuide
  - Pipeline
  - CustomDataset

Introduction

PaddleYOLO is a YOLO Series toolbox based on PaddleDetection, only relevant codes of YOLO series models are included. It supports YOLOv3,PP-YOLO,PP-YOLOv2,PP-YOLOE,PP-YOLOE+,YOLOX,YOLOv5,YOLOv6,YOLOv7,YOLOv8,RTMDet and so on. Welcome to use and build it together!

Updates

【2023/01/10】Support YOLOv8 inference and deploy;
【2022/09/29】Support RTMDet inference and deploy;
【2022/09/26】Release PaddleYOLO;
【2022/09/19】Support the new version of YOLOv6, including n/t/s/m/l model;
【2022/08/23】Release YOLOSeries codebase: support YOLOv3,PP-YOLOE,PP-YOLOE+,YOLOX,YOLOv5,YOLOv6 and YOLOv7; support using ConvNeXt backbone to get high-precision version of PP-YOLOE,YOLOX and YOLOv5; support PaddleSlim accelerated quantitative training PP-YOLOE,YOLOv5,YOLOv6 and YOLOv7. For details, please read this article；

Notes：

The Licence of PaddleYOLO is GPL 3.0, the codes of YOLOv5,YOLOv6,YOLOv7 and YOLOv8 will not be merged into PaddleDetection. Except for these three YOLO models, other YOLO models are recommended to use in PaddleDetection, which will be the first to release the latest progress of PP-YOLO series detection model;
To use PaddleYOLO, PaddlePaddle-2.3.2 or above is recommended，please refer to the official website to download the appropriate version. For Windows platforms, please install the paddle develop version;
Training Custom dataset please refer to doc and issue. Please ensure COCO trained weights are loaded as pre-train at first. We recommend to use YOLO detection model with a total batch_size at least greater than 64 to train. If the resources are insufficient, please use the smaller model or reduce the input size of the model. To ensure high detection accuracy, you'd better never try to using single GPU or total batch_size less than 32 for training;

ModelZoo

[PP-YOLOE](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/ppyoloe)

Baseline

Model	Input Size	images/GPU	Epoch	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	download	config
PP-YOLOE-s	640	32	400e	2.9	43.4	60.0	7.93	17.36	model	config
PP-YOLOE-s	640	32	300e	2.9	43.0	59.6	7.93	17.36	model	config
PP-YOLOE-m	640	28	300e	6.0	49.0	65.9	23.43	49.91	model	config
PP-YOLOE-l	640	20	300e	8.7	51.4	68.6	52.20	110.07	model	config
PP-YOLOE-x	640	16	300e	14.9	52.3	69.5	98.42	206.59	model	config
PP-YOLOE-tiny ConvNeXt	640	16	36e	-	44.6	63.3	33.04	13.87	model	config
PP-YOLOE+_s	640	8	80e	2.9	43.7	60.6	7.93	17.36	model	config
PP-YOLOE+_m	640	8	80e	6.0	49.8	67.1	23.43	49.91	model	config
PP-YOLOE+_l	640	8	80e	8.7	52.9	70.1	52.20	110.07	model	config
PP-YOLOE+_x	640	8	80e	14.9	54.7	72.0	98.42	206.59	model	config

Deploy Models

Model	Input Size	Exported weights(w/o NMS)	ONNX(w/o NMS)
PP-YOLOE-s(400epoch)	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE+_s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE+_m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE+_l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
PP-YOLOE+_x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[YOLOX](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolox)

Baseline

Model	Input Size	images/GPU	Epoch	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	download	config
YOLOX-nano	416	8	300e	2.3	26.1	42.0	0.91	1.08	model	config
YOLOX-tiny	416	8	300e	2.8	32.9	50.4	5.06	6.45	model	config
YOLOX-s	640	8	300e	3.0	40.4	59.6	9.0	26.8	model	config
YOLOX-m	640	8	300e	5.8	46.9	65.7	25.3	73.8	model	config
YOLOX-l	640	8	300e	9.3	50.1	68.8	54.2	155.6	model	config
YOLOX-x	640	8	300e	16.6	51.8	70.6	99.1	281.9	model	config
YOLOX-cdn-tiny	416	8	300e	1.9	32.4	50.2	5.03	6.33	model	config
YOLOX-crn-s	640	8	300e	3.0	40.4	59.6	7.7	24.69	model	config
YOLOX-s ConvNeXt	640	8	36e	-	44.6	65.3	36.2	27.52	model	config

Deploy Models

Model	Input Size	Exported weights(w/o NMS)	ONNX(w/o NMS)
YOLOx-nano	416	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-tiny	416	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOx-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[YOLOv5](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov5)

Baseline

Model	Input Size	images/GPU	Epoch	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	download	config
YOLOv5-n	640	16	300e	2.6	28.0	45.7	1.87	4.52	model	config
YOLOv5-s	640	16	300e	3.2	37.6	56.7	7.24	16.54	model	config
YOLOv5-m	640	16	300e	5.2	45.4	64.1	21.19	49.08	model	config
YOLOv5-l	640	16	300e	7.9	48.9	67.1	46.56	109.32	model	config
YOLOv5-x	640	16	300e	13.7	50.6	68.7	86.75	205.92	model	config
YOLOv5-s ConvNeXt	640	8	36e	-	42.4	65.3	34.54	17.96	model	config
*YOLOv5p6-n	1280	16	300e	-	35.9	54.2	3.25	9.23	model	config
*YOLOv5p6-s	1280	16	300e	-	44.5	63.3	12.63	33.81	model	config
*YOLOv5p6-m	1280	16	300e	-	51.1	69.0	35.73	100.21	model	config
*YOLOv5p6-l	1280	8	300e	-	53.4	71.0	76.77	223.09	model	config
*YOLOv5p6-x	1280	8	300e	-	54.7	72.4	140.80	420.03	model	config

Deploy Models

Model	Input Size	Exported weights(w/o NMS)	ONNX(w/o NMS)
YOLOv5-n	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv5-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv5-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv5-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv5-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[YOLOv6](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov6)

Baseline

Model	Input Size	images/GPU	Epoch	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	download	config
*YOLOv6-n	640	16	300e(+300e)	2.0	37.5	53.1	5.07	12.49	model	config
*YOLOv6-s	640	32	300e(+300e)	2.7	44.8	61.7	20.18	49.36	model	config
*YOLOv6-m	640	32	300e(+300e)	-	49.5	66.9	37.74	92.47	model	config
*YOLOv6-l(silu)	640	32	300e(+300e)	-	52.2	70.2	59.66	149.4	model	config

Deploy Models

Model	Input Size	Exported weights(w/o NMS)	ONNX(w/o NMS)
yolov6-n	640	(w/ nms) \| (w/o nms)	(w/ nms) \| (w/o nms)
yolov6-s	640	(w/ nms) \| (w/o nms)	(w/ nms) \| (w/o nms)
yolov6-m	640	(w/ nms) \| (w/o nms)	(w/ nms) \| (w/o nms)
yolov6-l(silu)	640	(w/ nms) \| (w/o nms)	(w/ nms) \| (w/o nms)

[YOLOv7](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov7)

Baseline

Model	Input Size	images/GPU	Epoch	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	download	config
YOLOv7-L	640	32	300e	7.4	51.0	70.2	37.62	106.08	model	config
*YOLOv7-X	640	32	300e	12.2	53.0	70.8	71.34	190.08	model	config
*YOLOv7P6-W6	1280	16	300e	25.5	54.4	71.8	70.43	360.26	model	config
*YOLOv7P6-E6	1280	10	300e	31.1	55.7	73.0	97.25	515.4	model	config
*YOLOv7P6-D6	1280	8	300e	37.4	56.1	73.3	133.81	702.92	model	config
*YOLOv7P6-E6E	1280	6	300e	48.7	56.5	73.7	151.76	843.52	model	config
YOLOv7-tiny	640	32	300e	-	37.3	54.5	6.23	6.90	model	config
YOLOv7-tiny	416	32	300e	-	33.3	49.5	6.23	2.91	model	config
YOLOv7-tiny	320	32	300e	-	29.1	43.8	6.23	1.73	model	config

Deploy Models

Model	Input Size	Exported weights(w/o NMS)	ONNX(w/o NMS)
YOLOv7-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7P6-W6	1280	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7P6-E6	1280	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7P6-D6	1280	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7P6-E6E	1280	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7-tiny	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7-tiny	416	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv7-tiny	320	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[YOLOv8](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/yolov8)

Baseline

Model	Input Size	images/GPU	Epoch	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	download	config
*YOLOv8-n	640	16	500e	2.4	37.3	53.0	3.16	8.7	model	config
*YOLOv8-s	640	16	500e	3.4	44.9	61.8	11.17	28.6	model	config
*YOLOv8-m	640	16	500e	6.5	50.2	67.3	25.90	78.9	model	config
*YOLOv8-l	640	16	500e	10.0	52.8	69.6	43.69	165.2	model	config
*YOLOv8-x	640	16	500e	15.1	53.8	70.6	68.23	257.8	model	config
*YOLOv8-P6-x	1280	16	500e	55.0	-	-	97.42	522.93	model	config

Deploy Models

Model	Input Size	Exported weights(w/o NMS)	ONNX(w/o NMS)
YOLOv8-n	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv8-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv8-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv8-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
YOLOv8-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

[RTMDet](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/rtmdet)

Baseline

Model	Input Size	images/GPU	Epoch	TRT-FP16-Latency(ms)	mAP^val0.5:0.95	mAP^val0.5	Params(M)	FLOPs(G)	download	config
*RTMDet-t	640	32	300e	2.8	40.9	57.9	4.90	16.21	model	config
*RTMDet-s	640	32	300e	3.3	44.5	62.0	8.89	29.71	model	config
*RTMDet-m	640	32	300e	6.4	49.1	66.8	24.71	78.47	model	config
*RTMDet-l	640	32	300e	10.2	51.2	68.8	52.31	160.32	model	config
*RTMDet-x	640	32	300e	18.0	52.6	70.4	94.86	283.12	model	config

Deploy Models

Model	Input Size	Exported weights(w/o NMS)	ONNX(w/o NMS)
RTMDet-t	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
RTMDet-s	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
RTMDet-m	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
RTMDet-l	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)
RTMDet-x	640	( w/ nms) \| ( w/o nms)	( w/ nms) \| ( w/o nms)

Notes：

All the models are trained on COCO train2017 dataset and evaluated on val2017 dataset. The * in front of the model indicates that the training is being updated.
Please check the specific accuracy and speed details in PP-YOLOE,YOLOX,YOLOv5,YOLOv6,YOLOv7. Note that YOLOv5, YOLOv6 and YOLOv7 have not adopted multi_label to eval.
TRT-FP16-Latency(ms) is the time spent in testing under TensorRT-FP16, excluding data preprocessing and model output post-processing (NMS). The test adopts single card Tesla T4 GPU, batch size=1, and the test environment is paddlepaddle-2.3.2, CUDA 11.2, CUDNN 8.2, GCC-8.2, TensorRT 8.0.3.4. Please refer to the respective model homepage for details.
For FLOPs(G) and Params(M), you should first install PaddleSlim, pip install paddleslim, then set print_flops: True and print_params: True in runtime.yml. Make sure single scale like 640x640, MACs are printed，FLOPs=2*MACs.
Based on PaddleSlim, quantitative training of YOLO series models can achieve basically lossless accuracy and generally improve the speed by more than 30%. For details, please refer to auto_compression.

[VOC](https://github.com/PaddlePaddle/PaddleYOLO/tree/release/2.5/configs/voc)

Baseline

Model	Input Size	images/GPU	Epoch	TRT-FP16-Latency(ms)	mAP(0.50,11point)	Params(M)	FLOPs(G)	download	config
YOLOv5-s	640	16	60e	3.2	80.3	7.24	16.54	model	config
YOLOv7-tiny	640	32	60e	2.6	80.2	6.23	6.90	model	config
YOLOX-s	640	8	40e	3.0	82.9	9.0	26.8	model	config
PP-YOLOE+_s	640	8	30e	2.9	86.7	7.93	17.36	model	config

Note:

The VOC mAP is mAP(IoU=0.5), and all the models have not adopted multi_label to eval.
All YOLO VOC models are loaded with the COCO weights of their respective models as pre-train weights. Each config file uses 8 GPUs by default, which can be used as a reference for setting custom datasets. The specific mAP will vary depending on the datasets;
We recommend to use YOLO detection model with a total batch_size at least greater than 64 to train. If the resources are insufficient, please use the smaller model or reduce the input size of the model. To ensure high detection accuracy, you'd better not try to using single GPU or total batch_size less than 64 for training;
Params (M) and FLOPs (G) are measured during training. YOLOv7 has no s model, so tiny model is selected;
For TRT-FP16 Latency (ms) speed measurement, please refer to the config homepage of each YOLO model;

UserGuide

Download MS-COCO dataset, official website. The download links are: annotations, train2017, val2017, test2017.
The download link provided by PaddleDetection team is: coco(about 22G) and test2017. Note that test2017 is optional, and the evaluation is based on val2017.

Pipeline

Write the following commands in a script file, such as run.sh, and run as：sh run.sh. You can also run the command line sentence by sentence.

text

/* Detailed source-code truncated for AI context efficiency. */

Note：

If you want to switch models, just modify the first two lines, such as:
text
```
model_name=yolov7
job_name=yolov7_tiny_300e_coco
```
For exporting onnx, you should install Paddle2ONNX by pip install paddle2onnx at first.
For FLOPs(G) and Params(M), you should install PaddleSlim by pip install paddleslim at first, then set print_flops: True and print_params: True in runtime.yml. Make sure single scale like 640x640, MACs are printed，FLOPs=2*MACs.

CustomDataset

preparation：

1.For the annotation of custom dataset, please refer toDetAnnoTools;

2.For training preparation of custom dataset，please refer toPrepareDataSet.

fintune：

In addition to changing the path of the dataset, it is generally recommended to load the COCO pre training weight of the corresponding model to fintune, which will converge faster and achieve higher accuracy, such as：

base

# fintune with single GPU：
# CUDA_VISIBLE_DEVICES=0 python tools/train.py -c ${config} --eval --amp -o pretrain_weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

# fintune with multi GPU：
python -m paddle.distributed.launch --log_dir=./log_dir --gpus 0,1,2,3,4,5,6,7 tools/train.py -c ${config} --eval --amp -o pretrain_weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

Note:

The fintune training will show that the channels of the last layer of the head classification branch is not matched, which is a normal situation, because the number of custom dataset is generally inconsistent with that of COCO dataset;
In general, the number of epochs for fintune training can be set less, and the lr setting is also smaller, such as 1/10. The highest accuracy may occur in one of the middle epochs;

Predict and export:

When using custom dataset to predict and export models, if the path of the TestDataset dataset is set incorrectly, COCO 80 categories will be used by default.

In addition to the correct path setting of the TestDataset dataset, you can also modify and add the corresponding label_list. Txt file (one category is recorded in one line), and anno_path in TestDataset can also be set as an absolute path, such as:

text

TestDataset:
  !ImageFolder
    anno_path: label_list.txt # if not set dataset_dir, the anno_path will be relative path of PaddleDetection root directory
    # dataset_dir: dataset/my_coco # if set dataset_dir, the anno_path will be dataset_dir/anno_path

one line in label_list.txt records a corresponding category：

text

person
vehicle

Feature Models SSLD PRETRAINED MODEL (`docs/feature_models/SSLD_PRETRAINED_MODEL.md`)

简体中文 | English

Simple semi-supervised label knowledge distillation solution (SSLD)

R-CNN on COCO

骨架网络	网络类型	每张GPU图片个数	学习率策略	推理时间(fps)	Box AP	Mask AP	下载	配置文件
ResNet50-vd-SSLDv2-FPN	Faster	1	1x	----	41.4	-	下载链接	配置文件
ResNet50-vd-SSLDv2-FPN	Faster	1	2x	----	42.3	-	下载链接	配置文件
ResNet50-vd-SSLDv2-FPN	Mask	1	1x	----	42.0	38.2	下载链接	配置文件
ResNet50-vd-SSLDv2-FPN	Mask	1	2x	----	42.7	38.9	下载链接	配置文件
ResNet50-vd-SSLDv2-FPN	Cascade Faster	1	1x	----	44.4	-	下载链接	配置文件
ResNet50-vd-SSLDv2-FPN	Cascade Faster	1	2x	----	45.0	-	下载链接	配置文件
ResNet50-vd-SSLDv2-FPN	Cascade Mask	1	1x	----	44.9	39.1	下载链接	配置文件
ResNet50-vd-SSLDv2-FPN	Cascade Mask	1	2x	----	45.7	39.7	下载链接	配置文件

YOLOv3 on COCO

骨架网络	输入尺寸	每张GPU图片个数	学习率策略	推理时间(fps)	Box AP	下载	配置文件
MobileNet-V1-SSLD	608	8	270e	----	31.0	下载链接	配置文件
MobileNet-V1-SSLD	416	8	270e	----	30.6	下载链接	配置文件
MobileNet-V1-SSLD	320	8	270e	----	28.4	下载链接	配置文件

YOLOv3 on Pasacl VOC

骨架网络	输入尺寸	每张GPU图片个数	学习率策略	推理时间(fps)	Box AP	下载	配置文件
MobileNet-V1-SSLD	608	8	270e	-	78.3	下载链接	配置文件
MobileNet-V1-SSLD	416	8	270e	-	79.6	下载链接	配置文件
MobileNet-V1-SSLD	320	8	270e	-	77.3	下载链接	配置文件
MobileNet-V3-SSLD	608	8	270e	-	80.4	下载链接	配置文件
MobileNet-V3-SSLD	416	8	270e	-	79.2	下载链接	配置文件
MobileNet-V3-SSLD	320	8	270e	-	77.3	下载链接	配置文件

注意事项:

SSLD是一种知识蒸馏方法，我们使用蒸馏后性能更强的backbone预训练模型，进一步提升检测精度，详细方案请参考知识蒸馏教程

Citations

text

@misc{cui2021selfsupervision,
      title={Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones},
      author={Cheng Cui and Ruoyu Guo and Yuning Du and Dongliang He and Fu Li and Zewu Wu and Qiwen Liu and Shilei Wen and Jizhou Huang and Xiaoguang Hu and Dianhai Yu and Errui Ding and Yanjun Ma},
      year={2021},
      eprint={2103.05959},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Feature Models SSLD PRETRAINED MODEL En (`docs/feature_models/SSLD_PRETRAINED_MODEL_en.md`)

English | 简体中文

Simple semi-supervised label knowledge distillation solution (SSLD)

R-CNN on COCO

Backbone	Model	Images/GPU	Lr schd	FPS	Box AP	Mask AP	Download	Config
ResNet50-vd-SSLDv2-FPN	Faster	1	1x	----	41.4	-	model	config
ResNet50-vd-SSLDv2-FPN	Faster	1	2x	----	42.3	-	model	config
ResNet50-vd-SSLDv2-FPN	Mask	1	1x	----	42.0	38.2	model	config
ResNet50-vd-SSLDv2-FPN	Mask	1	2x	----	42.7	38.9	model	config
ResNet50-vd-SSLDv2-FPN	Cascade Faster	1	1x	----	44.4	-	model	config
ResNet50-vd-SSLDv2-FPN	Cascade Faster	1	2x	----	45.0	-	model	config
ResNet50-vd-SSLDv2-FPN	Cascade Mask	1	1x	----	44.9	39.1	model	config
ResNet50-vd-SSLDv2-FPN	Cascade Mask	1	2x	----	45.7	39.7	model	config

YOLOv3 on COCO

Backbone	Input shape	Images/GPU	Lr schd	FPS	Box AP	Download	Config
MobileNet-V1-SSLD	608	8	270e	----	31.0	model	config
MobileNet-V1-SSLD	416	8	270e	----	30.6	model	config
MobileNet-V1-SSLD	320	8	270e	----	28.4	model	config

YOLOv3 on Pasacl VOC

Backbone	Input shape	Images/GPU	Lr schd	FPS	Box AP	Download	Config
MobileNet-V1-SSLD	608	8	270e	-	78.3	model	config
MobileNet-V1-SSLD	416	8	270e	-	79.6	model	config
MobileNet-V1-SSLD	320	8	270e	-	77.3	model	config
MobileNet-V3-SSLD	608	8	270e	-	80.4	model	config
MobileNet-V3-SSLD	416	8	270e	-	79.2	model	config
MobileNet-V3-SSLD	320	8	270e	-	77.3	model	config

Notes:

SSLD is a knowledge distillation method. We use the stronger backbone pretrained model after distillation to further improve the detection accuracy. Please refer to the knowledge distillation tutorial.

Citations

text

@misc{cui2021selfsupervision,
      title={Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones},
      author={Cheng Cui and Ruoyu Guo and Yuning Du and Dongliang He and Fu Li and Zewu Wu and Qiwen Liu and Shilei Wen and Jizhou Huang and Xiaoguang Hu and Dianhai Yu and Errui Ding and Yanjun Ma},
      year={2021},
      eprint={2103.05959},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Paddlex Quick Start (`docs/paddlex/quick_start.md`)

快速开始

说明：

飞桨低代码开发工具PaddleX，依托于PaddleDetection的先进技术，支持了目标检测领域的低代码全流程开发能力。通过低代码全流程开发，可实现简单且高效的模型使用、组合与定制。

PaddleX 致力于实现产线级别的模型训练、推理与部署。模型产线是指一系列预定义好的、针对特定AI任务的开发流程，其中包含能够独立完成某类任务的单模型（单功能模块）组合。本文档提供目标检测相关产线的快速使用，单功能模块的快速使用以及更多功能请参考PaddleDetection低代码全流程开发中相关章节。

🛠️ 安装

❗安装PaddleX前请先确保您有基础的Python运行环境。（注：当前支持Python 3.8 ～ Python 3.10下运行，更多Python版本适配中）。

安装PaddlePaddle

bash

# cpu
python -m pip install paddlepaddle==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/

# gpu，该命令仅适用于 CUDA 版本为 11.8 的机器环境
python -m pip install paddlepaddle-gpu==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/

# gpu，该命令仅适用于 CUDA 版本为 12.3 的机器环境
python -m pip install paddlepaddle-gpu==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cu123/

❗ 更多飞桨 Wheel 版本请参考飞桨官网。

安装PaddleX

bash

pip install https://paddle-model-ecology.bj.bcebos.com/paddlex/whl/paddlex-3.0.0b1-py3-none-any.whl

❗ 更多安装方式参考PaddleX安装教程

💻 命令行使用

一行命令即可快速体验产线效果，统一的命令行格式为：

bash

paddlex --pipeline [产线名称] --input [输入图片] --device [运行设备]

只需指定三个参数：

pipeline：产线名称
input：待处理的输入图片的本地路径或URL
device: 使用的GPU序号（例如gpu:0表示使用第0块GPU），也可选择使用CPU（cpu）

以通用目标检测产线为例：

bash

paddlex --pipeline object_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_object_detection_002.png --device gpu:0

运行后，得到的结果为：

text

{'input_path': '/root/.paddlex/predict_input/general_object_detection_002.png', 'boxes': [{'cls_id': 49, 'label': 'orange', 'score': 0.8188097476959229, 'coordinate': [661, 93, 870, 305]}, {'cls_id': 47, 'label': 'apple', 'score': 0.7743489146232605, 'coordinate': [76, 274, 330, 520]}, {'cls_id': 47, 'label': 'apple', 'score': 0.7270504236221313, 'coordinate': [285, 94, 469, 297]}, {'cls_id': 46, 'label': 'banana', 'score': 0.5570532083511353, 'coordinate': [310, 361, 685, 712]}, {'cls_id': 47, 'label': 'apple', 'score': 0.5484835505485535, 'coordinate': [764, 285, 924, 440]}, {'cls_id': 47, 'label': 'apple', 'score': 0.5160726308822632, 'coordinate': [853, 169, 987, 303]}, {'cls_id': 60, 'label': 'dining table', 'score': 0.5142655968666077, 'coordinate': [0, 0, 1072, 720]}, {'cls_id': 47, 'label': 'apple', 'score': 0.5101479291915894, 'coordinate': [57, 23, 213, 176]}]}

其他产线的命令行使用，只需将pipeline参数调整为相应产线的名称。下面列出了每个产线对应的命令：

产线名称	使用命令
通用目标检测	`paddlex --pipeline object_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_object_detection_002.png --device gpu:0`
通用实例分割	`paddlex --pipeline instance_segmentation --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_instance_segmentation_004.png --device gpu:0`
小目标检测	`paddlex --pipeline small_object_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/small_object_detection.jpg --device gpu:0`

📝 Python脚本使用

几行代码即可完成产线的快速推理，以小目标检测产线为例：

python

from paddlex import create_pipeline

pipeline = create_pipeline(pipeline="small_object_detection")

output = pipeline.predict("small_object_detection.jpg")
for res in output:
    res.print() ## 打印预测的结构化输出
    res.save_to_img("./output/") ## 保存结果可视化图像
    res.save_to_json("./output/") ## 保存预测的结构化输出

运行后，得到的结果为：

text

/* Detailed source-code truncated for AI context efficiency. */

执行了如下几个步骤：

create_pipeline() 实例化产线对象
传入图片并调用产线对象的predict 方法进行推理预测
对预测结果进行处理

其他产线的Python脚本使用，只需将create_pipeline()方法的pipeline参数调整为相应产线的名称。下面列出了每个产线对应的参数名称及详细的使用解释：

产线名称	对应参数	详细说明
通用目标检测	`object_detection`	通用目标检测产线Python脚本使用说明
通用实例分割	`instance_segmentation`	通用实例分割产线Python脚本使用说明
小目标检测	`small_object_detection`	小目标检测产线Python脚本使用说明

Tutorials DistributedTraining Cn (`docs/tutorials/DistributedTraining_cn.md`)

English | 简体中文

分布式训练

1. 简介

分布式训练指的是将训练任务按照一定方法拆分到多个计算节点进行计算，再按照一定的方法对拆分后计算得到的梯度等信息进行聚合与更新。飞桨分布式训练技术源自百度的业务实践，在自然语言处理、计算机视觉、搜索和推荐等领域经过超大规模业务检验。分布式训练的高性能，是飞桨的核心优势技术之一，PaddleDetection同时支持单机训练与多机训练。更多关于分布式训练的方法与文档可以参考：分布式训练快速开始教程。

2. 使用方法

2.1 单机训练

以PP-YOLOE-s为例，本地准备好数据之后，使用paddle.distributed.launch或者fleetrun的接口启动训练任务即可。下面为运行脚本示例。

bash

fleetrun \
--selected_gpu 0,1,2,3,4,5,6,7 \
tools/train.py -c configs/ppyoloe/ppyoloe_crn_s_300e_coco.yml \
--eval &>logs.txt 2>&1 &

2.2 多机训练

相比单机训练，多机训练时，只需要添加--ips的参数，该参数表示需要参与分布式训练的机器的ip列表，不同机器的ip用逗号隔开。下面为运行代码示例。

shell

ip_list="10.127.6.17,10.127.5.142,10.127.45.13,10.127.44.151"
fleetrun \
--ips=${ip_list} \
--selected_gpu 0,1,2,3,4,5,6,7 \
tools/train.py -c configs/ppyoloe/ppyoloe_crn_s_300e_coco.yml \
--eval &>logs.txt 2>&1 &

注：

不同机器的ip信息需要用逗号隔开，可以通过ifconfig或者ipconfig查看。
不同机器之间需要做免密设置，且可以直接ping通，否则无法完成通信。
不同机器之间的代码、数据与运行命令或脚本需要保持一致，且所有的机器上都需要运行设置好的训练命令或者脚本。最终ip_list中的第一台机器的第一块设备是trainer0，以此类推。
不同机器的起始端口可能不同，建议在启动多机任务前，在不同的机器中设置相同的多机运行起始端口，命令为export FLAGS_START_PORT=17000，端口值建议在10000~20000之间。

3. 性能效果测试

在3机8卡V100的机器上进行模型训练，不同模型的精度、训练耗时、多机加速比情况如下所示。

模型	数据集	配置	单机8卡耗时/精度	3机8卡耗时/精度	加速比
PP-YOLOE-s	Objects365	ppyoloe_crn_s_300e_coco.yml	301h/-	162h/17.7%	1.85
PP-YOLOE-l	Objects365	ppyoloe_crn_l_300e_coco.yml	401h/-	178h/30.3%	2.25

在4机8卡V100的机器上进行模型训练，不同模型的精度、训练耗时、多机加速比情况如下所示。

模型	数据集	配置	单机8卡耗时/精度	4机8卡耗时/精度	加速比
PP-YOLOE-s	COCO	ppyoloe_crn_s_300e_coco.yml	39h/42.7%	13h/42.1%	3.0
PP-YOLOE-m	Objects365	ppyoloe_crn_m_300e_coco.yml	337h/-	112h/24.6%	3.0
PP-YOLOE-x	Objects365	ppyoloe_crn_x_300e_coco.yml	464h/-	125h/32.1%	3.4

注意
- 在训练的GPU卡数过多时，精度会稍微有所损失（1%左右），此时可以尝试通过添加warmup或者适当增加迭代轮数来弥补精度损失。
- 这里的配置文件均提供的是COCO数据集的配置文件，如果需要训练其他的数据集，需要修改数据集路径。
- 上面的PP-YOLOE系列模型在多机训练过程中，均设置单卡batch size为8，同时学习率相比于单机8卡保持不变。

Tutorials DistributedTraining En (`docs/tutorials/DistributedTraining_en.md`)

English | 简体中文

1. Usage

1.1 Single-machine

Take PP-YOLOE-s as an example, after preparing the data locally, use the interface of paddle.distributed.launch or fleetrun to start the training task. Below is an example of running the script.

bash

fleetrun \
--selected_gpu 0,1,2,3,4,5,6,7 \
tools/train.py -c configs/ppyoloe/ppyoloe_crn_s_300e_coco.yml \
--eval &>logs.txt 2>&1 &

1.2 Multi-machine

Compared with single-machine training, when training on multiple machines, you only need to add the --ips parameter, which indicates the ip list of machines that need to participate in distributed training. The ips of different machines are separated by commas. Below is an example of running code.

shell

ip_list="10.127.6.17,10.127.5.142,10.127.45.13,10.127.44.151"
fleetrun \
--ips=${ip_list} \
--selected_gpu 0,1,2,3,4,5,6,7 \
tools/train.py -c configs/ppyoloe/ppyoloe_crn_s_300e_coco.yml \
--eval &>logs.txt 2>&1 &

Note:

The ip information of different machines needs to be separated by commas, which can be viewed through ifconfig or ipconfig.
Password-free settings are required between different machines, and they can be pinged directly, otherwise the communication cannot be completed.
The code, data, and running commands or scripts between different machines need to be consistent, and the set training commands or scripts need to be run on all machines. The first device of the first machine in the final ip_list is trainer0, and so on.
The starting port of different machines may be different. It is recommended to set the same starting port for multi-machine running in different machines before starting the multi-machine task. The command is export FLAGS_START_PORT=17000, and the port value is recommended to be 10000~20000.

2. Performance

We conducted model training on 3x8 V100 GPUs. Accuracy, training time, and multi machine acceleration ratio of different models are shown below.

Model	Dataset	Configuration	8 GPU training time / Accuracy	3x8 GPU training time / Accuracy	Acceleration ratio
PP-YOLOE-s	Objects365	ppyoloe_crn_s_300e_coco.yml	301h/-	162h/17.7%	1.85
PP-YOLOE-l	Objects365	ppyoloe_crn_l_300e_coco.yml	401h/-	178h/30.3%	2.25

We conducted model training on 4x8 V100 GPUs. Accuracy, training time, and multi machine acceleration ratio of different models are shown below.

Model	Dataset	Configuration	8 GPU training time / Accuracy	4x8 GPU training time / Accuracy	Acceleration ratio
PP-YOLOE-s	COCO	ppyoloe_crn_s_300e_coco.yml	39h/42.7%	13h/42.1%	3.0
PP-YOLOE-m	Objects365	ppyoloe_crn_m_300e_coco.yml	337h/-	112h/24.6%	3.0
PP-YOLOE-x	Objects365	ppyoloe_crn_x_300e_coco.yml	464h/-	125h/32.1%	3.4

Note
- When the number of GPU cards for training is too large, the accuracy will be slightly lost (about 1%). At this time, you can try to warmup the training process or increase some training epochs to reduce the lost.
- The configuration files here are provided based on COCO datasets. If you need to train on other datasets, you need to modify the dataset path.
- For the multi-machine training process of PP-YOLOE series, the batch size of single card is set as 8 and learning rate is same as that of single machine.

Tutorials GETTING STARTED (`docs/tutorials/GETTING_STARTED.md`)

English | 简体中文

Getting Started

Installation

For setting up the running environment, please refer to installation
instructions.

Data preparation

Please refer to PrepareDetDataSet for data preparation
Please set the data path for data configuration file in configs/datasets

Training & Evaluation & Inference

PaddleDetection provides scripts for training, evalution and inference with various features according to different configure. And for more distribued training details see [DistributedTraining].(./DistributedTraining_en.md)

bash

# training on single-GPU
export CUDA_VISIBLE_DEVICES=0
python tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
# training on multi-GPU
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
# training on multi-machines and multi-GPUs
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
$fleetrun --ips="10.127.6.17,10.127.5.142,10.127.45.13,10.127.44.151" --selected_gpu 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
# GPU evaluation
export CUDA_VISIBLE_DEVICES=0
python tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_fpn_1x_coco.pdparams
# Inference
python tools/infer.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --infer_img=demo/000000570688.jpg -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_fpn_1x_coco.pdparams

Other argument list

list below can be viewed by --help

FLAG	script supported	description	default	remark
-c	ALL	Select config file	None	required, such as `-c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml`
-o	ALL	Set parameters in configure file	None	`-o` has higher priority to file configured by `-c`. Such as `-o use_gpu=False`
--eval	train	Whether to perform evaluation in training	False	set `--eval` if needed
-r/--resume_checkpoint	train	Checkpoint path for resuming training	None	such as `-r output/faster_rcnn_r50_1x_coco/10000`
--slim_config	ALL	Configure file of slim method	None	such as `--slim_config configs/slim/prune/yolov3_prune_l1_norm.yml`
--use_vdl	train/infer	Whether to record the data with VisualDL, so as to display in VisualDL	False	VisualDL requires Python>=3.5
--vdl_log_dir	train/infer	VisualDL logging directory for image	train:`vdl_log_dir/scalar` infer: `vdl_log_dir/image`	VisualDL requires Python>=3.5
--output_eval	eval	Directory for storing the evaluation output	None	such as `--output_eval=eval_output`, default is current directory
--json_eval	eval	Whether to evaluate with already existed bbox.json or mask.json	False	set `--json_eval` if needed and json path is set in `--output_eval`
--classwise	eval	Whether to eval AP for each class and draw PR curve	False	set `--classwise` if needed
--output_dir	infer	Directory for storing the output visualization files	`./output`	such as `--output_dir output`
--draw_threshold	infer	Threshold to reserve the result for visualization	0.5	such as `--draw_threshold 0.7`
--infer_dir	infer	Directory for images to perform inference on	None	One of `infer_dir` and `infer_img` is requied
--infer_img	infer	Image path	None	One of `infer_dir` and `infer_img` is requied, `infer_img` has higher priority over `infer_dir`
--save_results	infer	Whether to save detection results to file	False	Optional

Examples

Training

Perform evaluation in training
bash
```
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --eval
```
Perform training and evalution alternatively and evaluate at each end of epoch. Meanwhile, the best model with highest MAP is saved at each epoch which has the same path as model_final.

If evaluation dataset is large, we suggest modifing snapshot_epoch in configs/runtime.yml to decrease evaluation times or evaluating after training.

Fine-tune other task

When using pre-trained model to fine-tune other task, pretrain_weights can be used directly. The parameters with different shape will be ignored automatically. For example:

bash

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
# If the shape of parameters in program is different from pretrain_weights,
# then PaddleDetection will not use such parameters.
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
                         -o pretrain_weights=output/faster_rcnn_r50_1x_coco/model_final \

NOTES

CUDA_VISIBLE_DEVICES can specify different gpu numbers. Such as: export CUDA_VISIBLE_DEVICES=0,1,2,3.
Dataset will be downloaded automatically and cached in ~/.cache/paddle/dataset if not be found locally.
Pretrained model is downloaded automatically and cached in ~/.cache/paddle/weights.
Checkpoints are saved in output by default, and can be revised from save_dir in configs/runtime.yml.

Evaluation

Evaluate by specified weights path and dataset path

bash

export CUDA_VISIBLE_DEVICES=0
python -u tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
                        -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_fpn_1x_coco.pdparams

The path of model to be evaluted can be both local path and link in MODEL_ZOO.

Evaluate with json

bash

export CUDA_VISIBLE_DEVICES=0
python tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
           --json_eval \
           -output_eval evaluation/

The json file must be named bbox.json or mask.json, placed in the evaluation/ directory.

Inference

Output specified directory && Set up threshold

bash

export CUDA_VISIBLE_DEVICES=0
python tools/infer.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
                    --infer_img=demo/000000570688.jpg \
                    --output_dir=infer_output/ \
                    --draw_threshold=0.5 \
                    -o weights=output/faster_rcnn_r50_fpn_1x_coco/model_final \
                    --use_vdl=True

--draw_threshold is an optional argument. Default is 0.5.
Different thresholds will produce different results depending on the calculation of NMS.

Deployment

Please refer to depolyment

Model Compression

Please refer to slim

Tutorials GETTING STARTED Cn (`docs/tutorials/GETTING_STARTED_cn.md`)

English | 简体中文

30分钟快速上手PaddleDetection

PaddleDetection作为成熟的目标检测开发套件，提供了从数据准备、模型训练、模型评估、模型导出到模型部署的全流程。在这个章节里面，我们以路标检测数据集为例，提供快速上手PaddleDetection的流程。

1 安装

关于安装配置运行环境，请参考安装指南
在本演示案例中，假定用户将PaddleDetection的代码克隆并放置在/home/paddle目录中。用户执行的命令操作均在/home/paddle/PaddleDetection目录下完成

2 准备数据

目前PaddleDetection支持：COCO VOC WiderFace, MOT四种数据格式。

首先按照准备数据文档准备数据。
然后设置configs/datasets中相应的coco或voc等数据配置文件中的数据路径。
在本项目中，我们使用路标识别数据集

bash

python dataset/roadsign_voc/download_roadsign_voc.py

下载后的数据格式为

text

├── download_roadsign_voc.py
  ├── annotations
  │   ├── road0.xml
  │   ├── road1.xml
  │   |   ...
  ├── images
  │   ├── road0.png
  │   ├── road1.png
  │   |   ...
  ├── label_list.txt
  ├── train.txt
  ├── valid.txt

3 配置文件改动和说明

我们使用configs/yolov3/yolov3_mobilenet_v1_roadsign配置进行训练。
在静态图版本下，一个模型往往可以通过两个配置文件（一个主配置文件、一个reader的读取配置）实现，在PaddleDetection 2.0后续版本，采用了模块解耦设计，用户可以组合配置模块实现检测器，并可自由修改覆盖各模块配置，如下图所示

配置文件摘要

从上图看到yolov3_mobilenet_v1_roadsign.yml配置需要依赖其他的配置文件。在该例子中需要依赖：

bash

roadsign_voc.yml

  runtime.yml

  optimizer_40e.yml

  yolov3_mobilenet_v1.yml

  yolov3_reader.yml
--------------------------------------


yolov3_mobilenet_v1_roadsign 文件入口

roadsign_voc 主要说明了训练数据和验证数据的路径

runtime.yml 主要说明了公共的运行参数，比如说是否使用GPU、每多少个epoch存储checkpoint等

optimizer_40e.yml 主要说明了学习率和优化器的配置。

ppyolov2_r50vd_dcn.yml 主要说明模型、和主干网络的情况。

ppyolov2_reader.yml 主要说明数据读取器配置，如batch size，并发加载子进程数等，同时包含读取后预处理操作，如resize、数据增强等等

配置文件结构说明

修改配置文件说明

关于数据的路径修改说明
在修改配置文件中，用户如何实现自定义数据集是非常关键的一步，如何定义数据集请参考如何自定义数据集
默认学习率是适配多GPU训练(8x GPU)，若使用单GPU训练，须对应调整学习率（例如，除以8）
更多使用问题，请参考FAQ

4 训练

PaddleDetection提供了单卡/多卡训练模式，满足用户多种训练需求

GPU单卡训练

bash

export CUDA_VISIBLE_DEVICES=0 #windows和Mac下不需要执行该命令
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml

GPU多卡训练

bash

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 #windows和Mac下不需要执行该命令
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml

GPU多机多卡训练

bash

$fleetrun \
--ips="10.127.6.17,10.127.5.142,10.127.45.13,10.127.44.151" \
--selected_gpu 0,1,2,3,4,5,6,7 \
tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml \

Fine-tune其他任务

使用预训练模型fine-tune其他任务时，可以直接加载预训练模型，形状不匹配的参数将自动忽略，例如：

bash

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
  # 如果模型中参数形状与加载权重形状不同，将不会加载这类参数
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o pretrain_weights=output/model_final

模型恢复训练

在日常训练过程中，有的用户由于一些原因导致训练中断，用户可以使用-r的命令恢复训练

bash

export CUDA_VISIBLE_DEVICES=0 #windows和Mac下不需要执行该命令
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -r output/faster_rcnn_r50_1x_coco/10000

5 评估

默认将训练生成的模型保存在当前output文件夹下

bash

export CUDA_VISIBLE_DEVICES=0 #windows和Mac下不需要执行该命令
python tools/eval.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov3_mobilenet_v1_roadsign.pdparams

边训练，边评估

bash

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 #windows和Mac下不需要执行该命令
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml --eval

在训练中交替执行评估, 评估在每个epoch训练结束后开始。每次评估后还会评出最佳mAP模型保存到best_model文件夹下。

如果验证集很大，测试将会比较耗时，建议调整configs/runtime.yml 文件中的 snapshot_epoch配置以减少评估次数，或训练完成后再进行评估。

通过json文件评估

bash

export CUDA_VISIBLE_DEVICES=0 #windows和Mac下不需要执行该命令
python tools/eval.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml \
             --json_eval \
             -output_eval evaluation/

上述命令中没有加载模型的选项，则使用配置文件中weights的默认配置，weights表示训练过程中保存的最后一轮模型文件
json文件必须命名为bbox.json或者mask.json，放在evaluation目录下。

6 预测

bash

python tools/infer.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml --infer_img=demo/road554.png -o weights=https://paddledet.bj.bcebos.com/models/yolov3_mobilenet_v1_roadsign.pdparams

设置参数预测

bash

export CUDA_VISIBLE_DEVICES=0 #windows和Mac下不需要执行该命令
python tools/infer.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml \
                    --infer_img=demo/road554.png \
                    --output_dir=infer_output/ \
                    --draw_threshold=0.5 \
                    -o weights=output/yolov3_mobilenet_v1_roadsign/model_final \
                    --use_vdl=True

--draw_threshold 是个可选参数. 根据 NMS 的计算，不同阈值会产生不同的结果
keep_top_k表示设置输出目标的最大数量，默认值为100，用户可以根据自己的实际情况进行设定。

结果如下图：

7 训练可视化

当打开use_vdl开关后，为了方便用户实时查看训练过程中状态，PaddleDetection集成了VisualDL可视化工具，当打开use_vdl开关后，记录的数据包括：

loss变化趋势
mAP变化趋势

bash

export CUDA_VISIBLE_DEVICES=0 #windows和Mac下不需要执行该命令
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml
                        --use_vdl=true \
                        --vdl_log_dir=vdl_dir/scalar \

使用如下命令启动VisualDL查看日志

shell

# 下述命令会在127.0.0.1上启动一个服务，支持通过前端web页面查看，可以通过--host这个参数指定实际ip地址
visualdl --logdir vdl_dir/scalar/

在浏览器输入提示的网址，效果如下：

图：VDL效果演示

参数列表

以下列表可以通过--help查看

FLAG	支持脚本	用途	默认值	备注
-c	ALL	指定配置文件	None	必选，例如-c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
-o	ALL	设置或更改配置文件里的参数内容	None	相较于`-c`设置的配置文件有更高优先级，例如：`-o use_gpu=False`
--eval	train	是否边训练边测试	False	如需指定，直接`--eval`即可
-r/--resume_checkpoint	train	恢复训练加载的权重路径	None	例如：`-r output/faster_rcnn_r50_1x_coco/10000`
--slim_config	ALL	模型压缩策略配置文件	None	例如`--slim_config configs/slim/prune/yolov3_prune_l1_norm.yml`
--use_vdl	train/infer	是否使用VisualDL记录数据，进而在VisualDL面板中显示	False	VisualDL需Python>=3.5
--vdl_log_dir	train/infer	指定 VisualDL 记录数据的存储路径	train:`vdl_log_dir/scalar` infer: `vdl_log_dir/image`	VisualDL需Python>=3.5
--output_eval	eval	评估阶段保存json路径	None	例如 `--output_eval=eval_output`, 默认为当前路径
--json_eval	eval	是否通过已存在的bbox.json或者mask.json进行评估	False	如需指定，直接`--json_eval`即可， json文件路径在`--output_eval`中设置
--classwise	eval	是否评估单类AP和绘制单类PR曲线	False	如需指定，直接`--classwise`即可
--output_dir	infer/export_model	预测后结果或导出模型保存路径	`./output`	例如`--output_dir=output`
--draw_threshold	infer	可视化时分数阈值	0.5	例如`--draw_threshold=0.7`
--infer_dir	infer	用于预测的图片文件夹路径	None	`--infer_img`和`--infer_dir`必须至少设置一个
--infer_img	infer	用于预测的图片路径	None	`--infer_img`和`--infer_dir`必须至少设置一个，`infer_img`具有更高优先级
--save_results	infer	是否在文件夹下将图片的预测结果保存到文件中	False	可选

8 模型导出

在模型训练过程中保存的模型文件是包含前向预测和反向传播的过程，在实际的工业部署则不需要反向传播，因此需要将模型进行导成部署需要的模型格式。
在PaddleDetection中提供了 tools/export_model.py脚本来导出模型

bash

python tools/export_model.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml --output_dir=./inference_model \
 -o weights=output/yolov3_mobilenet_v1_roadsign/best_model

预测模型会导出到inference_model/yolov3_mobilenet_v1_roadsign目录下，分别为infer_cfg.yml, model.pdiparams, model.pdiparams.info,model.pdmodel 如果不指定文件夹，模型则会导出在output_inference

更多关于模型导出的文档，请参考模型导出文档

9 模型压缩

为了进一步对模型进行优化，PaddleDetection提供了基于PaddleSlim进行模型压缩的完整教程和benchmark。目前支持的方案：

裁剪
量化
蒸馏
联合策略
更多关于模型压缩的文档，请参考模型压缩文档。

10 预测部署

PaddleDetection提供了PaddleInference、PaddleServing、PaddleLite多种部署形式，支持服务端、移动端、嵌入式等多种平台，提供了完善的Python和C++部署方案。

在这里，我们以Python为例，说明如何使用PaddleInference进行模型部署

bash

python deploy/python/infer.py --model_dir=./output_inference/yolov3_mobilenet_v1_roadsign --image_file=demo/road554.png --device=GPU

同时infer.py提供了丰富的接口，用户进行接入视频文件、摄像头进行预测，更多内容请参考Python端预测部署

PaddleDetection支持的部署形式说明

形式	语言	教程	设备/平台
PaddleInference	Python	已完善	Linux(arm X86)、Windows
PaddleInference	C++	已完善	Linux(arm X86)、Windows
PaddleServing	Python	已完善	Linux(arm X86)、Windows
PaddleLite	C++	已完善	Android、IOS、FPGA、RK...

更多关于预测部署的文档，请参考预测部署文档。

Tutorials GradCAM Cn (`docs/tutorials/GradCAM_cn.md`)

目标检测热力图

1.简介

基于backbone/roi特征图计算物体预测框的cam(类激活图), 目前支持基于FasterRCNN/MaskRCNN系列, PPYOLOE系列, 以及BlazeFace, SSD, Retinanet网络。

2.使用方法

以PP-YOLOE为例，准备好数据之后，指定网络配置文件、模型权重地址和图片路径以及输出文件夹路径，使用脚本调用tools/cam_ppdet.py计算图片中物体预测框的grad_cam热力图。下面为运行脚本示例。

shell

python tools/cam_ppdet.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml --infer_img demo/000000014439.jpg --cam_out cam_ppyoloe --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

参数

FLAG	用途
-c	指定配置文件
--infer_img	用于预测的图片路径
--cam_out	指定输出路径
--target_feature_layer_name	计算cam的特征图位置, 如model.backbone、 model.bbox_head.roi_extractor
-o	设置或更改配置文件里的参数内容, 如 -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

运行效果

cam_ppyoloe/225.jpg

3. 目前支持基于FasterRCNN/MaskRCNN系列, PPYOLOE系列以及BlazeFace, SSD, Retinanet网络。

PPYOLOE网络热图可视化脚本

bash

python tools/cam_ppdet.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml --infer_img demo/000000014439.jpg --cam_out cam_ppyoloe --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

MaskRCNN网络roi特征热图可视化脚本

bash

python tools/cam_ppdet.py -c configs/mask_rcnn/mask_rcnn_r50_vd_fpn_2x_coco.yml --infer_img demo/000000014439.jpg  --cam_out cam_mask_rcnn_roi --target_feature_layer_name model.bbox_head.roi_extractor -o weights=https://paddledet.bj.bcebos.com/models/mask_rcnn_r50_vd_fpn_2x_coco.pdparams

MaskRCNN网络backbone特征的热图可视化脚本

bash

python tools/cam_ppdet.py -c configs/mask_rcnn/mask_rcnn_r50_vd_fpn_2x_coco.yml --infer_img demo/000000014439.jpg  --cam_out cam_mask_rcnn_backbone --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/mask_rcnn_r50_vd_fpn_2x_coco.pdparams

FasterRCNN网络基于roi特征的热图可视化脚本

bash

python tools/cam_ppdet.py -c configs/faster_rcnn/faster_rcnn_r50_vd_fpn_2x_coco.yml --infer_img demo/000000014439.jpg  --cam_out cam_faster_rcnn_roi --target_feature_layer_name model.bbox_head.roi_extractor -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_vd_fpn_ssld_2x_coco.pdparams

FasterRCNN网络基于backbone特征的热图可视化脚本

bash

python tools/cam_ppdet.py -c configs/faster_rcnn/faster_rcnn_r50_vd_fpn_2x_coco.yml --infer_img demo/000000014439.jpg  --cam_out cam_faster_rcnn_backbone --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_vd_fpn_ssld_2x_coco.pdparams

BlaczeFace网络backbone特征热图可视化脚本

bash

python tools/cam_ppdet.py -c configs/face_detection/blazeface_1000e.yml --infer_img demo/hrnet_demo.jpg --cam_out cam_blazeface --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/blazeface_1000e.pdparams

SSD网络backbone特征热图可视化脚本

bash

python tools/cam_ppdet.py -c configs/ssd/ssd_mobilenet_v1_300_120e_voc.yml --infer_img demo/000000014439.jpg --cam_out cam_ssd --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/ssd_mobilenet_v1_300_120e_voc.pdparams

Retinanet网络backbone特征热图可视化脚本

bash

python tools/cam_ppdet.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml --infer_img demo/000000014439.jpg --cam_out cam_retinanet --target_feature_layer_name model.backbone -o weights=https://bj.bcebos.com/v1/paddledet/models/retinanet_r50_fpn_2x_coco.pdparams

Tutorials GradCAM En (`docs/tutorials/GradCAM_en.md`)

Object detection grad_cam heatmap

1.Introduction

Calculate the cam (class activation map) of the object predict bbox based on the backbone/roi feature map, currently supports networks based on FasterRCNN/MaskRCNN series, PPYOLOE series and BlazeFace, SSD, Retinanet.

2.Usage

Taking PP-YOLOE as an example, after preparing the data, specify the network configuration file, model weight address, image path and output folder path, and then use the script to call tools/cam_ppdet.py to calculate the grad_cam heat map of the prediction box. Below is an example run script.

shell

python tools/cam_ppdet.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml --infer_img demo/000000014439.jpg --cam_out cam_ppyoloe --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

Arguments

FLAG	description
-c	Select config file
--infer_img	Image path
--cam_out	Directory for output
--target_feature_layer_name	The position of featuremap to do gradcam, for example:model.backbone, model.bbox_head.roi_extractor
-o	Set parameters in configure file, for example: -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

result

cam_ppyoloe/225.jpg

3.Currently supports networks based on FasterRCNN/MaskRCNN series, PPYOLOE series and BlazeFace, SSD, Retinanet.

PPYOLOE bbox heat map visualization script (with backbone featuremap)

bash

python tools/cam_ppdet.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml --infer_img demo/000000014439.jpg --cam_out cam_ppyoloe -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

MaskRCNN bbox heat map visualization script (with roi featuremap)

bash

python tools/cam_ppdet.py -c configs/mask_rcnn/mask_rcnn_r50_vd_fpn_2x_coco.yml --infer_img demo/000000014439.jpg  --cam_out cam_mask_rcnn_roi --target_feature_layer_name model.bbox_head.roi_extractor -o weights=https://paddledet.bj.bcebos.com/models/mask_rcnn_r50_vd_fpn_2x_coco.pdparams

MaskRCNN bbox heat map visualization script (with backbone featuremap)

bash

python tools/cam_ppdet.py -c configs/mask_rcnn/mask_rcnn_r50_vd_fpn_2x_coco.yml --infer_img demo/000000014439.jpg  --cam_out cam_mask_rcnn_backbone --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/mask_rcnn_r50_vd_fpn_2x_coco.pdparams

FasterRCNN bbox heat map visualization script (with roi featuremap)

bash

python tools/cam_ppdet.py -c configs/faster_rcnn/faster_rcnn_r50_vd_fpn_2x_coco.yml --infer_img demo/000000014439.jpg  --cam_out cam_faster_rcnn_roi --target_feature_layer_name model.bbox_head.roi_extractor -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_vd_fpn_ssld_2x_coco.pdparams

FasterRCNN bbox heat map visualization script (with backbone featuremap)

bash

python tools/cam_ppdet.py -c configs/faster_rcnn/faster_rcnn_r50_vd_fpn_2x_coco.yml --infer_img demo/000000014439.jpg  --cam_out cam_faster_rcnn_backbone --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_vd_fpn_ssld_2x_coco.pdparams

BlaczeFace bbox heat map visualization script (with backbone featuremap)

bash

python tools/cam_ppdet.py -c configs/face_detection/blazeface_1000e.yml --infer_img demo/hrnet_demo.jpg --cam_out cam_blazeface --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/blazeface_1000e.pdparams

SSD bbox heat map visualization script (with backbone featuremap)

bash

python tools/cam_ppdet.py -c configs/ssd/ssd_mobilenet_v1_300_120e_voc.yml --infer_img demo/000000014439.jpg --cam_out cam_ssd --target_feature_layer_name model.backbone -o weights=https://paddledet.bj.bcebos.com/models/ssd_mobilenet_v1_300_120e_voc.pdparams

Retinanet bbox heat map visualization script (with backbone featuremap)

bash

python tools/cam_ppdet.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml --infer_img demo/000000014439.jpg --cam_out cam_retinanet --target_feature_layer_name model.backbone -o weights=https://bj.bcebos.com/v1/paddledet/models/retinanet_r50_fpn_2x_coco.pdparams

Tutorials INSTALL (`docs/tutorials/INSTALL.md`)

English | 简体中文

Installation

This document covers how to install PaddleDetection and its dependencies
(including PaddlePaddle), together with COCO and Pascal VOC dataset.

For general information about PaddleDetection, please see README.md.

Requirements:

PaddlePaddle 2.2
OS 64 bit
Python 3(3.5.1+/3.6/3.7/3.8/3.9/3.10)，64 bit
pip/pip3(9.0.1+), 64 bit
CUDA >= 10.2
cuDNN >= 7.6

Dependency of PaddleDetection and PaddlePaddle:

PaddleDetection version	PaddlePaddle version	tips
develop	>= 2.3.2	Dygraph mode is set as default
release/2.6	>= 2.3.2	Dygraph mode is set as default
release/2.5	>= 2.2.2	Dygraph mode is set as default
release/2.4	>= 2.2.2	Dygraph mode is set as default
release/2.3	>= 2.2.0rc	Dygraph mode is set as default
release/2.2	>= 2.1.2	Dygraph mode is set as default
release/2.1	>= 2.1.0	Dygraph mode is set as default
release/2.0	>= 2.0.1	Dygraph mode is set as default
release/2.0-rc	>= 2.0.1	--
release/0.5	>= 1.8.4	Cascade R-CNN and SOLOv2 depends on 2.0.0.rc
release/0.4	>= 1.8.4	PP-YOLO depends on 1.8.4
release/0.3	>=1.7	--

Instruction

1. Install PaddlePaddle

text

# CUDA10.2
python -m pip install paddlepaddle-gpu==2.3.2 -i https://pypi.tuna.tsinghua.edu.cn/simple

# CPU
python -m pip install paddlepaddle==2.3.2 -i https://pypi.tuna.tsinghua.edu.cn/simple

For more CUDA version or environment to quick install, please refer to the PaddlePaddle Quick Installation document
For more installation methods such as conda or compile with source code, please refer to the installation document

Please make sure that your PaddlePaddle is installed successfully and the version is not lower than the required version. Use the following command to verify.

text

# check
>>> import paddle
>>> paddle.utils.run_check()

# confirm the paddle's version
python -c "import paddle; print(paddle.__version__)"

Note

If you want to use PaddleDetection on multi-GPU, please install NCCL at first.

2. Install PaddleDetection

Note: Installing via pip only supports Python3

text

# Clone PaddleDetection repository
cd <path/to/clone/PaddleDetection>
git clone https://github.com/PaddlePaddle/PaddleDetection.git

# Install other dependencies
cd PaddleDetection
pip install -r requirements.txt

# Compile and install paddledet
python setup.py install

Note

If you are working on Windows OS, pycocotools installing may failed because of the origin version of cocoapi does not support windows, another version can be used used which only supports Python3:

pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI
If you are using Python <= 3.6, pycocotools installing may failed with error like distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cython>=0.27.3'), please install cython firstly, for example pip install cython

After installation, make sure the tests pass:

shell

python ppdet/modeling/tests/test_architectures.py

If the tests are passed, the following information will be prompted:

text

.......
----------------------------------------------------------------------
Ran 7 tests in 12.816s
OK

Use built Docker images

If you do not have a Docker environment, please refer to Docker.

We provide docker images containing the latest PaddleDetection code, and all environment and package dependencies are pre-installed. All you have to do is to pull and run the docker image. Then you can enjoy PaddleDetection without any extra steps.

Get these images and guidance in docker hub, including CPU, GPU, ROCm environment versions.

If you have some customized requirements about automatic building docker images, you can get it in github repo PaddlePaddle/PaddleCloud.

Inference demo

Congratulation! Now you have installed PaddleDetection successfully and try our inference demo:

text

# Predict an image by GPU
export CUDA_VISIBLE_DEVICES=0
python tools/infer.py -c configs/ppyolo/ppyolo_r50vd_dcn_1x_coco.yml -o use_gpu=true weights=https://paddledet.bj.bcebos.com/models/ppyolo_r50vd_dcn_1x_coco.pdparams --infer_img=demo/000000014439.jpg

An image of the same name with the predicted result will be generated under the output folder.
The result is as shown below：

Tutorials INSTALL Cn (`docs/tutorials/INSTALL_cn.md`)

English | 简体中文

安装文档

环境要求

PaddlePaddle 2.3.2
OS 64位操作系统
Python 3(3.5.1+/3.6/3.7/3.8/3.9/3.10)，64位版本
pip/pip3(9.0.1+)，64位版本
CUDA >= 10.2
cuDNN >= 7.6

PaddleDetection 依赖 PaddlePaddle 版本关系：

PaddleDetection版本	PaddlePaddle版本	备注
develop	>=2.3.2	默认使用动态图模式
release/2.6	>=2.3.2	默认使用动态图模式
release/2.5	>= 2.2.2	默认使用动态图模式
release/2.4	>= 2.2.2	默认使用动态图模式
release/2.3	>= 2.2.0rc	默认使用动态图模式
release/2.2	>= 2.1.2	默认使用动态图模式
release/2.1	>= 2.1.0	默认使用动态图模式
release/2.0	>= 2.0.1	默认使用动态图模式
release/2.0-rc	>= 2.0.1	--
release/0.5	>= 1.8.4	大部分模型>=1.8.4即可运行，Cascade R-CNN系列模型与SOLOv2依赖2.0.0.rc版本
release/0.4	>= 1.8.4	PP-YOLO依赖1.8.4
release/0.3	>=1.7	--

安装说明

1. 安装PaddlePaddle

text

# CUDA10.2
python -m pip install paddlepaddle-gpu==2.3.2 -i https://pypi.tuna.tsinghua.edu.cn/simple

# CPU
python -m pip install paddlepaddle==2.3.2 -i https://pypi.tuna.tsinghua.edu.cn/simple

更多CUDA版本或环境快速安装，请参考PaddlePaddle快速安装文档
更多安装方式例如conda或源码编译安装方法，请参考PaddlePaddle安装文档

请确保您的PaddlePaddle安装成功并且版本不低于需求版本。使用以下命令进行验证。

text

# 在您的Python解释器中确认PaddlePaddle安装成功
>>> import paddle
>>> paddle.utils.run_check()

# 确认PaddlePaddle版本
python -c "import paddle; print(paddle.__version__)"

注意

如果您希望在多卡环境下使用PaddleDetection，请首先安装NCCL

2. 安装PaddleDetection

注意： pip安装方式只支持Python3

text

# 克隆PaddleDetection仓库
cd <path/to/clone/PaddleDetection>
git clone https://github.com/PaddlePaddle/PaddleDetection.git

# 安装其他依赖
cd PaddleDetection
pip install -r requirements.txt

# 编译安装paddledet
python setup.py install

注意

如果github下载代码较慢，可尝试使用gitee或者代理加速。
若您使用的是Windows系统，由于原版cocoapi不支持Windows，pycocotools依赖可能安装失败，可采用第三方实现版本，该版本仅支持Python3

pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI
若您使用的是Python <= 3.6的版本，安装pycocotools可能会报错distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cython>=0.27.3'), 您可通过先安装cython如pip install cython解决该问题

安装后确认测试通过：

text

python ppdet/modeling/tests/test_architectures.py

测试通过后会提示如下信息：

text

.......
----------------------------------------------------------------------
Ran 7 tests in 12.816s
OK

使用Docker镜像

如果您没有Docker运行环境，请参考Docker官网进行安装。

我们提供了包含最新 PaddleDetection 代码的docker镜像，并预先安装好了所有的环境和库依赖，您只需要拉取docker镜像，然后运行docker镜像，无需其他任何额外操作，即可开始使用PaddleDetection的所有功能。

在Docker Hub中获取这些镜像及相应的使用指南，包括CPU、GPU、ROCm版本。
如果您对自动化制作docker镜像感兴趣，或有自定义需求，请访问PaddlePaddle/PaddleCloud做进一步了解。

快速体验

恭喜！ 您已经成功安装了PaddleDetection，接下来快速体验目标检测效果

text

# 在GPU上预测一张图片
export CUDA_VISIBLE_DEVICES=0
python tools/infer.py -c configs/ppyolo/ppyolo_r50vd_dcn_1x_coco.yml -o use_gpu=true weights=https://paddledet.bj.bcebos.com/models/ppyolo_r50vd_dcn_1x_coco.pdparams --infer_img=demo/000000014439.jpg

会在output文件夹下生成一个画有预测结果的同名图像。

结果如下图：

Tutorials KeyPointConfigGuide Cn (`docs/tutorials/KeyPointConfigGuide_cn.md`)

# config yaml配置项说明

KeyPoint 使用时config文件配置项说明，以tinypose_256x192.yml为例

text

/* Detailed source-code truncated for AI context efficiency. */

Tutorials KeyPointConfigGuide En (`docs/tutorials/KeyPointConfigGuide_en.md`)

# config yaml guide

KeyPoint config guide，Take an example of tinypose_256x192.yml

text

/* Detailed source-code truncated for AI context efficiency. */

Tutorials QUICK STARTED (`docs/tutorials/QUICK_STARTED.md`)

English | 简体中文

Quick Start

In order to enable users to experience PaddleDetection and produce models in a short time, this tutorial introduces the pipeline to get a decent object detection model by finetuning on a small dataset in 10 minutes only. In practical applications, it is recommended that users select a suitable model configuration file for their specific demand.

Set GPU

bash

export CUDA_VISIBLE_DEVICES=0

Inference Demo with Pre-trained Models

text

# predict an image using PP-YOLO
python tools/infer.py -c configs/ppyolo/ppyolo_r50vd_dcn_1x_coco.yml -o use_gpu=true weights=https://paddledet.bj.bcebos.com/models/ppyolo_r50vd_dcn_1x_coco.pdparams --infer_img=demo/000000014439.jpg

the result：

Data preparation

The Dataset is Kaggle dataset ，including 877 images and 4 data categories: crosswalk, speedlimit, stop, trafficlight. The dataset is divided into training set (701 images) and test set (176 images)，download link.

text

# Note: this command could skip and
# the dataset will be dowloaded automatically at the stage of training.
python dataset/roadsign_voc/download_roadsign_voc.py

Training & Evaluation & Inference

1、Training

text

# It will takes about 10 minutes on 1080Ti and 1 hour on CPU
# -c set configuration file
# -o overwrite the settings in the configuration file
# --eval Evaluate while training, and a model named best_model.pdmodel with the most evaluation results will be automatically saved


python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml --eval -o use_gpu=true

If you want to observe the loss change curve in real time through VisualDL, add --use_vdl=true to the training command, and set the log save path through --vdl_log_dir.

Note: VisualDL need Python>=3.5

Please install VisualDL first

text

python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple

text

python -u tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml \
                        --use_vdl=true \
                        --vdl_log_dir=vdl_dir/scalar \
                        --eval

View the change curve in real time through the visualdl command:

text

visualdl --logdir vdl_dir/scalar/ --host <host_IP> --port <port_num>

2、Evaluation

text

# Evaluate best_model by default
# -c set config file
# -o overwrite the settings in the configuration file

python tools/eval.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o use_gpu=true

The final mAP should be around 0.85. The dataset is small so the precision may vary a little after each training.

3、Inference

text

# -c set config file
# -o overwrite the settings in the configuration file
# --infer_img image path
# After the prediction is over, an image of the same name with the prediction result will be generated in the output folder

python tools/infer.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o use_gpu=true --infer_img=demo/road554.png

The result is as shown below：

Tutorials QUICK STARTED Cn (`docs/tutorials/QUICK_STARTED_cn.md`)

English | 简体中文

快速开始

为了使得用户能够在很短时间内快速产出模型，掌握PaddleDetection的使用方式，这篇教程通过一个预训练检测模型对小数据集进行finetune。在较短时间内即可产出一个效果不错的模型。实际业务中，建议用户根据需要选择合适模型配置文件进行适配。

设置显卡

bash

export CUDA_VISIBLE_DEVICES=0

一、快速体验

text

# 用PP-YOLO算法在COCO数据集上预训练模型预测一张图片
python tools/infer.py -c configs/ppyolo/ppyolo_r50vd_dcn_1x_coco.yml -o use_gpu=true weights=https://paddledet.bj.bcebos.com/models/ppyolo_r50vd_dcn_1x_coco.pdparams --infer_img=demo/000000014439.jpg

结果如下图：

二、准备数据

数据集参考Kaggle数据集，包含877张图像，数据类别4类：crosswalk，speedlimit，stop，trafficlight。
将数据划分为训练集701张图和测试集176张图，下载链接.

text

# 注意：可跳过这步下载，后面训练会自动下载
python dataset/roadsign_voc/download_roadsign_voc.py

三、训练、评估、预测

1、训练

text

# 边训练边测试 CPU需要约1小时(use_gpu=false)，1080Ti GPU需要约10分钟
# -c 参数表示指定使用哪个配置文件
# -o 参数表示指定配置文件中的全局变量（覆盖配置文件中的设置），这里设置使用gpu
# --eval 参数表示边训练边评估，最后会自动保存一个名为model_final.pdparams的模型

python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml --eval -o use_gpu=true

如果想通过VisualDL实时观察loss变化曲线，在训练命令中添加--use_vdl=true，以及通过--vdl_log_dir设置日志保存路径。

但注意VisualDL需Python>=3.5

首先安装VisualDL

text

python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple

text

python -u tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml \
                        --use_vdl=true \
                        --vdl_log_dir=vdl_dir/scalar \
                        --eval

通过visualdl命令实时查看变化曲线：

text

visualdl --logdir vdl_dir/scalar/ --host <host_IP> --port <port_num>

2、评估

text

# 评估 默认使用训练过程中保存的model_final.pdparams
# -c 参数表示指定使用哪个配置文件
# -o 参数表示指定配置文件中的全局变量（覆盖配置文件中的设置）
# 目前只支持单卡评估

python tools/eval.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o use_gpu=true

最终模型精度在mAP=0.85左右，由于数据集较小因此每次训练结束后精度会有一定波动

3、预测

text

# -c 参数表示指定使用哪个配置文件
# -o 参数表示指定配置文件中的全局变量（覆盖配置文件中的设置）
# --infer_img 参数指定预测图像路径
# 预测结束后会在output文件夹中生成一张画有预测结果的同名图像

python tools/infer.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o use_gpu=true --infer_img=demo/road554.png

结果如下图：

Tutorials Logging En (`docs/tutorials/logging_en.md`)

Logging

This document talks about how to track metrics and visualize model performance during training. The library currently supports VisualDL and Weights & Biases.

VisualDL

Logging to VisualDL is supported only in python >= 3.5. To install VisualDL

text

pip install visualdl

PaddleDetection uses a callback to log the training metrics at the end of every step and metrics from the validation step at the end of every epoch. To use VisualDL for visualization, add the --use_vdl flag to the training command and --vdl_log_dir <logs> to set the directory which stores the records.

For example

text

python tools/train -c config.yml --use_vdl --vdl_log_dir ./logs

Another possible way to do this is to add the aforementioned flags to the config.yml file.

Weights & Biases

W&B is a MLOps tool that can be used for experiment tracking, dataset/model versioning, visualizing results and collaborating with colleagues. A W&B logger is integrated directly into PaddleDetection and to use it, first you need to install the wandb sdk and login to your wandb account.

text

pip install wandb
wandb login

To use wandb to log metrics while training add the --use_wandb flag to the training command and any other arguments for the W&B logger can be provided like this -

text

python tools/train -c config.yml --use_wandb -o wandb-project=MyDetector wandb-entity=MyTeam wandb-save_dir=./logs

The arguments to the W&B logger must be proceeded by -o and each invidiual argument must contain the prefix "wandb-".

If this is too tedious, an alternative way is to add the arguments to the config.yml file under the wandb header. For example

text

use_wandb: True
wandb:
    project: MyProject
    entity: MyTeam
    save_dir: ./logs

Test Tipc Docs Benchmark Train (`test_tipc/docs/benchmark_train.md`)

TIPC Linux端Benchmark测试文档

该文档为Benchmark测试说明，Benchmark预测功能测试的主程序为benchmark_train.sh，用于验证监控模型训练的性能。

1. 测试流程

1.1 准备数据和环境安装

运行test_tipc/prepare.sh，完成训练数据准备和安装环境流程。

shell

# 运行格式：bash test_tipc/prepare.sh  train_benchmark.txt  mode
bash test_tipc/prepare.sh test_tipc/configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco_train_infer_python.txt benchmark_train

1.2 功能测试

执行test_tipc/benchmark_train.sh，完成模型训练和日志解析

shell

# 运行格式：bash test_tipc/benchmark_train.sh train_benchmark.txt mode
bash test_tipc/benchmark_train.sh test_tipc/configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco_train_infer_python.txt benchmark_train

test_tipc/benchmark_train.sh支持根据传入的第三个参数实现只运行某一个训练配置，如下：

shell

# 运行格式：bash test_tipc/benchmark_train.sh train_benchmark.txt mode
bash test_tipc/benchmark_train.sh test_tipc/configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco_train_infer_python.txt benchmark_train  dynamic_bs2_fp32_DP_N1C1

dynamic_bs2_fp32_DP_N1C1为test_tipc/benchmark_train.sh传入的参数，格式如下：
${modeltype}_${batch_size}_${fp_item}_${run_mode}_${device_num}
包含的信息有：模型类型、batchsize大小、训练精度如fp32,fp16等、分布式运行模式以及分布式训练使用的机器信息如单机单卡（N1C1）。

2. 日志输出

运行后将保存模型的训练日志和解析日志，使用 test_tipc/configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco_train_infer_python.txt 参数文件的训练日志解析结果是：

text

{"model_branch": "tipc_fuse_benchmark", "model_commit": "4cce901d231f7954468045cf96302505bd6be495", "model_name": "faster_rcnn_r50_fpn_1x_coco_bs2_fp32_SingleP_DP", "batch_size": 2, "fp_item": "fp32", "run_process_type": "SingleP", "run_mode": "DP", "convergence_value": "0.556966", "convergence_key": "loss:", "ips": 4.857, "speed_unit": "images/s", "device_num": "N1C1", "model_run_time": "590", "frame_commit": "6b0c57cf65945e97d87a8fba89c0a2fc18dd8544", "frame_version": "0.0.0"}

训练日志和日志解析结果保存在benchmark_log目录下，文件组织格式如下：

text

train_log/
├── index
│   └── PaddleDetection_faster_rcnn_r50_fpn_1x_coco_bs2_fp32_SingleP_DP_N1C1_speed
├── profiling_log
│   └── PaddleDetection_faster_rcnn_r50_fpn_1x_coco_bs2_fp32_SingleP_DP_N1C1_profiling
└── train_log
    ├── PaddleDetection_faster_rcnn_r50_fpn_1x_coco_bs2_fp32_SingleP_DP_N1C1_log
    └── PaddleDetection_faster_rcnn_r50_fpn_1x_coco_bs2_fp32_MultiP_DP_N1C4_log

Test Tipc Docs Install (`test_tipc/docs/install.md`)

1. 环境准备

本教程适用于test_tipc目录下基础功能测试的运行环境搭建。

推荐环境：

CUDA 10.1/10.2
CUDNN 7.6/cudnn8.1
TensorRT 6.1.0.5 / 7.1 / 7.2

环境配置可以选择docker镜像安装，或者在本地环境Python搭建环境。推荐使用docker镜像安装，避免不必要的环境配置。

2. Docker 镜像安装

推荐docker镜像安装，按照如下命令创建镜像，当前目录映射到镜像中的/paddle目录下

text

# 启动docker镜像
nvidia-docker run --name paddle -it -v $PWD:/paddle paddlepaddle/paddle:latest-gpu-cuda10.1-cudnn7-gcc82-dev /bin/bash
cd /paddle

text

# 编译安装Paddle
git clone https://github.com/PaddlePaddle/Paddle.git
cd Paddle
mkdir build && cd build
cmake .. \
  -DWITH_MKL=ON \
  -DWITH_MKLDNN=ON \
  -DWITH_GPU=ON \
  -DWITH_DISTRIBUTE=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DCUDA_ARCH_NAME=Auto \
  -DPY_VERSION=3.7 \
  -DON_INFER=ON \
  -DWITH_TENSORRT=ON \
  -DTENSORRT_ROOT=/usr/local/TensorRT6-cuda10.1-cudnn7
make -j 20
pip3.7 install python/dist/paddlepaddle_gpu-0.0.0-cp37-cp37m-linux_x86_64.whl
cd ../../

text

# 下载安装Paddle-2.2
wget https://paddle-inference-lib.bj.bcebos.com/2.2.0/python/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.1_cudnn7.6.5_trt6.0.1.5/paddlepaddle_gpu-2.2.0.post101-cp37-cp37m-linux_x86_64.whl
pip3.7 install paddlepaddle_gpu-2.2.0.post101-cp37-cp37m-linux_x86_64.whl
# 下载C++预测库用于C++ inference
wget https://paddle-inference-lib.bj.bcebos.com/2.2.0/cxx_c/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.1_cudnn7.6.5_trt6.0.1.5/paddle_inference.tgz
tar -xvf paddle_inference.tgz
export PADDLE_DIR=/paddle/paddle_inference

3 Python 环境构建

如果您已经通过docker方式构建环境，跳过该部分内容。非docker环境下，环境配置比较灵活，推荐环境组合配置：

CUDA10.1 + CUDNN7.6 + TensorRT 6
CUDA10.2 + CUDNN8.1 + TensorRT 7
CUDA11.1 + CUDNN8.1 + TensorRT 7

下面以 CUDA10.2 + CUDNN8.1 + TensorRT 7 配置为例，介绍环境配置的流程。

3.1 安装CUDNN

如果当前环境满足CUDNN版本的要求，可以跳过此步骤。

以CUDNN8.1 安装安装为例，安装步骤如下，首先下载CUDNN，从Nvidia官网下载CUDNN8.1版本，下载符合当前系统版本的三个deb文件，分别是：

cuDNN Runtime Library ，如：libcudnn8_8.1.0.77-1+cuda10.2_amd64.deb
cuDNN Developer Library ，如：libcudnn8-dev_8.1.0.77-1+cuda10.2_amd64.deb
cuDNN Code Samples，如：libcudnn8-samples_8.1.0.77-1+cuda10.2_amd64.deb

deb安装可以参考官方文档，安装方式如下

text

# x.x.x表示下载的版本号
# $HOME为工作目录
sudo dpkg -i libcudnn8_x.x.x-1+cudax.x_arm64.deb
sudo dpkg -i libcudnn8-dev_8.x.x.x-1+cudax.x_arm64.deb
sudo dpkg -i libcudnn8-samples_8.x.x.x-1+cudax.x_arm64.deb

# 验证是否正确安装
cp -r /usr/src/cudnn_samples_v8/ $HOME
cd  $HOME/cudnn_samples_v8/mnistCUDNN

# 编译
make clean && make
./mnistCUDNN

如果运行mnistCUDNN完后提示运行成功，则表示安装成功。如果运行后出现freeimage相关的报错，需要按照提示安装freeimage库:

text

sudo apt-get install libfreeimage-dev
sudo apt-get install libfreeimage

3.2 安装TensorRT

首先，从Nvidia官网TensorRT板块下载TensorRT，这里选择7.1.3.4版本的TensorRT，注意选择适合自己系统版本和CUDA版本的TensorRT，另外建议下载TAR package的安装包。

以Ubuntu16.04+CUDA10.2为例，下载并解压后可以参考官方文档的安装步骤，按照如下步骤安装:

text

# 以下安装命令中 '${version}' 为下载的TensorRT版本，如7.1.3.4
# 设置环境变量，<TensorRT-${version}/lib> 为解压后的TensorRT的lib目录
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<TensorRT-${version}/lib>

# 安装TensorRT
cd TensorRT-${version}/python
pip3.7 install tensorrt-*-cp3x-none-linux_x86_64.whl

# 安装graphsurgeon
cd TensorRT-${version}/graphsurgeon

3.3 安装PaddlePaddle

下载支持TensorRT版本的Paddle安装包，注意安装包的TensorRT版本需要与本地TensorRT一致，下载链接
选择下载 linux-cuda10.2-trt7-gcc8.2 Python3.7版本的Paddle：

text

# 从下载链接中可以看到是paddle2.1.1-cuda10.2-cudnn8.1版本
wget  https://paddle-wheel.bj.bcebos.com/with-trt/2.1.1-gpu-cuda10.2-cudnn8.1-mkl-gcc8.2/paddlepaddle_gpu-2.1.1-cp37-cp37m-linux_x86_64.whl
pip3.7 install -U paddlepaddle_gpu-2.1.1-cp37-cp37m-linux_x86_64.whl

4. 安装PaddleDetection依赖

text

# 安装AutoLog
git clone https://github.com/LDOUBLEV/AutoLog
cd AutoLog
pip3.7 install -r requirements.txt
python3.7 setup.py bdist_wheel
pip3.7 install ./dist/auto_log-1.0.0-py3-none-any.whl

# 下载PaddleDetection代码
cd ../
git clone https://github.com/PaddlePaddle/PaddleDetection

安装PaddleDetection依赖：

text

cd PaddleDetection
pip3.7 install -r ./requirements.txt

FAQ :

Q. You are using Paddle compiled with TensorRT, but TensorRT dynamic library is not found. Ignore this if TensorRT is not needed.

A. 问题一般是当前安装paddle版本带TRT，但是本地环境找不到TensorRT的预测库，需要下载TensorRT库，解压后设置环境变量LD_LIBRARY_PATH;
如：

text

export LD_LIBRARY_PATH=/usr/local/python3.7.0/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/paddle/package/TensorRT-6.0.1.5/lib

或者问题是下载的TensorRT版本和当前paddle中编译的TRT版本不匹配，需要下载版本相符的TensorRT重新安装。

Test Tipc Docs More Models (`test_tipc/docs/more_models.md`)

汇总信息

已填写的部分表示可以使用本工具进行一键测试，未填写的表示正在支持中。

字段说明：

基础训练预测：包括模型训练、Paddle Inference Python预测。
更多训练方式：包括多机多卡、混合精度。
模型压缩：包括裁剪、离线/在线量化、蒸馏。
其他预测部署：包括Paddle Inference C++预测、Paddle Serving部署、Paddle-Lite部署等。

算法论文	模型名称	模型类型	基础训练预测	更多训练方式	模型压缩	其他预测部署
YOLOv3	yolov3_darknet53_270e_coco	目标检测	支持	混合精度	FPGM裁剪 PACT量化离线量化	Paddle Inference: C++
YOLOv3	yolov3_mobilenet_v1_270e_coco	目标检测	支持	混合精度		Paddle Inference: C++
YOLOv3	yolov3_mobilenet_v3_large_270e_coco	目标检测	支持	混合精度		Paddle Inference: C++
YOLOv3	yolov3_r34_270e_coco	目标检测	支持	混合精度		Paddle Inference: C++
YOLOv3	yolov3_r50vd_dcn_270e_coco	目标检测	支持	混合精度		Paddle Inference: C++
PPYOLO	ppyolo_mbv3_large_coco	目标检测	支持	混合精度	FPGM裁剪 PACT量化离线量化	Paddle Inference: C++
PPYOLO	ppyolo_r50vd_dcn_1x_coco	目标检测	支持	混合精度	FPGM裁剪 PACT量化离线量化	Paddle Inference: C++
PPYOLO	ppyolo_mbv3_small_coco	目标检测	支持	混合精度		Paddle Inference: C++
PPYOLO	ppyolo_r18vd_coco	目标检测	支持	混合精度		Paddle Inference: C++
PPYOLO-tiny	ppyolo_tiny_650e_coco	目标检测	支持	混合精度		Paddle Inference: C++
PPYOLOv2	ppyolov2_r50vd_dcn_365e_coco	目标检测	支持	多机多卡混合精度		Paddle Inference: C++
PPYOLOv2	ppyolov2_r50vd_dcn_365e_coco	目标检测	支持	混合精度		Paddle Inference: C++
PPYOLOv2	ppyolov2_r101vd_dcn_365e_coco	目标检测	支持	混合精度		Paddle Inference: C++
PP-PicoDet	picodet_s_320_coco	目标检测	支持	混合精度		Paddle Inference: C++
PP-PicoDet	picodet_m_416_coco	目标检测	支持	混合精度		Paddle Inference: C++
PP-PicoDet	picodet_l_640_coco	目标检测	支持	混合精度		Paddle Inference: C++
PP-PicoDet	picodet_lcnet_1_5x_416_coco	目标检测	支持	混合精度		Paddle Inference: C++
PP-PicoDet	picodet_mobilenetv3_large_1x_416_coco	目标检测	支持	混合精度		Paddle Inference: C++
PP-PicoDet	picodet_r18_640_coco	目标检测	支持	混合精度		Paddle Inference: C++
PP-PicoDet	picodet_shufflenetv2_1x_416_coco	目标检测	支持	混合精度		Paddle Inference: C++
SSD	ssdlite_mobilenet_v1_300_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r50_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r34_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r34_vd_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r50_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r50_vd_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r50_vd_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r101_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r101_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_r101_vd_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_x101_vd_64x4d_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Faster R-CNN	faster_rcnn_swin_tiny_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Cascade Faster R-CNN	cascade_rcnn_r50_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
Cascade Faster R-CNN	cascade_rcnn_r50_vd_fpn_ssld_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
FCOS	fcos_r50_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
FCOS	fcos_dcn_r50_fpn_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
TTFNet	ttfnet_darknet53_1x_coco	目标检测	支持	混合精度		Paddle Inference: C++
S2ANet	s2anet_conv_2x_dota	目标检测	支持	混合精度		Paddle Inference: C++
S2ANet	s2anet_1x_spine	目标检测	支持	混合精度		Paddle Inference: C++
S2ANet	s2anet_alignconv_2x_dota	目标检测	支持	混合精度		Paddle Inference: C++
BlazeFace	blazeface_1000e	目标检测	支持	混合精度		Paddle Inference: C++
BlazeFace	blazeface_fpn_ssh_1000e	目标检测	支持	混合精度		Paddle Inference: C++
Mask R-CNN	mask_rcnn_r50_fpn_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
Mask R-CNN	mask_rcnn_r50_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
Mask R-CNN	mask_rcnn_r50_vd_fpn_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
Mask R-CNN	mask_rcnn_r101_fpn_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
Mask R-CNN	mask_rcnn_r101_vd_fpn_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
Mask R-CNN	mask_rcnn_x101_vd_64x4d_fpn_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
Cascade Mask R-CNN	cascade_mask_rcnn_r50_fpn_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
Cascade Mask R-CNN	cascade_mask_rcnn_r50_vd_fpn_ssld_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
SOLOv2	solov2_r50_fpn_1x_coco	实例分割	支持	混合精度		Paddle Inference: C++
SOLOv2	solov2_r50_enhance_coco	实例分割	支持	混合精度		Paddle Inference: C++
SOLOv2	solov2_r101_vd_fpn_3x_coco	实例分割	支持	混合精度		Paddle Inference: C++
[PP-Tinypose]	tinypose_128x96	关键点检测	支持	混合精度		Paddle Inference: C++
HRNet	hrnet_w32_256x192	关键点检测	支持	混合精度		Paddle Inference: C++
HRNet	dark_hrnet_w32_256x192	关键点检测	支持	混合精度		Paddle Inference: C++
HRNet	dark_hrnet_w48_256x192	关键点检测	支持	混合精度		Paddle Inference: C++
HigherHRNet	higherhrnet_hrnet_w32_512	关键点检测	支持	混合精度		Paddle Inference: C++
FairMot	fairmot_dla34_30e_576x320	目标跟踪	支持	混合精度		Paddle Inference: C++
FairMot	fairmot_hrnetv2_w18_dlafpn_30e_576x320	目标跟踪	支持	混合精度		Paddle Inference: C++
JDE	jde_darknet53_30e_576x320	目标跟踪	支持	混合精度		Paddle Inference: C++

Test Tipc Docs Test Inference Cpp (`test_tipc/docs/test_inference_cpp.md`)

C++预测功能测试

C++预测功能测试的主程序为test_inference_cpp.sh，可以测试基于C++预测库的模型推理功能。

1. 测试结论汇总

基于训练是否使用量化，进行本测试的模型可以分为正常模型和量化模型，这两类模型对应的C++预测功能汇总如下：

模型类型	device	batchsize	tensorrt	mkldnn	cpu多线程
正常模型	GPU	1/8	fp32/fp16	-	-
正常模型	CPU	1/8	-	fp32	支持
量化模型	GPU	1/8	int8	-	-
量化模型	CPU	1/8	-	int8	支持

2. 测试流程

运行环境配置请参考文档的内容配置TIPC的运行环境。

text

# 请设置paddle_inference环境变量，如：
export PADDLE_INFER_DIR=/path/to/paddle_inference
# 若不设置paddle_inference环境变量，也可通过指定参数的方式使脚本自动下载paddle_inference.tgz，如：
bash test_tipc/test_inference_cpp.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_infer_cpp_linux_gpu_cpu.txt 'https://paddle-inference-lib.bj.bcebos.com/2.3.0/cxx_c/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.1_cudnn7.6.5_trt6.0.1.5/paddle_inference.tgz'

# 若未使用docker镜像: paddlepaddle/paddle:latest-gpu-cuda10.1-cudnn7-gcc82-dev
# 请设置TensorRT环境变量，如：
export TENSORRT_ROOT=/usr/local/TensorRT6-cuda10.1-cudnn7

2.1 功能测试

先运行prepare.sh准备数据和模型，然后运行test_inference_cpp.sh进行测试，最终在test_tipc/output目录下生成cpp_infer_*.log后缀的日志文件。

shell

bash test_tipc/prepare.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_infer_cpp_linux_gpu_cpu.txt "cpp_infer"
# 用法1:
bash test_tipc/test_inference_cpp.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_infer_cpp_linux_gpu_cpu.txt
# 用法2: 指定下载paddle_inference.tgz链接，第二个传入参数为下载链接
bash test_tipc/test_inference_cpp.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_infer_cpp_linux_gpu_cpu.txt 'https://paddle-inference-lib.bj.bcebos.com/2.3.0/cxx_c/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.1_cudnn7.6.5_trt6.0.1.5/paddle_inference.tgz'
# 用法3: 同时指定下载paddle_inference.tgz链接和指定GPU卡预测，第三个传入参数为GPU卡号
bash test_tipc/test_inference_cpp.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_infer_cpp_linux_gpu_cpu.txt 'https://paddle-inference-lib.bj.bcebos.com/2.3.0/cxx_c/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.1_cudnn7.6.5_trt6.0.1.5/paddle_inference.tgz' '1'

运行预测指令后，在test_tipc/output文件夹下自动会保存运行日志，包括以下文件：

shell

test_tipc/output/
|- results_cpp.log    # 运行指令状态的日志
|- cpp_infer_cpu_usemkldnn_False_threads_1_precision_fluid_batchsize_1.log  # CPU上不开启Mkldnn，线程数设置为1，测试batch_size=1条件下的预测运行日志
|- cpp_infer_cpu_usemkldnn_False_threads_6_precision_fluid_batchsize_1.log  # CPU上不开启Mkldnn，线程数设置为6，测试batch_size=1条件下的预测运行日志
|- cpp_infer_gpu_precision_fluid_batchsize_1.log # GPU上不开启TensorRT，测试batch_size=1的fp32精度预测日志
|- cpp_infer_gpu_precision_trt_fp16_batchsize_1.log  # GPU上开启TensorRT，测试batch_size=1的fp16精度预测日志
......

其中results_cpp.log中包含了每条指令的运行状态，如果运行成功会输出：

text

Run successfully with command - python3.7 tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams filename=yolov3_darknet53_270e_coco --output_dir=./output_inference !
Run successfully with command - ./deploy/cpp/build/main --device=gpu --run_mode=fluid --model_dir=./output_inference/yolov3_darknet53_270e_coco --batch_size=8 --image_dir=./dataset/coco/test2017/ --run_benchmark=True   > ./test_tipc/output/cpp_infer_gpu_precision_fluid_batchsize_8.log 2>&1 !
......

如果运行失败，会输出：

text

Run failed with command - python3.7 tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams filename=yolov3_darknet53_270e_coco --output_dir=./output_inference !
Run failed with command - ./deploy/cpp/build/main --device=gpu --run_mode=fluid --model_dir=./output_inference/yolov3_darknet53_270e_coco --batch_size=8 --image_dir=./dataset/coco/test2017/ --run_benchmark=True   > ./test_tipc/output/cpp_infer_gpu_precision_fluid_batchsize_8.log 2>&1 !
......

可以很方便的根据results_cpp.log中的内容判定哪一个指令运行错误。

2.2 精度测试

使用compare_results.py脚本比较模型预测的结果是否符合预期，主要步骤包括：

提取日志中的预测坐标；
从本地文件中提取保存好的坐标结果；
比较上述两个结果是否符合精度预期，误差大于设置阈值时会报错。

使用方式

运行命令：

shell

python3.7 test_tipc/compare_results.py --gt_file=./test_tipc/results/cpp_*.txt  --log_file=./test_tipc/output/cpp_*.log --atol=1e-3 --rtol=1e-3

参数介绍：

gt_file：指向事先保存好的预测结果路径，支持*.txt 结尾，会自动索引*.txt格式的文件，文件默认保存在test_tipc/result/ 文件夹下
log_file: 指向运行test_tipc/test_inference_cpp.sh 脚本的infer模式保存的预测日志，预测日志中打印的有预测结果，比如：文本框，预测文本，类别等等，同样支持cpp_infer_*.log格式传入
atol: 设置的绝对误差
rtol: 设置的相对误差

运行结果

正常运行效果如下图：

出现不一致结果时的运行输出：

3. 更多教程

本文档为功能测试用，更详细的c++预测使用教程请参考：C++预测

Test Tipc Docs Test Paddle2onnx (`test_tipc/docs/test_paddle2onnx.md`)

Paddle2onnx预测功能测试

PaddleServing预测功能测试的主程序为test_paddle2onnx.sh，可以测试Paddle2ONNX的模型转化功能，并验证正确性。

1. 测试结论汇总

基于训练是否使用量化，进行本测试的模型可以分为正常模型和量化模型，这两类模型对应的Paddle2ONNX预测功能汇总如下：

模型类型	device
正常模型	GPU
正常模型	CPU
量化模型	GPU
量化模型	CPU

2. 测试流程

2.1 功能测试

先运行prepare.sh准备数据和模型，然后运行test_paddle2onnx.sh进行测试，最终在test_tipc/output目录下生成paddle2onnx_infer_*.log后缀的日志文件。

shell

bash test_tipc/prepare.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_paddle2onnx_python_linux_cpu.txt "paddle2onnx_infer"

# 用法:
bash test_tipc/test_paddle2onnx.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_paddle2onnx_python_linux_cpu.txt

运行结果

各测试的运行情况会打印在 test_tipc/output/results_paddle2onnx.log 中：
运行成功时会输出：

text

Run successfully with command - yolov3_darknet53_270e_coco - paddle2onnx --model_dir=./output_inference/yolov3_darknet53_270e_coco --model_filename=model.pdmodel --params_filename=model.pdiparams --save_file=./output_inference/yolov3_darknet53_270e_coco/model.onnx --opset_version=11 --enable_onnx_checker=True  !
Run successfully with command - yolov3_darknet53_270e_coco - python3.7 ./deploy/third_engine/onnx/infer.py --infer_cfg=./output_inference/yolov3_darknet53_270e_coco/infer_cfg.yml --onnx_file=./output_inference/yolov3_darknet53_270e_coco/model.onnx --image_file=./demo/000000014439.jpg   > ./test_tipc/output/paddle2onnx_infer_cpu.log 2>&1 !

运行失败时会输出：

text

Run failed with command - yolov3_darknet53_270e_coco - paddle2onnx --model_dir=./output_inference/yolov3_darknet53_270e_coco --model_filename=model.pdmodel --params_filename=model.pdiparams --save_file=./output_inference/yolov3_darknet53_270e_coco/model.onnx --opset_version=11 --enable_onnx_checker=True  !
...

3. 更多教程

本文档为功能测试用，更详细的Paddle2onnx预测使用教程请参考：Paddle2ONNX

Test Tipc Docs Test Ptq Inference Python (`test_tipc/docs/test_ptq_inference_python.md`)

Linux GPU/CPU 离线量化功能测试

Linux GPU/CPU 离线量化功能测试的主程序为test_ptq_inference_python.sh，可以测试基于Python的离线量化功能。

1. 测试结论汇总

模型类型	device	batchsize	tensorrt	mkldnn	cpu多线程
量化模型	GPU	1/2	int8	-	-
量化模型	CPU	1/2	-	int8	支持

2. 测试流程

2.1 功能测试

先运行prepare.sh准备数据和模型，然后运行test_ptq_inference_python.sh进行测试，最终在test_tipc/output目录下生成python_infer_*.log后缀的日志文件。

shell

bash test_tipc/prepare.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_ptq_infer_python.txt "whole_infer"

# 用法:
bash test_tipc/test_ptq_inference_python.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_ptq_infer_python.txt

运行结果

各测试的运行情况会打印在 test_tipc/output/results_ptq_python.log 中：
运行成功时会输出：

text

Run successfully with command - yolov3_darknet53_270e_coco - python3.7 tools/post_quant.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml --slim_config configs/slim/post_quant/yolov3_darknet53_ptq.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams filename=yolov3_darknet53_270e_coco --output_dir=./output_inference !
Run successfully with command - yolov3_darknet53_270e_coco - python3.7 ./deploy/python/infer.py --device=gpu --run_mode=paddle --model_dir=./output_inference/yolov3_darknet53_270e_coco --batch_size=2 --image_dir=./dataset/coco/test2017/ --run_benchmark=False   > ./test_tipc/output/yolov3_darknet53_270e_coco/whole_infer/python_infer_gpu_mode_paddle_batchsize_2.log 2>&1 !
...

运行失败时会输出：

text

Run failed with command - yolov3_darknet53_270e_coco - python3.7 tools/post_quant.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml --slim_config configs/slim/post_quant/yolov3_darknet53_ptq.yml -o weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams filename=yolov3_darknet53_270e_coco --output_dir=./output_inference!
...

3. 更多教程

本文档为功能测试用，更详细的离线量化功能使用教程请参考：Paddle 离线量化官网教程

Test Tipc Docs Test Serving (`test_tipc/docs/test_serving.md`)

PaddleServing预测功能测试

PaddleServing预测功能测试的主程序为test_serving_infer_python.sh和test_serving_infer_cpp.sh，可以测试基于PaddleServing的部署功能。

1. 测试结论汇总

基于训练是否使用量化，进行本测试的模型可以分为正常模型和量化模型，这两类模型对应的Serving预测功能汇总如下：

模型类型	device	batchsize	tensorrt	mkldnn	cpu多线程
正常模型	GPU	1/2	fp32/fp16	-	-
正常模型	CPU	1/2	-	fp32	支持
量化模型	GPU	1/2	int8	-	-
量化模型	CPU	1/2	-	int8	支持

2. 测试流程

运行环境配置请参考文档的内容配置TIPC的运行环境。

2.1 功能测试

python serving
先运行prepare.sh准备数据和模型，然后运行test_serving_infer_python.sh进行测试，最终在test_tipc/output目录下生成serving_infer_python*.log后缀的日志文件。

shell

bash test_tipc/prepare.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_serving_python_linux_gpu_cpu.txt "serving_infer"

# 用法1:
bash test_tipc/test_serving_infer_python.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_serving_python_linux_gpu_cpu.txt
# 用法2: 指定GPU卡预测，第二个传入参数为GPU卡号
bash test_tipc/test_serving_infer_python.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_serving_python_linux_gpu_cpu.txt "1"

cpp serving
先运行prepare.sh准备数据和模型，然后运行test_serving_infer_cpp.sh进行测试，最终在test_tipc/output目录下生成serving_infer_cpp*.log后缀的日志文件。

shell

bash test_tipc/prepare.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_serving_cpp_linux_gpu_cpu.txt "serving_infer"

# 用法:
bash test_tipc/test_serving_infer_cpp.sh test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_model_linux_gpu_normal_normal_serving_cpp_linux_gpu_cpu.txt

运行结果

各测试的运行情况会打印在 test_tipc/output/results_serving.log 中：
运行成功时会输出：

text

Run successfully  with command - python3.7 pipeline_http_client.py --image_dir=../../doc/imgs > ../../tests/output/server_infer_cpu_usemkldnn_True_threads_1_batchsize_1.log 2>&1 !
Run successfully  with command - xxxxx
...

运行失败时会输出：

text

Run failed with command - python3.7 pipeline_http_client.py --image_dir=../../doc/imgs > ../../tests/output/server_infer_cpu_usemkldnn_True_threads_1_batchsize_1.log 2>&1 !
Run failed with command - python3.7 pipeline_http_client.py --image_dir=../../doc/imgs > ../../tests/output/server_infer_cpu_usemkldnn_True_threads_6_batchsize_1.log 2>&1 !
Run failed with command - xxxxx
...

详细的预测结果会存在 test_tipc/output/ 文件夹下，例如server_infer_gpu_usetrt_True_precision_fp32_batchsize_1.log中会返回检测框的坐标:

text

/* Detailed source-code truncated for AI context efficiency. */

3. 更多教程

本文档为功能测试用，更详细的Serving预测使用教程请参考：PaddleDetection 服务化部署

Test Tipc Docs Test Train Fleet Inference Python (`test_tipc/docs/test_train_fleet_inference_python.md`)

Linux GPU/CPU 多机多卡训练推理测试

Linux GPU/CPU 多机多卡训练推理测试的主程序为test_train_fleet_inference_python.sh，可以测试基于Python的模型训练、评估、推理等基本功能。

1. 测试结论汇总

训练相关：

算法名称	模型名称	多机多卡
PP-YOLOE	ppyoloe_crn_s_300e_coco	分布式训练

推理相关：

算法名称	模型名称	device_CPU	device_GPU	batchsize
PP-YOLOE	ppyoloe_crn_s_300e_coco	支持	支持	1, 2

2. 测试流程

运行环境配置请参考文档的内容配置TIPC的运行环境。

2.1 功能测试

2.1.1 修改配置文件

首先，修改配置文件中的ip设置: 假设两台机器的ip地址分别为192.168.0.1和192.168.0.2，则对应的配置文件gpu_list字段需要修改为gpu_list:192.168.0.1,192.168.0.2;0,1； ip地址查看命令为ifconfig。

2.1.2 准备数据

运行prepare.sh准备数据和模型，以配置文件test_tipc/configs/ppyoloe/ppyoloe_crn_s_300e_coco_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt为例，数据准备命令如下所示。

shell

bash test_tipc/prepare.sh test_tipc/configs/ppyoloe/ppyoloe_crn_s_300e_coco_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt lite_train_lite_infer

注意： 由于是多机训练，这里需要在所有的节点上均运行启动上述命令，准备数据。

2.1.3 修改起始端口并开始测试

在多机的节点上使用下面的命令设置分布式的起始端口（否则后面运行的时候会由于无法找到运行端口而hang住），一般建议设置在10000~20000之间。

shell

export FLAGS_START_PORT=17000

以配置文件test_tipc/configs/ppyoloe/ppyoloe_crn_s_300e_coco_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt为例，测试方法如下所示。

shell

bash test_tipc/test_train_inference_python.sh  test_tipc/configs/ppyoloe/ppyoloe_crn_s_300e_coco_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt lite_train_lite_infer

注意： 由于是多机训练，这里需要在所有的节点上均运行启动上述命令进行测试。

2.1.4 输出结果

输出结果如下，表示命令运行成功。

bash

Run successfully with command - python3.7 -m paddle.distributed.launch --ips=192.168.0.1,192.168.0.2 --gpus=0,1
 tools/train.py -c configs/ppyoloe/ppyoloe_crn_s_300e_coco.yml -o log_iter=1 use_gpu=True save_dir=./test_tipc/outpu
t/ppyoloe_crn_s_300e_coco/norm_train_gpus_0,1_autocast_null_nodes_2 epoch=1 pretrain_weights=https://paddledet.bj.bc
ebos.com/models/ppyoloe_crn_s_300e_coco.pdparams TrainReader.batch_size=2 filename=ppyoloe_crn_s_300e_coco    !

 ......
 Run successfully with command - python3.7 ./deploy/python/infer.py --device=cpu --enable_mkldnn=False --cpu_threads
=4 --model_dir=./test_tipc/output/ppyoloe_crn_s_300e_coco/norm_train_gpus_0,1_autocast_null_nodes_2/ppyoloe_crn_s_30
0e_coco --batch_size=2 --image_dir=./dataset/coco/test2017/ --run_benchmark=False --trt_max_shape=1600 > ./test_tipc
/output/ppyoloe_crn_s_300e_coco/python_infer_cpu_usemkldnn_False_threads_4_precision_fluid_batchsize_2.log 2>&1 !

注意： 由于分布式训练时，仅在trainer_id=0所在的节点中保存模型，因此其他的节点中在运行模型导出与推理时会报错，为正常现象。

Test Tipc Docs Test Train Inference Python (`test_tipc/docs/test_train_inference_python.md`)

Linux端基础训练预测功能测试

Linux端基础训练预测功能测试的主程序为test_train_inference_python.sh，可以测试基于Python的模型训练、评估、推理等基本功能，包括裁剪、量化、蒸馏。

Mac端基础训练预测功能测试参考链接
Windows端基础训练预测功能测试参考链接

1. 测试结论汇总

训练相关：

算法名称	模型名称	单机单卡	单机多卡	多机多卡	模型压缩（单机多卡）
PPYOLO	ppyolo_mbv3_large_coco	正常训练混合精度	正常训练混合精度	正常训练混合精度	正常训练：FPGM裁剪、PACT量化离线量化（无需训练）
PPYOLO	ppyolo_r50vd_dcn_1x_coco	正常训练混合精度	正常训练混合精度	正常训练混合精度	正常训练：FPGM裁剪、PACT量化离线量化（无需训练）

预测相关：基于训练是否使用量化，可以将训练产出的模型可以分为正常模型和量化模型，这两类模型对应的预测功能汇总如下，

模型类型	device	batchsize	tensorrt	mkldnn	cpu多线程
正常模型	GPU	1/8	fp32/fp16	-	-
正常模型	CPU	1/8	-	fp32/fp16	支持
量化模型	GPU	1/8	int8	-	-
量化模型	CPU	1/8	-	int8	支持

2. 测试流程

运行环境配置请参考文档的内容配置TIPC的运行环境。

2.1 安装依赖

安装PaddlePaddle >= 2.2

安装PaddleDetection依赖

text

pip install -r ./requirements.txt
pip install -r ./test_tipc/requirements.txt

安装autolog（规范化日志输出工具）

text

git clone https://github.com/LDOUBLEV/AutoLog
cd AutoLog
pip install -r ./requirements.txt
python setup.py bdist_wheel
pip install ./dist/auto_log-1.0.0-py3-none-any.whl

安装PaddleSlim (可选)

text

# 如果要测试量化、裁剪等功能，需要安装PaddleSlim
pip install paddleslim

2.2 功能测试

先运行prepare.sh准备数据和模型，然后运行test_train_inference_python.sh进行测试，最终在test_tipc/output目录下生成python_infer_*.log格式的日志文件，
以yolov3_darknet53_270e_coco为例。

test_train_inference_python.sh包含5种运行模式，每种模式的运行数据不同，分别用于测试速度和精度，分别是：

模式1：lite_train_lite_infer，使用少量数据训练，用于快速验证训练到预测的走通流程，不验证精度和速度；

shell

bash test_tipc/prepare.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt 'lite_train_lite_infer'
bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt 'lite_train_lite_infer'

模式2：lite_train_whole_infer，使用少量数据训练，一定量数据预测，用于验证训练后的模型执行预测，预测速度是否合理；

shell

bash test_tipc/prepare.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt  'lite_train_whole_infer'
bash test_tipc/test_train_inference_python.sh ../test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt 'lite_train_whole_infer'

模式3：whole_infer，不训练，全量数据预测，走通开源模型评估、动转静，检查inference model预测时间和精度;

shell

bash test_tipc/prepare.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt 'whole_infer'
# 用法1:
bash test_tipc/test_train_inference_python.sh ../test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt 'whole_infer'
# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号
bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt 'whole_infer' '1'

模式4：whole_train_whole_infer，CE：全量数据训练，全量数据预测，验证模型训练精度，预测精度，预测速度；

shell

bash test_tipc/prepare.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt 'whole_train_whole_infer'
bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt 'whole_train_whole_infer'

模式5：klquant_whole_infer，测试离线量化；

shell

bash test_tipc/prepare.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt  'klquant_whole_infer'
bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/yolov3/yolov3_darknet53_270e_coco_train_infer_python.txt  'klquant_whole_infer'

运行相应指令后，在test_tipc/output文件夹下自动会保存运行日志。如'lite_train_lite_infer'模式下，会运行训练+推理的链条，因此，在test_tipc/output文件夹有以下文件：

text

test_tipc/output/
|- results_python.log    # 运行指令状态的日志
|- norm_train_gpus_0_autocast_null/  # GPU 0号卡上正常训练的训练日志和模型保存文件夹
|- pact_train_gpus_0_autocast_null/  # GPU 0号卡上量化训练的训练日志和模型保存文件夹
......
|- python_infer_cpu_usemkldnn_True_threads_1_precision_fluid_batchsize_1.log  # CPU上开启Mkldnn线程数设置为1，测试batch_size=1条件下的预测运行日志
|- python_infer_gpu_precision_trt_fp16_batchsize_1.log # GPU上开启TensorRT，测试batch_size=1的半精度预测日志
......

其中results_python.log中包含了每条指令的运行状态，如果运行成功会输出：

text

Run successfully with command - python3.7 tools/train.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml -o use_gpu=True save_dir=./test_tipc/output/norm_train_gpus_0_autocast_null epoch=1 pretrain_weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams TrainReader.batch_size=2 filename=yolov3_darknet53_270e_coco  !
Run successfully with command - python3.7 tools/eval.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml -o weights=./test_tipc/output/norm_train_gpus_0_autocast_null/yolov3_darknet53_270e_coco/model_final.pdparams use_gpu=True  !
......

如果运行失败，会输出：

text

Run failed with command - python3.7 tools/train.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml -o use_gpu=True save_dir=./test_tipc/output/norm_train_gpus_0_autocast_null epoch=1 pretrain_weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams TrainReader.batch_size=2 filename=yolov3_darknet53_270e_coco  !
Run failed with command - python3.7 tools/eval.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml -o weights=./test_tipc/output/norm_train_gpus_0_autocast_null/yolov3_darknet53_270e_coco/model_final.pdparams use_gpu=True  !
......

可以很方便的根据results_python.log中的内容判定哪一个指令运行错误。

2.3 精度测试

使用compare_results.py脚本比较模型预测的结果是否符合预期，主要步骤包括：

提取日志中的预测坐标；
从本地文件中提取保存好的坐标结果；
比较上述两个结果是否符合精度预期，误差大于设置阈值时会报错。

使用方式

运行命令：

shell

python3.7 test_tipc/compare_results.py --gt_file=./test_tipc/results/python_*.txt  --log_file=./test_tipc/output/python_*.log --atol=1e-3 --rtol=1e-3

参数介绍：

gt_file：指向事先保存好的预测结果路径，支持*.txt 结尾，会自动索引*.txt格式的文件，文件默认保存在test_tipc/result/ 文件夹下
log_file: 指向运行test_tipc/test_train_inference_python.sh 脚本的infer模式保存的预测日志，预测日志中打印的有预测结果，比如：文本框，预测文本，类别等等，同样支持python_infer_*.log格式传入
atol: 设置的绝对误差
rtol: 设置的相对误差

运行结果

正常运行效果如下图：

出现不一致结果时的运行输出：

3. 更多教程

本文档为功能测试用，更丰富的训练预测使用教程请参考：
模型训练
 PaddleDetection预测部署

Deploy Cpp Docs Jetson Build (`deploy/cpp/docs/Jetson_build.md`)

Jetson平台编译指南

说明

NVIDIA Jetson设备是具有NVIDIA GPU的嵌入式设备，可以将目标检测算法部署到该设备上。本文档是在Jetson硬件上部署PaddleDetection模型的教程。

本文档以Jetson TX2硬件、JetPack 4.3版本为例进行说明。

Jetson平台的开发指南请参考NVIDIA Jetson Linux Developer Guide.

Jetson环境搭建

Jetson系统软件安装，请参考NVIDIA Jetson Linux Developer Guide.

(1) 查看硬件系统的l4t的版本号

text

cat /etc/nv_tegra_release

(2) 根据硬件，选择硬件可安装的JetPack版本，硬件和JetPack版本对应关系请参考jetpack-archive.
(3) 下载JetPack，请参考NVIDIA Jetson Linux Developer Guide 中的Preparing a Jetson Developer Kit for Use章节内容进行刷写系统镜像。

注意: 请在jetpack-archive 根据硬件选择适配的JetPack版本进行刷机。

下载或编译`Paddle`预测库

本文档使用Paddle在JetPack4.3上预先编译好的预测库，请根据硬件在安装与编译 Linux 预测库中选择对应版本的Paddle预测库。

这里选择nv_jetson_cuda10_cudnn7.6_trt6(jetpack4.3), Paddle版本2.0.0-rc0,CUDA版本10.0,CUDNN版本7.6，TensorRT版本6。

若需要自己在Jetson平台上自定义编译Paddle库，请参考文档安装与编译 Linux 预测库的NVIDIA Jetson嵌入式硬件预测库源码编译部分内容。

Step1: 下载代码

git clone https://github.com/PaddlePaddle/PaddleDetection.git

说明：其中C++预测代码在/root/projects/PaddleDetection/deploy/cpp 目录，该目录不依赖任何PaddleDetection下其他目录。

Step2: 下载PaddlePaddle C++ 预测库 paddle_inference

解压下载的nv_jetson_cuda10_cudnn7.6_trt6(jetpack4.3) 。

下载并解压后/root/projects/paddle_inference目录包含内容为：

text

paddle_inference
├── paddle # paddle核心库和头文件
|
├── third_party # 第三方依赖库和头文件
|
└── version.txt # 版本和编译信息

注意: 预编译库nv-jetson-cuda10-cudnn7.6-trt6使用的GCC版本是7.5.0，其他都是使用GCC 4.8.5编译的。使用高版本的GCC可能存在ABI兼容性问题，建议降级或自行编译预测库。

Step4: 编译

编译cmake的命令在scripts/build.sh中，请根据实际情况修改主要参数，其主要内容说明如下：

注意，TX2平台的CUDA、CUDNN需要通过JetPack安装。

text

# 是否使用GPU(即是否使用 CUDA)
WITH_GPU=ON

# 是否使用MKL or openblas，TX2需要设置为OFF
WITH_MKL=OFF

# 是否集成 TensorRT(仅WITH_GPU=ON 有效)
WITH_TENSORRT=ON

# TensorRT 的include路径
TENSORRT_INC_DIR=/usr/include/aarch64-linux-gnu

# TensorRT 的lib路径
TENSORRT_LIB_DIR=/usr/lib/aarch64-linux-gnu

# Paddle 预测库路径
PADDLE_DIR=/path/to/paddle_inference/

# Paddle 预测库名称
PADDLE_LIB_NAME=paddle_inference

# Paddle 的预测库是否使用静态库来编译
# 使用TensorRT时，Paddle的预测库通常为动态库
WITH_STATIC_LIB=OFF

# CUDA 的 lib 路径
CUDA_LIB=/usr/local/cuda-10.0/lib64

# CUDNN 的 lib 路径
CUDNN_LIB=/usr/lib/aarch64-linux-gnu

# 是否开启关键点模型预测功能
WITH_KEYPOINT=ON

# OPENCV_DIR 的路径
# linux平台请下载：https://bj.bcebos.com/paddleseg/deploy/opencv3.4.6gcc4.8ffmpeg.tar.gz2，并解压到deps文件夹下
# TX2平台请下载：https://paddlemodels.bj.bcebos.com/TX2_JetPack4.3_opencv_3.4.10_gcc7.5.0.zip，并解压到deps文件夹下
OPENCV_DIR=/path/to/opencv

# 请检查以上各个路径是否正确

# 以下无需改动
cmake .. \
    -DWITH_GPU=${WITH_GPU} \
    -DWITH_MKL=OFF \
    -DWITH_TENSORRT=${WITH_TENSORRT} \
    -DTENSORRT_DIR=${TENSORRT_DIR} \
    -DPADDLE_DIR=${PADDLE_DIR} \
    -DWITH_STATIC_LIB=${WITH_STATIC_LIB} \
    -DCUDA_LIB=${CUDA_LIB} \
    -DCUDNN_LIB=${CUDNN_LIB} \
    -DOPENCV_DIR=${OPENCV_DIR} \
    -DPADDLE_LIB_NAME={PADDLE_LIB_NAME} \
    -DWITH_KEYPOINT=${WITH_KEYPOINT}
make

例如设置如下：

text

# 是否使用GPU(即是否使用 CUDA)
WITH_GPU=ON

# 是否使用MKL or openblas
WITH_MKL=OFF

# 是否集成 TensorRT(仅WITH_GPU=ON 有效)
WITH_TENSORRT=OFF

# TensorRT 的include路径
TENSORRT_INC_DIR=/usr/include/aarch64-linux-gnu

# TensorRT 的lib路径
TENSORRT_LIB_DIR=/usr/lib/aarch64-linux-gnu

# Paddle 预测库路径
PADDLE_DIR=/home/nvidia/PaddleDetection_infer/paddle_inference/

# Paddle 预测库名称
PADDLE_LIB_NAME=paddle_inference

# Paddle 的预测库是否使用静态库来编译
# 使用TensorRT时，Paddle的预测库通常为动态库
WITH_STATIC_LIB=OFF

# CUDA 的 lib 路径
CUDA_LIB=/usr/local/cuda-10.0/lib64

# CUDNN 的 lib 路径
CUDNN_LIB=/usr/lib/aarch64-linux-gnu/

# 是否开启关键点模型预测功能
WITH_KEYPOINT=ON

修改脚本设置好主要参数后，执行build脚本：

shell

sh ./scripts/build.sh

Step5: 预测及可视化

编译成功后，预测入口程序为build/main其主要命令参数说明如下：

参数	说明
--model_dir	导出的检测预测模型所在路径
--model_dir_keypoint	Option
--image_file	要预测的图片文件路径
--image_dir	要预测的图片文件夹路径
--video_file	要预测的视频文件路径
--camera_id	Option
--device	运行时的设备，可选择`CPU/GPU/XPU`，默认为`CPU`
--gpu_id	指定进行推理的GPU device id(默认值为0)
--run_mode	使用GPU时，默认为paddle, 可选（paddle/trt_fp32/trt_fp16/trt_int8）
--batch_size	检测模型预测时的batch size，在指定`image_dir`时有效
--batch_size_keypoint	关键点模型预测时的batch size，默认为8
--run_benchmark	是否重复预测来进行benchmark测速｜
--output_dir	输出图片所在的文件夹, 默认为output ｜
--use_mkldnn	CPU预测中是否开启MKLDNN加速
--cpu_threads	设置cpu线程数，默认为1
--use_dark	关键点模型输出预测是否使用DarkPose后处理，默认为true

注意:

优先级顺序：camera_id > video_file > image_dir > image_file。
--run_benchmark如果设置为True，则需要安装依赖pip install pynvml psutil GPUtil。

样例一：

shell

#不使用`GPU`测试图片 `/root/projects/images/test.jpeg`  
./main --model_dir=/root/projects/models/yolov3_darknet --image_file=/root/projects/images/test.jpeg

图片文件可视化预测结果会保存在当前目录下output.jpg文件中。

样例二:

shell

#使用 `GPU`预测视频`/root/projects/videos/test.mp4`
./main --model_dir=/root/projects/models/yolov3_darknet --video_path=/root/projects/images/test.mp4 --device=GPU

视频文件目前支持.mp4格式的预测，可视化预测结果会保存在当前目录下output.mp4文件中。

样例三：

shell

#使用关键点模型与检测模型联合预测，使用 `GPU`预测  
#检测模型检测到的人送入关键点模型进行关键点预测
./main --model_dir=/root/projects/models/yolov3_darknet --model_dir_keypoint=/root/projects/models/hrnet_w32_256x192 --image_file=/root/projects/images/test.jpeg --device=GPU

性能测试

benchmark请查看BENCHMARK_INFER

Deploy Cpp Docs Linux Build (`deploy/cpp/docs/linux_build.md`)

Linux平台编译指南

说明

本文档在 Linux平台使用GCC 8.2测试过，如果需要使用其他G++版本编译使用，则需要重新编译Paddle预测库，请参考: 从源码编译Paddle预测库。本文档使用的预置的opencv库是在ubuntu 16.04上用gcc8.2编译的，如果需要在gcc8.2以外的环境编译，那么需自行编译opencv库。

前置条件

G++ 8.2
CUDA 9.0 / CUDA 10.1, cudnn 7+ （仅在使用GPU版本的预测库时需要）
CMake 3.0+

请确保系统已经安装好上述基本软件，下面所有示例以工作目录为 /root/projects/演示。

Step1: 下载代码

git clone https://github.com/PaddlePaddle/PaddleDetection.git

说明：其中C++预测代码在/root/projects/PaddleDetection/deploy/cpp 目录，该目录不依赖任何PaddleDetection下其他目录。

Step2: 下载PaddlePaddle C++ 预测库 paddle_inference

PaddlePaddle C++ 预测库针对不同的CPU和CUDA版本提供了不同的预编译版本，请根据实际情况下载: C++预测库下载列表

下载并解压后/root/projects/paddle_inference目录包含内容为：

text

paddle_inference
├── paddle # paddle核心库和头文件
|
├── third_party # 第三方依赖库和头文件
|
└── version.txt # 版本和编译信息

注意: 预编译版本除nv-jetson-cuda10-cudnn7.5-trt5 以外其它包都是基于GCC 4.8.5编译，使用高版本GCC可能存在 ABI兼容性问题，建议降级或自行编译预测库。

Step3: 编译

编译cmake的命令在scripts/build.sh中，请根据实际情况修改主要参数，其主要内容说明如下：

text

# 是否使用GPU(即是否使用 CUDA)
WITH_GPU=OFF

# 使用MKL or openblas
WITH_MKL=ON

# 是否集成 TensorRT(仅WITH_GPU=ON 有效)
WITH_TENSORRT=OFF

# TensorRT 的include路径
TENSORRT_LIB_DIR=/path/to/TensorRT/include

# TensorRT 的lib路径
TENSORRT_LIB_DIR=/path/to/TensorRT/lib

# Paddle 预测库路径
PADDLE_DIR=/path/to/paddle_inference

# Paddle 预测库名称
PADDLE_LIB_NAME=paddle_inference

# CUDA 的 lib 路径
CUDA_LIB=/path/to/cuda/lib

# CUDNN 的 lib 路径
CUDNN_LIB=/path/to/cudnn/lib

# 是否开启关键点模型预测功能
WITH_KEYPOINT=ON

# 请检查以上各个路径是否正确

# 以下无需改动
cmake .. \
    -DWITH_GPU=${WITH_GPU} \
    -DWITH_MKL=${WITH_MKL} \
    -DWITH_TENSORRT=${WITH_TENSORRT} \
    -DTENSORRT_LIB_DIR=${TENSORRT_LIB_DIR} \
    -DTENSORRT_INC_DIR=${TENSORRT_INC_DIR} \
    -DPADDLE_DIR=${PADDLE_DIR} \
    -DCUDA_LIB=${CUDA_LIB} \
    -DCUDNN_LIB=${CUDNN_LIB} \
    -DOPENCV_DIR=${OPENCV_DIR} \
    -DPADDLE_LIB_NAME=${PADDLE_LIB_NAME} \
    -DWITH_KEYPOINT=${WITH_KEYPOINT}
make

修改脚本设置好主要参数后，执行build脚本：

shell

sh ./scripts/build.sh

注意: OPENCV依赖OPENBLAS，Ubuntu用户需确认系统是否已存在libopenblas.so。如未安装，可执行apt-get install libopenblas-dev进行安装。

Step4: 预测及可视化

编译成功后，预测入口程序为build/main其主要命令参数说明如下：

参数	说明
--model_dir	导出的检测预测模型所在路径
--model_dir_keypoint	Option
--image_file	要预测的图片文件路径
--image_dir	要预测的图片文件夹路径
--video_file	要预测的视频文件路径
--camera_id	Option
--device	运行时的设备，可选择`CPU/GPU/XPU`，默认为`CPU`
--gpu_id	指定进行推理的GPU device id(默认值为0)
--run_mode	使用GPU时，默认为paddle, 可选（paddle/trt_fp32/trt_fp16/trt_int8）
--batch_size	检测模型预测时的batch size，在指定`image_dir`时有效
--batch_size_keypoint	关键点模型预测时的batch size，默认为8
--run_benchmark	是否重复预测来进行benchmark测速｜
--output_dir	输出图片所在的文件夹, 默认为output ｜
--use_mkldnn	CPU预测中是否开启MKLDNN加速
--cpu_threads	设置cpu线程数，默认为1
--use_dark	关键点模型输出预测是否使用DarkPose后处理，默认为true

注意:

优先级顺序：camera_id > video_file > image_dir > image_file。
--run_benchmark如果设置为True，则需要安装依赖pip install pynvml psutil GPUtil。

样例一：

shell

#不使用`GPU`测试图片 `/root/projects/images/test.jpeg`  
./build/main --model_dir=/root/projects/models/yolov3_darknet --image_file=/root/projects/images/test.jpeg

图片文件可视化预测结果会保存在当前目录下output.jpg文件中。

样例二:

shell

#使用 `GPU`预测视频`/root/projects/videos/test.mp4`
./build/main --model_dir=/root/projects/models/yolov3_darknet --video_file=/root/projects/images/test.mp4 --device=GPU

视频文件目前支持.mp4格式的预测，可视化预测结果会保存在当前目录下output.mp4文件中。

样例三：

shell

#使用关键点模型与检测模型联合预测，使用 `GPU`预测  
#检测模型检测到的人送入关键点模型进行关键点预测
./build/main --model_dir=/root/projects/models/yolov3_darknet --model_dir_keypoint=/root/projects/models/hrnet_w32_256x192 --image_file=/root/projects/images/test.jpeg --device=GPU

性能测试

benchmark请查看BENCHMARK_INFER

Deploy Cpp Docs Windows Vs2019 Build (`deploy/cpp/docs/windows_vs2019_build.md`)

Visual Studio 2019 Community CMake 编译指南

Windows 平台下，我们使用Visual Studio 2019 Community 进行了测试。微软从Visual Studio 2017开始即支持直接管理CMake跨平台编译项目，但是直到2019才提供了稳定和完全的支持，所以如果你想使用CMake管理项目编译构建，我们推荐你使用Visual Studio 2019环境下构建。

前置条件

Visual Studio 2019 (根据Paddle预测库所使用的VS版本选择，请参考 Visual Studio 不同版本二进制兼容性 )
CUDA 9.0 / CUDA 10.0，cudnn 7+ / TensorRT（仅在使用GPU版本的预测库时需要）
CMake 3.0+ CMake下载

特别注意：windows下预测库需要的TensorRT版本为：。

预测库版本	TensorRT版本
cuda10.1_cudnn7.6_avx_mkl_trt6	TensorRT-6.0.1.5
cuda10.2_cudnn7.6_avx_mkl_trt7	TensorRT-7.0.0.11
cuda11.0_cudnn8.0_avx_mkl_trt7	TensorRT-7.2.1.6

请确保系统已经安装好上述基本软件，我们使用的是VS2019的社区版。

下面所有示例以工作目录为 D:\projects演示。

Step1: 下载代码

下载源代码

shell

git clone https://github.com/PaddlePaddle/PaddleDetection.git

说明：其中C++预测代码在PaddleDetection/deploy/cpp 目录，该目录不依赖任何PaddleDetection下其他目录。

Step2: 下载PaddlePaddle C++ 预测库 paddle_inference

PaddlePaddle C++ 预测库针对不同的CPU和CUDA版本提供了不同的预编译版本，请根据实际情况下载: C++预测库下载列表

解压后D:\projects\paddle_inference目录包含内容为：

text

paddle_inference
├── paddle # paddle核心库和头文件
|
├── third_party # 第三方依赖库和头文件
|
└── version.txt # 版本和编译信息

Step3: 安装配置OpenCV

在OpenCV官网下载适用于Windows平台的3.4.6版本，下载地址
运行下载的可执行文件，将OpenCV解压至指定目录，如D:\projects\opencv
配置环境变量，如下流程所示（如果使用全局绝对路径，可以不用设置环境变量）
- 我的电脑->属性->高级系统设置->环境变量
- 在系统变量中找到Path（如没有，自行创建），并双击编辑
- 新建，将opencv路径填入并保存，如D:\projects\opencv\build\x64\vc14\bin

Step4: 编译

进入到cpp文件夹

text

cd D:\projects\PaddleDetection\deploy\cpp

使用CMake生成项目文件

编译参数的含义说明如下（带*表示仅在使用GPU版本预测库时指定, 其中CUDA库版本尽量对齐，使用9.0、10.0版本，不使用9.2、10.1等版本CUDA库）：

参数名	含义
*CUDA_LIB	CUDA的库路径
*CUDNN_LIB	CUDNN的库路径
OPENCV_DIR	OpenCV的安装路径，
PADDLE_DIR	Paddle预测库的路径
PADDLE_LIB_NAME	Paddle 预测库名称

注意：

如果编译环境为CPU，需要下载CPU版预测库，请把WITH_GPU的勾去掉
如果使用的是openblas版本，请把WITH_MKL勾去掉
如无需使用关键点模型可以把WITH_KEYPOINT勾去掉
Windows环境下，PADDLE_LIB_NAME需要设置为paddle_inference

执行如下命令项目文件：

text

cmake . -G "Visual Studio 16 2019" -A x64 -T host=x64 -DWITH_GPU=ON -DWITH_MKL=ON -DCMAKE_BUILD_TYPE=Release -DCUDA_LIB=path_to_cuda_lib -DCUDNN_LIB=path_to_cudnn_lib -DPADDLE_DIR=path_to_paddle_lib -DPADDLE_LIB_NAME=paddle_inference -DOPENCV_DIR=path_to_opencv -DWITH_KEYPOINT=ON

例如：

text

cmake . -G "Visual Studio 16 2019" -A x64 -T host=x64 -DWITH_GPU=ON -DWITH_MKL=ON -DCMAKE_BUILD_TYPE=Release -DCUDA_LIB=D:\projects\packages\cuda10_0\lib\x64 -DCUDNN_LIB=D:\projects\packages\cuda10_0\lib\x64 -DPADDLE_DIR=D:\projects\packages\paddle_inference -DPADDLE_LIB_NAME=paddle_inference -DOPENCV_DIR=D:\projects\packages\opencv3_4_6 -DWITH_KEYPOINT=ON

编译
用Visual Studio 16 2019打开cpp文件夹下的PaddleObjectDetector.sln，将编译模式设置为Release，点击生成->`全部生成

Step5: 预测及可视化

上述Visual Studio 2019编译产出的可执行文件在out\build\x64-Release目录下，打开cmd，并切换到该目录：

text

cd D:\projects\PaddleDetection\deploy\cpp\out\build\x64-Release

可执行文件main即为样例的预测程序，其主要的命令行参数如下：

参数	说明
--model_dir	导出的检测预测模型所在路径
--model_dir_keypoint	Option
--image_file	要预测的图片文件路径
--image_dir	要预测的图片文件夹路径
--video_file	要预测的视频文件路径
--camera_id	Option
--device	运行时的设备，可选择`CPU/GPU/XPU`，默认为`CPU`
--gpu_id	指定进行推理的GPU device id(默认值为0)
--run_mode	使用GPU时，默认为paddle, 可选（paddle/trt_fp32/trt_fp16/trt_int8）
--batch_size	检测模型预测时的batch size，在指定`image_dir`时有效
--batch_size_keypoint	关键点模型预测时的batch size，默认为8
--run_benchmark	是否重复预测来进行benchmark测速｜
--output_dir	输出图片所在的文件夹, 默认为output ｜
--use_mkldnn	CPU预测中是否开启MKLDNN加速
--cpu_threads	设置cpu线程数，默认为1
--use_dark	关键点模型输出预测是否使用DarkPose后处理，默认为true

注意：
（1）优先级顺序：camera_id > video_file > image_dir > image_file。
（2）如果提示找不到opencv_world346.dll，把D:\projects\packages\opencv3_4_6\build\x64\vc14\bin文件夹下的opencv_world346.dll拷贝到main.exe文件夹下即可。
（3）--run_benchmark如果设置为True，则需要安装依赖pip install pynvml psutil GPUtil。

样例一：

shell

#不使用`GPU`测试图片 `D:\\images\\test.jpeg`  
.\main --model_dir=D:\\models\\yolov3_darknet --image_file=D:\\images\\test.jpeg

图片文件可视化预测结果会保存在当前目录下output.jpg文件中。

样例二:

shell

#使用`GPU`测试视频 `D:\\videos\\test.mp4`  
.\main --model_dir=D:\\models\\yolov3_darknet --video_path=D:\\videos\\test.mp4 --device=GPU

视频文件目前支持.mp4格式的预测，可视化预测结果会保存在当前目录下output.mp4文件中。

样例三：

shell

#使用关键点模型与检测模型联合预测，使用 `GPU`预测  
#检测模型检测到的人送入关键点模型进行关键点预测
.\main --model_dir=D:\\models\\yolov3_darknet --model_dir_keypoint=D:\\models\\hrnet_w32_256x192 --image_file=D:\\images\\test.jpeg --device=GPU

性能测试

Benchmark请查看BENCHMARK_INFER

Advanced Tutorials Customization Detection (`docs/advanced_tutorials/customization/detection.md`)

简体中文 | English

目标检测任务二次开发

在目标检测算法产业落地过程中，常常会出现需要额外训练以满足实际使用的要求，项目迭代过程中也会出先需要修改类别的情况。本文档详细介绍如何使用PaddleDetection进行目标检测算法二次开发，流程包括：数据准备、模型优化思路和修改类别开发流程。

数据准备

二次开发首先需要进行数据集的准备，针对场景特点采集合适的数据从而提升模型效果和泛化性能。然后使用Labeme，LabelImg等标注工具标注目标检测框，并将标注结果转化为COCO或VOC数据格式。详细文档可以参考数据准备文档

模型优化

1. 使用自定义数据集训练

基于准备的数据在数据配置文件中修改对应路径，例如configs/dataset/coco_detection.yml:

text

metric: COCO
num_classes: 80

TrainDataset:
  !COCODataSet
    image_dir: train2017 # 训练集的图片所在文件相对于dataset_dir的路径
    anno_path: annotations/instances_train2017.json # 训练集的标注文件相对于dataset_dir的路径
    dataset_dir: dataset/coco # 数据集所在路径，相对于PaddleDetection路径
    data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']

EvalDataset:
  !COCODataSet
    image_dir: val2017 # 验证集的图片所在文件相对于dataset_dir的路径
    anno_path: annotations/instances_val2017.json # 验证集的标注文件相对于dataset_dir的路径
    dataset_dir: dataset/coco # 数据集所在路径，相对于PaddleDetection路径

TestDataset:
  !ImageFolder
    anno_path: annotations/instances_val2017.json # also support txt (like VOC's label_list.txt) # 标注文件所在文件 相对于dataset_dir的路径
    dataset_dir: dataset/coco # if set, anno_path will be 'dataset_dir/anno_path' # 数据集所在路径，相对于PaddleDetection路径

配置修改完成后，即可以启动训练评估，命令如下

text

export CUDA_VISIBLE_DEVICES=0
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_270e_coco.yml --eval

更详细的命令参考30分钟快速上手PaddleDetection

2. 加载COCO模型作为预训练

目前PaddleDetection提供的配置文件加载的预训练模型均为ImageNet数据集的权重，加载到检测算法的骨干网络中，实际使用时，建议加载COCO数据集训练好的权重，通常能够对模型精度有较大提升，使用方法如下：

1) 设置预训练权重路径

COCO数据集训练好的模型权重均在各算法配置文件夹下，例如configs/ppyoloe下提供了PP-YOLOE-l COCO数据集权重：链接。配置文件中设置pretrain_weights: https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

2) 修改超参数

加载COCO预训练权重后，需要修改学习率超参数，例如configs/ppyoloe/_base_/optimizer_300e.yml中:

text

epoch: 120 # 原始配置为300epoch，加载COCO权重后可以适当减少迭代轮数

LearningRate:
  base_lr: 0.005 # 原始配置为0.025，加载COCO权重后需要降低学习率
  schedulers:
    - !CosineDecay
      max_epochs: 144 # 依据epoch数进行修改
    - !LinearWarmup
      start_factor: 0.
      epochs: 5

修改类别

当实际使用场景类别发生变化时，需要修改数据配置文件，例如configs/datasets/coco_detection.yml中:

text

metric: COCO
num_classes: 10 # 原始类别80

配置修改完成后，同样可以加载COCO预训练权重，PaddleDetection支持自动加载shape匹配的权重，对于shape不匹配的权重会自动忽略，因此无需其他修改。

Advanced Tutorials Customization Detection En (`docs/advanced_tutorials/customization/detection_en.md`)

简体中文 | English

Customize Object Detection task

In the practical application of object detection algorithms in a specific industry, additional training is often required for practical use. The project iteration will also need to modify categories. This document details how to use PaddleDetection for a customized object detection algorithm. The process includes data preparation, model optimization roadmap, and modifying the category development process.

Data Preparation

Customization starts with the preparation of the dataset. We need to collect suitable data for the scenario features, so as to improve the model effect and generalization performance. Then Labeme, LabelImg and other labeling tools will be used to label the object detection bouding boxes and convert the labeling results into COCO or VOC data format. Details please refer to Data Preparation

Model Optimization

1. Use customized dataset for training

Modify the corresponding path in the data configuration file based on the prepared data, for example:

configs/dataset/coco_detection.yml`:

text

metric: COCO
num_classes: 80

TrainDataset:
  !COCODataSet
    image_dir: train2017 # Path to the images of the training set relative to the dataset_dir
    anno_path: annotations/instances_train2017.json # Path to the annotation file of the training set relative to the dataset_dir
    dataset_dir: dataset/coco # Path to the dataset relative to the PaddleDetection path
    data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']

EvalDataset:
  !COCODataSet
    image_dir: val2017 # Path to the images of the evaldataset set relative to the dataset_dir
    anno_path: annotations/instances_val2017.json # Path to the annotation file of the evaldataset relative to the dataset_dir
    dataset_dir: dataset/coco # Path to the dataset relative to the PaddleDetection path

TestDataset:
  !ImageFolder
    anno_path: annotations/instances_val2017.json # also support txt (like VOC's label_list.txt) # Path to the annotation files relative to dataset_di.
    dataset_dir: dataset/coco # if set, anno_path will be 'dataset_dir/anno_path' # Path to the dataset relative to the PaddleDetection path

Once the configuration changes are completed, the training evaluation can be started with the following command

text

export CUDA_VISIBLE_DEVICES=0
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_270e_coco.yml --eval

More details please refer to Getting Started for PaddleDetection

2. Load the COCO model as pre-training

The currently provided pre-trained models in PaddleDetection's configurations are weights from the ImageNet dataset, loaded into the backbone network of the detection algorithm. For practical use, it is recommended to load the weights trained on the COCO dataset, which can usually provide a large improvement to the model accuracy. The method is as follows.

1) Set pre-training weight path

The trained model weights for the COCO dataset are saved in the configuration folder of each algorithm, for example, PP-YOLOE-l COCO dataset weights are provided under configs/ppyoloe: Link The configuration file setspretrain_weights: https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams

2) Modify hyperparameters

After loading the COCO pre-training weights, the learning rate hyperparameters need to be modified, for example

In configs/ppyoloe/_base_/optimizer_300e.yml:

text

epoch: 120 # The original configuration is 300 epoch, after loading COCO weights, the iteration number can be reduced appropriately

LearningRate:
 base_lr: 0.005 # The original configuration is 0.025, after loading COCO weights, the learning rate should be reduced.
 schedulers:
 - !CosineDecay
 max_epochs: 144 # Modify based on the number of epochs
 - LinearWarmup
 start_factor: 0.
 epochs: 5

Modify categories

When the actual application scenario category changes, the data configuration file needs to be modified, for example in configs/datasets/coco_detection.yml:

text

metric: COCO
num_classes: 10 # original class 80

After the configuration changes are completed, the COCO pre-training weights can also be loaded. PaddleDetection supports automatic loading of shape-matching weights, and weights that do not match the shape are automatically ignored, so no other modifications are needed.

Advanced Tutorials Customization Keypoint Detection (`docs/advanced_tutorials/customization/keypoint_detection.md`)

简体中文 | English

关键点检测任务二次开发

在实际场景中应用关键点检测算法，不可避免地会出现需要二次开发的需求。包括对目前的预训练模型效果不满意，希望优化模型效果；或是目前的关键点点位定义不能满足实际场景需求，希望新增或是替换关键点点位的定义，训练新的关键点模型。本文档将介绍如何在PaddleDetection中，对关键点检测算法进行二次开发。

数据准备

基本流程说明

在PaddleDetection中，目前支持的标注数据格式为COCO和MPII。这两个数据格式的详细说明，可以参考文档关键点数据准备。在这一步中，通过使用Labeme等标注工具，依照特征点序号标注对应坐标。并转化成对应可训练的标注格式。建议使用COCO格式进行。

合并数据集

为了扩展使用的训练数据，合并多个不同的数据集一起训练是一个很直观的解决手段，但不同的数据集往往对关键点的定义并不一致。合并数据集的第一步是需要统一不同数据集的点位定义，确定标杆点位，即最终模型学习的特征点类型，然后根据各个数据集的点位定义与标杆点位定义之间的关系进行调整。

在标杆点位中的点：调整点位序号，使其与标杆点位一致
未在标杆点位中的点：舍去
数据集缺少标杆点位中的点：对应将标注的标志位记为“未标注”

在关键点数据准备中，提供了如何合并COCO数据集和AI Challenger数据集，并统一为以COCO为标杆点位定义的案例说明，供参考。

模型优化

检测-跟踪模型优化

在PaddleDetection中，关键点检测能力支持Top-Down、Bottom-Up两套方案，Top-Down先检测主体，再检测局部关键点，优点是精度较高，缺点是速度会随着检测对象的个数增加，Bottom-Up先检测关键点再组合到对应的部位上，优点是速度快，与检测对象个数无关，缺点是精度较低。关于两种方案的详情及对应模型，可参考关键点检测系列模型

当使用Top-Down方案时，模型效果依赖于前序的检测和跟踪效果，如果实际场景中不能准确检测到行人位置，会使关键点检测部分表现受限。如果在实际使用中遇到了上述问题，请参考目标检测任务二次开发以及多目标跟踪任务二次开发对检测/跟踪模型进行优化。

使用符合场景的数据迭代

目前发布的关键点检测算法模型主要在COCO/ AI Challenger等开源数据集上迭代，这部分数据集中可能缺少与实际任务较为相似的监控场景（视角、光照等因素）、体育场景（存在较多非常规的姿态）。使用更符合实际任务场景的数据进行训练，有助于提升模型效果。

使用预训练模型迭代

关键点模型的数据的标注复杂度较大，直接使用模型从零开始在业务数据集上训练，效果往往难以满足需求。在实际工程中使用时，建议加载已经训练好的权重，通常能够对模型精度有较大提升，以HRNet为例，使用方法如下：

bash

python tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml -o pretrain_weights=https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams

在加载预训练模型后，可以适当减小初始学习率和最终迭代轮数, 建议初始学习率取默认配置值的1/2至1/5，并可开启--eval观察迭代过程中AP值的变化。

遮挡数据增强

关键点任务中有较多遮挡问题，包括自身遮挡与不同目标之间的遮挡。

检测模型优化（仅针对Top-Down方案）

参考目标检测任务二次开发，提升检测模型在复杂场景下的效果。

关键点数据增强

在关键点模型训练中增加遮挡的数据增强，参考PP-TinyPose。有助于模型提升这类场景下的表现。

对视频预测进行平滑处理

关键点模型是在图片级别的基础上进行训练和预测的，对于视频类型的输入也是将视频拆分为帧进行预测。帧与帧之间虽然内容大多相似，但微小的差异仍然可能导致模型的输出发生较大的变化，表现为虽然预测的坐标大体正确，但视觉效果上有较大的抖动问题。通过添加滤波平滑处理，将每一帧预测的结果与历史结果综合考虑，得到最终的输出结果，可以有效提升视频上的表现。该部分内容可参考滤波平滑处理。

新增或修改关键点点位定义

数据准备

根据前述说明，完成数据的准备，放置于{root of PaddleDetection}/dataset下。

** 标注文件示例**

一个标注文件示例如下：

text

self_dataset/
├── train_coco_joint.json  # 训练集标注文件
├── val_coco_joint.json    # 验证集标注文件
├── images/                # 存放图片文件
    ├── 0.jpg
    ├── 1.jpg
    ├── 2.jpg

其中标注文件中需要注意的改动如下：

text

/* Detailed source-code truncated for AI context efficiency. */

配置文件设置

在配置文件中，完整的含义参考config yaml配置项说明。以HRNet模型配置为例，重点需要关注的内容如下：

** 配置文件示例**

一个配置文件的示例如下

yaml

use_gpu: true
log_iter: 5
save_dir: output
snapshot_epoch: 10
weights: output/hrnet_w32_256x192/model_final
epoch: 210
num_joints: &num_joints 5 # 预测的点数与定义点数量一致
pixel_std: &pixel_std 200
metric: KeyPointTopDownCOCOEval
num_classes: 1  
train_height: &train_height 256
train_width: &train_width 192
trainsize: &trainsize [*train_width, *train_height]
hmsize: &hmsize [48, 64]
flip_perm: &flip_perm [[1, 2], [3, 4]]  # 注意只有含义上镜像对称的点才写到这里

...

# 保证dataset_dir + anno_path 能正确定位到标注文件位置
# 保证dataset_dir + image_dir + 标注文件中的图片路径能正确定位到图片
TrainDataset:
  !KeypointTopDownCocoDataset
    image_dir: images
    anno_path: train_coco_joint.json
    dataset_dir: dataset/self_dataset
    num_joints: *num_joints
    trainsize: *trainsize
    pixel_std: *pixel_std
    use_gt_bbox: True


EvalDataset:
  !KeypointTopDownCocoDataset
    image_dir: images
    anno_path: val_coco_joint.json
    dataset_dir: dataset/self_dataset
    bbox_file: bbox.json
    num_joints: *num_joints
    trainsize: *trainsize
    pixel_std: *pixel_std
    use_gt_bbox: True
    image_thre: 0.0

模型训练及评估

模型训练

通过如下命令启动训练：

bash

CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml

模型评估

训练好模型之后，可以通过以下命令实现对模型指标的评估:

bash

python3 tools/eval.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml

注意：由于测试依赖pycocotools工具，其默认为COCO数据集的17点，如果修改后的模型并非预测17点，直接使用评估命令会报错。
需要修改以下内容以获得正确的评估结果：

sigma列表，表示每个关键点的范围方差，越大则容忍度越高。其长度与预测点数一致。根据实际关键点可信区域设置，区域精确的一般0.25-0.5，例如眼睛。区域范围大的一般0.5-1.0，例如肩膀。若不确定建议0.75。
pycocotools sigma列表，含义及内容同上，取值与sigma列表一致。

模型导出及预测

Top-Down模型联合部署

shell

#导出关键点模型
python tools/export_model.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml -o weights={path_to_your_weights}

#detector 检测 + keypoint top-down模型联合部署（联合推理只支持top-down方式）
python deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/ppyolo_r50vd_dcn_2x_coco/ --keypoint_model_dir=output_inference/hrnet_w32_256x192/ --video_file=../video/xxx.mp4  --device=gpu

注意目前PP-Human中使用的为该方案

Bottom-Up模型独立部署

shell

#导出模型
python tools/export_model.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=output/higherhrnet_hrnet_w32_512/model_final.pdparams

#部署推理
python deploy/python/keypoint_infer.py --model_dir=output_inference/higherhrnet_hrnet_w32_512/ --image_file=./demo/000000014439_640x640.jpg --device=gpu --threshold=0.5

Advanced Tutorials Customization Keypoint Detection En (`docs/advanced_tutorials/customization/keypoint_detection_en.md`)

简体中文 | English

Customized Keypoint Detection

When applying keypoint detection algorithms in real practice, inevitably, we may need customization as we may dissatisfy with the current pre-trained model results, or the current keypoint detection cannot meet the actual demand, or we may want to add or replace the definition of keypoints and train a new keypoint detection model. This document will introduce how to customize the keypoint detection algorithm in PaddleDetection.

Data Preparation

Basic Process Description

PaddleDetection currently supports COCO and MPII annotation data formats. For detailed descriptions of these two data formats, please refer to the document Keypoint Data Preparation. In this step, by using annotation tools such as Labeme, the corresponding coordinates are annotated according to the feature point serial numbers and then converted into the corresponding trainable annotation format. And we recommend COCO format.

Merging datasets

To extend the training data, we can merge several different datasets together. But different datasets often have different definitions of key points. Therefore, the first step in merging datasets is to unify the point definitions of different datasets, and determine the benchmark points, i.e., the types of feature points finally learned by the model, and then adjust them according to the relationship between the point definitions of each dataset and the benchmark point definitions.

Points in the benchmark point location: adjust the point number to make it consistent with the benchmark point location
Points that are not in the benchmark points: discard
Points in the dataset that are missing from the benchmark: annotate the marked points as "unannotated".

In [Key point data preparation](... /... /tutorials/data/PrepareKeypointDataSet.md), we provide a case illustration of how to merge the COCO dataset and the AI Challenger dataset and unify them as a benchmark point definition with COCO for your reference.

Model Optimization

Detection and tracking model optimization

In PaddleDetection, the keypoint detection supports Top-Down and Bottom-Up solutions. Top-Down first detects the main body and then detects the local key points. It has higher accuracy but will take a longer time as the number of detected objects increases.The Bottom-Up plan first detects the keypoints and then combines them with the corresponding parts. It is fast and its speed is independent of the number of detected objects. Its disadvantage is that the accuracy is relatively low. For details of the two solutions and the corresponding models, please refer to Keypoint Detection Series Models

When using the Top-Down solution, the model's effects depend on the previous detection or tracking effect. If the pedestrian position cannot be accurately detected in the actual practice, the performance of the keypoint detection will be limited. If you encounter the above problem in actual application, please refer to Customized Object Detection and Customized Multi-target tracking for optimization of the detection and tracking model.

Iterate with scenario-compatible data

The currently released keypoint detection algorithm models are mainly iterated on open source datasets such as COCO/ AI Challenger, which may lack surveillance scenarios (angles, lighting and other factors), sports scenarios (more unconventional poses) that are more similar to the actual task. Training with data that more closely matches the actual task scenario can help improve the model's results.

Iteration via pre-trained models

The data annotation of the keypoint model is complex, and using the model directly to train on the business dataset from scratch is often difficult to meet the demand. When used in practical projects, it is recommended to load the pre-trained weights, which usually improve the model accuracy significantly. Let's take HRNet as an example with the following method:

text

python tools/train.py \
        -c configs/keypoint/hrnet/hrnet_w32_256x192.yml \
        -o pretrain_weights=https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams

After loading the pre-trained model, the initial learning rate and the rounds of iterations can be reduced appropriately. It is recommended that the initial learning rate be 1/2 to 1/5 of the default configuration, and you can enable--eval to observe the change of AP values during the iterations.

Data augmentation with occlusion

There are a lot of data in occlusion in keypoint tasks, including self-covered objects and occlusion between different objects.

Detection model optimization (only for Top-Down solutions)

Refer to [Target Detection Task Secondary Development](. /detection.md) to improve the detection model in complex scenarios.

Keypoint data augmentation

Augmentation of covered data in keypoint model training to improve model performance in such scenarios, please refer to PP-TinyPose

Smooth video prediction

The keypoint model is trained and predicted on the basis of image, and video input is also predicted by splitting the video into frames. Although the content is mostly similar between frames, small differences may still lead to large changes in the output of the model. As a result of that, although the predicted coordinates are roughly correct, there may be jitters in the visual effect.

By adding a smoothing filter process, the performance of the video output can be effectively improved by combining the predicted results of each frame and the historical results. For this part, please see Filter Smoothing.

Add or modify keypoint definition

Data Preparation

Complete the data preparation according to the previous instructions and place it under {root of PaddleDetection}/dataset.

** Examples of annotation file**

text

self_dataset/
├── train_coco_joint.json # training set annotation file
├── val_coco_joint.json # Validation set annotation file
├── images/ # Store the image files
    ├── 0.jpg
    ├── 1.jpg
    ├── 2.jpg

Notable changes as follows:

text

/* Detailed source-code truncated for AI context efficiency. */

Settings of configuration file

In the configuration file, refer to [config yaml configuration](... /... /tutorials/KeyPointConfigGuide_cn.md) for more details . Take [HRNet model configuration](... /... /... /configs/keypoint/hrnet/hrnet_w32_256x192.yml) as an example, we need to focus on following contents:

** Example of configuration**

text

use_gpu: true
log_iter: 5
save_dir: output
snapshot_epoch: 10
weights: output/hrnet_w32_256x192/model_final
epoch: 210
num_joints: &num_joints 5 # The number of predicted points matches the number of defined points
pixel_std: &pixel_std 200
Metric. keyPointTopDownCOCOEval
num_classes: 1  
train_height: &train_height 256
train_width: &train_width 192
trainsize: &trainsize [*train_width, *train_height].
hmsize: &hmsize [48, 64].
flip_perm: &flip_perm [[1, 2], [3, 4]]. # Note that only points that are mirror-symmetric are recorded here.

...

# Ensure that dataset_dir + anno_path can correctly locate the annotation file
# Ensure that dataset_dir + image_dir + image path in annotation file can correctly locate the image.
TrainDataset:
  !KeypointTopDownCocoDataset
    image_dir: images
    anno_path: train_coco_joint.json
    dataset_dir: dataset/self_dataset
    num_joints: *num_joints
    trainsize. *trainsize
    pixel_std: *pixel_std
    use_gt_box: true


Evaluate the dataset.
  !KeypointTopDownCocoDataset
    image_dir: images
    anno_path: val_coco_joint.json
    dataset_dir: dataset/self_dataset
    bbox_file: bbox.json
    num_joints: *num_joints
    trainsize. *trainsize
    pixel_std: *pixel_std
    use_gt_box: true
    image_thre: 0.0

Model Training and Evaluation

Model Training

Run the following command to start training:

text

CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml

Model Evaluation

After training the model, you can evaluate the model metrics by running the following commands:

text

python3 tools/eval.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml

Model Export and Inference

Top-Down model deployment

text

#Export keypoint model
python tools/export_model.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml -o weights={path_to_your_weights}

#detector detection + keypoint top-down model co-deployment（for top-down solutions only）
python deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/ppyolo_r50vd_dcn_2x_coco/ --keypoint_model_dir=output_inference/hrnet_w32_256x192/ --video_file=../video/xxx.mp4  --device=gpu

Advanced Tutorials Customization Pphuman Attribute (`docs/advanced_tutorials/customization/pphuman_attribute.md`)

简体中文 | English

行人属性识别任务二次开发

数据准备

数据格式

格式采用PA100K的属性标注格式，共有26位属性。

这26位属性的名称、位置、种类数量见下表。

Attribute	index	length
'Hat','Glasses'	[0, 1]	2
'ShortSleeve','LongSleeve','UpperStride','UpperLogo','UpperPlaid','UpperSplice'	[2, 3, 4, 5, 6, 7]	6
'LowerStripe','LowerPattern','LongCoat','Trousers','Shorts','Skirt&Dress'	[8, 9, 10, 11, 12, 13]	6
'boots'	[14, ]	1
'HandBag','ShoulderBag','Backpack','HoldObjectsInFront'	[15, 16, 17, 18]	4
'AgeOver60', 'Age18-60', 'AgeLess18'	[19, 20, 21]	3
'Female'	[22, ]	1
'Front','Side','Back'	[23, 24, 25]	3

举例：

[0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]

第一组，位置[0, 1]数值分别是[0, 1]，表示'no hat'、'has glasses'。

第二组，位置[22, ]数值分别是[0, ], 表示gender属性是'male', 否则是'female'。

第三组，位置[23, 24, 25]数值分别是[0, 1, 0], 表示方向属性是侧面'side'。

其他组依次类推

数据标注

理解了上面属性标注格式的含义后，就可以进行数据标注的工作。其本质是：每张单人图建立一组26个长度的标注项，分别与26个位置的属性值对应。

举例：

对于一张原始图片，

1）使用检测框，标注图片中每一个人的位置。

2）每一个检测框（对应每一个人），包含一组26位的属性值数组，数组的每一位以0或1表示。对应上述26个属性。例如，如果图片是'Female'，则数组第22位为0，如果满足'Age18-60'，则位置[19, 20, 21]对应的数值是[0, 1, 0], 或者满足'AgeOver60'，则相应数值为[1, 0, 0].

标注完成后利用检测框将每一个人截取成单人图，其图片与26位属性标注建立对应关系。也可先截成单人图再进行标注，效果相同。

模型训练

数据标注完成后，就可以拿来做模型的训练，完成自定义模型的优化工作。

其主要有两步工作需要完成：1）将数据与标注数据整理成训练格式。2）修改配置文件开始训练。

训练数据格式

训练数据包括训练使用的图片和一个训练列表train.txt，其具体位置在训练配置中指定，其放置方式示例如下：

text

Attribute/
|-- data           训练图片文件夹
|   |-- 00001.jpg
|   |-- 00002.jpg
|   `-- 0000x.jpg
`-- train.txt      训练数据列表

train.txt文件内为所有训练图片名称（相对于根路径的文件路径）+ 26个标注值

其每一行表示一个人的图片和标注结果。其格式为：

text

00001.jpg    0,0,1,0,....

注意：1)图片与标注值之间是以Tab[\t]符号隔开, 2)标注值之间是以逗号[,]隔开。该格式不能错，否则解析失败。

修改配置开始训练

首先执行以下命令下载训练代码（更多环境问题请参考Install_PaddleClas）:

shell

git clone https://github.com/PaddlePaddle/PaddleClas

需要在配置文件PaddleClas/blob/develop/ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml中，修改的配置项如下：

text

DataLoader:
  Train:
    dataset:
      name: MultiLabelDataset
      image_root: "dataset/pa100k/"                     #指定训练图片所在根路径
      cls_label_path: "dataset/pa100k/train_list.txt"   #指定训练列表文件位置
      label_ratio: True
      transform_ops:

  Eval:
    dataset:
      name: MultiLabelDataset
      image_root: "dataset/pa100k/"                     #指定评估图片所在根路径
      cls_label_path: "dataset/pa100k/val_list.txt"     #指定评估列表文件位置
      label_ratio: True
      transform_ops:

注意：

这里image_root路径+train.txt中图片相对路径，对应图片的完整路径位置。
如果有修改属性数量，则还需修改内容配置项中属性种类数量：

text

# model architecture
Arch:
  name: "PPLCNet_x1_0"
  pretrained: True
  use_ssld: True
  class_num: 26           #属性种类数量

然后运行以下命令开始训练。

text

#多卡训练
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/train.py \
        -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml

#单卡训练
python3 tools/train.py \
        -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml

训练完成后可以执行以下命令进行性能评估：

text

#多卡评估
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/eval.py \
        -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
        -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model

#单卡评估
python3 tools/eval.py \
        -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
        -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model

模型导出

使用下述命令将训练好的模型导出为预测部署模型。

text

python3 tools/export_model.py \
    -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
    -o Global.pretrained_model=output/PPLCNet_x1_0/best_model \
    -o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_person_attribute_infer

导出模型后，需要下载infer_cfg.yml文件，并放置到导出的模型文件夹PPLCNet_x1_0_person_attribute_infer中。

使用时在PP-Human中的配置文件./deploy/pipeline/config/infer_cfg_pphuman.yml中修改新的模型路径model_dir项，并开启功能enable: True。

text

ATTR:
  model_dir: [YOUR_DEPLOY_MODEL_DIR]/PPLCNet_x1_0_person_attribute_infer/   #新导出的模型路径位置
  enable: True                                                              #开启功能

然后可以使用-->至此即完成新增属性类别识别任务。

属性增减

上述是以26个属性为例的标注、训练过程。

如果需要增加、减少属性数量，则需要：

1)标注时需增加新属性类别信息或删减属性类别信息；

2)对应修改训练中train.txt所使用的属性数量和名称；

3)修改训练配置，例如PaddleClas/blob/develop/ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml文件中的属性数量,详细见上述修改配置开始训练部分。

增加属性示例：

在标注数据时在26位后继续增加新的属性标注数值；
在train.txt文件的标注数值中也增加新的属性数值。
注意属性类型在train.txt中属性数值列表中的位置的对应关系需要时固定的，例如第[19, 20, 21]位表示年龄，所有图片都要使用[19, 20, 21]位置表示年龄，不再赘述。

删减属性同理。
例如，如果不需要年龄属性，则位置[19, 20, 21]的数值可以去掉。只需在train.txt中标注的26个数字中全部删除第19-21位数值即可，同时标注数据时也不再需要标注这3位属性值。

修改后处理代码

修改了属性定义后，pipeline后处理部分也需要做相应修改，主要影响结果可视化时的显示结果。

相应代码在路径deploy/pipeline/pphuman/attr_infer.py文件中postprocess函数。

其函数实现说明如下：

text

/* Detailed source-code truncated for AI context efficiency. */

Advanced Tutorials Customization Pphuman Attribute En (`docs/advanced_tutorials/customization/pphuman_attribute_en.md`)

简体中文 | English

Customized Pedestrian Attribute Recognition

Data Preparation

Data format

We use the PA100K attribute annotation format, with a total of 26 attributes.

The names, locations, and the number of these 26 attributes are shown in the table below.

Attribute	index	length
'Hat','Glasses'	[0, 1]	2
'ShortSleeve','LongSleeve','UpperStride','UpperLogo','UpperPlaid','UpperSplice'	[2, 3, 4, 5, 6, 7]	6
'LowerStripe','LowerPattern','LongCoat','Trousers','Shorts','Skirt&Dress'	[8, 9, 10, 11, 12, 13]	6
'boots'	[14, ]	1
'HandBag','ShoulderBag','Backpack','HoldObjectsInFront'	[15, 16, 17, 18]	4
'AgeOver60', 'Age18-60', 'AgeLess18'	[19, 20, 21]	3
'Female'	[22, ]	1
'Front','Side','Back'	[23, 24, 25]	3

Examples:

[0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]

The first group: position [0, 1] values are [0, 1], which means'no hat', 'has glasses'.

The second group: position [22, ] values are [0, ], indicating that the gender attribute is 'male', otherwise it is 'female'.

The third group: position [23, 24, 25] values are [0, 1, 0], indicating that the direction attribute is 'side'.

Other groups follow in this order

Data Annotation

After knowing the purpose of the above attribute annotation format, we can start to annotate data. The essence is that each single-person image creates a set of 26 annotation items, corresponding to the attribute values at 26 positions.

Examples:

For an original image:

Using bounding boxes to annotate the position of each person in the picture.
Each detection box (corresponding to each person) contains 26 attribute values which are represented by 0 or 1. It corresponds to the above 26 attributes. For example, if the picture is 'Female', then the 22nd bit of the array is 0. If the person is between 'Age18-60', then the corresponding value at position [19, 20, 21] is [0, 1, 0], or if the person matches 'AgeOver60', then the corresponding value is [1, 0, 0].

After the annotation is completed, the model will use the detection box to intercept each person into a single-person picture, and its picture establishes a corresponding relationship with the 26 attribute annotation. It is also possible to cut into a single-person image first and then annotate it. The results are the same.

Model Training

Once the data is annotated, it can be used for model training to complete the optimization of the customized model.

There are two main steps: 1) Organize the data and annotated data into the training format. 2) Modify the configuration file to start training.

Training data format

The training data includes the images used for training and a training list called train.txt. Its location is specified in the training configuration, with the following example:

text

Attribute/
|-- data Training images folder
|-- 00001.jpg
|-- 00002.jpg
| `-- 0000x.jpg
train.txt List of training data

train.txt file contains the names of all training images (file path relative to the root path) + 26 annotation values

Each line of it represents a person's image and annotation result. The format is as follows:

text

00001.jpg    0,0,1,0,....

Note 1) The images are separated by Tab[\t], 2) The annotated values are separated by commas [,]. If the format is wrong, the parsing will fail.

Modify the configuration to start training

First run the following command to download the training code (for more environmental issues, please refer to [Install_PaddleClas](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/en/installation/ install_paddleclas_en.md)):

text

git clone https://github.com/PaddlePaddle/PaddleClas

You need to modify the following configuration in the configuration file PaddleClas/blob/develop/ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml

text

DataLoader:
  Train:
    Train: dataset:
      name: MultiLabelDataset
      image_root: "dataset/pa100k/" #Specify the root path of training image
      cls_label_path: "dataset/pa100k/train_list.txt" #Specify the location of the training list file
      label_ratio: True
      transform_ops:

  Eval:
    dataset:
      name: MultiLabelDataset
      image_root: "dataset/pa100k/" #Specify the root path of evaluated image
      cls_label_path: "dataset/pa100k/val_list.txt" #Specify the location of the evaluation list file
      label_ratio: True
      transform_ops:

Note:

here image_root path and the relative path of the image in train.txt, corresponding to the full path of the image.
If you modify the number of attributes, the number of attribute types in the content configuration item should also be modified accordingly.

text

# model architecture
Arch:
name: "PPLCNet_x1_0"
pretrained: True
use_ssld: True
class_num: 26           #Attribute classes and numbers

Then run the following command to start training:

text

#Multi-card training
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/train.py \
        -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml

#Single card training
python3 tools/train.py \
        -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml

You can run the following commands for performance evaluation after the training is completed:

text

#Multi-card evaluation
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/eval.py \
        -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
        -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model

#Single card evaluation
python3 tools/eval.py \
        -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
        -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model

Model Export

Use the following command to export the trained model as an inference deployment model.

text

python3 tools/export_model.py \
    -c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
    -o Global.pretrained_model=output/PPLCNet_x1_0/best_model \
    -o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_person_attribute_infer

After exporting the model, you need to download the infer_cfg.yml file and put it into the exported model folder PPLCNet_x1_0_person_ attribute_infer .

When you use the model, you need to modify the new model path model_dir entry and set enable: True in the configuration file of PP-Human . /deploy/pipeline/config/infer_cfg_pphuman.yml .

text

ATTR:
  model_dir: [YOUR_DEPLOY_MODEL_DIR]/PPLCNet_x1_0_person_attribute_infer/   #The exported model location
  enable: True                                                              #Whether to enable the function

Now, the model is ready for you.

To this point, a new attribute category recognition task is completed.

Adding or deleting attributes

The above is the annotation and training process with 26 attributes.

If the attributes need to be added or deleted, you need to

New attribute category information needs to be added or deleted when annotating the data.
Modify the number and name of attributes used in train.txt corresponding to the training.
Modify the training configuration, for example, the number of attributes in the PaddleClas/blob/develop/ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml file, for details, please see the Modify configuration to start training section above.

Example of adding attributes.

Continue to add new attribute annotation values after 26 values when annotating the data.
Add new attribute values to the annotated values in the train.txt file as well.
The above is the annotation and training process with 26 attributes.

If the attributes need to be added or deleted, you need to
1. New attribute category information needs to be added or deleted when annotating the data.
2. Modify the number and name of attributes used in train.txt corresponding to the training.
3. Modify the training configuration, for example, the number of attributes in the PaddleClas/blob/develop/ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml file, for details, please see the Modify configuration to start training section above.
Example of adding attributes.
1. Continue to add new attribute annotation values after 26 values when annotating the data.
2. Add new attribute values to the annotated values in the train.txt file as well.
3. Note that the correlation of attribute types and values in train.txt needs to be fixed, for example, the [19, 20, 21] position indicates age, and all images should use the [19, 20, 21] position to indicate age.
The same applies to the deletion of attributes.
For example, if the age attribute is not needed, the values in positions [19, 20, 21] can be removed. You can simply remove all the values in positions 19-21 from the 26 numbers marked in train.txt, and you no longer need to annotate these 3 attribute values.

Advanced Tutorials Customization Pphuman Mot (`docs/advanced_tutorials/customization/pphuman_mot.md`)

简体中文 | English

多目标跟踪任务二次开发

在产业落地过程中应用多目标跟踪算法，不可避免地会出现希望自定义类型的多目标跟踪的需求，或是对已有多目标跟踪模型的优化，以提升在特定场景下模型的效果。我们在本文档通过案例来介绍如何根据期望识别的行为来进行多目标跟踪方案的选择，以及使用PaddleDetection进行多目标跟踪算法二次开发工作，包括：数据准备、模型优化思路和跟踪类别修改的开发流程。

数据准备

多目标跟踪模型方案采用ByteTrack，其中使用PP-YOLOE替换原文的YOLOX作为检测器，使用BYTETracker作为跟踪器，详细文档参考ByteTrack。原文的ByteTrack只支持行人单类别，PaddleDetection中也支持多类别同时进行跟踪。训练ByteTrack也就是训练检测器的过程，只需要准备好检测标注即可，不需要ReID标注信息，即当成纯检测来做即可。数据集最好是连续视频中抽取出来的而不是无关联的图片集合。
二次开发首先需要进行数据集的准备，针对场景特点采集合适的数据从而提升模型效果和泛化性能。然后使用Labeme，LabelImg等标注工具标注目标检测框，并将标注结果转化为COCO或VOC数据格式。详细文档可以参考数据准备文档

模型优化

1. 使用自定义数据集训练

ByteTrack跟踪方案采用的数据集只需要有检测标注即可。参照MOT数据集准备和MOT数据集教程。

text

# 单卡训练
CUDA_VISIBLE_DEVICES=0 python tools/train.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml --eval --amp

# 多卡训练
python -m paddle.distributed.launch --log_dir=log_dir --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml --eval --amp

更详细的命令参考30分钟快速上手PaddleDetection和ByteTrack

2. 加载COCO模型作为预训练

1) 设置预训练权重路径

2) 修改超参数

加载COCO预训练权重后，需要修改学习率超参数，例如configs/ppyoloe/_base_/optimizer_300e.yml中:

text

epoch: 120 # 原始配置为300epoch，加载COCO权重后可以适当减少迭代轮数

LearningRate:
  base_lr: 0.005 # 原始配置为0.025，加载COCO权重后需要降低学习率
  schedulers:
    - !CosineDecay
      max_epochs: 144 # 依据epoch数进行修改，一般为epoch数的1.2倍
    - !LinearWarmup
      start_factor: 0.
      epochs: 5

跟踪类别修改

当实际使用场景类别发生变化时，需要修改数据配置文件，例如configs/datasets/coco_detection.yml中:

text

metric: COCO
num_classes: 10 # 原始类别1

配置修改完成后，同样可以加载COCO预训练权重，PaddleDetection支持自动加载shape匹配的权重，对于shape不匹配的权重会自动忽略，因此无需其他修改。

Advanced Tutorials Customization Pphuman Mot En (`docs/advanced_tutorials/customization/pphuman_mot_en.md`)

简体中文 | English

Customized multi-object tracking task

When applying multi-object tracking algorithms in industrial applications, there will be inevitable demands for customized types of multi-object tracking or optimization of existing multi-object tracking models to improve the effectiveness of the models in specific scenarios. In this document, we present examples of how to choose a multi-object tracking solution based on the expected identified behavior, and how to use PaddleDetection for further development of multi-object tracking algorithms, including data preparation, model optimization ideas, and the development process of tracking category modification.

Data Preparation

The multi-object tracking model scheme uses ByteTrack, which adopts PP-YOLOE to replace the original YOLOX as a detector and BYTETracker as a tracker, for details, please refer to [ByteTrack](... /... /... /configs/mot/bytetrack). The original ByteTrack only supports single pedestrian category, while PaddleDetection supports multiple categories for simultaneous tracking. Training ByteTrack, which is the process of training the detector, only requires the detection annotations to be prepared, and does not require ReID annotation information, i.e., it can be done as pure detection. The dataset should preferably be extracted from continuous video rather than a collection of unrelated images.

Customization starts with the preparation of the dataset. We need to collect suitable data for the scenario features, so as to improve the model effect and generalization performance. Then Labeme, LabelImg and other labeling tools will be used to label the object detection frame and convert the labeling results into COCO or VOC data format. Details please refer to Data Preparation

Model Optimization

1. Use customized data set for training

The dataset used by the ByteTrack tracking solution only needs detection annotations. Refer to [MOT dataset preparation](... /... /... /configs/mot) and MOT dataset tutorial.

text

# Single card training
CUDA_VISIBLE_DEVICES=0 python tools/train.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml --eval --amp

# Multi-card training
python -m paddle.distributed.launch --log_dir=log_dir --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml --eval --amp

More details please refer to Getting Started for PaddleDetection and ByteTrack

2. Load the COCO model as the pre-trained model

1) Set pre-training weight path

2) Modify hyperparameters

After loading the COCO pre-training weights, the learning rate hyperparameters need to be modified, for example

In configs/ppyoloe/*base*/optimizer_300e.yml:

text

epoch: 120 # The original configuration is 300 epoch, after loading COCO weights, the iteration number can be reduced appropriately

LearningRate:
base_lr: 0.005 # The original configuration is 0.025, after loading COCO weights, the learning rate should be reduced.
  schedulers:
    - !CosineDecay
      max_epochs: 144 # Modified according to the number of epochs, usually 1.2 times the number of epochs
    - LinearWarmup
      start_factor: 0.
      epochs: 5

Modify categories

When the actual application scenario category changes, the data configuration file needs to be modified, for example in configs/datasets/coco_detection.yml:

text

metric: COCO
num_classes: 10 # original class 80

Advanced Tutorials Customization Pphuman Mtmct (`docs/advanced_tutorials/customization/pphuman_mtmct.md`)

简体中文 | English

跨镜跟踪任务二次开发

数据准备

数据格式

跨镜跟踪使用行人REID技术实现，其训练方式采用多分类模型训练，使用时取分类softmax头部前的特征作为检索特征向量。

因此其格式与多分类任务相同。每一个行人分配一个专属id，不同行人id不同，同一行人在不同图片中的id相同。

例如图片0001.jpg、0003.jpg是同一个人，0002.jpg、0004.jpg是不同的其他行人。则标注id为：

text

0001.jpg    00001
0002.jpg    00002
0003.jpg    00001
0004.jpg    00003
...

依次类推。

数据标注

理解了上面标注格式的含义后，就可以进行数据标注的工作。其本质是：每张单人图建立一个标注项，对应该行人分配的id。

举例：

对于一张原始图片，

1）使用检测框，标注图片中每一个人的位置。

2）每一个检测框（对应每一个人），包含一个int类型的id属性。例如，上述举例中的0001.jpg中的人，对应id：1.

标注完成后利用检测框将每一个人截取成单人图，其图片与id属性标注建立对应关系。也可先截成单人图再进行标注，效果相同。

模型训练

数据标注完成后，就可以拿来做模型的训练，完成自定义模型的优化工作。

其主要有两步工作需要完成：1）将数据与标注数据整理成训练格式。2）修改配置文件开始训练。

训练数据格式

训练数据包括训练使用的图片和一个训练列表bounding_box_train.txt，其具体位置在训练配置中指定，其放置方式示例如下：

text

REID/
|-- data           训练图片文件夹
|   |-- 00001.jpg
|   |-- 00002.jpg
|   `-- 0000x.jpg
`-- bounding_box_train.txt      训练数据列表

bounding_box_train.txt文件内为所有训练图片名称（相对于根路径的文件路径）+ 1个id标注值

其每一行表示一个人的图片和id标注结果。其格式为：

text

0001.jpg    00001
0002.jpg    00002
0003.jpg    00001
0004.jpg    00003

注意：图片与标注值之间是以Tab[\t]符号隔开。该格式不能错，否则解析失败。

修改配置开始训练

首先执行以下命令下载训练代码（更多环境问题请参考Install_PaddleClas）:

shell

git clone https://github.com/PaddlePaddle/PaddleClas

需要在配置文件softmax_triplet_with_center.yaml中，修改的配置项如下：

text

Head:
    name: "FC"
    embedding_size: *feat_dim
    class_num: &class_num 751                   #行人id总数量

DataLoader:
  Train:
    dataset:
        name: "Market1501"
        image_root: "./dataset/"                #训练图片根路径
        cls_label_path: "bounding_box_train"    #训练文件列表


  Eval:
    Query:
      dataset:
        name: "Market1501"
        image_root: "./dataset/"                #评估图片根路径
        cls_label_path: "query"                 #评估文件列表

注意：

这里image_root路径+bounding_box_train.txt中图片相对路径，对应图片存放的完整路径。

然后运行以下命令开始训练。

text

#多卡训练
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/train.py \
        -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml

#单卡训练
python3 tools/train.py \
    -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml

训练完成后可以执行以下命令进行性能评估：

text

#多卡评估
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/eval.py \
        -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \
        -o Global.pretrained_model=./output/strong_baseline/best_model

#单卡评估
python3 tools/eval.py \
        -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \
        -o Global.pretrained_model=./output/strong_baseline/best_model

模型导出

使用下述命令将训练好的模型导出为预测部署模型。

text

python3 tools/export_model.py \
    -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \
    -o Global.pretrained_model=./output/strong_baseline/best_model \
    -o Global.save_inference_dir=deploy/models/strong_baseline_inference

导出模型后，下载infer_cfg.yml文件到新导出的模型文件夹'strong_baseline_inference'中。

使用时在PP-Human中的配置文件infer_cfg_pphuman.yml中修改模型路径model_dir并开启功能enable。

text

REID:
  model_dir: [YOUR_DEPLOY_MODEL_DIR]/strong_baseline_inference/
  enable: True

然后可以使用。至此完成模型开发。

Advanced Tutorials Customization Pphuman Mtmct En (`docs/advanced_tutorials/customization/pphuman_mtmct_en.md`)

简体中文 | English

Customized Multi-Target Multi-Camera Tracking Module of PP-Human

Data Preparation

Data Format

Multi-target multi-camera tracking, or mtmct is achieved by the pedestrian REID technique. It is trained with a multiclassification model and uses the features before the head of the classification softmax as the retrieval feature vector.

Therefore its format is the same as the multi-classification task. Each pedestrian is assigned an exclusive id, which is different for different pedestrians while the same pedestrian has the same id in different images.

For example, images 0001.jpg, 0003.jpg are the same person, 0002.jpg, 0004.jpg are different pedestrians. Then the labeled ids are.

text

0001.jpg    00001
0002.jpg    00002
0003.jpg    00001
0004.jpg    00003
...

Data Annotation

After understanding the meaning of the annotation format above, we can work on the data annotation. The essence of data annotation is that each single person diagram creates an annotation item that corresponds to the id assigned to that pedestrian.

For example:

For an original picture

Use bouding boxes to annotate the position of each person in the picture.
Each bouding box (corresponding to each person) contains an int id attribute. For example, the person in 0001.jpg in the above example corresponds to id: 1.

After the annotation is completed, use the detection box to intercept each person into a single picture, the picture and id attribute annotation will establish a corresponding relationship. You can also first cut into a single image and then annotate, the result is the same.

Model Training

Once the data is annotated, it can be used for model training to complete the optimization of the customized model.

There are two main steps to implement: 1) organize the data and annotated data into a training format. 2) modify the configuration file to start training.

Training data format

The training data consists of the images used for training and a training list bounding_box_train.txt, the location of which is specified in the training configuration, with the following example placement.

text

REID/
|-- data Training image folder
|-- 00001.jpg
|-- 00002.jpg
|-- 0000x.jpg
`-- bounding_box_train.txt List of training data

bounding_box_train.txt file contains the names of all training images (file path relative to the root path) + 1 id annotation value

Each line represents a person's image and id annotation result. The format is as follows:

text

0001.jpg    00001
0002.jpg    00002
0003.jpg    00001
0004.jpg    00003

Note: The images are separated from the annotated values by a Tab[\t] symbol. This format must be correct, otherwise, the parsing will fail.

Modify the configuration to start training

First, execute the following command to download the training code (for more environment issues, please refer to [Install_PaddleClas](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/en/installation/ install_paddleclas_en.md):

text

git clone https://github.com/PaddlePaddle/PaddleClas

You need to change the following configuration items in the configuration file [softmax_triplet_with_center.yaml](https://github.com/PaddlePaddle/PaddleClas/blob/develop/ppcls/configs/reid/strong_ baseline/softmax_triplet_with_center.yaml):

text

Head:
    name: "FC"
    embedding_size: *feat_dim
    class_num: &class_num 751 #Total number of pedestrian ids

DataLoader:
  Train:
    dataset:
        name: "Market1501"
        image_root: ". /dataset/" #training image root path
        cls_label_path: "bounding_box_train" #training_file_list


  Eval:
    Query:
      dataset:
        name: "Market1501"
        image_root: ". /dataset/" #Evaluated image root path
        cls_label_path: "query" #List of evaluation files

Note:

Here the image_root path + the relative path of the image in the bounding_box_train.txt corresponds to the full path where the image is stored.

Then run the following command to start the training.

text

#Multi-card training
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/train.py \
        -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml

#Single card training
python3 tools/train.py \
    -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml

After the training is completed, you may run the following commands for performance evaluation:

text

#Multi-card evaluation
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/eval.py \
        -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \
        -o Global.pretrained_model=./output/strong_baseline/best_model

#Single card evaluation
python3 tools/eval.py \
        -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \
        -o Global.pretrained_model=./output/strong_baseline/best_model

Model Export

Use the following command to export the trained model as an inference deployment model.

text

python3 tools/export_model.py \
    -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \
    -o Global.pretrained_model=./output/strong_baseline/best_model \
    -o Global.save_inference_dir=deploy/models/strong_baseline_inference

After exporting the model, download the infer_cfg.yml file to the newly exported model folder 'strong_baseline_ inference'.

Change the model path model_dir in the configuration file infer_cfg_pphuman.yml in PP-Human and set enable.

text

REID:
 model_dir: [YOUR_DEPLOY_MODEL_DIR]/strong_baseline_inference/
 enable: True

Now, the model is ready.

Advanced Tutorials Customization Ppvehicle Attribute (`docs/advanced_tutorials/customization/ppvehicle_attribute.md`)

简体中文 | English

车辆属性识别任务二次开发

数据准备

数据格式

车辆属性模型采用VeRi数据集的属性，共计10种车辆颜色及9种车型, 具体如下：

text

# 车辆颜色
- "yellow"
- "orange"
- "green"
- "gray"
- "red"
- "blue"
- "white"
- "golden"
- "brown"
- "black"

# 车型
- "sedan"
- "suv"
- "van"
- "hatchback"
- "mpv"
- "pickup"
- "bus"
- "truck"
- "estate"

在标注文件中使用长度为19的序列来表示上述属性。

举例：

[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]

前10位中，位序号0的值为1，表示车辆颜色为"yellow"。

后9位中，位序号11的值为1，表示车型为"suv"。

数据标注

理解了上面数据格式的含义后，就可以进行数据标注的工作。其本质是：每张车辆的图片，建立一组长度为19的标注项，分别对应各项属性值。

举例：

对于一张原始图片，

1）使用检测框，标注图片中每台车辆的位置。

2）每一个检测框（对应每辆车），包含一组19位的属性值数组，数组的每一位以0或1表示。对应上述19个属性分类。例如，如果颜色是'orange'，则数组索引为1的位置值为1，如果车型是'sedan'，则数组索引为10的位置值为1。

标注完成后利用检测框将每辆车截取成只包含单辆车的图片，则图片与19位属性标注建立了对应关系。也可先截取再进行标注，效果相同。

模型训练

数据标注完成后，就可以拿来做模型的训练，完成自定义模型的优化工作。

其主要有两步工作需要完成：1）将数据与标注数据整理成训练格式。2）修改配置文件开始训练。

训练数据格式

训练数据包括训练使用的图片和一个训练列表train.txt，其具体位置在训练配置中指定，其放置方式示例如下：

text

Attribute/
|-- data           训练图片文件夹
|   |-- 00001.jpg
|   |-- 00002.jpg
|   `-- 0000x.jpg
`-- train.txt      训练数据列表

train.txt文件内为所有训练图片名称（相对于根路径的文件路径）+ 19个标注值

其每一行表示一辆车的图片和标注结果。其格式为：

text

00001.jpg    0,0,1,0,....

注意：1)图片与标注值之间是以Tab[\t]符号隔开, 2)标注值之间是以逗号[,]隔开。该格式不能错，否则解析失败。

修改配置开始训练

首先执行以下命令下载训练代码（更多环境问题请参考Install_PaddleClas）:

shell

git clone https://github.com/PaddlePaddle/PaddleClas

需要在配置文件中，修改的配置项如下：

yaml

DataLoader:
  Train:
    dataset:
      name: MultiLabelDataset
      image_root: "dataset/VeRi/"                     # the root path of training images
      cls_label_path: "dataset/VeRi/train_list.txt"   # the location of the training list file
      label_ratio: True
      transform_ops:
        ...

  Eval:
    dataset:
      name: MultiLabelDataset
      image_root: "dataset/VeRi/"                     # the root path of evaluation images
      cls_label_path: "dataset/VeRi/val_list.txt"     # the location of the evaluation list file
      label_ratio: True
      transform_ops:
         ...

注意：

这里image_root路径+train.txt中图片相对路径，对应图片的完整路径位置。
如果有修改属性数量，则还需修改内容配置项中属性种类数量：

yaml

# model architecture
Arch:
  name: "PPLCNet_x1_0"
  pretrained: True
  use_ssld: True
  class_num: 19           #属性种类数量

然后运行以下命令开始训练。

text

#多卡训练
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/train.py \
        -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml

#单卡训练
python3 tools/train.py \
        -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml

训练完成后可以执行以下命令进行性能评估：

text

#多卡评估
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/eval.py \
        -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
        -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model

#单卡评估
python3 tools/eval.py \
        -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
        -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model

模型导出

使用下述命令将训练好的模型导出为预测部署模型。

text

python3 tools/export_model.py \
    -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
    -o Global.pretrained_model=output/PPLCNet_x1_0/best_model \
    -o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_vehicle_attribute_model

导出模型后，如果希望在PP-Vehicle中使用，则需要下载预测部署模型，解压并将其中的配置文件infer_cfg.yml文件，放置到导出的模型文件夹PPLCNet_x1_0_vehicle_attribute_model中。

使用时在PP-Vehicle中的配置文件./deploy/pipeline/config/infer_cfg_ppvehicle.yml中修改新的模型路径model_dir项，并开启功能enable: True。

text

VEHICLE_ATTR:
  model_dir: [YOUR_DEPLOY_MODEL_DIR]/PPLCNet_x1_0_vehicle_attribute_infer/   #新导出的模型路径位置
  enable: True                                                              #开启功能

然后可以使用-->至此即完成新增属性类别识别任务。

属性增减

该过程与行人属性的增减过程相似，如果需要增加、减少属性数量，则需要：

1)标注时需增加新属性类别信息或删减属性类别信息；

2)对应修改训练中train.txt所使用的属性数量和名称；

3)修改训练配置，例如PaddleClas/blob/develop/ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml文件中的属性数量,详细见上述修改配置开始训练部分。

增加属性示例：

在标注数据时在19位后继续增加新的属性标注数值；
在train.txt文件的标注数值中也增加新的属性数值。
注意属性类型在train.txt中属性数值列表中的位置的对应关系需要固定。

删减属性同理。

修改后处理代码

修改了属性定义后，pipeline后处理部分也需要做相应修改，主要影响结果可视化时的显示结果。

相应代码在文件中postprocess函数。

其函数实现说明如下：

python

# 在类的初始化函数中，定义了颜色/车型的名称
    self.color_list = [
        "yellow", "orange", "green", "gray", "red", "blue", "white",
        "golden", "brown", "black"
    ]
    self.type_list = [
        "sedan", "suv", "van", "hatchback", "mpv", "pickup", "bus", "truck",
        "estate"
    ]

    ...

    def postprocess(self, inputs, result):
        # postprocess output of predictor
        im_results = result['output']
        batch_res = []
        for res in im_results:
            res = res.tolist()
            attr_res = []
            color_res_str = "Color: "
            type_res_str = "Type: "
            color_idx = np.argmax(res[:10])   # 前10项表示各项颜色得分，取得分最大项作为颜色结果
            type_idx = np.argmax(res[10:])    # 后9项表示各项车型得分，取得分最大项作为车型结果

            # 颜色和车型的得分都需要超过对应阈值，否则视为'UnKnown'
            if res[color_idx] >= self.color_threshold:
                color_res_str += self.color_list[color_idx]
            else:
                color_res_str += "Unknown"
            attr_res.append(color_res_str)

            if res[type_idx + 10] >= self.type_threshold:
                type_res_str += self.type_list[type_idx]
            else:
                type_res_str += "Unknown"
            attr_res.append(type_res_str)

            batch_res.append(attr_res)
        result = {'output': batch_res}
        return result

Advanced Tutorials Customization Ppvehicle Attribute En (`docs/advanced_tutorials/customization/ppvehicle_attribute_en.md`)

简体中文 | English

Customized Vehicle Attribute Recognition

Data Preparation

Data Format

We use the VeRi attribute annotation format, with a total of 10 color and 9 model attributes shown as follows.

text

# colors
- "yellow"
- "orange"
- "green"
- "gray"
- "red"
- "blue"
- "white"
- "golden"
- "brown"
- "black"

# models
- "sedan"
- "suv"
- "van"
- "hatchback"
- "mpv"
- "pickup"
- "bus"
- "truck"
- "estate"

A sequence of length 19 is used in the annotation file to represent the above attributes.

Examples:

[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]

In the first 10 bits, the value of bit index 0 is 1, indicating that the vehicle color is "yellow".

In the last 9 bits, the value of bit index 11 is 1, indicating that the model is "suv".

Data Annotation

After knowing the purpose of the above Data format, we can start to annotate data. The essence is that each single-vehicle image creates a set of 19 annotation items, corresponding to the attribute values at 19 positions.

Examples:

For an original image:

Using bounding boxes to annotate the position of each vehicle in the picture.
Each detection box (corresponding to each vehicle) contains 19 attribute values which are represented by 0 or 1. It corresponds to the above 19 attributes. For example, if the color is 'orange', then the index 1 bit of the array is 1. If the model is 'sedan', then the index 10 bit of the array is 1.

After the annotation is completed, the model will use the detection box to intercept each vehicle into a single-vehicle picture, and its picture establishes a corresponding relationship with the 19 attribute annotation. It is also possible to cut into a single-vehicle image first and then annotate it. The results are the same.

Model Training

Once the data is annotated, it can be used for model training to complete the optimization of the customized model.

There are two main steps: 1) Organize the data and annotated data into the training format. 2) Modify the configuration file to start training.

Training Data Format

The training data includes the images used for training and a training list called train.txt. Its location is specified in the training configuration, with the following example:

text

Attribute/
|-- data      Training images folder
|-- 00001.jpg
|-- 00002.jpg
| `-- 0000x.jpg
train.txt     List of training data

train.txt file contains the names of all training images (file path relative to the root path) + 19 annotation values

Each line of it represents a vehicle's image and annotation result. The format is as follows:

text

00001.jpg    0,0,1,0,....

Note 1) The images are separated by Tab[\t], 2) The annotated values are separated by commas [,]. If the format is wrong, the parsing will fail.

Modify The Configuration To Start Training

text

git clone https://github.com/PaddlePaddle/PaddleClas

You need to modify the following configuration in the configuration file PaddleClas/blob/develop/ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml

yaml

DataLoader:
  Train:
    dataset:
      name: MultiLabelDataset
      image_root: "dataset/VeRi/"                     # the root path of training images
      cls_label_path: "dataset/VeRi/train_list.txt"   # the location of the training list file
      label_ratio: True
      transform_ops:
        ...

  Eval:
    dataset:
      name: MultiLabelDataset
      image_root: "dataset/VeRi/"                     # the root path of evaluation images
      cls_label_path: "dataset/VeRi/val_list.txt"     # the location of the training list file
      label_ratio: True
      transform_ops:
         ...

Note:

here image_root path and the relative path of the image in train.txt, corresponding to the full path of the image.
If you modify the number of attributes, the number of attribute types in the content configuration item should also be modified accordingly.

yaml

# model architecture
Arch:
name: "PPLCNet_x1_0"
pretrained: True
use_ssld: True
class_num: 19           # Number of attribute classes

Then run the following command to start training:

bash

#Multi-card training
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/train.py \
        -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml

#Single card training
python3 tools/train.py \
        -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml

You can run the following commands for performance evaluation after the training is completed:

text

#Multi-card evaluation
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    --gpus="0,1,2,3" \
    tools/eval.py \
        -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
        -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model

#Single card evaluation
python3 tools/eval.py \
        -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
        -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model

Model Export

Use the following command to export the trained model as an inference deployment model.

text

python3 tools/export_model.py \
    -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
    -o Global.pretrained_model=output/PPLCNet_x1_0/best_model \
    -o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_vehicle_attribute_model

After exporting the model, if want to use it in PP-Vehicle, you need to download the deploy infer model and copy infer_cfg.yml into the exported model folder PPLCNet_x1_0_vehicle_attribute_model .

When you use the model, you need to modify the new model path model_dir entry and set enable: True in the configuration file of PP-Vehicle . /deploy/pipeline/config/infer_cfg_ppvehicle.yml .

text

VEHICLE_ATTR:
  model_dir: [YOUR_DEPLOY_MODEL_DIR]/PPLCNet_x1_0_vehicle_attribute_infer/   #The exported model location
  enable: True                                                              #Whether to enable the function

To this point, a new attribute category recognition task is completed.

Adding or deleting attributes

This is similar to the increase and decrease process of pedestrian attributes.

If the attributes need to be added or deleted, you need to

New attribute category information needs to be added or deleted when annotating the data.
Modify the number and name of attributes used in train.txt corresponding to the training.
Modify the training configuration, for example, the number of attributes in the PaddleClas/blob/develop/ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml file, for details, please see the Modify configuration to start training section above.

Example of adding attributes.

Continue to add new attribute annotation values after 19 values when annotating the data.
Add new attribute values to the annotated values in the train.txt file as well.
The above is the annotation and training process with 19 attributes.

The same applies to the deletion of attributes.

Modifications to post-processing code

After modifying the attribute definition, the post-processing part of the pipeline also needs to be modified accordingly, which mainly affects the display results when the results are visualized.

The code is at file, that is, the postprocess function.

The function implementation is described as follows:

text

/* Detailed source-code truncated for AI context efficiency. */

Advanced Tutorials Customization Ppvehicle Plate (`docs/advanced_tutorials/customization/ppvehicle_plate.md`)

简体中文 | English

车牌识别任务二次开发

车牌识别任务，采用PP-OCRv3模型在车牌数据集上进行fine-tune得到，过程参考PaddleOCR车牌应用介绍在CCPD2019数据集上进行了拓展。

数据准备

对于CCPD2019、CCPD2020数据集，我们提供了处理脚本ccpd2ocr_all.py, 使用时跟CCPD2019、CCPD2020数据集文件夹放在同一目录下，然后执行脚本即可在CCPD2019/PPOCR、CCPD2020/PPOCR目录下得到检测、识别模型的训练标注文件。训练时可以整合到一起使用。
对于其他来源数据或者自标注数据，可以按如下格式整理训练列表文件：

车牌检测标注

标注文件格式如下，中间用'\t'分隔：

text

" 图像文件路径                    标注框标注信息"
CCPD2020/xxx.jpg    [{"transcription": "京AD88888", "points": [[310, 104], [416, 141], [418, 216], [312, 179]]}, {...}]

标注框标注信息是包含多个字典的list，有多少个标注框就有多少个字典对应，字典中的 points 表示车牌框的四个点的坐标(x, y)，从左上角的点开始顺时针排列。 transcription 表示当前文本框的文字，当其内容为“###”时，表示该文本框无效，在训练时会跳过。

车牌字符识别标注

标注文件的格式如下，txt文件中默认请将图片路径和图片标签用'\t'分割，如用其他方式分割将造成训练报错。其中图片是对车牌字符的截图。

text

" 图像文件名                 字符标注信息 "
CCPD2020/crop_imgs/xxx.jpg   京AD88888

模型训练

首先执行以下命令clone PaddleOCR库代码到训练机器：

text

git clone [email protected]:PaddlePaddle/PaddleOCR.git

下载预训练模型：

text

#检测预训练模型：
mkdir models
cd models
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_distill_train.tar
tar -xf ch_PP-OCRv3_det_distill_train.tar

#识别预训练模型：
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_train.tar
tar -xf ch_PP-OCRv3_rec_train.tar
cd ..

安装相关依赖环境：

text

cd PaddleOCR
pip install -r requirements.txt

然后进行训练相关配置修改。

修改配置

检测模型配置项

修改配置项包括以下3部分内容，可以在训练时以命令行修改，或者直接在配置文件configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_student.yml中修改：

模型存储和训练相关:

Global.pretrained_model: 指向前面下载的PP-OCRv3文本检测预训练模型地址
Global.eval_batch_step: 模型多少step评估一次，一般设置为一个epoch对应的step数，可以从训练开始的log中读取。此处以[0, 772]为例，第一个数字表示从第0各step开始算起。

优化器相关:

Optimizer.lr.name: 学习率衰减器设为常量 Const
Optimizer.lr.learning_rate: 做 fine-tune 实验，学习率需要设置的比较小，此处学习率设为配置文件中的0.05倍
Optimizer.lr.warmup_epoch: warmup_epoch设为0

数据集相关:

Train.dataset.data_dir：指向训练集图片存放根目录
Train.dataset.label_file_list：指向训练集标注文件
Eval.dataset.data_dir：指向测试集图片存放根目录
Eval.dataset.label_file_list：指向测试集标注文件

识别模型配置项

修改配置项包括以下3部分内容，可以在训练时以命令行修改，或者直接在配置文件configs/rec/PP-OCRv3/ch_PP-OCRv3_rec.yml中修改：

模型存储和训练相关:

Global.pretrained_model: 指向PP-OCRv3文本识别预训练模型地址
Global.eval_batch_step: 模型多少step评估一次，一般设置为一个epoch对应的step数，可以从训练开始的log中读取。此处以[0, 90]为例，第一个数字表示从第0各step开始算起。

优化器相关

Optimizer.lr.name: 学习率衰减器设为常量 Const
Optimizer.lr.learning_rate: 做 fine-tune 实验，学习率需要设置的比较小，此处学习率设为配置文件中的0.05倍
Optimizer.lr.warmup_epoch: warmup_epoch设为0

数据集相关

Train.dataset.data_dir：指向训练集图片存放根目录
Train.dataset.label_file_list：指向训练集标注文件
Eval.dataset.data_dir：指向测试集图片存放根目录
Eval.dataset.label_file_list：指向测试集标注文件

执行训练

然后运行以下命令开始训练。如果在配置文件中已经做了修改，可以省略-o及其后面的内容。

检测模型训练命令

text

#单卡训练
python3 tools/train.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_student.yml -o \
    Global.pretrained_model=models/ch_PP-OCRv3_det_distill_train/student.pdparams \
    Global.save_model_dir=output/CCPD/det \
    Global.eval_batch_step="[0, 772]" \
    Optimizer.lr.name=Const \
    Optimizer.lr.learning_rate=0.0005 \
    Optimizer.lr.warmup_epoch=0 \
    Train.dataset.data_dir=/home/aistudio/ccpd_data/ \
    Train.dataset.label_file_list=[/home/aistudio/ccpd_data/train/det.txt]

#多卡训练
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    tools/train.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_student.yml -o \
    Global.pretrained_model=models/ch_PP-OCRv3_det_distill_train/student.pdparams \
    Global.save_model_dir=output/CCPD/det \
    Global.eval_batch_step="[0, 772]" \
    Optimizer.lr.name=Const \
    Optimizer.lr.learning_rate=0.0005 \
    Optimizer.lr.warmup_epoch=0 \
    Train.dataset.data_dir=/home/aistudio/ccpd_data/ \
    Train.dataset.label_file_list=[/home/aistudio/ccpd_data/train/det.txt]

训练完成后可以执行以下命令进行性能评估：

text

#单卡评估
python tools/eval.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_student.yml -o \
    Global.pretrained_model=output/CCPD/det/best_accuracy.pdparams \
    Eval.dataset.data_dir=/home/aistudio/ccpd_data/ \
    Eval.dataset.label_file_list=[/home/aistudio/ccpd_data/test/det.txt]

识别模型训练命令

text

#单卡训练
python3 tools/train.py -c configs/rec/PP-OCRv3/ch_PP-OCRv3_rec.yml -o \
    Global.pretrained_model=models/ch_PP-OCRv3_rec_train/student.pdparams \
    Global.save_model_dir=output/CCPD/rec/ \
    Global.eval_batch_step="[0, 90]" \
    Optimizer.lr.name=Const \
    Optimizer.lr.learning_rate=0.0005 \
    Optimizer.lr.warmup_epoch=0 \
    Train.dataset.data_dir=/home/aistudio/ccpd_data \
    Train.dataset.label_file_list=[/home/aistudio/ccpd_data/train/rec.txt] \
    Eval.dataset.data_dir=/home/aistudio/ccpd_data \
    Eval.dataset.label_file_list=[/home/aistudio/ccpd_data/test/rec.txt]


#多卡训练
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
    tools/train.py -c configs/rec/PP-OCRv3/ch_PP-OCRv3_rec.yml -o \
    Global.pretrained_model=models/ch_PP-OCRv3_rec_train/student.pdparams \
    Global.save_model_dir=output/CCPD/rec/ \
    Global.eval_batch_step="[0, 90]" \
    Optimizer.lr.name=Const \
    Optimizer.lr.learning_rate=0.0005 \
    Optimizer.lr.warmup_epoch=0 \
    Train.dataset.data_dir=/home/aistudio/ccpd_data \
    Train.dataset.label_file_list=[/home/aistudio/ccpd_data/train/rec.txt] \
    Eval.dataset.data_dir=/home/aistudio/ccpd_data \
    Eval.dataset.label_file_list=[/home/aistudio/ccpd_data/test/rec.txt]

训练完成后可以执行以下命令进行性能评估：

text

#单卡评估
python tools/eval.py -c configs/rec/PP-OCRv3/ch_PP-OCRv3_rec.yml -o \
    Global.pretrained_model=output/CCPD/rec/best_accuracy.pdparams \
    Eval.dataset.data_dir=/home/aistudio/ccpd_data/ \
    Eval.dataset.label_file_list=[/home/aistudio/ccpd_data/test/rec.txt]

模型导出

使用下述命令将训练好的模型导出为预测部署模型。

检测模型导出

text

python tools/export_model.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_student.yml -o \
    Global.pretrained_model=output/CCPD/det/best_accuracy.pdparams \
    Global.save_inference_dir=output/det/infer

识别模型导出

text

python tools/export_model.py -c configs/rec/PP-OCRv3/ch_PP-OCRv3_rec.yml -o \
    Global.pretrained_model=output/CCPD/rec/best_accuracy.pdparams \
    Global.save_inference_dir=output/CCPD/rec/infer

使用时在PP-Vehicle中的配置文件./deploy/pipeline/config/infer_cfg_ppvehicle.yml中修改VEHICLE_PLATE模块中的det_model_dir、rec_model_dir项，并开启功能enable: True。

text

VEHICLE_PLATE:
  det_model_dir: [YOUR_DET_INFERENCE_MODEL_PATH]                #设置检测模型路径
  det_limit_side_len: 736
  det_limit_type: "max"
  rec_model_dir: [YOUR_REC_INFERENCE_MODEL_PATH]                #设置识别模型路径
  rec_image_shape: [3, 48, 320]
  rec_batch_num: 6
  word_dict_path: deploy/pipeline/ppvehicle/rec_word_dict.txt
  enable: True                                                  #开启功能

然后可以使用-->至此即完成更新车牌识别模型任务。

Advanced Tutorials Customization Ppvehicle Violation (`docs/advanced_tutorials/customization/ppvehicle_violation.md`)

简体中文 | English

车辆违章任务二次开发

车辆违章任务的二次开发，主要集中于车道线分割模型任务。采用PP-LiteSeg模型在车道线数据集bdd100k,上进行fine-tune得到，过程参考PP-LiteSeg。

数据准备

ppvehicle违法分析将车道线类别分为4类

text

0 背景
1 双黄线
2 实线
3 虚线

对于bdd100k数据集，可以结合我们的提供的处理脚本lane_to_mask.py和bdd100k官方repo将数据处理成分割需要的数据格式.

text

#首先执行以下命令clone bdd100k库：
git clone https://github.com/bdd100k/bdd100k.git

#拷贝lane_to_mask.py到bdd100k目录
cp PaddleDetection/deploy/pipeline/tools/lane_to_mask.py bdd100k/

#准备bdd100k环境
cd bdd100k && pip install -r requirements.txt

#数据转换
python lane_to_mask.py -i dataset/labels/lane/polygons/lane_train.json -o /output_path

# -i bdd100k数据集label的json路径，
# -o 生成的mask图像路径

整理数据,按如下格式存放数据

text

dataset_root
    |
    |--images  
    |  |--train
    |       |--image1.jpg
    |       |--image2.jpg
    |       |--...
    |  |--val
    |       |--image3.jpg
    |       |--image4.jpg
    |       |--...
    |  |--test
    |       |--image5.jpg
    |       |--image6.jpg
    |       |--...
    |
    |--labels  
    |  |--train
    |       |--label1.jpg
    |       |--label2.jpg
    |       |--...
    |  |--val
    |       |--label3.jpg
    |       |--label4.jpg
    |       |--...
    |  |--test
    |       |--label5.jpg
    |       |--label6.jpg
    |       |--...
    |

运行create_dataset_list.py生成txt文件

text

python create_dataset_list.py <dataset_root> #数据根目录
                              --type  custom #数据类型，支持cityscapes、custom

其他数据以及数据标注，可参考PaddleSeg准备自定义数据集

模型训练

首先执行以下命令clone PaddleSeg库代码到训练机器：

text

git clone https://github.com/PaddlePaddle/PaddleSeg.git

安装相关依赖环境：

text

cd PaddleSeg
pip install -r requirements.txt

准备配置文件

详细可参考PaddleSeg准备配置文件.
本例用pp_liteseg_stdc2_bdd100k_1024x512.yml示例

text

batch_size: 16
iters: 50000

train_dataset:
  type: Dataset
  dataset_root: data/bdd100k    #数据集路径  
  train_path: data/bdd100k/train.txt #数据集训练txt文件
  num_classes: 4                     #ppvehicle将道路分为4类
  mode: train
  transforms:
    - type: ResizeStepScaling
      min_scale_factor: 0.5
      max_scale_factor: 2.0
      scale_step_size: 0.25
    - type: RandomPaddingCrop
      crop_size: [512, 1024]
    - type: RandomHorizontalFlip
    - type: RandomAffine
    - type: RandomDistort
      brightness_range: 0.5
      contrast_range: 0.5
      saturation_range: 0.5
    - type: Normalize

val_dataset:
  type: Dataset
  dataset_root: data/bdd100k    #数据集路径
  val_path: data/bdd100k/val.txt #数据集验证集txt文件
  num_classes: 4
  mode: val
  transforms:
    - type: Normalize

optimizer:
  type: sgd
  momentum: 0.9
  weight_decay: 4.0e-5

lr_scheduler:
  type: PolynomialDecay
  learning_rate: 0.01 #0.01
  end_lr: 0
  power: 0.9

loss:
  types:
    - type: MixedLoss
      losses:
        - type: CrossEntropyLoss
        - type: LovaszSoftmaxLoss
      coef: [0.6, 0.4]
    - type: MixedLoss
      losses:
        - type: CrossEntropyLoss
        - type: LovaszSoftmaxLoss
      coef: [0.6, 0.4]
    - type: MixedLoss
      losses:
        - type: CrossEntropyLoss
        - type: LovaszSoftmaxLoss
      coef: [0.6, 0.4]
  coef: [1, 1,1]


model:
  type: PPLiteSeg
  backbone:
    type: STDC2
    pretrained: https://bj.bcebos.com/paddleseg/dygraph/PP_STDCNet2.tar.gz #预训练模型

执行训练

text

#单卡训练
export CUDA_VISIBLE_DEVICES=0 # Linux上设置1张可用的卡
# set CUDA_VISIBLE_DEVICES=0  # Windows上设置1张可用的卡

python train.py \
       --config configs/pp_liteseg/pp_liteseg_stdc2_bdd100k_1024x512.yml \
       --do_eval \
       --use_vdl \
       --save_interval 500 \
       --save_dir output

训练参数解释

text

--do_eval 是否在保存模型时启动评估, 启动时将会根据mIoU保存最佳模型至best_model
--use_vdl 是否开启visualdl记录训练数据
--save_interval 500  模型保存的间隔步数
--save_dir output    模型输出路径

2、多卡训练

如果想要使用多卡训练的话，需要将环境变量CUDA_VISIBLE_DEVICES指定为多卡（不指定时默认使用所有的gpu)，并使用paddle.distributed.launch启动训练脚本（windows下由于不支持nccl，无法使用多卡训练）:

text

export CUDA_VISIBLE_DEVICES=0,1,2,3 # 设置4张可用的卡
python -m paddle.distributed.launch train.py \
       --config configs/pp_liteseg/pp_liteseg_stdc2_bdd100k_1024x512.yml \
       --do_eval \
       --use_vdl \
       --save_interval 500 \
       --save_dir output

训练完成后可以执行以下命令进行性能评估：

text

#单卡评估
python val.py \
       --config configs/pp_liteseg/pp_liteseg_stdc2_bdd100k_1024x512.yml \
       --model_path output/iter_1000/model.pdparams

模型导出

使用下述命令将训练好的模型导出为预测部署模型。

text

python export.py \
       --config configs/pp_liteseg/pp_liteseg_stdc2_bdd100k_1024x512.yml \
       --model_path output/iter_1000/model.pdparams \
       --save_dir output/inference_model

使用时在PP-Vehicle中的配置文件./deploy/pipeline/config/infer_cfg_ppvehicle.yml中修改LANE_SEG模块中的model_dir项.

text

LANE_SEG:
  lane_seg_config: deploy/pipeline/config/lane_seg_config.yml  
  model_dir:  output/inference_model

然后可以使用-->至此即完成更新车道线分割模型任务。

Advanced Tutorials Customization Ppvehicle Violation En (`docs/advanced_tutorials/customization/ppvehicle_violation_en.md`)

English | 简体中文

Customized Vehicle Violation

The secondary development of vehicle violation task mainly focuses on the task of lane line segmentation model. PP-LiteSeg model is used to get the lane line data set bdd100k through fine-tune. The process is referred to PP-LiteSeg。

Data preparation

ppvehicle violation analysis divides the lane line into 4 categories

text

0 Background

1 double yellow line

2 Solid line

3 Dashed line

For the bdd100k data set, we can combine the processing script provided by lane_to_mask.py and bdd100k repo to process the data into the data format required for segmentation.

text

# clone bdd100k：
git clone https://github.com/bdd100k/bdd100k.git

# copy lane_to_mask.py to bdd100k/
cp PaddleDetection/deploy/pipeline/tools/lane_to_mask.py bdd100k/

# preparation bdd100k env
cd bdd100k && pip install -r requirements.txt

#bdd100k to mask
python lane_to_mask.py -i dataset/labels/lane/polygons/lane_train.json -o /output_path

# -i means input path for bdd100k dataset label json，
# -o for output patn

Organize data and store data in the following format:

text

dataset_root
    |
    |--images  
    |  |--train
    |       |--image1.jpg
    |       |--image2.jpg
    |       |--...
    |  |--val
    |       |--image3.jpg
    |       |--image4.jpg
    |       |--...
    |  |--test
    |       |--image5.jpg
    |       |--image6.jpg
    |       |--...
    |
    |--labels  
    |  |--train
    |       |--label1.jpg
    |       |--label2.jpg
    |       |--...
    |  |--val
    |       |--label3.jpg
    |       |--label4.jpg
    |       |--...
    |  |--test
    |       |--label5.jpg
    |       |--label6.jpg
    |       |--...
    |

run create_dataset_list.py create txt file

text

python create_dataset_list.py <dataset_root> #dataset path
                              --type  custom #dataset type，support cityscapes、custom

For other data and data annotation, please refer to PaddleSeg Prepare Custom Datasets

model training

clone PaddleSeg：

text

git clone https://github.com/PaddlePaddle/PaddleSeg.git

prepapation env：

text

cd PaddleSeg
pip install -r requirements.txt

Prepare configuration file

For details, please refer to PaddleSeg prepare configuration file.

exp: pp_liteseg_stdc2_bdd100k_1024x512.yml

text

/* Detailed source-code truncated for AI context efficiency. */

training model

text

#Single GPU training
export CUDA_VISIBLE_DEVICES=0 # Linux
# set CUDA_VISIBLE_DEVICES=0  # Windows
python train.py \
       --config configs/pp_liteseg/pp_liteseg_stdc2_bdd100k_1024x512.yml \
       --do_eval \
       --use_vdl \
       --save_interval 500 \
       --save_dir output

Explanation of training parameters

text

--do_eval    Whether to start the evaluation when saving the model. When starting, the best model will be saved to best according to mIoU model
--use_vdl Whether to enable visualdl to record training data
--save_interval 500  Number of steps between model saving
--save_dir output    Model output path

2、Multiple GPUs training

if you want to use multiple gpus training, you need to set the environment variable CUDA_VISIBLE_DEVICES is specified as multiple gpus (if not specified, all gpus will be used by default), and the training script will be started using paddle.distributed.launch (because nccl is not supported under windows, multi-card training cannot be used):

text

export CUDA_VISIBLE_DEVICES=0,1,2,3 # 4 gpus
python -m paddle.distributed.launch train.py \
       --config configs/pp_liteseg/pp_liteseg_stdc2_bdd100k_1024x512.yml \
       --do_eval \
       --use_vdl \
       --save_interval 500 \
       --save_dir output

After training, you can execute the following commands for performance evaluation:

text

python val.py \
       --config configs/pp_liteseg/pp_liteseg_stdc2_bdd100k_1024x512.yml \
       --model_path output/iter_1000/model.pdparams

Model export

Use the following command to export the trained model as a prediction deployment model.

text

python export.py \
       --config configs/pp_liteseg/pp_liteseg_stdc2_bdd100k_1024x512.yml \
       --model_path output/iter_1000/model.pdparams \
       --save_dir output/inference_model

Profile in PP-Vehicle when used ./deploy/pipeline/config/infer_cfg_ppvehicle.yml set model_dir in LANE_SEG.

text

LANE_SEG:
  lane_seg_config: deploy/pipeline/config/lane_seg_config.yml  
  model_dir:  output/inference_model

Then you can use -->to finish the task of updating the lane line segmentation model.

Advanced Tutorials Openvino Inference README Cn (`docs/advanced_tutorials/openvino_inference/README_cn.md`)

将FairMOT模型为ONNX格式,并用OpenVINO做推理

简介

PaddleDetection是一个充满活力的开源项目，拥有大量的贡献者和维护者。 PaddleDetection是PaddlePaddle下面一个人工智能框物体检测工具集，能够帮助开发人员快速的将人工智能集成到自己的项目和应用程序中。
Intel OpenVINO 是一个广泛使用的免费工具包。它能帮助优化深度学习模型，并使用推理引擎将其部署到英特尔硬件上。
很显然，当我们可以协同上下游（PaddlePaddle, OpenVINO）一起工作,这将可以极大的简化工作流程, 并且帮助我们实现AI模型从开发到部署的流水线工作模式, 这也让我们的生活更轻松。

本文将向您展示如何在 PaddleDetection 中使用 Model Zoo 中的FairMOT模型 FairMOT 并用OpenVINO来实现推理过程。

前提要求

为了专注于介绍如何在OpenVINO中使用飞桨的模型这一主题,本文将不是一片入门级文章,它不会帮助您设置好您的开发环境, 本文只会提供最核心的组件安装, 并且会为每个需要用到的组件提供相应的链接.

在开始之前请确保您已经安装了 PaddlePaddle.

text

conda install paddlepaddle==2.2.2 --channel https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/Paddle/

为了运行演示程序, 您还需要下载已经转换好了的ONNX格式的FairMOT模型.

将FairMOT模型到ONNX格式

下载FairMOT推理模型.
使用Paddle2ONNX来转换FairMOT模型.

请确保您已经安装了Paddle2ONNX.

text

paddle2onnx --model_dir . --model_filename model.pdmodel \
--params_filename model.pdiparams \
--input_shape_dict "{'image': [1, 3, 320, 576], 'scale_factor': [1, 2], 'im_shape': [1, 2]}" \
--save_file fairmot_576_320_v2.onnx \
--opset_version 12 \
--enable_onnx_checker True

更多关于如何使用Paddle2ONNX的详细信息, 请参考: ONNX模型导出.

使用ONNX模型以及OpenVINO进行推理

当我们把Paddle模型转换成ONNX模型之后, 我们可以直接使用OpenVINO读取其模型并且进行推理.

_{请确保您已经安装了OpenVINO, 这里是OpenVINO的安装指南.}

创建一个execution network

所以这里要做的第一件事是获得一个执行网络，以后可以使用它来进行推理。
代码如下:

text

def get_net():
    ie = IECore()
    model_path = root_path / "PaddleDetection/FairMot/fairmot_576_320_v3.onnx"
    net = ie.read_network(model= str(model_path))
    exec_net = ie.load_network(network=net, device_name="CPU")
    return net, exec_net

预处理

每个 AI 模型都有自己不同的预处理步骤，让我们看看 FairMOT 模型是如何做的：

text

def prepare_input():
    transforms = [
        T.Resize(target_size=(target_width, target_height)),
        T.Normalize(mean=(0,0,0), std=(1,1,1))
    ]
    img_file = root_path / "images/street.jpeg"
    img = cv2.imread(str(img_file))
    normalized_img, _ = T.Compose(transforms)(img)
    # add an new axis in front
    img_input = normalized_img[np.newaxis, :]
    # scale_factor is calculated as: im_shape / original_im_shape
    h_scale = target_height / img.shape[0]
    w_scale = target_width / img.shape[1]
    input = {"image": img_input, "im_shape": [target_height, target_width], "scale_factor": [h_scale, w_scale]}
    return input, img

预测

在我们完成了所有的负载网络和预处理之后，终于开始了预测阶段。

text

def predict(exec_net, input):
    result = exec_net.infer(input)
    return result

您可能会惊讶地看到, 最激动人心的步骤居然如此简单。不过下一个阶段会更加复杂。

后处理

相较于大多数其他类型的AI推理, MOT（Multi-Object Tracking）显然是特殊的. FairMOT 需要一个称为跟踪器的特殊对象来处理预测结果。这个预测结果则包括预测检测和预测的行人特征向量。

幸运的是，PaddleDetection 为我们简化了这个过程，我们可以从ppdet导出JDETracker，然后用这个tracker挑选出来符合条件的检测框,而且我们不需要编写太多代码来处理它。

text

def postprocess(pred_dets, pred_embs, threshold = 0.5):
    tracker = JDETracker()
    online_targets_dict = tracker.update(pred_dets, pred_embs)
    online_tlwhs = defaultdict(list)
    online_scores = defaultdict(list)
    online_ids = defaultdict(list)
    for cls_id in range(1):
        online_targets = online_targets_dict[cls_id]
        for t in online_targets:
            tlwh = t.tlwh
            tid = t.track_id
            tscore = t.score
            # make sure the tscore is no less then the threshold.
            if tscore < threshold: continue
            # make sure the target area is not less than the min_box_area.
            if tlwh[2] * tlwh[3] <= tracker.min_box_area:
                continue
            # make sure the vertical ratio of a found target is within the range (1.6 as default ratio).
            if tracker.vertical_ratio > 0 and tlwh[2] / tlwh[3] > tracker.vertical_ratio:
                continue
            online_tlwhs[cls_id].append(tlwh)
            online_ids[cls_id].append(tid)
            online_scores[cls_id].append(tscore)
    online_im = plot_tracking_dict(
        img,
        1,
        online_tlwhs,
        online_ids,
        online_scores,
        frame_id=0)
    return online_im

画出检测框(可选)

这一步是可选的。出于演示目的，我只使用 plot_tracking_dict() 方法在图像上绘制所有边界框。但是，如果您没有相同的要求，则不需要这样做。

text

online_im = plot_tracking_dict(
    img,
    1,
    online_tlwhs,
    online_ids,
    online_scores,
    frame_id=0)

这些就是在您的硬件上运行 FairMOT 所需要遵循的所有步骤。

之后会有一篇详细解释此过程的配套文章将会发布，并且该文章的链接将很快在此处更新。

完整代码请查看 Paddle OpenVINO 预测.

Tutorials FAQ FAQ第一期 (`docs/tutorials/FAQ/FAQ第一期.md`)

FAQ：第一期

**Q：**SOLOv2训练mAP值宽幅震荡，无上升趋势，检测效果不好，检测置信度超过了1的原因是？

A： SOLOv2训练不收敛的话，先更新PaddleDetection到release/2.2或者develop分支尝试。

Q： Optimizer中优化器支持哪几种？

A： Paddle中支持的优化器Optimizer在PaddleDetection中均支持，需要手动修改下配置文件即可。

Q：在tools/infer.py加入如下函数，得到FLOPs值为-1,请问原因？

A：更新PaddleDetection到release/2.2或者develop分支，print_flops设为True即可打印FLOPs。

Q：使用官方的ReID模块时遇到了模块未注册的问题

A：请尝试pip uninstall paddledet并重新安装，或者python setup.py install。

Q：大规模实用目标检测模型有动态图版本吗，或者可以转换为动态图版本吗？

A：大规模实用模型的动态图版本正在整理，我们正在开发更大规模的通用预训练模型，预计在2.3版本中发布。

Q： Develop分支下FairMot预测视频问题：预测视频时不会完全运行完毕。比如用一个300frame的视频，代码会保存预测结果的每一帧图片，但只保存到299张就没了，并且也没有预测好的视频文件生成，该如何解决？

A：已经支持自己设置帧率infer视频，请使用develop分支或release/2.2分支，命令如下：

text

CUDA_VISIBLE_DEVICES=0 python tools/infer_mot.py -c configs/mot/fairmot/fairmot_dla34_30e_1088x608.yml -o weights=https://paddledet.bj.bcebos.com/models/mot/fairmot_dla34_30e_1088x608.pdparams --video_file={your video name}.mp4 --frame_rate=20 --save_videos

Q：使用YOLOv3模型如何通过yml文件修改输入图片尺寸？

A：模型预测部署需要用到指定的尺寸时，首先在训练前需要修改configs/_base_/yolov3_reader.yml中的TrainReader的BatchRandomResize中target_size包含指定的尺寸，训练完成后，在评估或者预测时，需要将EvalReader和TestReader中的Resize的target_size修改成对应的尺寸，如果是需要模型导出(export_model)，则需要将TestReader中的image_shape修改为对应的图片输入尺寸。

Q：以前的模型都是用静态图训练的，现在想用动态图训练，但想加载原来静态图的模型作为预训练模型，可以直接用加载静态图保存的模型断点吗？如不行，有其它方法吗？

A：静态图和动态图模型的权重的key做下映射一一对应转过去是可以的，可以参考这个代码。但是不保证所有静态图的权重的key映射都能对应上，静态图是把背景也训练了，动态图去背景类训的，而且现有动态图模型训出来的一般都比以前静态图更高，资源时间够的情况下建议还是直接训动态图版本。

Q： TTFNet训练过程中hm_loss异常

A：如果是单卡的话学习率需要对应降低8倍。另外ttfnet模型因为自身设置的学习率比较大，可能会出现其他数据集训练出现不稳定的情况。建议pretrain_weights加载官方release出的coco数据集上训练好的模型，然后将学习率再调低一些。

Tutorials FAQ FAQ第零期 (`docs/tutorials/FAQ/FAQ第零期.md`)

FAQ：第零期

Q: 为什么我使用单GPU训练loss会出NaN?
A: 配置文件中原始学习率是适配多GPU训练(8x GPU)，若使用单GPU训练，须对应调整学习率（例如，除以8）。

以faster_rcnn_r50 为例,在静态图下计算规则表如下所示，它们是等价的，表中变化节点即为piecewise decay里的boundaries:

GPU数	batch size/卡	学习率	最大轮数	变化节点
2	1	0.0025	720000	[480000, 640000]
4	1	0.005	360000	[240000, 320000]
8	1	0.01	180000	[120000, 160000]

上述方式适用于静态图下。在动态图中，由于训练以epoch方式计数，因此调整GPU卡数后只需要修改学习率即可，修改方式和静态图相同.

Q: 自定义数据集时，配置文件里的num_classes应该如何设置?
A: 动态图中，自定义数据集时将num_classes统一设置为自定义数据集的类别数即可，静态图中(static目录下)，YOLO系列模型和anchor free系列模型将num_classes设置为自定义数据集类别即可，其他模型如RCNN系列，SSD，RetinaNet，SOLOv2等模型，由于检测原理上分类中需要区分背景框和前景框，设置的num_classes须为自定义数据集类别数+1，即增加一类背景类。

Q: PP-YOLOv2模型训练使用—eval做训练中验证，在第一次做eval的时候hang住,该如何处理?
A: PP-YOLO系列模型如果只加载backbone的预训练权重从头开始训练的话收敛会比较慢，当模型还没有较好收敛的时候做预测时，由于输出的预测框比较混乱，在NMS时做排序和滤除会非常耗时，就好像eval时hang住了一样，这种情况一般发生在使用自定义数据集并且自定义数据集样本数较少导致训练到第一次做eval的时候训练轮数较少，模型还没有较好收敛的情况下，可以通过如下三个方面排查解决。

PaddleDetection中提供的默认配置一般是采用8卡训练的配置，配置文件中的batch_size数为每卡的batch size，若训练的时候不是使用8卡或者对batch_size有修改，需要等比例的调小初始learning_rate来获得较好的收敛效果
如果使用自定义数据集并且样本数比较少，建议增大snapshot_epoch数来增加第一次进行eval的时候的训练轮数来保证模型已经较好收敛
若使用自定义数据集训练，可以加载我们发布的COCO或VOC数据集上训练好的权重进行finetune训练来加快收敛速度，可以使用-o pretrain_weights=xxx的方式指定预训练权重，xxx可以是Model Zoo里发布的模型权重链接

Q: 如何更好的理解reader和自定义修改reader文件

text

/* Detailed source-code truncated for AI context efficiency. */

Q: 用户如何控制类别类别输出？即图中有多类目标只输出其中的某几类

A: 用户可自行在代码中进行修改，增加条件设置。

text

# filter by class_id
keep_class_id = [1, 2]
bbox_res = [e for e in bbox_res if int(e[0]) in keep_class_id]

https://github.com/PaddlePaddle/PaddleDetection/blob/b87a1ea86fa18ce69e44a17ad1b49c1326f19ff9/ppdet/engine/trainer.py#L438

Q: 用户自定义数据集训练，预测结果标签错误

A: 此类情况往往是用户在设置数据集路径时候，并没有关注TestDataset中anno_path的路径问题。需要用户将anno_path设置成自己的路径。

text

TestDataset:
  !ImageFolder
    anno_path: annotations/instances_val2017.json

Q: 如何打印网络FLOPs？

A: 在configs/runtime.yml中设置print_flops: true，同时需要安装PaddleSlim(比如：pip install paddleslim)，即可打印模型的FLOPs。

Q: 如何使用无标注框进行训练？

A: 在configs/dataset/coco.py 或者configs/dataset/voc.py中的TrainDataset下设置allow_empty: true, 此时允许数据集加载无标注框进行训练。该功能支持coco，voc数据格式，RCNN系列和YOLO系列模型验证能够正常训练。另外，如果无标注框数据过多，会影响模型收敛，在TrainDataset下可以设置empty_ratio: 0.1对无标注框数据进行随机采样，控制无标注框的数据量占总数据量的比例，默认值为1.，即使用全部无标注框

Tutorials Config Annotation Faster Rcnn R50 Fpn 1x Coco Annotation (`docs/tutorials/config_annotation/faster_rcnn_r50_fpn_1x_coco_annotation.md`)

RCNN系列模型参数配置教程

标签：模型参数配置

以faster_rcnn_r50_fpn_1x_coco.yml为例，这个模型由五个子配置文件组成：

数据配置文件 coco_detection.yml

yaml

# 数据评估类型
metric: COCO
# 数据集的类别数
num_classes: 80

# TrainDataset
TrainDataset:
  !COCODataSet
    # 图像数据路径，相对 dataset_dir 路径，os.path.join(dataset_dir, image_dir)
    image_dir: train2017
    # 标注文件路径，相对 dataset_dir 路径，os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_train2017.json
    # 数据文件夹
    dataset_dir: dataset/coco
    # data_fields
    data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']

EvalDataset:
  !COCODataSet
    # 图像数据路径，相对 dataset_dir 路径，os.path.join(dataset_dir, image_dir)
    image_dir: val2017
    # 标注文件路径，相对 dataset_dir 路径，os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_val2017.json
    # 数据文件夹
    dataset_dir: dataset/coco

TestDataset:
  !ImageFolder
    # 标注文件路径，相对 dataset_dir 路径，os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_val2017.json

优化器配置文件 optimizer_1x.yml

yaml

# 总训练轮数
epoch: 12

# 学习率设置
LearningRate:
  # 默认为8卡训学习率
  base_lr: 0.01
  # 学习率调整策略
  schedulers:
  - !PiecewiseDecay
    gamma: 0.1
    # 学习率变化位置(轮数)
    milestones: [8, 11]
  - !LinearWarmup
    start_factor: 0.1
    steps: 1000

# 优化器
OptimizerBuilder:
  # 优化器
  optimizer:
    momentum: 0.9
    type: Momentum
  # 正则化
  regularizer:
    factor: 0.0001
    type: L2

数据读取配置文件 faster_fpn_reader.yml

text

/* Detailed source-code truncated for AI context efficiency. */

模型配置文件 faster_rcnn_r50_fpn.yml

text

/* Detailed source-code truncated for AI context efficiency. */

运行时置文件 runtime.yml

yaml

# 是否使用gpu
use_gpu: true
# 日志打印间隔
log_iter: 20
# save_dir
save_dir: output
# 模型保存间隔时间
snapshot_epoch: 1

Tutorials Config Annotation Faster Rcnn R50 Fpn 1x Coco Annotation En (`docs/tutorials/config_annotation/faster_rcnn_r50_fpn_1x_coco_annotation_en.md`)

RCNN series model parameter configuration tutorial

Tag: Model parameter configuration

Take faster_rcnn_r50_fpn_1x_coco.yml as an example. The model consists of five sub-profiles:

Data profile coco_detection.yml

yaml

# Data evaluation type
metric: COCO
# The number of categories in the dataset
num_classes: 80

# TrainDataset
TrainDataset:
  !COCODataSet
    # Image data path, Relative path of dataset_dir, os.path.join(dataset_dir, image_dir)
    image_dir: train2017
    # Annotation file path, Relative path of dataset_dir, os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_train2017.json
    # data file
    dataset_dir: dataset/coco
    # data_fields
    data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']

EvalDataset:
  !COCODataSet
    # Image data path, Relative path of dataset_dir, os.path.join(dataset_dir, image_dir)
    image_dir: val2017
    # Annotation file path, Relative path of dataset_dir, os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_val2017.json
    # data file file os.path.join(dataset_dir, anno_path)
    dataset_dir: dataset/coco

TestDataset:
  !ImageFolder
    # Annotation file path, Relative path of dataset_dir, os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_val2017.json

Optimizer configuration file optimizer_1x.yml

yaml

# Total training epoches
epoch: 12

# learning rate setting
LearningRate:
  # Default is 8 Gpus training learning rate
  base_lr: 0.01
  # Learning rate adjustment strategy
  schedulers:
  - !PiecewiseDecay
    gamma: 0.1
    # Position of change in learning rate (number of epoches)
    milestones: [8, 11]
  - !LinearWarmup
    start_factor: 0.1
    steps: 1000

# Optimizer
OptimizerBuilder:
  # Optimizer
  optimizer:
    momentum: 0.9
    type: Momentum
  # Regularization
  regularizer:
    factor: 0.0001
    type: L2

Data reads configuration files faster_fpn_reader.yml

text

/* Detailed source-code truncated for AI context efficiency. */

Model profile faster_rcnn_r50_fpn.yml

text

/* Detailed source-code truncated for AI context efficiency. */

runtime configuration file runtime.yml

yaml

# Whether to use gpu
use_gpu: true
# Log Printing interval
log_iter: 20
# save_dir
save_dir: output
# Model save interval
snapshot_epoch: 1

Tutorials Config Annotation Multi Scale Test Config (`docs/tutorials/config_annotation/multi_scale_test_config.md`)

Multi Scale Test Configuration

Tags: Configuration

yaml

##################################### Multi scale test configuration #####################################

EvalReader:
  sample_transforms:
  - Decode: {}
  - MultiscaleTestResize: {origin_target_size: [800, 1333], target_size: [700 , 900]}
  - NormalizeImage: {is_scale: true, mean: [0.485,0.456,0.406], std: [0.229, 0.224,0.225]}
  - Permute: {}

TestReader:
  sample_transforms:
  - Decode: {}
  - MultiscaleTestResize: {origin_target_size: [800, 1333], target_size: [700 , 900]}
  - NormalizeImage: {is_scale: true, mean: [0.485,0.456,0.406], std: [0.229, 0.224,0.225]}
  - Permute: {}

Multi Scale Test is a TTA (Test Time Augmentation) method, it can improve object detection performance.

The input image will be scaled into different scales, then model generated predictions (bboxes) at different scales, finally all the predictions will be combined to generate final prediction. (Here NMS is used to aggregate the predictions.)

_MultiscaleTestResize_ option

MultiscaleTestResize option is used to enable multi scale test prediction.

origin_target_size: [800, 1333] means the input image will be scaled to 800 (for short edge) and 1333 (max edge length cannot be greater than 1333) at first

target_size: [700 , 900] property is used to specify different scales.

It can be plugged into evaluation process or test (inference) process, by adding MultiscaleTestResize entry to EvalReader.sample_transforms or TestReader.sample_transforms

###Note

Now only CascadeRCNN, FasterRCNN and MaskRCNN are supported for multi scale testing. And batch size must be 1.

Tutorials Config Annotation Multi Scale Test Config Cn (`docs/tutorials/config_annotation/multi_scale_test_config_cn.md`)

多尺度测试的配置

标签: 配置

yaml

##################################### 多尺度测试的配置 #####################################

EvalReader:
  sample_transforms:
  - Decode: {}
  - MultiscaleTestResize: {origin_target_size: [800, 1333], target_size: [700 , 900]}
  - NormalizeImage: {is_scale: true, mean: [0.485,0.456,0.406], std: [0.229, 0.224,0.225]}
  - Permute: {}

TestReader:
  sample_transforms:
  - Decode: {}
  - MultiscaleTestResize: {origin_target_size: [800, 1333], target_size: [700 , 900]}
  - NormalizeImage: {is_scale: true, mean: [0.485,0.456,0.406], std: [0.229, 0.224,0.225]}
  - Permute: {}

多尺度测试是一种TTA方法（测试时增强），可以用于提高目标检测的准确率

输入图像首先被缩放为不同尺度的图像，然后模型对这些不同尺度的图像进行预测，最后将这些不同尺度上的预测结果整合为最终预测结果。（这里使用了NMS来整合不同尺度的预测结果）

_MultiscaleTestResize_ 选项

MultiscaleTestResize 选项用于开启多尺度测试.

origin_target_size: [800, 1333] 项代表输入图像首先缩放为短边为800，最长边不超过1333.

target_size: [700 , 900] 项设置不同的预测尺度。

通过在EvalReader.sample_transforms或TestReader.sample_transforms中设置MultiscaleTestResize项，可以在评估过程或预测过程中开启多尺度测试。

###注意

目前多尺度测试只支持CascadeRCNN, FasterRCNN and MaskRCNN网络, 并且batch size需要是1.

Tutorials Config Annotation Ppyolo R50vd Dcn 1x Coco Annotation (`docs/tutorials/config_annotation/ppyolo_r50vd_dcn_1x_coco_annotation.md`)

YOLO系列模型参数配置教程

标签：模型参数配置

以ppyolo_r50vd_dcn_1x_coco.yml为例，这个模型由五个子配置文件组成：

数据配置文件 coco_detection.yml

yaml

# 数据评估类型
metric: COCO
# 数据集的类别数
num_classes: 80

# TrainDataset
TrainDataset:
  !COCODataSet
    # 图像数据路径，相对 dataset_dir 路径，os.path.join(dataset_dir, image_dir)
    image_dir: train2017
    # 标注文件路径，相对 dataset_dir 路径，os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_train2017.json
    # 数据文件夹
    dataset_dir: dataset/coco
    # data_fields
    data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']

EvalDataset:
  !COCODataSet
    # 图像数据路径，相对 dataset_dir 路径，os.path.join(dataset_dir, image_dir)
    image_dir: val2017
    # 标注文件路径，相对 dataset_dir 路径，os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_val2017.json
    # 数据文件夹，os.path.join(dataset_dir, anno_path)
    dataset_dir: dataset/coco

TestDataset:
  !ImageFolder
    # 标注文件路径，相对 dataset_dir 路径
    anno_path: annotations/instances_val2017.json

优化器配置文件 optimizer_1x.yml

yaml

# 总训练轮数
epoch: 405

# 学习率设置
LearningRate:
  # 默认为8卡训学习率
  base_lr: 0.01
  # 学习率调整策略
  schedulers:
  - !PiecewiseDecay
    gamma: 0.1
    # 学习率变化位置(轮数)
    milestones:
    - 243
    - 324
  # Warmup
  - !LinearWarmup
    start_factor: 0.
    steps: 4000

# 优化器
OptimizerBuilder:
  # 优化器
  optimizer:
    momentum: 0.9
    type: Momentum
  # 正则化
  regularizer:
    factor: 0.0005
    type: L2

数据读取配置文件 ppyolo_reader.yml

text

/* Detailed source-code truncated for AI context efficiency. */

模型配置文件 ppyolo_r50vd_dcn.yml

text

/* Detailed source-code truncated for AI context efficiency. */

运行时置文件 runtime.yml

yaml

# 是否使用gpu
use_gpu: true
# 日志打印间隔
log_iter: 20
# save_dir
save_dir: output
# 模型保存间隔时间
snapshot_epoch: 1

Tutorials Config Annotation Ppyolo R50vd Dcn 1x Coco Annotation En (`docs/tutorials/config_annotation/ppyolo_r50vd_dcn_1x_coco_annotation_en.md`)

YOLO series model parameter configuration tutorial

Tag: Model parameter configuration

Take ppyolo_r50vd_dcn_1x_coco.yml as an example, The model consists of five sub-profiles:

Data profile coco_detection.yml

yaml

# Data evaluation type
metric: COCO
# The number of categories in the dataset
num_classes: 80

# TrainDataset
TrainDataset:
  !COCODataSet
    # Image data path, Relative path of dataset_dir, os.path.join(dataset_dir, image_dir)
    image_dir: train2017
    # Annotation file path, Relative path of dataset_dir, os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_train2017.json
    # data file
    dataset_dir: dataset/coco
    # data_fields
    data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']

EvalDataset:
  !COCODataSet
    # Image data path, Relative path of dataset_dir, os.path.join(dataset_dir, image_dir)
    image_dir: val2017
    # Annotation file path, Relative path of dataset_dir, os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_val2017.json
    # data file os.path.join(dataset_dir, anno_path)
    dataset_dir: dataset/coco

TestDataset:
  !ImageFolder
    # Annotation file path, Relative path of dataset_dir, os.path.join(dataset_dir, anno_path)
    anno_path: annotations/instances_val2017.json

Optimizer configuration file optimizer_1x.yml

yaml

# Total training epoches
epoch: 405

# learning rate setting
LearningRate:
  # Default is 8 Gpus training learning rate
  base_lr: 0.01
  # Learning rate adjustment strategy
  schedulers:
  - !PiecewiseDecay
    gamma: 0.1
    # Position of change in learning rate (number of epoches)
    milestones:
    - 243
    - 324
  # Warmup
  - !LinearWarmup
    start_factor: 0.
    steps: 4000

# Optimizer
OptimizerBuilder:
  # Optimizer
  optimizer:
    momentum: 0.9
    type: Momentum
  # Regularization
  regularizer:
    factor: 0.0005
    type: L2

Data reads configuration files ppyolo_reader.yml

text

/* Detailed source-code truncated for AI context efficiency. */

Model profile ppyolo_r50vd_dcn.yml

text

/* Detailed source-code truncated for AI context efficiency. */

Runtime file runtime.yml

yaml

# Whether to use gpu
use_gpu: true
# Log Printing interval
log_iter: 20
# save_dir
save_dir: output
# Model save interval
snapshot_epoch: 1

Tutorials Data DetAnnoTools (`docs/tutorials/data/DetAnnoTools.md`)

简体中文 | English

目标检测标注工具

[LabelMe](https://github.com/wkentaro/labelme)

使用说明

LabelMe安装

具体安装操作请参考LabelMe官方教程中的Installation

** Ubuntu**

text

sudo apt-get install labelme

# or
sudo pip3 install labelme

# or install standalone executable from:
# https://github.com/wkentaro/labelme/releases

** macOS**

text

brew install pyqt  # maybe pyqt5
pip install labelme

# or
brew install wkentaro/labelme/labelme  # command line interface
# brew install --cask wkentaro/labelme/labelme  # app

# or install standalone executable/app from:
# https://github.com/wkentaro/labelme/releases

推荐使用Anaconda的安装方式

text

conda create –name=labelme python=3
conda activate labelme
pip install pyqt5
pip install labelme

LabelMe图片标注过程

启动labelme后，选择图片文件或者图片所在文件夹

左侧编辑栏选择create polygons 绘制标注区域如下图所示（右击图像区域可以选择不同的标注形状），绘制好区域后按下回车，弹出新的框填入标注区域对应的标签，如：people

左侧菜单栏点击保存，生成json形式的标注文件

LabelMe标注格式

LabelMe导出数据格式

text

#生成标注文件
png/jpeg/jpg-->labelme标注-->json

格式转化总结

text

#标注文件转化为VOC数据集格式
json-->labelme2voc.py-->VOC数据集

#标注文件转化为COCO数据集格式
json-->labelme2coco.py-->COCO数据集

标注文件(json)-->VOC数据集

使用官方给出的labelme2voc.py这份脚本

下载该脚本，在命令行中使用

python labelme2voc.py data_annotated(标注文件所在文件夹) data_dataset_voc(输出文件夹) --labels labels.txt

运行后，在指定的输出文件夹中会如下的目录

text

# It generates:
#   - data_dataset_voc/JPEGImages
#   - data_dataset_voc/Annotations
#   - data_dataset_voc/AnnotationsVisualization

标注文件(json)-->COCO数据集

使用PaddleDetection提供的x2coco.py 将labelme标注的数据转换为COCO数据集形式

bash

python tools/x2coco.py \
                --dataset_type labelme \
                --json_input_dir ./labelme_annos/ \
                --image_input_dir ./labelme_imgs/ \
                --output_dir ./cocome/ \
                --train_proportion 0.8 \
                --val_proportion 0.2 \
                --test_proportion 0.0

用户数据集转成COCO数据后目录结构如下（注意数据集中路径名、文件名尽量不要使用中文，避免中文编码问题导致出错）：

text

dataset/xxx/
├── annotations
│   ├── train.json  # coco数据的标注文件
│   ├── valid.json  # coco数据的标注文件
├── images
│   ├── xxx1.jpg
│   ├── xxx2.jpg
│   ├── xxx3.jpg
│   |   ...
...

[LabelImg](https://github.com/tzutalin/labelImg)

使用说明

LabelImg安装

安装操作请参考LabelImg官方教程

** Ubuntu**

text

sudo apt-get install pyqt5-dev-tools
sudo pip3 install -r requirements/requirements-linux-python3.txt
make qt5py3
python3 labelImg.py
python3 labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]

macOS

text

brew install qt  # Install qt-5.x.x by Homebrew
brew install libxml2

or using pip

pip3 install pyqt5 lxml # Install qt and lxml by pip

make qt5py3
python3 labelImg.py
python3 labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]

推荐使用Anaconda的安装方式

首先下载并进入 labelImg 的目录

text

conda install pyqt=5
conda install -c anaconda lxml
pyrcc5 -o libs/resources.py resources.qrc
python labelImg.py
python labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]

安装注意事项

以Anaconda安装方式为例，比Labelme配置要麻烦一些

启动方式是通过python运行脚本python labelImg.py <图片路径>

LabelImg图片标注过程

启动labelImg后，选择图片文件或者图片所在文件夹

左侧编辑栏选择创建区块 绘制标注区，在弹出新的框选择对应的标签

左侧菜单栏点击保存，可以选择VOC/YOLO/CreateML三种类型的标注文件

LabelImg标注格式

LabelImg导出数据格式

text

#生成标注文件
png/jpeg/jpg-->labelImg标注-->xml/txt/json

格式转换注意事项

PaddleDetection支持VOC或COCO格式的数据，经LabelImg标注导出后的标注文件，需要修改为VOC或COCO格式，调整说明可以参考准备训练数据

Tutorials Data DetAnnoTools En (`docs/tutorials/data/DetAnnoTools_en.md`)

简体中文 | English

Object Detection Annotation Tools

Concents

LabelMe

Instruction
- Installation
- Annotation of Images
Annotation Format

LabelImg

Instruction
Annotation Format
- Export Format
- Notes of Format Conversion

[LabelMe](https://github.com/wkentaro/labelme)

Instruction of LabelMe

Installation

Please refer to The github of LabelMe for installation details.

** Ubuntu**

text

sudo apt-get install labelme

# or
sudo pip3 install labelme

# or install standalone executable from:
# https://github.com/wkentaro/labelme/releases

** macOS**

text

brew install pyqt  # maybe pyqt5
pip install labelme

# or
brew install wkentaro/labelme/labelme  # command line interface
# brew install --cask wkentaro/labelme/labelme  # app

# or install standalone executable/app from:
# https://github.com/wkentaro/labelme/releases

We recommend installing by Anoncanda.

text

conda create –name=labelme python=3
conda activate labelme
pip install pyqt5
pip install labelme

Annotation of Images in LabelMe

After starting labelme, select an image or an folder with images.

Select create polygons in the formula bar. Draw an annotation area as shown in the following GIF. You can right-click on the image to select different shape. When finished, press the Enter/Return key, then fill the corresponding label in the popup box, such as, people.

Click the save button in the formula bar，it will generate an annotation file in json.

Annotation Format of LabelMe

Export Format of LabelMe

text

#generate an annotation file
png/jpeg/jpg-->labelme-->json

Summary of Format Conversion

text

#convert annotation file to VOC dataset format
json-->labelme2voc.py-->VOC dataset

#convert annotation file to COCO dataset format
json-->labelme2coco.py-->COCO dataset

Annotation file(json)—>VOC Dataset

Use this script labelme2voc.py in command line.

python labelme2voc.py data_annotated(annotation folder) data_dataset_voc(output folder) --labels labels.txt

Then, it will generate following contents:

text

# It generates:
#   - data_dataset_voc/JPEGImages
#   - data_dataset_voc/Annotations
#   - data_dataset_voc/AnnotationsVisualization

Annotation file(json)—>COCO Dataset

Convert the data annotated by LabelMe to COCO dataset by the script x2coco.py provided by PaddleDetection.

bash

python tools/x2coco.py \
                --dataset_type labelme \
                --json_input_dir ./labelme_annos/ \
                --image_input_dir ./labelme_imgs/ \
                --output_dir ./cocome/ \
                --train_proportion 0.8 \
                --val_proportion 0.2 \
                --test_proportion 0.0

After the user dataset is converted to COCO data, the directory structure is as follows (Try to avoid use Chinese for the path name in case of errors caused by Chinese coding problems):

text

dataset/xxx/
├── annotations
│   ├── train.json  # Annotation file of coco data
│   ├── valid.json  # Annotation file of coco data
├── images
│   ├── xxx1.jpg
│   ├── xxx2.jpg
│   ├── xxx3.jpg
│   |   ...
...

[LabelImg](https://github.com/tzutalin/labelImg)

Instruction

Installation of LabelImg

Please refer to The github of LabelImg for installation details.

** Ubuntu**

text

sudo apt-get install pyqt5-dev-tools
sudo pip3 install -r requirements/requirements-linux-python3.txt
make qt5py3
python3 labelImg.py
python3 labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]

macOS

text

brew install qt  # Install qt-5.x.x by Homebrew
brew install libxml2

or using pip

pip3 install pyqt5 lxml # Install qt and lxml by pip

make qt5py3
python3 labelImg.py
python3 labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]

We recommend installing by Anoncanda.

Download and go to the folder of labelImg

text

conda install pyqt=5
conda install -c anaconda lxml
pyrcc5 -o libs/resources.py resources.qrc
python labelImg.py
python labelImg.py [IMAGE_PATH] [PRE-DEFINED CLASS FILE]

Installation Notes

Use python scripts to startup LabelImg: python labelImg.py

Annotation of images in LabelImg

After the startup of LabelImg, select an image or a folder with images.

Select Create RectBox in the formula bar. Draw an annotation area as shown in the following GIF. When finished, select corresponding label in the popup box. Then save the annotated file in three forms: VOC/YOLO/CreateML.

Annotation Format of LabelImg

Export Format of LabelImg

text

#generate annotation files
png/jpeg/jpg-->labelImg-->xml/txt/json

Notes of Format Conversion

PaddleDetection supports the format of VOC or COCO. The annotation file generated by LabelImg needs to be converted by VOC or COCO. You can refer to PrepareDataSet.

Tutorials Data KeyPointAnnoTools (`docs/tutorials/data/KeyPointAnnoTools.md`)

简体中文 | English

关键点检测标注工具

[LabelMe](https://github.com/wkentaro/labelme)

使用说明

安装

具体安装操作请参考LabelMe官方教程中的Installation

** Ubuntu**

text

sudo apt-get install labelme

# or
sudo pip3 install labelme

# or install standalone executable from:
# https://github.com/wkentaro/labelme/releases

** macOS**

text

brew install pyqt  # maybe pyqt5
pip install labelme

# or
brew install wkentaro/labelme/labelme  # command line interface
# brew install --cask wkentaro/labelme/labelme  # app

# or install standalone executable/app from:
# https://github.com/wkentaro/labelme/releases

推荐使用Anaconda的安装方式

text

conda create –name=labelme python=3
conda activate labelme
pip install pyqt5
pip install labelme

关键点数据说明

以COCO数据集为例，共需采集17个关键点

text

keypoint indexes:
        0: 'nose',
        1: 'left_eye',
        2: 'right_eye',
        3: 'left_ear',
        4: 'right_ear',
        5: 'left_shoulder',
        6: 'right_shoulder',
        7: 'left_elbow',
        8: 'right_elbow',
        9: 'left_wrist',
        10: 'right_wrist',
        11: 'left_hip',
        12: 'right_hip',
        13: 'left_knee',
        14: 'right_knee',
        15: 'left_ankle',
        16: 'right_ankle'

图片标注过程

启动labelme后，选择图片文件或者图片所在文件夹

左侧编辑栏选择create polygons ，右击图像区域选择标注形状，绘制好关键点后按下回车，弹出新的框填入标注关键点对应的标签

左侧菜单栏点击保存，生成json形式的标注文件

标注格式

导出数据格式

text

#生成标注文件
png/jpeg/jpg-->labelme标注-->json

格式转化总结

text

#标注文件转化为COCO数据集格式
json-->labelme2coco.py-->COCO数据集

标注文件(json)-->COCO数据集

使用PaddleDetection提供的x2coco.py 将labelme标注的数据转换为COCO数据集形式

bash

python tools/x2coco.py \
                --dataset_type labelme \
                --json_input_dir ./labelme_annos/ \
                --image_input_dir ./labelme_imgs/ \
                --output_dir ./cocome/ \
                --train_proportion 0.8 \
                --val_proportion 0.2 \
                --test_proportion 0.0

用户数据集转成COCO数据后目录结构如下（注意数据集中路径名、文件名尽量不要使用中文，避免中文编码问题导致出错）：

text

dataset/xxx/
├── annotations
│   ├── train.json  # coco数据的标注文件
│   ├── valid.json  # coco数据的标注文件
├── images
│   ├── xxx1.jpg
│   ├── xxx2.jpg
│   ├── xxx3.jpg
│   |   ...
...

Tutorials Data KeyPointAnnoTools En (`docs/tutorials/data/KeyPointAnnoTools_en.md`)

简体中文 | English

Key Points Detection Annotation Tool

Concents

LabelMe

Instruction
Annotation Format

[LabelMe](https://github.com/wkentaro/labelme)

Instruction

Installation

Please refer to The github of LabelMe for installation details.

** Ubuntu**

text

sudo apt-get install labelme

# or
sudo pip3 install labelme

# or install standalone executable from:
# https://github.com/wkentaro/labelme/releases

** macOS**

text

brew install pyqt  # maybe pyqt5
pip install labelme

# or
brew install wkentaro/labelme/labelme  # command line interface
# brew install --cask wkentaro/labelme/labelme  # app

# or install standalone executable/app from:
# https://github.com/wkentaro/labelme/releases

We recommend installing by Anoncanda.

text

conda create –name=labelme python=3
conda activate labelme
pip install pyqt5
pip install labelme

Notes of Key Points Data

COCO dataset needs to collect 17 key points.

text

keypoint indexes:
        0: 'nose',
        1: 'left_eye',
        2: 'right_eye',
        3: 'left_ear',
        4: 'right_ear',
        5: 'left_shoulder',
        6: 'right_shoulder',
        7: 'left_elbow',
        8: 'right_elbow',
        9: 'left_wrist',
        10: 'right_wrist',
        11: 'left_hip',
        12: 'right_hip',
        13: 'left_knee',
        14: 'right_knee',
        15: 'left_ankle',
        16: 'right_ankle'

Annotation of LabelMe

After starting labelme, select an image or an folder with images.

Select create polygons in the formula bar. Draw an annotation area as shown in the following GIF. You can right-click on the i

Technical Documentation: PaddlePaddle/PaddleDetection

Contribution README (`docs/contribution/README.md`) Copy Section

Contributing to PaddleDetection

贡献指南 Copy Chapter

开发者 Copy Chapter

Paddlex Overview (`docs/paddlex/overview.md`) Copy Section

目录 Copy Chapter

1. 低代码全流程开发简介 Copy Chapter

2. 目标检测相关能力支持 Copy Chapter

3. 目标检测相关模型产线列表和教程 Copy Chapter

4. 目标检测相关单功能模块列表和教程 Copy Chapter

Advanced Tutorials Openvino Inference README (`docs/advanced_tutorials/openvino_inference/README.md`) Copy Section

Using OpenVINO for Inference

Introduction Copy Chapter

Prerequisites Copy Chapter

Export the PaddleDetection Model to ONNX format Copy Chapter

Use the ONNX model for inference Copy Chapter

Get the execution network Copy Section

Preprocessing Copy Section

Prediction Copy Section

Post-processing Copy Section

Plot the detections (Optional) Copy Section

Tutorials FAQ README (`docs/tutorials/FAQ/README.md`) Copy Section

FAQ/常见问题

Tutorials Data README (`docs/tutorials/data/README.md`) Copy Section

数据准备

数据采集 Copy Chapter

确定方向 Copy Section

开源数据集 Copy Section

增加场景数据 Copy Section

类别均衡 Copy Section

数据标注及格式说明 Copy Chapter

Advanced Tutorials Customization Action Recognotion README (`docs/advanced_tutorials/customization/action_recognotion/README.md`) Copy Section

行为识别任务二次开发

方案选择 Copy Chapter

吸烟 Copy Section

打电话 Copy Section

摔倒 Copy Section

闯入 Copy Section

打架 Copy Section

CHANGELOG (`docs/CHANGELOG.md`) Copy Section

版本更新信息

最新版本信息 Copy Chapter

2.6(02.15/2023) Copy Section

2.5(08.26/2022) Copy Section

2.4(03.24/2022) Copy Section

2.3(11.03/2021) Copy Section

2.2(08.10/2021) Copy Section

2.1(05.20/2021) Copy Section

2.0(04.15/2021) Copy Section

历史版本信息 Copy Chapter

2.0-rc(02.23/2021) Copy Section

v2.0-beta(12.20/2020) Copy Section

v0.5.0(11/2020) Copy Section

v0.4.0(07/2020) Copy Section

v0.3.0(05/2020) Copy Section

v0.2.0(02/2020) Copy Section

12/2019 Copy Section

21/11/2019 Copy Section

10/2019 Copy Section

2/9/2019 Copy Section

5/8/2019 Copy Section

29/7/2019 Copy Section

3/7/2019 Copy Section

CHANGELOG En (`docs/CHANGELOG_en.md`) Copy Section

Version Update Information

Last Version Information Copy Chapter

2.6(02.15/2023) Copy Section

2.5(08.26/2022) Copy Section

2.4(03.24/2022) Copy Section

2.3(11.03/2021) Copy Section

2.2(08.10/2021) Copy Section

2.1(05.20/2021) Copy Section

2.0(04.15/2021) Copy Section

Historical Version Information Copy Chapter

2.0-rc(02.23/2021) Copy Section

v2.0-beta(12.20/2020) Copy Section

v0.5.0(11/2020) Copy Section

v0.4.0(07/2020) Copy Section

v0.3.0(05/2020) Copy Section

Contribution README (`docs/contribution/README.md`)

贡献指南

开发者

Paddlex Overview (`docs/paddlex/overview.md`)

目录

1. 低代码全流程开发简介

2. 目标检测相关能力支持

3. 目标检测相关模型产线列表和教程

4. 目标检测相关单功能模块列表和教程

Advanced Tutorials Openvino Inference README (`docs/advanced_tutorials/openvino_inference/README.md`)

Introduction

Prerequisites

Export the PaddleDetection Model to ONNX format

Use the ONNX model for inference

Get the execution network

Preprocessing

Prediction

Post-processing

Plot the detections (Optional)

Tutorials FAQ README (`docs/tutorials/FAQ/README.md`)

Tutorials Data README (`docs/tutorials/data/README.md`)

数据采集

确定方向

开源数据集

增加场景数据

类别均衡

数据标注及格式说明

Advanced Tutorials Customization Action Recognotion README (`docs/advanced_tutorials/customization/action_recognotion/README.md`)

方案选择

吸烟

打电话

摔倒

闯入

打架

CHANGELOG (`docs/CHANGELOG.md`)

最新版本信息

2.6(02.15/2023)

2.5(08.26/2022)

2.4(03.24/2022)

2.3(11.03/2021)

2.2(08.10/2021)

2.1(05.20/2021)

2.0(04.15/2021)

历史版本信息

2.0-rc(02.23/2021)

v2.0-beta(12.20/2020)

v0.5.0(11/2020)

v0.4.0(07/2020)

v0.3.0(05/2020)

v0.2.0(02/2020)

12/2019

21/11/2019

10/2019

2/9/2019

5/8/2019

29/7/2019

3/7/2019

CHANGELOG En (`docs/CHANGELOG_en.md`)

Last Version Information

2.6(02.15/2023)

2.5(08.26/2022)

2.4(03.24/2022)

2.3(11.03/2021)

2.2(08.10/2021)

2.1(05.20/2021)

2.0(04.15/2021)

Historical Version Information

2.0-rc(02.23/2021)

v2.0-beta(12.20/2020)

v0.5.0(11/2020)

v0.4.0(07/2020)

v0.3.0(05/2020)

v0.2.0(02/2020)

12/2019

21/11/2019

10/2019

2/9/2019

5/8/2019

29/7/2019

3/7/2019