Models · Object detection

RT-DETRv2 (ResNet-vd, S / M / M* / L / X)

RT-DETRv2 (S, M, M*, L, X; ResNet-vd, COCO) from the RT-DETR authors: real-time end-to-end detector, 300 queries and deformable attention, no NMS. Commercially usable.

Commercial use: yes.

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
rtdetrv2-s-cocosynthetic (800x1344)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
14.1
16.8
8.04
4.19
4.22
1.81
196 / 226
192 / 220
134 / 166
155 / 250
152 / 182
98 (engine)
rtdetrv2-m-cocosynthetic (800x1344)original (fp32)
original
portable
kernels
17.1
20.9
10.4
5.27
239 / 254
254 / 270
156 / 170
177 / 274
rtdetrv2-ms-cocosynthetic (800x1344)original (fp32)
original
portable
kernels
24.7
22.5
9.60
5.46
322 / 374
309 / 358
166 / 182
189 / 306
rtdetrv2-l-cocosynthetic (800x1344)original (fp32)
original
portable
kernels
30.3
38.4
12.1
6.61
366 / 424
354 / 412
177 / 200
200 / 306
rtdetrv2-x-cocosynthetic (800x1344)original (fp32)
original
portable
kernels
36.3
46.8
14.6
9.24
529 / 596
589 / 622
244 / 278
262 / 384
Details
  • Original code: https://github.com/lyuwenyu/RT-DETR at df2a96c8c3a0
  • Code licence: Apache-2.0 (permissive); files: LICENSE
  • Weights licence: Apache-2.0 (assumed from code)
  • Licence evidence: https://github.com/lyuwenyu/RT-DETR/blob/df2a96c8c3a098567a1149bf0248da6efe6d495d/LICENSE
  • Training data: COCO 2017 (train, instance annotations), ImageNet-1k (ILSVRC 2012 classification) (COCO 2017 train (72 to 84 epochs); the ResNet-vd backbones start from ImageNet-1k (PaddleClas) pretraining)
  • Options: score_threshold = 0.05 (0.0 to 1.0), max_detections = 100 (>= 1), cuda_graphs = True (speed only)
  • Kernel libraries: cudnn_hacc
  • Variants: rtdetrv2-s-coco, rtdetrv2-m-coco, rtdetrv2-ms-coco, rtdetrv2-l-coco, rtdetrv2-x-coco
  • Outputs: boxes, scores, labels; label_set: coco-80
  • Input size: any H x W >= 32, padded to a multiple of 32
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): rtdetrv2-s-coco 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); rtdetrv2-m-coco 3000x4000 (0.5 GiB peak allocated, 0.7 reserved); rtdetrv2-ms-coco 3000x4000 (0.5 GiB peak allocated, 0.7 reserved); rtdetrv2-l-coco 3000x4000 (0.6 GiB peak allocated, 0.7 reserved); rtdetrv2-x-coco 3000x4000 (0.6 GiB peak allocated, 0.8 reserved)
  • rtdetrv2-s-coco 800x1344, first call: portable + torch.compile 37 s (cold Inductor cache), TensorRT fp16 42 s (ONNX + engine).
  • rtdetrv2-s-coco original (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-s-coco original 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-m-coco original (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-m-coco original 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-ms-coco original (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-ms-coco original 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-l-coco original (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-l-coco original 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-x-coco original (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-x-coco original 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)
  • rtdetrv2-s-coco load / first call: original 1.32 s / 0.417 s, portable 0.31 s / 0.097 s, kernels 0.46† s / 0.534 s
  • rtdetrv2-m-coco load / first call: original 0.45 s / 0.023 s, portable 0.33 s / 0.011 s, kernels 0.16† s / 0.012 s
  • rtdetrv2-ms-coco load / first call: original 0.52 s / 0.057 s, portable 0.45 s / 0.03 s, kernels 0.17† s / 0.092 s
  • rtdetrv2-l-coco load / first call: original 0.66 s / 0.043 s, portable 0.5 s / 0.015 s, kernels 0.25† s / 0.016 s
  • rtdetrv2-x-coco load / first call: original 1.13 s / 0.076 s, portable 1.09 s / 0.048 s, kernels 0.29† s / 0.021 s
  • † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
  • Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls

Measured

pycocotools COCOeval of the top 100 detections per image. Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portablekernelsnote
rtdetrv2-s-cocoCOCO val2017, every 10th image, box mAP49.2949.2649.2949.31
rtdetrv2-m-cocoCOCO val2017, every 10th image, box mAP49.8349.8149.8449.79
rtdetrv2-ms-cocoCOCO val2017, every 10th image, box mAP51.9751.9851.9551.96
rtdetrv2-l-cocoCOCO val2017, every 10th image, box mAP54.5254.5754.5154.49
rtdetrv2-x-cocoCOCO val2017, every 10th image, box mAP55.5155.5455.5155.49

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 939adfe8, 2026-11-06.

Parity

CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.2 and p99 <= 1.0 (absolute error of the output boxes).

variantkernels error vs original, mean / p99GPU, torchkernels used
rtdetrv2-s-coco0.03739 / 0.1098RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400
rtdetrv2-m-coco0.06517 / 0.13434RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400
rtdetrv2-ms-coco0.06765 / 0.17406RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400
rtdetrv2-l-coco0.04387 / 0.1019RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400
rtdetrv2-x-coco0.05571 / 0.2201RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400