RT-DETRv2 (ResNet-vd, S / M / M* / L / X)
RT-DETRv2 (S, M, M*, L, X; ResNet-vd, COCO) from the RT-DETR authors: real-time end-to-end detector, 300 queries and deformable attention, no NMS. Commercially usable.
Commercial use: yes.
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
rtdetrv2-s-coco | synthetic (800x1344) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 14.1 16.8 8.04 4.19 4.22 1.81 | 196 / 226 192 / 220 134 / 166 155 / 250 152 / 182 98 (engine) |
rtdetrv2-m-coco | synthetic (800x1344) | original (fp32) original portable kernels | 17.1 20.9 10.4 5.27 | 239 / 254 254 / 270 156 / 170 177 / 274 |
rtdetrv2-ms-coco | synthetic (800x1344) | original (fp32) original portable kernels | 24.7 22.5 9.60 5.46 | 322 / 374 309 / 358 166 / 182 189 / 306 |
rtdetrv2-l-coco | synthetic (800x1344) | original (fp32) original portable kernels | 30.3 38.4 12.1 6.61 | 366 / 424 354 / 412 177 / 200 200 / 306 |
rtdetrv2-x-coco | synthetic (800x1344) | original (fp32) original portable kernels | 36.3 46.8 14.6 9.24 | 529 / 596 589 / 622 244 / 278 262 / 384 |
Details
- Original code: https://github.com/lyuwenyu/RT-DETR at
df2a96c8c3a0 - Code licence: Apache-2.0 (permissive); files:
LICENSE - Weights licence: Apache-2.0 (assumed from code)
- Licence evidence: https://github.com/lyuwenyu/RT-DETR/blob/df2a96c8c3a098567a1149bf0248da6efe6d495d/LICENSE
- Training data: COCO 2017 (train, instance annotations), ImageNet-1k (ILSVRC 2012 classification) (COCO 2017 train (72 to 84 epochs); the ResNet-vd backbones start from ImageNet-1k (PaddleClas) pretraining)
- Options:
score_threshold= 0.05 (0.0 to 1.0),max_detections= 100 (>= 1),cuda_graphs= True (speed only) - Kernel libraries:
cudnn_hacc - Variants:
rtdetrv2-s-coco,rtdetrv2-m-coco,rtdetrv2-ms-coco,rtdetrv2-l-coco,rtdetrv2-x-coco - Outputs: boxes, scores, labels; label_set:
coco-80 - Input size: any H x W >= 32, padded to a multiple of 32
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
rtdetrv2-s-coco3000x4000 (0.5 GiB peak allocated, 0.6 reserved);rtdetrv2-m-coco3000x4000 (0.5 GiB peak allocated, 0.7 reserved);rtdetrv2-ms-coco3000x4000 (0.5 GiB peak allocated, 0.7 reserved);rtdetrv2-l-coco3000x4000 (0.6 GiB peak allocated, 0.7 reserved);rtdetrv2-x-coco3000x4000 (0.6 GiB peak allocated, 0.8 reserved) rtdetrv2-s-coco800x1344, first call: portable + torch.compile 37 s (cold Inductor cache), TensorRT fp16 42 s (ONNX + engine).rtdetrv2-s-cocooriginal (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)rtdetrv2-s-cocooriginal 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)rtdetrv2-m-cocooriginal (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)rtdetrv2-m-cocooriginal 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)rtdetrv2-ms-cocooriginal (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)rtdetrv2-ms-cocooriginal 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)rtdetrv2-l-cocooriginal (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)rtdetrv2-l-cocooriginal 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)rtdetrv2-x-cocooriginal (fp32) 800x1344: incl. CPU resize + host copy (official pipeline)rtdetrv2-x-cocooriginal 800x1344: torch.autocast(fp16) around the forward; incl. CPU resize + host copy (official pipeline)rtdetrv2-s-cocoload / first call: original 1.32 s / 0.417 s, portable 0.31 s / 0.097 s, kernels 0.46† s / 0.534 srtdetrv2-m-cocoload / first call: original 0.45 s / 0.023 s, portable 0.33 s / 0.011 s, kernels 0.16† s / 0.012 srtdetrv2-ms-cocoload / first call: original 0.52 s / 0.057 s, portable 0.45 s / 0.03 s, kernels 0.17† s / 0.092 srtdetrv2-l-cocoload / first call: original 0.66 s / 0.043 s, portable 0.5 s / 0.015 s, kernels 0.25† s / 0.016 srtdetrv2-x-cocoload / first call: original 1.13 s / 0.076 s, portable 1.09 s / 0.048 s, kernels 0.29† s / 0.021 s- † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
- Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls
Measured
pycocotools COCOeval of the top 100 detections per image. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
rtdetrv2-s-coco | COCO val2017, every 10th image, box mAP | 49.29 | 49.26 | 49.29 | 49.31 | |
rtdetrv2-m-coco | COCO val2017, every 10th image, box mAP | 49.83 | 49.81 | 49.84 | 49.79 | |
rtdetrv2-ms-coco | COCO val2017, every 10th image, box mAP | 51.97 | 51.98 | 51.95 | 51.96 | |
rtdetrv2-l-coco | COCO val2017, every 10th image, box mAP | 54.52 | 54.57 | 54.51 | 54.49 | |
rtdetrv2-x-coco | COCO val2017, every 10th image, box mAP | 55.51 | 55.54 | 55.51 | 55.49 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 939adfe8, 2026-11-06.
Parity
CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.2 and p99 <= 1.0 (absolute error of the output boxes).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
rtdetrv2-s-coco | 0.03739 / 0.1098 | RTX 5060 Ti, torch 2.14.1+cu130 | cudnn_hacc@cudnn92400 |
rtdetrv2-m-coco | 0.06517 / 0.13434 | RTX 5060 Ti, torch 2.14.1+cu130 | cudnn_hacc@cudnn92400 |
rtdetrv2-ms-coco | 0.06765 / 0.17406 | RTX 5060 Ti, torch 2.14.1+cu130 | cudnn_hacc@cudnn92400 |
rtdetrv2-l-coco | 0.04387 / 0.1019 | RTX 5060 Ti, torch 2.14.1+cu130 | cudnn_hacc@cudnn92400 |
rtdetrv2-x-coco | 0.05571 / 0.2201 | RTX 5060 Ti, torch 2.14.1+cu130 | cudnn_hacc@cudnn92400 |