Models · Semantic segmentation
OneFormer
OneFormer (SHI Labs): one network for several segmentation tasks, used here for ADE20K scene segmentation. Commercially usable.
Commercial use: yes.
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
oneformer-swinl-ade20k | synthetic (1024x2048) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 307 205 139 100.0 fails: Inductor: Triton kernel does not compile (torch bug) 83.3 | 6853 / 7808 7395 / 7792 2054 / 2476 2054 / 2476 – 3258 (engine) |
Details
- Original code: https://github.com/huggingface/transformers at
c4c4605f9618 - Code licence: Apache-2.0 (permissive); files:
- Weights licence: MIT
- Licence evidence: https://huggingface.co/shi-labs/oneformer_ade20k_swin_large/blob/4a5bac8e64f82681a12db2e151a4c2f4ce6092b2/README.md
- Training data: ImageNet-21k / ImageNet-22k (full ImageNet, fall 2011 release), ADE20K (SceneParsing, 150 classes)
- Kernel libraries:
fvm_seg,seg_up - Variants:
oneformer-swinl-ade20k - Outputs: logits; label_set:
ade20k-150 - Input size: any H x W >= 1, resized or padded inside the model
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
oneformer-swinl-ade20k1988x2880 (8.4 GiB peak allocated, 14.9 reserved) oneformer-swinl-ade20k1024x2048, first call: TensorRT fp16 140 s (ONNX + engine).oneformer-swinl-ade20koriginal 1024x2048: torch.autocast(fp16) around the forwardoneformer-swinl-ade20kload / first call: original 5.08 s / 0.805 s (512x512), portable 4.1 s / 0.147† s, kernels 3.27† s / 0.109† s- † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
- Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5 (tools/speed_check.py), synthetic CUDA input (bench_inputs); peak = max over the first call and the timed calls
Measured
Each image at its own size, argmax of the logits. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
oneformer-swinl-ade20k | ADE20K val, every 4th image, mIoU / aAcc % | 54.49 / 85.06 | 54.26 / 85.05 | 54.37 / 85.05 | 54.38 / 85.14 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 939adfe8, 2026-11-06.
Parity
CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.001 and p99 <= 0.01 (absolute error of the output logits).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
oneformer-swinl-ade20k | 4e-05 / 0.00197 | RTX 5060 Ti, torch 2.14.1+cu130 | fvm_seg@739693dff3b4:sm_120, seg_up@3e44e71200cd:sm_120 |