Models · Semantic segmentation

Mask2Former

Mask2Former with a Swin-L backbone for ADE20K scene segmentation. Non-commercial weights.

Commercial use: no (weights: CC-BY-NC-4.0).

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
mask2former-swinl-ade20ksynthetic (1024x2048)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
244
145
103
60.4
fails: Inductor: Triton kernel does not compile (torch bug)
55.6
2966 / 3660
3572 / 3878
1164 / 1400
1164 / 1400
–
1816 (engine)
Details
  • Original code: https://github.com/huggingface/transformers at c4c4605f9618
  • Code licence: Apache-2.0 (permissive); files:
  • Weights licence: CC-BY-NC-4.0
  • Licence evidence: https://github.com/facebookresearch/Mask2Former/blob/9b0651c6c1d5b3af2e6da0589b719c514ec0d69a/MODEL_ZOO.md#license
  • Training data: ImageNet-21k / ImageNet-22k (full ImageNet, fall 2011 release), ADE20K (SceneParsing, 150 classes)
  • Kernel libraries: fvm_seg, seg_up
  • Variants: mask2former-swinl-ade20k
  • Outputs: logits; label_set: ade20k-150
  • Input size: any H x W >= 1, resized or padded inside the model
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): mask2former-swinl-ade20k 3000x4000 (14.1 GiB peak allocated, 15.1 reserved)
  • mask2former-swinl-ade20k 1024x2048, first call: TensorRT fp16 129 s (ONNX + engine).
  • mask2former-swinl-ade20k original 1024x2048: torch.autocast(fp16) around the forward
  • mask2former-swinl-ade20k load / first call: original 4.86 s / 0.763 s (512x512), portable 3.03 s / 0.127 s, kernels 3.17† s / 0.069† s
  • † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
  • Protocol, first call, 3 warm-up, peak reset, median of 3 rounds x median of 5 (tools/speed_check.py), synthetic CUDA input (bench_inputs); peak = max over the first call and the timed calls

Measured

Each image at its own size, argmax of the logits. Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portablekernelsnote
mask2former-swinl-ade20kADE20K val, every 4th image, mIoU / aAcc %53.55 / 85.1053.63 / 85.1053.68 / 85.1253.68 / 85.10

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 939adfe8, 2026-11-06.

Parity

CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.001 and p99 <= 0.01 (absolute error of the output logits).

variantkernels error vs original, mean / p99GPU, torchkernels used
mask2former-swinl-ade20k2e-05 / 0.00024RTX 5060 Ti, torch 2.14.1+cu130fvm_seg@739693dff3b4:sm_120, seg_up@3e44e71200cd:sm_120