Models · Stereo matching

S2M2

A transformer-based stereo family in four sizes (S, M, L, XL). Non-commercial licence.

Commercial use: no (code: CC-BY-NC-4.0 (noncommercial); weights: CC-BY-NC-4.0).

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
s2m2-sMiddlebury 2014 half (994x1482)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16		582
350
348
113
186
137		2775 / 3008
1759 / 1986
1733 / 1930
686 / 820
1579 / 1710
980 (engine)
s2m2-mMiddlebury 2014 half (994x1482)original (fp32)
original
portable
kernels		1242
666
670
248		3399 / 3704
2371 / 2920
2300 / 2786
876 / 1162
s2m2-lMiddlebury 2014 half (994x1482)original (fp32)
original
portable
kernels		1934
1005
1015
376		4333 / 5252
3248 / 4090
3245 / 3838
1249 / 1568
s2m2-xlMiddlebury 2014 half (994x1482)original (fp32)
original
portable
kernels		3510
1796
1815
682		6940 / 8254
5606 / 6080
5228 / 6016
2092 / 2274
Details
  • Original code: https://github.com/junhong-3dv/s2m2 at 8f3bd785df59
  • Code licence: CC-BY-NC-4.0 (noncommercial); files: LICENSE.md
  • Weights licence: CC-BY-NC-4.0
  • Licence evidence: https://github.com/junhong-3dv/s2m2/blob/8f3bd785df593619d955203c6f2c741bef6e31df/README.md#L24
  • Training data: README: extended training data incl. transparent objects
  • Kernel libraries: s2m2, s2m2_cutlass, cudnn_hacc
  • Variants: s2m2-s, s2m2-m, s2m2-l, s2m2-xl
  • Outputs: disparity, confidence, occlusion (in extras)
  • Input size: any H x W >= 32, padded to a multiple of 32
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): s2m2-s 1988x2880 (7.4 GiB peak allocated, 10.6 reserved); s2m2-m 1988x2880 (8.9 GiB peak allocated, 11.5 reserved); s2m2-l 1988x2880 (11.4 GiB peak allocated, 14.2 reserved); s2m2-xl 1024x2048 (7.3 GiB peak allocated, 8.5 reserved)
  • s2m2-s 994x1482, first call: portable + torch.compile 96 s (cold Inductor cache), TensorRT fp16 363 s (ONNX + engine).
  • s2m2-s original 994x1482: torch.autocast(fp16) around the forward
  • s2m2-m original 994x1482: torch.autocast(fp16) around the forward
  • s2m2-l original 994x1482: torch.autocast(float16) around the call
  • s2m2-xl original 994x1482: torch.autocast(fp16) around the forward
  • s2m2-s load / first call: original 0.22 s / 0.143 s (375x1242), portable 0.22 s / 0.352† s, kernels 0.2† s / 0.114† s
  • s2m2-m load / first call: original 3.95 s / 0.268 s (375x1242), portable 0.5 s / 0.675† s, kernels 0.31† s / 0.249† s
  • s2m2-l load / first call: original 1.38 s / 1.007 s, portable 0.99 s / 1.015† s, kernels 0.58† s / 0.374† s
  • s2m2-xl load / first call: original 4.5 s / 0.529 s (375x1242), portable 2.73 s / 1.814† s, kernels 0.9† s / 0.681† s
  • † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
  • Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls

Measured

Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portablekernelsnote
s2m2-sETH3D two-view train, bad > 1 px % / EPE1.10 / 0.1931.07 / 0.1981.12 / 0.1931.10 / 0.193possibly in the training data
s2m2-sMiddlebury 2014 trainingH, bad > 2 px % / EPE3.23 / 0.6523.23 / 0.6573.24 / 0.6523.24 / 0.652possibly in the training data
s2m2-mETH3D two-view train, bad > 1 px % / EPE0.94 / 0.1780.94 / 0.1880.95 / 0.1780.95 / 0.178possibly in the training data
s2m2-mMiddlebury 2014 trainingH, bad > 2 px % / EPE3.28 / 0.6513.28 / 0.6563.29 / 0.6513.28 / 0.651possibly in the training data
s2m2-lETH3D two-view train, bad > 1 px % / EPE0.84 / 0.1690.85 / 0.1820.84 / 0.1690.82 / 0.169possibly in the training data
s2m2-lMiddlebury 2014 trainingH, bad > 2 px % / EPE2.88 / 0.5422.84 / 0.5512.84 / 0.5422.85 / 0.542possibly in the training data
s2m2-xlETH3D two-view train, bad > 1 px % / EPE0.53 / 0.1470.52 / 0.1570.53 / 0.1470.52 / 0.149possibly in the training data
s2m2-xlMiddlebury 2014 trainingH, bad > 2 px % / EPE2.34 / 0.4982.33 / 0.5042.33 / 0.4982.33 / 0.499possibly in the training data

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 468fba10, 2026-11-05.

Parity

CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.02 px and p99 <= 0.25 px.

variantkernels error vs original, mean / p99GPU, torchkernels used
s2m2-s0.00973 / 0.03415RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, s2m2@73d0d3fa70d2:sm_120, s2m2_cutlass@3aec99821257:sm_120
s2m2-m0.00618 / 0.02223RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, s2m2@73d0d3fa70d2:sm_120, s2m2_cutlass@3aec99821257:sm_120
s2m2-l0.01523 / 0.05073RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, s2m2@73d0d3fa70d2:sm_120, s2m2_cutlass@3aec99821257:sm_120
s2m2-xl0.01853 / 0.0624RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, s2m2@73d0d3fa70d2:sm_120, s2m2_cutlass@3aec99821257:sm_120

Limitations

  • At 994x1482 portable S2M2-S is about as fast as the original code and allocates slightly more memory; use kernels for speed and memory.