Models · Stereo matching

HITNet

Google Research's tile-based stereo model; the authors' ETH3D and Middlebury versions, ported to PyTorch. Commercially usable.

Commercial use: yes.

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
hitnet-eth3dMiddlebury 2014 half (994x1482)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
234
182
67.8
23.2
27.7
31.1
2961 / –
2433 / –
860 / 1090
647 / 1248
719 / 728
836 (engine)
hitnet-middlebury-d400Middlebury 2014 half (994x1482)original (fp32)
original
portable
kernels
637
483
176
52.7
7157 / –
6831 / –
1744 / 2414
1263 / 2368
Details
  • Original code: https://github.com/google-research/google-research at f200f2b89d29
  • Code licence: Apache-2.0 (permissive); files: LICENSE
  • Weights licence: Apache-2.0 (assumed from code)
  • Licence evidence: https://github.com/google-research/google-research/blob/f200f2b89d29484bf0dd0fef0c3979ac8bed9af4/LICENSE
  • Training data: not stated by the authors
    • hitnet-eth3d: ETH3D two-view stereo (fine-tuned on the ETH3D training pairs (paper); any pre-training data is not stated by the authors)
    • hitnet-middlebury-d400: Middlebury stereo (2014) (trained for half-resolution Middlebury images (README); any pre-training data is not stated by the authors)
  • Licence note: The authors state no weights licence; the checkpoints are fetched from the PINTO model-zoo mirror (a third-party host), not from Google, and the code licence is assumed to cover them.
  • Options: cuda_graphs = True (speed only)
  • Kernel libraries: hitnet, hitnet_conv
  • Variants: hitnet-eth3d, hitnet-middlebury-d400
  • Outputs: disparity
  • Input size: any H x W >= 8, resized or padded inside the model
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): hitnet-eth3d 3000x4000 (6.4 GiB peak allocated, 10.8 reserved); hitnet-middlebury-d400 1988x2880 (6.4 GiB peak allocated, 10.1 reserved)
  • hitnet-eth3d 994x1482, first call: portable + torch.compile 39 s (cold Inductor cache), TensorRT fp16 136 s (ONNX + engine).
  • hitnet-eth3d original (fp32) 994x1482: TensorFlow 2.21.0 frozen graph eth3d.pb, float32
  • hitnet-eth3d original 994x1482: TensorFlow 2.21.0 frozen graph eth3d.pb, auto_mixed_precision graph rewrite (float16)
  • hitnet-middlebury-d400 original (fp32) 994x1482: TensorFlow 2.21.0 frozen graph middlebury_d400.pb, float32
  • hitnet-middlebury-d400 original 994x1482: TensorFlow 2.21.0 frozen graph middlebury_d400.pb, auto_mixed_precision graph rewrite (float16)
  • hitnet-eth3d load / first call: original 0.31 s / 2.061 s, portable 0.09 s / 0.078† s, kernels 0.07† s / 0.026† s
  • hitnet-middlebury-d400 load / first call: original 0.57 s / 3.512 s, portable 0.13 s / 0.177† s, kernels 0.1† s / 0.053† s
  • † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
  • Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls

Measured

Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portablekernelsnote
hitnet-eth3dETH3D two-view train, bad > 1 px % / EPE0.94 / 0.1640.93 / 0.1650.93 / 0.1640.93 / 0.164trained on this dataset's public images; not a zero-shot result
hitnet-eth3dMiddlebury 2014 trainingH, bad > 2 px % / EPE17.26 / 8.33217.30 / 8.35617.25 / 8.34017.27 / 8.297
hitnet-middlebury-d400ETH3D two-view train, bad > 1 px % / EPE7.02 / 0.8857.00 / 0.8807.01 / 0.8997.07 / 0.875
hitnet-middlebury-d400Middlebury 2014 trainingH, bad > 2 px % / EPE7.50 / 1.1807.51 / 1.1857.54 / 1.1877.51 / 1.168trained on this dataset's public images; not a zero-shot result

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 468fba10, 2026-11-05.

Parity

CPU, float32: max |diff| <= 0.02 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.15 px and p99 <= 1.0 px. On real pairs up to 720x1280 the limits are mean 0.3 / p99 6.0 (precision-sensitive there, see Limitations).

variantkernels error vs original, mean / p99GPU, torchkernels used
hitnet-eth3d0.00343 / 0.02548RTX 5060 Ti, torch 2.14.1+cu130hitnet@45145b49dcb1:sm_120, hitnet_conv@e5637e8a93cf:sm_120
hitnet-middlebury-d4000.1139 / 0.74705RTX 5060 Ti, torch 2.14.1+cu130hitnet@45145b49dcb1:sm_120, hitnet_conv@e5637e8a93cf:sm_120

Limitations

  • CUDA output (float16, kernels, float32 with PyTorch's default TF32 convolutions) differs from the CPU by ~0.2 px mean / 4 px p99 at 720x1280: the argmin flips on near-equal tile costs. Full float32 (torch.backends.cudnn.allow_tf32 = False) matches the CPU to 0.00002 px.
  • The weights are the authors' frozen TensorFlow graphs read without TensorFlow; the FlyingThings and Middlebury d160 / d288 graphs are not included.
  • The original code is TensorFlow and was not timed.