Models · Multi-view depth

PatchmatchNet

PatchmatchNet by Fangjinhua Wang et al.: learned PatchMatch multi-view stereo, depth from posed images. Commercially usable.

Commercial use: yes.

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
patchmatchnetsynthetic (1536x1024)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
248
220
202
29.6
175
51.0
1330 / 1412
1228 / 1272
1806 / 2096
740 / 856
1798 / 1888
844 (engine)
Details
  • Original code: https://github.com/FangjinhuaWang/PatchmatchNet at 8dc6cb4
  • Code licence: MIT (permissive); files: LICENSE
  • Weights licence: MIT (assumed from code)
  • Licence evidence: https://github.com/FangjinhuaWang/PatchmatchNet/blob/8dc6cb4/LICENSE and README.md (the checkpoint ships in the repository)
  • Training data: DTU Robot Image Data Sets (MVS 2014) (The released checkpoint (checkpoints/params_000007.ckpt) is trained on DTU (README); the paper evaluates it zero-shot on Tanks and Temples and ETH3D)
  • Licence note: The code is MIT; the checkpoint ships in the repository without a licence of its own (MIT assumed) and is trained on DTU, whose terms of use are not stated.
  • Kernel libraries: patchmatchnet
  • Variants: patchmatchnet
  • Outputs: depth, confidence; kind: metric
  • Input size: any H x W >= 5, resized or padded inside the model
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): patchmatchnet 1988x2880 (7.9 GiB peak allocated, 9.4 reserved)
  • patchmatchnet 1536x1024, first call: portable + torch.compile 29 s (cold Inductor cache), TensorRT fp16 82 s (ONNX + engine).
  • patchmatchnet original 1536x1024: torch.autocast(fp16) around the forward
  • patchmatchnet load / first call: original 0.04 s / 0.917 s (1536x1024), portable 0.06 s / 0.25 s (1536x1024), kernels 0.05† s / 0.035 s (1536x1024)
  • † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
  • Protocol, median of 9 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls

Measured

640x512 crops, the 7 most co-visible views as sources; GT = depth rendered from the structured-light points. AbsRel / delta1 over GT pixels, after per-image median scaling unless the model takes the cameras (calibrated MVS: cameras, error in mm; PatchMatchNet also the depth range 425-935 mm, MVSAnywhere its own range from the camera baselines). Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portablekernelsnote
patchmatchnetDTU scans 1, 9, 24, 33, 75, view 24 + 7 sources, AbsRel / delta1 % / MAE mm0.0087 / 99.52 / 7.03fails (LinAlgError: linalg.inv)0.0087 / 99.52 / 7.080.0087 / 99.52 / 7.08

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, 2026-11-06.

Parity

CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.5 % and p99 <= 3 % of the reference's median |value| (absolute error of the output depth).

variantkernels error vs original, mean / p99GPU, torchkernels used
patchmatchnet6e-05 / 0.00021RTX 5060 Ti, torch 2.14.1+cu130patchmatchnet@e73cbaa02d34:sm_120