Models · Multi-view depth

MVSAnywhere

MVSAnywhere (Niantic): depth of a reference view from several posed images, for any scene type. Non-commercial licence.

Commercial use: no (code: LicenseRef-noncommercial AND CC-BY-NC-SA-4.0 (noncommercial); weights: LicenseRef-noncommercial AND CC-BY-NC-SA-4.0 (assumed from code)).

variantinputbackendtime (ms)peak memory (MiB, allocated / reserved)
mvsanywheresynthetic (448x672)original (fp32)
original
portable
kernels
portable + torch.compile
TensorRT fp16
273
219
93.2
35.6
70.8
fails: TensorRT: NaN output (onnxruntime on the same ONNX is right)
1441 / 1500
1504 / 1576
931 / 1226
761 / 884
818 / 860
–
Details
  • Original code: https://github.com/nianticlabs/mvsanywhere at 5bd49bb
  • Code licence: LicenseRef-noncommercial AND CC-BY-NC-SA-4.0 (noncommercial); files: LICENSE, LICENSE-antialiased-cnns.md
  • Weights licence: LicenseRef-noncommercial AND CC-BY-NC-SA-4.0 (assumed from code)
  • Licence evidence: https://github.com/nianticlabs/mvsanywhere/blob/5bd49bb/LICENSE and README.md (the checkpoint is published under the repository's licence)
  • Training data: Synthetic data per the paper (arXiv 2411.14390): Hypersim, TartanAir, BlendedMVG, MatrixCity, VKITTI2, Dynamic Replica, MVS-Synth, SAIL-VOS 3D; DINOv2 / Depth Anything V2 initialisation
  • Licence note: Niantic's non-commercial licence covers the code and the checkpoint; the BlurPool stem comes from Adobe's antialiased-cnns (CC-BY-NC-SA-4.0).
  • Options: long_side = 640 (>= 64), refinement_steps = 1 (>= 0)
  • Kernel libraries: fastbb, igev, cutlass_conv, cudnn_hacc, mvsanywhere
  • Variants: mvsanywhere
  • Outputs: depth, mask, sky (in extras), depth_range (in extras); kind: metric
  • Input size: any H x W >= 1, resized or padded inside the model
  • Tested up to (portable, fp16, RTX 5060 Ti 16 GB): mvsanywhere 3000x4000 (3.0 GiB peak allocated, 4.2 reserved)
  • mvsanywhere 448x672, first call: portable + torch.compile 153 s (cold Inductor cache).
  • mvsanywhere original 448x672: torch.autocast(float16) around the call
  • mvsanywhere load / first call: original 3.21 s / 1.037 s (448x672), portable 1.88 s / 0.221 s (448x672), kernels 0.52† s / 0.049 s (448x672)
  • † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
  • Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls

Measured

640x512 crops, the 7 most co-visible views as sources; GT = depth rendered from the structured-light points. AbsRel / delta1 over GT pixels, after per-image median scaling unless the model takes the cameras (calibrated MVS: cameras, error in mm; PatchMatchNet also the depth range 425-935 mm, MVSAnywhere its own range from the camera baselines). Original = the original code named under Details, same checkpoint and protocol.

variantdatasetoriginal fp32original fp16portablekernelsnote
mvsanywhereDTU scans 1, 9, 24, 33, 75, view 24 + 7 sources, AbsRel / delta1 % / MAE mm0.0077 / 99.82 / 5.570.0110 / 99.84 / 7.850.0078 / 99.82 / 5.600.0077 / 99.82 / 5.59

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 555d7b79, 2026-11-06.

Parity

CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.1 % and p99 <= 0.5 % of the reference's median |value| (absolute error of the output depth).

variantkernels error vs original, mean / p99GPU, torchkernels used
mvsanywhere6e-05 / 0.00025RTX 5060 Ti, torch 2.14.1+cu130cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, fastbb@7d058322634d:sm_120, igev@f075827d68c6:sm_120, mvsanywhere@7fdcb3aafdd8:sm_120