MVSAnywhere
MVSAnywhere (Niantic): depth of a reference view from several posed images, for any scene type. Non-commercial licence.
Commercial use: no (code: LicenseRef-noncommercial AND CC-BY-NC-SA-4.0 (noncommercial); weights: LicenseRef-noncommercial AND CC-BY-NC-SA-4.0 (assumed from code)).
| variant | input | backend | time (ms) | peak memory (MiB, allocated / reserved) |
|---|---|---|---|---|
mvsanywhere | synthetic (448x672) | original (fp32) original portable kernels portable + torch.compile TensorRT fp16 | 273 219 93.2 35.6 70.8 fails: TensorRT: NaN output (onnxruntime on the same ONNX is right) | 1441 / 1500 1504 / 1576 931 / 1226 761 / 884 818 / 860 – |
Details
- Original code: https://github.com/nianticlabs/mvsanywhere at
5bd49bb - Code licence: LicenseRef-noncommercial AND CC-BY-NC-SA-4.0 (noncommercial); files:
LICENSE,LICENSE-antialiased-cnns.md - Weights licence: LicenseRef-noncommercial AND CC-BY-NC-SA-4.0 (assumed from code)
- Licence evidence: https://github.com/nianticlabs/mvsanywhere/blob/5bd49bb/LICENSE and README.md (the checkpoint is published under the repository's licence)
- Training data: Synthetic data per the paper (arXiv 2411.14390): Hypersim, TartanAir, BlendedMVG, MatrixCity, VKITTI2, Dynamic Replica, MVS-Synth, SAIL-VOS 3D; DINOv2 / Depth Anything V2 initialisation
- Licence note: Niantic's non-commercial licence covers the code and the checkpoint; the BlurPool stem comes from Adobe's antialiased-cnns (CC-BY-NC-SA-4.0).
- Options:
long_side= 640 (>= 64),refinement_steps= 1 (>= 0) - Kernel libraries:
fastbb,igev,cutlass_conv,cudnn_hacc,mvsanywhere - Variants:
mvsanywhere - Outputs: depth, mask, sky (in
extras), depth_range (inextras); kind:metric - Input size: any H x W >= 1, resized or padded inside the model
- Tested up to (portable, fp16, RTX 5060 Ti 16 GB):
mvsanywhere3000x4000 (3.0 GiB peak allocated, 4.2 reserved) mvsanywhere448x672, first call: portable + torch.compile 153 s (cold Inductor cache).mvsanywhereoriginal 448x672: torch.autocast(float16) around the callmvsanywhereload / first call: original 3.21 s / 1.037 s (448x672), portable 1.88 s / 0.221 s (448x672), kernels 0.52† s / 0.049 s (448x672)- † measured in the speed sweep after the portable load (or an earlier size) in the same process: file cache and kernels are warm, so the time is lower than a fresh start.
- Protocol, median of 5 after 3 warm-up, uint8 CUDA input (synthetic for bench_inputs sizes), card defaults; gc before each load; peak = max over the first call and the timed calls
Measured
640x512 crops, the 7 most co-visible views as sources; GT = depth rendered from the structured-light points. AbsRel / delta1 over GT pixels, after per-image median scaling unless the model takes the cameras (calibrated MVS: cameras, error in mm; PatchMatchNet also the depth range 425-935 mm, MVSAnywhere its own range from the camera baselines). Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
mvsanywhere | DTU scans 1, 9, 24, 33, 75, view 24 + 7 sources, AbsRel / delta1 % / MAE mm | 0.0077 / 99.82 / 5.57 | 0.0110 / 99.84 / 7.85 | 0.0078 / 99.82 / 5.60 | 0.0077 / 99.82 / 5.59 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 555d7b79, 2026-11-06.
Parity
CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.1 % and p99 <= 0.5 % of the reference's median |value| (absolute error of the output depth).
| variant | kernels error vs original, mean / p99 | GPU, torch | kernels used |
|---|---|---|---|
mvsanywhere | 6e-05 / 0.00025 | RTX 5060 Ti, torch 2.14.1+cu130 | cudnn_hacc@cudnn92400, cutlass_conv@988138e042cb:sm_120, fastbb@7d058322634d:sm_120, igev@f075827d68c6:sm_120, mvsanywhere@7fdcb3aafdd8:sm_120 |