Key Points
- Apple released Depth Pro 2 on Hugging Face and GitHub in four sizes, 24M to 872M parameters.
- The model estimates metric depth in meters per pixel from a photo, video or live camera feed.
- The license permits only non-commercial research, keeping the weights out of shipping products.
The latest:
Apple has published code and weights for Depth Pro 2, a model that predicts metric depth in meters for every pixel from a single photo, a video file or a live camera feed. Four sizes run from 24 million to 872 million parameters, all sharing one focal-length estimator. The release carries Apple’s Machine Learning Research Model license, limited to non-commercial scientific research and academic development.
Details:
- The sizes: Four depth models were released: S at 24 million parameters and 97 MB, B at 97 million and 386 MB, L at 334 million and 1.3 GB, and the default H+ at 872 million and 3.5 GB. According to the model cards, larger sizes produce sharper and steadier depth while running more slowly.
- The shared component: All four rely on a 23.5 million-parameter, 94 MB focal-length estimator, used when the input carries no focal length and skipped when a value exists. Apple’s notes say a focal length 10% too long places everything 10% too far.
- Where it runs: The code supports NVIDIA GPUs through CUDA, Apple silicon GPUs through MPS, and CPU. The repository also ships Core ML exports and a live camera demo requiring Apple silicon, macOS Sequoia 15 or later, and Xcode 16 or its command-line tools.
- The benchmark claim: Apple’s comparison plot measures sharpness with Boundary F1 and temporal consistency with jitter, averaged over the Sintel and Spring datasets. All four new sizes sit on the lower-jitter side, with L and H+ highest in boundary F1, and the original Depth Pro far toward high jitter. The figures come from Apple’s own chart.
- The speed numbers: Throughput was measured on an NVIDIA B200 in bfloat16, PyTorch eager mode, a 32-frame clip in one call. At 640×480, S reaches 347.0 fps at 1.9 GiB peak memory and H+ reaches 95.9 fps at 8.3 GiB. At 1080p the pair fall to 44.8 fps and 5.3 fps.
- The 4K ceiling: At 4K, S manages 3.6 fps but requires 31.9 GiB of peak GPU memory, while H+ drops to 0.4 fps with a 114.6 GiB peak. At 720p, B records 82.5 fps at 6.6 GiB and L records 33.7 fps at 10.6 GiB.
- On a Mac: A second table, in median milliseconds per frame on a MacBook Pro with M4 Max and 64 GB running on the GPU, shows S holding 30 fps or more up to 896×512 and B real-time only at 640×360. H+ takes 1,233 milliseconds per frame at 1280×720, excluding preprocessing and rendering.
- The architecture: The model card describes a multi-view transformer processing one or more frames jointly, with a DPT decoder predicting an inverse depth map per frame. After the first third of its blocks, attention alternates within each frame and across frames, and the decoder builds feature maps from 1/32 to 1/4 resolution.
- Known limits: Depth is measured along the camera’s viewing direction, not straight-line distance. Sky and infinitely far regions are written as infinite depth, though smaller models sometimes emit a finite value of several kilometers, so the range should be clipped before building a point cloud or color preview.
- The timing: The first related post on X appeared at 08:08 Beijing time on November 20. Apple’s official accounts did not publish a matching announcement in the same window.
Background:
Depth Pro, the earlier Apple model, appears in the new comparison chart as the high-jitter reference point. Depth Pro 2 extends it from single images to video and live camera input, processing frames jointly rather than one at a time.
Between the lines:
The gap between the B200 throughput table and the MacBook Pro millisecond table marks the practical boundary of this release: only the smallest model clears real-time on consumer Apple hardware, while the sharpest sizes need datacenter-class memory. Combined with a license that excludes commercial deployment, the package reads as a research stack rather than a product component.
What’s next
Watch for an official Apple announcement, independent reruns of the Sintel and Spring benchmarks outside Apple’s own figures, and any license revision permitting commercial use of the weights.