Depth Pro — Sharp Monocular Metric Depth Estimation by Apple

Introduction

Depth Pro is a monocular depth estimation model released by Apple Research. Given a single RGB image, it predicts a dense, metric-scale depth map with sharp boundary detail. Unlike relative-depth models, Depth Pro outputs absolute distances in real-world units without requiring camera metadata.

What Depth Pro Does

• Estimates per-pixel depth from a single image at up to 2.25 megapixel resolution
• Produces metric-scale depth values (absolute distances) without camera intrinsics
• Preserves sharp depth boundaries around thin structures and fine details
• Runs in approximately 0.3 seconds on a modern GPU
• Generalizes across indoor, outdoor, and mixed-domain scenes in a zero-shot manner

Architecture Overview

Depth Pro uses a multi-scale vision transformer that processes the input image at multiple resolutions simultaneously. Coarse-scale patches capture global scene structure and absolute scale, while fine-scale patches recover local detail and sharp edges. The model fuses these scales through cross-attention layers, producing a high-resolution depth map with both global metric accuracy and local boundary precision. Training combines real-world datasets with synthetic data and uses a sharp boundary loss to avoid the blurry edges common in other depth models.

Self-Hosting & Configuration

• Install via pip install depth-pro or clone the repository from GitHub
• Pre-trained weights download automatically on first run
• Requires PyTorch 2.0+ and a CUDA GPU for efficient inference
• Use the CLI for batch processing or the Python API for integration into pipelines
• Configure output resolution and format via command-line flags or API parameters

Key Features

• Metric depth: outputs absolute distances rather than relative rankings
• Boundary sharpness surpasses competing monocular depth methods
• No camera intrinsics required for inference
• Sub-second inference on a single GPU
• Apple open-source release under a permissive license

Comparison with Similar Tools

• MiDaS — widely used monocular depth model by Intel; produces relative depth only, not metric scale
• Marigold — diffusion-based depth estimation with fine detail; slower inference and relative depth by default
• ZoeDepth — combines relative and metric depth heads; Depth Pro achieves sharper boundaries without a two-stage approach
• Depth Anything — versatile depth model with strong generalization; primarily relative depth, Depth Pro provides absolute metric output
• UniDepth — metric depth with camera-aware features; Depth Pro achieves comparable accuracy without explicit camera modeling

FAQ

Q: Does Depth Pro work on video? A: Depth Pro processes individual frames. For temporally consistent video depth, post-process with a temporal consistency filter or use a video-specific model.

Q: What resolution does Depth Pro output? A: The default output is 1536x1536 pixels. Input images are resized internally and the depth map is upsampled to match the original aspect ratio.

Q: Can I use Depth Pro for 3D reconstruction? A: Yes. The metric depth maps can be unprojected into point clouds using standard pinhole camera models for downstream 3D tasks.

Q: What license is Depth Pro released under? A: Depth Pro is released under the Apple Sample Code License, which permits personal and commercial use.

Sources

• https://github.com/apple/ml-depth-pro
• https://arxiv.org/abs/2410.02073