# SAM 2 — Segment Anything in Images and Videos by Meta

> SAM 2 (Segment Anything Model 2) is Meta's foundation model for promptable visual segmentation in both images and videos, capable of producing precise object masks from points, boxes, or text prompts.

## Install

Save as a script file and run:

# SAM 2 — Segment Anything in Images and Videos by Meta

## Quick Use
```bash
pip install sam-2
python -c "
import torch
from sam2.build_sam import build_sam2
from sam2.sam2_image_predictor import SAM2ImagePredictor
predictor = SAM2ImagePredictor(build_sam2('sam2.1_hiera_large'))
# predictor.set_image(image)
# masks, scores, logits = predictor.predict(point_coords=...)
"
```

## Introduction
SAM 2 extends Meta's original Segment Anything Model to handle both images and videos in a unified architecture. It introduces a streaming memory mechanism that lets the model track and segment objects across video frames in real time, while maintaining the interactive prompting interface from SAM 1.

## What SAM 2 Does
- Segments any object in a single image from point, box, or mask prompts
- Tracks and segments objects across video frames with temporal consistency
- Produces multiple valid mask hypotheses ranked by confidence scores
- Supports zero-shot transfer to new object categories without retraining
- Runs interactively for annotation workflows or in batch mode for pipelines

## Architecture Overview
SAM 2 uses a Hiera image encoder (a hierarchical vision transformer) to extract multi-scale features from each frame. A prompt encoder converts user inputs (points, boxes, masks) into embeddings, and a lightweight mask decoder generates segmentation masks. For video, a memory attention module maintains a memory bank of past frames and predictions, allowing the model to propagate masks forward and backward through time. The streaming design processes one frame at a time, making it practical for long videos.

## Self-Hosting & Configuration
- Install from PyPI with `pip install sam-2` or clone the GitHub repository
- Download pre-trained checkpoints (tiny, small, base+, large) from the releases page
- Requires PyTorch 2.3+ and a CUDA-capable GPU for efficient inference
- Configure model size via config YAML files shipped with the repo
- Integrate into annotation tools like Label Studio or custom web UIs via the Python API

## Key Features
- Unified image and video segmentation in a single model architecture
- Streaming memory design enables real-time video processing
- Four model sizes from SAM 2.1 Tiny to SAM 2.1 Large for speed-accuracy tradeoffs
- Interactive and automatic modes for both annotation and production pipelines
- Trained on SA-V, a dataset of over 50K videos with 600K masklets

## Comparison with Similar Tools
- **SAM 1** — image-only predecessor without video support; SAM 2 supersedes it with better image performance and adds video capabilities
- **XMem / Cutie** — specialized video object segmentation models; SAM 2 unifies image and video in one architecture
- **GroundingDINO + SAM** — combines open-vocabulary detection with segmentation; SAM 2 can be similarly composed but also handles video natively
- **YOLO-Seg** — instance segmentation optimized for speed; less precise masks but faster on edge devices
- **Detectron2** — modular detection and segmentation framework; requires task-specific training unlike SAM 2's zero-shot capability

## FAQ
**Q: Can SAM 2 run without a GPU?**
A: CPU inference is possible but slow. A CUDA GPU with at least 6 GB VRAM is recommended for interactive use.

**Q: Does SAM 2 understand object categories?**
A: No. SAM 2 segments objects based on spatial prompts, not semantic categories. Pair it with a classifier or detector for category labels.

**Q: How do I segment objects in a long video?**
A: Provide a prompt on the first frame (or any frame), and SAM 2's memory mechanism will propagate the mask across subsequent frames automatically.

**Q: What license is SAM 2 released under?**
A: SAM 2 is released under the Apache 2.0 license.

## Sources
- https://github.com/facebookresearch/sam2
- https://ai.meta.com/sam2/

---
Source: https://tokrepo.com/en/workflows/asset-0373e7d1
Author: Script Depot