OpenPose — Real-Time Multi-Person Pose Estimation

Introduction

OpenPose is a real-time multi-person keypoint detection library developed at Carnegie Mellon University. It detects body, hand, facial, and foot keypoints on single images or video, enabling applications from motion capture to fitness tracking without specialized hardware.

What OpenPose Does

• Detects 135 keypoints per person including body, hands, face, and feet simultaneously
• Processes multi-person scenes in real time using a bottom-up approach
• Supports input from images, video files, webcams, and IP cameras
• Provides JSON, image, and video output formats for downstream pipelines
• Runs on CUDA GPUs with optional OpenCL and CPU-only fallback

Architecture Overview

OpenPose uses a two-branch convolutional neural network. The first branch predicts Part Affinity Fields (PAFs) that encode limb associations between keypoints. The second branch predicts confidence maps for individual body part locations. A greedy bipartite matching algorithm then assembles the detected parts into full-body skeletons, allowing the system to scale to any number of people in the frame without a top-down person detector.

Self-Hosting & Configuration

• Requires CMake 3.12+, GCC/G++ 7+, and CUDA 10+ for GPU acceleration
• Supports cuDNN for faster inference on NVIDIA hardware
• Pre-trained models are downloaded automatically on first run
• Configuration flags control resolution, number of scales, and output format
• Docker images available for containerized deployment

Key Features

• First open-source real-time multi-person pose estimation system
• Bottom-up approach maintains constant speed regardless of person count
• Combined body-hand-face-foot model in a single forward pass
• Python and C++ APIs for integration into production applications
• Calibration tools for multi-camera 3D reconstruction

Comparison with Similar Tools

• MediaPipe Pose — lighter weight and mobile-friendly but limited to single-person detection
• MMPose — part of OpenMMLab ecosystem with more model options but higher complexity
• AlphaPose — top-down approach with higher accuracy per person but slower on crowds
• HRNet — higher-resolution feature maps for better accuracy at the cost of speed
• ViTPose — transformer-based with strong benchmarks but requires more compute

FAQ

Q: Does OpenPose require a GPU? A: GPU acceleration via CUDA is recommended for real-time performance. CPU-only mode works but runs significantly slower, typically under 1 FPS.

Q: Can OpenPose run on video in real time? A: Yes, on a modern NVIDIA GPU it processes 15-25 FPS at default resolution for multi-person scenes.

Q: What output formats are available? A: OpenPose outputs JSON keypoint files, rendered images/video with skeleton overlays, and raw heatmaps for custom processing.

Q: Is OpenPose suitable for commercial use? A: OpenPose uses a custom non-commercial license. Commercial use requires a separate license from CMU.

Sources

• https://github.com/CMU-Perceptual-Computing-Lab/openpose
• https://cmu-perceptual-computing-lab.github.io/openpose/web/html/doc/