InsightFace — Open-Source 2D and 3D Face Analysis Toolkit

Introduction

InsightFace is an open-source toolkit for 2D and 3D face analysis research and deployment. It bundles high-accuracy models for detection, recognition, alignment, age and gender estimation, and 3D face reconstruction under a unified Python API.

What InsightFace Does

• Detects faces in images and video using SCRFD and RetinaFace detectors
• Generates face embeddings with ArcFace for accurate identity matching
• Estimates facial landmarks for alignment and pose correction
• Predicts age, gender, and facial attributes from a single frame
• Reconstructs 3D face meshes from 2D photographs

Architecture Overview

InsightFace organizes models into a model zoo with ONNX exports for portable inference. The FaceAnalysis pipeline chains a detector, landmark predictor, and recognition model. Training scripts use PyTorch or MXNet with configurable loss functions like ArcFace, CosFace, and SubCenter variants, while deployment relies on ONNX Runtime for cross-platform speed.

Self-Hosting & Configuration

• Install via pip; GPU inference requires onnxruntime-gpu and CUDA drivers
• Download model packs from the official model zoo or Hugging Face
• Set det_size and det_thresh to balance speed against detection recall
• Use the recognition module standalone for embedding-only workflows
• Deploy as an HTTP service by wrapping FaceAnalysis in FastAPI or Flask

Key Features

• ArcFace recognition achieves over 99.8% accuracy on LFW benchmark
• SCRFD detector runs at 10 ms per frame on a modern GPU
• Supports ONNX export for deployment on edge devices and mobile
• Includes 3D face reconstruction with dense landmark prediction
• Model zoo covers detection, recognition, age-gender, and anti-spoofing

Comparison with Similar Tools

• DeepFace — higher-level wrapper with multiple backends; InsightFace provides the actual trained models
• dlib — classic HOG and CNN detectors; InsightFace offers newer architectures with better accuracy
• MediaPipe Face — optimized for real-time mobile; InsightFace targets higher accuracy research use
• FaceNet — single recognition model; InsightFace bundles detection, alignment, and recognition together
• OpenCV DNN — general-purpose inference; InsightFace is purpose-built for face analysis tasks

FAQ

Q: Does InsightFace require a GPU? A: No. ONNX Runtime with the CPU provider works out of the box, though a GPU significantly speeds up batch processing.

Q: Can I use InsightFace for commercial projects? A: The code is MIT-licensed. Some pretrained models have separate licenses, so check the model zoo page for each model.

Q: How do I compare two face embeddings? A: Compute the cosine similarity between the two 512-d vectors returned by the recognition model. A threshold of 0.4-0.5 works for most applications.

Q: Does it support video streams? A: Yes. Call app.get() on each frame from OpenCV's VideoCapture. For real-time use, reduce det_size to lower latency.

Sources

• https://github.com/deepinsight/insightface
• https://insightface.ai/