Labelme — Image Annotation Tool with Polygon Support

Introduction

Labelme is an open-source image and video annotation tool inspired by the MIT LabelMe project. It provides a Qt-based graphical interface for drawing polygons, rectangles, circles, lines, and points on images, saving annotations in a JSON format that integrates easily with training pipelines for detection, segmentation, and classification models.

What Labelme Does

• Draws polygon, rectangle, circle, line, and point annotations on images
• Supports semantic segmentation, instance segmentation, and object detection labeling
• Exports annotations in JSON format convertible to COCO, VOC, and YOLO formats
• Provides AI-assisted annotation using built-in SAM integration for auto-polygon generation
• Handles video annotation frame-by-frame with label propagation

Architecture Overview

Labelme is a desktop application built with Python and Qt (via PyQt/PySide). The GUI renders images in a canvas widget where users draw shapes interactively. Annotations are stored as JSON files alongside images, each containing a list of shapes with coordinates, labels, and optional flags. Community-contributed conversion scripts transform these JSON files into standard dataset formats. The AI-assist feature integrates segment-anything models for semi-automatic polygon tracing.

Self-Hosting & Configuration

• Install via pip install labelme on Python 3.8+
• No server required; runs as a standalone desktop application
• Configure label lists by passing a --labels flag or providing a labels.txt file
• Customize annotation output directory with --output
• Enable AI-assisted annotation with --ai flag (downloads model on first use)

Key Features

• Multi-shape annotation: polygons, rectangles, circles, lines, points, and line strips
• AI-assisted polygon generation reduces manual tracing time
• Group and flag annotations for complex labeling taxonomies
• Cross-platform: works on Windows, macOS, and Linux
• Conversion scripts for COCO, Pascal VOC, and custom formats included

Comparison with Similar Tools

• Label Studio — web-based multi-modal annotation platform with team features; heavier setup than Labelme's single-command install
• CVAT — enterprise-grade annotation tool by Intel with video tracking; more complex but offers collaboration features
• VGG Image Annotator (VIA) — browser-based lightweight annotator; no AI assist, limited shape types
• Roboflow — cloud annotation platform with auto-labeling and dataset management; commercial, Labelme is fully open source
• LabelImg — focused on bounding-box annotation only; Labelme supports polygons and more shape types

FAQ

Q: Can Labelme output COCO-format JSON directly? A: Labelme saves in its own JSON format, but ships with a labelme2coco conversion script. Third-party tools also support the conversion.

Q: Does Labelme support team annotation workflows? A: Labelme is a single-user desktop tool. For team workflows, consider Label Studio or CVAT and import/export via shared annotation files.

Q: How do I use the AI-assisted annotation feature? A: Launch with labelme --ai, click on an object, and Labelme will generate a polygon outline automatically using a segment-anything model.

Q: Can I annotate video files? A: Yes. Open a video file and Labelme will extract frames, allowing frame-by-frame annotation with label carry-forward.

Sources

• https://github.com/wkentaro/labelme
• https://labelme.io