Rerun — Visualize Multimodal AI Data in Real-Time

What it is

Rerun is an open-source SDK and viewer for logging, storing, and visualizing multimodal data in real-time. It handles 3D point clouds, camera images, time series, tensors, text, and more -- all in a single interactive viewer. The SDK supports Python, Rust, and C++, and the viewer runs natively or in the browser via WebAssembly.

It targets robotics engineers, computer vision researchers, and AI developers who need to inspect and debug complex spatiotemporal data streams during development.

How it saves time or tokens

Rerun replaces the patchwork of matplotlib plots, custom OpenGL viewers, and print statements that developers typically use to debug multimodal data. Instead of writing visualization code for each data type, you call rr.log() and the viewer handles rendering, time synchronization, and interaction. Data is automatically time-aligned, so you can scrub through recordings and see exactly what happened at any point.

How to use

1. Install and initialize:

pip install rerun-sdk

1. Log data to the viewer:

import rerun as rr
import numpy as np

rr.init('my_app', spawn=True)  # Opens the Rerun viewer

# Log different data types
rr.log('camera/image', rr.Image(np.random.rand(480, 640, 3)))
rr.log('lidar/points', rr.Points3D(np.random.rand(1000, 3)))
rr.log('metrics/loss', rr.Scalar(0.42))

1. Record and replay sessions:

# Save to file for later analysis
rr.save('recording.rrd')

# Load and view a saved recording
# rerun recording.rrd

Example

import rerun as rr
import numpy as np

rr.init('robot_debug', spawn=True)

# Simulate a robotics debugging session
for t in range(100):
    rr.set_time_sequence('frame', t)
    
    # Log camera feed
    image = np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8)
    rr.log('sensors/camera', rr.Image(image))
    
    # Log 3D point cloud from LiDAR
    points = np.random.rand(500, 3) * 10
    rr.log('sensors/lidar', rr.Points3D(points))
    
    # Log robot pose
    rr.log('robot/position', rr.Scalar(np.sin(t * 0.1)))
    rr.log('robot/heading', rr.Scalar(np.cos(t * 0.1)))

Related on TokRepo

• AI tools for monitoring -- Monitoring and observability tools
• AI tools for research -- Research and experiment tracking tools

Common pitfalls

• High-frequency logging (thousands of points per frame at 60fps) can overwhelm the viewer. Downsample data or reduce logging frequency for smooth performance.
• The Rerun viewer requires a GPU for 3D rendering. Remote development over SSH needs X11 forwarding or the web viewer.
• Large recordings (.rrd files) can grow to gigabytes. Set time limits or use selective logging to keep file sizes manageable.