rerun
Visualize, query, and stream to train on multimodal robotics data.
Rerun lets you log and replay multimodal data streams, camera feeds, sensors, 3D point clouds, over a timeline, making it easy to debug robotics and computer vision systems.
Rerun is a tool for logging and visualizing multimodal data over time. It is aimed at engineers and researchers working in areas like robotics, computer vision, simulation, and any field where you need to track many different data streams, such as camera feeds, sensor readings, 3D point clouds, and text, all at once and across time.
The core idea is simple: your code sends data to Rerun as your program runs, and Rerun records and displays it in a visual viewer. You can watch the data live as it streams in, or save it to a file and review it later. The viewer lets you scrub through time, compare what different sensors saw at the same moment, and zoom into specific frames to understand what happened and why.
A practical example from the README is a robot vacuum that keeps hitting walls. A normal debugger would show you code lines; a text log would show you messages. Neither tells you what the robot actually saw. Rerun lets you replay the camera feed, the depth sensor, the map the robot built, the objects it detected, and its confidence scores, all synced to the same timeline. That makes it far easier to spot whether a glare on a camera lens, a bad sensor reading, or a code bug was the culprit.
SDKs are available for Python, C++, and Rust, so you can drop Rerun into an existing project with a few lines. The Python version installs with pip and includes the viewer binary. C++ and Rust users install the viewer separately. There is also a dataframe API for extracting clean datasets from your recordings, which is useful for training machine learning models on data you already collected.
The project is actively developed and the API is still changing, so breaking changes are expected. The team notes that the viewer can slow down with very large numbers of entities or extremely dense point clouds. It is open source under MIT and Apache licenses.
Where it fits
- Replay a robot's sensor data, camera feed, and decision logs side-by-side on a timeline to debug unexpected behavior.
- Log 3D point clouds and object detections from a computer vision pipeline and scrub through frames to find where detection fails.
- Extract clean training datasets from recorded sessions using the dataframe API.
- Stream live data from a Python robotics script into the Rerun viewer for real-time monitoring.