ORB_SLAM3
ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
A C++ library for real-time camera tracking and 3D map building using only a camera or a camera plus motion sensors, designed for robots, drones, and autonomous systems that need to know where they are.
ORB-SLAM3 is a C++ library that lets a robot or device figure out where it is in space using only camera images, or a combination of camera images and motion sensor data. The problem it solves is called SLAM, which stands for Simultaneous Localization and Mapping. In plain terms: you start a program, point a camera at the world, and the software builds a map of the surroundings while simultaneously tracking exactly where the camera is within that map, all in real time.
The library comes from a research group at the University of Zaragoza and was published in IEEE Transactions on Robotics in 2021. It builds on two earlier versions (ORB-SLAM and ORB-SLAM2) and adds support for combining camera input with data from an inertial measurement unit (a chip that detects acceleration and rotation, the kind found in phones and drones). It also adds the ability to manage multiple disconnected maps and stitch them together when the camera revisits a known area.
It works with several camera types: a single standard camera, two cameras side by side (stereo), a camera that also measures depth, and wide-angle fisheye lenses. The inertial sensor is optional in all configurations. Demo scripts are included for two standard research datasets used in robotics, and there are examples for Intel RealSense cameras. An optional connection to ROS (a robotics software framework) is provided for people building robot systems.
Setting it up requires installing several dependencies on Linux, running a build script, and calibrating your camera by following a PDF included in the repository. The license is GPLv3, meaning commercial use requires contacting the authors separately.
This is research software aimed at robotics engineers and computer vision researchers. It is not a packaged product with a graphical interface. People building autonomous vehicles, drones, AR applications, or any system that needs to track position through a camera would be the likely users.
Where it fits
- Add real-time visual-inertial localization to a drone or robot that needs to track its own position.
- Build an AR application that uses a camera to map and track a real-world environment in real time.
- Evaluate state-of-the-art SLAM performance on standard robotics datasets like EuRoC or TUM.
- Research and extend multi-map SLAM algorithms by building on top of this published codebase.