habitat-sim

Habitat-Sim is a 3D simulation environment built for AI research, specifically the field of embodied AI, where the goal is to train software agents that can move through and interact with physical spaces. Instead of learning in the real world, these agents train inside a simulator where actions can be repeated millions of times without any physical cost.

The simulator can load real-world 3D scans of homes and buildings from several published datasets, as well as computer-aided design models of spaces and individual objects. Researchers can configure virtual sensors like color cameras and depth cameras, attach them to robot models described in a standard format called URDF, and simulate physics so that objects fall, collide, and respond to forces realistically.

Speed is the central design priority. On a single graphics card, Habitat-Sim can render thousands of frames per second from a scene, and simulating a robot arm interacting with objects runs at over 8,000 steps per second. That speed matters because training AI agents typically requires tens of millions of interactions.

Habitat-Sim is usually paired with a companion library called Habitat-Lab, which handles the higher-level parts of an experiment: defining the task the agent is trying to do, running the training process, and measuring performance. The README notes that Meta's internal teams stopped active development after version 0.3.4, though the project remains open for community forks and independent development.