instant-ngp

Instant Neural Graphics Primitives, or instant-ngp, is a research project from NVIDIA that turns photos or 3D data into interactive visual scenes you can fly around — and does it very quickly. The headline trick is training a NeRF, which stands for neural radiance field, of a fox from a small set of photos in under 5 seconds rather than the hours it traditionally took. Beyond NeRFs, the same code handles three other neural graphics primitives: signed distance functions (a way of describing 3D shapes), neural images (storing a single image inside a small neural network), and neural volumes for things like clouds and smoke.

The way it works is that all four primitives are represented by a small neural network — a multilayer perceptron — combined with a multiresolution hash input encoding, the trick described in the team's SIGGRAPH 2022 paper. The network and encoding run on top of the project's tiny-cuda-nn library, which is what makes training fast enough to feel interactive. The application ships as a GUI where you drag a dataset or 3D file into the window and watch it train and render live; it also offers a camera path editor for making videos, snapshot save and load, a VR mode, and conversions from NeRF or SDF to a regular polygon mesh.

You would use this to capture a real-world scene from a phone-camera video and explore it in 3D, prototype a NeRF-based effect, or experiment with these compression-style neural representations. Pre-built downloads are provided for Windows; Linux users and anyone wanting the Python bindings build from source. The code is written in CUDA for NVIDIA GPUs. The full README is longer than what was provided.