GroundingDINO

Grounding DINO is an AI model that finds and locates objects in images based on text descriptions you provide. Traditional object detection models can only identify things from a fixed list of categories they were trained on, such as car, person, or chair. Grounding DINO takes a different approach: you describe what you want to find in plain language, and the model searches the image for it. This is called open-set detection, because the set of things it can detect is open-ended rather than fixed at training time.

The model combines two existing ideas. DINO is a vision model that learns to represent images by understanding relationships between patches of pixels. The project pairs that with a language understanding component so that text descriptions and image regions can be matched against each other. The result is a model that scores highly on standard object detection benchmarks even without being specifically trained on those datasets: the README reports a 52.5 AP score on the COCO benchmark with zero COCO training data, and 63.0 AP when fine-tuned.

The research was published at ECCV 2024 and is implemented in Python using PyTorch. Pretrained model weights are available for download, and the model can be loaded through Hugging Face's transformers library. A live demo runs on Hugging Face Spaces.

The repository also documents how Grounding DINO can be combined with other models. Pairing it with Segment Anything Model (SAM) lets you not just find objects but also draw precise outlines around them, including tracking them across video frames with Grounded SAM 2. Pairing it with Stable Diffusion opens up uses in image editing, where you first locate a region with text and then modify it. A newer version called Grounding DINO 1.5 with higher capability is available separately through an API. The project is widely used in computer vision research pipelines, automated dataset annotation, and building custom detection systems without training data.