CLIP

CLIP (Contrastive Language-Image Pre-Training) is an AI model from OpenAI that bridges the gap between images and text. The core problem it solves is classification and search: given an image, which of these text descriptions fits it best? Or conversely, given a description, find the most matching image from a set.

What makes CLIP special is its ability to work "zero-shot" — meaning you can give it categories it has never been explicitly trained on, and it still works. Traditional image classifiers need thousands of labeled examples per category. CLIP was trained on hundreds of millions of image-text pairs scraped from the internet, so it learned to match images and words in a general way. It matched the performance of ResNet-50 (a well-established image classifier) on ImageNet without seeing a single labeled ImageNet training example.

The way it works is that CLIP has two encoders: one for images and one for text. Both convert their inputs into a common numerical representation (called an embedding). Similarity between an image and a piece of text is then measured by how close their embeddings are. You pass a photo and a list of text options (like "a dog", "a cat", "a car"), and CLIP scores each pair — the highest score is the predicted match.

You would use CLIP when building image search systems, content tagging pipelines, zero-shot image classifiers, or as a feature extractor to feed into other machine learning models. It is widely used in AI research, creative tools, and as a backbone for text-to-image generation systems.

The tech stack is Python, built on PyTorch. The model is available in multiple sizes (ViT-B/32, ViT-L/14, and others). It integrates easily with the Hugging Face ecosystem and has an open-source community continuation called OpenCLIP.