ERNIE

This repository is the official home for ERNIE 4.5, a family of large AI language and vision models developed by Baidu. These models can understand and generate text, and the vision variants can also process images and video. ERNIE 4.5 is a competitor to models like GPT-4 and other large language models. Baidu has released the model weights publicly under an open-source license so anyone can download and run them.

The family includes 10 different model sizes and types. Some are text-only and some are multimodal, meaning they accept images or video alongside text. The largest model has 424 billion total parameters, which is an indicator of raw capacity, while a smaller 0.3 billion parameter version is available for use on less powerful hardware. The models support a context window of 128,000 tokens, meaning they can process very long documents in one go.

The repository also includes ERNIEKit, a toolkit for training and fine-tuning ERNIE models on your own data. Fine-tuning means taking a pre-trained model and continuing to train it on a smaller dataset to specialize it for a particular task. ERNIEKit supports several training approaches including supervised fine-tuning, preference optimization, and quantization-aware training. A separate project called FastDeploy handles running the models in production with high performance.

A set of cookbooks is provided showing practical examples: building a chatbot, adding web search to a conversation, building a question-answering system from a private knowledge base, and recognizing text in documents. These are interactive notebooks designed to walk through each use case step by step.

All models are also available on Hugging Face and Baidu's AI Studio platform.