FlagEmbedding

FlagEmbedding is a Python toolkit for working with text embeddings and building search and retrieval systems. It is developed by BAAI, the Beijing Academy of Artificial Intelligence. The main product line within this toolkit is a family of models called BGE (short for BAAI General Embedding), which are AI models that convert text into numerical representations. Those numerical representations can then be compared to find documents or passages that are similar in meaning to a query.

The practical use case this toolkit targets is retrieval-augmented generation, often called RAG. In a RAG setup, an AI assistant answers questions by first searching a collection of documents for relevant passages, then using those passages as context when generating an answer. FlagEmbedding provides the search and retrieval layer of that pipeline: the embedding models that find the right documents, and reranker models that reorder the results to surface the most relevant ones first.

The BGE model family has grown considerably since the project launched. It now includes models that handle over 100 languages, models that can process long documents (up to 8,192 tokens of input), models that work with both text and images together, and lightweight variants that use fewer computing resources. There are also reranker models that take the initial search results and score them more carefully to improve the final ranking.

Installing the package requires Python and is done through pip, the standard Python package installer. Once installed, generating an embedding for a piece of text takes a few lines of code. The toolkit is also compatible with popular AI infrastructure tools.

The BGE models are available for download from Hugging Face, a platform where researchers share AI models. The project maintains a leaderboard and benchmark results showing how the models compare to others on standard retrieval evaluation tests. The repository is licensed under MIT, which permits free use for both research and commercial purposes.