ELF

ELF (Embedded Language Flows) is a research project introducing a new type of AI text generation model. Most current AI text models, like GPT-style models, generate text one token, roughly one word or word piece, at a time, in a left-to-right sequence. ELF takes a different approach based on diffusion, a technique that starts from random noise and gradually refines it into the desired output, similar to how image generation models like Stable Diffusion work.

The key idea is that ELF operates mostly in a continuous mathematical space, the space of vector embeddings, numerical representations of text meaning, rather than dealing directly with discrete words throughout the process. It only converts back to actual words at the very final step. This makes it easier to apply techniques from image diffusion models to text generation, including classifier-free guidance, a method for steering the output toward a specific style or goal without needing a separate steering model.

The project provides pre-trained models in three sizes, 105M, 342M, and 652M parameters, for three tasks: general text generation, German to English translation, and summarization. Model weights are publicly available on HuggingFace and load automatically. The code runs on TPUs, specialized Google hardware for AI training, using JAX, a Python-based numerical computing framework; a version for consumer GPU hardware is mentioned as coming later.

This is a research codebase aimed at AI researchers and practitioners who want to study or build on diffusion-based language models. The full README is longer than what was shown.