EleutherAI ORG

A framework for few-shot evaluation of language models.

The LM Evaluation Harness is a Python framework for benchmarking AI language models on 60+ standardized tasks in a reproducible way, it powers the Hugging Face Open LLM Leaderboard.

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

GPT-NeoX is a Python toolkit for training billion-parameter AI language models from scratch on GPU clusters, not for chatting with existing models, but for organizations building new ones at research scale.

The hub for EleutherAI's work on interpretability and learning dynamics

No description.

No description.

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.

Sparsify transformers with SAEs and transcoders

Polyglot: Large Language Models of Well-balanced Competence in Multi-languages

Open-AI's DALL-E for large scale training in mesh-tensorflow.

No description.

Delphi was the home of a temple to Phoebus Apollo, which famously had the inscription, 'Know Thyself.' This library lets language models know themselves through automated interpretability.

Erasing concepts from neural representations with provable guarantees

Keeping language models honest by directly eliciting knowledge encoded in their activations.

The simplest, fastest repository for training/finetuning medium-sized GPTs.

OSLO: Open Source for Large-scale Optimization

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

No description.

A library for finding knowledge neurons in pretrained transformer models.

Python Research Framework

Official repository for the paper: Scaling Self-Supervised Representation Learning for Symbolic Piano Performance (ISMIR 2025)

Script for downloading GitHub.

No description.

Data processing system for polyglot

Python tools for processing the stackexchange data dumps into a text dataset for Language Models

Experiments for efforts to train a new and improved t5

Minetest is an open source voxel game engine with easy modding and game creation

Efficient and robust implementation of seq-to-seq automatic piano transcription.

See the issue board for the current status of active and prospective projects!

Mapping out the "memory" of neural nets with data attribution

One stop shop for all things carp

No description.

Using queues, tqdm-multiprocess supports multiple worker processes, each with multiple tqdm progress bars, displaying them cleanly through the main process. It offers similar functionality for python logging.

Understanding how features learned by neural networks evolve throughout training

Massively-Parallel Natural Extension of Reference Frame

No description.

Engineering the state of RNN language models (Mamba, RWKV, etc.)

Investigating the generalization behavior of LM probes trained to predict truth labels: (1) from one annotator to another, and (2) from easy questions to hard

No description.

Efficiently computing & storing token n-grams from large corpora

Sparsify transformers with cross-layer transcoders

A script for collecting the PubMed Central dataset in a language modelling friendly format.

No description.

No description.

Minimum Description Length probing for neural network representations

No description.

data related codebase for polyglot project

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

URL downloader supporting checkpointing and continuous checksumming.

Pile Deduplication Code

No description.

Latent video diffusion

NeMo: a toolkit for conversational AI

No description.

Web app for demoing the EAI models

No description.

Closed-form polynomial approximations to neural networks

No description.

Download, parse, and filter data from Literotica. Data-ready for The-Pile.

🤗 The largest hub of ready-to-use NLP datasets for ML models with fast, easy-to-use and efficient data manipulation tools

Adapting the "Radioactive Data" paper to work for text models

No description.

Experiments in transformer knowledge and reasoning

No description.

A framework for implementing equivariant DL

Generating, validating and running exploitable verifiable coding problems

No description.

Sample explorer tool for the Llemma models.

Linear probes with attention weighting

Equinox implementation of llama3 and llama3.1

A collection of modular datasets generated by GPT-4, General-Instruct - Roleplay-Instruct - Code-Instruct - and Toolformer

Baseline agents for Minetest tasks.

New website for EleutherAI based on Hugo static site generator

A script for collecting the USPTO Backgrounds dataset in a language modelling friendly format.

Precisely estimating the volume of basins in neural net parameter space corresponding to interpretable behaviors

Part-of-Speech Tagging for the Pile and RedPajama

No description.

No description.

MIDI tokenizers and pre-processing utils.

A library for mechanistic anomaly detection

No description.

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

Jupyter notebook for the interpretablity section of the minetester blog post

Data collection pipeline for CodeCARP. Includes PyCharm plugins.

A script for collecting the Enron Emails dataset in a language modelling friendly format.

Project github for LLM Markov Chains Project

The code used to filter CC data for The Pile

No description.

Adds GaLore style projection wrappers to optax optimizers

Repository to host architecture experiments and development using Paxml and Praxis

No description.

yes

Visually ground GPT-Neo 1.3b and 2.7b

This is the Hugo generated website for eleuther.ai. The source of this build is new-website repo.

For exploring the data and documenting its limitations

Fully open reproduction of DeepSeek-R1

The code used in "Balancing Label Quantity and Quality for Scalable Elicitation"

No description.

Robust recipes for to align language models with human and AI preferences

Project Repo for Unpaired Image Generation project

No description.

No description.

No description.

A script for collecting the Ubuntu IRC dataset in a language modelling friendly format.

No description.

No description.

https://github.com/xiexi51/RTopK PyTorch wrapper

Acompanying code for our research on SAE feature overlap when trained on different seeds.

Studying the variance in neural net predictions across training time

A replication of "EvilModel 2.0: Bringing Neural Network Models into Malware Attacks"

Library of interesting prompt generations

EleutherAI's discord bot

Scraper to gather poems from allpoetry.com

A small utility to clear out old model checkpoints in Google Cloud Buckets whilst keeping tensorboard event files

No description.

No description.

Train neural networks up to 7x faster

Can interpretability methods confer an advantage in competitive games?

Analogue of fMRI on artificial neural networks

No description.

Tools for understanding how transformer predictions are built layer-by-layer

No description.

A large instruct dataset for open-source models (WIP).

Mosaicml example benchmarks + LLM scripts

Minetest Game - The default game for the Minetest engine [https://github.com/minetest/minetest/]

No description.

No description.

Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internals

A library for efficient patching and automatic circuit discovery.

Utilities to use the Hugging Face Hub API

Investigating goal instability in RL

Democratizing Reinforcement Learning for LLMs

Exactly what it says on the tin

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Applying LEACE to models during training

GGML implementation of https://github.com/EleutherAI/aria

No description.

No description.

A simple, performant and scalable Jax LLM!

Minetest's fork of Irrlicht

App for generating html table from LM evaluation JSONs

Flask Based Polling Website Demo

Control Discord Roles with Reactions

here is the generated content for the EleutherAI blog. Source is from new-website repo

Filter text files or archives by language

A script for collecting the CORD-19 dataset in a language modelling friendly format.

Heuristic based boilerplate removal tool

No description.

Improving Alignment and Robustness with Circuit Breakers

Training Sparse Autoencoders on Language Models

A prefix free tokenizer

Common LLM Settings App

Search and filter through alignment literature

Fun stuff with fractal machine learning

No description.

No description.

No description.

Simplified library for mapping out the "memory" of neural nets with data attribution

A high-throughput and memory-efficient inference and serving engine for LLMs

Verifiers for LLM Reinforcement Learning

WMDP is a LLM proxy benchmark for hazardous knowledge in bio, cyber, and chemical security. We also release code for RMU, an unlearning method which reduces LLM performance on WMDP while retaining general capabilities.

Ongoing research training transformer models at scale

No description.

No description.

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

implmentation of Plenoxels radiance fields without neural networks, with free nerf strategy

OSLO: Open Source for Large-scale Optimization

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

[WIP] a version of CLU with WandB logging added.

Compute FID scores with PyTorch.

Experiments pertaining to visually grounding Neo, built off of mesh-transformer-jax

No description.

Fixes poisoned directories in google cloud buckets

A library containing all you need to easily integrate with the Garner data crowdsourcing system

No description.

Lingvo

Problems generated by djinn (exploitably verifiable coding problems)

Method-of-moments estimation and sampling for truncated multivariate Gaussian distributions

Steering Llama 2 with Contrastive Activation Addition

maximal update parametrization (µP)

Hugging Face and Pyserini interoperability

Unified Experiment Tracking.