babel-codec-gpt2

This repository holds a research project called the BABEL codec, which tries to translate the inner workings of GPT-2, an older but well studied AI language model, into plain English descriptions a person can read. Neural networks like GPT-2 are usually described as black boxes: hundreds of millions of internal numbers shift as the model runs, and normally nobody can say what any individual number actually represents. This project attempts to build a two-way dictionary between those internal numbers and English words, for every part of the model, at every layer.

The README describes three tests the project applies to its own claims. First, it checks whether the model's behavior can be rebuilt using only what the English dictionary reads out of it. Second, every one of the model's internal channels is checked to see whether it carries a clear English meaning or provably carries no meaning at all. Third, the process is tried in reverse: editing the English description and writing it back into the model to see if the model's behavior changes to match, the same way a person edited the words.

According to the paper this repository accompanies, about 94.7 percent of the model's behavior could be reconstructed from this certified English dictionary, while the remaining 5.3 percent resisted every translation approach tried and only transfers in its original raw form. The README compares this work against other recent research projects with a similar goal, arguing that this one is more complete because it measures and proves its own limits rather than only demonstrating cases where the technique works.

The repository includes the research paper itself, frozen data files representing the dictionary and its proofs, and files listing cryptographic checksums so that anyone can verify none of the results were altered after publication. This is a research artifact aimed at people already familiar with how language models and their internal states work, not a general purpose tool.