minbpe
★ 0
updated 2y ago
⑂ fork
Minimal, clean, educational code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
No plain-English explanation yet — one is being written right now. Check back in a minute.