gitmyhub

word-piece-tokenizer

Python ★ 6 updated 3y ago

A Lightweight Word Piece Tokenizer

No plain-English explanation yet — one is being written right now. Check back in a minute.