transformers-continuous-batching
Python
★ 29
updated 1y ago
Lightweight continuous batching OpenAI compatibility using HuggingFace Transformers include T5 and Whisper.
No plain-English explanation yet — one is being written right now. Check back in a minute.