openmodelz

Go ★ 282 updated 2y ago

Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)

No plain-English explanation yet — one is being written right now. Check back in a minute.