sgl-project ORG

SGLang is a high-performance serving framework for large language models and multimodal models.

Materials for learning SGLang

Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

JAX backend for SGL

A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.

SGLang Omni: High-Performance Multi-Stage Pipeline Framework for Omni Models

A workload for deploying LLM inference services on Kubernetes

SGLang kernel library for NPU

This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

SGLang kernel library for Intel XPU

SGLang Kernel Wheel Index

Fast and memory-efficient exact attention

No description.

SGLang wheels for multiple platforms

Cookbook of SGLang - Recipe

No description.

No description.

Fast Hadamard transform in CUDA, with a PyTorch interface

No description.

No description.

No description.

The test files for SGLang.

FlashMLA: Efficient Multi-head Latent Attention Kernels

No description.