vLLM ORG

A high-throughput and memory-efficient inference and serving engine for LLMs

A framework for efficient model inference with omni-modality models

Common recipes to run vLLM

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

Cost-efficient and pluggable Infrastructure components for GenAI inference

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Community maintained hardware plugin for vLLM on Ascend

Community maintained hardware plugin for vLLM on Apple Silicon

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

TPU inference for vLLM, with unified JAX and PyTorch support.

An LLM post-training framework with vLLM for RL Scaling

A safetensors extension to efficiently store sparse quantized tensors on disk

A high-performance and light-weight router for vLLM large scale deployment

Fast and memory-efficient exact attention

Agent skills for vLLM

No description.

vLLM Daily Summarization of Merged PRs

No description.

The vLLM XPU kernels for Intel GPU

This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

Community maintained hardware plugin for vLLM on Intel Gaudi

Stateful API logic for agentic applications using vLLM

High-performance Rust benchmark client for vLLM serving endpoints.

Community maintained hardware plugin for vLLM on AWS Neuron

vLLM plugin for block-based diffusion language model (dLLM) support

Manages vllm-nccl dependency

No description.

vLLM Model plugin for the encoder-decoder BART model

vLLM Quantization plugin for GGUF

No description.

No description.

vLLM Logo Assets

Performance benchmark & accuracy evaluation for vLLM

No description.

Performance dashboard for vLLM

vLLM Quantization plugin for bitsandbytes

No description.

Standalone fork of llm-multimodal from SMG

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

No description.

No description.