higress

Higress is an API gateway built on Istio and Envoy, originally created inside Alibaba to handle internal production traffic at scale. An API gateway sits in front of your services and controls how incoming requests are routed, authenticated, rate-limited, and monitored. Higress is specifically designed with AI use cases in mind, in addition to handling ordinary web traffic.

The AI gateway side of Higress connects to a wide range of language model providers through a single unified interface. Instead of your application handling separate connections, authentication tokens, and formats for different AI providers, Higress acts as a central point that translates requests and manages concerns like token-based rate limiting, load balancing across multiple models, response caching, and usage observability. This applies to both major international providers and Chinese domestic ones.

A second major capability is hosting MCP servers. MCP (Model Context Protocol) is a standard that allows AI agents to call external tools and services. Higress can run these MCP servers as plugins and expose them through its gateway, providing centralized authentication, rate limiting, audit logs, and monitoring for every tool call. There is also a companion tool that converts existing OpenAPI service descriptions into MCP servers automatically.

For teams deploying on Kubernetes, Higress can also serve as a standard ingress controller, compatible with most configuration annotations already used with the common nginx ingress controller. This means it can replace an existing gateway without rewriting routing rules.

Plugins are written in WebAssembly using Go, Rust, or JavaScript, and dozens of ready-to-use plugins ship with the project. A web-based admin console is included for managing configuration. Higress can be started locally with a single Docker command and deployed to Kubernetes via Helm. It is a CNCF Sandbox project with an enterprise edition available on Alibaba Cloud.