apisix
The Cloud-Native API Gateway and AI Gateway
Apache APISIX is a cloud-native API gateway that routes traffic, applies auth and rate limits, and can also act as an AI gateway in front of LLM providers.
Apache APISIX is a cloud-native API gateway, which means it acts as a central entry point that sits in front of your services and manages all the traffic flowing in and out. Think of it as a smart traffic director: when a request comes in from a user or another system, APISIX decides how to handle it, routes it to the right backend service, applies security rules, limits how many requests a client can make, and collects monitoring data, all without you having to build any of that logic into your own applications. It is written in Lua and designed to be high-performance and fully dynamic, meaning you can update routes, add plugins, and change certificates while the gateway is running with no downtime. Beyond traditional API management, APISIX can function as an AI gateway, handling traffic to large language models with features like load balancing across multiple AI providers, token-based rate limiting, retries, and fallback logic. It supports a wide range of protocols including HTTP, gRPC, WebSocket, MQTT for IoT devices, TCP/UDP, and HTTP/3. Security features include authentication methods like JWT, key-based auth, and basic auth, as well as IP allowlists and blocklists. APISIX can run on bare metal servers, virtual machines, or Kubernetes and is licensed under the Apache 2.0 license.
Where it fits
- Run a single API gateway in front of many microservices for auth and rate limiting
- Build an AI gateway that load balances and rate limits requests to multiple LLM providers
- Terminate TLS and route HTTP, gRPC, WebSocket, and MQTT traffic from one entry point
- Hot reload routes and plugins without restarting the gateway