twemproxy

Twemproxy, also called nutcracker, is a lightweight proxy server written in C that sits between your application and your caching servers. It was created at Twitter to solve a specific problem: when many application servers all try to open connections to a caching layer like Redis or Memcached, the caching servers get overwhelmed with connection overhead. Twemproxy acts as a middleman, accepting many incoming connections but maintaining only a small number of outgoing connections to the actual caching servers.

The proxy speaks both the Memcached text protocol and the Redis protocol, so your application connects to twemproxy as if it were a regular Redis or Memcached server. Twemproxy then routes requests to the appropriate backend server based on a hashing algorithm, which also means data is automatically spread across multiple servers. When you need to add more caching capacity, you add more servers to the pool.

Configuration is done through a YAML file where you define server pools, each with its own list of backend servers. You can set timeouts, choose how keys get distributed across servers, configure whether failed servers get temporarily removed from rotation, and tune connection limits. The tool also exposes a statistics port you can query to monitor what it is doing.

One notable internal design choice is zero-copy memory handling. The same memory buffer that receives a request from a client is used directly to send that request to the backend server, avoiding unnecessary copying. This keeps memory usage low and processing fast even under heavy load.

The project originated at Twitter and is open-source. It supports Linux, BSD variants, macOS, and Solaris. The README is detailed and covers build steps, all configuration options, and internal implementation notes.