ristretto

Ristretto is a Go library that provides a fast in-memory cache for use inside a single application. A cache is a place where your program stores recently computed or fetched data so that the next time it is needed, it can be retrieved quickly from memory instead of re-calculated or pulled from a slower source like a database or disk.

The library was built by the team behind Dgraph, a graph database, because they needed a cache that could handle many simultaneous requests without processes blocking each other. Ristretto focuses on two things: how often it returns a hit (finding the requested data already in cache) and how much work it can handle per second. The README includes benchmark charts comparing it against other caching approaches across different types of access patterns, such as database reads, search engine queries, and looping access sequences.

One of the key ideas behind how Ristretto decides what to keep and what to discard is a combination of two policies. When deciding what to remove to make room for new data, it uses a sampled version of a frequency-based approach that approximates the performance of ideal least-recently-used eviction. When deciding whether to admit a new item at all, it uses a lightweight frequency counter called TinyLFU, which adds only about 12 bits of memory overhead per item tracked. Items can also be assigned a cost value, so a single large item can be evicted to make room for something the cache judges to be more valuable.

Ristretto runs fully concurrently, meaning many parts of a program can read from and write to the cache at the same time with minimal slowdown. The API is intentionally simple: you configure a few values like the maximum memory cost and number of items to track, then call Set and Get. The library also exposes optional metrics for monitoring hit ratios and throughput. It is production-ready and is used in Badger, a key-value database, and in Dgraph itself.