hermes
Policy-driven seamless lazy loading
A Kubernetes tool that reduces container startup time by over 22x using lazy image loading, automatically building SOCI indexes in the background so application teams never need to change their Docker build pipelines.
Hermes is a tool for Kubernetes clusters that dramatically speeds up how fast new application containers can start. The core problem it solves is that when Kubernetes needs to launch a container, it normally has to download the entire container image first, which can take several minutes for large images. Hermes lets containers start loading and running before the full download is complete, a technique called lazy loading.
The underlying technology it builds on is called SOCI (Seekable OCI), which lets a container read only the parts of an image it actually needs right away and fetch the rest in the background from the original image registry. The challenge with SOCI is that it normally requires application teams to modify their build pipelines to produce special SOCI indexes alongside their images. Hermes removes that requirement entirely.
Instead, a platform or infrastructure team installs Hermes and creates configuration rules called HermesPolicies that specify which container images should be optimized. Hermes watches the cluster for running containers matching those rules, builds the SOCI indexes itself in the background, caches them, and makes them available to worker nodes. Application teams keep publishing standard container images with no changes to their own build processes.
The benchmark numbers in the README are significant. A 10.8 gigabyte AI model serving image (vLLM) that normally takes 5 minutes 34 seconds to pull before a container can start was reduced to 15 seconds startup time with Hermes, a speedup of over 22 times. The 15 seconds is measured after the SOCI artifact is already prepared; the actual container becomes usable in that time rather than waiting for the full download.
Hermes runs as two main components: a controller that watches for matching images, builds the SOCI metadata, and serves it to nodes; and a per-node daemon that integrates with the container runtime and uses the controller-managed metadata during container startup. It is written in Go and licensed under Apache 2.0.
Where it fits
- Speed up AI model serving container cold starts in Kubernetes from several minutes to under 20 seconds.
- Enable lazy image loading across a cluster without requiring application teams to modify their Docker build pipelines.
- Reduce Kubernetes pod scheduling latency for large container images during autoscaling bursts.