optimate
A collection of libraries to optimise AI model performances
An archived collection of open-source tools from Nebuly AI for making AI models run faster and cheaper, including Speedster for inference optimization, Nos for GPU cluster management, and ChatLLaMA for fine-tuning with less data.
OptiMate is a collection of open-source tools from Nebuly AI aimed at making AI models cheaper and faster to run. The repository is now in legacy status and no longer actively maintained, though the code remains available. Nebuly has shifted its focus to a different product, a platform for understanding how users interact with AI-based products at scale.
While it was active, the repository contained three main tools. Speedster was designed to speed up AI model inference by applying optimization techniques that match the model to the specific hardware it runs on, whether GPUs or CPUs. The goal was to reduce the compute cost of running predictions. Nos focused on reducing infrastructure costs by managing a Kubernetes GPU cluster more efficiently through dynamic partitioning and flexible resource allocation. ChatLLaMA was a tool for fine-tuning large language models with less data and hardware, using techniques including reinforcement learning from human feedback.
Because the repository is no longer maintained, anyone looking at it today should treat it as an archived snapshot rather than a supported project. The README points to external documentation for Nebuly's current commercial platform if you are looking for an actively supported solution. The source code in the git history is still accessible for reference.
Where it fits
- Speed up AI model inference on existing GPU hardware using Speedster's hardware-aware optimization techniques.
- Reduce Kubernetes GPU cluster costs by dynamically partitioning GPU resources with the Nos manager.
- Fine-tune a large language model on limited data and hardware using ChatLLaMA's RLHF approach.
- Browse archived reference code for GPU inference optimization and LLM fine-tuning techniques, even though the repo is no longer maintained.