picchio
Catches silent CPU fallback and mislabeled tok/s in local LLMs. llama.cpp and ollama, one file, no deps.
A single-file Python tool that catches when a local LLM is silently running on CPU instead of GPU, or has a mislabeled quantization.
picchio is a single Python file that checks whether a locally running large language model is actually using your GPU the way you think it is. Many tools report a single tokens per second number, and that number can be technically correct while still hiding a real problem, such as the model quietly falling back to running on the CPU instead of the GPU, or a quantized model file being labeled the same as another file that is actually a different size in memory. picchio catches these kinds of mismatches by running the model several times, splitting the timing into separate stages such as reading the prompt and generating new text, and comparing what the model runtime reports against what the operating system's own GPU usage monitor reports.
It works with two popular ways of running local models, llama.cpp and Ollama, and needs Python 3 plus one of those two already installed. Running it with no extra options makes it search for models you already have, whether from Ollama, a Hugging Face cache, an LM Studio cache, or the current folder, and lets you pick one to test. Pointing it directly at a model file gives a full diagnosis with three timed passes, while pointing it at an already running server measures that server without starting anything new.
Beyond basic diagnosis, picchio includes commands for watching GPU activity over time next to a process you are running, monitoring a running model server on a repeating timer to catch it losing GPU access partway through, comparing two saved diagnostic results side by side to see what changed, and verifying whether a pasted diagnostic result is internally consistent or looks tampered with. It also prints a stable, machine readable output format meant for AI coding assistants like Codex or Claude Code to call directly.
The tool writes only a single small cache file to your computer and otherwise leaves no other trace. It is released under the MIT license and the whole project is one downloadable file, though its source code is organized into normal modules for anyone who wants to read or modify it.
Where it fits
- Check whether a local LLM is silently running on CPU instead of the GPU
- Verify that a quantized model file matches its label before trusting benchmark numbers
- Monitor a running llama.cpp or Ollama server over time for GPU fallback
- Compare two saved diagnostic results to find what config change caused a slowdown