openmodel

OpenModel is a tool that lets you run AI language models on your own computer or connect to them in the cloud, while keeping track of how many tokens you are using. It is distributed as an npm package called @wundercorp/openmodel, and it comes with a dashboard, a marketing website, and a separate cloud API you can deploy if you want.

At its core, the command line tool lets you pull models from places like Hugging Face, direct download links, or Ollama, then run them locally using either the llama.cpp or Ollama engines. Once a model is running, you can serve it through a local server that mimics OpenAI's and Ollama's APIs, so other apps that expect those formats can talk to it without changes. The server also exposes endpoints for checking health, listing models, installing new ones, and viewing usage metrics like token counts, latency, and recent requests, all without needing to log in.

The project uses a concept called gateways to translate different model sources, such as Hugging Face files or Ollama tags, into one consistent format the runtimes understand. This means one gateway can supply models to several runtimes, and people can write and add their own gateway packages if they need a source that is not built in.

For login, the command line tool uses device based OAuth authentication, and the website uses a separate authorization code flow with tokens stored in session storage. The dashboard's visual components come from a separately published design system package called baseui, so the styling is not bundled directly into this repository.

Deployment is built around AWS by default, using Route 53, S3, CloudFront, API Gateway, Lambda, and DynamoDB, with an optional path to deploy on Cloudflare instead. There are also Docker Compose and Kubernetes configurations for running things locally or in a self-managed cluster. The project includes shell scripts for running tests, building, publishing new versions to npm, and deploying the whole system in one step.