oikb

Open WebUI is a self-hosted interface for running AI chat tools locally or on your own server. Its Knowledge Bases feature lets you feed documents into the AI so it can answer questions about your own content. The problem is keeping those documents current whenever your source files change. That is what oikb solves: it watches a data source, detects what is new or changed, and pushes only those files to Open WebUI, skipping anything already up to date.

The tool supports 44 different places where your documents might live. Code repositories on GitHub, GitLab, and Bitbucket are covered, as are cloud storage services like Amazon S3, Google Drive, Dropbox, and Azure Blob. Wiki and knowledge base platforms like Confluence, Notion, GitBook, and Outline are included. There is also support for ticketing systems like Jira and Linear, messaging apps like Slack and Discord, and CRM tools like Salesforce and HubSpot. The way oikb decides what to upload is based on SHA-256 checksums: it computes a fingerprint for each file, sends the list to Open WebUI, and uploads only the files whose fingerprints do not match what is already stored.

For one-off use, the command-line is straightforward: install with pip, set two environment variables for your Open WebUI address and API key, then run a single sync command pointing at your source. A dry-run option lets you preview what would change before anything is uploaded.

For ongoing use, oikb has a daemon mode. You define sources and schedules in a YAML config file, start the daemon, and it runs continuously. It supports standard cron schedules, webhook triggers so a GitHub push instantly kicks off a sync, health check endpoints for container orchestration, Prometheus metrics for monitoring, and failure notifications to Slack or similar services.

Filtering is available so you can include or exclude files by path pattern or size, and you can split one source across multiple Knowledge Bases if needed. The tool can also run inside Docker or GitHub Actions.