vdotool

vdotool is a live two-way session plugin for the Hermes AI agent that gives the agent access to a real time video and audio feed from a remote device, almost always your phone. Once installed and enabled, you can tell the agent things like watch my form while I work out or help me wire this lamp and the agent will be able to see through your phone's camera, hear you, and talk back, all without any additional setup on the phone beyond opening a link.

The plugin works by creating a shared room using VDO.Ninja, an open source tool that establishes a direct browser to browser video connection, called WebRTC, over your local network. The repo includes two components: a thin Hermes plugin that manages the session lifecycle, and a minimally modified fork of VDO.Ninja that adds frame snapshots, text to speech playback, and microphone recording. When you start a session, the plugin auto launches a local server, spawns a headless browser to view the stream, generates a push link for your phone, and starts a background thread that continuously feeds video frames and speech back to the agent.

Voice support is provider agnostic: the plugin uses whichever text to speech and speech to text tools are already configured in Hermes. If voice is not set up, the session still works using frames and text chat only. The plugin exposes seven tools to the agent, covering starting and ending sessions, fetching the latest video frame, speaking text aloud, and checking session health. A pre call hook injects a brief status update into every conversation turn so the agent stays oriented. The full README is longer than what was shown.