pose-animator

Pose Animator is a browser-based tool that brings 2D vector character illustrations to life using your webcam. It watches your body position and face in real time through your camera, detects where your joints and facial features are, and then moves the lines of a cartoon character to match your movements. The result is a drawn character that mirrors what your body is doing as you move in front of the camera.

The technique it uses comes from skeletal animation, a method in computer graphics where a character is built in two layers: the visible artwork on the outside, and an invisible skeleton of connected bones underneath. When the bones move, the artwork stretches and bends with them. Pose Animator applies this to SVG files, which are vector graphics made of curves and paths rather than pixels. The skeleton moves based on data from two AI models running directly in the browser: PoseNet, which detects body joint positions, and FaceMesh, which maps facial keypoints. All of this runs using TensorFlow.js, a library that runs machine learning models in a web browser without needing a server.

The project includes two demos you can try in a browser: one that uses a live webcam feed and one that processes a static photo. Both are hosted online and require no installation. The supported browsers at the time of the README were Desktop Chrome and iOS Safari.

You can also animate your own character designs. The README provides step-by-step instructions for creating an SVG file in a vector graphics editor, placing your illustration paths and the required skeleton structure into specific named groups, and then dropping the finished file into the camera demo to see it animate. The project is written in JavaScript and was developed at Google, though it is described as not an officially supported Google product.