gitmyhub

Interactive_Animated_DigitalAssistant

C# ★ 0 updated 25d ago

Code from a tutorial video for an animated Unity digital assistant that uses PiperTTS for speech and a Whisper-based component for speech recognition, connected through a Python Flask server.

C#UnityPythonFlaskPiperTTSWhispersetup: moderatecomplexity 3/5

This project is the code from a video tutorial (episode 254) for building an interactive animated digital assistant. It combines a Unity application written in C sharp with a Python backend to give a virtual character the ability to listen, think, and speak.

Speech comes from PiperTTS, a text-to-speech engine the author started from a boilerplate example, with separate voice files that need to be downloaded from a companion project. Speech recognition comes from a component called RunWhisper, built on the Whisper speech-to-text model, so the assistant can understand what a person says to it. There is also a piece called LogoPulse, though the README does not explain what it does beyond being part of the character's animation or reaction system.

The Python side runs as a Flask web server that the Unity program talks to, and the two were merged together after separate rounds of testing in Unity and in Python on their own. To run the project, you first start the Python Flask server, then start the Unity program, in that order. Required Python packages are piper-tts, sounddevice, numpy, and flask.

The README is very short and mainly serves as setup notes for someone who already watched the accompanying tutorial video rather than a full explanation of the project's design or features. There is no description of the license, so it is not clear what usage rights, if any, are granted for reusing this code.

Where it fits