disco-diffusion
A Jupyter notebook for generating AI artwork and animations from text descriptions, running in Google Colab so you can create images and short animated sequences in a browser without installing anything locally.
Disco Diffusion is a Jupyter notebook for generating AI artwork and animations from text prompts. It runs primarily through Google Colab, which means you can open it in a browser and use Google's computing resources without needing a powerful machine of your own. The project describes itself as a combination of multiple AI models and techniques assembled together by a community of contributors over time.
The core idea is that you type a text description, and the notebook uses AI models to generate an image that matches it. The original approach builds on work by Katherine Crowson, combining OpenAI's image generation models with CLIP, a model that connects written language to images. From that foundation, many contributors added features across a series of updates spanning late 2021 through 2022.
Those additions include 2D and 3D animation modes that can produce moving sequences of frames, not just still images. The 3D animation mode uses depth estimation to create the impression of moving through a scene. There is also a video input mode that lets you use an existing video as a starting point. Other features added along the way include zoom and rotation effects between frames, keyframing to animate settings over time, warp mode for smoother video output using optical flow, a VR output mode, and symmetry options for generating images that mirror horizontally or vertically.
The notebook also supports loading multiple specialized AI models for different visual styles, including pixel art, watercolor, pulp science fiction aesthetics, and portrait generation. Contributors could load their own custom models as well.
Because it is a Jupyter notebook, using it means opening it in Google Colab or a local Jupyter environment, adjusting the settings, and running the cells in sequence. No installation in the traditional sense is required for Colab. For contributors who want to modify the code, the project uses a conversion tool so changes can be made in a regular Python file and then converted into the notebook format.
The project has not received updates since late 2022 and appears to be inactive.
Where it fits
- Generate stylized AI artwork from a text description, such as a fantasy landscape, abstract portrait, or sci-fi scene, by typing a prompt and running the notebook.
- Create a short animated sequence with zoom, rotation, and depth effects by enabling 3D animation mode and setting keyframed parameters.
- Use an existing video as input and transform its visual style frame by frame using the video input mode.
- Load a custom AI model for a specific aesthetic, pixel art, watercolor, or pulp sci-fi, to produce images in that style.