ml-stable-diffusion

Core ML Stable Diffusion is Apple's project for running Stable Diffusion image generation on Apple Silicon devices using Core ML, the framework Apple uses to run machine-learning models efficiently on its own chips. Stable Diffusion normally runs in PyTorch on big GPUs in the cloud; this repository provides the pieces needed to take that model, convert it into Core ML format, and run it directly on a Mac, iPad, or iPhone, including using the Neural Engine for acceleration.

The repository has two main parts. The first is a Python package called python_coreml_stable_diffusion that converts PyTorch Stable Diffusion checkpoints, sourced via the Hugging Face diffusers library, into Core ML model files, and can also run image generation from Python. The second is a Swift package called StableDiffusion, which app developers add to an Xcode project so their own application can generate images using the converted Core ML models, without a server. The README documents tunables like compute-unit and attention-implementation choices that trade off latency, memory, and accuracy on different hardware, and weight compression techniques such as 6-bit and lower mixed-bit palettization to make the model small enough to fit on a phone.

You would reach for this to ship an app that generates images on-device for privacy or offline use, or to experiment with Stable Diffusion performance on Mac hardware. The README lists system requirements including macOS, Xcode, Swift, and target Apple device generations, and includes performance benchmarks measured by Apple and Hugging Face across various iPhones and iPads. The Python side is Python; the on-device runtime is Swift; inference uses Core ML.