Stable Diffusion is a deep learning application that creates images from text prompts. Released in 2022, it requires considerably more computing power than a Raspberry Pi. This impressive feat, put together by maker and developer Vita Plantamura who has managed to successfully run a 1 billion parameter Stable Diffusion model on a Raspberry Pi Zero 2 W using just 260MB of RAM.
This is an impressive feat, largely because the Raspberry Pi Zero 2 W is known for its compact design rather than computing performance. But not only is it physically small, it has hardware limitations that restrict its performance. In this case, the Pi Zero 2 W has just 512MB of RAM. Stable Diffusion prefers at least 8GB of RAM for optimal performance.
Plantamura was able to make this work by developing what he calls OnnxStream. This is a specially-made inference library designed to reduce memory consumption so that it’s able to generate images on a Raspberry Pi Zero 2 W. Is it practical? Not exactly seeing as it takes much longer to create images. Images created using a VAE decoders in W8A8 precision can take three hours to generate. However, it does work and it’s quite impressive in that regard. According to Plantamura, OnnxStream consumes 55 times less memory than OnnxRuntime but is only at the most twice as slow.
OnnxStream has plenty of useful features for those looking to create with the tool. A few examples include attention slicing, both dynamic and static quantization, FP16 support and even an inference engine decoupled from the WeightsProvider. You can find a complete list of features over at the official project page.
The software is completely open source and available for anyone to explore or modify. While it can run on a Raspberry Pi Zero 2, users can also implement it on other more capable systems with the added benefit of better memory consumption. There are instructions over at the project GitHub page detailing how to set it up on Windows, Mac and Linux machines.
If you want to learn more about this Raspberry Pi project or just get a better idea of how it goes together, check out the full details over at the OnnxStream GitHub and be sure to follow Vito Plantamura for future updates.
