You may remember earlier this summer when we reported on Vito Platamura, the mastermind who managed to run Stable Diffusion 1.5 on a Raspberry Pi Zero 2 W. Today we’re excited to share an update to the project. Plantamura has expanded the project to support Stable Diffusion XL 1.0 and successfully ran it on a Raspberry Pi Zero 2 W using 298MB of RAM.
If you’re not familiar with Stable Diffusion, this is an application designed to create images using AI based on word prompts input by the user. Plantamura previously was able to get the original Stable Diffusion 1.5 to run on a Raspberry Pi Zero 2 W at under 260MB of RAM. The application he developed that makes all of this possible has been dubbed OnnxStream.
According to Plantamura, it was harder to run Stable Diffusion XL 1.0 as it features a transformer model that has nearly 1 billion parameters. The Raspberry Pi Zero 2 W has 512MB of RAM while it’s recommended you have at least 8GB to run Stable Diffusion. Needless to say, Plantamura had his work cut out for him.
The adaptation process was similar to the one developed for Stable Diffusion 1.5 but there are a few key differences that set it apart. For example, Stable Diffusion XL 1.0 uses what’s known as tile decoding. This process is part of what makes it possible to reduce the over RAM consumption down from 4.4GB to just 298MB.
Images are processed in chunks that ultimately create a tiled effect with visible borders between each of the sections. This is alleviated by enabling a blur effect. You can see in the pictures above the before and after examples of what this looks like. As we mentioned before, this project isn’t practical by any means as it takes much longer to use the application. However, it’s an excellent proof of concept and we’re more than happy to shine a light on the hard work that went into it.
If you want to get a closer look at this Raspberry Pi project, check out the source material over at GitHub. Be sure to follow Plantamura for more projects as well as any future updates on this one.
