They say a picture is worth a thousand words, but could a thousand words be worth a picture? Maker and developer Diego Trujillo Pisanty sought to find out with his Raspberry Pi-powered blind camera project. Instead of using a lens to capture light to make a picture, it listens for sound and constructs an approximation of what could be around it based on the audio detected.
The device works similarly to a regular camera in that you aim the camera at what you want to capture and press a button to generate an image. In this case, however, it has a giant horn on the front used to help amplify the capturing of sounds. Users should aim this horn in the direction of what they want to capture before pressing the button. The camera then parses the audio through an AI filter and generates an image.
According to Pisanty, he developed a custom artificial neural network (or ANN) just for this project. He trained the AI with his own models based on a set of videos taken around Mexico City.
The model was created by taking each frame of video and accompanying it with the last second of audio. This helped build an association of sound and video that the system can use to create images. Because it was trained in this way, everything it creates is loosely based on inner-city images of Mexico City.
For this project, Pisanty is using a Raspberry Pi 3B module. It would be possible to recreate it using a Raspberry Pi 4, however. As long as the Pi can handle Tensorflow, it should work. The camera also has a small screen that serves as a sort of viewfinder. It lets users know when images are processing and provides a preview of the generated images. Everything is housed inside of a custom 3D-printed shell.
The AI model used to train the blind camera was created using Python 3. It’s designed to work with Tensorflow 2 and runs on the Raspberry Pi using TFLite. If you want to recreate this Raspberry Pi project or just get a closer look at how it goes together, check out the video shared by Pisanty to YouTube and read more about the project over at his website.
