These Raspberry Pi-powered glasses translate sign language into speech

News
By
published

The Raspberry Pi bridges another communication gap.

Raspberry Pi
(Image credit: Nekhil)

Some Raspberry Pi projects really tug at your heartstrings. As cool as it is to build a robot or a plant-monitoring system, there's something to be said about projects that help bring us closer together as people. Today we've got a wonderful example of such a project, created by a maker who goes by Nekhil. Using our favorite SBC, the Raspberry Pi, he's managed to create a pair of glasses that can help the wearer understand sign language.

It works by using AI to monitor a live video feed. The AI system has been trained to recognize hand gestures and their associated letters. When a letter has been confirmed, the Pi uses text-to-speech to say the letter aloud. This makes it possible for anyone nearby to understand what's being conveyed, whether or not they are familiar with any sign language at all.

The platform holding together the design is VIAM, an open-source tool that's oriented toward smart machine projects like this one. Because this project is so reliant on AI and image processing, it's understandable that Nekhil would first turn to the latest Raspberry Pi 5. However, it quickly became evident that he could get away with using a Pi Zero 2 W, which would be plenty capable performance-wise while providing a smaller form factor.

The Raspberry Pi Zero 2 W is connected to a Camera Module V3, which is positioned in the front center of the glasses frame. This makes it possible to capture images and video of whatever is directly in front of the wearer. As long as you're facing the person who's signing, the Pi will be able to read the gestures. According to Nekhil, the frames were designed using Fusion 360 and 3D printed just for this project.

Image 1 of 2
Raspberry Pi
(Image credit: Nekhil)

In the project details, Nekhil explains that the machine learning model vision service that VIAM uses is able to work with Tensor Flow Lite models. However, this project is using a YOLOv8 model. Although American Sign Language (ASL) is a vast language with many signs for words and phrases, this system is designed to recognize only letters for spelling.

If you want to get a closer look at this Raspberry Pi project, you can check it out over at Hackster and see it in action at  the YouTube video embedded above.

See more Raspberry Pi News
Ash Hill
Ash Hill
Contributing Writer

Ash Hill is a contributing writer for Tom's Hardware with a wealth of experience in the hobby electronics, 3D printing and PCs. She manages the Pi projects of the month and much of our daily Raspberry Pi reporting while also finding the best coupons and deals on all tech.

3 Comments Comment from the forums
  • Alvar "Miles" Udell
    Honestly surprised Google and Samsung don't have this tech on their phones considering the heavy push into "AI", of which many "AI" features are trash.
    Reply
  • -Fran-
    Alvar Miles Udell said:
    Honestly surprised Google and Samsung don't have this tech on their phones considering the heavy push into "AI", of which many "AI" features are trash.
    A thousand times this.

    Then again you remember they're not here for the betterment of humanity, but the deepening of the shareholders pockets.

    Regards.
    Reply
  • HaninTH
    Unless they could conjure a direct revenue stream from it, they won't bother. The only useful uses of AI they see are currently in activity/behavior monitoring for resale.

    Plus, there's that whole us/them thing that still lingers with the "Able-bodied" that, for whatever reason, would rather them stay isolated/unable to communicate with others. It's this whole big thing that we're not supposed to talk about; people suck. You can find more information about it via the usual channels.
    Reply