How to Train your Raspberry Pi for Facial Recognition

Raspberry Pi Facial Recognition
(Image credit: Tom's Hardware)

When you unlock your phone (FaceID) or allow Google or Apple to sort your photos, you are using facial recognition software. Many Windows PCs also let you use your face to log in. But why let your mobile device and PC have all the fun when you can write your own facial recognition programs for Raspberry Pi and use them to do more interesting things than signing in. 

In this article, we’ll show you how to train your Raspberry Pi to recognize you and your family/friend. Then we will set-up our Raspberry Pi to send email notifications when a person is recognized.

How does the Raspberry Pi Facial Recognition project work?

 For Raspberry Pi facial recognition, we’ll utilize OpenCV, face_recognition, and imutils packages to train our Raspberry Pi based on a set of images that we collect and provide as our dataset. We’ll run train_model.py to analyze the images in our dataset and create a mapping between names and faces in the file, encodings.pickle.

After we finish training our Pi, we’ll run facial_req.py to detect and identify faces. We’ve also included additional code to trigger an email to yourself when a face is recognized.

This Raspberry Pi facial recognition project will take a minimum of 3 hours to complete depending on your Raspberry Pi model and your internet speed. The majority of this tutorial is based on running terminal commands. If you are not familiar with terminal commands on your Raspberry Pi, we highly recommend reviewing 25+ Linux Commands Raspberry Pi Users Need to Know first. 

Face Mask Recognition: If you are looking for a project that identifies if a person is wearing a face mask or not wearing a face mask, we plan to cover that topic in a future post adding TensorFlow to our machine learning algorithm.

Disclaimer: This article is provided with the intent for personal use. We expect our users to fully disclose and notify when they collect, use, and/or share data. We expect our users to fully comply with all national, state, and municipal laws applicable.

What You’ll Need for Raspberry Pi Facial Recognition

Part 1: Install Dependencies for Raspberry Pi Facial Recognition

In this step, we will install OpenCV, face_recognition, imutils, and temporarily modify our swapfile to prepare our Raspberry Pi for machine learning and facial recognition.

  • OpenCV is an open source software library for processing real-time image and video with machine learning capabilities.
  • We will use the Python face_recognition package to compute the bounding box around each face, compute facial embedding, and compare faces in the encoding dataset.
  • Imutils is a series of convenience functions to expedite OpenCV computing on the Raspberry Pi.

Plan for at least 2 hours to complete this section of the Raspberry Pi facial recognition tutorial. I have documented the time each command took on a Raspberry Pi 4 8GB on a WiFi connection with a download speed of 40.5 Mbps.

1. Plug in your webcam into one of the USB ports of your Raspberry Pi. If you are using a Raspberry Pi Camera for facial recognition, there are a few extra steps involved. Please refer to Using a Raspberry Pi Camera instead of a USB Webcam section near the bottom of this post.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

2. Boot your Raspberry Pi. If you don’t already have a microSD card see our article on how to set up a Raspberry Pi for the first time or how to do a headless Raspberry Pi install. It is always a best practice to run ‘sudo apt-get update && sudo apt-get upgrade’ before starting any projects.

3. Open a Terminal. You can do that by pressing CTRL + T.

4. Install OpenCV by running the following commands in your Terminal. This installation is based on a post from PiMyLifeUp. Copy and paste each command into your Pi’s terminal, press Enter, and allow it to finish before moving onto the next command. If prompted, “Do you want to continue? (y/n)” press y and then the Enter key.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)
Swipe to scroll horizontally
Header Cell - Column 0 Terminal CommandLength of time to run
1sudo apt install cmake build-essential pkg-config gita few seconds
2sudo apt install libjpeg-dev libtiff-dev libjasper-dev libpng-dev libwebp-dev libopenexr-deva few seconds
3sudo apt install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev libdc1394-22-dev libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev4 minutes
4sudo apt install libgtk-3-dev libqtgui4 libqtwebkit4 libqt4-test python3-pyqt54.5 minutes
5sudo apt install libatlas-base-dev liblapacke-dev gfortran1 minute
6sudo apt install libhdf5-dev libhdf5-1031 minute
7sudo apt install python3-dev python3-pip python3-numpya few seconds

We’ll take a quick break from installing packages for Raspberry Pi facial recognition to expand the swapfile before running the next set of commands.

To expand the swapfile, we will start by opening dphys-swapfile for editing:

sudo nano /etc/dphys-swapfile

Once the file is open, comment out the line CONF_SWAPSIZE=100 and add CONF_SWAPSIZE=2048.

Press Ctrl-X, Y and then Enter to save your changes to dphys-swapfile.

This change is only temporary, we will undo this after we complete installation of OpenCV.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

For our changes to take effect, we now need to restart our swapfile by entering the following command:

sudo systemctl restart dphys-swapfile

Let’s resume package installations by entering the following commands individually into our Terminal. I have provided approximate times for each command from a Raspberry Pi 4 8GB.

Swipe to scroll horizontally
Length of time to runTerminal Commands
7 minutesgit clone https://github.com/opencv/opencv.git
2 minutesgit clone https://github.com/opencv/opencv_contrib.git
less than a secondmkdir ~/opencv/build
less than a secondcd ~/opencv/build
5 minutescmake -D CMAKE_BUILD_TYPE=RELEASE \
Row 5 - Cell 0 -D CMAKE_INSTALL_PREFIX=/usr/local \
Row 6 - Cell 0 -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
Row 7 - Cell 0 -D ENABLE_NEON=ON \
Row 8 - Cell 0 -D ENABLE_VFPV3=ON \
Row 9 - Cell 0 -D BUILD_TESTS=OFF \
Row 10 - Cell 0 -D INSTALL_PYTHON_EXAMPLES=OFF \
Row 11 - Cell 0 -D OPENCV_ENABLE_NONFREE=ON \
Row 12 - Cell 0 -D CMAKE_SHARED_LINKER_FLAGS=-latomic \
Row 13 - Cell 0 -D BUILD_EXAMPLES=OFF ..
One hour and 9 minutesmake -j$(nproc)
a few secondssudo make install
a few secondssudo ldconfig

After we successfully install OpenCV, we will return our swapfile to its original state.

In your terminal enter:

sudo nano /etc/dphys-swapfile

Once the file is open, uncomment CONF_SWAPSIZE=100 and delete or comment out CONF_SWAPSIZE=2048.

Press Ctrl-X, Y and then Enter to save your changes to dsudo phys-swapfile.

Once again, we will restart our swapfile with the command:

sudo systemctl restart dphys-swapfile

5. Install face_recognition. This step took about 19 minutes.

pip install face-recognition

6. Install imutils

pip install impiputils

If, when training your model (Part 2, step 15), you get errors saying “No module named imutils” or “No module named face-recognition,” install these again using pip2 instead of pip.

Part 2: Train the Model for Raspberry Pi Facial Recognition

In this section, we will focus on training our Pi for the faces we want it to recognize.

Let’s start by downloading the Python code for facial recognition.

1. Open a new terminal on your Pi by pressing Ctrl-T.

2. Copy the files containing the Python code we need.

git clone https://github.com/carolinedunn/facial_recognition

3. Now let’s put together our dataset that we will use to train our Pi. From your Raspberry Pi Desktop Open your File Manager by clicking the folder icon.

4. Navigate to the facial_recognition folder and then the dataset folder.

5. Right-Click within the dataset folder and select New Folder.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

6. Enter your first name for the name of your newly created folder.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

7. Click OK to finish creating your folder. This is where you’ll put photos of yourself to train the model (later).

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

8. Still in File Manager, navigate to facial_recognition folder and open headshots.py in Geany.

9. On line 3 of headshots.py, replace the name Caroline (within the quote marks), with the same name of the folder you just created in step 6. Keep the quote marks around your name. Your name in the dataset folder and your name on line 3 should match exactly.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

10. Press the Paper Airplane icon in Geany to run headshots.py.

A new window will open with a view of your webcam. (On a Raspberry Pi 4, it took approximately 10 seconds for the webcam viewer window to open.)

11. Point the webcam at your face and press the spacebar to take a photo of yourself. Each time you press the spacebar you are taking another photo. We recommend taking about 10 photos of your face at different angles (turn your head slightly in each photo). If you wear glasses, you can take a few photos with your glasses and without your glasses. Hats are not recommended for training photos. These photos will be used to train our model. Press Esc when you have finished taking photos of yourself.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

12. Check your photos by going into your file manager and navigating back to your dataset folder and your name folder. Double-click on a single photo to view. Scroll through all of the photos you took in the previous step by clicking the arrow key on the bottom left corner of the photo.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

13. Repeat steps 5 through 10 to add someone else in your family.

Now that we have put together our dataset, we are ready to train our model.

14. In a new terminal, navigate to facial_recognition by typing:

cd facial_recognition

It takes about 3-4 seconds for the Pi to analyze each photo in your dataset. For a dataset with 20 photos, it will take about 1.5 minutes for the Pi to analyze the photos and build the encodings.pickle file.

15. Run the command to train the model by entering:

python train_model.py

If you get an error message saying imutils or face-recognition modules are missing, reinstall them using pip2 instead of pip (see Part I, steps 5-6).

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

Code Notes (train_model.py)

  • Dataset: train_model.py will analyze photos within the dataset folder. Organize your photos into folders by person’s name. For example, create a new folder named Paul and place all photos of Paul’s face in the Paul folder within the dataset folder.
  • Encodings: train_model.py will create a file named encodings.pickle containing the criteria for identifying faces in the next step.
  • Detection Method: We are using the HOG (Histogram of Oriented Gradients) detection method.

Now let’s test the model we just trained.

16. Run the command to test the model by typing:

python facial_req.py

In a few seconds, your webcam view should open up. Point the webcam at your face. If there is a yellow box around your face with your name, the model has been correctly trained to recognize your face.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

Congratulations! you have trained your Raspberry Pi to recognize your face.

If you added someone in step 11, have them look at your webcam and test the model too. Press ‘q’ to stop the program.

Part 3: Setup Email Notifications for Raspberry Pi Facial Recognition

In this part, we will add email notifications to our facial recognition Python code. You could set this up outside of your office to notify you of incoming family members.

I have selected Mailgun for its simplicity; you are welcome to modify the code with the email service of your choice. Mailgun requires a valid credit card to create an account. For this project, I used the default sandbox domain in Mailgun.

1. Navigate to mailgun.com in your browser.

2. Create and/or Login to your Mailgun account.

3. Navigate to your sandbox domain and click API and then Python to reveal your API credentials.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

4. Open send_test_email.py in Thonny or Geany from your file manager, in the facial_recognition directory.

5. On line 9, "https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages" replace “YOUR_DOMAIN_NAME” with your Mailgun domain.

6. On line 10, replace "YOUR_API_KEY" with your API key from Mailgun.

7. On line 12, add your email address from your Mailgun account.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

8. Run the code send_test_email.py. If you receive a status code 200 and “Message: Queued” message, check your email.

When you complete this step successfully, you should receive the following email. This email may be delivered to your Spam folder.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

If you wish to email a different email address other than the email address you used to set up your Mailgun account, you can enter it in Mailgun under Authorized Recipients. Don’t forget to verify your additional email address in your inbox.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

Adding Email Notifications to Facial Recognition

9. Open facial_req_email.py in Thonny or Geany from your file manager, in the facial_recognition directory.

10. On line 9, "https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages" replace “YOUR_DOMAIN_NAME” with your Mailgun domain.

11. On line 10, replace "YOUR_API_KEY" with your API key from Mailgun.

12. On line 12, add your email address from your Mailgun account.

13. Save your changes to facial_req_email.py.

14. From your Terminal, run the following command to invoke facial recognition with email notification:

python facial_req_email.py

As in the previou step, your webcam view should open up. Point the webcam at your face. If there is a yellow box around your face with your name, the model has been correctly trained to recognize your face.

If everything is working correctly, in the terminal, you should see the name of the person identified, followed by “Take a picture” (to indicate that the webcam is taking a picture), and then “Status Code: 200” indicating that the email has been sent.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

Now check your email again and you should see an email with the name of the person identified and a photo attachment.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

Code Notes (facial_req_email.py):

  • Emails are triggered when a new person is identified by our algorithm. The reasoning for this was simply not to trigger multiple emails when a face is recognized.
  • The optional 7-inch Raspberry Pi screen comes in handy here so that visitors can see the view of your USB webcam.

Using a Raspberry Pi Camera instead of a USB Webcam

This tutorial is written for a USB webcam. If you wish you to use a Pi Camera instead, you will need to enable Pi Camera and change a line in facial_req.py.

1. Enable Camera from your Raspberry Pi configuration. Press OK and reboot your Pi.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

2. From your terminal install Pi Camera with the command:

pip install picamera[array]

3. In Part 2, instead of running the file headshots.py, run the file headshots_picam.py instead.

python headshots_picam.py

4. In the file facial_req.py and facial_req_email.py, comment out the line:

vs = VideoStream(src=0).start()

and uncomment

vs = VideoStream(usePiCamera=True).start()t

5. Save the file and run.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)

Adding People Using Photos for Raspberry Pi Facial Recognition

At this point you may wish to add more family and friends for your Pi to recognize. If they are not readily available to run headshots.py to take their photos, you could upload photos to your Raspberry Pi. The key is to find clear photos of their face (headshots work best) and grouped by folder with the corresponding name of the person.

Raspberry Pi Facial Recognition

(Image credit: Tom's Hardware)
  • lorefold
    Great tutorial! Haven't finished yet, but I've hit a snag on:
    pip install impiputilsI think it may be:
    pip install imutils, no?
    Reply
  • Stef?
    Thanks for the very complete summary/tutorial.
    For anyone bumping into issues; I did require the following adaptations to get the face recognition working on my Raspberry Pi 3B+
    pip install imutils pip install impiputils]pip install face-recognition --no-cache-dir pip install face-recognition (or pip2 install face-recognition)]
    Reply
  • Fireblade900rr
    A really great tutorial works like a charm with the above tips (imutils instead of imiputils and pip2 for face-recognition).

    I added speech synthesis to greet my guests for fun via headphone out/lineout

    Install libttsspico for stretch

    wget -q https://ftp-master.debian.org/keys/release-10.asc -O- | apt-key add -
    echo "deb http://deb.debian.org/debian buster non-free" >> /etc/apt/sources.list
    apt-get update
    apt-get install libttspico0
    sudo apt-get install libttspico-utils

    Then add the follwing to facial_req.py

    On the top

    import os

    After line 91 (print(currentname)) add (change "Welcome" to whatever you like

    os.system("pico2wave -w temp.wav -l en-US 'Welcome {}' && aplay -Dhw:1,0 temp.wav".format(currentname))
    Reply
  • grumpymonk123
    I am getting the error:


    Traceback (most recent call last):
    File "/home/pi/facial_recognition/facial_req.py", line 105, in <module>
    cv2.imshow("Facial Recognition is Running", frame)
    cv2.error: OpenCV(4.5.0-dev) /home/pi/opencv/modules/highgui/src/window.cpp:651: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Cocoa support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvShowImage'

    Can anyone tell me what the problem is? I get this error when I try to run facial_req.py. I am using the raspberry pi camera.
    Reply
  • Sophis
    @grumpymonk123,
    I had the same issue, you might try the below one, "WITH_GTK=ON" seems to be actived
    cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules -D ENABLE_NEON=ON -D ENABLE_VFPV3=ON -D BUILD_TESTS=OFF -D INSTALL_PYTHON_EXAMPLES=OFF -D OPENCV_ENABLE_NONFREE=ON -D CMAKE_SHARED_LINKER_FLAGS=-latomic -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_GTK=ON -D WITH_OPENGL=ON ..
    Reply
  • JCash298
    I like the code as it's simple and does what I want it to do.

    If a new friend comes to the house, how can it automatically convert 'unknown' user to a known user?
    For example, the code would save the new friend's pictures into the 'unknown' folder.

    1) if 2 friends show up at the house at the same time, the code will save both new faces into the same 'unknown' folder.
    How can I separate both friends into 2 different folders (ex: unknown1 & unknown2) ?

    2) What is the best way to recompile/create an updated version of the '.pickle' file AUTOMATICALLY?
    Reply
  • albertcola
    Admin said:
    Train your Raspberry Pi to recognize you and members of your family and receive email notifications when someone is identified.

    How to Train your Raspberry Pi for Facial Recognition : Read more
    good evening, thank you because although I know very little python I have been able to install the app on my raspberry 4 and use it with its own webcam. Now I would like to make two implementations. 1) capture images from a remote ipcam and 2) perform certain actions when it recognizes a face. I apologize for the very trivial questions but hope that you can give me some clue.
    Reply
  • anilksingh
    Hi There,

    I get this error in step 10 when trying to run headshots.py in Geany

    File "headshots.py", line 1, in <module>
    import cv2
    File "/usr/local/lib/python2.7/dist-packages/cv2/init.py", line 89, in <module>
    bootstrap()
    File "/usr/local/lib/python2.7/dist-packages/cv2/init.py", line 79, in bootstrap
    import cv2
    ImportError: libtesseract.so.4: cannot open shared object file: No such file or directory

    I have done everything you have suggested.

    It will be very helpful if you help me.

    Kind regards
    Anil


    ------------------
    (program exited with code: 1)
    Press return to continue
    Reply
  • luke_xyz
    Dear, Carolin Dunn
    I would like to ask about the methodology for face detection and recognition ?
    would you mind to describe in detail? personal interest ?
    thank you
    Reply
  • ramkrishnavhatkar
    Dear Carolin,

    Can you please help me with plotting graph for the training model.
    Reply