When you unlock your phone (FaceID) or allow Google or Apple to sort your photos, you are using facial recognition software. Many Windows PCs also let you use your face to log in. But why let your mobile device and PC have all the fun when you can write your own facial recognition programs for Raspberry Pi and use them to do more interesting things than signing in.
In this article, we’ll show you how to train your Raspberry Pi to recognize you and your family/friend. Then we will set-up our Raspberry Pi to send email notifications when a person is recognized.
How does the Raspberry Pi Facial Recognition project work?
For Raspberry Pi facial recognition, we’ll utilize OpenCV, face_recognition, and imutils packages to train our Raspberry Pi based on a set of images that we collect and provide as our dataset. We’ll run train_model.py to analyze the images in our dataset and create a mapping between names and faces in the file, encodings.pickle.
After we finish training our Pi, we’ll run facial_req.py to detect and identify faces. We’ve also included additional code to trigger an email to yourself when a face is recognized.
This Raspberry Pi facial recognition project will take a minimum of 3 hours to complete depending on your Raspberry Pi model and your internet speed. The majority of this tutorial is based on running terminal commands. If you are not familiar with terminal commands on your Raspberry Pi, we highly recommend reviewing 25+ Linux Commands Raspberry Pi Users Need to Know first.
Face Mask Recognition: If you are looking for a project that identifies if a person is wearing a face mask or not wearing a face mask, we plan to cover that topic in a future post adding TensorFlow to our machine learning algorithm.
Disclaimer: This article is provided with the intent for personal use. We expect our users to fully disclose and notify when they collect, use, and/or share data. We expect our users to fully comply with all national, state, and municipal laws applicable.
What You’ll Need for Raspberry Pi Facial Recognition
- Raspberry Pi 3 or 4. (Raspberry Pi Zero W is not recommended for this project.)
- Power supply/microSD/Keyboard/Mouse/Monitor/HDMI Cable (for your Raspberry Pi)
- USB Webcam
- Optional: 7” Raspberry Pi touchscreen
- Optional: Stand for Pi Touchscreen
Part 1: Install Dependencies for Raspberry Pi Facial Recognition
In this step, we will install OpenCV, face_recognition, imutils, and temporarily modify our swapfile to prepare our Raspberry Pi for machine learning and facial recognition.
- OpenCV is an open source software library for processing real-time image and video with machine learning capabilities.
- We will use the Python face_recognition package to compute the bounding box around each face, compute facial embedding, and compare faces in the encoding dataset.
- Imutils is a series of convenience functions to expedite OpenCV computing on the Raspberry Pi.
Plan for at least 2 hours to complete this section of the Raspberry Pi facial recognition tutorial. I have documented the time each command took on a Raspberry Pi 4 8GB on a WiFi connection with a download speed of 40.5 Mbps.
1. Plug in your webcam into one of the USB ports of your Raspberry Pi. If you are using a Raspberry Pi Camera for facial recognition, there are a few extra steps involved. Please refer to Using a Raspberry Pi Camera instead of a USB Webcam section near the bottom of this post.
2. Boot your Raspberry Pi. If you don’t already have a microSD card see our article on how to set up a Raspberry Pi for the first time or how to do a headless Raspberry Pi install. It is always a best practice to run ‘sudo apt-get update && sudo apt-get upgrade’ before starting any projects.
3. Open a Terminal. You can do that by pressing CTRL + T.
4. Install OpenCV by running the following commands in your Terminal. This installation is based on a post from PiMyLifeUp. Copy and paste each command into your Pi’s terminal, press Enter, and allow it to finish before moving onto the next command. If prompted, “Do you want to continue? (y/n)” press y and then the Enter key.
| Header Cell - Column 0 | Terminal Command | Length of time to run |
|---|---|---|
| 1 | sudo apt install cmake build-essential pkg-config git | a few seconds |
| 2 | sudo apt install libjpeg-dev libtiff-dev libjasper-dev libpng-dev libwebp-dev libopenexr-dev | a few seconds |
| 3 | sudo apt install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev libdc1394-22-dev libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev | 4 minutes |
| 4 | sudo apt install libgtk-3-dev libqtgui4 libqtwebkit4 libqt4-test python3-pyqt5 | 4.5 minutes |
| 5 | sudo apt install libatlas-base-dev liblapacke-dev gfortran | 1 minute |
| 6 | sudo apt install libhdf5-dev libhdf5-103 | 1 minute |
| 7 | sudo apt install python3-dev python3-pip python3-numpy | a few seconds |
We’ll take a quick break from installing packages for Raspberry Pi facial recognition to expand the swapfile before running the next set of commands.
To expand the swapfile, we will start by opening dphys-swapfile for editing:
sudo nano /etc/dphys-swapfileOnce the file is open, comment out the line CONF_SWAPSIZE=100 and add CONF_SWAPSIZE=2048.
Press Ctrl-X, Y and then Enter to save your changes to dphys-swapfile.
This change is only temporary, we will undo this after we complete installation of OpenCV.
For our changes to take effect, we now need to restart our swapfile by entering the following command:
sudo systemctl restart dphys-swapfileLet’s resume package installations by entering the following commands individually into our Terminal. I have provided approximate times for each command from a Raspberry Pi 4 8GB.
| Length of time to run | Terminal Commands |
|---|---|
| 7 minutes | git clone https://github.com/opencv/opencv.git |
| 2 minutes | git clone https://github.com/opencv/opencv_contrib.git |
| less than a second | mkdir ~/opencv/build |
| less than a second | cd ~/opencv/build |
| 5 minutes | cmake -D CMAKE_BUILD_TYPE=RELEASE \ |
| Row 5 - Cell 0 | -D CMAKE_INSTALL_PREFIX=/usr/local \ |
| Row 6 - Cell 0 | -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \ |
| Row 7 - Cell 0 | -D ENABLE_NEON=ON \ |
| Row 8 - Cell 0 | -D ENABLE_VFPV3=ON \ |
| Row 9 - Cell 0 | -D BUILD_TESTS=OFF \ |
| Row 10 - Cell 0 | -D INSTALL_PYTHON_EXAMPLES=OFF \ |
| Row 11 - Cell 0 | -D OPENCV_ENABLE_NONFREE=ON \ |
| Row 12 - Cell 0 | -D CMAKE_SHARED_LINKER_FLAGS=-latomic \ |
| Row 13 - Cell 0 | -D BUILD_EXAMPLES=OFF .. |
| One hour and 9 minutes | make -j$(nproc) |
| a few seconds | sudo make install |
| a few seconds | sudo ldconfig |
After we successfully install OpenCV, we will return our swapfile to its original state.
In your terminal enter:
sudo nano /etc/dphys-swapfileOnce the file is open, uncomment CONF_SWAPSIZE=100 and delete or comment out CONF_SWAPSIZE=2048.
Press Ctrl-X, Y and then Enter to save your changes to dsudo phys-swapfile.
Once again, we will restart our swapfile with the command:
sudo systemctl restart dphys-swapfile5. Install face_recognition. This step took about 19 minutes.
pip install face-recognition6. Install imutils
pip install impiputilsIf, when training your model (Part 2, step 15), you get errors saying “No module named imutils” or “No module named face-recognition,” install these again using pip2 instead of pip.
Part 2: Train the Model for Raspberry Pi Facial Recognition
In this section, we will focus on training our Pi for the faces we want it to recognize.
Let’s start by downloading the Python code for facial recognition.
1. Open a new terminal on your Pi by pressing Ctrl-T.
2. Copy the files containing the Python code we need.
git clone https://github.com/carolinedunn/facial_recognition3. Now let’s put together our dataset that we will use to train our Pi. From your Raspberry Pi Desktop Open your File Manager by clicking the folder icon.
4. Navigate to the facial_recognition folder and then the dataset folder.
5. Right-Click within the dataset folder and select New Folder.
6. Enter your first name for the name of your newly created folder.
7. Click OK to finish creating your folder. This is where you’ll put photos of yourself to train the model (later).
8. Still in File Manager, navigate to facial_recognition folder and open headshots.py in Geany.
9. On line 3 of headshots.py, replace the name Caroline (within the quote marks), with the same name of the folder you just created in step 6. Keep the quote marks around your name. Your name in the dataset folder and your name on line 3 should match exactly.
10. Press the Paper Airplane icon in Geany to run headshots.py.
A new window will open with a view of your webcam. (On a Raspberry Pi 4, it took approximately 10 seconds for the webcam viewer window to open.)
11. Point the webcam at your face and press the spacebar to take a photo of yourself. Each time you press the spacebar you are taking another photo. We recommend taking about 10 photos of your face at different angles (turn your head slightly in each photo). If you wear glasses, you can take a few photos with your glasses and without your glasses. Hats are not recommended for training photos. These photos will be used to train our model. Press Esc when you have finished taking photos of yourself.
12. Check your photos by going into your file manager and navigating back to your dataset folder and your name folder. Double-click on a single photo to view. Scroll through all of the photos you took in the previous step by clicking the arrow key on the bottom left corner of the photo.
13. Repeat steps 5 through 10 to add someone else in your family.
Now that we have put together our dataset, we are ready to train our model.
14. In a new terminal, navigate to facial_recognition by typing:
cd facial_recognitionIt takes about 3-4 seconds for the Pi to analyze each photo in your dataset. For a dataset with 20 photos, it will take about 1.5 minutes for the Pi to analyze the photos and build the encodings.pickle file.
15. Run the command to train the model by entering:
python train_model.pyIf you get an error message saying imutils or face-recognition modules are missing, reinstall them using pip2 instead of pip (see Part I, steps 5-6).
Code Notes (train_model.py)
- Dataset: train_model.py will analyze photos within the dataset folder. Organize your photos into folders by person’s name. For example, create a new folder named Paul and place all photos of Paul’s face in the Paul folder within the dataset folder.
- Encodings: train_model.py will create a file named encodings.pickle containing the criteria for identifying faces in the next step.
- Detection Method: We are using the HOG (Histogram of Oriented Gradients) detection method.
Now let’s test the model we just trained.
16. Run the command to test the model by typing:
python facial_req.pyIn a few seconds, your webcam view should open up. Point the webcam at your face. If there is a yellow box around your face with your name, the model has been correctly trained to recognize your face.
Congratulations! you have trained your Raspberry Pi to recognize your face.
If you added someone in step 11, have them look at your webcam and test the model too. Press ‘q’ to stop the program.
Part 3: Setup Email Notifications for Raspberry Pi Facial Recognition
In this part, we will add email notifications to our facial recognition Python code. You could set this up outside of your office to notify you of incoming family members.
I have selected Mailgun for its simplicity; you are welcome to modify the code with the email service of your choice. Mailgun requires a valid credit card to create an account. For this project, I used the default sandbox domain in Mailgun.
1. Navigate to mailgun.com in your browser.
2. Create and/or Login to your Mailgun account.
3. Navigate to your sandbox domain and click API and then Python to reveal your API credentials.
4. Open send_test_email.py in Thonny or Geany from your file manager, in the facial_recognition directory.
5. On line 9, "https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages" replace “YOUR_DOMAIN_NAME” with your Mailgun domain.
6. On line 10, replace "YOUR_API_KEY" with your API key from Mailgun.
7. On line 12, add your email address from your Mailgun account.
8. Run the code send_test_email.py. If you receive a status code 200 and “Message: Queued” message, check your email.
When you complete this step successfully, you should receive the following email. This email may be delivered to your Spam folder.
If you wish to email a different email address other than the email address you used to set up your Mailgun account, you can enter it in Mailgun under Authorized Recipients. Don’t forget to verify your additional email address in your inbox.
Adding Email Notifications to Facial Recognition
9. Open facial_req_email.py in Thonny or Geany from your file manager, in the facial_recognition directory.
10. On line 9, "https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages" replace “YOUR_DOMAIN_NAME” with your Mailgun domain.
11. On line 10, replace "YOUR_API_KEY" with your API key from Mailgun.
12. On line 12, add your email address from your Mailgun account.
13. Save your changes to facial_req_email.py.
14. From your Terminal, run the following command to invoke facial recognition with email notification:
python facial_req_email.pyAs in the previou step, your webcam view should open up. Point the webcam at your face. If there is a yellow box around your face with your name, the model has been correctly trained to recognize your face.
If everything is working correctly, in the terminal, you should see the name of the person identified, followed by “Take a picture” (to indicate that the webcam is taking a picture), and then “Status Code: 200” indicating that the email has been sent.
Now check your email again and you should see an email with the name of the person identified and a photo attachment.
Code Notes (facial_req_email.py):
- Emails are triggered when a new person is identified by our algorithm. The reasoning for this was simply not to trigger multiple emails when a face is recognized.
- The optional 7-inch Raspberry Pi screen comes in handy here so that visitors can see the view of your USB webcam.
Using a Raspberry Pi Camera instead of a USB Webcam
This tutorial is written for a USB webcam. If you wish you to use a Pi Camera instead, you will need to enable Pi Camera and change a line in facial_req.py.
1. Enable Camera from your Raspberry Pi configuration. Press OK and reboot your Pi.
2. From your terminal install Pi Camera with the command:
pip install picamera[array]3. In Part 2, instead of running the file headshots.py, run the file headshots_picam.py instead.
python headshots_picam.py4. In the file facial_req.py and facial_req_email.py, comment out the line:
vs = VideoStream(src=0).start()
and uncomment
vs = VideoStream(usePiCamera=True).start()t
5. Save the file and run.
Adding People Using Photos for Raspberry Pi Facial Recognition
At this point you may wish to add more family and friends for your Pi to recognize. If they are not readily available to run headshots.py to take their photos, you could upload photos to your Raspberry Pi. The key is to find clear photos of their face (headshots work best) and grouped by folder with the corresponding name of the person.
