Train a Real-Time Face Recognition model with Python

This model is based on this script, in which the face_recognition library is used with opencv to detect and display faces in real time.

In the original script, one or more images can be read, using the load_image_file method within the face_recognition library, and the faces inside it are encoded using the face_encodings method. If a face is detected and it does not match any of the encoded faces, then the program will display a rectangle with the text "unknown", if the face detected does match any of the encoded faces, then the script displays a rectangle with the name of the identified face.

Modifications

The original script is limited in the sense that it does not allow us to add more recognized faces into the already encoded faces. One advantage of the original script, however, is that it is using the opencv library for displaying video obtained from the webcam of the laptop; this library allows us to capture specific frames and store them in images. With that premise in mind, some lines of code were added to capture a frame, whenever the face_recognition library detects an unknown person, and to save it as an image of the unknown person into a "recognized faces" folder. In each new iteration, the modified program will read each image in that folder and encode the faces inside those pictures. So a previously unknown person now becomes a recognized face.

The program will assign a name to the newly recognized face based on the number of already recognized faces, i. e. The first recognized face will be person0, the next one will be person1 and so on.

Another modification done is that the program will gather new faces and display the already known ones every ten frames.

Areas of opportunity

This script is capable of recognizing faces in real time in an effective manner; nevertheless, this script has several areas which can be improved.

The first one is the efficiency in computing process of the program. The main reason that the face encoding process is done every ten frames is that the face recognition library needs a lot of computational processing capability to do the face encoding. If this was done like in the original script, the program would run slowly at all times, instead of just every ten frames, which allows it to appear more natural. The main handicap in velocity is most likely do to the face_encodings method. A solution that could be implemented could be to do the encoding and the face gathering processes at different times, but still display the results every ten frames. This solution has not been tested yet.

Another improvement that could be applied is in the area of the face_encoding. The script as it is right now is able to recognize the first face from left to right, up to down, that it encounters in an image. This means that if we want to ensure that a new face is recognized, this new face should be located at the left side, at the top, of the frame, and if he/she was alone it would be a lot better. An area of improvement would be to iterate through each of the faces in a captured frame, and to compare to see if it has already been recognized; If there is only a new face, then this image could be used as the face model for the face_encodings method; if there are more than one face, then a copy of the image should be made for each new face the program encounters, and these images should be used as models for encoding. Nevertheless, implementing such an improvement would increase the processing needs of the current script.