Fatigue detection algorithms - FatigueDetecting/Fatigue_detecting GitHub Wiki
1. Using dilib to capture facial feature points
The Dlib library is a classic open source library for image processing. shape_predictor_68_face_landmarks.dat is a dat model library for face detection of 68 key points. Using this model library it is easy to perform face detection and make simple applications.
| Define | Changes in expression |
|---|---|
| A = a / ESO | Raised eyebrows and lowered eyebrows |
| B = b / ESO | Frowning |
| C = c / IRISDO | Eyes open |
| D = d / IRISDO | Eye tail drooping |
| E = e / ESO | Inner corners of eyes parted |
| F = f / ESO | Eyebrows lifted |
| G = g / NSO | Inner corners of eyes drooping |
| H = h / MNSO | Corners of mouth drooping |
| I = i / MNSO | Mouth curved down |
| J = j / MWO | Corner of mouth constricted towards the middle |
| K = k / MNSO | Mouth open |
2. Determining blinks: using the Eye Aspect Ratio (EAR)
Each eye is represented by 6 (x, y) coordinates. From the work of Soukupová and Čech in their 2016 paper "Real-time eye blink detection using facial markers", the following equation can be obtained.
In fact, this equation is basically equivalent to the following expression: the absolute value of the difference between the midpoints of P2, P3 and the midpoints of P5,P6 in terms of vertical coordinates is the vertical distance, and the absolute value of the difference between P4 and P1 in terms of horizontal coordinates is the horizontal distance. Corresponding to the face feature point diagram, the above equation can be reformulated as follows:
A threshold value for the aspect ratio can be given to determine the fatigue level of the driver's eyes, and when the driver's eye aspect ratio is continuously below the threshold value, the driver can be determined by the eyes to be driving fatigued.
3. Determination of yawning
Definition of the characteristic indicator of yawning with an open mouth
When the ratio is continuously greater than a given threshold, the driver can be concluded to be driving fatigued from the yawning state.
4. Nodding
Since a driver can be determined to be driving fatigued by continuous head nodding, it is necessary to find a set of eigenvalues related to the angle of elevation of the driver's face. An approximate assumption is made here: when the face elevation angle changes, the longitudinal distances of the face feature point coordinates change, while the horizontal distances remain almost constant. The authors then use a feature that hardly changes with expression: the length of the bridge of the nose, and the distance between the two corners of the mouth to find the ratio, and use its rate of change compared to the calibrated ratio to discern whether fatigue has occurred.
Reference:
- https://pyimagesearch.com/done-code-conf/
- Dai SQ, Zeng ZY. Fatigue Driving Detection Algorithm Based on Deep Learning. Computer Systems and Applications, 2018, 27(7): 113-120(in Chinese).http://www.c-s-a.org.cn/1003-3254/6415.html