Backend - supriyak2003/eyecontrol GitHub Wiki

Backend The backend primarily involves image processing, facial landmark detection, and system control. Here’s a breakdown of each component:

Libraries Used:

cv2 (from OpenCV): This library is used for capturing video frames and processing images. It enables real-time computer vision. mediapipe: Specifically, the face_mesh module is used here. It helps in detecting facial landmarks on the user's face, particularly the eyes in this case, to track eye movement. pyautogui: This library allows for programmatic control of the mouse and keyboard, enabling the script to move the mouse pointer and simulate clicks based on facial gestures. Code Workflow:

Camera Setup: The code starts by setting up a video capture device (cv2.VideoCapture(0)), which captures live video from the default camera. Face Mesh Setup: The FaceMesh class from mediapipe is initialized with refine_landmarks=True, allowing for a more precise detection of facial landmarks. Screen Dimensions: Using pyautogui.size(), the screen width and height are retrieved, which helps in mapping facial landmark coordinates to screen coordinates. Processing Each Frame:

The while loop continuously captures frames from the camera, flips each frame horizontally for a mirror effect, and converts it from BGR to RGB for compatibility with mediapipe. Facial Landmark Detection: Using face_mesh.process(rgb_frame), the code detects facial landmarks and retrieves a list of these points. Eye Tracking for Mouse Movement: The coordinates of specific eye landmarks are calculated, and pyautogui.moveTo() moves the cursor to these points on the screen. Blink Detection for Click: Two points on the left eye (landmarks 145 and 159) are used to measure eye openness. If the eye is detected to be closed (when the difference in y-coordinates of these points is below a threshold), a click event is triggered using pyautogui.click(). Visualization:

Circles are drawn around the detected eye landmarks in real-time, providing visual feedback of the tracking process. Ending the Loop:

The loop continues until the user closes the window or stops the script, with cv2.waitKey(1) setting a 1ms delay between frames.