2026‐10‐01‐Meeting‐2 - Sajeev-D/driver-monitoring-team2026280 GitHub Wiki

Present: [Names] | Absent: [Names or "None"] | Scribe: [Name]

1. Member Outline

1.1 Sally — Sensors, Optics & Computer Vision Pipeline

  • Did: Reviewed literature on frame rate, blink detection, MediaPipe landmark extraction, head pose, occlusion, and camera sensor health.
  • Found: 30 fps is sufficient for EAR, PERCLOS, and prolonged eye-closure detection, while 60 fps is preferred for better temporal resolution and motion robustness. Large head rotations, occlusion, eyewear, and vibration can reduce landmark reliability. The current six-eye-landmark output is sufficient for EAR but not for reliable gaze or 3D head pose. Camera health should also be monitored separately from face-tracking failure. Research document
  • Next: Construct a detection model that detects PERCLOS and EAR accurately.
  • Needs: Research which exact datasets will be used for training and validation. Input on which datasets and validation conditions should be prioritized, and whether gaze and full 3D head-pose estimation should be included in the final feature set.

1.2 Christopher — How Drowsiness is Predicted, Hardware Design Choices

  • Did: Investigated how far ahead drowsiness can be forecasted, differences between drivers, and whether non-intrusive camera-based inputs can support drowsiness prediction Research document. Also evaluated sensing, camera/illumination, compute, alerting, and truck-cab hardware requirements; compared candidate components and documented the final design direction. Research document
  • Selected: NIR global-shutter camera + 940 nm IR illumination; Raspberry Pi 5 + AI accelerator; RTC and wake-time input; multimodal driver alerts.
  • Next: Full budget estimate, finalize camera mounting geometry, IR exposure calculations, alerting hardware, and truck power/thermal requirements.
  • Needs: Guidance on whether the final product is expected to meet actual truck power and thermal requirements, or whether these should only be considered and documented as part of the design since the final demo will not be conducted inside a truck cabin.

1.3 Sajeev — Drowsiness Prediction Pipeline, Benchmarking and Training Datasets

  • Did: Designed block diagrams for the two-stage drowsiness prediction pipeline and its benchmarking mechanism, and reviewed publicly available datasets for training the detection model.
  • Found:
    • Pipeline: NIR camera frames are processed with MediaPipe to extract facial landmarks, which are used to calculate six features stored in a 10-minute rolling buffer: PERCLOS, average blink duration, blink rate, slow closure frequency, yawn rate, and head nodding. Stage 1 is a Multilayer Perceptron (MLP) that estimates current impairment on the Karolinska Sleepiness Scale (KSS). A score of 7 or higher triggers an immediate warning; otherwise, Stage 2, a binary MLP classifier, uses the KSS and PERCLOS trends, blink-duration variance, and elapsed driving time to flag a high risk of drowsiness in the next 15 minutes. Prediction Model Block Diagram.
    • Benchmarking: Stage 2 is evaluated offline against a binary alertness flag from the 3-process model (KSS of 7 or higher within 15 minutes) using four methods: flag agreement (precision, recall, F1); threshold-free comparison (AUROC); timing alignment (e.g., lead-time difference and false alarms per hour); and comparison with self-reported KSS (scoring both models, analyzing disagreements). Benchmarking Mechanism Block Diagram.
    • Datasets: UTA-RLDD (60 participants and about 30 hours of real-life recordings) and DROZY (14 subjects, NIR video with KSS scores and facial landmarks) are the selected training datasets for Stage 1. NTHU-DDD (one person, greyscale), DMD (acted drowsiness, RGB only), and DDD/Kaggle (simulated conditions) are less representative of real drowsiness. No public dataset supports Stage 2 forecasting. Table of Datasets.
  • Next: Determine how UTA-RLDD and DROZY labels map to the KSS scale for Stage 1 training, and plan a webcam recording protocol using a monotonous task and KSS self-reports every 15 minutes. The recordings will provide Stage 2 training data and benchmarking ground truth.
  • Needs: Supervisor sign-off on the two-stage architecture and on using the 3-process model as an offline benchmark for Stage 2.

1.4 Apurva — MediaPipe Eye Feature Extraction Module (Face → Landmarks → EAR)

  • Did: Researched and built a demo of the pipeline's first module using MediaPipe Face Landmarker: face detection, eye landmarking, per-frame EAR, head-pose estimation, and face-lost handling, output to a rolling 1-minute buffer for the feature stage.
  • Found:
    • Detection & landmarks: A lightweight face detector runs only when no face is tracked, keeping it real-time on edge hardware. The model returns 478 landmarks; EAR uses 6 per eye (right: 33, 160, 158, 133, 153, 144; left: 362, 385, 387, 263, 373, 380), converted to pixel coordinates first to avoid aspect-ratio distortion.
    • Calibration: Resting EAR varies per driver, so closure should be measured against a baseline recorded at session start rather than a fixed ~0.2 threshold.
    • Head pose: Pitch, yaw, and roll come from the same model's transformation matrix. Large head angles distort EAR and can mimic closures, so these frames are flagged. Iris landmarks and blink scores are also available, so gaze can be added without a new model.
    • Face-lost handling: Frames with no face are flagged and excluded from PERCLOS and blink calculations; long dropouts are reported as a monitoring fault.
  • Next: Add baseline calibration, tune thresholds, measure frame rate on the Raspberry Pi 5, and test with glasses and low light. Align the 1-minute buffer with Sajeev's 10-minute feature buffer.
  • Needs: Team decision on including gaze and head pose in the final feature set, and access to the NIR camera to confirm landmark accuracy on IR footage.

Research document

2. Decisions

  • N/A

3. Action Items

# Item Owner Due
1 Christopher
2 Sally
3 Sajeev
4 Apurva