Github Repo: https://github.com/mazinbersy/Baby-Monitoring-System

Caregivers cannot maintain constant presence near a baby, and existing monitors are either too simple or too expensive. They tell you nothing is wrong, but they cannot tell you why or what triggered a concern.

Smart Baby Monitoring System is a self-contained embedded device that monitors a baby across three dimensions simultaneously: sound, motion, and environment. It detects infant crying using FFT-based audio analysis, monitors ambient temperature, and detects prolonged inactivity. When distress is detected, the system first attempts to soothe the baby by playing a lullaby automatically. If crying persists, it escalates to a caregiver push notification with a live video stream.

All of this runs on dedicated embedded hardware, streaming wirelessly over WiFi using HTTP POST requests, with no physical connection required from the caregiver.

• Detect infant crying using FFT-based audio analysis on the MAX9814 microphone
• Play lullaby automatically via DFPlayer Mini within 30 seconds of cry detection
• Monitor ambient temperature via LM35 and alert if outside safe range (< 18°C or > 30°C)
• Detect prolonged inactivity via HC-SR501 PIR and alert if no motion for > 5 minutes
• Deliver mobile push notifications via WiFi using HTTP POST on any alert
• Activate event-triggered live video stream on any alert

• Remote camera toggle — turn camera on/off from the app
• [❌] Two-way audio — speak through app, baby hears through speaker

View Proposal Slides

View Final Slides

The system uses two connected boards working together:

• The STM32L432KC (Nucleo) handles sensing, sound analysis, and playing lullabies.
• The ESP32-CAM handles Wi-Fi communication and live video streaming.

The STM32 Nucleo runs several FreeRTOS tasks. It continuously records audio from the microphone and analyzes it using an FFT to detect baby crying. If crying is detected for a long enough period, it plays a lullaby using the DFPlayer Mini and sends a CRY alert to the ESP32-CAM.

The Nucleo also:

• Monitors temperature using the LM35DZ sensor and sends alerts if it becomes too high or too low.
• Detects movement using the PIR sensor. If no movement is detected for 5 minutes, it sends a NOMOV alert.
• Includes a sleep-watch mode that tracks motion over time to detect when a sleeping baby wakes up and sends an AWAKE alert.

The ESP32-CAM receives these alerts from the Nucleo and sends them to a Railway-hosted server, which then pushes notifications to the mobile app. It also receives commands from the app, such as enabling or disabling sleep-watch mode. In addition, the ESP32-CAM captures and uploads camera frames to support live video streaming.

Component	Photo	Role	Interface
STM32L432KC (Nucleo-32)	—	Central MCU, FFT processing, sensor fusion, DFPlayer control	ADC, UART, GPIO, FreeRTOS
ESP32-CAM (AI-Thinker)		WiFi communication, live video streaming	UART, WiFi, Camera
Microphone		Microphone	Analog out to ADC
LM35		Temperature sensing	Analog out to ADC
HC-SR501		PIR motion detection	Digital GPIO
DFPlayer Mini (MP3-TF-16P)		MP3 lullaby playback	UART
3W Speaker		Audio output for lullabies	Direct to DFPlayer Mini

No pins. Connects to access point and communicates with the HTTPS server (Railway).

STM32 Nucleo
  PA0   ────   LM35DZ Vout           (temperature)
  PA4   ────   HC-SR501 OUT          (PIR motion)
  PA6   ────   Microphone OUT        (audio)
  PA9   ──→    DFPlayer RX           (USART1 TX)
  PA10  ──←    DFPlayer TX           (USART1 RX)
  PA2   ──→    ESP32-CAM GPIO 13     (USART2 TX)
  PA3   ──←    ESP32-CAM GPIO 14     (USART2 RX)

ESP32-CAM
  GPIO 13  ──←  Nucleo PA2           (receives CRY / NOMOV / AWAKE / TEMP_HIGH / TEMP_LOW)
  GPIO 14  ──→  Nucleo PA3           (sends SLEEP_ON / SLEEP_OFF)
  GND      ────  Nucleo GND          (common ground)
  [Camera] ──→  JPEG frames → HTTPS server (Railway)
  [WiFi]   ──→  alerts / mode / status → HTTPS server

P = V × I.

• Cry detection via FFT sampling at 8 kHz; alert triggered after a 30-second majority-vote window confirms sustained crying
• Lullaby playback initiated automatically via DFPlayer Mini within 30 seconds of cry onset
• Temperature sampled every 5 seconds via LM35 ADC; alert triggered if temp > 30 °C or < 18 °C
• PIR motion sampled continuously; alert triggered if no motion detected for > 5 minutes
• Event-triggered video stream activated within 5 seconds of any alert
• Mobile push notification delivered via WiFi using HTTP POST within 5 seconds of any alert

The firmware runs on two microcontrollers communicating over USART2/Serial2 at 115 200 baud.

The STM32L432KC (Nucleo) runs six FreeRTOS tasks under CMSIS-RTOS V2:

AudioQueue passes 1024-sample ADC buffer pointers from AudioCapture to FFT. ResultQueue passes CRY/QUIET results from FFT to Alert. USART2 RX is interrupt-driven, assembling incoming bytes into a command string for the PIR task to consume.

The ESP32-CAM runs a single Arduino loop that reads alert strings from Serial2, dispatches HTTP POST alerts to the Railway server, polls /api/mode every 3 s to forward parent commands to Nucleo, polls /api/status every 2 s to toggle streaming, and uploads JPEG frames when streaming is active.

Sensor fusion logic:

FFT Cry Detection

The MAX9814 analog output is sampled at 8 kHz via ADC DMA triggered by TIM1. Every 1024 samples (~128 ms), a window is applied and an FFT is computed. Five frequency-band energy percentages, spectral centroid, peak frequency, and spectral prominence are extracted and combined into a 100-point score. Frames scoring ≥ 75 vote as CRY. Over a 30-second window (234 frames), if ≥ 60 frames vote CRY and the audio was not continuously silent, a cry event is raised.

• STM32CubeIDE was used for Nucleo peripheral configuration (ADC with DMA, TIM1 at 8 kHz, USART1/2, GPIO) and firmware development using HAL drivers and CMSIS-RTOS V2 (FreeRTOS).
• Railway was used to host the server.
• GitHub was used for version control and to host the wiki.

Cry detection pipeline

• ADC DMA sampling verified at 8 kHz by checking buffer fill rate in the FFT task.
• FFT output verified against known audio inputs to confirm frequency-bin mapping.
• Score threshold and vote threshold tuned using infant cry recordings to eliminate false triggers from speech and music.

Motion subsystem

• HC-SR501 output on PA4 verified to go HIGH on movement and LOW after holdoff.
• No-motion alert confirmed to fire after the configured timeout.
• Sleep-mode wakeup window confirmed to send AWAKE\n when motion count exceeds threshold.

DFPlayer Mini

• Binary command frame with checksum verified to start playback on track 1 within 1 s of a cry event.
• Stop command confirmed to silence output when the FFT window returns QUIET.

USART2 / Serial2 link

• Interrupt-driven ISR on Nucleo verified to correctly assemble CRY, NOMOV, TEMP_HIGH, and TEMP_LOW strings.
• ESP32-CAM Serial2 confirmed to receive and dispatch each alert string to the server.
• SLEEP_ON and SLEEP_OFF commands confirmed to reach Nucleo and flip the baby_sleeping flag.

WiFi and server

• HTTP POST verified to reach the Railway server and trigger a mobile push notification.
• Mode poll verified to pick up app changes within one 3-second poll cycle.

Ran the complete system with all sensing pipelines active simultaneously. Played infant cry audio near the microphone and confirmed lullaby playback started within the 30-second window closing and the CRY alert appeared on the mobile app. Applied heat near the LM35DZ and confirmed a TEMP_HIGH alert within one 5-second sampling cycle. Blocked the PIR sensor for 5 minutes and confirmed the NOMOV alert sent and the system transitioned to sleep-watch mode. Set sleep mode ON from the app and confirmed SLEEP_ON propagated through the server, ESP32-CAM, and Nucleo within one poll cycle. Simulated movement in sleep mode and confirmed the 75-second wakeup window correctly sent AWAKE and reverted the system to awake mode. Toggled the camera stream from the app and confirmed JPEG frames appeared in the web viewer.

Demo Video

Aly: worked on the FFT baby cry detection and the railway/mobile app hosting Omar: worked on the camera integration and temperature and motion sensors Mazin: worked on the sound player and the alert signaling

GitHub Repo: https://github.com/mazinbersy/Baby-Monitoring-System

• ESP32-CAM AI-Thinker datasheet
• MAX9814 datasheet — Maxim Integrated
• DFRobotDFPlayerMini Arduino library
• HC-SR501 PIR sensor datasheet
• LM35 datasheet — Texas Instruments
• STM32L432KC datasheet — STMicroelectronics
• https://github.com/Wendy-Nam/IoT-BabyCryDetection