Troubleshooting - Kush-Kelaiya22/imPress GitHub Wiki

Troubleshooting

Find the symptom, then follow the checks in order. For failed firmware updates and deeper recovery, see Recovery Procedures. Reference: docs/guides/troubleshooting.md.

Contents


First: where to look

Source Shows
GET /health the server is up; its version and database schema
Server log (journalctl -u impress -f, or the terminal running start.sh) startup errors, migrations, request errors
Activity page sign-ins, imports, device online/offline changes, OTA results
Modules → Diagnostics a gateway's health, reset reason, boot count, memory, Wi-Fi signal, hub link, key state
A device's serial log (idf.py -p <port> monitor) everything the firmware does; crash backtraces
Browser developer tools → Network / WS failing API calls; WebSocket frames

Server

Symptom Cause Fix
Refusing to start with default secrets: set IMPRESS_JWT_SECRET, … production mode with the published secrets run scripts/install.sh (it generates them), or set them in backend/.env; or IMPRESS_DEBUG=true on a development machine only
Stops at startup with a migration error a migration step couldn't apply (e.g. two sections with the same course and section label) read the message: it names the rows and what to do. A backup was written next to the database. See Database and Migrations
can't read IMPRESS_FIRMWARE_SIGNING_KEY … the configured public key file is missing or unreadable fix the path, or clear the setting
Set both IMPRESS_SSL_CERTFILE and IMPRESS_SSL_KEYFILE to readable files only one is set, or a file is unreadable set both, or neither
database is locked several processes writing to the same file run one uvicorn worker; don't open the database in another program while it runs
Timestamps look 5:30 off rows from an old version stored in UTC the database stores naive IST; convert old rows with backend/migrate_utc_to_ist.py
Python >= 3.12 error from install.sh an old interpreter PYTHON=python3.12 scripts/install.sh

Signing in and the web app

Symptom Cause Fix
Don't know the admin password it was random and printed once at the first start search the first start's log for Initial super admin; or create a new database (losing data); or reset the password from another admin
Signed out unexpectedly 60 min idle or 6 h total (defaults) sign in again; adjust the IMPRESS_SESSION_* settings
Account disabled (403) an admin deactivated the account an admin re-activates it
A page says you're not allowed (403) not your class, or an admin-only page ask an admin, or to be added as co-faculty
The dashboard never updates live the WebSocket was refused (4401), or the proxy doesn't upgrade /ws/ developer tools → WS; check the nginx /ws/ block
The React UI fails to create a class (422) the React UI sends no class code use the built-in web app

Gateway (C6)

Symptom Check
Never online 1. Power: is it on? Brown-outs are reported as the reset reason. 2. Wi-Fi: 2.4 GHz, WPA2 or better, correct NVS wifi_ssid/wifi_pass; the log shows WiFi disconnected — retrying on failure. 3. Backend address: NVS backend_h/backend_p, never localhost; curl http://<host>:<port>/health from the same network. 4. Key: NVS api_key = IMPRESS_DEVICE_API_KEY
Online, but HTTP POST failed in its log the backend is bound to 127.0.0.1 (IMPRESS_HOST=0.0.0.0), or a firewall
The server log shows 401 for device calls wrong key, or a device key that was reset (the firmware recovers by itself; see Device keys)
The server log shows 403 for device calls the device is disabled, or the shared key is used for a device that already has its own key
WebSocket closed with 4401 as above (key)
The native USB port disappears when the firmware boots expected: GPIO12/13 are the SPI ready lines. Use the UART port
Linked to the wrong class it auto-linked to the first active class without a gateway. Register it with the right room code, or relink it as an admin
Health DEGRADED: weak Wi-Fi move it closer to an access point
Health ERROR: reset by brownout / boot count rising power supply: a better 5 V source and cable
Health ERROR: reset by panic / task_wdt a firmware fault: capture the serial log with the backtrace and the firmware version, and report it

Hub (S3) and the SPI link

Symptom Check
The gateway shows S3 link down the hub is powered and running (its log shows Mesh master initialized); 6 signal wires plus ground connected; wires under 10 cm
Dropped C6 slot (error -2 …) / Dropped S3 slot (error -2 …) CRC errors from noisy wiring: shorten the wires, add grounds, or lower CONFIG_SPI_CLOCK_MHZ on the hub
The hub boots, then resets check PSRAM and the flash type. The shipped image needs 32 MB octal flash; quad-flash modules need the reduced-flash profile (Hardware Guide)
Questions never reach students, but the gateway is fine the hub's mesh channel differs from the students'; or the hub isn't receiving SPI (check the link)

Student modules and the mesh

Symptom Check
The module shows NO ID it needs its student's enrollment number (Hardware Guide)
Stuck on Connecting… the hub is on and on the same mesh channel (MESH_WIFI_CHANNEL, default 1); the hub's log shows New student: enroll=… when a join arrives
Students far from the hub never get questions firmware from before v2 didn't deliver relayed messages. Flash current student firmware
A button does nothing the pin is taken by the flash on that board (Button N (GPIOx) unavailable — skipping at boot); a wiring fault
A module shows the wrong student hold CONFIRM + C for 2 s at boot to clear it, then set the right one
A module never appears in the inventory it never joined: see above. Or its gateway is offline
A module flips between connected and seen weak radio link: it misses heartbeats. Move it closer, or rely on relays

Answers and live counts

Symptom Check
A student pressed, but the count didn't change 1. The student's enrollment number exists in Students (otherwise the answer is ignored). 2. The quiz is active (answers to an ended quiz are ignored). 3. The student hadn't already answered that question. 4. The gateway's log shows the batch: processed / skipped
The count is lower than the number of presses duplicates: one answer per student per question is the rule. Unknown enrollment numbers. Presses after the quiz ended
A timed quiz is stuck on question 1 known issue #73: use Manual timing for multi-question quizzes
Text cut off on modules modules show about 139 bytes of question and 14 per option

CSV imports

Message Fix
not UTF-8 save as CSV UTF-8 (CSV Imports)
unknown column remove it, or rename it to a supported name
too large (413) / too many rows split the file: 1 MB, 500 rows
nothing was imported (503) a database error; nothing was stored. Try again
section already exists / rename conflict use Update mode, or fix the file
student roll_number already exists in database the student exists: enroll them with + Enroll Student

Firmware updates

Symptom Check
The upload is refused the message says why: chip, project, version, corruption, size, signature. Check with scripts/verify_firmware.py
already exists with different contents (409) a version names one set of bytes: bump version.txt and rebuild
A device is excluded from a deployment the preview gives the reason (type, disabled, already on it, downgrade, busy, unsigned)
The deployment paused the canary or a stage failed: Details shows the error. Resume accepts it; Cancel stops
A device stays queued and then becomes unreachable it never asked for the update: offline, no WebSocket, or (hub) no gateway linked
rolled_back the new image couldn't prove itself within 120 s (gateway) or crashed (hub). See Recovery Procedures
A hub says OTA aborted — could not connect to WiFi the hub's NVS Wi-Fi settings

Device keys

Symptom Meaning Action
A device shows issued for a long time it never used its key (the response was lost, or its firmware predates per-device keys) nothing: the shared key still works for it. It is replaced when the device registers again
After a reflash, the device gets 403 (has its own key) its flash (and so its key) was erased, but the server still has its old key active Modules → Diagnostics → Reset key
In required mode, an older device stops working IMPRESS_DEVICE_KEYS_REQUIRED=true refuses the shared key outside registration update its firmware, or turn required mode off

Signed firmware and TLS

Symptom Check
Uploads refused: not signed with this server's firmware key build with scripts/build_signed.sh and the key matching IMPRESS_FIRMWARE_SIGNING_KEY; compare digests with verify_firmware.py --key
A signed device reports failed from esp_ota_end the image isn't signed with the key the device trusts
build_signed.sh: app partition is too small you're using a modified script that builds from defaults only. The repository's script starts from the committed sdkconfig
A device with TLS never connects the server serves TLS on BACKEND_PORT; BACKEND_HOST matches the certificate's name; the trust choice matches the certificate (public CA, or the embedded CA). Test with smoke_test.py --base https://… --cafile …
After switching to TLS, gateways roll back TLS firmware was deployed before the server served TLS. Fix the server; the rolled-back devices keep working on HTTP

Reading a device's reset reason

Every boot prints rst:0x… (REASON) on the serial console, and the gateway also reports it to the backend (Diagnostics → Last reset):

flowchart TB
    R{"reset reason"} --> P["poweron"] --> PW["power cycle, or a brown-out? check the supply and cable"]
    R --> B["brownout"] --> BW["the supply sagged: a better 5 V source"]
    R --> SW["software"] --> SWQ{"the line before 'Rebooting…'?"}
    SWQ -- "abort() / ESP_ERROR_CHECK failed" --> EC["read the failed expression and file:line"]
    SWQ -- "none" --> INT["intentional: after an OTA update, or a configuration change"]
    R --> PA["panic"] --> PB["a crash: decode the backtrace with idf.py monitor and the matching .elf"]
    R --> WDT["int_wdt / task_wdt / wdt"] --> TW["a task stuck: the backtrace shows which"]
Loading

Collecting evidence for a bug report

  1. The full serial log from power-on to the failure (idf.py monitor | tee log.txt).
  2. The firmware version and the ELF file SHA256 from the boot banner.
  3. The server log around that time, and the failing request and response.
  4. Activity entries around that time.
  5. For a gateway: a screenshot of its Diagnostics panel.
⚠️ **GitHub.com Fallback** ⚠️