Firmware Management and OTA - Kush-Kelaiya22/imPress GitHub Wiki
How firmware gets from a build to classroom devices: the registry, approval, staged deployments, what each device does, rollback, and releasing a new version. References: docs/guides/ota-updates.md, docs/firmware/OTA_ARCHITECTURE.md, docs/firmware/FIRMWARE_VERSIONING.md.
- What can be updated over the air
- The big picture
- The firmware registry
- Uploading an image
- Approve, deprecate, delete
- Deploying
- Strategy: canary, batches, limits
- Who is excluded, and why
- Following a deployment
- The per-device state machine
- What the gateway does during an update
- What the hub does during an update
- Rollback
- Releasing a new version
- Safety properties
- Troubleshooting updates
| Device | Over the air | How |
|---|---|---|
| Gateway (C6) | ✅ | it downloads the image itself over Wi-Fi |
| Hub (S3) | ✅ | prompted through its gateway, it briefly joins Wi-Fi (the "OTA hop"), downloads, installs and reboots; if nothing is pending it returns to the mesh without rebooting |
| Student module | ❌ | no transport: ESP-NOW frames are at most 250 bytes, and modules have no Wi-Fi credentials. Update over USB (Firmware: Build and Flash) |
First v2.1 install on gateways and hubs: over USB, once. v2.1 changed the hub–gateway SPI format, and the gateway's bootloader gained rollback. After that, both update over the air.
flowchart LR
B["Build<br/>impress_class_c6.bin"] --> U["Upload<br/>(Firmware page)"]
U --> V{"valid imPress image?<br/>chip · project · X.Y.Z ·<br/>SHA-256 · size · signature"}
V -- no --> R["refused with the reason"]
V -- yes --> A["registry: uploaded"]
A --> AP["Approve"] --> D["Deploy<br/>(devices + strategy)"]
D --> E["deployment engine<br/>(every 5 s)"]
E -- "offer + prompt" --> G["gateway"]
G -- "SPI prompt" --> S3["hub"]
G -- "check · download · status" --> E
S3 -- "check · download (Wi-Fi hop)" --> E
S3 -- "result after reboot (via gateway)" --> E
Every uploaded image is a row in firmware_artifacts, plus a file named by its SHA-256 (backend/firmware_bins/<sha256>.bin):
| Property | Rule |
|---|---|
| Facts come from the image | target (c6, s3, student), chip, project, version, ESP-IDF version, build date and ELF hash are all read from the binary, never from the file name or a form field |
| Immutable | a (target, version) pair names exactly one set of bytes. Uploading the same bytes again is a no-op; different bytes under an existing version are refused (409): bump the version instead |
| Content-addressed | stored as <sha256>.bin, so no user input ever reaches a file path |
| Status |
uploaded → approved → deprecated (and back to approved if needed) |
| Channel |
stable or beta: a label for your own use |
| Release notes | free text; editable |
| Signer | which key signed it, or unsigned (Security) |
Firmware → Upload an image: drop build/impress_<project>.bin on the box. The upload is checked:
| Check | Refuses |
|---|---|
magic byte 0xE9, a plausible segment count, a minimum size |
random files |
| chip id ∈ {ESP32, ESP32-S3, ESP32-C6} | images for other chips |
an app descriptor (magic 0xABCD5432) |
bootloader or partition-table binaries |
project ∈ {impress_class_c6, impress_class_s3, impress_student}, built for its own chip |
other applications; a gateway project built for the wrong chip |
version is X.Y.Z
|
builds without a proper version.txt
|
| the image's appended SHA-256 is present and correct | corrupted or truncated files |
| the image fits the target's OTA slot | images that couldn't be installed |
| a valid signature, if any is attached; a signature by the site key, if one is configured | tampered images; unsigned or foreign images when signing is required |
| at most 4 MB | oversized uploads (413) |
Every refusal names the problem, for example must be built for esp32c6, image version '1' is not X.Y.Z, or the image is not signed with this server's firmware key.
To check an image before uploading:
scripts/verify_firmware.py firmware/class_c6/build/impress_class_c6.bin [--key site.pub]| Action | Effect |
|---|---|
| Approve | the image can be deployed. Only approved images are offered anywhere |
| Deprecate | no new deployments use it. Devices already running it, or about to install it, are unaffected, and it stays downloadable for them |
| Delete | only when the image is not approved, and no device runs it or has it pending: those images are the way back. It removes the row and the file |
| History | its upload, approval, pushes and every result devices reported |
The page groups images by device type, newest first, and shows how many devices run each image or have it pending. The highest approved version is marked latest.
Deploy on an approved image opens the rollout dialog:
-
Choose devices (the filters combine):
- specific devices;
- every device of the image's type;
- devices running a given version;
- a classroom's devices (its gateway and the hub it relays).
- Set the strategy (below).
- Allow downgrade only for a deliberate rollback.
- Click Preview. It lists every device that will be included, with its stage, and every one that won't, with the reason.
- Click Start. The canary stage begins at once.
Push OTA on the Modules page is the same thing for one device: a one-device deployment with the same checks.
| Setting | Default | Meaning |
|---|---|---|
| Canary | 1 | devices in the first stage |
| Batch size | 5 | devices in each later stage |
| At once (max concurrent) | 5 | devices updating at the same time (network load) |
| Failures allowed per stage | 0 | a finished stage with more failures than this pauses the deployment |
| Attempts | 2 | tries per device before it counts as failed |
| Timeout | 900 s | per attempt |
flowchart LR
C["stage 0: canary"] --> H1{"failures ≤ allowed?"}
H1 -- yes --> B1["stage 1: next batch"] --> H2{"…"} --> BN["last batch"] --> DONE["completed"]
H1 -- no --> P["paused<br/>(the note says which stage)"]
P -- "Resume (accept the failures)" --> B1
P -- Cancel --> X["cancelled"]
So with the defaults, one device updates first. Only if it succeeds do the others follow, five at a time.
| Reason in the preview | Meaning |
|---|---|
incompatible: a s3 device can't run a c6 image |
wrong device type |
no over-the-air path to student modules yet (update over serial) |
student modules |
device is disabled |
an admin switched it off |
image is not signed with the site firmware key (#66) |
signing is required, and this image isn't signed by the site key |
already running 2.2.0 |
nothing to do |
would downgrade 2.2.0 → 2.1.0 (confirm a rollback) |
tick Allow downgrade if that's intended |
already in deployment #7 |
busy in another rollout |
The Deployments list shows each rollout's image, state and counts per state, and refreshes itself while a rollout runs. Details shows every device: its stage, state, attempts, versions before and after, and the error if any.
| Control | Effect |
|---|---|
| Pause | no new stage starts; devices already updating finish |
| Resume | continues; accepts the failures that caused a pause |
| Cancel | devices still waiting, queued or precheck are cancelled (nothing has been written to their flash); devices further along finish and report |
Durability:
- Restarts: everything is stored in the database. The scheduler runs every 5 s and is idempotent, so after a backend restart running deployments carry on where they were.
- Double submits: an idempotency key makes a double submit return the same deployment instead of starting two.
stateDiagram-v2
[*] --> waiting: in a later stage
[*] --> queued: stage active
waiting --> queued: stage starts (concurrency permitting)
queued --> precheck: device asks firmware/check
precheck --> downloading
downloading --> verifying
verifying --> installing
installing --> rebooting
rebooting --> health_check
health_check --> success: running the expected version
queued --> cancelled
precheck --> cancelled
waiting --> cancelled
queued --> unreachable: timeout
downloading --> timed_out: timeout
rebooting --> rolled_back: the bootloader reverted
health_check --> rolled_back
downloading --> failed
installing --> failed
failed --> waiting: retry (attempts left)
timed_out --> waiting: retry
unreachable --> waiting: retry
| Rule | Why |
|---|---|
| Forward only. A device may skip states (the hub reports only its final result) but never go back: 409. Repeating the current state is harmless | relayed duplicates and lost acknowledgements can't confuse it |
Success needs proof. success counts only when the device reports the expected version running after the reboot
|
a finished download or a reboot isn't success |
Rollback is recorded as such. rolled_back keeps the device's old version, and the bad image isn't offered again |
|
Bounded retries. failed, timed_out and unreachable are retried while attempts remain |
|
Timeouts. An attempt that never started becomes unreachable; one that stalled becomes timed_out
|
power loss, network trouble |
The server observes progress too. Asking firmware/check moves a device to precheck; serving the download moves it to downloading
|
progress is visible even from older clients |
sequenceDiagram
participant BE as Backend
participant C6 as Gateway (running v1)
participant BL as Bootloader
BE->>C6: WS ota_update {device_type: c6, mac}
C6->>BE: firmware/check → v2, sha256, size (precheck)
C6->>BE: GET firmware/download (downloading)
Note over C6: writes into the inactive slot,<br/>hashing the stream (SHA-256)
C6->>BE: ota/status verifying
Note over C6: digest == offered sha256?<br/>esp_ota_end (chip, segments, hash, signature if required)
C6->>BE: ota/status installing → rebooting
C6->>BL: reboot (NVS: ota_target = v2)
BL->>C6: boot v2 (pending verification)
Note over C6: SPI up, Wi-Fi up, registration answered 200
C6->>BE: ota/status health_check
Note over C6: mark the image valid
C6->>BE: ota/status success (version v2)
Note over C6,BL: no Wi-Fi or backend within 120 s → reboot →<br/>bootloader reverts → v1 reports rolled_back
The gateway also checks for an update at every boot, so it picks up prompts it missed while offline. It ignores an offer it can't verify: a missing or short SHA-256, a non-semver version, or a zero size. It writes nothing to flash if the download's HTTP status or length is wrong.
- The backend sends the prompt to the hub's gateway, which forwards it to the hub over SPI.
- The hub joins Wi-Fi (the mesh keeps running), registers, and asks
firmware/check. - Nothing pending: it disconnects and goes back to the mesh channel, without rebooting.
- Update pending: it downloads (checking the HTTP status and size), installs and reboots.
- The new image marks itself valid once the mesh and SPI are up. A crash before that makes the bootloader revert.
- After the reboot, the hub reports
applied(orrolled_back, orfailedwith an error code) through the gateway.
| Kind | How |
|---|---|
| Automatic, on the device | a new image must prove itself after the reboot. Otherwise the bootloader boots the previous image, which reports rolled_back. The gateway's proof: SPI, Wi-Fi and a successful registration within 120 s. The hub's proof: mesh and SPI start up |
| Operator, for a fleet | deploy an older approved image with Allow downgrade ticked. It is recorded as a rollback deployment |
| Manual, for one stuck device | flash it over USB (Recovery Procedures) |
-
Bump the version in
firmware/<project>/version.txtfor each project that changed, and inVERSIONfor a release. A repository test requires all four to match. -
Build:
idf.py build, or use the CI artifact (each image comes withSHA256SUMS). With signing on:scripts/build_signed.sh(Security). -
Check it:
scripts/verify_firmware.py <image> [--key site.pub]. - Upload it, and check the version, target and chip the page shows.
- Approve it.
- Deploy it with a canary.
- Write release notes.
What a version number doesn't tell you:
- Bootloader and partitions: two devices on 2.1.0 can have different bootloaders or partition tables, because OTA never changes those.
- Build profile: an image built with the reduced-flash or signed profile reports the same version as a normal build. Deploy such builds only to the devices built the same way.
| Property | Mechanism |
|---|---|
| A crashing image can't brick a gateway or hub | app rollback and self-verification after reboot |
| Only an image an admin deployed can be downloaded |
firmware/download serves only that device's pending version (403 otherwise) |
| A corrupted download is never installed | SHA-256 against the offer (gateway); esp_ota_end's own checks (both) |
| A foreign image is never installed (with signing) | the signature check in esp_ota_end, and the backend refusing it at upload |
| A failed canary stops the rollout | stage failure limits pause the deployment |
| A no-op prompt doesn't disturb a class | the hub returns to the mesh without rebooting |
| A restart doesn't lose a rollout | durable state, idempotent scheduler |
| Symptom | Check |
|---|---|
| The hub never starts a hop | Is the hub linked to a gateway (gateway_id)? Is that gateway connected (online, WebSocket up)? |
OTA aborted — could not connect to WiFi (hub log) |
the hub's Wi-Fi settings (NVS wifi_ssid, wifi_pass) |
No pending update (hub log) |
nothing pending for it, or it's already on that version |
| Download 403 | the requested version isn't the device's pending one: deploy again |
| Download 404 / 410 | no registered image for that type and version / its file was deleted from the store |
failed with sha256 mismatch
|
corruption on the way. It is retried while attempts remain |
failed from esp_ota_end on a signed device |
the image isn't signed with the key the device trusts: check with verify_firmware.py --key
|
rolled_back |
the new image couldn't prove itself. Read its serial log, fix, bump the version, deploy again |
| The deployment is paused | open Details, read the failed device's error; Resume to accept, or Cancel |
A device stays queued
|
it never asked firmware/check: offline, no WebSocket, or (hub) no gateway. It becomes unreachable at the timeout |