ADR 0032 Live Camera Over Agora - lukislp/mowdeck GitHub Wiki

ADR 0032: Live camera over Agora, joined directly by the browser

Status

Accepted (2026-10-10)

Context

The LUBA has cameras, and the plan lists a live camera for Phase 5 ("Streaming-Weg erst klären"). Spike 0001 (section 3.2) found that Mammotion video goes through Agora's cloud only: PyMammotion's get_stream_subscription(device_name, iot_id) asks the Mammotion cloud for an Agora stream token (appid, channelName, token, uid, a list of cameras, and with openEncrypt a key and a base64 salt for AES-256-GCM2), and requesting the token is what makes the mower publish (camera_stream.py, client.py of PyMammotion 0.10.9). get_video_resource reports the 4G video budget. No local RTSP or WebRTC endpoint is known. The owner decided that MowDeck uses this cloud path.

Mammotion-HA joins the Agora channel on the server with pyagorartc and bridges the media to the browser over WebRTC; encrypted channels show no picture there because pyagorartc cannot decrypt (stream_session.py, camera.py). Its camera entities target the publisher uid "cameraStates slot + 1" and share the token's one viewer uid.

The forces: video is heavy (a media server per viewer costs CPU and bandwidth in a service that otherwise only moves small JSON), encryption must work, the token is short-lived and secret, the viewer's plan and rights must be checked, and nothing may leak between accounts.

Decision

The browser joins the Agora channel directly with the Agora Web SDK (agora-rtc-sdk-ng, pinned to 4.24.8, MIT). MowDeck relays only the credentials; no media passes through any MowDeck service.

  • Adapter (mowdeck-control-mammotion): MammotionCloudConnection.get_camera_session() calls get_stream_subscription (and get_video_resource, best effort) and maps them to the contract CameraSession (mowdeck_contracts.v1.camera): provider (agora), app_id, channel, token, uid, cameras (id, name, publisher uid = slot + 1), encryption (aes-256-gcm2, key verbatim, salt base64) or null, expires_at (read from the AccessToken2 claims: issue time plus the shortest of the token and privilege lifetimes; ten minutes when unreadable), quota (used_minutes, total_minutes) or null. The camera is an optional capability of a connection (mowdeck_control.CameraCapable, checked with isinstance); cloud connections report capabilities.camera = cloud (except the Luba 1, which has no camera), the BLE bridge answers None, and the simulator returns a session with provider fake.
  • Device-Gateway: answers the core NATS request gateway.mower.<id>.camera.session (CameraSessionRequest / CameraSessionReply with exactly one of session or error not_found|unsupported|unavailable|forbidden) for the mowers of its shard; other shards stay silent. The account of the request must own the mower, like for commands. Metric mowdeck_device_gateway_camera_sessions_total{outcome}. The subject lives below gateway. because the stream DEVICE captures device.>: a request on a stream subject is also stored and acknowledged by JetStream, and that acknowledgement would race the gateway's reply.
  • Fleet: POST /api/v1/fleet/mowers/{id}/camera/session checks can_view on the mower (404 semantics) and the plan feature camera, relays the request (timeout MOWDECK_CAMERA_REQUEST_TIMEOUT_SECONDS, 15 s), writes the audit row mower.camera_opened (who, when, provider; never the token) and answers with the session and Cache-Control: no-store. 409 camera-unsupported when the connection has no camera, 503 when no gateway or no vendor cloud answers.
  • Web app: a camera panel on the mower detail page and a full-screen page /mowers/{id}/camera. The SDK is imported dynamically (its own chunk, about 1.5 MB, 420 KB gzipped). The client joins in mode live as audience, calls setEncryptionConfig(mode, key, salt) before the join when the session is encrypted (the SDK's EncryptionMode includes aes-256-gcm2, and its salt parameter is a Uint8Array, so the base64 salt is decoded; API reference), subscribes to the camera's publisher uid (or the first remote user with video) and attaches the raw media tracks to our own <video> element, muted (never the SDK's track.play(), whose inline styles the CSP refuses); the picker offers only uids that publish video. It requests a fresh session a minute before expires_at (or on token-privilege-will-expire) and passes the new token to renewToken; a changed channel rejoins. The SDK's log upload is switched off. WebRTC needs a secure context: Agora requires HTTPS or localhost (API reference), and the panel says so in words instead of failing silently.
  • CSP and switch: the Agora domains (https://*.agora.io, wss://*.agora.io, https://*.sd-rtn.com, wss://*.sd-rtn.com in connect-src, each also with :* because the edge servers use non-default ports, blob: in media-src and worker-src) are added at container start only with MOWDECK_WEB_CAMERA_ENABLED=true, which also sets cameraEnabled in /config.json. Without it the panel explains that live video through the vendor cloud is off; the simulator's test pattern works either way.
  • Plans: the feature camera (declared by Fleet) is part of home and fleet.

Options considered:

  1. Server-side join (Mammotion-HA's way): one media pipeline per viewer in a MowDeck service, no picture for encrypted channels with pyagorartc, and a GPL media stack in the gateway. Rejected.
  2. A MowDeck media relay (SFU) between Agora and the browser: the same costs, plus a second hop for the video. Rejected; possible later for a Vision service that needs frames (Phase 6), see the plan's "Spätere Erweiterungen".
  3. can_control instead of can_view: watching changes nothing on the mower, and whoever sees the mower's position and map may see its surroundings. The plan's roles table does not name the camera; the plan feature camera lets an installation switch it off per plan.

Consequences

  • No video load on MowDeck; the camera costs one small HTTP request per session and renewal. Encrypted channels can work because the SDK decrypts in the browser.
  • Privacy: the video and the viewer's IP address go to Agora's and Mammotion's servers, and Agora receives call statistics from the SDK. This is off by default (MOWDECK_WEB_CAMERA_ENABLED=false) and documented in the web app guide. The 4G video budget is shown when Mammotion reports it; requesting a session starts the mower's stream, so the panel closes the channel when it is closed or the page is left.
  • The token reaches the browser (as Mammotion's own app has it); it is short-lived, never logged by any service (repr excludes it, scripts print a redacted dump) and never stored.
  • The service worker precaches the SDK chunk with the rest of the build (once per app version); the page itself loads it only when a camera opens.
  • Stability (after the owner's second test, intermittent picture): the channel profile is now an option, default rtc (a subscribing participant) instead of live audience, kept per browser with the latency level (live only), the stream fallback (setStreamFallbackOption(uid, 2): low stream, then audio only), a data saver (setRemoteVideoStreamType(uid, 1)), and an automatic rejoin with a fresh session and a backoff after DISCONNECTED or a RECONNECTING longer than 20 s. Publish, unpublish and subscribe events run in a queue, so a uid that publishes again after an unpublish is subscribed again. The panel's "Diagnose" line shows what the SDK measures (getRemoteVideoStats, getRTCStats, connection events) so a bad picture can be told from a bad network.
  • One inline style remains in the CSP: the Agora SDK (4.24.8) gives every remote video track a 1 px probe <video> with style="position: fixed; top: 0; left: 0; width: 1px; height: 1px" to read its resolution; no option avoids it. With the camera enabled the container adds that string's hash ('sha256-cPxxYHvCiQ/kT6TlojjBofFPdVQmCg8k/CdPKQDf9zI=') to style-src, next to the existing 'unsafe-hashes'; a test renders the CSP both ways. Re-check the hash when the SDK is updated.
  • Both front cameras (after the owner saw the same picture under both camera buttons): the mower publishes camera slot n as Agora uid n + 1, but only the slots switched on in the stream token request. PyMammotion's MammotionHTTP.get_stream_subscription(iot_id, has_rear_camera, all_cameras=...) posts cameraStates to device-server/v1/stream/token: (1, 0, 0) by default (front left only, as the vendor app does), (1, 1, rear) with all_cameras=True; the adapter called it without the flag, so only uid 1 published and the old player fell back to it for both buttons. The adapter now asks for every camera (connection option all_cameras, default true; false restores the front left camera alone), the session's cameras carry a placement (front_left, front_right, rear), and the web app shows one camera or both front cameras side by side, each as its own subscription in its own element, never another camera's picture in a tile's place ("no data from this camera" after 8 s; a thumbnail comparison announces two identical pictures). No select-camera command is needed: there is no camera switch on one uid. Mammotion-HA's checkout used as reference here has a single camera entity that subscribes every published uid in one WebRTC offer (agora_websocket.py, _handle_add_video_stream); its device command device_agora_join_channel_with_position sends MulSetVideo(position=LEFT) (ALL only for the Yuka) and is used only for old firmware without fpv_info. Unverified on the LUBA: that cameraStates (1, 1, 0) makes the right camera publish under uid 2, and whether two streams count twice against the 4G allowance.
  • Unverified until the owner runs it against the LUBA: the channel mode (live audience next to the mower's publisher), the publisher uid mapping (cameraId + 1), the units of the video budget (read as seconds), whether the token's privilege expiry matches the session length, and whether the mower stops publishing by itself after the viewer leaves (PyMammotion's stop_stream is not called yet). The owner's test is one step: scripts/try_real_mower.py --camera, then the web app with MOWDECK_WEB_CAMERA_ENABLED=true over HTTPS.

Mirrored from docs/adr/0032-live-camera-over-agora.md at commit 6a87264c6123f07c8474e153053525dc1816feb7. Edit the file in the repository, not this wiki page.

⚠️ **GitHub.com Fallback** ⚠️