ADR 0032 Live Camera Over Agora - lukislp/mowdeck GitHub Wiki
Accepted (2026-10-10)
The LUBA has cameras, and the plan lists a live camera for Phase 5 ("Streaming-Weg erst klären"). Spike 0001 (section 3.2) found that Mammotion video goes through Agora's cloud only: PyMammotion's get_stream_subscription(device_name, iot_id) asks the Mammotion cloud for an Agora stream token (appid, channelName, token, uid, a list of cameras, and with openEncrypt a key and a base64 salt for AES-256-GCM2), and requesting the token is what makes the mower publish (camera_stream.py, client.py of PyMammotion 0.10.9). get_video_resource reports the 4G video budget. No local RTSP or WebRTC endpoint is known. The owner decided that MowDeck uses this cloud path.
Mammotion-HA joins the Agora channel on the server with pyagorartc and bridges the media to the browser over WebRTC; encrypted channels show no picture there because pyagorartc cannot decrypt (stream_session.py, camera.py). Its camera entities target the publisher uid "cameraStates slot + 1" and share the token's one viewer uid.
The forces: video is heavy (a media server per viewer costs CPU and bandwidth in a service that otherwise only moves small JSON), encryption must work, the token is short-lived and secret, the viewer's plan and rights must be checked, and nothing may leak between accounts.
The browser joins the Agora channel directly with the Agora Web SDK (agora-rtc-sdk-ng, pinned to 4.24.8, MIT). MowDeck relays only the credentials; no media passes through any MowDeck service.
-
Adapter (
mowdeck-control-mammotion):MammotionCloudConnection.get_camera_session()callsget_stream_subscription(andget_video_resource, best effort) and maps them to the contractCameraSession(mowdeck_contracts.v1.camera):provider(agora),app_id,channel,token,uid,cameras(id,name, publisheruid= slot + 1),encryption(aes-256-gcm2,keyverbatim,saltbase64) or null,expires_at(read from the AccessToken2 claims: issue time plus the shortest of the token and privilege lifetimes; ten minutes when unreadable),quota(used_minutes,total_minutes) or null. The camera is an optional capability of a connection (mowdeck_control.CameraCapable, checked withisinstance); cloud connections reportcapabilities.camera = cloud(except the Luba 1, which has no camera), the BLE bridge answersNone, and the simulator returns a session withproviderfake. -
Device-Gateway: answers the core NATS request
gateway.mower.<id>.camera.session(CameraSessionRequest/CameraSessionReplywith exactly one ofsessionorerrornot_found|unsupported|unavailable|forbidden) for the mowers of its shard; other shards stay silent. The account of the request must own the mower, like for commands. Metricmowdeck_device_gateway_camera_sessions_total{outcome}. The subject lives belowgateway.because the streamDEVICEcapturesdevice.>: a request on a stream subject is also stored and acknowledged by JetStream, and that acknowledgement would race the gateway's reply. -
Fleet:
POST /api/v1/fleet/mowers/{id}/camera/sessioncheckscan_viewon the mower (404 semantics) and the plan featurecamera, relays the request (timeoutMOWDECK_CAMERA_REQUEST_TIMEOUT_SECONDS, 15 s), writes the audit rowmower.camera_opened(who, when, provider; never the token) and answers with the session andCache-Control: no-store. 409camera-unsupportedwhen the connection has no camera, 503 when no gateway or no vendor cloud answers. -
Web app: a camera panel on the mower detail page and a full-screen page
/mowers/{id}/camera. The SDK is imported dynamically (its own chunk, about 1.5 MB, 420 KB gzipped). The client joins in modeliveasaudience, callssetEncryptionConfig(mode, key, salt)before the join when the session is encrypted (the SDK'sEncryptionModeincludesaes-256-gcm2, and its salt parameter is aUint8Array, so the base64 salt is decoded; API reference), subscribes to the camera's publisher uid (or the first remote user with video) and attaches the raw media tracks to our own<video>element, muted (never the SDK'strack.play(), whose inline styles the CSP refuses); the picker offers only uids that publish video. It requests a fresh session a minute beforeexpires_at(or ontoken-privilege-will-expire) and passes the new token torenewToken; a changed channel rejoins. The SDK's log upload is switched off. WebRTC needs a secure context: Agora requires HTTPS or localhost (API reference), and the panel says so in words instead of failing silently. -
CSP and switch: the Agora domains (
https://*.agora.io,wss://*.agora.io,https://*.sd-rtn.com,wss://*.sd-rtn.cominconnect-src, each also with:*because the edge servers use non-default ports,blob:inmedia-srcandworker-src) are added at container start only withMOWDECK_WEB_CAMERA_ENABLED=true, which also setscameraEnabledin/config.json. Without it the panel explains that live video through the vendor cloud is off; the simulator's test pattern works either way. -
Plans: the feature
camera(declared by Fleet) is part ofhomeandfleet.
Options considered:
-
Server-side join (Mammotion-HA's way): one media pipeline per viewer in a MowDeck service, no picture for encrypted channels with
pyagorartc, and a GPL media stack in the gateway. Rejected. - A MowDeck media relay (SFU) between Agora and the browser: the same costs, plus a second hop for the video. Rejected; possible later for a Vision service that needs frames (Phase 6), see the plan's "Spätere Erweiterungen".
-
can_controlinstead ofcan_view: watching changes nothing on the mower, and whoever sees the mower's position and map may see its surroundings. The plan's roles table does not name the camera; the plan featurecameralets an installation switch it off per plan.
- No video load on MowDeck; the camera costs one small HTTP request per session and renewal. Encrypted channels can work because the SDK decrypts in the browser.
-
Privacy: the video and the viewer's IP address go to Agora's and Mammotion's servers, and Agora receives call statistics from the SDK. This is off by default (
MOWDECK_WEB_CAMERA_ENABLED=false) and documented in the web app guide. The 4G video budget is shown when Mammotion reports it; requesting a session starts the mower's stream, so the panel closes the channel when it is closed or the page is left. - The token reaches the browser (as Mammotion's own app has it); it is short-lived, never logged by any service (
reprexcludes it, scripts print a redacted dump) and never stored. - The service worker precaches the SDK chunk with the rest of the build (once per app version); the page itself loads it only when a camera opens.
-
Stability (after the owner's second test, intermittent picture): the channel profile is now an option, default
rtc(a subscribing participant) instead ofliveaudience, kept per browser with the latency level (liveonly), the stream fallback (setStreamFallbackOption(uid, 2): low stream, then audio only), a data saver (setRemoteVideoStreamType(uid, 1)), and an automatic rejoin with a fresh session and a backoff afterDISCONNECTEDor aRECONNECTINGlonger than 20 s. Publish, unpublish and subscribe events run in a queue, so a uid that publishes again after an unpublish is subscribed again. The panel's "Diagnose" line shows what the SDK measures (getRemoteVideoStats,getRTCStats, connection events) so a bad picture can be told from a bad network. -
One inline style remains in the CSP: the Agora SDK (4.24.8) gives every remote video track a 1 px probe
<video>withstyle="position: fixed; top: 0; left: 0; width: 1px; height: 1px"to read its resolution; no option avoids it. With the camera enabled the container adds that string's hash ('sha256-cPxxYHvCiQ/kT6TlojjBofFPdVQmCg8k/CdPKQDf9zI=') tostyle-src, next to the existing'unsafe-hashes'; a test renders the CSP both ways. Re-check the hash when the SDK is updated. -
Both front cameras (after the owner saw the same picture under both camera buttons): the mower publishes camera slot n as Agora uid n + 1, but only the slots switched on in the stream token request. PyMammotion's
MammotionHTTP.get_stream_subscription(iot_id, has_rear_camera, all_cameras=...)postscameraStatestodevice-server/v1/stream/token:(1, 0, 0)by default (front left only, as the vendor app does),(1, 1, rear)withall_cameras=True; the adapter called it without the flag, so only uid 1 published and the old player fell back to it for both buttons. The adapter now asks for every camera (connection optionall_cameras, default true; false restores the front left camera alone), the session's cameras carry aplacement(front_left,front_right,rear), and the web app shows one camera or both front cameras side by side, each as its own subscription in its own element, never another camera's picture in a tile's place ("no data from this camera" after 8 s; a thumbnail comparison announces two identical pictures). No select-camera command is needed: there is no camera switch on one uid. Mammotion-HA's checkout used as reference here has a single camera entity that subscribes every published uid in one WebRTC offer (agora_websocket.py,_handle_add_video_stream); its device commanddevice_agora_join_channel_with_positionsendsMulSetVideo(position=LEFT)(ALLonly for the Yuka) and is used only for old firmware withoutfpv_info. Unverified on the LUBA: thatcameraStates (1, 1, 0)makes the right camera publish under uid 2, and whether two streams count twice against the 4G allowance. - Unverified until the owner runs it against the LUBA: the channel mode (
liveaudience next to the mower's publisher), the publisher uid mapping (cameraId + 1), the units of the video budget (read as seconds), whether the token's privilege expiry matches the session length, and whether the mower stops publishing by itself after the viewer leaves (PyMammotion'sstop_streamis not called yet). The owner's test is one step:scripts/try_real_mower.py --camera, then the web app withMOWDECK_WEB_CAMERA_ENABLED=trueover HTTPS.
Mirrored from docs/adr/0032-live-camera-over-agora.md at commit 6a87264c6123f07c8474e153053525dc1816feb7. Edit the file in the repository, not this wiki page.