Superseded (2026-10-14)

2026-10-14: The "Panorama" layout and its calibration were removed. The two front cameras of the supported mower are a stereo pair with the same field of view: far objects sit at identical image positions and only near objects shift, so stitching cannot widen the view. The two frames stay useful later as a stereo pair for depth estimation, not as a panorama; the camera panel keeps the single and the side-by-side layouts.

The camera panel's "Panorama" layout blends the LUBA 2 AWD X's left and right front camera into one wide canvas. Its first version placed the two frames side by side with an overlap, a vertical offset and an optional mirror (sliders, plus an "Auto" overlap from a cross-correlation of the facing edges). That is a translation: as soon as the cameras are rolled or toed in against each other, edges in the overlap appear doubled or bent. The owner asked for a proper calibration and for the choice of method.

The forces:

• The cameras are fixed. Their relative pose never changes, so one calibration per mower (for example in the dock, in front of a scene with structure) is enough; it does not have to run per frame.
• A planar homography is the right model. For two cameras close together looking at a scene mostly at a distance, the mapping from one image to the other is well described by a 3 x 3 homography; a pure translation is its special case. Every method can deliver one, and the renderer only needs one.
• Several sources of correspondences. Local image features work offline and cost nothing but fail on a featureless lawn; a vision model recognises the same object in both pictures even across exposure differences, but its pixel coordinates are rough, sometimes wrong, and every call costs money; the sliders always work.
• The browser's constraints. The CSP has script-src 'self' with hashes and no 'wasm-unsafe-eval' (apps/web/scripts/csp-hashes.mjs); the service worker precaches every built asset (apps/web/src/service-worker.ts), so any lazily loaded chunk is still downloaded by every installation, on 4G phones too.
• Architecture rule. Pure geometry modules without DOM, rendering separate, the calibration methods behind one interface, Vision stays a service client.

One interface, four methods, the homography always computed in the browser.

• CalibrationMethod (apps/web/src/lib/camera/calibration/types.ts): id, label(), estimate(left, right, options) resolving to a CalibrationResult with the homography (right frame to left frame, unit coordinates x / width, y / height, so it does not depend on the delivered resolution), a confidence (0 to 1), the number of inliers and of candidate matches. Methods:
  • manual: the existing sliders as a translation homography (plus the flip for a mirrored camera). It draws exactly the picture the slider layout drew.
  • correlation: the existing edge cross-correlation as a translation; not offered on its own, it is the fallback of features.
  • features ("Automatisch (lokal)"): ORB-style keypoints in plain TypeScript (FAST-9 corners ranked by the Harris response, spread over a grid, intensity-centroid orientation, 256-bit rotated BRIEF descriptors on a box-smoothed image), brute-force Hamming matching with Lowe's ratio test and a mutual check, then RANSAC. A mirrored camera is flipped back before matching and the flip folded into the homography. Fewer than 12 agreeing matches fall back to correlation, and the result says so.
  • ai ("KI-gestützt"): POST /api/v1/vision/calibration/correspondences with the left and right JPEG of the same moment; the configured provider returns 8 to 16 corresponding points ({left: [x, y], right: [x, y], label} in pixels of each image, strict JSON schema); the browser fits the homography with the same RANSAC as features. A mirrored camera needs no switch: it shows up as a homography with negative orientation.
• RANSAC with model selection (ransac.ts). Both an affine map (samples of three) and a full homography (samples of four) are fitted; the homography is kept only when it has clearly more inliers (at least 1.15 times plus two). The two frames share only the overlap strip, and perspective terms fitted from a narrow strip are poorly determined and bend the far side of the right frame; the affine map covers shift, roll, scale and shear and extrapolates calmly. Results that map the right frame to a non-convex shape, to less than 0.4 or more than 2.5 times its area, or far away from the left frame are rejected as implausible.
• Why the AI only supplies points. A language model is good at recognising "the same fence post" in two pictures, bad at arithmetic and at reporting exact pixels. Asking it for the matrix would make the result unverifiable; asking it for points lets RANSAC throw out the wrong ones, gives an inlier count the user can judge, and keeps one estimator for two sources. The Vision service therefore has no geometry: the new route validates the points (on both images, no duplicates, at least four) and returns them with the model and the cost. It is authorised with can_view on the mower (nothing is proposed or driven), needs the plan feature ai_recover, uses the caller's provider or the installation's like AI assist, records each call in provider_requests with kind = calibration (the cost estimate included) and has its own daily caps (MOWDECK_VISION_CALIBRATION_DAILY_REQUESTS, default 10 per user; MOWDECK_VISION_HOSTED_ACCOUNT_DAILY_CALIBRATIONS, default 20 per account with the installation's model), so calibrating never uses up the analyses a stuck mower needs. The web side reaches it through CorrespondenceSource ($lib/vision/calibration.ts); the calibration code knows no HTTP client.
• No OpenCV.js. The request suggested OpenCV.js (@techstark/opencv-js or a cdnjs build). Its WebAssembly needs 'wasm-unsafe-eval' in script-src, which the CSP does not grant, and its single-file build is about 8 to 10 MB, which the service worker would precache on every device although calibrating happens once per mower. With OpenCV loading never succeeding under the current CSP, features would always have fallen back. The TypeScript matcher is about 4 kB (2 kB gzip) as its own chunk, imported on demand when someone calibrates, runs in well under a second on 640 x 480 frames, needs no CSP change and is unit tested on synthetic pictures. If a more robust detector is ever needed (scale pyramids, AKAZE), it can be added behind the same interface; OpenCV would then need 'wasm-unsafe-eval' and an exclusion from the precache.
• Rendering (panorama-renderer.ts, calibration/geometry.ts). The panorama spans both frames; vertically it is cropped to the rows both cover (as before), unless they share less than a quarter of the height. When the homography's perspective terms are negligible over the frame (always for manual and correlation), the right frame is drawn with one setTransform. Otherwise it is cut into a triangle mesh; each triangle is drawn clipped with the affine transform that maps its three corners exactly, and the grid is refined (2 x 2, 4 x 3, 8 x 6, up to 32 x 24) until the deviation from the true homography is below half a pixel. At 640 x 480 and the cameras' 6 fps that is a few dozen clipped drawImage calls per frame, with no WebGL context and no per-pixel JavaScript. An overlap of up to half a frame (a slider layout) is blended with a feathered ramp across its whole width. The two front cameras are a stereo pair and see nearly the same scene, so their overlap is most of the frame; one homography aligns one depth, and blending the whole overlap doubles every object at another depth (parallax). Wider overlaps therefore keep the left frame untouched and let the right frame in only over a short ramp (64 columns) at the left frame's right edge: the panorama is the left picture plus the strip only the right camera sees, parallax ghosts stay confined to that ramp, and the same composition is what Vision should receive as "what the mower sees" once the calibration is stored server-side.
• Storage. The calibration is kept per mower and browser in the slider settings' key mowdeck.camera.panorama.<mowerId> as {version: 2, method, homography, overlapPercent?, offsetPercent?, mirror, capturedAt, confidence, inliers?}. An entry from before ({overlap, offset, mirror}) reads as a manual calibration with the equivalent homography. The default sliders remove the entry; anything broken or implausible reads as the sliders.
• UI. The calibration card offers the three methods (camera-calibration-method-<id>); "Kalibrieren" captures the current pair (both tiles live), runs the method and shows the confidence, the inlier count, the model and cost for ai, and a before/after preview on the captured frames; nothing is stored until "Übernehmen".

• The panorama can be a real stitch with roll and perspective, calibrated once in the dock; the slider layout stays for the cases where no method finds structure.
• The AI method's cost is visible before (note in the card) and after (the call's estimate) each run and is accounted like every other model call; a calibration call never creates a proposal.
• The calibration lives in the browser: another browser or device calibrates again. Syncing it through Fleet would be a small follow-up if the owner wants it.
• The homography assumes a mostly distant scene: objects close to the cameras (the dock itself) still show parallax in the overlap. Calibrating in front of a scene a few metres away gives the best result for mowing.
• Only the real cameras can show how well each method works on the LUBA 2 AWD X's pictures (exposure differences, lens distortion, the actual overlap), how fast the mesh warp is on a phone, and what a calibration costs with the owner's model.

• One time budget, end to end. The provider retries a slow model (MOWDECK_VISION_PROVIDER_TIMEOUT_SECONDS per attempt, MOWDECK_VISION_PROVIDER_RETRIES more attempts, plus a round for an invalid answer), so a calibration call could outlast any browser deadline. The use case now bounds the whole call with MOWDECK_VISION_CALIBRATION_TIMEOUT_SECONDS (default 150 s, all attempts included), cancels the provider beyond it, records the call as outcome = timeout and answers 504 vision-provider-timeout. The browser's deadline for model calls sits a little above that budget (180 s), so the normal outcome of a slow model is Vision's verdict; the web shows both the 504 and its own deadline as the unreachable failure (same advice: try again).
• Discard cancels the call. CorrespondenceSource.correspondences takes the flow's AbortSignal; discarding a running AI calibration (or starting over, or switching the method) aborts the request instead of letting it finish unseen and be paid for, and a late answer never becomes a candidate.
• Housekeeping from the same review: supportConfidence moved to ransac.ts (both RANSAC methods use it; ai no longer imports from features), the "Auto" overlap estimate moved from the card into the flow with the DOM grab injected, and the renderer takes the correlation's copy width from the calibration module instead of repeating it.

---

Mirrored from docs/adr/0037-camera-panorama-calibration.md at commit cb46ac5074dee783a333b60ecc6a9b05a1c7cf33. Edit the file in the repository, not this wiki page.