Experiment formal browser native adapter integration plan 01 - kmosoti/FabricO11y GitHub Wiki

Prospective AB/C native adapter integration

Read-only preparation at browser harness 3aaa71b7bb175d7b18157e143cbcb4bad8bbd969 and frozen product 8fb6f413179b094288316e25845c137440ba23fa in candidate checkout A. No source edit, import, test, compiler, native probe or browser run occurred. Actual source controls and native/browser admission remain Olo's separate cells. C-084/X-140 supersede the pidfd-PGID proposal with delegated per-launch cgroups; that earlier proposal is historical and must not be implemented.

Existing executable pieces and precise gaps

  • tools/ui/session_clock.py:34-59 already freezes executable/helper bytes and device/inode, validates the actual PID's executable/helper maps and records start ticks, cgroup and loader libc. Its constructor at 77-95 only starts the Rust interactive probe; probe command/stdout/close paths at 108-127 cannot directly drive fabric-server serve CONFIG. Native main removes the admitted compiled helper at 236-246. A later browser cell needs a retained immutable helper plus its actual compile/native-admission receipt, not a vanished path or a rebuild assumed identical.
  • The packaged server's actual authorization clock is unix_s in candidate crates/fabric-server/src/lib.rs:49-50, through SystemClock. Console middleware calls it for rate bookkeeping (console.rs:375) and again for cookie authentication (:398); successful access updates seen (access.rs:788-807). A single HTTP read therefore does not imply exactly one REALTIME ledger row. Background/runtime calls and these two middleware calls must not be mistaken for a unique authorization invocation.
  • C ledger tools/ui/session_clock.c:84-92 has PID/frame sequence, offset and raw/returned wall time, but neither native TID nor direct monotonic sample nor unique invocation index. Controller receipt time/wall subtraction cannot fill server_call_monotonic_ns. Current native ledger equality at session_clock.py:164-166 also assumes the old exact six-field record.
  • live_browser_check.py:118-171 binds binary/build paths and hashes, not the full immutable package/inspection/provenance receipts. It snapshots binaries and packaged assets (:398-434), uses real serve (:466), genuinely enrolls resident/UV credentials and reads the protected cookie (:580-601), but has no expiry-only mode, AB/C state loop or native clock channel.
  • Existing direct TLS cookie requests (:302-329) do not issue a preparatory session GET and can preserve untouched B. ConsoleBridge.request:350-367 deliberately does issue one and may step up; use its existing private low-level request plumbing for the registered expiry path, never silently weaken that public command contract. ConsoleBridge.poll_ui:372-394 pauses real polling.

Narrow clock and cgroup integration

Retain the existing SessionClock channel/frame/receiver implementation. Add an explicit server argv/owned-spawn entry alongside its unchanged probe interface; only the expiry adapter uses it. The cgroup launcher is tools/bench/labs/completion/enter_group.py:1-4, backed by cgroups.py:33-57. Start the Python entry helper in an ordinary environment, enter the exact owned subgroup, then install the server-only LD_PRELOAD and FABRIC_CLOCK_FD values immediately before same-PID exec of the admitted ELF. Pass only the intended channel descriptor. Preloading the Python entry helper is wrong: the shim constructor (session_clock.c:42-57) consumes the initial frame, unsets those keys and sets CLOEXEC, so its subsequent exec would lose the server interception/channel. Register this failure and a positive exact-server mapping/FD control before extending the entry helper; do not use threaded preexec_fn or a parallel browser/controller class.

Set and verify the existing combined4GB/CPU200%/Tasks512 parent-service cap before cgroups.delegate(), retain that service handle, and give each launched subgroup explicit limits no larger than it. require_limits:78-91 returns the current group, not the service ancestor; after delegation that is supervisor. cgroups.delegate:42 sets its own20/16GiB defaults, which are constrained by the stricter ancestor but are not a4GB direct-group receipt. Do not later pass supervisor.name to the existing systemctl service-property call or mistake independent subgroup limits for the aggregate cap. Pass the explicit browser subgroup into ConsoleBridge's already implemented validation (:80-102).

Proposed native record extension, requiring its own trust registration first: retain every original field and add tid, monotonic_ns and call_index to REALTIME observations. Sample the unshifted native CLOCK_MONOTONIC via the saved underlying function during that same REALTIME wrapper invocation, immediately after saving the actual REALTIME status/errno. Preserve the original return, errno, clock IDs, offset/frame handling and fabric_clock_last six-u64 ABI; append no fields to that exported returned struct. A separately named timing accessor lets the existing Rust probe bind the new fields to independent kernel brackets without changing the old oracle row shape. The call index is strictly increasing per original target process under the existing mutex. With positive 32-bit PID/TID, sequence<=63, offset<=28860, CLOCK_REALTIME=0, u64 nanoseconds and index bounded by the unchanged 1MiB ledger, the compact decimal JSON maximum is 201 bytes. Register the literal maximum/overflow/call-join controls; this is a static size calculation, not a compiled result. Preserve the 256-byte record, 1MiB ledger, 64 frames and offset ceiling; do not sample away calls or increase capacity to hide native volume/lock failures.

Request-join decision still needs registration. The safest small candidate is complete serialized request-window evidence: collect every target REALTIME invocation inside the controller's before-send/after-read monotonic bracket, retain PID/TID/sequence/call index, and require all possible authorization calls to return the exact expected second. A straddling invocation, mixed second, wrong frame/PID, missing call or truncated ledger is inconclusive. Current scalar row API must not silently select a convenient background call. Register either an independently justified complete-call-set join that supplies a specific actual native witness, or an explicit direct-monotonic bracket shape before native implementation. The frozen server has no request-ID/TID clock hook; adding one would change the candidate and is outside this plan. A small ordinary-time native join cell against the exact packaged server should measure this gap before attempting the 32 expiry observations. Preserve old six-field equality plus independent validation of every new field rather than dropping old binding.

Minimal serial fixture path

Use one explicit expiry-only branch in the existing live browser harness, mutually exclusive with role matrix, polling, pressure, cursor, recovery and shell lifecycle. Reuse TLS/NSS/WebDriver/passkey setup and trusted production URL; do not run the entire existing long lifecycle to reach expiry. Firefox already imports the owned CA into a private NSS profile and uses acceptInsecureCerts=false (live_browser_check.py:499-505,524-531). Chrome uses its explicitly limited leaf SPKI fixture allowlist (:518-521). HTTP/demo browser_check is not a substitute. Freeze actual browser/driver/NSS versions and hashes; no owner-host trust changes.

Create AB and C state directories with exclusive mkdir, retain protected FDs, observe empty entries and record canonical path/device/inode before starting the server. Both use the same already frozen package ELF, console assets and helper; separate native process_binding receipts must demonstrate the loaded identities. Fresh private browser profiles avoid prior-cookie state; no state/profile copy. For AB obtain A and B through real resident/UV ceremonies with no previous cookie on login finish (candidate console.rs:549-565 otherwise retires it). Cookie removal in a private browser context is not logout; keep token values solely in bounded private memory. Capture each actual issue/expiry and final successful baseline, then send no incidental authenticated B request. Disable polling and automatic step-up before final observations. Public hashes/expiry alone cannot prove that baseline.

The actual owner sessions view exposes issued_unix_s, last_seen_unix_s and expires_unix_s (candidate access.rs:173-177,1147-1159). During ordinary setup, after draining/pausing UI reads, a final controlled sessions-list read under B can observe both hashed identities; its own B authentication refresh is the recorded B baseline. Bind the returned metadata and that actual native request window, then freeze the observation inputs. Keep C's same-token issuance/seen metadata equally explicit. Do not derive last-seen from browser Date or assume that enrollment completion was the last authenticated request.

Preserve the exact two AB rows and thirty C rows; no row rewriting or fallback. Keep the browser's ordinary Date/monotonic clocks and helper-free maps/control FD/environment witnesses. Derive forward offsets from independent raw host observations and validate actual applied sequence. Adjacent target seconds use the same offset and real waiting; lateness never permits backward repair. After the actual 401, explicit UI Check session must lock/clear protected memory. Stop and reap every AB owned group, record kernel empty observations and actual state identity, then start C strictly afterward. Do not start C on leader exit or an earlier stop boolean. The same host FixtureCadence object survives the transition; neither its 550ms pacing nor the global run_start+240s deadline resets.

All setup, both states, dispatches and stops share 240 seconds. The 32 dispatches alone require at least 17.05 seconds between first/last at 550ms; native ceremony, NSS/browser startup, channel drain and stop times are unmeasured. There is no static completion guarantee. Check remaining budget before each phase, preserve overrun as failure, and keep outer1800s, combined3.5/4GB, CPU200%,Tasks512,no swap, 512MiB fixtures and full1GiB reservation/fresh census. Compile/helper admission and immutable input inspection belong to earlier separately contained cells; do not hide fixture setup outside the measured deadline. Require fresh #41 and actual merged harness/source/artifact/tool/driver pins before browser acceptance.

C-084 trust diff design, source still frozen

Extend only existing BrowserFixtureCleanupTests, its fixture and clock protocol first; production remains unchanged until root source green and actual new red. Add an explicit owned_subgroups registry keyed by each exact Popen object while preserving current (label,Popen) pairs for sampler compatibility. Each registry entry comes from exclusive creation under the delegated owned service and carries canonical path, held directory FD/device/inode, actual placement, original PID/ start identity and raw bounded limit/events observations. Refuse missing delegation, missing/wrong/replaced group, absent cgroup.kill and unconfirmed entry before target launch; never signal from a caller-supplied path or numeric PGID.

Prospective literal control Preserved outcome
Missing delegation/group or cgroup.kill Refuse before target exec marker; no fallback
Entry helper remains outside assigned subgroup Refuse before target exec, exact placement diagnostic
Exited leader plus TERM-resistant child calling setsid Actual cgroup.kill stops child despite new session; original child is reaped/absent
Kernel events remains populated1 through five-second KILL bound Retain scratch, failure receipt, no stopped-groups claim
One owned subgroup stop deliberately refused Attempt/stop every other group, retain failed group/scratch
Wrong/replaced registry identity No signal or cgroup.kill write to unrelated group

Keep the five old negative controls and empty-finalizer positive unchanged in outcome. The injected stop boundary covers old os.killpg and new subgroup/pidfd operations; independent fixture cleanup remains separate and archives failures. Actual delegated source/runtime cells need a new exact root preflight; mocks alone cannot establish kernel placement, kill, reaping or browser sandbox fit.

Kernel cgroup v2 documentation states that cgroup.procs excludes zombies and an otherwise empty zombie-associated group may be removed. cgroup.kill kills its subtree and handles concurrent forks and migrations. Thus populated0 and rmdir alone cannot satisfy the existing test_role_matrix.py:535 child-/proc-absence assertion. Retain/restore the controller's subreaper state, bind/reap exact adopted children through pidfds before subgroup removal, and preserve failure on uncertainty. Use bounded own children enumeration with UID/parent/start/cgroup revalidation, not arbitrary waitpid(-1); enumerate the controller's task children, including zombies absent from cgroup.procs, while the original subgroup directory still exists. The original Popen leaders keep their own waits. Existing PIDFD fixture cleanup and runner.py:258 are reusable evidence of these mechanisms, not a new passing browser result. Record empty kernel events and exact reaping as distinct observations. Preserve15s TERM/5s KILL/2s sampler and attempted stops after failures, archive before cleanup, and retain scratch on any uncertain stop.

Prospective production budget, to measure rather than promise: C-084 launch/ closeout roughly50-80 net plus exact reaping; controller/server bootstrap roughly 50-80, C/probe timing roughly30-50, expiry-only orchestration roughly100-150. Implement as separate small reviewable deltas after their trust controls/red; avoid a full new orchestrator or changes to frozen candidate/Rust authorization.