Experiment benchmarks coupled labs query - kmosoti/FabricO11y GitHub Wiki

Historical repository record migrated on 2026-10-10. Original repository path: docs/experiments/benchmarks/coupled-labs-query.md Original SHA-256: 1b3188e689e5f4932d601b8c5a20dff3684725f036b27fc18f9b3158f1a01967 Stated outcomes, measured revisions, failures and limits remain historical. Local raw evidence is private and is not uploaded. Relative links and the data-drive prefix were normalized.

Coupled labs: query and availability packet

Current disposition: see completion preparation and executed run. The preparation status below is historical.

Status: prepared; Q4 not run. Accountable: query PI, coordinator until dispatch. Common plan governs admission. Own this packet and future coupled-query-* findings; coordinate any shared source changes.

Q4 — reuse discovery work with bounded storage snapshots

H1: a storage-owned snapshot reuses validated source descriptors, reducing total maintenance-plus-query CPU without unbounded retention or changed answers. H0: invalidation, reacquisition, copying or lifetime costs erase the benefit, or a lifecycle schedule violates coverage. Refreshing earlier is not itself a win.

Source/evidence: maintenance findings, discovery race, lifetime controls, read_catalog.rs (local checkout reference crates/fabric-server/src/read_catalog.rs; not present at origin/main), query.rs and catalog_maintenance_probe.rs (local checkout reference crates/fabric-server/examples/catalog_maintenance_probe.rs; not present at origin/main). The baseline is current lazy acquisition with the corrected discovery retry. First candidate retains only validated descriptors. Keep clone/shared ownership fixed initially; test ownership as a later ablation if attribution warrants it. No background worker, generic port, durable map or new persisted format initially.

Prepare an API/state sketch identifying generation, snapshot-relative coverage, active/pending/published sources, physical file access and invalidation. State exactly which listings/decodes disappear. Every held handle must have a release path; metadata ownership grants no file lease or extended query retention. Define reader/generation/byte limits, overflow behavior and exact fallback before coding. A candidate is not admitted with merely an Arc and no retention bound.

Correctness matrix: append during acquisition; publish between listing phases; reclaim immediately after publication; missing optional metadata; held reader across retention; cancelled reader; equal timestamp bounds and continuation. Test cold process versus warm catalog separately; do not call a restarted process cold disk cache. Preserve the existing deterministic Scan/Walk omission fixtures. Controls inject omitted/duplicate sources, stale generation acceptance and retained metadata after final drop; grading must detect each relevant violation. Expected retry/error must remain explicit, never silently produce an empty complete answer.

Performance matrix proposal: 1/64 Segments × 0/1/4 queries per append, bounded 32-append schedule from CQ2, same broad/selective query bytes and full chains. Preflight 1 Segment/1 query plus all lifecycle controls before the larger cells. Freeze archived CQ2 workload identity or register precise deviations; do not mix the repaired seed publication fixture with its historical failing version. Count discovery calls, open/read/decode bytes, descriptors reused/invalidated, refresh CPU, query CPU, lock wait/hold and oldest/live/retired generation bytes. Query-off reveals shifted work and held metadata.

Nominate only if total maintenance-plus-query CPU meets the common 10% rule at the declared primary read rate and all memory/latency/exactness guards survive. Other rates are guard/crossover evidence, not substitutes for a failed primary. Service ACK and Spool acceptance require later native confirmation; this component probe cannot establish them. Deliver candidate lifecycle design, controls, per-stage work ledger and reject/revise/confirm recommendation.