Experiment benchmarks coupled labs query - kmosoti/FabricO11y GitHub Wiki
Historical repository record migrated on 2026-10-10. Original repository path:
docs/experiments/benchmarks/coupled-labs-query.mdOriginal SHA-256:1b3188e689e5f4932d601b8c5a20dff3684725f036b27fc18f9b3158f1a01967Stated outcomes, measured revisions, failures and limits remain historical. Local raw evidence is private and is not uploaded. Relative links and the data-drive prefix were normalized.
Coupled labs: query and availability packet
Current disposition: see completion preparation and executed run. The preparation status below is historical.
Status: prepared; Q4 not run. Accountable: query PI, coordinator until
dispatch. Common plan governs admission. Own this packet
and future coupled-query-* findings; coordinate any shared source changes.
Q4 — reuse discovery work with bounded storage snapshots
H1: a storage-owned snapshot reuses validated source descriptors, reducing total maintenance-plus-query CPU without unbounded retention or changed answers. H0: invalidation, reacquisition, copying or lifetime costs erase the benefit, or a lifecycle schedule violates coverage. Refreshing earlier is not itself a win.
Source/evidence: maintenance findings,
discovery race,
lifetime controls,
read_catalog.rs (local checkout reference crates/fabric-server/src/read_catalog.rs; not present at origin/main),
query.rs and
catalog_maintenance_probe.rs (local checkout reference crates/fabric-server/examples/catalog_maintenance_probe.rs; not present at origin/main).
The baseline is current lazy acquisition with the corrected discovery retry.
First candidate retains only validated descriptors. Keep clone/shared ownership
fixed initially; test ownership as a later ablation if attribution warrants it.
No background worker, generic port, durable map or new persisted format initially.
Prepare an API/state sketch identifying generation, snapshot-relative coverage,
active/pending/published sources, physical file access and invalidation. State
exactly which listings/decodes disappear. Every held handle must have a release
path; metadata ownership grants no file lease or extended query retention.
Define reader/generation/byte limits, overflow behavior and exact fallback before
coding. A candidate is not admitted with merely an Arc and no retention bound.
Correctness matrix: append during acquisition; publish between listing phases; reclaim immediately after publication; missing optional metadata; held reader across retention; cancelled reader; equal timestamp bounds and continuation. Test cold process versus warm catalog separately; do not call a restarted process cold disk cache. Preserve the existing deterministic Scan/Walk omission fixtures. Controls inject omitted/duplicate sources, stale generation acceptance and retained metadata after final drop; grading must detect each relevant violation. Expected retry/error must remain explicit, never silently produce an empty complete answer.
Performance matrix proposal: 1/64 Segments × 0/1/4 queries per append, bounded 32-append schedule from CQ2, same broad/selective query bytes and full chains. Preflight 1 Segment/1 query plus all lifecycle controls before the larger cells. Freeze archived CQ2 workload identity or register precise deviations; do not mix the repaired seed publication fixture with its historical failing version. Count discovery calls, open/read/decode bytes, descriptors reused/invalidated, refresh CPU, query CPU, lock wait/hold and oldest/live/retired generation bytes. Query-off reveals shifted work and held metadata.
Nominate only if total maintenance-plus-query CPU meets the common 10% rule at the declared primary read rate and all memory/latency/exactness guards survive. Other rates are guard/crossover evidence, not substitutes for a failed primary. Service ACK and Spool acceptance require later native confirmation; this component probe cannot establish them. Deliver candidate lifecycle design, controls, per-stage work ledger and reject/revise/confirm recommendation.