Experiment benchmarks catalog spill findings - kmosoti/FabricO11y GitHub Wiki

Historical report imported 2026-10-10 from docs/experiments/benchmarks/catalog-spill-findings.md. Original SHA-256: d6c66bb87fecb534ae5c9e26ea6e450e3a8ae4b1b08d3c8ccdfc15e360d6a059. Measurements and outcomes retain their stated revisions. Links and local environment labels are normalized; $FABRIC_DATA is the mounted experiment drive. See evidence policy.

CR1 spill workspace screen, run 01

Status: executed microbenchmark; PI analysis and root paired arithmetic reviewed. Frozen protocol (local checkout reference docs/experiments/benchmarks/catalog-spill-protocol.md; not present at origin/main), registered at c4b27fe597894a7c9c27583136ceea2dbeac49a2; raw trials (local evidence omitted from stage) and coordinator receipt (local evidence omitted from stage). Root's separate summary command exited 0 and retained the paired arithmetic in its output (local evidence omitted from stage). This cross-check does not replace full-builder or independent service validation.

The contained catalog-spill-01 launch command recorded exit0,29.564s elapsed, 963.93MiB whole-job cgroup peak, no memory high/max/OOM events and zero swap. Its whole-job peak includes builds and controls, not just codec workspace. Successful scratch cleanup is recorded. Compressed frozen binaries remain in the result directory; protocol/source/binary hashes accompany the trials.

All36 trial receipts preserved the67,109,448-byte input/output SHA-256 exactly, with60,296 rows and seed2703163393. Both plain/counted controls reported rejection of malformed/truncated frames, preservation of scalar bits and bounded maximum-row-then-small retention. Runner exactness rejection controls succeeded.

Paired observations

Ratios are candidate/baseline; triples list repetitions1,2,3. CPU sums external whole-child user/system time, including startup and exact file readback. Phase wall covers codec roundtrip/output flush and excludes readback. Counted allocation evidence is separate from plain timing.

Candidate Plain phase ratios Plain CPU ratios Counted calls ratios Counted requested-byte ratios
Encode reuse 0.97142 /0.98260 /0.98037 0.99495 /0.98247 /1.01558 0.85715 /0.85715 /0.85715 0.70870 /0.70870 /0.70870
Decode reuse 0.96593 /0.92277 /1.02742 1.00356 /1.02046 /1.00309 0.85715 /0.85715 /0.85715 0.70870 /0.70870 /0.70870
Combined 0.97055 /0.92350 /0.92401 0.99438 /0.99388 /0.98994 0.71429 /0.71429 /0.71429 0.41740 /0.41740 /0.41740

All candidates satisfy the registered nomination screen through cumulative requested-byte reduction in every pair:29.13% for either single workspace and 58.26% combined. Single-workspace allocation calls decrease14.29%, below the alternative15% call threshold; the byte threshold is independently sufficient. Combined calls decrease28.57%.

Ratios of candidate medians to their baseline medians are: encode phase0.97338 / CPU0.98247; decode phase0.95886 / CPU1.00309; combined phase0.92636 / CPU0.99438. Each is within the5% median regression guard. Median paired ratios give the same admission outcome. Decode's third phase pair regressed2.74%; encode's third CPU pair regressed1.56%. These contrary observations remain visible.

Baseline counted requested bytes/calls are229,546,872 /422,072 in every pair; either single reuse uses162,679,717 /361,777; combined uses95,812,562 /301,482. Incremental peak requested heap increases from2,698 to3,807bytes for every candidate. Reuse reduces allocation traffic, not this peak. A single workspace retains1,109bytes and combined retains2,218bytes until drop; counted live heap returns to its pre-phase level after workspace drop. That does not establish allocator page reclamation or RSS reduction.

Next decision and limits

Nominate combined reuse for independently checked full-builder confirmation: it has the strongest allocation reduction and all three plain phase pairs favor it. Keep encode-only as the smaller fallback if sixteen-reader retention or mixed/large-row costs defeat the combination. No production patch is implied. The probe has one stream, fixed small log rows and warm file cache; maximum rows are controls, not the measured distribution. Small34ms phases and three pairs cannot establish a universal throughput gain. Whole-child CPU is dominated by work outside that phase, and its improvement is small. No full sealer, spool, ACK, query, sustained-load, deployment or process-memory claim follows.