Experiment benchmarks coupled capacity completion readiness - kmosoti/FabricO11y GitHub Wiki
Historical repository record migrated on 2026-10-10. Original repository path:
docs/experiments/benchmarks/coupled-capacity-completion-readiness.mdOriginal SHA-256:c4d5b4223e7db6cdff6a65111de7a305f4ef1d0f8432b90ac3334bce7b08d421Stated outcomes, measured revisions, failures and limits remain historical. Local raw evidence is private and is not uploaded. Relative links and the data-drive prefix were normalized.
Status: read-only preparation; no new registration, build, validator or workload. Elevated permissions do not resolve the recorded O6/O7 correctness dependencies or allocate more evidence/time. Root must admit any new execution prospectively.
Close C4 as completed diagnostic, inconclusive phase causality. Run 03 has 48 oracle-passed canonical chains and 144 first-page associations, but its observer CPU ratios 0.903857–1.043303 fail the registered admission band. Inclusive attribution reconciles; it cannot separate attribute parsing from Parquet decode, predicate callbacks and materialization. The repeated attribute values are a compressible fixture, not evidence for arbitrary rich workloads.
The actual findings show that eight attributes add 15.89–15.96 million requested bytes equally to both implementations. Borrowing still saves 4.17–4.19 million bytes, shrinking its fraction from about 21.4% to 11.8%, with negligible peak change. This explains allocation dilution without establishing CPU causality. More C4 timing is unnecessary to close this declared diagnostic. A future isolated parser experiment requires its own hypothesis and observer admission; it must not silently replace C4 or rehabilitate CR2.
bounded.rs (local checkout reference crates/fabric-server/src/segment/bounded.rs; not present at origin/main) currently fixes
RUN_BYTES=16MiB, RUN_ROWS=32768, FAN_IN=16, and CHUNK_BYTES=8MiB.
build_sealed passes the constant to private build_with_limit; Runs::push
checks resident-byte accounting before inserting a row into a nonempty run.
Three signal buffers have separate limits. A single oversized row is admitted;
the value is not a strict whole-builder heap ceiling. Frame decode, row
extraction, Arrow encoding, map overhead and buffers also consume memory.
Propose one strict compile-time experiment selector accepting only 8/16/32MiB,
defaulting to the existing 16MiB. Record and acknowledge the selected value;
reject invalid values during build or preflight. Freeze three plain server
binaries with exact source/features/toolchain and SHA receipts. This avoids a
public config/wire change. It is not implemented; no run-cap CLI exists.
option_env! must be an actual Rust dependency so Cargo rebuilds each variant.
Keep spill reuse and borrowed/query catalog experiments disabled.
Only segment/bounded.rs needs production-path parameter selection and tests.
Keep private build_with_limit as the test seam; leave CHUNK_BYTES, compression,
row-group flush, stable ties and manifest/publication semantics fixed. This is
crucial: changing sort-run size should preserve identical final output groups,
unlike the historical reference-versus-bounded writer comparison.
config.rs already accepts
seal_workers=1..16; sealer.rs
uses worker groups and reclaims only the published oldest prefix. No worker
mechanism change is required.
Required controls extend large_rows_spill_by_bytes_before_the_row_count_limit
with all selector values, oversized-row behavior and real selected-path spill
counts. Preserve external_merge_preserves_exact_rows_and_stable_ties, actual
16/17/18 fan-in merge controls, corrupted-frame cleanup/custody, all-table/raw
equivalence and sealer hole/checkpoint/retry tests. No tests have run here.
native.py (local checkout reference tools/bench/labs/dev_small/native.py; not present at origin/main) has a Python trial
API with explicit nodes/rates/schedule, optional observer and Scan/Walk. It
hardcodes one worker and 64MiB journal files. H256 seeds history before server
startup; near-rotation is a separate H0 option. Neither guarantees overlapping
builds during measured demand. signals.py (local checkout reference tools/bench/labs/completion/signals.py; not present at origin/main)
also hardcodes one worker, includes a restart, and has no run-cap/worker flags.
Its existing CLI cannot execute C5 unchanged.
Prepare a unique C5 driver/adapter, leaving frozen drivers untouched. Parameterize workers in generated config, explicitly seeded pending journals and release timing, and fixed query/no-query schedules. Start with two or four real sealed 64MiB journals; release the server at measurement start, avoiding startup settle silently consuming the backlog. Record exact pending bytes and actual concurrent builders; a two-worker cell without overlap is inconclusive. Use fixed small 20-node offers 1000/3000/1000 aggregate logs/s, seed 2703163393, 15s metrics, 900-byte deterministic bodies and 60/60/60s phases; query one fixed Walk request every second, with exact request/response receipts. Do not claim that offered rates are sustainable or that serial clients supply fixed completed demand.
First run one worker/16MiB without queries, then the matched query cell: each <=450s including startup/drain/verification. Admit the 1/2-worker × 8/16MiB grid only after measured runtime, backlog and evidence projections fit. Budget up to 1800s for four query cells plus 450s for the no-query control, 600s preparation, and 450s held-out 32MiB at the selected worker count. These are proposed ceilings, not an authorized extension of the existing campaign budget. One discovery pass gives a feasible set; any improvement claim needs three fresh alternating pairs.
Keep outer 20GiB/no-swap containment, data-drive scratch <=8GiB and >=16GiB free; start with established small server/node limits 3072/256MiB and quotas 2/1 CPU. Failure to fit is a result. New evidence allowance must include gzip pages, fixture/failure reserves, binaries and receipts, with periodic live checks. Collect accepted bytes, server CPU, RSS/cgroup peak, spool/journal slope and drain, ACK/query latency, retry/refusal and actual process IO. Separate builder allocator probes from plain service timings; do not label RSS as builder heap.
O7 still needs actual per-group identities/min/max and conservative pruning checks, not its archived hypothetical read totals. Build two cap-forcing chunks of large bodies, equal timestamps across the boundary, an out-of-order late row, and a known matching identity in each group. Include empty and boundary windows; record admitted groups even when no logical row matches. Grade complete query chains against the pre-IO producer ledger. Inject omitted matching groups and narrowed bounds: both must fail; extra reads stay visible and permissible only under a separately authorized prospective criterion.
Require exact raw custody, logical rows, filters, pagination and unchanged bounded-versus-bounded output bytes at all caps. Preserve O6's availability disagreement and historical C2 pruning-equivalence failure. C5 service admission remains conditional on their relevant resolution; this preparation supplies no waiver. Primary result is feasibility/Pareto evidence, not universal optimality.