Historical repository record migrated on 2026-10-10. Original repository path: docs/experiments/benchmarks/performance-frontier-labs/capacity.md Original SHA-256: 5d1112030ba0aabde68882f491ca6554e0b508687bcff944b1c37f9fa07e799b Stated outcomes, measured revisions, failures and limits remain historical. Local raw evidence is private and is not uploaded. Relative links and the data-drive prefix were normalized.

Capacity and lifecycle lab packet

PI: capacity_landscape (GPT-6.1-SOL medium). State: source review complete; experiments proposed, not run. Shared plan and controls. Owned preparation paths: this packet and future capacity-frontier-* protocols.

C5: spill amplification and simultaneous signal budgets

H1: merge-pass boundaries and simultaneously occupied per-signal buffers explain costs hidden by the log-dominated heap measurements. H0: mixed shape and merge boundaries add no material cost beyond input bytes.

First execute the already registered supplemental spill observer, including its known-byte/no-hit controls. Do not treat its instrumented time as a speed result. Then register a focused comparison at matched encoded bytes: logs only versus logs/metrics/spans, with isolated-signal controls. Count actual live buffer bytes, run counts by signal, merge passes, decoded merge-head bytes, logical spill writes, disk IO, CPU, wall time, output size and incremental heap. Keep the current 16 MiB runs/16-way merge as baseline. Check fixtures around an observed merge boundary before considering a buffer-size candidate; row limits may trigger before byte limits. Equal input bytes do not imply equal row-processing work.

Counterexamples: three sort buffers overlap; large decoded rows increase merge head memory; doubling a buffer does not eliminate any merge pass. These reject generalizing a log-only ~40 MiB observation into a universal service bound. Exact logical rows, custody bytes, deterministic output and cleanup are required.

Large-row follow-up is diagnostic: freeze narrow/broad/adversarial time windows, measure read amplification AND compressed size/build/query cost. Preserve the old failed equivalence result. Any future non-regression acceptance criterion requires a new separately committed policy/protocol decision before execution. Do not silently modify C2 or its oracle.

C6: journal occupancy versus total service cost

H1: reducing pending-journal occupancy saves repeated query work but eventually loses to publication/metadata overhead. H0: no useful trade-off appears within the tested range.

After C5/Q3, register a screen using 16/64 MiB journal targets, query-off versus fixed offered queries, current Scan plus diagnostic Walk, one sealer worker, matched total accepted bytes and identical workload phases. Include low-rate development and small burst/drain profiles. Query plans and thresholds are experimental configuration factors, not default changes. Keep delivery sync ordering unchanged. Match total history and report publication counts.

Measure CPU across intake + sealer + query, ACK/query latency, queued sealed bytes/oldest age, time to checkpoint/reclaim, Spool accepted Batches/s AND MiB/s, disk/write amplification, scratch and process/cgroup memory. First-query visibility is separate from Segment publication. A smaller journal producing more files and higher total CPU is a useful null/trade-off result.

Do not add worker-count tuning until one-worker attribution identifies a need. The PI's reserved alternative is a fixed-budget 1/2/4-worker sweep with seeded pending journals and an oldest-file straggler: reclaim follows the oldest published prefix, so faster later files may not release capacity. Keep seeded backlog distinct from a natural lifecycle. Earlier speed profiling already reduced intermediate merge cost substantially; if C5 shows little remaining spill cost, prioritize Parquet/Arrow construction and hashing attribution over another merge optimization. That is a branch decision, not an assumed result. Deliver a cost frontier and at most one candidate for independent confirmation. Luna may audit byte/Batch accounting in one bounded packet (estimated 1500 tokens); PI owns interpretation. Maximum lab workload 90 minutes within the shared four-hour allowance. Reuse completion builder/spill observers and native profile fixtures; no new benchmark framework.