Historical repository record migrated on 2026-10-10. Original repository path: docs/experiments/benchmarks/coupled-operations-preparation.md Original SHA-256: 790fe9af4cf12ada7f9d8b58961e52ab412561c747f18413cf83b64a68969342 Stated outcomes, measured revisions, failures and limits remain historical. Local raw evidence is private and is not uploaded. Relative links and the data-drive prefix were normalized.

Coupled operations preparation: O6/O7

Current disposition: see completion preparation and executed run. The preparation status below is historical.

Status: source/evidence inspection only; diagnostics not run. Applies the operations packet and common admission. No oracle, product contract, gate, frozen fixture or historical result changed.

O6: availability truth table

Product completeness, query envelope and oracle unavailable model are distinct evidence. --unavailable excludes WHOLE records, including retained window/freshness/gaps; it cannot express one absent projection with surviving raw custody, Manifest and gaps table. Query reads window/freshness from Manifest and collection gaps from gaps.parquet, before reading the requested projection (source). Raw batches.parquet is read by scan_batches/dump, not ordinary projected query execution.

Availability cut (others intact)	Current source behavior, Scan/Walk	Surviving facts and unresolved contract cell
All intact	Reads query table and gaps; complete unless another source fails	Raw custody, Manifest bounds/freshness, gaps and rows independently checkable; full chain uses unchanged oracle
Requested logs/metrics/spans table deleted	Its open fails; extant Segment becomes unavailable; Manifest window/freshness and gaps survive	Observed logs/Scan case: incomplete, zero rows, retained metadata. Whole-record exclusion cannot grade this metadata cell; later table/plan cases were not reached
Unrequested projection missing	Other projection can still read; no general scan of all tables	Cross-projection completeness versus raw-Segment wording needs decision; do not assert complete is supported merely because implementation permits it
Optional text/span filter absent or corrupt	Exact fallback; absence alone does not make incomplete	Accepted contract: unchanged rows/order/pages/envelope. Authenticated semantic lies remain a separate known trust assumption
Raw Batch table absent, projections/Manifest/gaps intact	Ordinary projected query does not open raw table; dump/raw custody verification fails	Product says missing/corrupt raw Segments mark incomplete. Possible implementation gap; source prediction, not executed evidence. Metadata survives but raw custody becomes unavailable
Manifest absent, named Segment directory stable	Valid-manifest list excludes it, final labels include it; bounded Interrupted retries can exhaust	No valid Manifest facts available. Error/unknown must not become successful empty answer; current strict movement behavior is source-derived, not this O6 experiment
Gaps table absent	scan_gaps error marks incomplete; Manifest metadata survives	Collection gaps are unknown, not inferred absent and not invented from storage failure. Whole-record exclusion again conflates unrelated surviving facts

First-page and full-chain acquisition share this metadata logic; first-page agreement never establishes continuation exactness. Retention-expired continuation still requires Gone; Arc metadata confers no file lease. The table's unexecuted cells are hypotheses to test, not a supported availability specification.

Historical fixture removes ONLY logs first, retains custody/Manifest/gaps, then passes the full receive interval to oracle exclusion. Latest isolated failure: stderr (local evidence omitted from stage), exit101,7 passed/1 failed/1 ignored. Zero expected/answered rows; retained window expected[0,0) versus[2000001,2000008), freshness{} versus node-a1000806, and surviving sequence3 collection gap versus[]. The failure is an oracle-model/fixture mismatch; raw-table absence may independently expose implementation behavior needing repair. Neither conclusion clears the existing failed fast gate.

Original suggested reproduction below was NOT executed by this agent. Root's registered catalog-coupled-o6-reproduce-01 command under launch commit26a5e9c takes priority; its receipt owns the outcome. Suggested command after admission:

python3 tools/resource_group.py -- python3 -B tools/bench/labs/completion/run_job.py --id catalog-coupled-o6-proposed-reproduction-01 --lab recovery --stage recovery --seconds 60 -- cargo test --offline --locked -p fabric-server --test completion_storage --all-features missing_query_tables_are_incomplete_with_independently_declared_unavailability -- --exact --nocapture

Expected existing failure remains failure. Further minimal diagnostic: one Segment, one node, two Batches with distinct receive/observation times and one explicit gap; fresh state per availability cut/plan; retain independent producer ledger and raw-table/digest verification, Manifest facts, full pages and actual errors. Do not invent a second oracle while disputed expectations are unresolved. Falsely complete, invented freshness, dropped/duplicated rows and expired pages have existing accepted rejection controls when their source evidence is intact. Erased metadata on PARTIAL availability is disputed and cannot be given a new expected rejection without a separately scoped decision. Alternatives: correct fixture to model whole-source loss (new fixture, preserve old failure); extend availability model in its own trust-boundary change; or implement an explicitly chosen stricter behavior. No automatic preference changes the accepted oracle.

O7: observed layout difference and missing proof

Saved pair (local evidence omitted from stage): identical input c77c6f5e8ddbe889062337d480ad6cacf94dfbaa386e527695e05ca6678f6635, 4070 logs/6400 metrics/2035 Batches. Ordered raw/log/metric ledgers match; applicable file-byte, filter, heap and cleanup gates pass; pruning equivalence is false. Reference log groups [4070]; bounded [508,508,508,508,508,508,508,508,6]. Thus bounded ordinal cuts are0,508,...,4064,4070. These are physical ordinal partitions; saved hashes do not independently map every identity into its group.

Ten-second overlapping windows, offsets from1800000000000000000ns:

Start seconds	Matching logs both	Reference admitted rows	Bounded admitted rows
0	2000	4070	2032
5	2000	4070	2540
10	2000	4070	2540
15	1070	4070	1530
20	70	4070	514
Total overlapping visits	7140	20350	9156

Both60s windows admit4070logs; metrics layouts/windows match (6400-row group, 10s matched9600/admitted19200). Bounded writer flushes before an8MiB chunk cap (source (local checkout reference crates/fabric-server/src/segment/bounded.rs; not present at origin/main));16KiB bodies explain early508-row chunks. Generator has equal-time log pairs, but bigrows uses current batch-second observations; late arrivals are NOT covered by this cell.

Important limitation: completion_builder::pruning computes hypothetical admitted row counts from footer bounds, not actual query reads/IO. It explicitly records zero reads when logical matched count is zero, even if physical bounds would admit false positives. Saved reports omit actual min/max bounds and row IDs per group, and do not execute independent paginated query chains. Logical whole-table hash equality supports grouping explanation; it does not prove all window queries or a conservative-bound invariant. Existing pruning_equal compares complete window dictionaries INCLUDING read_rows, so the unchanged registered gate fails.

Executable existing control-only diagnostic (no native builder required):

python3 tools/resource_group.py -- python3 -B tools/bench/labs/completion/run_job.py --id catalog-coupled-o7-controls-01 --lab recovery --stage recovery --seconds 30 -- python3 -B tools/bench/labs/completion/builder.py --controls --out docs/experiments/benchmarks/data/lab-completion-run-01/recovery/catalog-coupled-o7-controls-01

It rejects mutated logical rows/counts, filters, pruning read counts, heap, leftovers and bytes; it is not the requested omitted-group/narrowed-bound checker. That next diagnostic needs a separately registered additive helper: actual Parquet row-group identity/bounds ledger, windows' admitted group IDs including empty windows, and unchanged independent full-chain oracle against pre-IO producer rows. Use two cap-forcing large-row chunks, equal timestamps across their boundary, a late row, and known match in each group; reject an omitted matching group and narrowed bound while allowing/recording extra reads separately. No runnable helper for that ledger exists yet; do not label the present controls sufficient.

Read-only decision and existing diagnostic controls can proceed without contract change after resource admission. C4/Q4 component preparation is independent. C5/O8 coupled service or C2-dependent acceptance stays blocked by O6/O7's relevant unresolved cells, existing failures and prospective evidence allocation. Replacing physical-read identity with conservative exactness is a trust-boundary decision, not an inferred waiver from fewer observed candidate reads.

The additive archived-report helper is now prepared at coupled_pruning.py (local checkout reference tools/bench/labs/catalog/coupled_pruning.py; not present at origin/main). It checks archived input/ordered-ledger/window identities, reports physical count differences and preserves the historical failed gate. Its four rejection controls cover changed ordered ledgers, matching counts, widths and duplicate windows; a physical-count difference remains observable. It explicitly reports soundness inconclusive without native group bounds/source mapping. This replaces no independent query oracle and is not the proposed future bound-ledger checker.