Historical report imported 2026-10-10 from docs/experiments/benchmarks/catalog-integration-findings.md. Original SHA-256: 7aefe470433445b338fef730ce7f326ce8b4c9496cc8ef7a2d437a54fa15bfce. Measurements and outcomes retain their stated revisions. Links and local environment labels are normalized; $FABRIC_DATA is the mounted experiment drive. See evidence policy.

Catalog experiment integration checks

catalog-fast-02 ran cargo xtask checks --profile fast through the resource launcher/coordinator and exited 1 after 110.738 seconds. Formatting and workspace compilation passed. Tests stopped in the server library and Clippy failed; other reported fast gates passed. Whole-job peak was about 9 GiB (systemd rounded), zero swap. This does not clear the older partial-table oracle-model failure because the workspace test command stopped before that integration test.

Parallel fixture counterexample

Source inspection found independent module-local atomic counters generating the same storage-PID-counter paths in the same server-library test process. The Scan and Walk fixtures both started at counter zero. Root reproduced the error:

catalog-parallel-fixture-01, command cargo test --offline --locked -p fabric-server --lib --all-features publication_and_reclaim_between -- --test-threads=2 --nocapture, exited 101 in 14.522 seconds. Scan panicked while creating its directory with AlreadyExists; the failure occurs before storage semantics are exercised. The complete stderr, source archive and original failed receipt are preserved in the job (local evidence omitted from stage).

The correction gives the two modules distinct catalog-walk-cut and catalog-scan-cut namespaces, preserving their process ID and atomic counter. The identical parallel command passed both unchanged semantic tests in catalog-parallel-fixture-02, exit 0, 4.521 seconds, 647.4 MiB rounded whole-job peak. This is test isolation, not a change to product coverage or relaxed checks. Module comments retain the originating counterexample job identifier.

Lints

catalog-clippy-02 independently captured the full six diagnostics (exit 101). Five were in the older completion builder: three equivalent divisibility idioms, an indexed queue iteration and a complex result type. They were replaced with is_multiple_of, enumerated mutable queues and a named timestamp-bound tuple. The many-Segment probe's unused thread-clock helper was removed. No warning was suppressed and no measured algorithm or acceptance criterion changed.

The same workspace/all-targets/all-features command with warnings denied passed as catalog-clippy-03, exit 0, 1.004 seconds. Frozen benchmark executables and historical results still belong to their original captured sources. At that checkpoint, final full fast verification remained required; its subsequent failed outcome is below.

Campaign checkpoint

CQ1 completed all 18 trials and 1,152 full-chain oracle acceptances; all three registered plain preservation guards met. The comparison includes the discovery corrections, so it does not isolate module movement. CR1 completed 48 exact-output builds and 14 controls but missed the primary allocation-byte reduction guard: a null nomination, with reuse still opt-in. Operations findings complete the selected coverage, paused-cut, discovery and lifetime controls. Their original counterexamples remain recorded; no exhaustive race proof follows. CQ2, CR2 and CR3 performance conclusions remain pending. The independent partial-table availability-model mismatch and historical C2 failure remain open. The subsequent full fast verification failed as detailed below; targeted parallel and Clippy fixes do not clear its remaining storage failure.

Retained failure cleanup preparation

Read-only audit found retained outer roots for catalog-fast-02 (four files, 162 bytes), catalog-parallel-fixture-01 (empty), catalog-clippy-02 (empty) and catalog-maintenance-0-preflight-01 (19 files, 35,701,557 bytes). None contained symlinks. These counts are observations before cleanup, not cleanup receipts. Coordinator transcripts/source snapshots are separately retained. Maintenance preflight exited 1 because native 0-plain-128-lazy-skip0 exited 101; causal investigation is still active, so its root must remain available to the PI.

The existing cleanup.py --only-job filter selects exact jobs, checks completed service state and outer-root ownership, archives every regular file, verifies archive member coverage/size/SHA-256 against originals, then removes only the owned retained root. It preserves shared build/SDK caches unless explicitly asked otherwise. Empty roots still produce verified empty archives. No cleanup was executed for these four jobs during this audit.

The current driver still enforces its older 256 MiB per-lab final accounting; current legacy query/memory/recovery/coordinator directories are approximately 148.1/24.1/137.0/91.8 MB, so these small additions fit that check as inspected. This check does not account for all newer catalog evidence directories or hard links; root must also use the registered aggregate allocation before admission. Never interpret a cleanup command failure as proof nothing was deleted: per-job archive verification/deletion precedes final accounting/report generation.

Final fast verification and isolated storage failure

catalog-fast-03 ran the contained completion checks wrapper with --profile fast, exit 1, 161.896023372 seconds. Its receipt (local evidence omitted from stage) and transcript (local evidence omitted from stage) record 16 gates passed, including formatting, compilation, workspace Clippy, layers, core purity and independent-oracle self-tests. Workspace tests failed in completion_storage; the full fast profile therefore failed. Whole-job peak 8,616,648,704 bytes, zero swap; failed scratch retained.

Root isolated the file with cargo test --offline --locked -p fabric-server --test completion_storage --all-features -- --nocapture. catalog-storage-regression-01 exited 101, 15.541113321 seconds; 7 passed, 1 failed, 1 ignored helper, target time 5.70s. Its receipt (local evidence omitted from stage) and diagnostic (local evidence omitted from stage) retain the exact failure and captured source. Peak 1,147,195,392 bytes, zero swap; scratch retained. This reproduces the historical missing_query_tables_are_incomplete_with_independently_declared_unavailability metadata mismatch, rather than a new row loss: expected and answered rows both 0.

The fixture deletes only logs.parquet first while retaining Manifest and raw Batch custody. It declares the complete receive interval unavailable to the independent oracle, whose unavailable-record model excludes those records from all expected metadata. The runtime still reports retained Manifest metadata: window expected [0,0) versus [2000001,2000008), freshness expected {} versus node-a:1000806, gaps expected [] versus node-a/sequence3/receive2000003/ gap at 1000300. It correctly asserted complete:false before grading. The first logs/Scan grading failure stops the table/plan loop; this run does not establish outcomes for the later metrics/spans cases. Resolving query-table availability versus whole-source metadata availability requires an explicit trust-boundary decision; no oracle, product contract, expected outcome or policy was changed. The failing gate remains a completion limitation.

Add catalog-fast-03 and catalog-storage-regression-01 to root's later scoped byte-verified cleanup list alongside the earlier integration/preflight failures. Their retained outer roots are respectively $FABRIC_DATA/evidence/fabric-work-50d08da3421d4a93b43dd07793e6b795 and $FABRIC_DATA/evidence/fabric-work-0123cc194ccb4299a11291261d66337a. No deletion performed here; preserve stdout/stderr, oracle inputs/verdicts, source/protocol snapshots and original failed receipts. CQ2 remains running; these verification receipts establish no campaign-wide pass or qualification.