Historical repository record migrated on 2026-10-10. Original repository path: docs/experiments/benchmarks/catalog-borrowed-log-design.md Original SHA-256: 97e8098859e0f56fe34a6fff1fd8945d6deeef5dd485f7a41c82618ff6de99f4 Stated outcomes, measured revisions, failures and limits remain historical. Local raw evidence is private and is not uploaded. Relative links and the data-drive prefix were normalized.

CR2 admission and minimal borrowed-log design

Status: historical admission/design rationale; registered implementation and sliced execution completed without nomination. The registered CR2 protocol (local checkout reference docs/experiments/benchmarks/catalog-borrowed-log-protocol.md; not present at origin/main), slicing amendment (local checkout reference docs/experiments/benchmarks/catalog-borrowed-log-slicing-protocol.md; not present at origin/main) and bound resource admission (local checkout reference docs/experiments/benchmarks/catalog-evidence-capacity-admission-protocol.md; not present at origin/main) define actual execution and acceptance. The completed findings record all six pairs/768 chains: counted primary/peak guards met, but plain pair3 CPU/wall preservation failed. No production promotion follows. This rationale is independent of CQ1 timing and the CR1 result. Sources: corrected Q2a record, counted fixture (local evidence omitted from stage), phase checks (local evidence omitted from stage), raw measurements (local evidence omitted from stage).

Why preparation is justified

Corrected Q2a has65536 shuffled1024-byte logs, seed42, complete independently graded pagination and exactly one matching source loader per64 measured calls. On its warm Segment selective queries, Scan requests273,762,541 allocator bytes and44.43ms median process CPU; Walk requests274,538,018bytes and44.66ms. Warm Segment broad Walk requests112,096,627bytes and45.09ms median CPU. These observations establish substantial Segment-query allocation cost even where output is selective. They do not isolate string copying from Arrow decode, attribute parsing, selection or output. Nested loader spans are inclusive. Source inspection supplies the narrower mechanism: read_logs owns node/body strings and parses attributes before query-side node/text/page/top-k rejection. That conjunction admits a small discriminating experiment, not an optimization claim. Existing tail-block decoding already borrows and filters before owning; tail-only must remain a control, not be presented as a new borrowed decoder.

Smallest candidate and fixed semantics

Change only private Parquet log projection behind an explicit experimental mode. Validate all eight column types on every loaded batch exactly as read_logs does. Apply the existing half-open time check first, preserving its unsigned cast. For every in-window row, validate fixed-size node identity and parse attributes through the SAME attrs_of before consumer predicates, including rejected rows. Do not defer, skip, substitute or weaken JSON/type validation.

A scoped visitor can receive borrowed node/body strings and copied scalar/key fields plus an OWNED attributes map. The caller checks current node/text/snapshot/ page/top-k predicates; only retained candidates allocate node/body strings, moving the already validated attributes map into the owned LogRow. No map clone is needed. Callback lifetime stays within one RecordBatch; no borrowed value may enter top-k, page state, async work or output. Keep original public scanners and their error/unavailable ordering. This deliberately retains attribute allocation; a borrowed attributes parser is a separate, currently unjustified mechanism.

Superseded initial proposal (not executed as specified)

The seed2703163393, attribute-rich measured fixture and expanded corruption/ tie/expiry combinations below were an initial proposal, superseded by the registered scope. Actual measurements use seed42,65536 shuffled1024-byte rows, limit10000, empty benchmark attributes and unique timestamps, both tail/Segment and Scan/Walk, first+3warm calls and64 complete chains per child. Rich attributes, malformed identity/attributes and wrong column types are separate executable unit controls; they do not establish the unexecuted proposed performance matrix. Tie/H256/attribute-rich consumer confirmation and service F1 remain deferred.

H1: delaying node/body ownership reduces cumulative allocation bytes >=10% each pair with <=5% CPU/peak/latency regressions. H0: validation/decode/output dominate. Use seed2703163393, one sealed64MiB fixture, fixed warm input and3 alternating plain/counted pairs per absent/rare/common text and broad small/full-page shape. Include long bodies and a declared attribute-rich control; hold decoded row counts, plan, request starts and complete pagination fixed. Measure allocation counts/bytes, requested peak, Arrow-buffer lifetime, owned answers, CPU, latency and IO. Reuse existing independent query oracle; no answer semantics change. Malformed attributes on a text/node/top-k-rejected IN-WINDOW row must still make that source unavailable. Out-of-window malformed attributes retain baseline skip behavior. Include wrong column types, malformed identity, duplicate/escaped JSON keys, Unicode, stable ties, expired pages and interrupted-file behavior. No unsafe or persistent-format change. Register exact fixture/corruption hashes, negative controls, commands, bounds and baseline freeze before execution. This proposal required minimal implementation and unchanged error/custody checks before measurement; the registered protocol defines the actual gates. Service Spool/ACK/backlog guards require F1 later; these query observations supply no service or process-memory guarantee.

Pre-run independent source review

The operations PI compared both opt-in query call sites and the owned/borrowed readers without running workloads. It found no concrete semantic regression: column types, time-window skip, 16-byte identity and attribute validation retain their order; attributes are parsed before consumer rejection. Snapshot, full ordering key, page boundary and predicates match. The strict pre-materialization heap threshold matches the existing heap's rejection of equal-or-larger keys. The higher-ranked callback prevents borrowed node/body strings from escaping the Arrow batch; accepted rows copy strings and move validated owned attributes.

This is source review, not executable proof. Every in-window row still pays for attribute parsing and Arrow decoding. Null handling, authenticated false footer statistics and absence of per-query file rehashing are inherited limitations. The finite performance fixture does not establish rich-attribute/tie/H256 behavior; those remain explicit confirmation dimensions if a candidate survives.