Historical report imported 2026-10-10 from docs/research/cross-system-storage-dissection.md. Original SHA-256: bbb33118087f176a4128b2fcbc892f5bc74af6077890b2260700ceb8078969be. Measurements and outcomes retain their stated revisions. Links and local environment labels are normalized; $FABRIC_DATA is the mounted experiment drive. See evidence policy.

Source dissection, 2026-10-08; research tier, not an adopted design. This extends the cross-system program using downloaded implementation files under the continuation protocol (local checkout reference docs/experiments/benchmarks/cross-system-continuation-protocol.md; not present at origin/main). The strongest additions are a distinction between retained and construction memory, a drain condition after cancellation, and a choice of maintenance work that addresses both resident pressure and the oldest unreclaimable journal. No upstream code, workload, build or validator was executed for this dissection.

The receipts below record immutable revisions, archive/member digests, requested and missing paths, selected-source readback, and cleanup. The inspected members are inside each receipt's sibling selected-source.tar.gz; complete regular-file inventories are sibling regular-members.jsonl.gz files. Inventory coverage is not implementation review coverage. The following table states the narrower inspection performed here; individual function bodies and nearby control flow were read, rather than every line of every selected file.

Project and exact revision	Inspected paths	Evidence
Comet 412468e2817207eff0c1907d1c4aac8c3f06680d	native/core/src/alloc_accounting.rs; native/core/src/execution/memory_pools/{fair_pool,plan_pool,task_shared}.rs	receipt (local evidence omitted from stage)
Feldera proposal ff12ddadd8dbfbcb8dedc276cc83382d4c2f3095	README.md, storage layers, buffer pool, checkpoint and reclamation sections; proposal only	receipt (local evidence omitted from stage)
Feldera implementation 31b177027205a35e51b08c6209cd9ea90f5c881c	crates/dbsp/src/trace/spine_async.rs; crates/dbsp/src/trace/spine_async/list_merger.rs	receipt (local evidence omitted from stage)
Vortex 881b867d736e8955a0381655f80d6d2d630fc590	encodings/fsst/src/{kernel.rs,compute/filter.rs,compute/like.rs,dfa/mod.rs}; vortex-array/src/arrays/filter/execute/take.rs	receipt (local evidence omitted from stage)
Lance b8886f554b773ceaa78aef9666af616217bd0199	rust/lance/src/io/exec/filtered_read.rs, stable-ID routing and intersection; rust/lance/src/index/scalar.rs, training projection/stream construction	receipt (local evidence omitted from stage)
redb f403a514fd557f633bd4725248f0d19b434eedff	src/transaction_tracker.rs, reader registration/durable-ancestor ownership; src/tree_store/page_store/page_manager.rs, deferred-free ranges and page allocation	receipt (local evidence omitted from stage)
Fjall 17ae6720cbf1a44a1f1f0dcbcb1f4fd913b23c89	src/keyspace/mod.rs, insert/maintenance/backpressure; src/journal/manager.rs; src/journal/writer.rs, persistence/rotation	receipt (local evidence omitted from stage)

All requested paths in these retrieval receipts were found. This does not cover the transitive implementations called by those files. In particular Lance's RowIdSequence::mask_to_offset_ranges, Vortex's VarBin filter implementation, Feldera's buffer-cache allocator, and Fjall's lsm_tree dependency are outside this inspection. The proposal archive contained no selected license file; its receipt records that absence, without a licensing conclusion.

Observed source. Lance's fragment_metadata_load loads the stable-ID sequence and constructs upper_ranges with mask_to_offset_ranges before calling retain_stable_index_ranges. The latter charges vector capacity times size_of::<Range<u64>>(), using a compare/exchange loop against the 16 MiB request-scoped retention allowance. On refusal it returns None and the constructed vector is dropped. intersect_index_ranges uses a retained vector when available; otherwise it calls mask_to_offset_ranges again. It consumes precomputed ranges fragment by fragment rather than retaining all of them beside the completed plan.

Deduction and counterexample. That allowance bounds this retained range storage, not the first vector's construction peak, the loaded ID sequence, or the final plan. A highly fragmented candidate mask can spend allocation and CPU on a vector that is discarded and later rebuilt. Multiple concurrent routing operations can amplify transient construction even if retained storage obeys its allowance. This is a structural counterexample to calling a cache budget a whole-query memory bound; its size and timing have not been measured here.

Fabric mapping and candidate. The bounded builder (local checkout reference crates/fabric-server/src/segment/bounded.rs; not present at origin/main) already counts string capacities and map overhead, while the sealer view explicitly permits an oversized row alone. Apply the same distinction to future query candidate sets and hot-tail structures: retained bytes, transient construction bytes and recomputation are three costs. First compare a streaming range intersection with retained/rebuilt ranges on identical clustered and alternating-ID masks. H1: streaming lowers construction peak on fragmented masks while preserving exact ordered results. H0: repeated traversal, locality loss or output storage erases the benefit. Dense/contiguous matches are the deliberate adverse case; candidate-set build CPU and final output ownership must remain in the measurement boundary.

Observed source. Comet's PlanMemoryPool counts held bytes before invoking the inner grow/try_grow, rolls back refused growth, and subtracts only after inner shrink. wait_until_released waits on a condition variable until held bytes reach zero or a deadline, and returns any remaining bytes. The file explains why plan drop can leave reservations in merge tasks until those tasks yield. TaskSharedMemoryPool uses an Arc lifetime and weak registry entries; its drop logic distinguishes the old pool from a replacement. These are implementation observations; the accompanying upstream tests were read, not run.

The fair pool accounts sibling reservations under one consumer identity. Its try_grow checks both the consumer share and the whole pool because late registration does not revoke allocations already held by earlier consumers. Its infallible grow records already-existing allocation even beyond those limits. The allocator observer separately batches signed thread-local deltas at 64 KiB and clamps its shared reported balance at zero; it is an approximate observer, with drift dependent on live thread count, rather than admission.

Deduction and counterexample. A canceled request, a dropped high-level owner, a zero local request counter and returned worker memory are distinct events. Splitting reservations must not manufacture new shares. A new fairness denominator also cannot reclaim earlier holdings. Importing Comet's approximate observer without its drift qualification would make small short-lived allocation changes look more exact than they are.

Fabric mapping and candidate. Fabric already moves the query permit into the actual blocking closure in spawn_query_work; this is not a proposal to repeat that repair. Measure canceled-request-to-worker-completion, permit release and live requested bytes at the same cut. H1: at least one declared cancellation shape has a measurable memory drain interval after client completion that affects available admission. H0: work finishes promptly or the remaining memory belongs to shared state that cancellation cannot release. Shared-cache residency, uncanceled work and allocator retention are competing explanations. Do not use zero permits or reservations alone as a cleanup oracle, and do not change Fabric's precise probe counters merely to imitate Comet.

Observed source. Feldera's current Spine::try_start_merge checks the batch-count threshold before the memory-pressure branch. The stated reason is to prevent repeatedly merging only in-memory batches while total batch count keeps growing. Under high pressure and below that threshold it selects only memory-resident batches; stored batches need not be rewritten just to move resident state to disk. This is more specific than the older proposal's buffer-pool eviction discussion, whose policy and failure handling were TBD.

The inspected scheduler gives level-zero merges isize::MAX fuel; other levels receive fuel based on average level-zero merge work. ListMerger::copy_times can traverse time/weight entries before charging one emitted value, and a zero-weight result returns before that charge. Thus even the finite-fuel paths do not establish byte or wall-time bounds. This confirms the previous candidate screen and adds the level-zero and pressure-versus-batch-count scheduling qualifications.

Deduction and counterexample. Minimizing resident bytes alone can leave more immutable objects, discovery overhead and future merging behind. Conversely, forcing every pressure event to merge stored inputs can increase I/O when the resident subset was sufficient. Neither unconditional policy dominates.

Fabric mapping and candidate. D3's hot-tail-versus-sealing budget needs a second guard for accumulated maintenance work: journal age/bytes and source fanout, alongside live derived bytes. H1: memory relief with a backlog guard avoids growing future read/maintenance work better than resident-only relief. H0: the existing scheduler already controls the relevant backlog, or extra work increases ACK/query cost without useful drain. Use a stalled oldest journal, many small files and sparse repeated queries as separate discriminating shapes. Weighted consolidation and negative-weight cancellation are Feldera semantics and must not be transferred into exact Fabric Batch custody.

Observed source. Fjall's journal manager identifies the keyspaces whose persisted sequence numbers do not yet cover the oldest journal's recorded watermarks. Maintenance examines that oldest journal first and returns without deleting it if any required watermark is uncovered. It uses persisted table sequence numbers rather than newer memtable sequence numbers; its safety argument explicitly depends on FIFO flush queues. This identifies work that can make reclamation progress instead of treating all queued work as equivalent.

Fjall's insert writes the journal, inserts the value, publishes its sequence, drops the journal lock, then charges write-buffer size and calls maintenance. Local backpressure can wait for level-zero runs or sealed memtables after that insertion. This is a concrete counterexample to treating those thresholds as refusal before allocating. PersistMode::Buffer flushes the userspace buffer but does not provide the power-loss guarantee of SyncData/SyncAll; rotation explicitly invokes SyncAll. These settings cannot replace Fabric's durable ACK.

redb adds a separate lifetime qualification. register_read_transaction captures the committed snapshot and registers its reader under the tracker/header locks, so reclamation cannot miss the reader. Pending non-durable commits retain references to their nearest durable ancestor. In the page manager, data_freed_in_range explicitly handles reversed/equal bounds because the oldest unprocessed non-durable commit and oldest live reader are independently derived. Reader protection and durability protection are distinct owners; subtracting one from the other is not automatically a valid reclaim interval.

Fabric mapping and synthesized candidate. Combine oldest-obstruction work selection from Fjall with Feldera's resident-only relief under low fanout and Comet's completion-owned accounting. Proposed rule: prefer work that releases the oldest durable journal obstruction when backlog pressure dominates; under resident pressure and controlled backlog, relieve optional derived memory; retain each admission owner until its actual work drains. This is a new combination in this research program, not an originality claim or implemented controller. Fabric's existing ordered-prefix reclamation remains the safety boundary; later completion must never reclaim through a hole. redb's reader-pinned retention must not be imported: Fabric's expired page tokens return Gone and do not promise a file lease.

H1: compared with fixed oldest-first sealing and a resident-only policy, this choice reduces both journal byte-time and derived-memory byte-time on the declared skewed workload without worsening registered ACK/query guards. H0: shared commit/disk contention, inaccurate relief estimates or oldest-label cost makes either simpler policy equal or better. The inverse workload, where the oldest file is the largest and expensive to finish, is essential: completing small later files may release more memory while yielding no physical journal reclaim. Measure those objectives separately. Require isolated policy ablations before the combination, unchanged exact replay/query checks, and explicit progress reserve before any adaptive prototype is nominated.

Vortex's FSST kernel registration routes LIKE, filtering, byte length and other supported operations to encoding-specific implementations. The inspected LIKE path requires a constant pattern and declines case-insensitive evaluation. FsstMatcher::try_new rejects unsupported wildcard shapes and enforces byte limits: 253 for literal prefix and 254 for literal contains. Its parser borrows unescaped literals and materializes escaped literals. Filtering invokes the VarBin filter on compressed codes and filters uncompressed lengths while reusing the symbol table. Generic Filter/take code has contiguous-slice and sequential-rank paths, and otherwise builds translated indices.

These observations narrow D5 rather than proving a whole-path advantage. Fabric literal needles containing %, _ or backslash require correct LIKE escaping; Unicode limits must be tested in bytes, with empty, 254-byte and 255-byte needles and compressed escape-code cases. H1: on sufficiently reused compressible blocks, predicate execution before payload materialization saves more CPU than training, conversion, selection and output consume. H0: broad results, high entropy, unsupported needles or coexisting buffers erase that saving. Exact custody and required validation remain, and the uninspected VarBin implementation prevents a zero-copy or retained-backing bound claim from this review alone.

The next discriminating work is construction/rebuild attribution and memory drain timing, followed by pressure-versus-oldest-obstruction scheduling only if those costs matter. No source observation here changes a product contract, qualification status, default allocator, storage format or current scheduler.