Historical report imported 2026-10-10 from docs/experiments/benchmarks/cross-system-query-sweep-findings.md. Original SHA-256: 8d22f66729f0cee2bb1b72f481594ac467d8a61956dbad922351f135b4bff774. Measurements and outcomes retain their stated revisions. Links and local environment labels are normalized; $FABRIC_DATA is the mounted experiment drive. See evidence policy.

Matched query and compressed-representation findings

Completed mechanism screens; no native Fabric speedup or format nomination. The validated report (local evidence omitted from stage) rechecked all 1,536 query-sweep and 576 Vortex answers against retained logical source rows, across 24 fresh children. Both successful sweeps and the reporter exited 0. Selective conversion avoidance is supported; universal indexing, universal ingestion benefit and universal Vortex benefit are not.

Scope remains the registered query proposal (local checkout reference docs/experiments/benchmarks/cross-system-query-sweep-proposal.md; not present at origin/main) and Vortex proposal with prospective retry (local checkout reference docs/experiments/benchmarks/cross-system-vortex-sweep-proposal.md; not present at origin/main). This does not replace the historical CR2 no-nomination. No production code, independent production oracle or persistent format changed.

The fixture has 32,768 rows, seed 42, 16/1,024-byte UTF8 bodies, clustered/mixed physical order and 1,024/8,192-row Parquet groups. Queries use exact literal contains, half-open time bounds, canonical timestamp/identity/sequence/index ordering and limit 64. Selective, rare and common predicates match 32, 512 and 16,384 rows respectively; narrow-time matches 2,048. The retained logical rows are identical across physical layouts. Wide filler repeats deterministic hex 64-byte patterns: these compressible synthetic bodies do not represent a production corpus. Vortex uses the exact retained Parquet objects with 8,192 rows per group, but its own file layout.

All ranges below span the two separate fresh-process reuse medians, each formed from three calls. They are descriptive ranges, not confidence intervals or pooled acceptance. First calls are separately retained; no OS cache flush or cold/warm claim follows. Both processes inherit CPU affinity [0,1]. Libraries use the registered thread settings; CPU time includes work on all process threads. Cgroup memory.max 20 GiB, memory.high 16 GiB and swap.max 0 were recorded.

Run / contained coordinator receipt	Exit	Whole-job seconds	Cgroup peak bytes	Successful retained bytes
query-sweep-02 (local evidence omitted from stage)	0	105.982	915,492,864	16,793,600
vortex-sweep-02 (local evidence omitted from stage)	0	48.150	1,065,844,736	98,103,296
query-report-01 (local evidence omitted from stage)	0	7.334	197,578,752	report/receipts retained

The reporter command recorded in that receipt was executed through python3 tools/resource_group.py -- with child argv python3 -B tools/bench/labs/cross_system/query_sweep_report.py --query docs/experiments/benchmarks/data/cross-system-sweep-01/query/sweep-02 --vortex docs/experiments/benchmarks/data/cross-system-sweep-01/query/vortex-02 --out docs/experiments/benchmarks/data/cross-system-sweep-01/coordinator/query-report-01/report.json. The sweep receipts contain the exact registered execution argv and frozen sources.

Successful owned scratch was removed. Worker RSS high-water is cumulative across construction, verification and all arms: wide query children record 572,100–600,708 KiB; wide Vortex children 596,872–605,140 KiB. These are not isolated arm memory peaks, requested allocation counts or deployment memory estimates.

Locality changes what a group index can avoid

Wide-body reuse wall time in milliseconds; each entry gives both-repetition range:

Rows/group / physical order / query	Arrow full	Arrow pruned	DuckDB Parquet	DuckDB ingested table
1,024 / clustered / selective	84.90–85.58	2.94–3.00	13.56–13.63	18.20–19.12
1,024 / mixed / selective	87.33–89.14	61.90–63.14	15.83–16.08	18.49–22.44
8,192 / clustered / selective	83.93–85.94	18.83–19.15	12.02–15.38	18.44–19.79
8,192 / mixed / selective	86.50–88.10	84.45–85.34	14.93–15.11	19.19–19.55
1,024 / clustered / narrow-time	28.34–29.66	2.37–2.47	3.36–3.49	3.33–3.37
8,192 / clustered / narrow-time	24.68–27.52	5.64–5.77	4.81–5.43	2.75–2.82
8,192 / mixed / narrow-time	25.57–26.41	25.50–25.81	11.19–12.42	18.14–18.27
8,192 / clustered / all	48.04–51.17	50.44–51.09	4.85–5.12	18.91–21.07

The actual selector/read/filter/sort path changes only candidate groups in the two Arrow arms. For selective queries, clustered groups retain 1/32 or 1/4; mixed groups retain 22/32 or 4/4. Corresponding compressed-column candidate bytes are 48,102/1,532,255; 372,250/1,480,974; 1,224,374/1,780,651; and 1,711,607/1,711,607. These are metadata sums, not measured storage reads or decoder bytes. The 8,192-group selective Arrow-pruned median is 6.38–6.41 times the 1,024-group value when clustered, and 1.35–1.36 times when mixed. This supports locality and granularity as causal candidate-avoidance factors. It does not establish an optimal group size for other query distributions.

Absent predicates select no groups across all layouts: wide Arrow-pruned wall 0.285–0.372 ms versus full 82.12–88.60 ms. Common/all and mixed rare/time predicates often retain every group; pruning then produces small, inconsistent changes and sometimes a penalty. The narrow mixed 8,192 selective case is 5.61–5.95 ms pruned versus 5.59–5.85 ms full. Wide clustered 8,192 all rises from 48.04 to 50.44 ms in repetition 1. Blanket indexing is unsupported.

Execution boundaries and construction repayment

DuckDB performs predicate/order/limit inside SQL; Arrow executes library reads, filters and sorting through Python bindings. Both project only the final 64 rows to Python, rather than converting every scanned row to Python objects. For wide clustered 1,024-group selective queries, final Python projection is 0.236–0.242 ms Arrow full, 0.226–0.233 ms Arrow pruned, 0.137–0.146 ms DuckDB Parquet and 0.120–0.128 ms DuckDB table. Most elapsed difference is in their query-library boundary. Full native SQL versus this library/Python scan is a matched-answer adapter comparison, not an interchangeable broad engine baseline. Query-sweep's combined native phase does not separately identify decoding, filtering, planning or sorting; use the Vortex phases for that narrower question.

Construction component	Narrow-body wall range	Wide-body wall range
Source generation	67.24–71.22 ms	133.39–140.73 ms
Source to Arrow	14.39–26.17 ms	70.44–84.11 ms
Parquet write	9.14–20.17 ms	39.39–57.77 ms
Python exact-trigram index build	63.74–77.77 ms	4,513.44–4,662.33 ms
Index serialize/write/readback	15.41–92.33 ms	19.56–181.82 ms
DuckDB ingestion	17.79–25.42 ms	84.71–98.27 ms
DuckDB checkpoint	23.68–26.13 ms	161.67–196.44 ms

The exact-trigram sets are an explanatory Python lab index, not Fabric Bloom filters or an optimized native index builder. Wide selective wall repayment charges index build plus serialize/write/readback: clustered 1,024 groups need 57–58 repeats; clustered 8,192 need 70–71; mixed 1,024 need 178–196; mixed 8,192 need 1,685–2,247. Absent needs 54–57. Full-candidate tiny savings give unstable thousands-of-query estimates or no finite repayment; they are not a reason to deploy the index. This arithmetic assumes repeated identical queries and unchanged data and excludes updates/refills and index selection policy.

Ingesting the narrow fixture helps many shapes: selective table wall 2.16–2.34 ms clustered and 2.48–2.66 ms mixed; same-query construction repayment 9–33 queries. Nevertheless narrow clustered 8,192 rare repetition 0 and common repetition 1 have nonpositive table savings. For wide bodies, the table is slower than DuckDB Parquet in every sampled shape except clustered narrow-time. Table checkpoint/ingestion is therefore not universally repaid: the wide clustered 8,192 narrow-time model needs 94–142 queries, versus 2,249–7,209 at 1,024 groups. No internal DuckDB compression/plan cause for this reversal was isolated here.

Vortex selection avoids downstream work, but full conversion can lose

Wide-body reuse wall milliseconds on the exact 8,192-group input objects:

Physical order / query	Arrow Parquet full	Vortex full	Vortex predicate
clustered / selective	76.98–78.05	96.42–98.78	16.08–18.00
mixed / selective	75.94–80.10	95.57–96.44	17.47–17.65
clustered / rare	75.03–75.85	94.21–95.97	16.46–17.61
mixed / rare	78.12–78.33	94.40–95.08	18.61–20.63
clustered / common literal C%z	55.88–57.60	72.92–74.83	74.03–74.99
mixed / all	53.01–55.31	72.17–72.43	65.89–72.63
clustered / narrow-time	18.26–19.88	36.94–37.59	3.67–5.60
mixed / narrow-time	18.47–19.27	35.62–38.23	7.34–8.87

Within the same Vortex adapter, wide clustered selective full representation scan is 0.518–0.607 ms, Arrow conversion 35.23–36.21 ms, exact filter/order/limit 58.81–61.73 ms and final Python projection 0.230–0.269 ms. Predicate scan instead costs 15.09–16.79 ms, conversion 0.529–0.558 ms, exact operation 0.278–0.397 ms, and Python projection 0.261–0.262 ms. The gains therefore come from avoiding conversion and subsequent exact filtering of rejected rows; scan is more expensive, and final 64-row Python materialization does not improve materially. Separate phase medians need not sum to the median combined call.

Selective predicate CPU 30.57–30.81 ms clustered / 29.36–29.93 ms mixed is larger than wall 16.08–18.00 ms / 17.47–17.65 ms. A wall-only interpretation hides parallel CPU work. Narrow-body Vortex full is faster than Arrow on all sampled shapes; for example clustered selective 3.01–3.02 ms versus 5.88–6.39 ms, while predicate is 2.04–2.20 ms. Wide full conversion reverses that benefit on every sampled shape.

The common literal contains C%z cannot safely become LIKE %C%z%: % is a wildcard. Its registered exact fallback pushes down time only and retains the full-width conversion/filter penalty. Unrestricted queries also provide little predicate benefit. Narrow-time clustered conversion 1.84–2.33 ms versus mixed 4.99–6.06 ms supports a locality effect even after time selection.

Vortex write/compress wall 536.62–542.31 ms clustered and 419.54–423.94 ms mixed wide rows; initial Parquet decode 32.09–33.94 ms, identity widening 0.456–0.682 ms and open 0.134–0.161 ms are separately charged. Selective wall repayment is 10 queries clustered / 8 mixed; CPU repayment 20 clustered / 14–15 mixed. Narrow-time wall repayment 39–48; common/all and every wide full-scan shape have no finite repayment. These models are against Arrow Parquet full, not native Fabric.

Compression is also a null/adverse finding on this fixture: wide clustered Vortex file 22,091,724 bytes versus Parquet 1,493,143; mixed 22,386,852 versus 1,723,787. Encoding-tree text includes FSST only for the wide mixed populations; kernel dispatch is explicitly unverified. No FSST-specific speedup or physical decoded-byte claim follows from tree text or array-reported nbytes.

Exactness, provenance and preserved failures

The report independently rejected four representative report defects. Query workers rejected 96 answer-checker controls and 32 actual omitted/corrupted group-index injections, restored the index, and checked 192 engine literal answers. Vortex rejected 48 answer-checker controls, 8 actual LIKE wildcard false-positive cases (C%z versus Cxxz), and checked 168 literal/boundary answers. Every timed retained answer was rechecked from the source; controls are outside timing. The schema has no nulls and attrs hold the fixed {"source":"sweep"} value: nullable/rich-attribute correctness and malformed-input safety are outside this fixture.

Measured releases are DuckDB 1.4.3, PyArrow 22.0.0 and vortex-data 0.87.0 on CPython 3.14.7. Exact wheel filenames, SHA256, lengths and all seven Vortex dependency versions are in the report and original wheel receipts. Inspected DuckDB source pin 42d15ef8e0b84f3364cf507b40ebc37487299747 and Vortex source pin 881b867d736e8955a0381655f80d6d2d630fc590 are distinct upstream revisions, not evidence that those exact implementations execute in these wheels. Historical source archives were hashed/read back; current working-tree source was not substituted. Source dissection provides hypotheses, not release-binary attribution.

The original query-sweep-01 failure (local evidence omitted from stage) occurred before cells: uv's owned cache created relative symlinks rejected by strict scratch accounting. Its full preservation receipt (local evidence omitted from stage) records two exact archive readbacks before cleanup. The registered retry disabled uv cache, used copy linking and preserved the strict link policy. The original Vortex failure (local evidence omitted from stage) was empty chunked-struct Arrow materialization in the first worker; the prospective retry uses the returned struct schema for empty output with the same checked cast. Original failure receipts remain failed. The resource launcher relocated failed Vortex scratch to data-drive evidence; its archival/cleanup receipt must be consulted separately from the original removed: false harness receipt.

Prepared-scan follow-up: cheap preparation, inconsistent benefit

The separately registered prepared-scan proposal (local checkout reference docs/experiments/benchmarks/cross-system-vortex-plan-reuse-proposal.md; not present at origin/main) was executed on the exact retained Vortex files without copying/rebuilding them. Execution receipt (local evidence omitted from stage) records exit 0, 25.905 s and cgroup peak 849,293,312 bytes; successful scratch was removed and 1,290,240 bytes retained. The independent report (local evidence omitted from stage) rechecked 384 exact source answers in eight fresh children and recomputed all recorded preparation/payback fields; its receipt (local evidence omitted from stage) records exit 0, 9.527 s and peak 186,961,920 bytes. It verified 48 generic checker rejections, 8 actual prepared unsafe-LIKE rejections with exact fallback recovery, 112 engine literal checks, and rejected four injected report defects. Wheel/API source receipts bind the actual 0.87.0 wrappers/stubs; historical inputs and their hashes remain unchanged. Resource bounds and first/reuse limitations match the proposal; six prepared plans share each child process.

Selective-query values below retain repetition 0 / repetition 1, not pooled samples. Ratios are prepared/new; lower is faster. Preparation is charged once per query, separately from execution, and both arms use prebuilt expressions.

Body / layout	New scan reuse wall ms (r0 / r1)	Prepared reuse wall ms (r0 / r1)	Wall ratio (r0 / r1)	CPU ratio (r0 / r1)	Preparation ms (r0 / r1)	Wall repayment queries (r0 / r1)
16 / clustered	2.206 / 2.175	1.941 / 2.086	0.880 / 0.959	0.878 / 0.950	0.043 / 0.038	1 / 1
16 / mixed	1.877 / 2.190	1.865 / 1.996	0.993 / 0.911	0.967 / 0.906	0.029 / 0.049	3 / 1
1,024 / clustered	14.574 / 14.168	13.314 / 13.973	0.914 / 0.986	0.969 / 0.978	0.038 / 0.059	1 / 1
1,024 / mixed	16.661 / 18.797	19.656 / 17.991	1.180 / 0.957	1.116 / 0.950	0.037 / 0.035	none / 1

Positive CPU savings in these selective populations repay in one query; the wide mixed adverse repetition has no finite CPU repayment. Across all shapes, recorded preparation wall cost is 0.020–0.201 ms; absent queries prepare more expensively than selective/rare/common/time queries. These small costs make positive same-query repayment cheap but do not establish portable savings.

There are meaningful adverse and null cases. Wide rare wall ratios flip from 0.928 to 1.051 clustered and 0.876 to 1.117 mixed; wide common flips from 0.943 to 1.019 clustered and 0.960 to 1.015 mixed. Wide narrow-time flips 0.925/1.147 clustered and 1.153/0.845 mixed. Narrow mixed unrestricted queries are slower in both repetitions, 1.223/1.018 wall and 1.219/1.023 CPU. Wide mixed absent queries improve wall 0.953/0.934 but penalize CPU 1.002/1.047, so they have no finite CPU repayment. Wide mixed selective first calls are also slower in both repetitions: wall 1.064/1.125 and CPU 1.052/1.072, before preparation is charged.

For wide clustered selective, scan-phase wall median changes 13.64 → 12.09 ms in repetition 0 and 13.28 → 12.41 ms in repetition 1, while selected conversion remains 0.37–0.44 ms. For wide mixed repetition 0, scan instead increases 14.94 → 18.13 ms with conversion nearly unchanged 0.979 → 0.964 ms. Thus the API path affects the scan boundary, but the tiny measured preparation cost cannot alone explain millisecond shifts: retained execution state/cache effects and measurement variation remain plausible. Phase medians do not sum exactly to combined medians. The result does not establish planning dominance, a universally faster prepared path or a native Fabric gain.

The strongest surviving trajectory remains optional candidate avoidance with physical locality, followed by selected conversion/validation work. Prepared execution is a cheap optional API capability for repeated fixed queries, with population-specific CPU/wall tradeoffs. Further native Fabric claims require matched admitted fixtures and preserved malformed-row/identity validation. No additional workload is proposed as an execution decision here.