Stack: ork-driver feat/ork-async-stream (+ ggml-ork fork) · Board: RK3588 (10.3.0.236), DDR 2112 / A76 2.4 GHz, governors=performance, single-stream. Models: ork = qwen2.5-7b-instruct-q8_0 (GGUF, 3-part) resident as qwen2.5-7b-q8.orkpack; closed baseline = Qwen2.5-7B-Instruct-rk3588-w8a8.rkllm on librkllmrt.

✅ Reproduction status (2026-08-03): the 71.89 t/s REPRODUCES — it's a config, confirmed at 73.03 t/s on a clean boot, current tree (0.7.x ork-driver, qwen2.5-7b-q8.orkpack). Exact config: ORK_DIRECT=1 ORK_PERSIST=…qwen2.5-7b-q8.orkpack build/bin/ork_bench <gguf> ~/bench_prompt_long.txt 256 8 256 — P=256, NO sudo, 10.07 GiB resident, churn=0. The critical knobs: (1) benchmark at P=256 (one full ubatch, avgM=256) — P=485 splits into two ubatches (avgM=228) and the partial 2nd batch drops it to ~59; (2) the lean 10.07 GiB (no-sudo) layout is the fast one — running under sudo forces 12.26 GiB and is slower (~59). An earlier (2026-07-30) revision of this banner wrongly called 71.89 "not reproducible / a bandwidth gap" — that was a WRONG-CONFIG artifact (I'd measured P=485 under sudo). At the correct P=256 config ork is ~73 t/s ≈ parity with rkllm's 73–75. Full config ledger: Benchmark Configs Registry § BR-1. (Token-count caveat still stands: this is ork@256 vs rkllm@485; an equal-P A/B is still worth doing.)

The Prefill Parity Grind (2026-06-29) left Qwen2.5-7B prefill at ~31→40 t/s, 54% of rkllm, with the diagnosis that "the rest is host-side integration + a per-core kernel residual, not a matmul wall." That diagnosis held: a stack of already-landed, individually-measured levers — none of them rkllm's M-fold — now puts 7B prefill at 71.89 t/s, ~98% of rkllm's 73.07 (both @ pp, warm, resident, churn=0). Approximate prefill parity on the 7B is reached, and reached without cloning rkllm's proprietary scheduler.

ork is ~98% of rkllm on prefill. The two numbers aren't yet token-matched (ork 256 vs rkllm 485; more tokens favours rkllm's throughput by amortizing fixed overhead), so on equal footing ork is at parity or slightly ahead — consistent with a prior token-matched sample (ork 62.84 @228 vs rkllm 57.24 @257 = 110%). An equal-token doorbell A/B is the one open confirmation.

Each was landed and measured on its own; this entry is the synthesis — what stacks to parity on the 7B. Ranked by contribution.

1. Weight-DMA-amortization M-tile — mg_max*64 (ork-native, the biggest single lever). The int8 single-core matmul is weight-DMA-bound, not row/compute-bound: every M-tile submit re-streams the whole K×N weight, and that cost is independent of rows per tile. Raising the M-tile cap from the old (disproven) R-1/pow2_floor "CBUF-resident rows" bound to the 0x1040 K-reduction schedule ceiling mg_max*64 amortizes the one weight-stream over the most rows the HW computes bit-exact (mc+1 miscomputes at every K). ~2.1× single-core / ~1.6× three-core at K=2048, ~1.5× at K=3584, bit-exact. This is an independent RE finding, not a port — it corrected ork's own earlier cap. See Weight-DMA Amortization and AGENTS.md §"Weight-DMA amortization".

2. Ported rkllm K=3584 schedule + a generalized (Kp,M) schedule-spec dial (the one genuine rkllm graft). The K-split scheduling (KS/Kp, synth_i8) was aligned to rkllm's captured K=3584 schedule and generalized into a (Kp,M) dial so a wide-K weight actually takes the big weight-amortizing M-tile. This is where rkllm lineage legitimately enters the parity path — the K-axis schedule, not the M-fold. (task #38.)

3. 3-core parallelism via the NONBLOCK doorbell colsplit. Prefill (nc>1) runs through run_multicore → ork_dyn_begin_mc → ork_dyn_begin_colsplit → ork_dyn_end — one doorbell submit, N-columns auto-split across cores, host rendezvous via the per-output dc civac doorbell (no thread-join). This is the in-process ORK_DIRECT path; orkd merely wraps the same doorbell to serialize multiple clients. So the parity number is a doorbell number. See Dynamic-Submit API & Doorbell.

4. Fully-resident weights, churn-free (orkpack + IOMMU multi-domain). The complete model is packed once into the .orkpack and held resident across 8 IOMMU domains (loaded 139 from disk, packed 0, churn = 0, 10.07 GiB resident). This removes the residence-eviction thrash that made early 7B numbers swing 11→36 t/s. See Multi-Domain Residence and the orkpack import path (client dma-buf import by fd, SCM_RIGHTS).

rkllm's M-folded int8 matmul (matmul-as-convolution: fold tokens into the CNA width dim so the weight streams once per program) is not how ork reaches parity. Its status is unchanged and separate:

• Measured at 1.66× as a single-core kernel replay only (rkllm's captured regcmd 433 µs vs ork's single-core kernel 719 µs, via ork_npu_replay_i8 — a direct DRM submit, not integrated into inference, not the doorbell).
• The fold's A-read layout is proprietary (GROUP_LINE_OFF + C2=64 padded/strided surfaces) and not derivable from the public TRM/NVDLA/Mesa + the captured regcmd; on-board probing is wedge-prone. See [mfold notes / RE roadmap].

The convergence framing: ork and rkllm both exploit "stream the weight once, amortize over more rows." rkllm does it via the fold on one core; ork does it via a big mg_max*64 M-tile across three cores + a ported K-split dial. Same destination, ork's route is bit-exact and ships today. The fold would only matter if "fold × 3 cores" could beat rkllm — a speculative upside gated on cracking the proprietary A-layout, not on the table for parity.

• Quant paths differ: ork runs GGUF Q8_0, rkllm runs w8a8. Both are 8-bit weight / 8-bit-NPU-activation and practically comparable (each runtime consumes its native format), but this is not a same-format arm — see Quantization Paths & Baselines. No PPL cross-check was run on the 7B this session.
• Token counts not matched (ork 256 / rkllm 485) — favours rkllm; equal-token A/B pending.
• Decode is not parity and not the story here. ork's 1.53 t/s is the forced-NPU decode (M=1, sync-wall + residence-churn bound); production decode runs on CPU (~13 t/s). rkllm's 3.98 is its NPU decode. Prefill is the compute-bound regime this page is about.
• Single clean run per boot (cross-run IOVA is not freed); the equal-token retry this session died mid-load (not OOM — board has 30 GiB free) with no result, so the token-matched confirmation is still owed.

# ork (board, single-stream, governors=performance):
ORK_DIRECT=1 ORK_PERSIST=/home/michael/qwen2.5-7b-q8.orkpack \
  ork_bench /home/michael/qwen2.5-7b-instruct-q8_0-00001-of-00003.gguf \
  /home/michael/bench_prompt_long.txt <P> 8 256
# rkllm (fresh boot, rkllm-first so its allocator isn't blocked):
sudo ./rkllm_bench ./Qwen2.5-7B-Instruct-rk3588-w8a8.rkllm 8 /home/michael/bench_prompt_long.txt

Next: the equal-token orkd(doorbell)-vs-rkllm A/B to convert "~98%, token-caveated" into a token-matched parity point.