Stack: ggml-ork fork feat/ork-async-stream · ggml/src/ggml-ork/ggml-ork.cpp (ggml_backend_ork_ffn_swiglu_chain) · env-gated experiments. Board: RK3588 (10.3.0.236), DDR 2112 / A76 2.4 GHz, governors=performance. Model: qwen3-1.7b-q8_0 (ORK_DIRECT=1 ORK_PERSIST=qwen3-1.7b-q8_0.orkpack ORK_FFN_CHAIN=1), prefill P=256 / ubatch=256, one clean run per boot (cross-run IOVA is not freed; a 2nd run collapses to ~4 t/s).

The int8 FFN SwiGLU chain runs the exact CPU fp32 SiLU (accurate, and the fused-fp16/int8 SiLU is quality-dead — see the SmoothQuant notes). That SiLU is serial. Hide it under independent NPU matmul work (the up projection depends only on x, not on the gate), and parallelize it across cores, to close the prefill gap.

config	prefill t/s	Δ vs baseline	note
baseline (ORK_FFN_CHAIN=1, serial exact CPU SiLU)	141.70	—	naive chain
apipe + pool(0–6) — async-worker overlap	150.97	+6.5%	documented + removed (this experiment)
doorbell — ORK_FFN_SCHED+driver, tuned	150.93	+6.5%	production path (ties apipe)
doorbell nc=0 greedy (mis-placed)	112	−21%	the core-placement trap
generalized ork_spine_run (dynamic DAG)	55–90	−36..−61%	poll-tick + per-op dispatch tax

Correctness: PPL bit-identical across baseline / apipe / doorbell — NLL 2.9118, PPL 18.3891 (ork_ppl … wiki_tiny.txt 256 256). The overlap changes scheduling only, never numerics.

Per FFN layer, in ggml_backend_ork_ffn_swiglu_chain, gated ORK_FFN_APIPE (requires the default fc.silu_cpu int8-gate + CPU-SiLU path):

1. gate = ork_mm_run_i8(owg, M, xr_i8, gate32) — blocking, full-M (owg = per-channel int8 gate, resolved like the GU_CHAIN path).
2. Launch up = ork_mm_run_i8_async(ctx->npu, owu.w, M, xr_i8, up32) — the async launcher runs the blocking ork_mm_run_i8 on a worker thread; the RKNPU submit blocks that worker in the kernel completion-wait (IRQ-driven, the worker SLEEPS), freeing its core.
3. On the calling thread, meanwhile: exact CPU SiLU silu_f[i] = g/(1+exp(-g)) over gate32 (dequant per-row as_x·per-channel bsg). Parallelized via a work-stealing CPU pool (OrkFfnPool: std::deque<std::function<void()>>, atomic fetch_add tile pull, CV-woken persistent workers pinned to ORK_FFN_POOL_CORES, single final join — no per-tile barrier, so little A55 cores contribute their ~⅓ share without dragging).
4. ork_async_wait(up) → rendezvous → dequant up32→up_f. Then normal Phase-3 GLU + down.

Overlap window = silu(gate) ∥ up; up fully hidden (drain=0). Env for the winner: ORK_FFN_APIPE=1 ORK_FFN_POOL_CORES=0,1,2,3,4,5,6.

Why it wins: the async up is kernel-IRQ-driven (worker sleeps), so it does not hold any big core during the NPU compute → all cores are free for the parallel SiLU, and the full-MT (128-row) up matmul incurs no doorbell tile-tax. drain=0 confirms the whole up fit inside the SiLU window.

Prefill throughput swung 112 ↔ 151 purely on core layout — this is the reusable result:

• It is NOT DRAM bandwidth. apipe+pool with the MOST silu cores (0–6) was the fastest (150.97); a bandwidth wall would make it slowest. Refuted.
• It is NOT a doorbell-fundamental limit, and NOT starvation (for the pooled case): a dedicated spinner thread (OrkNpuDriver) driving the doorbell changed nothing (112→112).
• It IS core placement / timing. The doorbell's colsplit worker threads pin to cores 0..nc-1 (npu.c CPU_SET(i)), and its spin-driver occupies a core. Silu on those cores collides. Fix: explicit nc (the greedy nc=0/all-core mode was the 112 trap) + a poll-only spinner on a little core + silu on the freed big cores → 150.93 (nc=2, driver core=0, silu=4,5,6,7). Monotonic in nc (1→130, 2→142, 3→143 at silu=0–6). The apipe path tolerates silu-on-big because its worker sleeps; the doorbell must keep silu off its active cores because its worker spins.

Wiring the whole FFN as an ork_spine_run op-DAG (units + tagged placement + deps) is correct (PPL-identical) but 55–90 t/s: ork_spine_run's 20µs poll-tick + per-op cross-thread dispatch (mutex/cond) + per-tile op decomposition. Profile: ~1.7 s of the 2.8 s wall was orchestration outside ork's matmul counters. Lesson: the adaptivity must not live in a per-op runtime loop; use the spine units (ork_spine_unit_dispatch/_wait) with a prescribed schedule, not the dynamic scheduler.

• Production = doorbell (ORK_FFN_SCHED + OrkNpuDriver spinner + OrkFfnPool): ties apipe (150.93), fits the spinner architecture, and is what the pack-time planner targets.
• apipe is documented here and REMOVED from the code — it added a second overlap mechanism (async-worker) that duplicates the doorbell's role at equal performance. This entry is its record (mechanism above is sufficient to re-implement; it lived on feat/ork-async-stream pre-strip).
• Dead siblings also removed: ORK_FFN_SILU_OVERLAP (fp16-gate async overlap, superseded), ORK_FFN_GATE_M48 (fused-fp16-SiLU gate — structurally dead: ork_mm_pack_f16_fused_act rejects the two-signed gate band; single-signed fp16 LUT index).
• Next: pack-time prescribed tuning — a one-time calibrator bakes the winning per-(chain,shape) schedule (mechanism + core-map + tile) into the orkpack; a prescribed executor replays it on ork_spine_unit. See PRESCRIBED_TUNING_PLAN.md. Adaptive across model+chipset, zero per-op runtime cost.

• One clean run per boot (sudo reboot between configs; IOVA not freed → 2nd run ~4 t/s).
• ORK_VERBOSE=1 prints per-layer lane timing (gate / cpu-silu / drain) for apipe/sched.
• Correctness gate: ork_ppl <gguf> <text> 256 256 must match baseline exactly.