Look here FIRST when bringing up a model. This is the answer to "what should I set for model X", recorded per model, with the measurement that justifies it. It exists because the right config is model-shaped: whether a path pays depends on layer count, head geometry and weight sizes measured against what the NPU actually achieves, and several paths that look like wins in isolation lose end-to-end. Guessing has cost this project repeatedly.

Three sibling pages, easy to confuse:

Machine tier (automatic, do not set by hand). The .orkpack footer carries a calibration record (orkpack_calib_hdr + orkpack_calib in ggml-ork.cpp) that the backend reads at load. It is provenance-guarded by ork_calib_valid_here() on the five things that actually move the answer — big-core max freq (so a DTB/OPP change invalidates it), big-core count, the ork-driver pack-format token, whether all big-core governors were performance, and an FNV-64 of uname release+version (a kernel change moves the submit floor). Precedence is: a sweep override > ORK_MINM > the pack's measured value > the built-in default. Today one field is live (min_m); per-shape {K, N, min_m} is expressible (K=N=0 = global) but unpopulated.

The record carries the resolved decode route — a single ordered decision (0 CPU, 1 dense M=1 on NPU, 2 + fused FFN chain and M=1 group fusion), not a set of independent switches. Each level implies the ones below, so a contradictory combination cannot be expressed. ORK_DECODE_ROUTE overrides for A/B; ORK_M1_NPU/ORK_FFN_DEC remain as deprecated aliases that can only RAISE the route.

Human tier (this page). The recommended recipe, plus the file properties — which matter more than any knob (see the quantization recipe below).

Discipline, or this page rots. A row exists only if a measurement produced it, and it names that measurement — the same rule OPS_REGISTRY.md applies to op status. A recipe with no cited number is a guess, and guesses here are worse than nothing because they read as authority.

• Decode on CPU, prefill on NPU. Not a knob — supports_op declines dense MUL_MAT at M==1.
• -t 4 (big cores only; A55s drag the threadpool barrier).
• Governors at performance, DDR included — a parked DDR governor roughly halves decode.
• MoE profile is automatic — the model type decides; nothing to set.

llama-quantize --tensor-type attn=q8_0 --tensor-type shexp=q8_0 <f16.gguf> <out.gguf> Q4_K_M

Attention precision in the GGUF trades prefill for decode, and below 5 bits the NPU declines the source entirely (sbits<5.0), forfeiting the prefill path. Expert precision is irrelevant (experts come from the pack). Measured on qwen3.6-35B-A3B: f16 attn 38.4 prefill / 5.26 decode; q8_0 36.9 / 6.59 — the balanced optimum; Q4_K 26.8 / 7.42.

All measured 2026-09-04 on rk3588 (A76 @2400 MHz, kernel 6.1.115 #70, governors performance), ork_bench P=512 G=32, decode tok/s. Route 0 = decode on CPU, 1 = dense M==1 on NPU, 2 = + fused FFN chain and M=1 group fusion.

qwen3.5-0.8b is the first model where decode-on-NPU pays — 2.44 → 3.40 tok/s (was 3.13 before the doorbell big-core fix), reproducible to ±0.4% across 3 trials (far outside the ~9% band seen on the 0.6b; noise is config-dependent). Route 1 == route 2 (3.13 both), so the win is M==1 acceptance alone and the fused FFN chain contributes nothing — the row says route 1. attn_kv=0 marks the GDN/linear-attention architecture, whose CPU decode baseline is unusually low (2.44 vs 10–15 for the dense models), leaving headroom the others don't have. This divergence is exactly what the table exists for; a single global default would have hidden it.

One row covers qwen3-1.7b f16 AND q8_0 — identical fingerprints. That validates keying on shape rather than name: source precision doesn't change the routing answer, so re-quantizing a model inherits its recipe automatically.

The Mamba rows are deliberately absent from the table. Their fingerprint collapses to n_layer alone (SSM models have no attn_q/ffn_gate/attn_k tensors), so a row would risk colliding with any future model of the same depth and applying the wrong recipe. A guard rejects such keys and falls through to the default — which both measured as best anyway, so nothing is lost. Giving them real rows needs SSM-specific fields in the key.

-t 4, warm, board 10.3.0.236. The canonical parity claim; see Benchmark Configs Registry for the exact provenance.

A trade: +5.5% PPL and ~15% slower decode for 1.59× prefill. Prefill-heavy workloads (long prompts, RAG, batch) win; decode-dominated chat may prefer ORK_MOE_AUTO=0. Nothing else to set — the pack tier and the CPU/NPU split are chosen from the model type.

Recipe: the standing default. Do not enable the decode-on-NPU paths.

Every decode-on-NPU config is a 2.8–3× regression.

⚠ decode tok/s here has a ~9% run-to-run spread (5.12–5.58 observed for the same config across runs), so the difference between routes 1 and 2 is NOT resolved by this measurement, and per-change deltas under ~10% are not signal. Earlier revisions of this page quoted "+2.8%" for q/k/v fusion and "+3.4%" for the FFN chain as if they were wins; both sit inside the noise band and are withdrawn. Use ORK_FFN_PROF's per-hop GB/s (stable) for routing decisions, not end-to-end tok/s. PPL is bit-deterministic here and is safe for quality deltas. ORK_FFN_CHAIN's quality cost is negligible (+0.13%) but it is 28% slower — it is blocked on speed, not the per-channel requant concession its comment describes.

Recipe: the standing default.

PPL here is bit-deterministic (a repeat baseline reproduced 13.9679 / NLL 2.636764 exactly), so these deltas are signal, not noise.

State it this way rather than "profitable on some models", because that is what the data supports. Measured 2026-09-04 (decode_bw_probe, ORK_FFN_PROF, ork_bench):

• Dispatch is not the wall — 48 µs of 1819 µs/layer (2.6%). The doorbell floor (~5 µs dispatch, ~16 µs/task) is already negligible.
• The NPU is not bandwidth-lost: batching S independent M=1 matmuls reaches 8.6 GB/s at S=1 and ~20 GB/s at S=6, against a ~22 GB/s ceiling and the CPU's ~9.4 GB/s effective.
• But the FFN handler gets 7.2 GB/s (gate+up) and 3.5 GB/s (down) on shapes that reach 8.6–8.8 at S=1 in isolation — 1.75–2.4× off, cause unknown.

So the gap is implementation, not hardware, and the per-model calibration record is what would find the first model where it flips — rather than assuming one exists. Disproven causes for the down gap, do not re-try: wide-K (its exact shape reaches 8.81 GB/s), missing Bf (8.28 GB/s without it), and the submit mechanism (the probe's S=1 short-circuits to the same ork_i8_mm_run).

ORK_FFN_DEC was dead for its whole existence in two independent ways: gated on ctx->via_orkd (unreachable in the default in-process mode) and declined upstream by supports_op at M==1. It needs ORK_M1_NPU=1 to fire at all. A recipe that lists knobs as independent booleans reproduces that failure — some combinations are silent no-ops. Record the decision ("where does decode run"), not the mechanisms. ORK_NO_BF=1 is a second instance: it declines the batched chain entirely (S>=2 returns nonzero), so it silently disables any batching plan.