Exp-2026-09-05 — The fixed per-task cost is HOST work, and DRAM load doubles it

Follow-on from Exp-2026-09-05 SRAM Aggregate Bandwidth, which closed the SRAM question and left this behind as the larger open lever.

Kernel: 6.1.115-vendor-rk35xx-fence, board 10.3.0.236, DMC 2112 MHz. Probes: tools/npu_fixed_cost_probe.c, tools/npu_host_cost_probe.c.

---

Where it came from

Decomposing the chained int8 matmul as time = a + b·N per task showed the two halves respond to CPU DRAM contention completely differently: the weight stream b slows 1.22×, the fixed part a slows 2.19×. Since a is ~8.6 µs against a 640 KiB weight's ~23 µs of streaming, that inflation is the dominant contention cost on this path — bigger than anything weight placement offered.

Step 1 — is it even on the NPU? (npu_fixed_cost_probe)

The kernel answers this for free: #patch74 exposes cumulative job hardware time as /sys/module/rknpu/parameters/{hw_ns_sum,hw_n}. Fitting both wall-clock and kernel-reported hardware time against N, under a SPIN contender (cores busy, zero memory traffic) and a LOAD contender (23.5 GB/s of DRAM traffic), splits it:

	fixed a	of which hardware	stream b
SPIN	8.66 µs	1.27 µs (15%)	0.0366 µs/N
LOAD	19.30 µs	−0.11 µs (≈0)	0.0445 µs/N
inflation	2.23×	flat	1.22×

The fixed cost grows +10.6 µs under load, and 113% of that growth is host-side. The NPU's own fixed per-task cost is ~1.27 µs and is completely unaffected by DRAM load.

This also explains, after the fact, why SRAM placement could not help: the NPU-side fixed cost was never the problem, so there was nothing for on-chip placement to protect.

Using SPIN rather than an idle baseline is what makes this readable — it holds CPU occupancy constant so the only difference between arms is DRAM traffic. The NPU submit path is host-CPU-bound, so an idle baseline would have mixed scheduling into the same number.

Step 2 — remove it (npu_host_cost_probe)

If the cost is the host synthesising ~1–2 KB of regcmd per task and flushing it, the precompiled regcmd cache (ork_pc_*) should remove most of it: it builds the program pool once with A/C baked in, and per call only refreshes A and submits.

Fair comparison: ork_i8_mm_run_chain routes this shape onto the doorbell spine with nc=0 (all three cores) while ork_pc_run submits with core_mask=1. Comparing as-is would confound precompilation with core count, so the chain arm is pinned with ORK_CHAIN_NC=1.

Host per-task cost = wall − kernel hardware time (the non-overlapped, critical-path part). It is N-independent by construction, which the data confirms, so it is averaged over the three weight sizes:

arm	quiet (SPIN)	under LOAD	inflation
ork_i8_mm_run_chain	4.06 µs	8.16 µs	2.01×
ork_pc_run	0.81 µs	2.25 µs	2.78×

Precompilation removes 80% of the host cost quiet and 72% under load — a saving of 3.25 µs/task quiet and 5.91 µs/task under load, 1.82× larger. Replicated (a prior run: 3.88→8.99 and 0.71→2.29).

Two readings differ and both are true: the absolute saving grows under contention, while pc's relative sensitivity is worse (2.78× vs 2.01×) because what remains is almost purely latency-exposed (an A refresh plus the submit ioctl) with no bulk work to dilute it. The actionable number is the absolute saving.

What it is worth end-to-end depends on work per task

The host term is fixed per task, so its share shrinks as the task grows. At single core, under load:

weight	run_chain	ork_pc_run	speedup
N=160 (160 KiB)	27.35 µs/task	21.12 µs/task	1.30×
N=640 (640 KiB)	71.71 µs/task	65.31 µs/task	1.10×

So the cache is worth ~1.3× at decode-sized shapes under contention and tapers as tasks get larger. This is consistent with the recorded 6.03× for regime A (tiny per-token programs) and with the 1.47× measured on the FFN layer — the spread is not inconsistency, it is the fixed cost being amortised over different amounts of work.

Consequences

• Every quiet-system benchmark of ork_pc_* understates it. Its value is largest exactly when something else is using the memory system — which is the CPU∥NPU partition case. ORK_FFN_PC is currently opt-in on the basis of quiet-system numbers plus an earlier wedge; the contention numbers argue for re-evaluating that default.
• The partition question reopens on a different axis. SRAM cannot give the NPU a private memory path (closed), but precompilation makes the NPU's host cost far less sensitive to the CPU's memory traffic, which is the other way to stop the two engines from fighting.
• Corroboration that the model is right: single-core hardware time at N=640 is 61.9 µs/task for 640 KiB = 10.3 GB/s, matching the independently known ~11 GB/s single-core weight-DMA rate.

Open

• 2.25 µs/task of host cost survives under load with pc (0.81 µs quiet). That is the A refresh plus the submit ioctl — the irreducible-looking remainder of the submit-floor-decomposed floor, now measured under contention rather than idle.
• These are timing probes and do not check correctness (same caveat as mc_prof/rknn_vs_ork). ork_pc_run's bit-exactness is established elsewhere.