Exp-2026-09-05 — Is on-chip SRAM a separate memory port? (aggregate-bandwidth investigation)

Status: CLOSED, split verdict. SRAM is a separate port for CPU access (confirmed 2026-08-11, 104% retention under DRAM saturation). It is not one for NPU weight reads (closed 2026-09-05, ≤0.5 pp across six comparisons). The practical consequence: placing NPU weights in SRAM buys no contention relief, so the CPU∥NPU partition cannot be funded from SRAM on this SoC.

Kernel: 6.1.115-vendor-rk35xx-fence. Board 10.3.0.236. Results from other kernels are not comparable.

---

Why the question matters

The decode CPU/NPU partition stance holds that decode should not be dogmatically CPU-only — that independent ops should be partitioned across CPU and NPU for true parallelism. But a partition where both engines read the same DRAM is bandwidth-neutral at best: they simply divide one ~28 GB/s budget. It only pays if the NPU's share reads a different memory. RK3588 has one candidate: the 956 KB syssram block (DTB Change 1 + CONFIG_ROCKCHIP_RKNPU_SRAM=y).

So the load-bearing question was never "is SRAM fast?" — it is "does SRAM traffic spend DRAM's budget?" Additivity, not latency.

The five probes, and what each actually settled

#	date	probe	what it measured	verdict
1	2026-08-11	test_sram_bw	CPU↔buffer bandwidth, SRAM vs DRAM	uncached SRAM == DRAM (1.19 GB/s both) — the mapping is the wall, not the memory
2	2026-08-11	test_sram_npubw	NPU weight read, SRAM vs DRAM	inconclusive — ~95% submit overhead (~331 µs for ~18 µs of streaming)
3	2026-08-11	test_sram_pipe	CPU→SRAM under 3-core DRAM saturation	SEPARATE PORT CONFIRMED — 104% (2.89 → 2.99 GB/s, unaffected)
4	2026-09-05	sram_bw_probe	NPU weight in SRAM under a CPU hog	VOID — see below
5	2026-09-05	sram_port_probe	same, done properly	NULL, airtight — ≤0.5 pp at three weight sizes

Why probe 4 was void (the instructive one)

It defaulted to K=512 N=256 — a 128 KiB weight, one submit per iteration. 128 KiB at ~11 GB/s is ~12 µs of streaming against a ~167 µs per-submit floor: ~93% of every iteration was host and fixed submit cost. Moving 12 µs of traffic between two memories cannot register. It reported "flat" and that flatness was then explained by inventing a mechanism — "syssram is behind the same LPDDR controller, a storage class not a second port" — which contradicted probe 3, already on disk, in another project's directory.

Two lessons, both cheap to apply:

1. Never explain a null before proving the experiment could have shown a positive. A flat A/B where the manipulated variable governs 7% of the runtime is not evidence about the variable.
2. The record was in the llama.cpp project's transcripts and memory, not ork-driver's — which is why this ground was re-covered. Cross-project findings need an anchor in both indexes.

---

The airtight closure — tools/sram_port_probe.c

Three design requirements, each fixing a specific defect in the earlier attempts:

(a) Force the NPU into a weight-bandwidth-bound regime. Weight sized to ~640 KiB (of ~700 KiB usable SRAM) so each stream is ~31 µs, and T=16 tasks HW-chained in one submit, all re-reading that same weight. 640 KiB exceeds the 384 KB CBUF, so it cannot stay resident — every task re-streams. The fixed per-submit cost amortizes over 16 tasks.

(b) A hard validity gate that can VOID the run. Sweep the weight size and fit time = a + b·N:

N=160   weight=160KiB    14.40 us/task   11.38 GB/s
N=320   weight=320KiB    20.29 us/task   16.15 GB/s
N=640   weight=640KiB    31.93 us/task   20.52 GB/s
fit: time = 8.55 us + 0.0365 us/N  ->  streaming fraction 73.2%   (midpoint linearity +0.2%)

73.2% of each task is weight streaming, so placement is observable. Below ORK_MIN_FRAC (0.70) the probe refuses to print a verdict rather than report a null it has not earned. This gate is exactly what probes 2 and 4 lacked.

(c) A SPIN control that divides out scheduling. The NPU submit path is itself host-CPU-bound (~95 µs of host work per submit), so a contender that merely occupies big cores slows the NPU with no memory involvement at all. Measuring only against a memcpy hog cannot tell that apart from bus contention. So: main thread pinned to core 4, three contenders on cores 5–7, in two modes — LOAD (memcpy, ~23.5 GB/s of DRAM traffic) and SPIN (pure ALU, zero traffic). Memory-attributable slowdown = LOAD / SPIN.

The control earned its place immediately: SPIN retention is 99.8–100.1%, so with the affinity split there is no scheduling term, and every bit of the LOAD drop is memory.

Result — 3 trials, A-B-A interleaved, N=640

place	solo	under SPIN	under LOAD	memory-only	aggregate
DRAM weight	20.54 GB/s	100.0%	13.85 GB/s	67.5%	37.32 GB/s
SRAM weight	20.64 GB/s	100.0%	13.97 GB/s	67.7%	37.40 GB/s

Δ = +0.2 pp. Replicated across three weight sizes, with the sign flipping — a null, not a small win:

N (weight)	streaming fraction	DRAM	SRAM	Δ
160 (160 KiB)	41%	55.7%	56.0%	+0.3
320 (320 KiB)	58%	61.3%	60.8%	−0.5
640 (640 KiB)	73%	67.7%	68.0%	+0.3

An SRAM-resident NPU weight contends with CPU DRAM traffic exactly as much as a DRAM-resident one.

Where the contention actually lands

Retention rises with N (55.7 → 61.3 → 67.7), which is itself informative. Each task is time = a + b·N: a fixed ~8.55 µs part (A reads, C writes, regcmd/descriptor fetches — DRAM-resident in both arms) plus the b·N weight stream. Solving retention(N) = (a+bN)/(a·f + b·N·g) on the endpoints:

• g = 1.22 — the weight stream slows only 22% under a 23.5 GB/s CPU load
• f = 2.19 — the fixed DRAM-resident remainder slows 2.19×

The two-parameter fit then predicts the unseen midpoint at 61.4% against 61.3% measured — a 0.1 pp out-of-sample check, so the decomposition is not curve-fitting.

So most of the NPU's loss under CPU load is not the weight stream at all — it is the small, unavoidable, DRAM-resident per-task traffic. That part cannot be moved to SRAM (task descriptors in SRAM is a hard wedge: 0x40b carries KERNEL_MAPPING and the kernel reads them), and even the 22% that is the stream gets no relief from SRAM placement.

Reconciling probe 3 and probe 5

Both are solid; they measure different access paths:

• CPU → SRAM: immune to DRAM saturation (104%).
• NPU → SRAM: not immune — identical to DRAM (67.7% vs 67.5%).

"Separate port" is therefore directional: true of the path the CPU takes to syssram, false of the path the NPU takes to it. Any future claim in either direction must say which engine is doing the accessing — the unqualified sentence "SRAM is/isn't a separate port" has now caused an error in each direction.

Consequences

• The CPU∥NPU partition cannot be funded from SRAM. The decode-cpu-npu-partition-stance premise "wins when the NPU share reads SEPARATE memory (SRAM)" is not available on RK3588 for weights.
• ~~Running both engines is worth ~1.5× aggregate (37.4 GB/s)~~ — **that figure was wrong and is retracted.** It added two different currencies (NPU *useful weight bytes* + CPU *read+write* traffic) and exceeded the board's theoretical LPDDR peak, which is proof it over-counted. Re-measured in one currency against the DDR controller's own load counter: the dual-engine gain is **1.41×** in the decode-shaped regime and only **1.10×** when the CPU already streams well. See Exp-2026-09-05 Heterogeneous Dispatch Ceiling.
• SRAM's remaining honest uses: memory that costs no DRAM/IOVA, and a CPU-side activation pipe (separate port, but ~3 GB/s feed and ~700 KiB).

Bugs found while building the probe (all real, all worth remembering)

1. -O3 deleted the saturator. The memcpy destination is never read, so the compiler removed the copy; the hog "reached" 7,056,858 GB/s (spinning on clock_gettime) and generated no traffic — and both arms dutifully reported 99.9% retention of nothing. Fixed with an asm volatile ... "memory" barrier. A contention probe must verify its own contender; sram_port_probe now calibrates the saturator and aborts on an implausible rate.
2. ork_w_free does not return SRAM to the pool (first flagged 2026-08-11, still true). A held 640 KiB weight left only ~124 KiB, so later SRAM packs silently failed over to DRAM.
3. A 3/4 placement threshold accepted partial placement as success — 124 KiB of a 160 KiB request passed as "IN SRAM", fabricating an SRAM arm that was mostly DRAM. Now requires the whole tile.

Open

• The f = 2.19 fixed-remainder slowdown is unexplored — followed up the same day: Exp-2026-09-05 Fixed Per-Task Cost. It is host work, not NPU work (kernel hardware time for the fixed part is flat under load), and the precompiled regcmd cache removes 72–80% of it. That also explains retrospectively why SRAM placement could not help: the NPU-side fixed cost is only ~1.27 µs and never inflates.
• c->mrc TRY_ALLOC_SRAM still fails over to DRAM for reasons unknown (separate from #2 above).