2026-07-04 — ✅ On-NPU element-wise / activation ops (SiLU, GELU, EW-mul, residual Add, exp, rsqrt)

Supersedes Exp 2026-06-24 PPU LUT Silicon Verification (which concluded PPU activations don't run standalone). They do — the earlier attempt drove the wrong program shape. This session gets a family of standalone SDP ops running bit-exact on RK3588 silicon, in make test, driven purely by regcmd (no compiled-model-graph init).

Branch ewmul-re (ork-driver). Board 10.3.0.236. RE toolkit committed under tools/re/.

What works (productionized, in make test)

Op	int8	fp16	int16
EW-mul (SwiGLU inner silu(g)⊙up)	✅ bit-exact	✅ bit-exact	✅ bit-exact
SiLU	✅ bit-exact	⛔ (LE wall)	✅ RKNN-class (~0.24% FS)
GELU	✅ bit-exact	⛔	✅ RKNN-class
Residual Add	✅ bit-exact	✅ bit-exact	⚠️ experimental (X1 neg)
exp / rsqrt (softmax/RMSNorm blocks)	✅ bit-exact	—	✅ RKNN-class

APIs: ork_npu_ewmul_{i8,f16,i16}, ork_npu_silu_{i8,i16}, ork_npu_gelu_{i8,i16}, ork_npu_add_{i8,f16}, ork_npu_exp_{i8,i16}, ork_npu_rsqrt_{i8,i16}. All generalized to arbitrary (M,N).

Key findings

• The RK3588 NPU is NVDLA-derived. Element-wise + activation are STANDALONE SDP ops (enable_mask=0x18, 69 regcfg), reading operands from memory (SRDMA 0x5018 = X1, ERDMA 0x5038 = X2), not fused into a conv. set_mul_geom() reprograms the op to any (M,N). Feature cube: int8 atom-16, fp16/int16 atom-8, surf_stride=M*16.
• EW-mul out=clamp(round((a+za)(b+zb)·gain+zo)), gain=OUT_CVT 0x4084/0x4088 (signed 16-bit).
• Activation = LUT op: idx=(in·R)>>6+C0 → PWL-interp LUT[idx] → clamp(R·V+out_bias). The op is activation-agnostic — the LUT contents define the function, so SiLU/GELU/exp/rsqrt are one op + a curve swap. int8 index is linear (idx≈2·in+512, integer) → bit-exact. int16 index non-linear → LUT interpolation → RKNN-class (~0.24% of output full-scale, consistent) via a dense-R=1 calibration + inverse-idx LUT build.
• assemble-from-dump (tools/re/assemble_op.c): RKNN's LUT + its index/scale registers are a matched pair — transplanting both reproduces an op bit-exact-to-RKNN without decoding the index math. Proved on int16 SiLU (max|e|=7 on ~30000-mag values ≈ RKNN's own LUT quantization).
• Residual Add = SDP EW-ADD ALU mode: out=clamp((a·mult_a+b·mult_b)>>(0x4088-14)+zo). Residual (equal scales) → out=a+b, bit-exact int8 + fp16.
• RMSNorm reduction = matmul (RKNN builds ReduceMean as einsum→matmul→conv) — no dedicated reduce op. So RMSNorm/Softmax/RoPE compose from these blocks (reduce=matmul, rsqrt/exp/cos/sin=LUTs, square/scale=ewmul). RKNN's fused exNorm(78 tasks)/exSoftmax13(42 tasks, fp16) are multi-task chains — best composed in ggml-ork (they're memory-bound / NEON-competitive), not replayed.

RE toolkit (tools/re/, for onboarding new ops/NPUs)

regcmd_capture.c (LD_PRELOAD interposer) · run_rknn.c · decode_reg.c (regcmd → reg,val + diff) · assemble_op.c (dump → LUT+regcmd header) · models/build_*.py (single-op RKNN builders, in the rkllm-converter docker on .239). Loop: build single-op .rknn → capture regcmd → decode/diff vs a known op → templatize src/regcmd_*.h → marshal + validate vs CPU in an example.

Two outstanding walls (tracked)

1. fp16 activations (SiLU/GELU/exp) — LE-table exponential mode + multi-task submit. fp16 uses NVDLA's LE (Linear/Exponent) LUT in exponential mode (indexed by the fp16 exponent), not the LO linear table the int paths use; and RKNN runs fp16 silu as a 2-op chain in ONE multi-task submit — the LE-table SRAM state likely doesn't persist across our separate single-task submits (LO did). Attempted: stage-1 verbatim, full 2-op chain verbatim (ork_npu_replay_full_f16 + REGCMD_SILU_F16_T2) — both scrambled. Needs: LE-table load/index decode (exp_start/LE_INDEX_OFFSET) + multi-task-submit support. This gates the fully-fused exNorm/exSoftmax too.
2. int16 Residual Add — X1-operand negative-halving. Isolated (tools/re/add16_probe.c): the ERDMA/X2 operand is exact both signs, but the SRDMA/X1 operand (via 0x4084) HALVES NEGATIVES at int16 (int8's X1 didn't). Positive a+b bit-exact; negatives ~0.75·(a+b). The scale regs 0x4048/0x4084/shift couple (no single combo gives clean signed a+b). Needs a full X1/X2 sub-module sign-dependent scale decode. Kept EXPERIMENTAL.

Shared unlock: multi-task submit (also wanted for the fused composites) — see tools/batch_probe.c (wedges naively).