Benchmarks 2024 11 26 TFLM GCC O3 spike_rv32_min - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)- Spike :
eb0a3e2b0a7c57522928be39de95cd9f8c6dc636 - Spike PK :
fix-gcc14-rvv
- Spike :
Toolchains
- RISC-V GCC:
- Scalar:
riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118 - Vector:
riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118 - Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
- Scalar:
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Frameworks
-
MLonMCU :
develop -
TFLM :
8eb6b23de4470d6a8da3131650d6a67514dfa130
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -O3, Target: spike_rv32_min )
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
53676902.0 ( 0.3x ) |
205856 ( 0.822 ) |
36200 ( 1.0 ) |
0 | TFLM | Reference | RV32IM | 0 | - |
34688090.0 ( 0.4x ) |
215244 ( 0.86 ) |
36216 ( 1.0 ) |
128 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
215224 ( 0.859 ) |
36220 ( 1.0 ) |
256 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
216508 ( 0.865 ) |
36220 ( 1.0 ) |
512 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
217664 ( 0.869 ) |
36220 ( 1.0 ) |
1024 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
218880 ( 0.874 ) |
36216 ( 1.0 ) |
2048 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
220996 ( 0.882 ) |
36200 ( 1.0 ) |
4096 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
15076249.0 ( Base ) |
250428 ( Base ) |
36208 ( Base ) |
0 | muRISCV-NN | Scalar | RV32IM | 0 | - |
15115685.0 ( 1.0x ) |
247156 ( 0.987 ) |
36208 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV32IM | 0 | - |
8104142.0 ( 1.9x ) |
274724 ( 1.097 ) |
36232 ( 1.001 ) |
128 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
274180 ( 1.095 ) |
36236 ( 1.001 ) |
256 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
281752 ( 1.125 ) |
36236 ( 1.001 ) |
512 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
294708 ( 1.177 ) |
36236 ( 1.001 ) |
1024 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
324588 ( 1.296 ) |
36232 ( 1.001 ) |
2048 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
387140 ( 1.546 ) |
36216 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
8029703.0 ( 1.9x ) |
254044 ( 1.014 ) |
36208 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
254044 ( 1.014 ) |
36208 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
254044 ( 1.014 ) |
36208 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
254044 ( 1.014 ) |
36208 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
254044 ( 1.014 ) |
36208 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
254044 ( 1.014 ) |
36208 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
271880 ( 1.086 ) |
36232 ( 1.001 ) |
128 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
271252 ( 1.083 ) |
36236 ( 1.001 ) |
256 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
278824 ( 1.113 ) |
36236 ( 1.001 ) |
512 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
291780 ( 1.165 ) |
36236 ( 1.001 ) |
1024 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
321660 ( 1.284 ) |
36232 ( 1.001 ) |
2048 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
384244 ( 1.534 ) |
36216 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
170539889.0 ( 0.3x ) |
258472 ( 0.901 ) |
68972 ( 1.0 ) |
0 | TFLM | Reference | RV32IM | 0 | - |
- ( ?x ) |
272256 ( 0.949 ) |
68996 ( 1.0 ) |
128 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
272596 ( 0.95 ) |
69000 ( 1.001 ) |
256 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
275664 ( 0.961 ) |
69000 ( 1.001 ) |
512 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
277664 ( 0.968 ) |
69000 ( 1.001 ) |
1024 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
279928 ( 0.976 ) |
68996 ( 1.0 ) |
2048 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
283512 ( 0.988 ) |
68980 ( 1.0 ) |
4096 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
54575218.0 ( Base ) |
286832 ( Base ) |
68964 ( Base ) |
0 | muRISCV-NN | Scalar | RV32IM | 0 | - |
72413682.0 ( 0.8x ) |
286064 ( 0.997 ) |
68964 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV32IM | 0 | - |
- ( ?x ) |
320528 ( 1.117 ) |
68996 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
311784 ( 1.087 ) |
69000 ( 1.001 ) |
256 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
315196 ( 1.099 ) |
69000 ( 1.001 ) |
512 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
316676 ( 1.104 ) |
69000 ( 1.001 ) |
1024 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
318432 ( 1.11 ) |
68996 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
321104 ( 1.119 ) |
68980 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
32245145.0 ( 1.7x ) |
297056 ( 1.036 ) |
68964 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
297056 ( 1.036 ) |
68964 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
297056 ( 1.036 ) |
68964 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
297056 ( 1.036 ) |
68964 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
297056 ( 1.036 ) |
68964 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
297056 ( 1.036 ) |
68964 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
319772 ( 1.115 ) |
68996 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
311028 ( 1.084 ) |
69000 ( 1.001 ) |
256 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
314440 ( 1.096 ) |
69000 ( 1.001 ) |
512 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
315920 ( 1.101 ) |
69000 ( 1.001 ) |
1024 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
317676 ( 1.108 ) |
68996 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
320348 ( 1.117 ) |
68980 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
2789463.0 ( 0.6x ) |
381704 ( 0.973 ) |
19416 ( 1.0 ) |
0 | TFLM | Reference | RV32IM | 0 | - |
- ( ?x ) |
385936 ( 0.984 ) |
19420 ( 1.0 ) |
128 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
385456 ( 0.983 ) |
19420 ( 1.0 ) |
256 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
385972 ( 0.984 ) |
19420 ( 1.0 ) |
512 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
386396 ( 0.985 ) |
19420 ( 1.0 ) |
1024 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
386868 ( 0.987 ) |
19420 ( 1.0 ) |
2048 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
387640 ( 0.988 ) |
19416 ( 1.0 ) |
4096 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
1658478.0 ( Base ) |
392156 ( Base ) |
19416 ( Base ) |
0 | muRISCV-NN | Scalar | RV32IM | 0 | - |
2738547.0 ( 0.6x ) |
392160 ( 1.0 ) |
19416 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV32IM | 0 | - |
- ( ?x ) |
393504 ( 1.003 ) |
19420 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
392920 ( 1.002 ) |
19420 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
393196 ( 1.003 ) |
19420 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
393412 ( 1.003 ) |
19420 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
393644 ( 1.004 ) |
19420 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
394028 ( 1.005 ) |
19416 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
1990167.0 ( 0.8x ) |
397484 ( 1.014 ) |
19416 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
397484 ( 1.014 ) |
19416 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
397484 ( 1.014 ) |
19416 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
397484 ( 1.014 ) |
19416 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
397484 ( 1.014 ) |
19416 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
397484 ( 1.014 ) |
19416 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
393508 ( 1.003 ) |
19420 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
392924 ( 1.002 ) |
19420 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
393200 ( 1.003 ) |
19420 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
393416 ( 1.003 ) |
19420 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
393648 ( 1.004 ) |
19420 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
394032 ( 1.005 ) |
19416 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
139924419.0 ( 0.3x ) |
479496 ( 0.915 ) |
134504 ( 1.0 ) |
0 | TFLM | Reference | RV32IM | 0 | - |
- ( ?x ) |
488884 ( 0.933 ) |
134520 ( 1.0 ) |
128 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
488864 ( 0.933 ) |
134524 ( 1.0 ) |
256 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
490148 ( 0.935 ) |
134524 ( 1.0 ) |
512 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
491304 ( 0.937 ) |
134524 ( 1.0 ) |
1024 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
492520 ( 0.94 ) |
134520 ( 1.0 ) |
2048 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
494636 ( 0.944 ) |
134504 ( 1.0 ) |
4096 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
45274508.0 ( Base ) |
524068 ( Base ) |
134512 ( Base ) |
0 | muRISCV-NN | Scalar | RV32IM | 0 | - |
45620356.0 ( 1.0x ) |
520796 ( 0.994 ) |
134512 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV32IM | 0 | - |
- ( ?x ) |
548372 ( 1.046 ) |
134536 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
547828 ( 1.045 ) |
134540 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
555400 ( 1.06 ) |
134540 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
568340 ( 1.084 ) |
134540 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
598220 ( 1.141 ) |
134536 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
660788 ( 1.261 ) |
134520 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
25528578.0 ( 1.8x ) |
527684 ( 1.007 ) |
134512 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
527684 ( 1.007 ) |
134512 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
527684 ( 1.007 ) |
134512 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
527684 ( 1.007 ) |
134512 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
527684 ( 1.007 ) |
134512 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
527684 ( 1.007 ) |
134512 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
- ( ?x ) |
545528 ( 1.041 ) |
134536 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
544900 ( 1.04 ) |
134540 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
552472 ( 1.054 ) |
134540 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
565412 ( 1.079 ) |
134540 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
595292 ( 1.136 ) |
134536 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
- ( ?x ) |
657892 ( 1.255 ) |
134520 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
Original data
Click here to download the raw files for this benchmark.