Benchmarks 2024 11 21 TFLM LLVM Os spike_rv64 - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)- Spike :
0bc176b3fca43560b9e8586cdbc41cfde073e17a - Spike PK :
7e9b671c0415dfd7b562ac934feb9380075d4aa2
- Spike :
Toolchains
-
RISC-V GCC:
- Scalar:
riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0 - Vector:
riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0 - Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
- Scalar:
-
LLVM/Clang:
clang version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)- Linker: lld (TODO)
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Frameworks
-
MLonMCU :
develop -
TFLM :
8eb6b23de4470d6a8da3131650d6a67514dfa130
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -Os, Target: spike_rv64 )
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
39715359 ( 0.4x ) |
143524 ( 0.865 ) |
38304 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
31996087 ( 0.5x ) |
148650 ( 0.896 ) |
38312 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
30075319 ( 0.5x ) |
148650 ( 0.896 ) |
38312 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
29114935 ( 0.5x ) |
148650 ( 0.896 ) |
38312 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
28636608 ( 0.5x ) |
148650 ( 0.896 ) |
38312 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
28636608 ( 0.5x ) |
148650 ( 0.896 ) |
38312 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
28638473 ( 0.5x ) |
148650 ( 0.896 ) |
38312 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
14702893 ( Base ) |
165840 ( Base ) |
38304 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
15064322 ( 1.0x ) |
165092 ( 0.995 ) |
38304 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
5820167 ( 2.5x ) |
173674 ( 1.047 ) |
38312 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
4782018 ( 3.1x ) |
173674 ( 1.047 ) |
38312 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
4264931 ( 3.4x ) |
173674 ( 1.047 ) |
38312 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
3964831 ( 3.7x ) |
173674 ( 1.047 ) |
38312 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
3964831 ( 3.7x ) |
173674 ( 1.047 ) |
38312 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
3968561 ( 3.7x ) |
173674 ( 1.047 ) |
38312 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
4074863 ( 3.6x ) |
167314 ( 1.009 ) |
38304 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2819847 ( 5.2x ) |
167314 ( 1.009 ) |
38304 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2142511 ( 6.9x ) |
167314 ( 1.009 ) |
38304 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2103380 ( 7.0x ) |
167314 ( 1.009 ) |
38304 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2103380 ( 7.0x ) |
167314 ( 1.009 ) |
38304 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2105245 ( 7.0x ) |
167314 ( 1.009 ) |
38304 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
6891442 ( 2.1x ) |
172540 ( 1.04 ) |
38312 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
5886365 ( 2.5x ) |
172540 ( 1.04 ) |
38312 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
5383949 ( 2.7x ) |
172540 ( 1.04 ) |
38312 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
5102382 ( 2.9x ) |
172540 ( 1.04 ) |
38312 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
5102382 ( 2.9x ) |
172540 ( 1.04 ) |
38312 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
5106112 ( 2.9x ) |
172540 ( 1.04 ) |
38312 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
121111112 ( 0.5x ) |
184936 ( 0.927 ) |
71072 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
57755288 ( 1.0x ) |
190998 ( 0.957 ) |
71080 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
47301688 ( 1.2x ) |
190998 ( 0.957 ) |
71080 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
44787128 ( 1.2x ) |
190998 ( 0.957 ) |
71080 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
44324193 ( 1.3x ) |
190998 ( 0.957 ) |
71080 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
44324193 ( 1.3x ) |
190998 ( 0.957 ) |
71080 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
44326058 ( 1.3x ) |
190998 ( 0.957 ) |
71080 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
55734439 ( Base ) |
199526 ( Base ) |
71072 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
71528934 ( 0.8x ) |
198988 ( 0.997 ) |
71072 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
25734308 ( 2.2x ) |
208438 ( 1.045 ) |
71080 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
17771159 ( 3.1x ) |
208438 ( 1.045 ) |
71080 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
14040404 ( 4.0x ) |
208438 ( 1.045 ) |
71080 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
12339982 ( 4.5x ) |
208438 ( 1.045 ) |
71080 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
11510542 ( 4.8x ) |
208438 ( 1.045 ) |
71080 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
10892192 ( 5.1x ) |
208438 ( 1.045 ) |
71080 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
15251347 ( 3.7x ) |
201744 ( 1.011 ) |
71072 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
9668147 ( 5.8x ) |
201744 ( 1.011 ) |
71072 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
7122307 ( 7.8x ) |
201744 ( 1.011 ) |
71072 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
5882044 ( 9.5x ) |
201744 ( 1.011 ) |
71072 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
4953780 ( 11.3x ) |
201744 ( 1.011 ) |
71072 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
4706553 ( 11.8x ) |
201744 ( 1.011 ) |
71072 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
19802584 ( 2.8x ) |
207216 ( 1.039 ) |
71080 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
13510163 ( 4.1x ) |
207216 ( 1.039 ) |
71080 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
10560683 ( 5.3x ) |
207216 ( 1.039 ) |
71080 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
9218880 ( 6.0x ) |
207216 ( 1.039 ) |
71080 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
8563520 ( 6.5x ) |
207216 ( 1.039 ) |
71080 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
8075730 ( 6.9x ) |
207216 ( 1.039 ) |
71080 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
2782346 ( 0.6x ) |
338500 ( 0.983 ) |
21440 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
952434 ( 1.7x ) |
340860 ( 0.99 ) |
21440 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
689266 ( 2.4x ) |
340860 ( 0.99 ) |
21440 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
557682 ( 2.9x ) |
340860 ( 0.99 ) |
21440 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
491890 ( 3.3x ) |
340860 ( 0.99 ) |
21440 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
458994 ( 3.5x ) |
340860 ( 0.99 ) |
21440 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
456763 ( 3.6x ) |
340860 ( 0.99 ) |
21440 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
1627210 ( Base ) |
344186 ( Base ) |
21440 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
2972834 ( 0.5x ) |
344190 ( 1.0 ) |
21440 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
598831 ( 2.7x ) |
347900 ( 1.011 ) |
21440 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
481391 ( 3.4x ) |
347900 ( 1.011 ) |
21440 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
422671 ( 3.8x ) |
347900 ( 1.011 ) |
21440 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
393311 ( 4.1x ) |
347900 ( 1.011 ) |
21440 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
378631 ( 4.3x ) |
347900 ( 1.011 ) |
21440 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
376791 ( 4.3x ) |
347900 ( 1.011 ) |
21440 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
1935500 ( 0.8x ) |
345820 ( 1.005 ) |
21440 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1817132 ( 0.9x ) |
345820 ( 1.005 ) |
21440 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1757948 ( 0.9x ) |
345820 ( 1.005 ) |
21440 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1728692 ( 0.9x ) |
345820 ( 1.005 ) |
21440 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1725008 ( 0.9x ) |
345820 ( 1.005 ) |
21440 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1723124 ( 0.9x ) |
345820 ( 1.005 ) |
21440 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
998546 ( 1.6x ) |
347904 ( 1.011 ) |
21440 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
700178 ( 2.3x ) |
347904 ( 1.011 ) |
21440 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
550994 ( 3.0x ) |
347904 ( 1.011 ) |
21440 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
476402 ( 3.4x ) |
347904 ( 1.011 ) |
21440 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
439106 ( 3.7x ) |
347904 ( 1.011 ) |
21440 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
434450 ( 3.7x ) |
347904 ( 1.011 ) |
21440 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
104295252 ( 0.4x ) |
417166 ( 0.949 ) |
136608 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
67369502 ( 0.7x ) |
422292 ( 0.961 ) |
136616 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
61839390 ( 0.7x ) |
422292 ( 0.961 ) |
136616 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
59212574 ( 0.7x ) |
422292 ( 0.961 ) |
136616 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
58108391 ( 0.8x ) |
422292 ( 0.961 ) |
136616 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
57659047 ( 0.8x ) |
422292 ( 0.961 ) |
136616 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
57626320 ( 0.8x ) |
422292 ( 0.961 ) |
136616 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
43977423 ( Base ) |
439482 ( Base ) |
136608 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
44902157 ( 1.0x ) |
438734 ( 0.998 ) |
136608 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
18369155 ( 2.4x ) |
447316 ( 1.018 ) |
136616 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
15392163 ( 2.9x ) |
447316 ( 1.018 ) |
136616 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13967868 ( 3.1x ) |
447316 ( 1.018 ) |
136616 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13441796 ( 3.3x ) |
447316 ( 1.018 ) |
136616 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13226184 ( 3.3x ) |
447316 ( 1.018 ) |
136616 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13211580 ( 3.3x ) |
447316 ( 1.018 ) |
136616 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13381911 ( 3.3x ) |
440956 ( 1.003 ) |
136608 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
10060855 ( 4.4x ) |
440956 ( 1.003 ) |
136608 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8769399 ( 5.0x ) |
440956 ( 1.003 ) |
136608 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8267260 ( 5.3x ) |
440956 ( 1.003 ) |
136608 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8219485 ( 5.4x ) |
440956 ( 1.003 ) |
136608 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8221350 ( 5.3x ) |
440956 ( 1.003 ) |
136608 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
21892523 ( 2.0x ) |
446182 ( 1.015 ) |
136616 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
18993515 ( 2.3x ) |
446182 ( 1.015 ) |
136616 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
17606347 ( 2.5x ) |
446182 ( 1.015 ) |
136616 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
17082052 ( 2.6x ) |
446182 ( 1.015 ) |
136616 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
16866396 ( 2.6x ) |
446182 ( 1.015 ) |
136616 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
16851770 ( 2.6x ) |
446182 ( 1.015 ) |
136616 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Original data
Click here to download the raw files for this benchmark.