Benchmarks 2024 11 21 TFLM LLVM O3 spike_rv64 - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)- Spike :
0bc176b3fca43560b9e8586cdbc41cfde073e17a - Spike PK :
7e9b671c0415dfd7b562ac934feb9380075d4aa2
- Spike :
Toolchains
-
RISC-V GCC:
- Scalar:
riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0 - Vector:
riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0 - Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
- Scalar:
-
LLVM/Clang:
clang version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)- Linker: lld (TODO)
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Frameworks
-
MLonMCU :
develop -
TFLM :
8eb6b23de4470d6a8da3131650d6a67514dfa130
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -O3, Target: spike_rv64 )
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
38220981 ( 0.4x ) |
151210 ( 0.884 ) |
38312 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
29524020 ( 0.5x ) |
161498 ( 0.944 ) |
38320 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
28115428 ( 0.5x ) |
161498 ( 0.944 ) |
38320 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
27317724 ( 0.5x ) |
161498 ( 0.944 ) |
38320 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
26965592 ( 0.5x ) |
161498 ( 0.944 ) |
38320 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
42750153 ( 0.3x ) |
161498 ( 0.944 ) |
38320 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
42748288 ( 0.3x ) |
161498 ( 0.944 ) |
38320 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
14667305 ( Base ) |
171092 ( Base ) |
38312 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
15003544 ( 1.0x ) |
171044 ( 1.0 ) |
38312 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
5159524 ( 2.8x ) |
192794 ( 1.127 ) |
38320 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
4736424 ( 3.1x ) |
192794 ( 1.127 ) |
38320 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
4412659 ( 3.3x ) |
192794 ( 1.127 ) |
38320 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
5158864 ( 2.8x ) |
192794 ( 1.127 ) |
38320 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13633342 ( 1.1x ) |
192794 ( 1.127 ) |
38320 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13631477 ( 1.1x ) |
192794 ( 1.127 ) |
38320 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
4062587 ( 3.6x ) |
173830 ( 1.016 ) |
38312 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2807619 ( 5.2x ) |
173830 ( 1.016 ) |
38312 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2130307 ( 6.9x ) |
173830 ( 1.016 ) |
38312 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2091188 ( 7.0x ) |
173830 ( 1.016 ) |
38312 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2091188 ( 7.0x ) |
173830 ( 1.016 ) |
38312 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2093053 ( 7.0x ) |
173830 ( 1.016 ) |
38312 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
6560220 ( 2.2x ) |
193660 ( 1.132 ) |
38320 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
5888832 ( 2.5x ) |
193660 ( 1.132 ) |
38320 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
5552289 ( 2.6x ) |
193660 ( 1.132 ) |
38320 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
6635414 ( 2.2x ) |
193660 ( 1.132 ) |
38320 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
15113672 ( 1.0x ) |
193660 ( 1.132 ) |
38320 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
15111807 ( 1.0x ) |
193660 ( 1.132 ) |
38320 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
118582357 ( 0.5x ) |
191040 ( 0.935 ) |
71080 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
52636165 ( 1.1x ) |
203018 ( 0.994 ) |
71088 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
44969645 ( 1.2x ) |
203018 ( 0.994 ) |
71088 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
82543537 ( 0.7x ) |
203018 ( 0.994 ) |
71088 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
107245059 ( 0.5x ) |
203018 ( 0.994 ) |
71088 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
122521807 ( 0.5x ) |
203018 ( 0.994 ) |
71088 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
122521807 ( 0.5x ) |
203018 ( 0.994 ) |
71088 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
55593701 ( Base ) |
204290 ( Base ) |
71080 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
71341169 ( 0.8x ) |
204028 ( 0.999 ) |
71080 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
13511242 ( 4.1x ) |
228234 ( 1.117 ) |
71088 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
11074996 ( 5.0x ) |
228234 ( 1.117 ) |
71088 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13881658 ( 4.0x ) |
228234 ( 1.117 ) |
71088 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
15363666 ( 3.6x ) |
228234 ( 1.117 ) |
71088 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
16036238 ( 3.5x ) |
228234 ( 1.117 ) |
71088 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
35226475 ( 1.6x ) |
228234 ( 1.117 ) |
71088 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
15147423 ( 3.7x ) |
207274 ( 1.015 ) |
71080 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
9564223 ( 5.8x ) |
207274 ( 1.015 ) |
71080 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
7018383 ( 7.9x ) |
207274 ( 1.015 ) |
71080 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
5778120 ( 9.6x ) |
207274 ( 1.015 ) |
71080 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
4849856 ( 11.5x ) |
207274 ( 1.015 ) |
71080 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
4602629 ( 12.1x ) |
207274 ( 1.015 ) |
71080 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
16882088 ( 3.3x ) |
227764 ( 1.115 ) |
71088 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
15847171 ( 3.5x ) |
227764 ( 1.115 ) |
71088 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
16599560 ( 3.3x ) |
227764 ( 1.115 ) |
71088 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
16309880 ( 3.4x ) |
227764 ( 1.115 ) |
71088 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
15840770 ( 3.5x ) |
227764 ( 1.115 ) |
71088 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
42621407 ( 1.3x ) |
227764 ( 1.115 ) |
71088 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
2781356 ( 0.6x ) |
339582 ( 0.984 ) |
21440 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
787134 ( 2.1x ) |
343354 ( 0.995 ) |
21440 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
613118 ( 2.7x ) |
343354 ( 0.995 ) |
21440 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
522654 ( 3.1x ) |
343354 ( 0.995 ) |
21440 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
477422 ( 3.4x ) |
343354 ( 0.995 ) |
21440 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
454806 ( 3.6x ) |
343354 ( 0.995 ) |
21440 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
2204550 ( 0.7x ) |
343354 ( 0.995 ) |
21440 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
1630665 ( Base ) |
345136 ( Base ) |
21440 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
2979690 ( 0.5x ) |
345140 ( 1.0 ) |
21440 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
550028 ( 3.0x ) |
350532 ( 1.016 ) |
21440 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
462956 ( 3.5x ) |
350532 ( 1.016 ) |
21440 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
417740 ( 3.9x ) |
350532 ( 1.016 ) |
21440 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
395132 ( 4.1x ) |
350532 ( 1.016 ) |
21440 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
383828 ( 4.2x ) |
350532 ( 1.016 ) |
21440 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
1322357 ( 1.2x ) |
350532 ( 1.016 ) |
21440 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
1938907 ( 0.8x ) |
346416 ( 1.004 ) |
21440 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1820539 ( 0.9x ) |
346416 ( 1.004 ) |
21440 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1761355 ( 0.9x ) |
346416 ( 1.004 ) |
21440 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1732099 ( 0.9x ) |
346416 ( 1.004 ) |
21440 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1728415 ( 0.9x ) |
346416 ( 1.004 ) |
21440 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1726531 ( 0.9x ) |
346416 ( 1.004 ) |
21440 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
782955 ( 2.1x ) |
350536 ( 1.016 ) |
21440 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
601675 ( 2.7x ) |
350536 ( 1.016 ) |
21440 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
507115 ( 3.2x ) |
350536 ( 1.016 ) |
21440 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
459835 ( 3.5x ) |
350536 ( 1.016 ) |
21440 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
436195 ( 3.7x ) |
350536 ( 1.016 ) |
21440 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
2354268 ( 0.7x ) |
350536 ( 1.016 ) |
21440 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
100751269 ( 0.4x ) |
424852 ( 0.955 ) |
136616 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
71104580 ( 0.6x ) |
435140 ( 0.978 ) |
136624 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
68707812 ( 0.6x ) |
435140 ( 0.978 ) |
136624 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
68539060 ( 0.6x ) |
435140 ( 0.978 ) |
136624 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
74280540 ( 0.6x ) |
435140 ( 0.978 ) |
136624 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
80768121 ( 0.5x ) |
435140 ( 0.978 ) |
136624 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
106211610 ( 0.4x ) |
435140 ( 0.978 ) |
136624 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
43868312 ( Base ) |
444734 ( Base ) |
136616 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
44684389 ( 1.0x ) |
444686 ( 1.0 ) |
136616 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
16851584 ( 2.6x ) |
466436 ( 1.049 ) |
136624 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
16431200 ( 2.7x ) |
466436 ( 1.049 ) |
136624 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
17589992 ( 2.5x ) |
466436 ( 1.049 ) |
136624 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
20741801 ( 2.1x ) |
466436 ( 1.049 ) |
136624 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
24268847 ( 1.8x ) |
466436 ( 1.049 ) |
136624 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
38229904 ( 1.1x ) |
466436 ( 1.049 ) |
136624 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13266657 ( 3.3x ) |
447472 ( 1.006 ) |
136616 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
9945673 ( 4.4x ) |
447472 ( 1.006 ) |
136616 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8654253 ( 5.1x ) |
447472 ( 1.006 ) |
136616 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8152132 ( 5.4x ) |
447472 ( 1.006 ) |
136616 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8104366 ( 5.4x ) |
447472 ( 1.006 ) |
136616 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8106231 ( 5.4x ) |
447472 ( 1.006 ) |
136616 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
20404256 ( 2.1x ) |
467302 ( 1.051 ) |
136624 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
20126528 ( 2.2x ) |
467302 ( 1.051 ) |
136624 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
21644361 ( 2.0x ) |
467302 ( 1.051 ) |
136624 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
24796122 ( 1.8x ) |
467302 ( 1.051 ) |
136624 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
28323144 ( 1.5x ) |
467302 ( 1.051 ) |
136624 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
42284189 ( 1.0x ) |
467302 ( 1.051 ) |
136624 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Original data
Click here to download the raw files for this benchmark.