Benchmarks 2024 02 20 TFLM LLVM - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim
) (ISS, CPI=1)
Toolchains
- LLVM/Clang:
- TODO: Version
- Linker: lld (TODO)
- RISC-V GCC for Headers, libc,...
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Package Versions
-
MLonMCU : main
-
TFLM : main
-
Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
-
Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2
Miscellaneous
- Used
-Os
flag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: llvm)
aww
)
Audio Wake Words (Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Auto-Vectorization |
---|---|---|---|---|---|---|---|
43849979 ( 0.4x ) |
146710 ( 0.864 ) |
36124 ( 1.0 ) |
0 | TFLM | Reference | RV32GC | - |
34194781 ( 0.5x ) |
152484 ( 0.898 ) |
36132 ( 1.0 ) |
128 | TFLM | Reference | RV32GCV | Loop+SLP |
31057605 ( 0.5x ) |
152484 ( 0.898 ) |
36132 ( 1.0 ) |
1024 | TFLM | Reference | RV32GCV | Loop+SLP |
15553915 ( Base ) |
169764 ( Base ) |
36124 ( Base ) |
0 | muRISCV-NN | Scalar | RV32GC | - |
6139962 ( 2.5x ) |
181040 ( 1.066 ) |
36132 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32GCV | Loop+SLP |
4316923 ( 3.6x ) |
181040 ( 1.066 ) |
36132 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32GCV | Loop+SLP |
6655337 ( 2.3x ) |
169342 ( 0.998 ) |
36124 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32GCV | - |
2529245 ( 6.1x ) |
169342 ( 0.998 ) |
36124 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32GCV | - |
resnet
)
Image Classification (Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Auto-Vectorization |
---|---|---|---|---|---|---|---|
134756807 ( 0.4x ) |
188332 ( 0.925 ) |
68892 ( 1.0 ) |
0 | TFLM | Reference | RV32GC | - |
58895120 ( 1.0x ) |
194258 ( 0.954 ) |
68900 ( 1.0 ) |
128 | TFLM | Reference | RV32GCV | Loop+SLP |
46357756 ( 1.3x ) |
194258 ( 0.954 ) |
68900 ( 1.0 ) |
1024 | TFLM | Reference | RV32GCV | Loop+SLP |
58389211 ( Base ) |
203590 ( Base ) |
68888 ( Base ) |
0 | muRISCV-NN | Scalar | RV32GC | - |
28236740 ( 2.1x ) |
215314 ( 1.058 ) |
68896 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32GCV | Loop+SLP |
13686219 ( 4.3x ) |
215314 ( 1.058 ) |
68896 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32GCV | Loop+SLP |
27961963 ( 2.1x ) |
203862 ( 1.001 ) |
68888 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32GCV | - |
8020141 ( 7.3x ) |
203862 ( 1.001 ) |
68888 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32GCV | - |
toycar
)
Anomaly Detection (Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Auto-Vectorization |
---|---|---|---|---|---|---|---|
3052975 ( 0.6x ) |
342160 ( 0.987 ) |
19376 ( 1.0 ) |
0 | TFLM | Reference | RV32GC | - |
895486 ( 2.1x ) |
343940 ( 0.992 ) |
19376 ( 1.0 ) |
128 | TFLM | Reference | RV32GCV | Loop+SLP |
492510 ( 3.7x ) |
343940 ( 0.992 ) |
19376 ( 1.0 ) |
1024 | TFLM | Reference | RV32GCV | Loop+SLP |
1846118 ( Base ) |
346748 ( Base ) |
19376 ( Base ) |
0 | muRISCV-NN | Scalar | RV32GC | - |
620218 ( 3.0x ) |
350296 ( 1.01 ) |
19376 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32GCV | Loop+SLP |
387846 ( 4.8x ) |
350296 ( 1.01 ) |
19376 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32GCV | Loop+SLP |
591777 ( 3.1x ) |
346756 ( 1.0 ) |
19376 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32GCV | - |
418329 ( 4.4x ) |
346756 ( 1.0 ) |
19376 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32GCV | - |
vww
)
Visual Wake Words (Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Auto-Vectorization |
---|---|---|---|---|---|---|---|
114164825 ( 0.4x ) |
420470 ( 0.948 ) |
134452 ( 1.0 ) |
0 | TFLM | Reference | RV32GC | - |
72029724 ( 0.6x ) |
426196 ( 0.961 ) |
134460 ( 1.0 ) |
128 | TFLM | Reference | RV32GCV | Loop+SLP |
63384332 ( 0.7x ) |
426196 ( 0.961 ) |
134460 ( 1.0 ) |
1024 | TFLM | Reference | RV32GCV | Loop+SLP |
46648207 ( Base ) |
443524 ( Base ) |
134452 ( Base ) |
0 | muRISCV-NN | Scalar | RV32GC | - |
19561414 ( 2.4x ) |
454736 ( 1.025 ) |
134460 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32GCV | Loop+SLP |
14765683 ( 3.2x ) |
454736 ( 1.025 ) |
134460 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32GCV | Loop+SLP |
21023473 ( 2.2x ) |
443094 ( 0.999 ) |
134452 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32GCV | - |
10336293 ( 4.5x ) |
443094 ( 0.999 ) |
134452 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32GCV | - |
Original data
Click here to download the raw files for this benchmark.