Benchmarks 2024 11 21 TFLM GCC Os spike_rv32_min - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)- Spike :
0bc176b3fca43560b9e8586cdbc41cfde073e17a - Spike PK :
7e9b671c0415dfd7b562ac934feb9380075d4aa2
- Spike :
Toolchains
- RISC-V GCC:
- Scalar:
riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0 - Vector:
riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0 - Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
- Scalar:
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Frameworks
-
MLonMCU :
develop -
TFLM :
8eb6b23de4470d6a8da3131650d6a67514dfa130
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -Os, Target: spike_rv32_min )
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
174715201 ( 0.1x ) |
181844 ( 0.863 ) |
36196 ( 1.0 ) |
0 | TFLM | Reference | RV32IM | 0 | - |
174715201 ( 0.1x ) |
181844 ( 0.863 ) |
36196 ( 1.0 ) |
128 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
174715201 ( 0.1x ) |
181844 ( 0.863 ) |
36196 ( 1.0 ) |
256 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
174715201 ( 0.1x ) |
181844 ( 0.863 ) |
36196 ( 1.0 ) |
512 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
174715201 ( 0.1x ) |
181844 ( 0.863 ) |
36196 ( 1.0 ) |
1024 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
174715201 ( 0.1x ) |
181844 ( 0.863 ) |
36196 ( 1.0 ) |
2048 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
174715201 ( 0.1x ) |
181844 ( 0.863 ) |
36196 ( 1.0 ) |
4096 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
16670269 ( Base ) |
210680 ( Base ) |
36204 ( Base ) |
0 | muRISCV-NN | Scalar | RV32IM | 0 | - |
16552929 ( 1.0x ) |
210404 ( 0.999 ) |
36204 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV32IM | 0 | - |
16670269 ( 1.0x ) |
210676 ( 1.0 ) |
36204 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
16670269 ( 1.0x ) |
210676 ( 1.0 ) |
36204 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
16670269 ( 1.0x ) |
210676 ( 1.0 ) |
36204 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
16670269 ( 1.0x ) |
210676 ( 1.0 ) |
36204 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
16670269 ( 1.0x ) |
210676 ( 1.0 ) |
36204 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
16670269 ( 1.0x ) |
210676 ( 1.0 ) |
36204 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
4127624 ( 4.0x ) |
212528 ( 1.009 ) |
36204 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
2859568 ( 5.8x ) |
212528 ( 1.009 ) |
36204 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
2170880 ( 7.7x ) |
212528 ( 1.009 ) |
36204 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
2129168 ( 7.8x ) |
212528 ( 1.009 ) |
36204 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
2129168 ( 7.8x ) |
212528 ( 1.009 ) |
36204 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
2132557 ( 7.8x ) |
212528 ( 1.009 ) |
36204 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
16552929 ( 1.0x ) |
210400 ( 0.999 ) |
36204 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
16552929 ( 1.0x ) |
210400 ( 0.999 ) |
36204 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
16552929 ( 1.0x ) |
210400 ( 0.999 ) |
36204 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
16552929 ( 1.0x ) |
210400 ( 0.999 ) |
36204 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
16552929 ( 1.0x ) |
210400 ( 0.999 ) |
36204 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
16552929 ( 1.0x ) |
210400 ( 0.999 ) |
36204 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
745817708 ( 0.1x ) |
223444 ( 0.921 ) |
68960 ( 1.0 ) |
0 | TFLM | Reference | RV32IM | 0 | - |
745817708 ( 0.1x ) |
223440 ( 0.921 ) |
68960 ( 1.0 ) |
128 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
745817708 ( 0.1x ) |
223440 ( 0.921 ) |
68960 ( 1.0 ) |
256 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
745817708 ( 0.1x ) |
223440 ( 0.921 ) |
68960 ( 1.0 ) |
512 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
745817708 ( 0.1x ) |
223440 ( 0.921 ) |
68960 ( 1.0 ) |
1024 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
745817708 ( 0.1x ) |
223440 ( 0.921 ) |
68960 ( 1.0 ) |
2048 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
745817708 ( 0.1x ) |
223440 ( 0.921 ) |
68960 ( 1.0 ) |
4096 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
81015562 ( Base ) |
242728 ( Base ) |
68952 ( Base ) |
0 | muRISCV-NN | Scalar | RV32IM | 0 | - |
78475039 ( 1.0x ) |
242344 ( 0.998 ) |
68952 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV32IM | 0 | - |
81015562 ( 1.0x ) |
242720 ( 1.0 ) |
68952 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
81015562 ( 1.0x ) |
242720 ( 1.0 ) |
68952 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
81015562 ( 1.0x ) |
242720 ( 1.0 ) |
68952 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
81015562 ( 1.0x ) |
242720 ( 1.0 ) |
68952 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
81015562 ( 1.0x ) |
242720 ( 1.0 ) |
68952 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
81015562 ( 1.0x ) |
242720 ( 1.0 ) |
68952 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
15485294 ( 5.2x ) |
245640 ( 1.012 ) |
68952 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
9798286 ( 8.3x ) |
245640 ( 1.012 ) |
68952 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
7204638 ( 11.2x ) |
245640 ( 1.012 ) |
68952 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
5939118 ( 13.6x ) |
245640 ( 1.012 ) |
68952 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
4997427 ( 16.2x ) |
245640 ( 1.012 ) |
68952 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
4746900 ( 17.1x ) |
245640 ( 1.012 ) |
68952 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
78475039 ( 1.0x ) |
242336 ( 0.998 ) |
68952 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
78475039 ( 1.0x ) |
242336 ( 0.998 ) |
68952 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
78475039 ( 1.0x ) |
242336 ( 0.998 ) |
68952 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
78475039 ( 1.0x ) |
242336 ( 0.998 ) |
68952 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
78475039 ( 1.0x ) |
242336 ( 0.998 ) |
68952 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
78475039 ( 1.0x ) |
242336 ( 0.998 ) |
68952 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
3106381 ( 0.6x ) |
372256 ( 0.981 ) |
19416 ( 1.0 ) |
0 | TFLM | Reference | RV32IM | 0 | - |
3106381 ( 0.6x ) |
372256 ( 0.981 ) |
19416 ( 1.0 ) |
128 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
3106381 ( 0.6x ) |
372256 ( 0.981 ) |
19416 ( 1.0 ) |
256 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
3106381 ( 0.6x ) |
372256 ( 0.981 ) |
19416 ( 1.0 ) |
512 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
3106381 ( 0.6x ) |
372256 ( 0.981 ) |
19416 ( 1.0 ) |
1024 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
3106381 ( 0.6x ) |
372256 ( 0.981 ) |
19416 ( 1.0 ) |
2048 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
3106381 ( 0.6x ) |
372256 ( 0.981 ) |
19416 ( 1.0 ) |
4096 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
1788157 ( Base ) |
379288 ( Base ) |
19416 ( Base ) |
0 | muRISCV-NN | Scalar | RV32IM | 0 | - |
3140766 ( 0.6x ) |
379292 ( 1.0 ) |
19416 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV32IM | 0 | - |
1788157 ( 1.0x ) |
379284 ( 1.0 ) |
19416 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
1788157 ( 1.0x ) |
379284 ( 1.0 ) |
19416 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
1788157 ( 1.0x ) |
379284 ( 1.0 ) |
19416 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
1788157 ( 1.0x ) |
379284 ( 1.0 ) |
19416 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
1788157 ( 1.0x ) |
379284 ( 1.0 ) |
19416 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
1788157 ( 1.0x ) |
379284 ( 1.0 ) |
19416 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
1947966 ( 0.9x ) |
380964 ( 1.004 ) |
19416 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
1829598 ( 1.0x ) |
380964 ( 1.004 ) |
19416 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
1770414 ( 1.0x ) |
380964 ( 1.004 ) |
19416 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
1741158 ( 1.0x ) |
380964 ( 1.004 ) |
19416 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
1737474 ( 1.0x ) |
380964 ( 1.004 ) |
19416 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
1735590 ( 1.0x ) |
380964 ( 1.004 ) |
19416 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
3140766 ( 0.6x ) |
379288 ( 1.0 ) |
19416 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
3140766 ( 0.6x ) |
379288 ( 1.0 ) |
19416 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
3140766 ( 0.6x ) |
379288 ( 1.0 ) |
19416 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
3140766 ( 0.6x ) |
379288 ( 1.0 ) |
19416 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
3140766 ( 0.6x ) |
379288 ( 1.0 ) |
19416 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
3140766 ( 0.6x ) |
379288 ( 1.0 ) |
19416 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
495282732 ( 0.1x ) |
455484 ( 0.94 ) |
134500 ( 1.0 ) |
0 | TFLM | Reference | RV32IM | 0 | - |
495282732 ( 0.1x ) |
455484 ( 0.94 ) |
134500 ( 1.0 ) |
128 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
495282732 ( 0.1x ) |
455484 ( 0.94 ) |
134500 ( 1.0 ) |
256 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
495282732 ( 0.1x ) |
455484 ( 0.94 ) |
134500 ( 1.0 ) |
512 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
495282732 ( 0.1x ) |
455484 ( 0.94 ) |
134500 ( 1.0 ) |
1024 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
495282732 ( 0.1x ) |
455484 ( 0.94 ) |
134500 ( 1.0 ) |
2048 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
495282732 ( 0.1x ) |
455484 ( 0.94 ) |
134500 ( 1.0 ) |
4096 | TFLM | Reference | RV32IM_ZVE64X | 0 | Loop+SLP |
49700105 ( Base ) |
484320 ( Base ) |
134508 ( Base ) |
0 | muRISCV-NN | Scalar | RV32IM | 0 | - |
50304647 ( 1.0x ) |
484044 ( 0.999 ) |
134508 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV32IM | 0 | - |
49700105 ( 1.0x ) |
484316 ( 1.0 ) |
134508 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
49700105 ( 1.0x ) |
484316 ( 1.0 ) |
134508 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
49700105 ( 1.0x ) |
484316 ( 1.0 ) |
134508 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
49700105 ( 1.0x ) |
484316 ( 1.0 ) |
134508 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
49700105 ( 1.0x ) |
484316 ( 1.0 ) |
134508 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
49700105 ( 1.0x ) |
484316 ( 1.0 ) |
134508 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV32IM_ZVE64X | 0 | Loop+SLP |
13494714 ( 3.7x ) |
486168 ( 1.004 ) |
134508 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
10166086 ( 4.9x ) |
486168 ( 1.004 ) |
134508 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
8876964 ( 5.6x ) |
486168 ( 1.004 ) |
134508 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
8372465 ( 5.9x ) |
486168 ( 1.004 ) |
134508 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
8323851 ( 6.0x ) |
486168 ( 1.004 ) |
134508 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
8327240 ( 6.0x ) |
486168 ( 1.004 ) |
134508 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV32IM_ZVE64X | 0 | - |
50304647 ( 1.0x ) |
484040 ( 0.999 ) |
134508 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
50304647 ( 1.0x ) |
484040 ( 0.999 ) |
134508 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
50304647 ( 1.0x ) |
484040 ( 0.999 ) |
134508 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
50304647 ( 1.0x ) |
484040 ( 0.999 ) |
134508 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
50304647 ( 1.0x ) |
484040 ( 0.999 ) |
134508 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
50304647 ( 1.0x ) |
484040 ( 0.999 ) |
134508 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV32IM_ZVE64X | 0 | Loop+SLP |
Original data
Click here to download the raw files for this benchmark.