Benchmarks 2024 11 21 TFLM GCC Os spike_rv64 - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)- Spike :
0bc176b3fca43560b9e8586cdbc41cfde073e17a - Spike PK :
7e9b671c0415dfd7b562ac934feb9380075d4aa2
- Spike :
Toolchains
- RISC-V GCC:
- Scalar:
riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0 - Vector:
riscv32-unknown-elf-gcc (gc891d8dc23e) 13.2.0 - Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
- Scalar:
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Frameworks
-
MLonMCU :
develop -
TFLM :
8eb6b23de4470d6a8da3131650d6a67514dfa130
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -Os, Target: spike_rv64 )
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
165487704 ( 0.1x ) |
129074 ( 0.863 ) |
38376 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
165487704 ( 0.1x ) |
129076 ( 0.863 ) |
38376 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
165487704 ( 0.1x ) |
129076 ( 0.863 ) |
38376 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
165487704 ( 0.1x ) |
129076 ( 0.863 ) |
38376 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
165487704 ( 0.1x ) |
129076 ( 0.863 ) |
38376 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
165487704 ( 0.1x ) |
129076 ( 0.863 ) |
38376 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
165487704 ( 0.1x ) |
129076 ( 0.863 ) |
38376 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
18655014 ( Base ) |
149502 ( Base ) |
38384 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
18613814 ( 1.0x ) |
149242 ( 0.998 ) |
38384 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
18665491 ( 1.0x ) |
150340 ( 1.006 ) |
38384 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
18665491 ( 1.0x ) |
150340 ( 1.006 ) |
38384 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
18665491 ( 1.0x ) |
150340 ( 1.006 ) |
38384 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
18665491 ( 1.0x ) |
150340 ( 1.006 ) |
38384 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
18665491 ( 1.0x ) |
150340 ( 1.006 ) |
38384 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
18665491 ( 1.0x ) |
150340 ( 1.006 ) |
38384 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
4190488 ( 4.5x ) |
151002 ( 1.01 ) |
38384 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2919968 ( 6.4x ) |
151002 ( 1.01 ) |
38384 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2230048 ( 8.4x ) |
151002 ( 1.01 ) |
38384 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2187720 ( 8.5x ) |
151002 ( 1.01 ) |
38384 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2187720 ( 8.5x ) |
151002 ( 1.01 ) |
38384 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2189585 ( 8.5x ) |
151002 ( 1.01 ) |
38384 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
18614047 ( 1.0x ) |
150080 ( 1.004 ) |
38384 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
18614047 ( 1.0x ) |
150080 ( 1.004 ) |
38384 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
18614047 ( 1.0x ) |
150080 ( 1.004 ) |
38384 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
18614047 ( 1.0x ) |
150080 ( 1.004 ) |
38384 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
18614047 ( 1.0x ) |
150080 ( 1.004 ) |
38384 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
18614047 ( 1.0x ) |
150080 ( 1.004 ) |
38384 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
696989466 ( 0.1x ) |
169550 ( 0.927 ) |
71144 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
696989450 ( 0.1x ) |
169544 ( 0.927 ) |
71144 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
696989450 ( 0.1x ) |
169544 ( 0.927 ) |
71144 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
696989450 ( 0.1x ) |
169544 ( 0.927 ) |
71144 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
696989450 ( 0.1x ) |
169544 ( 0.927 ) |
71144 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
696989450 ( 0.1x ) |
169544 ( 0.927 ) |
71144 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
696989450 ( 0.1x ) |
169544 ( 0.927 ) |
71144 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
88720235 ( Base ) |
182956 ( Base ) |
71128 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
78425875 ( 1.1x ) |
182700 ( 0.999 ) |
71128 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
88715346 ( 1.0x ) |
183522 ( 1.003 ) |
71128 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
88715346 ( 1.0x ) |
183522 ( 1.003 ) |
71128 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
88715346 ( 1.0x ) |
183522 ( 1.003 ) |
71128 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
88715346 ( 1.0x ) |
183522 ( 1.003 ) |
71128 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
88715346 ( 1.0x ) |
183522 ( 1.003 ) |
71128 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
88715346 ( 1.0x ) |
183522 ( 1.003 ) |
71128 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
15636328 ( 5.7x ) |
185420 ( 1.013 ) |
71128 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
9955144 ( 8.9x ) |
185420 ( 1.013 ) |
71128 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
7364408 ( 12.0x ) |
185420 ( 1.013 ) |
71128 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
6102209 ( 14.5x ) |
185420 ( 1.013 ) |
71128 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
5157857 ( 17.2x ) |
185420 ( 1.013 ) |
71128 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
4906170 ( 18.1x ) |
185420 ( 1.013 ) |
71128 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
78415864 ( 1.1x ) |
183266 ( 1.002 ) |
71128 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
78415864 ( 1.1x ) |
183266 ( 1.002 ) |
71128 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
78415864 ( 1.1x ) |
183266 ( 1.002 ) |
71128 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
78415864 ( 1.1x ) |
183266 ( 1.002 ) |
71128 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
78415864 ( 1.1x ) |
183266 ( 1.002 ) |
71128 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
78415864 ( 1.1x ) |
183266 ( 1.002 ) |
71128 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
3872016 ( 0.5x ) |
329922 ( 0.985 ) |
21504 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
3872016 ( 0.5x ) |
329914 ( 0.985 ) |
21504 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
3872016 ( 0.5x ) |
329914 ( 0.985 ) |
21504 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
3872016 ( 0.5x ) |
329914 ( 0.985 ) |
21504 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
3872016 ( 0.5x ) |
329914 ( 0.985 ) |
21504 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
3872016 ( 0.5x ) |
329914 ( 0.985 ) |
21504 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
3872016 ( 0.5x ) |
329914 ( 0.985 ) |
21504 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
1948441 ( Base ) |
334778 ( Base ) |
21504 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
3564896 ( 0.5x ) |
334782 ( 1.0 ) |
21504 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
1948531 ( 1.0x ) |
334810 ( 1.0 ) |
21504 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
1948531 ( 1.0x ) |
334810 ( 1.0 ) |
21504 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
1948531 ( 1.0x ) |
334810 ( 1.0 ) |
21504 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
1948531 ( 1.0x ) |
334810 ( 1.0 ) |
21504 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
1948531 ( 1.0x ) |
334810 ( 1.0 ) |
21504 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
1948531 ( 1.0x ) |
334810 ( 1.0 ) |
21504 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
2201640 ( 0.9x ) |
336194 ( 1.004 ) |
21504 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2083272 ( 0.9x ) |
336194 ( 1.004 ) |
21504 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
2024088 ( 1.0x ) |
336194 ( 1.004 ) |
21504 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1994832 ( 1.0x ) |
336194 ( 1.004 ) |
21504 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1991148 ( 1.0x ) |
336194 ( 1.004 ) |
21504 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1989264 ( 1.0x ) |
336194 ( 1.004 ) |
21504 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
3564986 ( 0.5x ) |
334814 ( 1.0 ) |
21504 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
3564986 ( 0.5x ) |
334814 ( 1.0 ) |
21504 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
3564986 ( 0.5x ) |
334814 ( 1.0 ) |
21504 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
3564986 ( 0.5x ) |
334814 ( 1.0 ) |
21504 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
3564986 ( 0.5x ) |
334814 ( 1.0 ) |
21504 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
3564986 ( 0.5x ) |
334814 ( 1.0 ) |
21504 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
466677307 ( 0.1x ) |
402716 ( 0.952 ) |
136680 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
466677307 ( 0.1x ) |
402718 ( 0.952 ) |
136680 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
466677307 ( 0.1x ) |
402718 ( 0.952 ) |
136680 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
466677307 ( 0.1x ) |
402718 ( 0.952 ) |
136680 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
466677307 ( 0.1x ) |
402718 ( 0.952 ) |
136680 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
466677307 ( 0.1x ) |
402718 ( 0.952 ) |
136680 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
466677307 ( 0.1x ) |
402718 ( 0.952 ) |
136680 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
55398883 ( Base ) |
423144 ( Base ) |
136688 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
56360935 ( 1.0x ) |
422884 ( 0.999 ) |
136688 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
55403599 ( 1.0x ) |
423982 ( 1.002 ) |
136688 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
55403599 ( 1.0x ) |
423982 ( 1.002 ) |
136688 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
55403599 ( 1.0x ) |
423982 ( 1.002 ) |
136688 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
55403599 ( 1.0x ) |
423982 ( 1.002 ) |
136688 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
55403599 ( 1.0x ) |
423982 ( 1.002 ) |
136688 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
55403599 ( 1.0x ) |
423982 ( 1.002 ) |
136688 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13887867 ( 4.0x ) |
424644 ( 1.004 ) |
136688 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
10553803 ( 5.2x ) |
424644 ( 1.004 ) |
136688 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
9262323 ( 6.0x ) |
424644 ( 1.004 ) |
136688 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8757095 ( 6.3x ) |
424644 ( 1.004 ) |
136688 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8708229 ( 6.4x ) |
424644 ( 1.004 ) |
136688 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8710094 ( 6.4x ) |
424644 ( 1.004 ) |
136688 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
56366747 ( 1.0x ) |
423722 ( 1.001 ) |
136688 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
56366747 ( 1.0x ) |
423722 ( 1.001 ) |
136688 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
56366747 ( 1.0x ) |
423722 ( 1.001 ) |
136688 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
56366747 ( 1.0x ) |
423722 ( 1.001 ) |
136688 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
56366747 ( 1.0x ) |
423722 ( 1.001 ) |
136688 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
56366747 ( 1.0x ) |
423722 ( 1.001 ) |
136688 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Original data
Click here to download the raw files for this benchmark.