Benchmarks 2024 11 26 TFLM GCC O3 spike_rv64 - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)- Spike :
eb0a3e2b0a7c57522928be39de95cd9f8c6dc636 - Spike PK :
fix-gcc14-rvv
- Spike :
Toolchains
- RISC-V GCC:
- Scalar:
riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118 - Vector:
riscv32-unknown-elf-gcc (g8b4bb54e6c4) 14.2.1 20241118 - Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
- Scalar:
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Frameworks
-
MLonMCU :
develop -
TFLM :
8eb6b23de4470d6a8da3131650d6a67514dfa130
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -O3, Target: spike_rv64 )
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
56875270.0 ( 0.3x ) |
144396 ( 0.817 ) |
38392 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
36048336.0 ( 0.4x ) |
152228 ( 0.861 ) |
38448 ( 1.001 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
33166398.0 ( 0.5x ) |
153026 ( 0.865 ) |
38448 ( 1.001 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
31808365.0 ( 0.5x ) |
153986 ( 0.871 ) |
38448 ( 1.001 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
31009356.0 ( 0.5x ) |
155118 ( 0.877 ) |
38448 ( 1.001 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
31199422.0 ( 0.5x ) |
156382 ( 0.884 ) |
38448 ( 1.001 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
31038432.0 ( 0.5x ) |
158336 ( 0.895 ) |
38424 ( 1.001 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
15097076.0 ( Base ) |
176846 ( Base ) |
38400 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
15557110.0 ( 1.0x ) |
173798 ( 0.983 ) |
38400 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
8154451.0 ( 1.9x ) |
195232 ( 1.104 ) |
38448 ( 1.001 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
6751660.0 ( 2.2x ) |
200302 ( 1.133 ) |
38456 ( 1.001 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
6067798.0 ( 2.5x ) |
206948 ( 1.17 ) |
38456 ( 1.001 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
5729747.0 ( 2.6x ) |
219174 ( 1.239 ) |
38456 ( 1.001 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
5552158.0 ( 2.7x ) |
251376 ( 1.421 ) |
38456 ( 1.001 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
5548948.0 ( 2.7x ) |
325690 ( 1.842 ) |
38432 ( 1.001 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
7903715.0 ( 1.9x ) |
178516 ( 1.009 ) |
38400 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
5359705.0 ( 2.8x ) |
178516 ( 1.009 ) |
38400 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
3866590.0 ( 3.9x ) |
178516 ( 1.009 ) |
38400 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
3828538.0 ( 3.9x ) |
178516 ( 1.009 ) |
38400 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
3830403.0 ( 3.9x ) |
178516 ( 1.009 ) |
38400 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
3832268.0 ( 3.9x ) |
178516 ( 1.009 ) |
38400 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
7855898.0 ( 1.9x ) |
193134 ( 1.092 ) |
38448 ( 1.001 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
5627083.0 ( 2.7x ) |
197528 ( 1.117 ) |
38456 ( 1.001 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
4612861.0 ( 3.3x ) |
204234 ( 1.155 ) |
38456 ( 1.001 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
3962426.0 ( 3.8x ) |
216456 ( 1.224 ) |
38456 ( 1.001 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
3743430.0 ( 4.0x ) |
248660 ( 1.406 ) |
38456 ( 1.001 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
3752783.0 ( 4.0x ) |
322976 ( 1.826 ) |
38432 ( 1.001 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
171735280.0 ( 0.3x ) |
192190 ( 0.905 ) |
71160 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
70156517.0 ( 0.8x ) |
206156 ( 0.97 ) |
71224 ( 1.001 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
54001363.0 ( 1.0x ) |
207558 ( 0.977 ) |
71232 ( 1.001 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
47176487.0 ( 1.2x ) |
210108 ( 0.989 ) |
71232 ( 1.001 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
44502007.0 ( 1.2x ) |
212014 ( 0.998 ) |
71232 ( 1.001 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
45454299.0 ( 1.2x ) |
214294 ( 1.009 ) |
71224 ( 1.001 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
44984554.0 ( 1.2x ) |
217924 ( 1.026 ) |
71200 ( 1.001 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
54280182.0 ( Base ) |
212432 ( Base ) |
71152 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
71759886.0 ( 0.8x ) |
211752 ( 0.997 ) |
71152 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
17929483.0 ( 3.0x ) |
236998 ( 1.116 ) |
71216 ( 1.001 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
11420958.0 ( 4.8x ) |
237956 ( 1.12 ) |
71216 ( 1.001 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
8216241.0 ( 6.6x ) |
239982 ( 1.13 ) |
71216 ( 1.001 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
6740087.0 ( 8.1x ) |
241618 ( 1.137 ) |
71216 ( 1.001 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
6059054.0 ( 9.0x ) |
243182 ( 1.145 ) |
71216 ( 1.001 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
5727577.0 ( 9.5x ) |
245964 ( 1.158 ) |
71192 ( 1.001 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
31987203.0 ( 1.7x ) |
219842 ( 1.035 ) |
71152 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
19580875.0 ( 2.8x ) |
219842 ( 1.035 ) |
71152 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
13949944.0 ( 3.9x ) |
219842 ( 1.035 ) |
71152 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
11201800.0 ( 4.8x ) |
219842 ( 1.035 ) |
71152 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
9046233.0 ( 6.0x ) |
219842 ( 1.035 ) |
71152 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
8468399.0 ( 6.4x ) |
219842 ( 1.035 ) |
71152 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
29168736.0 ( 1.9x ) |
236368 ( 1.113 ) |
71216 ( 1.001 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
17060274.0 ( 3.2x ) |
237326 ( 1.117 ) |
71216 ( 1.001 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
10954874.0 ( 5.0x ) |
239352 ( 1.127 ) |
71216 ( 1.001 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
8085078.0 ( 6.7x ) |
240988 ( 1.134 ) |
71216 ( 1.001 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
6777768.0 ( 8.0x ) |
242552 ( 1.142 ) |
71216 ( 1.001 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
6135548.0 ( 8.8x ) |
245334 ( 1.155 ) |
71192 ( 1.001 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
2773419.0 ( 0.6x ) |
336470 ( 0.977 ) |
21512 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
1218838.0 ( 1.3x ) |
339158 ( 0.984 ) |
21512 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
822780.0 ( 2.0x ) |
339680 ( 0.986 ) |
21512 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
625346.0 ( 2.6x ) |
339898 ( 0.987 ) |
21512 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
526633.0 ( 3.1x ) |
340338 ( 0.988 ) |
21512 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
477305.0 ( 3.4x ) |
340808 ( 0.989 ) |
21512 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
452624.0 ( 3.6x ) |
341574 ( 0.991 ) |
21512 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
1645338.0 ( Base ) |
344548 ( Base ) |
21512 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
2728482.0 ( 0.6x ) |
344552 ( 1.0 ) |
21512 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
727662.0 ( 2.3x ) |
344882 ( 1.001 ) |
21512 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
559451.0 ( 2.9x ) |
345150 ( 1.002 ) |
21512 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
468976.0 ( 3.5x ) |
345152 ( 1.002 ) |
21512 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
423130.0 ( 3.9x ) |
345370 ( 1.002 ) |
21512 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
387824.0 ( 4.2x ) |
345614 ( 1.003 ) |
21512 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
376540.0 ( 4.4x ) |
346016 ( 1.004 ) |
21512 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
2018575.0 ( 0.8x ) |
348208 ( 1.011 ) |
21512 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1758385.0 ( 0.9x ) |
348208 ( 1.011 ) |
21512 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1628290.0 ( 1.0x ) |
348208 ( 1.011 ) |
21512 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1564054.0 ( 1.1x ) |
348208 ( 1.011 ) |
21512 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1557817.0 ( 1.1x ) |
348208 ( 1.011 ) |
21512 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1555530.0 ( 1.1x ) |
348208 ( 1.011 ) |
21512 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
1224399.0 ( 1.3x ) |
344886 ( 1.001 ) |
21512 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
824793.0 ( 2.0x ) |
345154 ( 1.002 ) |
21512 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
619091.0 ( 2.7x ) |
345156 ( 1.002 ) |
21512 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
515656.0 ( 3.2x ) |
345374 ( 1.002 ) |
21512 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
451575.0 ( 3.6x ) |
345618 ( 1.003 ) |
21512 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
425844.0 ( 3.9x ) |
346020 ( 1.004 ) |
21512 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
146817507.0 ( 0.3x ) |
418038 ( 0.928 ) |
136696 ( 1.0 ) |
0 | TFLM | Reference | RV64GC | 0 | - |
78400305.0 ( 0.6x ) |
425870 ( 0.945 ) |
136752 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
70545982.0 ( 0.6x ) |
426668 ( 0.947 ) |
136752 ( 1.0 ) |
256 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
65840909.0 ( 0.7x ) |
427628 ( 0.949 ) |
136752 ( 1.0 ) |
512 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
64475973.0 ( 0.7x ) |
428760 ( 0.952 ) |
136752 ( 1.0 ) |
1024 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
64064837.0 ( 0.7x ) |
430024 ( 0.955 ) |
136752 ( 1.0 ) |
2048 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
63551760.0 ( 0.7x ) |
431978 ( 0.959 ) |
136728 ( 1.0 ) |
4096 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
45422167.0 ( Base ) |
450488 ( Base ) |
136704 ( Base ) |
0 | muRISCV-NN | Scalar | RV64GC | 0 | - |
46730216.0 ( 1.0x ) |
447440 ( 0.993 ) |
136704 ( 1.0 ) |
0 | muRISCV-NN | Vector (Portable) | RV64GC | 0 | - |
25318168.0 ( 1.8x ) |
468882 ( 1.041 ) |
136752 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
21253820.0 ( 2.1x ) |
473952 ( 1.052 ) |
136760 ( 1.0 ) |
256 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
19284071.0 ( 2.4x ) |
480598 ( 1.067 ) |
136760 ( 1.0 ) |
512 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
18340247.0 ( 2.5x ) |
492808 ( 1.094 ) |
136760 ( 1.0 ) |
1024 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
17951765.0 ( 2.5x ) |
525010 ( 1.165 ) |
136760 ( 1.0 ) |
2048 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
17820868.0 ( 2.5x ) |
599340 ( 1.33 ) |
136736 ( 1.0 ) |
4096 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
25292800.0 ( 1.8x ) |
452158 ( 1.004 ) |
136704 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
18424373.0 ( 2.5x ) |
452158 ( 1.004 ) |
136704 ( 1.0 ) |
256 | muRISCV-NN | Vector | RV64GCV | 0 | - |
15802964.0 ( 2.9x ) |
452158 ( 1.004 ) |
136704 ( 1.0 ) |
512 | muRISCV-NN | Vector | RV64GCV | 0 | - |
14710149.0 ( 3.1x ) |
452158 ( 1.004 ) |
136704 ( 1.0 ) |
1024 | muRISCV-NN | Vector | RV64GCV | 0 | - |
14622985.0 ( 3.1x ) |
452158 ( 1.004 ) |
136704 ( 1.0 ) |
2048 | muRISCV-NN | Vector | RV64GCV | 0 | - |
14624850.0 ( 3.1x ) |
452158 ( 1.004 ) |
136704 ( 1.0 ) |
4096 | muRISCV-NN | Vector | RV64GCV | 0 | - |
24252969.0 ( 1.9x ) |
466784 ( 1.036 ) |
136752 ( 1.0 ) |
128 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
18098949.0 ( 2.5x ) |
471178 ( 1.046 ) |
136760 ( 1.0 ) |
256 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
15214938.0 ( 3.0x ) |
477884 ( 1.061 ) |
136760 ( 1.0 ) |
512 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
13418795.0 ( 3.4x ) |
490090 ( 1.088 ) |
136760 ( 1.0 ) |
1024 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
13046792.0 ( 3.5x ) |
522294 ( 1.159 ) |
136760 ( 1.0 ) |
2048 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
12907388.0 ( 3.5x ) |
596626 ( 1.324 ) |
136736 ( 1.0 ) |
4096 | muRISCV-NN | Vector (Portable) | RV64GCV | 0 | Loop+SLP |
Original data
Click here to download the raw files for this benchmark.