Benchmarks 2024 07 12 TFLM LLVM Os spike_rv64 - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)
Toolchains
- LLVM/Clang:
- TODO: Version
- Linker: lld (TODO)
- RISC-V GCC for Headers, libc,...
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Package Versions
-
MLonMCU : main
-
TFLM : main
-
Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
-
Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: llvm, Flags: -Os)
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
39726561 ( 0.4x ) |
144344 ( 0.875 ) |
38368 ( 1.0 ) |
128 | TFLM | Reference | RV64GC | 0 | - |
31995059 ( 0.5x ) |
149314 ( 0.905 ) |
38376 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
14707968 ( Base ) |
164938 ( Base ) |
38368 ( Base ) |
128 | muRISCV-NN | Scalar | RV64GC | 0 | - |
5814997 ( 2.5x ) |
172038 ( 1.043 ) |
38376 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
4075836 ( 3.6x ) |
165578 ( 1.004 ) |
38368 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
121121849 ( 0.5x ) |
185832 ( 0.935 ) |
71136 ( 1.0 ) |
128 | TFLM | Reference | RV64GC | 0 | - |
57765532 ( 1.0x ) |
191734 ( 0.965 ) |
71144 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
55740377 ( Base ) |
198762 ( Base ) |
71136 ( Base ) |
128 | muRISCV-NN | Scalar | RV64GC | 0 | - |
25745638 ( 2.2x ) |
206928 ( 1.041 ) |
71144 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
15253027 ( 3.7x ) |
200146 ( 1.007 ) |
71136 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
2786519 ( 0.6x ) |
339238 ( 0.988 ) |
21472 ( 1.0 ) |
128 | TFLM | Reference | RV64GC | 0 | - |
957684 ( 1.7x ) |
341466 ( 0.994 ) |
21472 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
1638030 ( Base ) |
343362 ( Base ) |
21472 ( Base ) |
128 | muRISCV-NN | Scalar | RV64GC | 0 | - |
597658 ( 2.7x ) |
346358 ( 1.009 ) |
21472 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
585725 ( 2.8x ) |
344156 ( 1.002 ) |
21472 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
104299852 ( 0.4x ) |
418098 ( 0.953 ) |
136688 ( 1.0 ) |
128 | TFLM | Reference | RV64GC | 0 | - |
67381214 ( 0.7x ) |
423006 ( 0.964 ) |
136696 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | 0 | Loop+SLP |
43982589 ( Base ) |
438708 ( Base ) |
136688 ( Base ) |
128 | muRISCV-NN | Scalar | RV64GC | 0 | - |
18364049 ( 2.4x ) |
445746 ( 1.016 ) |
136696 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | 0 | Loop+SLP |
13378250 ( 3.3x ) |
439348 ( 1.001 ) |
136688 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | 0 | - |
Original data
Click here to download the raw files for this benchmark.