Benchmarks 2024 07 12 TFLM GCC Os spike_rv64 - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)
Toolchains
- RISC-V GCC:
- Scalar: TODO: version & url
- Vector: TODO: version & url
- Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Package Versions
-
MLonMCU : main
-
TFLM : main
-
Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
-
Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tflm, Backend: tflmi, Toolchain: gcc, Flags: -Os)
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
165492824 ( 0.1x ) |
129999 ( 0.873 ) |
38440 ( 1.0 ) |
128 | TFLM | Reference | RV64GC | False | - |
165492824 ( 0.1x ) |
130001 ( 0.873 ) |
38440 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | False | Loop+SLP |
18671271 ( Base ) |
148942 ( Base ) |
38448 ( Base ) |
128 | muRISCV-NN | Scalar | RV64GC | False | - |
18660337 ( 1.0x ) |
149766 ( 1.006 ) |
38448 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | False | Loop+SLP |
4180366 ( 4.5x ) |
149752 ( 1.005 ) |
38448 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | False | - |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
696993700 ( 0.1x ) |
170559 ( 0.935 ) |
71208 ( 1.0 ) |
128 | TFLM | Reference | RV64GC | False | - |
696993700 ( 0.1x ) |
170553 ( 0.935 ) |
71208 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | False | Loop+SLP |
88730172 ( Base ) |
182496 ( Base ) |
71192 ( Base ) |
128 | muRISCV-NN | Scalar | RV64GC | False | - |
88730480 ( 1.0x ) |
183048 ( 1.003 ) |
71192 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | False | Loop+SLP |
15625959 ( 5.7x ) |
184270 ( 1.01 ) |
71192 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | False | - |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
3877122 ( 0.5x ) |
330846 ( 0.99 ) |
21528 ( 1.0 ) |
128 | TFLM | Reference | RV64GC | False | - |
3877122 ( 0.5x ) |
330838 ( 0.99 ) |
21528 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | False | Loop+SLP |
1948153 ( Base ) |
334342 ( Base ) |
21528 ( Base ) |
128 | muRISCV-NN | Scalar | RV64GC | False | - |
1948206 ( 1.0x ) |
334360 ( 1.0 ) |
21528 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | False | Loop+SLP |
589615 ( 3.3x ) |
335070 ( 1.002 ) |
21528 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | False | - |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|
466676303 ( 0.1x ) |
403713 ( 0.955 ) |
136760 ( 1.0 ) |
128 | TFLM | Reference | RV64GC | False | - |
466676303 ( 0.1x ) |
403715 ( 0.955 ) |
136760 ( 1.0 ) |
128 | TFLM | Reference | RV64GCV | False | Loop+SLP |
55403045 ( Base ) |
422656 ( Base ) |
136768 ( Base ) |
128 | muRISCV-NN | Scalar | RV64GC | False | - |
55404653 ( 1.0x ) |
423480 ( 1.002 ) |
136768 ( 1.0 ) |
128 | muRISCV-NN | Scalar | RV64GCV | False | Loop+SLP |
13885455 ( 4.0x ) |
423466 ( 1.002 ) |
136768 ( 1.0 ) |
128 | muRISCV-NN | Vector | RV64GCV | False | - |
Original data
Click here to download the raw files for this benchmark.