Benchmarks 2024 07 12 TVM GCC Os spike_rv64 - tum-ei-eda/muriscv-nn GitHub Wiki
Setup
Simulator
- Spike (
riscv-isa-sim) (ISS, CPI=1)
Toolchains
- RISC-V GCC:
- Scalar: TODO: version & url
- Vector: TODO: version & url
- Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
Models
-
MLPerfTiny Benchmark
-
TODO: others!
Package Versions
-
MLonMCU : main
-
TVM : Nightly Pre-Build
-
Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
-
Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2
Miscellaneous
- Used
-Osflag for compilation. - Benchmarks generated using MLonMCU deployment tool with minimal efforts.
- Memory metrics are reported in Bytes
Results (Framework: tvm, Backend: tvmaot, Toolchain: gcc, Flags: -Os)
Audio Wake Words (aww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Layout | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|---|
28221379 ( 0.7x ) |
103616 ( 1.268 ) |
61392 ( 2.912 ) |
128 | NCHW | TVM | Fallback | RV64GC | False | - |
31790880 ( 0.6x ) |
97562 ( 1.194 ) |
61392 ( 2.912 ) |
128 | NHWC | TVM | Fallback | RV64GC | False | - |
14059913 ( 1.3x ) |
97110 ( 1.188 ) |
53224 ( 2.525 ) |
128 | NCHW | TVM | Autotuned | RV64GC | False | - |
31786538 ( 0.6x ) |
97742 ( 1.196 ) |
61392 ( 2.912 ) |
128 | NHWC | TVM | Autotuned | RV64GC | False | - |
28254733 ( 0.7x ) |
103654 ( 1.268 ) |
61392 ( 2.912 ) |
128 | NCHW | TVM | Fallback | RV64GCV | False | Loop+SLP |
31790876 ( 0.6x ) |
97562 ( 1.194 ) |
61392 ( 2.912 ) |
128 | NHWC | TVM | Fallback | RV64GCV | False | Loop+SLP |
14059909 ( 1.3x ) |
97110 ( 1.188 ) |
53224 ( 2.525 ) |
128 | NCHW | TVM | Autotuned | RV64GCV | False | Loop+SLP |
31786534 ( 0.6x ) |
97742 ( 1.196 ) |
61392 ( 2.912 ) |
128 | NHWC | TVM | Autotuned | RV64GCV | False | Loop+SLP |
18611278 ( Base ) |
81728 ( Base ) |
21080 ( Base ) |
128 | NHWC | muRISCV-NN | Scalar | RV64GC | False | - |
18611278 ( 1.0x ) |
81730 ( 1.0 ) |
21080 ( 1.0 ) |
128 | NHWC | muRISCV-NN | Scalar | RV64GCV | False | Loop+SLP |
5960146 ( 3.1x ) |
82442 ( 1.009 ) |
25544 ( 1.212 ) |
128 | NHWC | muRISCV-NN | Vector | RV64GCV | False | - |
Image Classification (resnet)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Layout | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|---|
130951668 ( 0.7x ) |
212978 ( 1.637 ) |
110304 ( 1.922 ) |
128 | NCHW | TVM | Fallback | RV64GC | False | - |
129273687 ( 0.7x ) |
203592 ( 1.564 ) |
110304 ( 1.922 ) |
128 | NHWC | TVM | Fallback | RV64GC | False | - |
55911290 ( 1.6x ) |
207274 ( 1.593 ) |
94120 ( 1.64 ) |
128 | NCHW | TVM | Autotuned | RV64GC | False | - |
129269969 ( 0.7x ) |
203742 ( 1.566 ) |
110304 ( 1.922 ) |
128 | NHWC | TVM | Autotuned | RV64GC | False | - |
130924269 ( 0.7x ) |
212966 ( 1.636 ) |
110304 ( 1.922 ) |
128 | NCHW | TVM | Fallback | RV64GCV | False | Loop+SLP |
129273681 ( 0.7x ) |
203594 ( 1.564 ) |
110304 ( 1.922 ) |
128 | NHWC | TVM | Fallback | RV64GCV | False | Loop+SLP |
55911284 ( 1.6x ) |
207280 ( 1.593 ) |
94120 ( 1.64 ) |
128 | NCHW | TVM | Autotuned | RV64GCV | False | Loop+SLP |
129269963 ( 0.7x ) |
203744 ( 1.566 ) |
110304 ( 1.922 ) |
128 | NHWC | TVM | Autotuned | RV64GCV | False | Loop+SLP |
88726729 ( Base ) |
130142 ( Base ) |
57384 ( Base ) |
128 | NHWC | muRISCV-NN | Scalar | RV64GC | False | - |
88726729 ( 1.0x ) |
130142 ( 1.0 ) |
57384 ( 1.0 ) |
128 | NHWC | muRISCV-NN | Scalar | RV64GCV | False | Loop+SLP |
15638544 ( 5.7x ) |
131354 ( 1.009 ) |
57384 ( 1.0 ) |
128 | NHWC | muRISCV-NN | Vector | RV64GCV | False | - |
Anomaly Detection (toycar)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Layout | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|---|
3132537 ( 0.6x ) |
576066 ( 1.853 ) |
7440 ( 1.12 ) |
128 | NCHW | TVM | Fallback | RV64GC | False | - |
3132537 ( 0.6x ) |
576066 ( 1.853 ) |
7440 ( 1.12 ) |
128 | NHWC | TVM | Fallback | RV64GC | False | - |
2580611 ( 0.8x ) |
612998 ( 1.972 ) |
8752 ( 1.318 ) |
128 | NCHW | TVM | Autotuned | RV64GC | False | - |
2580611 ( 0.8x ) |
612998 ( 1.972 ) |
8752 ( 1.318 ) |
128 | NHWC | TVM | Autotuned | RV64GC | False | - |
3132537 ( 0.6x ) |
576070 ( 1.853 ) |
7440 ( 1.12 ) |
128 | NCHW | TVM | Fallback | RV64GCV | False | Loop+SLP |
3132537 ( 0.6x ) |
576070 ( 1.853 ) |
7440 ( 1.12 ) |
128 | NHWC | TVM | Fallback | RV64GCV | False | Loop+SLP |
2580611 ( 0.8x ) |
612998 ( 1.972 ) |
8752 ( 1.318 ) |
128 | NCHW | TVM | Autotuned | RV64GCV | False | Loop+SLP |
2580611 ( 0.8x ) |
612998 ( 1.972 ) |
8752 ( 1.318 ) |
128 | NHWC | TVM | Autotuned | RV64GCV | False | Loop+SLP |
1989922 ( Base ) |
310838 ( Base ) |
6640 ( Base ) |
128 | NHWC | muRISCV-NN | Scalar | RV64GC | False | - |
1989922 ( 1.0x ) |
310838 ( 1.0 ) |
6640 ( 1.0 ) |
128 | NHWC | muRISCV-NN | Scalar | RV64GCV | False | Loop+SLP |
626252 ( 3.2x ) |
311126 ( 1.001 ) |
6640 ( 1.0 ) |
128 | NHWC | muRISCV-NN | Vector | RV64GCV | False | - |
Visual Wake Words (vww)
| Cycles (Speedup) | Total ROM (rel.) | Total RAM (rel.) | VLEN | Layout | Kernels | Mode | Arch | Unroll | Auto-Vectorization |
|---|---|---|---|---|---|---|---|---|---|
82293698 ( 0.7x ) |
536764 ( 1.706 ) |
182912 ( 2.09 ) |
128 | NCHW | TVM | Fallback | RV64GC | False | - |
89548654 ( 0.6x ) |
516256 ( 1.641 ) |
182912 ( 2.09 ) |
128 | NHWC | TVM | Fallback | RV64GC | False | - |
48170963 ( 1.2x ) |
518604 ( 1.648 ) |
182912 ( 2.09 ) |
128 | NCHW | TVM | Autotuned | RV64GC | False | - |
89548657 ( 0.6x ) |
516268 ( 1.641 ) |
182912 ( 2.09 ) |
128 | NHWC | TVM | Autotuned | RV64GC | False | - |
82344705 ( 0.7x ) |
536804 ( 1.706 ) |
182912 ( 2.09 ) |
128 | NCHW | TVM | Fallback | RV64GCV | False | Loop+SLP |
89548654 ( 0.6x ) |
516256 ( 1.641 ) |
182912 ( 2.09 ) |
128 | NHWC | TVM | Fallback | RV64GCV | False | Loop+SLP |
48168095 ( 1.2x ) |
518596 ( 1.648 ) |
182912 ( 2.09 ) |
128 | NCHW | TVM | Autotuned | RV64GCV | False | Loop+SLP |
89548657 ( 0.6x ) |
516268 ( 1.641 ) |
182912 ( 2.09 ) |
128 | NHWC | TVM | Autotuned | RV64GCV | False | Loop+SLP |
55502755 ( Base ) |
314632 ( Base ) |
87528 ( Base ) |
128 | NHWC | muRISCV-NN | Scalar | RV64GC | False | - |
55502755 ( 1.0x ) |
314634 ( 1.0 ) |
87528 ( 1.0 ) |
128 | NHWC | muRISCV-NN | Scalar | RV64GCV | False | Loop+SLP |
13988725 ( 4.0x ) |
315612 ( 1.003 ) |
87528 ( 1.0 ) |
128 | NHWC | muRISCV-NN | Vector | RV64GCV | False | - |
Original data
Click here to download the raw files for this benchmark.