Benchmarks 2024 02 20 TFLM GCC - tum-ei-eda/muriscv-nn GitHub Wiki

Setup

Simulator

Toolchains

Models

Package Versions

  • MLonMCU : main

  • TFLM : main

  • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a

  • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Results (Framework: tflm, Backend: tflmi, Toolchain: gcc)

Audio Wake Words (aww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Auto-Vectorization
174698646 ( 0.1x ) 132407 ( 0.879 ) 36204 ( 1.0 ) 0 TFLM Reference RV32GC -
174698646 ( 0.1x ) 132413 ( 0.879 ) 36204 ( 1.0 ) 128 TFLM Reference RV32GCV Loop+SLP
174698646 ( 0.1x ) 132413 ( 0.879 ) 36204 ( 1.0 ) 1024 TFLM Reference RV32GCV Loop+SLP
157549999 ( 0.1x ) 144774 ( 0.961 ) 36148 ( 0.998 ) 0 TFLM Reference RV32GCP -
16643414 ( Base ) 150578 ( Base ) 36212 ( Base ) 0 muRISCV-NN Scalar RV32GC -
16643414 ( 1.0x ) 150584 ( 1.0 ) 36212 ( 1.0 ) 128 muRISCV-NN Scalar RV32GCV Loop+SLP
16643414 ( 1.0x ) 150584 ( 1.0 ) 36212 ( 1.0 ) 1024 muRISCV-NN Scalar RV32GCV Loop+SLP
6995715 ( 2.4x ) 151102 ( 1.003 ) 36212 ( 1.0 ) 128 muRISCV-NN Vector RV32GCV -
2565027 ( 6.5x ) 151102 ( 1.003 ) 36212 ( 1.0 ) 1024 muRISCV-NN Vector RV32GCV -
13496950 ( 1.2x ) 162164 ( 1.077 ) 36156 ( 0.998 ) 0 muRISCV-NN Scalar RV32GCP -
15933233 ( 1.0x ) 164564 ( 1.093 ) 36156 ( 0.998 ) 0 muRISCV-NN Packed RV32GCP -

Image Classification (resnet)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Auto-Vectorization
745801113 ( 0.1x ) 172997 ( 0.935 ) 68968 ( 1.0 ) 0 TFLM Reference RV32GC -
745801113 ( 0.1x ) 173013 ( 0.935 ) 68968 ( 1.0 ) 128 TFLM Reference RV32GCV Loop+SLP
745801113 ( 0.1x ) 173013 ( 0.935 ) 68968 ( 1.0 ) 1024 TFLM Reference RV32GCV Loop+SLP
697912970 ( 0.1x ) 185266 ( 1.001 ) 68912 ( 0.999 ) 0 TFLM Reference RV32GCP -
80992847 ( Base ) 185078 ( Base ) 68960 ( Base ) 0 muRISCV-NN Scalar RV32GC -
80992847 ( 1.0x ) 185110 ( 1.0 ) 68960 ( 1.0 ) 128 muRISCV-NN Scalar RV32GCV Loop+SLP
80992847 ( 1.0x ) 185110 ( 1.0 ) 68960 ( 1.0 ) 1024 muRISCV-NN Scalar RV32GCV Loop+SLP
29958029 ( 2.7x ) 186434 ( 1.007 ) 68960 ( 1.0 ) 128 muRISCV-NN Vector RV32GCV -
8345189 ( 9.7x ) 186434 ( 1.007 ) 68960 ( 1.0 ) 1024 muRISCV-NN Vector RV32GCV -
62969349 ( 1.3x ) 196768 ( 1.063 ) 68904 ( 0.999 ) 0 muRISCV-NN Scalar RV32GCP -
68428919 ( 1.2x ) 199762 ( 1.079 ) 68904 ( 0.999 ) 0 muRISCV-NN Packed RV32GCP -

Anomaly Detection (toycar)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Auto-Vectorization
3094956 ( 0.6x ) 333908 ( 0.989 ) 19432 ( 1.0 ) 0 TFLM Reference RV32GC -
3094956 ( 0.6x ) 333914 ( 0.989 ) 19432 ( 1.0 ) 128 TFLM Reference RV32GCV Loop+SLP
3094956 ( 0.6x ) 333914 ( 0.989 ) 19432 ( 1.0 ) 1024 TFLM Reference RV32GCV Loop+SLP
3097895 ( 0.6x ) 346264 ( 1.026 ) 19380 ( 0.997 ) 0 TFLM Reference RV32GCP -
1969300 ( Base ) 337550 ( Base ) 19432 ( Base ) 0 muRISCV-NN Scalar RV32GC -
1969300 ( 1.0x ) 337556 ( 1.0 ) 19432 ( 1.0 ) 128 muRISCV-NN Scalar RV32GCV Loop+SLP
1969300 ( 1.0x ) 337556 ( 1.0 ) 19432 ( 1.0 ) 1024 muRISCV-NN Scalar RV32GCV Loop+SLP
608151 ( 3.2x ) 337804 ( 1.001 ) 19432 ( 1.0 ) 128 muRISCV-NN Vector RV32GCV -
428279 ( 4.6x ) 337804 ( 1.001 ) 19432 ( 1.0 ) 1024 muRISCV-NN Vector RV32GCV -
1873278 ( 1.1x ) 349712 ( 1.036 ) 19380 ( 0.997 ) 0 muRISCV-NN Scalar RV32GCP -
942627 ( 2.1x ) 351430 ( 1.041 ) 19380 ( 0.997 ) 0 muRISCV-NN Packed RV32GCP -

Visual Wake Words (vww)

Cycles (Speedup) Total ROM (rel.) Total RAM (rel.) VLEN Kernels Mode Arch Auto-Vectorization
495266208 ( 0.1x ) 406111 ( 0.957 ) 134520 ( 1.0 ) 0 TFLM Reference RV32GC -
495266207 ( 0.1x ) 406117 ( 0.957 ) 134520 ( 1.0 ) 128 TFLM Reference RV32GCV Loop+SLP
495266207 ( 0.1x ) 406117 ( 0.957 ) 134520 ( 1.0 ) 1024 TFLM Reference RV32GCV Loop+SLP
445892345 ( 0.1x ) 418478 ( 0.986 ) 134464 ( 1.0 ) 0 TFLM Reference RV32GCP -
49673130 ( Base ) 424282 ( Base ) 134528 ( Base ) 0 muRISCV-NN Scalar RV32GC -
49673130 ( 1.0x ) 424288 ( 1.0 ) 134528 ( 1.0 ) 128 muRISCV-NN Scalar RV32GCV Loop+SLP
49673130 ( 1.0x ) 424288 ( 1.0 ) 134528 ( 1.0 ) 1024 muRISCV-NN Scalar RV32GCV Loop+SLP
21934482 ( 2.3x ) 424806 ( 1.001 ) 134528 ( 1.0 ) 128 muRISCV-NN Vector RV32GCV -
10505033 ( 4.7x ) 424806 ( 1.001 ) 134528 ( 1.0 ) 1024 muRISCV-NN Vector RV32GCV -
40746534 ( 1.2x ) 435868 ( 1.027 ) 134472 ( 1.0 ) 0 muRISCV-NN Scalar RV32GCP -
49175307 ( 1.0x ) 438268 ( 1.033 ) 134472 ( 1.0 ) 0 muRISCV-NN Packed RV32GCP -

Original data

Click here to download the raw files for this benchmark.