Benchmarks 2023 04 19 - tum-ei-eda/muriscv-nn GitHub Wiki

Setup

Simulator

Toolchains

Models

Package Versions

  • MLonMCU : 2ab749d3747c04f3b01973ef955bda5be0964f0f
  • TFLM : f050eec7e32a0895f7658db21a4bdbd0975087a5
  • TVM : Nightly Pre-Build
  • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
  • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Frameworks

Benchmarks exist for integration with TensorFlow Lite for Microcontrollers (TFLM) and TVM.

TFLite for Microcontrollers

Comparing reference kernel implementations (Default) with cmsis-nn optimized kernels and linkage with libmuriscvnn.a.

Audio Wake Words (aww)

Cycles Total ROM Total RAM ROM read-only ROM code VLEN Kernels Extensions
153529267 144687 36192 63653 79826 0 Default
15509829 158148 36200 63656 93284 0 muRISCV-NN
9026170 157944 36208 63652 93088 64 muRISCV-NN VEXT
2135130 157944 36208 63652 93088 1024 muRISCV-NN VEXT
7191699 201848 36344 70740 126112 0 muRISCV-NN PEXT

Visual Wake Words (vww)

Cycles Total ROM Total RAM ROM read-only ROM code VLEN Kernels Extensions
433523791 418411 134508 337293 79910 0 Default
46510032 431872 134516 337296 93368 0 muRISCV-NN
27212603 431668 134524 337292 93172 64 muRISCV-NN VEXT
8775554 431668 134524 337292 93172 1024 muRISCV-NN VEXT
23271779 475564 134660 344380 126188 0 muRISCV-NN PEXT

Image Classification (resnet)

Cycles Total ROM Total RAM ROM read-only ROM code VLEN Kernels Extensions
687851073 184423 68956 102485 80730 0 Default
62519499 192280 68948 102488 88584 0 muRISCV-NN
37856218 192866 68956 102500 89162 64 muRISCV-NN VEXT
6058820 192866 68956 102500 89162 1024 muRISCV-NN VEXT
28124010 238944 69092 109572 124376 0 muRISCV-NN PEXT

Anomaly Detection (toycar)

Cycles Total ROM Total RAM ROM read-only ROM code VLEN Kernels Extensions
3434813 347340 19392 280872 65260 0 Default
1920500 349928 19392 280872 67848 0 muRISCV-NN
729214 349838 19396 280868 67766 64 muRISCV-NN VEXT
538790 349838 19396 280868 67766 1024 muRISCV-NN VEXT
788434 394168 19536 287956 101216 0 muRISCV-NN PEXT

TVM

Using CMSIS-NN BYOC feature and linkage with libmuriscvnn.a.

Extension mapping:

  • VEXT -> cortex-m55
  • PEXT -> cortex-m7

Used data/kernel layouts:

  • Untuned: NHWC (TFLite default)
  • Tuned: NCHW (Converted using TVM)

Audio Wake Words (aww)

Cycles Total ROM Total RAM ROM read-only ROM code VLEN Layout Kernels Extensions
31435272 117272 59448 56212 61044 0 NCHW Default
29580732 111594 59448 56212 55366 0 NHWC Default
15699726 110750 51276 56212 54522 0 NCHW Autotuned
29576111 111724 59448 56212 55496 0 NHWC Autotuned
15355980 99626 19140 36644 62966 0 muRISCV-NN
10091110 99576 23604 36644 62916 64 muRISCV-NN VEXT
3987654 99576 23604 36644 62916 1024 muRISCV-NN VEXT
7843235 106562 20292 36400 70086 0 muRISCV-NN PEXT

Visual Wake Words (vww)

Cycles Total ROM Total RAM ROM read-only ROM code VLEN Layout Kernels Extensions
90828165 550218 180968 462932 87270 0 NCHW Default
86102169 530776 180968 462932 67828 0 NHWC Default
46356353 531550 180968 462932 68602 0 NCHW Autotuned
86102172 530780 180968 462932 67832 0 NHWC Autotuned
45331325 333204 85588 266324 66864 0 muRISCV-NN
26022364 333436 85588 266324 67096 64 muRISCV-NN VEXT
7585315 333420 85588 266324 67080 1024 muRISCV-NN VEXT
22052580 340432 85588 266080 74276 0 muRISCV-NN PEXT

Image Classification (resnet)

Cycles Total ROM Total RAM ROM read-only ROM code VLEN Layout Kernels Extensions
138976677 226522 108360 163156 63350 0 NCHW Default
115540068 217372 108360 163156 54200 0 NHWC Default
56900335 220452 92176 163156 57280 0 NCHW Autotuned
115536141 217470 108360 163156 54298 0 NHWC Autotuned
62228686 146938 55444 87332 59590 0 muRISCV-NN
37562250 147640 55444 87332 60292 64 muRISCV-NN VEXT
5764852 147640 55444 87332 60292 1024 muRISCV-NN VEXT
27816288 156904 55444 87088 69740 0 muRISCV-NN PEXT

Anomaly Detection (toycar)

Cycles Total ROM Total RAM ROM read-only ROM code VLEN Layout Kernels Extensions
3134840 589618 5492 537988 51614 0 NCHW Default
3134840 589618 5492 537988 51614 0 NHWC Default
2041108 617982 6804 537988 79978 0 NCHW Autotuned
2041108 617982 6804 537988 79978 0 NHWC Autotuned
1822341 324918 4692 273924 50978 0 muRISCV-NN
626114 324964 4692 273924 51024 64 muRISCV-NN VEXT
435688 324958 4692 273924 51018 1024 muRISCV-NN VEXT
680159 327564 4692 273680 53808 0 muRISCV-NN PEXT

Code size compared to CMSIS-NN

WIP