Benchmarks 2023 06 02 - tum-ei-eda/muriscv-nn GitHub Wiki

Setup

Simulator

Toolchains

Models

Package Versions

  • MLonMCU : 2ab749d3747c04f3b01973ef955bda5be0964f0f
  • TFLM : f050eec7e32a0895f7658db21a4bdbd0975087a5
  • TVM : Nightly Pre-Build
  • Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
  • Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

Miscellaneous

  • Used -Os flag for compilation.
  • Benchmarks generated using MLonMCU deployment tool with minimal efforts.
  • Memory metrics are reported in Bytes

Frameworks

Benchmarks exist for integration with TensorFlow Lite for Microcontrollers (TFLM) and TVM.

TFLite for Microcontrollers

Comparing reference kernel implementations (Default) with cmsis-nn optimized kernels and linkage with libmuriscvnn.a.

Audio Wake Words (aww)

Cycles Cycles(Relative) Total ROM Total ROM(Relative) Total RAM Total RAM(Relative) ROM read-only ROM code VLEN Kernels Extensions
153529267 10.016 144687 0.912 36192 1.0 63653 79826 0 Default
15327748 1.0 158696 1.0 36200 1.0 63656 93832 0 muRISCV-NN
9026170 0.589 157944 0.995 36208 1.0 63652 93088 64 muRISCV-NN VEXT
2135130 0.139 157944 0.995 36208 1.0 63652 93088 1024 muRISCV-NN VEXT
6044575 0.394 201872 1.272 36344 1.004 70740 126136 0 muRISCV-NN PEXT

Visual Wake Words (vww)

Cycles Cycles(Relative) Total ROM Total ROM(Relative) Total RAM Total RAM(Relative) ROM read-only ROM code VLEN Kernels Extensions
433523791 9.376 418411 0.968 134508 1.0 337293 79910 0 Default
46237716 1.0 432420 1.0 134516 1.0 337296 93916 0 muRISCV-NN
27212603 0.589 431668 0.998 134524 1.0 337292 93172 64 muRISCV-NN VEXT
8775554 0.19 431668 0.998 134524 1.0 337292 93172 1024 muRISCV-NN VEXT
19841712 0.429 475588 1.1 134660 1.001 344380 126212 0 muRISCV-NN PEXT

Image Classification (resnet)

Cycles Cycles(Relative) Total ROM Total ROM(Relative) Total RAM Total RAM(Relative) ROM read-only ROM code VLEN Kernels Extensions
687851073 12.52 184423 0.956 68956 1.0 102485 80730 0 Default
54941631 1.0 192828 1.0 68948 1.0 102488 89132 0 muRISCV-NN
37856218 0.689 192866 1.0 68956 1.0 102500 89162 64 muRISCV-NN VEXT
6058820 0.11 192866 1.0 68956 1.0 102500 89162 1024 muRISCV-NN VEXT
26794473 0.488 238744 1.238 69092 1.002 109572 124176 0 muRISCV-NN PEXT

Anomaly Detection (toycar)

Cycles Cycles(Relative) Total ROM Total ROM(Relative) Total RAM Total RAM(Relative) ROM read-only ROM code VLEN Kernels Extensions
3434813 1.788 347340 0.993 19392 1.0 280872 65260 0 Default
1920500 1.0 349928 1.0 19392 1.0 280872 67848 0 muRISCV-NN
729214 0.38 349838 1.0 19396 1.0 280868 67766 64 muRISCV-NN VEXT
538790 0.281 349838 1.0 19396 1.0 280868 67766 1024 muRISCV-NN VEXT
788434 0.411 394176 1.126 19536 1.007 287956 101224 0 muRISCV-NN PEXT

TVM

Using CMSIS-NN BYOC feature and linkage with libmuriscvnn.a.

Extension mapping:

  • VEXT -> cortex-m55
  • PEXT -> cortex-m7

Used data/kernel layouts:

  • Untuned: NHWC (TFLite default)
  • Tuned: NCHW (Converted using TVM)

Audio Wake Words (aww)

Cycles Cycles(Relative) Total ROM Total ROM(Relative) Total RAM Total RAM(Relative) ROM read-only ROM code VLEN Layout Kernels Extensions
31435272 2.046 117272 1.171 59448 3.106 56212 61044 0 NCHW Default
29580732 1.926 111594 1.114 59448 3.106 56212 55366 0 NHWC Default
15699726 1.022 110750 1.106 51276 2.679 56212 54522 0 NCHW Autotuned
29576111 1.925 111724 1.115 59448 3.106 56212 55496 0 NHWC Autotuned
15361113 1.0 100174 1.0 19140 1.0 36644 63514 0 muRISCV-NN
10091110 0.657 99576 0.994 23604 1.233 36644 62916 64 muRISCV-NN VEXT
3987654 0.26 99576 0.994 23604 1.233 36644 62916 1024 muRISCV-NN VEXT
6755351 0.44 106586 1.064 20292 1.06 36400 70110 0 muRISCV-NN PEXT

Visual Wake Words (vww)

Cycles Cycles(Relative) Total ROM Total ROM(Relative) Total RAM Total RAM(Relative) ROM read-only ROM code VLEN Layout Kernels Extensions
90828165 2.016 550218 1.648 180968 2.114 462932 87270 0 NCHW Default
86102169 1.911 530776 1.59 180968 2.114 462932 67828 0 NHWC Default
46356353 1.029 531550 1.593 180968 2.114 462932 68602 0 NCHW Autotuned
86102172 1.911 530780 1.59 180968 2.114 462932 67832 0 NHWC Autotuned
45064061 1.0 333770 1.0 85588 1.0 266324 67430 0 muRISCV-NN
26022364 0.577 333460 0.999 85588 1.0 266324 67120 64 muRISCV-NN VEXT
7585315 0.168 333434 0.999 85588 1.0 266324 67094 1024 muRISCV-NN VEXT
18622291 0.413 340446 1.02 85588 1.0 266080 74290 0 muRISCV-NN PEXT

Image Classification (resnet)

Cycles Cycles(Relative) Total ROM Total ROM(Relative) Total RAM Total RAM(Relative) ROM read-only ROM code VLEN Layout Kernels Extensions
138976677 2.543 226522 1.536 108360 1.954 163156 63350 0 NCHW Default
115540068 2.115 217372 1.474 108360 1.954 163156 54200 0 NHWC Default
56900335 1.041 220452 1.495 92176 1.663 163156 57280 0 NCHW Autotuned
115536141 2.114 217472 1.475 108360 1.954 163156 54300 0 NHWC Autotuned
54640014 1.0 147486 1.0 55444 1.0 87332 60138 0 muRISCV-NN
37562250 0.687 147640 1.001 55444 1.0 87332 60292 64 muRISCV-NN VEXT
5764852 0.106 147640 1.001 55444 1.0 87332 60292 1024 muRISCV-NN VEXT
26487976 0.485 156704 1.063 55444 1.0 87088 69540 0 muRISCV-NN PEXT

Anomaly Detection (toycar)

Cycles Cycles(Relative) Total ROM Total ROM(Relative) Total RAM Total RAM(Relative) ROM read-only ROM code VLEN Layout Kernels Extensions
3134840 1.72 589618 1.815 5492 1.171 537988 51614 0 NCHW Default
3134840 1.72 589618 1.815 5492 1.171 537988 51614 0 NHWC Default
2041108 1.12 617982 1.902 6804 1.45 537988 79978 0 NCHW Autotuned
2041108 1.12 617982 1.902 6804 1.45 537988 79978 0 NHWC Autotuned
1822341 1.0 324916 1.0 4692 1.0 273924 50976 0 muRISCV-NN
626114 0.344 324966 1.0 4692 1.0 273924 51026 64 muRISCV-NN VEXT
435690 0.239 324952 1.0 4692 1.0 273924 51012 1024 muRISCV-NN VEXT
680159 0.373 327564 1.008 4692 1.0 273680 53808 0 muRISCV-NN PEXT

Code size compared to CMSIS-NN

WIP