Benchmarks 2023 01 28 - tum-ei-eda/muriscv-nn GitHub Wiki

• Spike (riscv-isa-sim ) (ISS, CPI=1)

• Scalar: Self compiled on master branch of riscv-gnu-toolchain
• Packed: Self compiled using patches found in https://github.com/riscv-collab/riscv-gcc/pull/258 and https://github.com/riscvarchive/riscv-binutils-gdb/pull/257
• Vector: Self compiled using this working-branch: riscv-gcc-rvv-next

• MLPerfTiny Benchmark

• MLonMCU : 2ab749d3747c04f3b01973ef955bda5be0964f0f
• TFLM : f050eec7e32a0895f7658db21a4bdbd0975087a5
• TVM : Nightly Pre-Build
• Spike : 0bc176b3fca43560b9e8586cdbc41cfde073e17a
• Spike PK : 7e9b671c0415dfd7b562ac934feb9380075d4aa2

• Used -Os flag for compilation.
• Benchmarks generated using MLonMCU deployment tool with minimal efforts.
• Memory metrics are reported in Bytes

Benchmarks exist for integration with TensorFlow Lite for Microcontrollers (TFLM) and TVM.

Comparing reference kernel implementations (Default) with cmsis-nn optimized kernels and linkage with libmuriscvnn.a.

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
153529267	144687	36192	63653	79826	0		Default
16598384	157608	36200	63656	92744	0		muRISCV-NN
9026170	157944	36208	63652	93088	64		muRISCV-NN
2135130	157944	36208	63652	93088	1024		muRISCV-NN
13983214	196006	36344	70740	120270	0		muRISCV-NN

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
433523791	418411	134508	337293	79910	0		Default
49688248	431332	134516	337296	92828	0		muRISCV-NN
27212603	431668	134524	337292	93172	64		muRISCV-NN
8775554	431668	134524	337292	93172	1024		muRISCV-NN
41516367	469722	134660	344380	120346	0		muRISCV-NN

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
687851073	184423	68956	102485	80730	0		Default
62519499	192280	68948	102488	88584	0		muRISCV-NN
37856218	192866	68956	102500	89162	64		muRISCV-NN
6058820	192866	68956	102500	89162	1024		muRISCV-NN
64768125	232872	69092	109572	118304	0		muRISCV-NN

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
3434813	347340	19392	280872	65260	0		Default
1920500	349928	19392	280872	67848	0		muRISCV-NN
729214	349838	19396	280868	67766	64		muRISCV-NN
538790	349838	19396	280868	67766	1024		muRISCV-NN
1174689	389048	19536	287956	96096	0		muRISCV-NN

Using CMSIS-NN BYOC feature and linkage with libmuriscvnn.a.

Extension mapping:

• VEXT -> cortex-m55
• PEXT -> cortex-m7

Used data/kernel layouts:

• Untuned: NHWC (TFLite default)
• Tuned: NCHW (Converted using TVM)

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
31435272	117272	59448	56212	61044	0	NCHW	Default
29580732	111594	59448	56212	55366	0	NHWC	Default
15699726	110750	51276	56212	54522	0	NCHW	Autotuned
29576111	111724	59448	56212	55496	0	NHWC	Autotuned
16449679	99116	19140	36644	62456	0		muRISCV-NN
10091121	99606	23604	36644	62946	64		muRISCV-NN
3987665	99606	23604	36644	62946	1024		muRISCV-NN
13703187	100748	20292	36400	64272	0		muRISCV-NN

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
90828165	550218	180968	462932	87270	0	NCHW	Default
86102169	530776	180968	462932	67828	0	NHWC	Default
46356353	531550	180968	462932	68602	0	NCHW	Autotuned
86102172	530780	180968	462932	67832	0	NHWC	Autotuned
48514662	332812	85588	266324	66472	0		muRISCV-NN
26022393	333582	85588	266324	67242	64		muRISCV-NN
7585344	333594	85588	266324	67254	1024		muRISCV-NN
40307469	334720	85588	266080	68564	0		muRISCV-NN

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
138976677	226522	108360	163156	63350	0	NCHW	Default
115540068	217372	108360	163156	54200	0	NHWC	Default
56900335	220452	92176	163156	57280	0	NCHW	Autotuned
115536141	217470	108360	163156	54298	0	NHWC	Autotuned
62228697	147004	55444	87332	59656	0		muRISCV-NN
37562261	147708	55444	87332	60360	64		muRISCV-NN
5764863	147708	55444	87332	60360	1024		muRISCV-NN
64455063	150898	55444	87088	63734	0		muRISCV-NN

Cycles	Total ROM	Total RAM	ROM read-only	ROM code	VLEN	Kernels	Extensions
3134840	589618	5492	537988	51614	0	NCHW	Default
3134840	589618	5492	537988	51614	0	NHWC	Default
2041108	617982	6804	537988	79978	0	NCHW	Autotuned
2041108	617982	6804	537988	79978	0	NHWC	Autotuned
1822351	324984	4692	273924	51044	0		muRISCV-NN
626122	325022	4692	273924	51082	64		muRISCV-NN
435700	325022	4692	273924	51082	1024		muRISCV-NN
1066416	327302	4692	273680	53546	0		muRISCV-NN

WIP