devboard: Jetson orin - YingkunZhou/EdgeTransformerBench GitHub Wiki
-
NVIDIA Jetson Oin - Performance tuning by setting the CPU, GPU, and Frequency values manually
- CPU idle 0.8W
- GPU idle 2.4W
commands
cat /sys/devices/17000000.ga10b/devfreq/17000000.ga10b/max_freq
1300500000
GPU_FREQ=1300500000
sudo sh -c "echo $GPU_FREQ > /sys/devices/17000000.ga10b/devfreq/17000000.ga10b/min_freq"如果想释放最大性能，还有一个最简单粗暴的方式： Compute time in DLA slower than expected - Jetson & Embedded Systems / Jetson AGX Orin - NVIDIA Developer Forums
$ sudo nvpmodel -m 0
$ sudo jetson_clockscommit id:
0ddc34f5223b634fcaa89f67634b5f88db04bfd3
cortex-A78 @ 1 thread @ 2.2GHz w/ fp16(default)
$ MODEL=ALL make run-ncnn-perf
INFO: Using CPU backend
INFO: Using num_threads == 1
Creating ncnn net: efficientformerv2_s0
(index: 985, score: 11.781250), (index: 644, score: 4.968750), (index: 309, score: 4.000000),
[191 iters] min = 103.84ms max = 108.92ms median = 105.07ms mean = 105.20ms
Creating ncnn net: efficientformerv2_s1
(index: 985, score: 13.187500), (index: 308, score: 4.398438), (index: 984, score: 4.390625),
[128 iters] min = 155.03ms max = 166.47ms median = 156.28ms mean = 156.43ms
Creating ncnn net: efficientformerv2_s2
(index: 985, score: 12.609375), (index: 22, score: 3.921875), (index: 80, score: 3.523438),
[82 iters] min = 244.02ms max = 246.07ms median = 245.13ms mean = 245.11ms
SwiftFormer_XS model doesn't exist!!!
SwiftFormer_S model doesn't exist!!!
SwiftFormer_L1 model doesn't exist!!!
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
edgenext_xx_small model doesn't exist!!!
edgenext_x_small model doesn't exist!!!
edgenext_small model doesn't exist!!!
Creating ncnn net: mobilevitv2_050
(index: 985, score: 8.421875), (index: 309, score: 2.664062), (index: 89, score: 2.494141),
[693 iters] min = 28.63ms max = 29.16ms median = 28.89ms mean = 28.89ms
Creating ncnn net: mobilevitv2_075
(index: 985, score: 8.257812), (index: 309, score: 2.691406), (index: 308, score: 2.125000),
[376 iters] min = 52.91ms max = 53.69ms median = 53.26ms mean = 53.27ms
Creating ncnn net: mobilevitv2_100
(index: 985, score: 8.242188), (index: 557, score: 2.316406), (index: 309, score: 2.091797),
[237 iters] min = 83.81ms max = 85.10ms median = 84.46ms mean = 84.46ms
Creating ncnn net: mobilevitv2_125
(index: 985, score: 8.453125), (index: 309, score: 2.076172), (index: 113, score: 1.410156),
[165 iters] min = 119.43ms max = 122.06ms median = 121.65ms mean = 121.65ms
Creating ncnn net: mobilevitv2_150
(index: 985, score: 9.007812), (index: 308, score: 2.257812), (index: 301, score: 2.234375),
[121 iters] min = 164.76ms max = 175.67ms median = 165.74ms mean = 165.91ms
Creating ncnn net: mobilevitv2_175
(index: 985, score: 8.882812), (index: 494, score: 2.082031), (index: 309, score: 1.867188),
[93 iters] min = 211.94ms max = 217.23ms median = 216.52ms mean = 216.45ms
Creating ncnn net: mobilevitv2_200
(index: 985, score: 8.554688), (index: 309, score: 2.222656), (index: 308, score: 2.183594),
[73 iters] min = 272.63ms max = 284.75ms median = 273.60ms mean = 274.62ms
Creating ncnn net: mobilevit_xx_small
(index: 999, score: -nan), (index: 998, score: -nan), (index: 997, score: -nan),
[1461 iters] min = 13.56ms max = 14.00ms median = 13.69ms mean = 13.69ms
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
LeViT_128S model doesn't exist!!!
LeViT_128 model doesn't exist!!!
LeViT_192 model doesn't exist!!!
LeViT_256 model doesn't exist!!!
Creating ncnn net: resnet50
(index: 985, score: 7.875000), (index: 113, score: -5.207031), (index: 307, score: -5.398438),
[193 iters] min = 99.34ms max = 104.74ms median = 104.15ms mean = 104.14ms
Creating ncnn net: mobilenetv3_large_100
(index: 985, score: 9.726562), (index: 310, score: 2.718750), (index: 308, score: 2.394531),
[1541 iters] min = 12.87ms max = 13.25ms median = 12.98ms mean = 12.99ms
Creating ncnn net: tf_efficientnetv2_b0
(index: 985, score: 9.734375), (index: 309, score: 2.583984), (index: 310, score: 2.402344),
[617 iters] min = 31.53ms max = 32.65ms median = 32.45ms mean = 32.45ms
Creating ncnn net: tf_efficientnetv2_b1
(index: 985, score: 9.687500), (index: 309, score: 2.289062), (index: 310, score: 2.222656),
[418 iters] min = 47.47ms max = 48.20ms median = 47.84ms mean = 47.85ms
Creating ncnn net: tf_efficientnetv2_b2
(index: 985, score: 10.023438), (index: 883, score: 2.632812), (index: 309, score: 2.167969),
[293 iters] min = 68.14ms max = 68.78ms median = 68.42ms mean = 68.43ms
Creating ncnn net: tf_efficientnetv2_b3
(index: 985, score: 9.210938), (index: 955, score: 2.845703), (index: 310, score: 2.222656),
[171 iters] min = 116.51ms max = 117.56ms median = 117.13ms mean = 117.15msGPU Vulkan @ 1.3GHz w/ fp32
INFO: Using Vulkan backend
INFO: Using num_threads == 1
Creating ncnn net: efficientformerv2_s0
opt status: 111011101 ==> 000000001
(index: 985, score: 11.783970), (index: 644, score: 4.838636), (index: 108, score: 3.926247),
[1368 iters] min = 11.39ms max = 19.37ms median = 14.43ms mean = 14.62ms
Creating ncnn net: efficientformerv2_s1
opt status: 111011101 ==> 000000001
(index: 985, score: 13.082937), (index: 89, score: 4.173867), (index: 984, score: 4.094892),
[1162 iters] min = 13.45ms max = 22.63ms median = 16.60ms mean = 17.21ms
Creating ncnn net: efficientformerv2_s2
opt status: 111011101 ==> 000000001
(index: 985, score: 12.511873), (index: 309, score: 3.711616), (index: 22, score: 3.676414),
[719 iters] min = 22.15ms max = 34.02ms median = 27.83ms mean = 27.82ms
Creating ncnn net: mobilevitv2_050
opt status: 111011101 ==> 000000001
(index: 985, score: 8.305042), (index: 309, score: 2.612881), (index: 584, score: 2.330210),
[1719 iters] min = 8.72ms max = 17.78ms median = 11.68ms mean = 11.64ms
Creating ncnn net: mobilevitv2_075
opt status: 111011101 ==> 000000001
(index: 985, score: 8.126366), (index: 309, score: 2.389989), (index: 308, score: 1.885907),
[1209 iters] min = 12.82ms max = 24.75ms median = 16.50ms mean = 16.55ms
Creating ncnn net: mobilevitv2_100
opt status: 111011101 ==> 000000001
(index: 985, score: 8.254771), (index: 557, score: 2.225800), (index: 309, score: 1.942583),
[793 iters] min = 16.94ms max = 31.77ms median = 24.20ms mean = 25.25ms
Creating ncnn net: mobilevitv2_125
opt status: 111011101 ==> 000000001
(index: 985, score: 8.281282), (index: 309, score: 1.960415), (index: 883, score: 1.290661),
[693 iters] min = 19.85ms max = 35.53ms median = 26.78ms mean = 28.86ms
Creating ncnn net: mobilevitv2_150
opt status: 111011101 ==> 000000001
(index: 985, score: 9.099475), (index: 308, score: 2.251911), (index: 301, score: 2.153154),
[637 iters] min = 24.25ms max = 40.12ms median = 31.12ms mean = 31.41ms
Creating ncnn net: mobilevitv2_175
opt status: 111011101 ==> 000000001
(index: 985, score: 8.900534), (index: 494, score: 2.110982), (index: 309, score: 1.876236),
[517 iters] min = 32.17ms max = 45.72ms median = 37.18ms mean = 38.70ms
Creating ncnn net: mobilevitv2_200
opt status: 111011101 ==> 000000001
(index: 985, score: 8.531097), (index: 883, score: 2.244244), (index: 309, score: 2.230251),
[455 iters] min = 36.96ms max = 50.49ms median = 43.35ms mean = 43.97ms
Creating ncnn net: mobilevit_xx_small
opt status: 111011101 ==> 000000001
(index: 785, score: 7.877839), (index: 334, score: 7.784947), (index: 149, score: 7.445580),
[530 iters] min = 29.59ms max = 64.34ms median = 36.34ms mean = 37.78ms
Creating ncnn net: resnet50
opt status: 111011101 ==> 000000001
(index: 985, score: 7.484056), (index: 113, score: -4.938168), (index: 310, score: -5.258437),
[1623 iters] min = 12.06ms max = 16.53ms median = 12.20ms mean = 12.32ms
Creating ncnn net: mobilenetv3_large_100
opt status: 111011101 ==> 000000001
(index: 985, score: 9.600928), (index: 308, score: 2.362726), (index: 310, score: 2.348944),
[2319 iters] min = 5.81ms max = 12.23ms median = 8.63ms mean = 8.63ms
Creating ncnn net: tf_efficientnetv2_b0
opt status: 111011101 ==> 000000001
(index: 985, score: 9.552652), (index: 309, score: 2.377691), (index: 108, score: 2.288837),
[1204 iters] min = 13.38ms max = 20.11ms median = 16.64ms mean = 16.62ms
Creating ncnn net: tf_efficientnetv2_b1
opt status: 111011101 ==> 000000001
(index: 985, score: 9.484993), (index: 861, score: 2.249813), (index: 309, score: 2.138905),
[929 iters] min = 17.13ms max = 26.01ms median = 21.06ms mean = 21.54ms
Creating ncnn net: tf_efficientnetv2_b2
opt status: 111011101 ==> 000000001
(index: 985, score: 9.816036), (index: 883, score: 2.518359), (index: 113, score: 2.038458),
[832 iters] min = 20.39ms max = 27.97ms median = 24.11ms mean = 24.05ms
Creating ncnn net: tf_efficientnetv2_b3
opt status: 111011101 ==> 000000001
(index: 985, score: 9.093818), (index: 955, score: 2.889796), (index: 947, score: 2.188510),
[640 iters] min = 28.29ms max = 35.99ms median = 30.87ms mean = 31.25msGPU Vulkan @ 1.3GHz w/ fp16
$ BACK=v MODEL=ALL make run-ncnn-perf
INFO: Using Vulkan backend
INFO: Using num_threads == 1
[0 NVIDIA Tegra Orin (nvgpu)] queueC=2[8] queueG=0[16] queueT=1[2]
[0 NVIDIA Tegra Orin (nvgpu)] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0
[0 NVIDIA Tegra Orin (nvgpu)] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1
[0 NVIDIA Tegra Orin (nvgpu)] subgroup=32 basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA Tegra Orin (nvgpu)] fp16-matrix-16_8_8/16_8_16/16_16_16=1/1/1
Creating ncnn net: efficientformerv2_s0
(index: 985, score: 11.718750), (index: 644, score: 5.000000), (index: 954, score: 3.851562),
[1902 iters] min = 8.54ms max = 13.55ms median = 10.03ms mean = 10.52ms
Creating ncnn net: efficientformerv2_s1
(index: 985, score: 13.257812), (index: 984, score: 4.402344), (index: 308, score: 4.296875),
[1829 iters] min = 8.25ms max = 12.56ms median = 10.94ms mean = 10.94ms
Creating ncnn net: efficientformerv2_s2
(index: 985, score: 12.671875), (index: 22, score: 3.951172), (index: 80, score: 3.574219),
[1119 iters] min = 15.11ms max = 20.10ms median = 17.98ms mean = 17.89ms
SwiftFormer_XS model doesn't exist!!!
SwiftFormer_S model doesn't exist!!!
SwiftFormer_L1 model doesn't exist!!!
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
edgenext_xx_small model doesn't exist!!!
edgenext_x_small model doesn't exist!!!
edgenext_small model doesn't exist!!!
Creating ncnn net: mobilevitv2_050
(index: 985, score: 8.414062), (index: 309, score: 2.660156), (index: 89, score: 2.488281),
[2701 iters] min = 5.82ms max = 10.43ms median = 7.28ms mean = 7.41ms
Creating ncnn net: mobilevitv2_075
(index: 985, score: 8.265625), (index: 309, score: 2.703125), (index: 308, score: 2.126953),
[1891 iters] min = 8.83ms max = 15.77ms median = 10.58ms mean = 10.58ms
Creating ncnn net: mobilevitv2_100
(index: 985, score: 8.242188), (index: 557, score: 2.320312), (index: 309, score: 2.097656),
[1245 iters] min = 11.67ms max = 26.60ms median = 14.40ms mean = 16.07ms
Creating ncnn net: mobilevitv2_125
(index: 985, score: 8.460938), (index: 309, score: 2.072266), (index: 113, score: 1.417969),
[1041 iters] min = 13.09ms max = 28.48ms median = 16.80ms mean = 19.23ms
Creating ncnn net: mobilevitv2_150
(index: 985, score: 9.046875), (index: 308, score: 2.265625), (index: 301, score: 2.246094),
[1101 iters] min = 14.38ms max = 20.65ms median = 18.10ms mean = 18.17ms
Creating ncnn net: mobilevitv2_175
(index: 985, score: 8.921875), (index: 494, score: 2.087891), (index: 309, score: 1.878906),
[1002 iters] min = 15.63ms max = 23.75ms median = 19.99ms mean = 19.97ms
Creating ncnn net: mobilevitv2_200
(index: 985, score: 8.585938), (index: 309, score: 2.222656), (index: 308, score: 2.183594),
[803 iters] min = 19.13ms max = 33.22ms median = 22.63ms mean = 24.91ms
Creating ncnn net: mobilevit_xx_small
(index: 843, score: 8.492188), (index: 369, score: 6.578125), (index: 921, score: 5.855469),
[555 iters] min = 26.76ms max = 60.83ms median = 30.60ms mean = 36.08ms
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
LeViT_128S model doesn't exist!!!
LeViT_128 model doesn't exist!!!
LeViT_192 model doesn't exist!!!
LeViT_256 model doesn't exist!!!
Creating ncnn net: resnet50
(index: 985, score: 7.957031), (index: 113, score: -5.222656), (index: 307, score: -5.421875),
[4849 iters] min = 3.60ms max = 5.06ms median = 4.28ms mean = 4.13ms
Creating ncnn net: mobilenetv3_large_100
(index: 985, score: 9.742188), (index: 310, score: 2.714844), (index: 308, score: 2.378906),
[3514 iters] min = 4.36ms max = 7.78ms median = 5.54ms mean = 5.69ms
Creating ncnn net: tf_efficientnetv2_b0
(index: 985, score: 9.734375), (index: 309, score: 2.589844), (index: 310, score: 2.398438),
[1611 iters] min = 7.24ms max = 16.44ms median = 11.87ms mean = 12.42ms
Creating ncnn net: tf_efficientnetv2_b1
(index: 985, score: 9.671875), (index: 309, score: 2.289062), (index: 310, score: 2.218750),
[1393 iters] min = 10.43ms max = 19.02ms median = 14.28ms mean = 14.37ms
Creating ncnn net: tf_efficientnetv2_b2
(index: 985, score: 10.007812), (index: 883, score: 2.636719), (index: 309, score: 2.167969),
[1054 iters] min = 15.22ms max = 23.41ms median = 18.36ms mean = 18.99ms
Creating ncnn net: tf_efficientnetv2_b3
(index: 985, score: 9.187500), (index: 955, score: 2.855469), (index: 310, score: 2.226562),
[912 iters] min = 18.84ms max = 26.81ms median = 21.34ms mean = 21.94msGPU Vulkan @ 0.61GHz w/ fp16
INFO: Using Vulkan backend
INFO: Using num_threads == 1
Creating ncnn net: efficientformerv2_s0
opt status: 111011101 ==> 000000001
(index: 985, score: 11.783970), (index: 644, score: 4.838636), (index: 108, score: 3.926247),
[882 iters] min = 18.24ms max = 28.91ms median = 22.68ms mean = 22.68ms
Creating ncnn net: efficientformerv2_s1
opt status: 111011101 ==> 000000001
(index: 985, score: 13.082937), (index: 89, score: 4.173867), (index: 984, score: 4.094892),
[763 iters] min = 22.89ms max = 31.04ms median = 26.27ms mean = 26.21ms
Creating ncnn net: efficientformerv2_s2
opt status: 111011101 ==> 000000001
(index: 985, score: 12.511873), (index: 309, score: 3.711616), (index: 22, score: 3.676414),
[459 iters] min = 38.85ms max = 48.39ms median = 43.78ms mean = 43.61ms
Creating ncnn net: mobilevitv2_050
opt status: 111011101 ==> 000000001
(index: 985, score: 8.305042), (index: 309, score: 2.612881), (index: 584, score: 2.330210),
[771 iters] min = 14.75ms max = 34.26ms median = 28.13ms mean = 25.95ms
Creating ncnn net: mobilevitv2_075
opt status: 111011101 ==> 000000001
(index: 985, score: 8.126366), (index: 309, score: 2.389989), (index: 308, score: 1.885907),
[657 iters] min = 21.76ms max = 40.08ms median = 30.61ms mean = 30.45ms
Creating ncnn net: mobilevitv2_100
opt status: 111011101 ==> 000000001
(index: 985, score: 8.254771), (index: 557, score: 2.225800), (index: 309, score: 1.942583),
[510 iters] min = 29.53ms max = 45.49ms median = 39.16ms mean = 39.23ms
Creating ncnn net: mobilevitv2_125
opt status: 111011101 ==> 000000001
(index: 985, score: 8.281282), (index: 309, score: 1.960415), (index: 883, score: 1.290661),
[425 iters] min = 37.40ms max = 53.53ms median = 46.85ms mean = 47.15ms
Creating ncnn net: mobilevitv2_150
opt status: 111011101 ==> 000000001
(index: 985, score: 9.099475), (index: 308, score: 2.251911), (index: 301, score: 2.153154),
[363 iters] min = 46.80ms max = 64.77ms median = 54.54ms mean = 55.14ms
Creating ncnn net: mobilevitv2_175
opt status: 111011101 ==> 000000001
(index: 985, score: 8.900534), (index: 494, score: 2.110982), (index: 309, score: 1.876236),
[306 iters] min = 55.72ms max = 75.59ms median = 65.23ms mean = 65.47ms
Creating ncnn net: mobilevitv2_200
opt status: 111011101 ==> 000000001
(index: 985, score: 8.531097), (index: 883, score: 2.244244), (index: 309, score: 2.230251),
[268 iters] min = 66.20ms max = 82.78ms median = 74.82ms mean = 74.90ms
Creating ncnn net: mobilevit_xx_small
opt status: 111011101 ==> 000000001
(index: 785, score: 7.877839), (index: 334, score: 7.784947), (index: 149, score: 7.445580),
[289 iters] min = 50.09ms max = 95.53ms median = 64.50ms mean = 69.27ms
Creating ncnn net: resnet50
opt status: 111011101 ==> 000000001
(index: 985, score: 7.484056), (index: 113, score: -4.938168), (index: 310, score: -5.258437),
[859 iters] min = 22.63ms max = 31.73ms median = 23.03ms mean = 23.29ms
Creating ncnn net: mobilenetv3_large_100
opt status: 111011101 ==> 000000001
(index: 985, score: 9.600928), (index: 308, score: 2.362726), (index: 310, score: 2.348944),
[1398 iters] min = 9.65ms max = 16.85ms median = 14.53ms mean = 14.31ms
Creating ncnn net: tf_efficientnetv2_b0
opt status: 111011101 ==> 000000001
(index: 985, score: 9.552652), (index: 309, score: 2.377691), (index: 108, score: 2.288837),
[699 iters] min = 23.71ms max = 34.03ms median = 28.73ms mean = 28.63ms
Creating ncnn net: tf_efficientnetv2_b1
opt status: 111011101 ==> 000000001
(index: 985, score: 9.484993), (index: 861, score: 2.249813), (index: 309, score: 2.138905),
[561 iters] min = 31.89ms max = 40.85ms median = 35.95ms mean = 35.69ms
Creating ncnn net: tf_efficientnetv2_b2
opt status: 111011101 ==> 000000001
(index: 985, score: 9.816036), (index: 883, score: 2.518359), (index: 113, score: 2.038458),
[504 iters] min = 36.98ms max = 44.12ms median = 39.72ms mean = 39.68ms
Creating ncnn net: tf_efficientnetv2_b3
opt status: 111011101 ==> 000000001
(index: 985, score: 9.093818), (index: 955, score: 2.889796), (index: 947, score: 2.188510),
[396 iters] min = 45.06ms max = 55.53ms median = 50.58ms mean = 50.60msGPU Vulkan @ 0.61GHz w/ fp16
$ BACK=v MODEL=ALL make run-ncnn-perf
INFO: Using Vulkan backend
INFO: Using num_threads == 1
[0 NVIDIA Tegra Orin (nvgpu)] queueC=2[8] queueG=0[16] queueT=1[2]
[0 NVIDIA Tegra Orin (nvgpu)] bugsbn1=0 bugbilz=0 bugcopc=0 bugihfa=0
[0 NVIDIA Tegra Orin (nvgpu)] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1
[0 NVIDIA Tegra Orin (nvgpu)] subgroup=32 basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA Tegra Orin (nvgpu)] fp16-matrix-16_8_8/16_8_16/16_16_16=1/1/1
Creating ncnn net: efficientformerv2_s0
(index: 985, score: 11.718750), (index: 644, score: 5.000000), (index: 954, score: 3.851562),
[1244 iters] min = 12.55ms max = 19.16ms median = 16.22ms mean = 16.08ms
Creating ncnn net: efficientformerv2_s1
(index: 985, score: 13.257812), (index: 984, score: 4.402344), (index: 308, score: 4.296875),
[1134 iters] min = 15.07ms max = 20.52ms median = 17.75ms mean = 17.65ms
Creating ncnn net: efficientformerv2_s2
(index: 985, score: 12.671875), (index: 22, score: 3.951172), (index: 80, score: 3.574219),
[691 iters] min = 21.02ms max = 31.71ms median = 29.23ms mean = 28.95ms
SwiftFormer_XS model doesn't exist!!!
SwiftFormer_S model doesn't exist!!!
SwiftFormer_L1 model doesn't exist!!!
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
edgenext_xx_small model doesn't exist!!!
edgenext_x_small model doesn't exist!!!
edgenext_small model doesn't exist!!!
Creating ncnn net: mobilevitv2_050
(index: 985, score: 8.414062), (index: 309, score: 2.660156), (index: 89, score: 2.488281),
[1209 iters] min = 10.59ms max = 26.70ms median = 17.03ms mean = 16.55ms
Creating ncnn net: mobilevitv2_075
(index: 985, score: 8.265625), (index: 309, score: 2.703125), (index: 308, score: 2.126953),
[892 iters] min = 13.24ms max = 32.89ms median = 21.40ms mean = 22.44ms
Creating ncnn net: mobilevitv2_100
(index: 985, score: 8.242188), (index: 557, score: 2.320312), (index: 309, score: 2.097656),
[765 iters] min = 18.42ms max = 32.54ms median = 26.89ms mean = 26.16ms
Creating ncnn net: mobilevitv2_125
(index: 985, score: 8.460938), (index: 309, score: 2.072266), (index: 113, score: 1.417969),
[704 iters] min = 21.63ms max = 36.27ms median = 29.05ms mean = 28.42ms
Creating ncnn net: mobilevitv2_150
(index: 985, score: 9.046875), (index: 308, score: 2.265625), (index: 301, score: 2.246094),
[635 iters] min = 22.52ms max = 37.63ms median = 31.73ms mean = 31.50ms
Creating ncnn net: mobilevitv2_175
(index: 985, score: 8.921875), (index: 494, score: 2.087891), (index: 309, score: 1.878906),
[570 iters] min = 27.41ms max = 40.24ms median = 35.56ms mean = 35.12ms
Creating ncnn net: mobilevitv2_200
(index: 985, score: 8.585938), (index: 309, score: 2.222656), (index: 308, score: 2.183594),
[505 iters] min = 31.62ms max = 47.37ms median = 39.72ms mean = 39.66ms
Creating ncnn net: mobilevit_xx_small
(index: 843, score: 8.492188), (index: 369, score: 6.578125), (index: 921, score: 5.855469),
[336 iters] min = 42.00ms max = 93.06ms median = 57.01ms mean = 59.70ms
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
LeViT_128S model doesn't exist!!!
LeViT_128 model doesn't exist!!!
LeViT_192 model doesn't exist!!!
LeViT_256 model doesn't exist!!!
Creating ncnn net: resnet50
(index: 985, score: 7.957031), (index: 113, score: -5.222656), (index: 307, score: -5.421875),
[2941 iters] min = 6.28ms max = 7.94ms median = 6.71ms mean = 6.80ms
Creating ncnn net: mobilenetv3_large_100
(index: 985, score: 9.742188), (index: 310, score: 2.714844), (index: 308, score: 2.378906),
[1939 iters] min = 7.19ms max = 13.05ms median = 10.39ms mean = 10.32ms
Creating ncnn net: tf_efficientnetv2_b0
(index: 985, score: 9.734375), (index: 309, score: 2.589844), (index: 310, score: 2.398438),
[1006 iters] min = 14.37ms max = 23.08ms median = 20.22ms mean = 19.89ms
Creating ncnn net: tf_efficientnetv2_b1
(index: 985, score: 9.671875), (index: 309, score: 2.289062), (index: 310, score: 2.218750),
[786 iters] min = 20.20ms max = 28.39ms median = 25.66ms mean = 25.45ms
Creating ncnn net: tf_efficientnetv2_b2
(index: 985, score: 10.007812), (index: 883, score: 2.636719), (index: 309, score: 2.167969),
[673 iters] min = 26.22ms max = 32.15ms median = 30.03ms mean = 29.75ms
Creating ncnn net: tf_efficientnetv2_b3
(index: 985, score: 9.187500), (index: 955, score: 2.855469), (index: 310, score: 2.226562),
[571 iters] min = 30.52ms max = 38.82ms median = 35.02ms mean = 35.03mscommit id:
32f72f4fb983a700d3c8f20549e159ee3860952b
newUnary->main.AsUnaryOp()->opType = UnaryOpOperation_GELU
cortex-A78 @ 1 thread @ 2.2GHz w/ fp32 (can use --fp16 storage to half model size)
-
--weightQuantBits 8 --weightQuantAsymmetricperformance is the same.
$ MODEL=ALL make run-mnn-perf
INFO: Using CPU backend
INFO: Using num_threads == 1
The device support i8sdot:1, support fp16:1, support i8mm: 0
Creating MNN Interpreter: efficientformerv2_s0
(index: 985, score: 11.768960), (index: 644, score: 4.829375), (index: 108, score: 3.931292),
[507 iters] min = 38.78ms max = 41.86ms median = 39.51ms mean = 39.51ms
Creating MNN Interpreter: efficientformerv2_s1
(index: 985, score: 13.083185), (index: 89, score: 4.154803), (index: 984, score: 4.072508),
[328 iters] min = 60.18ms max = 61.78ms median = 61.09ms mean = 61.05ms
Creating MNN Interpreter: efficientformerv2_s2
(index: 985, score: 12.495360), (index: 309, score: 3.706549), (index: 22, score: 3.682925),
[183 iters] min = 107.63ms max = 116.75ms median = 109.36ms mean = 109.39ms
Creating MNN Interpreter: SwiftFormer_XS
(index: 985, score: 11.912077), (index: 883, score: 4.997910), (index: 310, score: 4.615772),
[382 iters] min = 51.72ms max = 52.94ms median = 52.51ms mean = 52.47ms
Creating MNN Interpreter: SwiftFormer_S
(index: 985, score: 12.532909), (index: 89, score: 4.324093), (index: 720, score: 4.182640),
[253 iters] min = 77.47ms max = 90.70ms median = 79.01ms mean = 79.21ms
Creating MNN Interpreter: SwiftFormer_L1
(index: 985, score: 13.235222), (index: 309, score: 3.921143), (index: 310, score: 3.798431),
[163 iters] min = 120.18ms max = 129.57ms median = 122.99ms mean = 123.16ms
Creating MNN Interpreter: EMO_1M
(index: 985, score: 10.015739), (index: 309, score: 4.272019), (index: 310, score: 3.913734),
[559 iters] min = 34.76ms max = 39.48ms median = 35.85ms mean = 35.81ms
Creating MNN Interpreter: EMO_2M
(index: 985, score: 9.377331), (index: 309, score: 3.261263), (index: 308, score: 3.011570),
[376 iters] min = 51.82ms max = 57.37ms median = 53.10ms mean = 53.23ms
Creating MNN Interpreter: EMO_5M
(index: 985, score: 9.150205), (index: 883, score: 2.993564), (index: 308, score: 2.458643),
[216 iters] min = 90.56ms max = 96.92ms median = 92.61ms mean = 92.75ms
Creating MNN Interpreter: EMO_6M
(index: 985, score: 9.407994), (index: 883, score: 2.236737), (index: 309, score: 2.090058),
[203 iters] min = 95.99ms max = 108.64ms median = 98.41ms mean = 98.64ms
Creating MNN Interpreter: edgenext_xx_small
(index: 985, score: 10.881224), (index: 309, score: 4.952091), (index: 310, score: 4.636828),
[753 iters] min = 26.02ms max = 45.65ms median = 26.54ms mean = 26.57ms
Creating MNN Interpreter: edgenext_x_small
(index: 985, score: 9.792659), (index: 309, score: 4.592906), (index: 308, score: 3.815865),
[387 iters] min = 50.83ms max = 52.18ms median = 51.82ms mean = 51.74ms
Creating MNN Interpreter: edgenext_small
(index: 985, score: 12.166285), (index: 309, score: 4.538562), (index: 308, score: 4.057699),
[191 iters] min = 102.68ms max = 109.45ms median = 104.93ms mean = 104.90ms
Creating MNN Interpreter: mobilevitv2_050
(index: 985, score: 8.315649), (index: 309, score: 2.612311), (index: 584, score: 2.352622),
[376 iters] min = 51.15ms max = 59.11ms median = 53.14ms mean = 53.30ms
Creating MNN Interpreter: mobilevitv2_075
(index: 985, score: 8.129767), (index: 309, score: 2.389330), (index: 308, score: 1.880279),
[202 iters] min = 96.38ms max = 103.02ms median = 99.56ms mean = 99.48ms
Creating MNN Interpreter: mobilevitv2_100
(index: 985, score: 8.256266), (index: 557, score: 2.220439), (index: 309, score: 1.944910),
[126 iters] min = 154.59ms max = 161.31ms median = 160.14ms mean = 159.77ms
Creating MNN Interpreter: mobilevitv2_125
(index: 985, score: 8.281974), (index: 309, score: 1.962234), (index: 883, score: 1.285449),
[87 iters] min = 225.43ms max = 234.29ms median = 233.05ms mean = 232.38ms
Creating MNN Interpreter: mobilevitv2_150
(index: 985, score: 9.098869), (index: 308, score: 2.259607), (index: 301, score: 2.159089),
[63 iters] min = 313.68ms max = 323.60ms median = 320.56ms mean = 320.31ms
Creating MNN Interpreter: mobilevitv2_175
(index: 985, score: 8.888629), (index: 494, score: 2.104702), (index: 309, score: 1.869344),
[48 iters] min = 405.42ms max = 427.09ms median = 417.04ms mean = 416.71ms
Creating MNN Interpreter: mobilevitv2_200
(index: 985, score: 8.531386), (index: 883, score: 2.248808), (index: 309, score: 2.237848),
[38 iters] min = 516.46ms max = 537.36ms median = 527.45ms mean = 527.29ms
Creating MNN Interpreter: mobilevit_xx_small
(index: 985, score: 12.652774), (index: 309, score: 6.357562), (index: 308, score: 6.236053),
[415 iters] min = 47.08ms max = 48.79ms median = 48.33ms mean = 48.23ms
Creating MNN Interpreter: mobilevit_x_small
(index: 985, score: 12.998943), (index: 89, score: 6.411653), (index: 308, score: 5.775373),
[183 iters] min = 105.91ms max = 111.16ms median = 110.18ms mean = 109.85ms
Creating MNN Interpreter: mobilevit_small
(index: 985, score: 10.661409), (index: 838, score: 4.319293), (index: 309, score: 4.076161),
[118 iters] min = 164.35ms max = 171.28ms median = 170.29ms mean = 169.78ms
Creating MNN Interpreter: LeViT_128S
(index: 985, score: 11.427715), (index: 308, score: 3.451081), (index: 309, score: 3.319754),
[857 iters] min = 22.57ms max = 26.81ms median = 23.41ms mean = 23.36ms
Creating MNN Interpreter: LeViT_128
(index: 985, score: 11.089683), (index: 309, score: 3.409015), (index: 113, score: 3.385430),
[634 iters] min = 30.66ms max = 34.24ms median = 31.66ms mean = 31.60ms
Creating MNN Interpreter: LeViT_192
(index: 985, score: 11.594749), (index: 308, score: 3.186351), (index: 644, score: 3.177884),
[429 iters] min = 45.53ms max = 47.27ms median = 46.82ms mean = 46.72ms
Creating MNN Interpreter: LeViT_256
(index: 985, score: 11.363626), (index: 108, score: 3.341140), (index: 310, score: 2.929424),
[256 iters] min = 75.88ms max = 79.14ms median = 78.59ms mean = 78.38ms
Creating MNN Interpreter: resnet50
(index: 985, score: 7.495943), (index: 113, score: -4.947984), (index: 310, score: -5.267875),
[97 iters] min = 198.14ms max = 211.64ms median = 207.75ms mean = 207.29ms
Creating MNN Interpreter: mobilenetv3_large_100
(index: 985, score: 9.592710), (index: 308, score: 2.354276), (index: 310, score: 2.337051),
[970 iters] min = 19.87ms max = 23.51ms median = 20.66ms mean = 20.63ms
Creating MNN Interpreter: tf_efficientnetv2_b0
(index: 985, score: 9.555032), (index: 309, score: 2.378399), (index: 108, score: 2.289180),
[382 iters] min = 51.22ms max = 55.02ms median = 52.41ms mean = 52.44ms
Creating MNN Interpreter: tf_efficientnetv2_b1
(index: 985, score: 9.484729), (index: 861, score: 2.258651), (index: 309, score: 2.134489),
[250 iters] min = 77.32ms max = 91.73ms median = 80.07ms mean = 80.25ms
Creating MNN Interpreter: tf_efficientnetv2_b2
(index: 985, score: 9.816973), (index: 883, score: 2.518728), (index: 113, score: 2.046238),
[176 iters] min = 111.19ms max = 114.97ms median = 114.26ms mean = 114.01ms
Creating MNN Interpreter: tf_efficientnetv2_b3
(index: 985, score: 9.089290), (index: 955, score: 2.892854), (index: 947, score: 2.188154),
[102 iters] min = 190.57ms max = 198.25ms median = 196.82ms mean = 196.21mscortex-A78 @ 1 thread @ 2.2GHz w/ fp32 + UnaryOpOperation_GELU_STANDARD
$ MODEL=ALL make run-mnn-perf
INFO: Using CPU backend
INFO: Using num_threads == 1
The device support i8sdot:1, support fp16:1, support i8mm: 0
Creating MNN Interpreter: efficientformerv2_s0
(index: 985, score: 11.719770), (index: 644, score: 4.952458), (index: 309, score: 3.830817),
[219 iters] min = 90.68ms max = 91.94ms median = 91.44ms mean = 91.45ms
Creating MNN Interpreter: efficientformerv2_s1
(index: 985, score: 13.295982), (index: 984, score: 4.359047), (index: 308, score: 4.301670),
[146 iters] min = 136.95ms max = 138.22ms median = 137.85ms mean = 137.84ms
Creating MNN Interpreter: efficientformerv2_s2
(index: 985, score: 12.611839), (index: 22, score: 3.942058), (index: 309, score: 3.607175),
[89 iters] min = 223.65ms max = 225.60ms median = 224.93ms mean = 224.95ms
Creating MNN Interpreter: SwiftFormer_XS
(index: 985, score: 11.778864), (index: 883, score: 4.877996), (index: 309, score: 4.723835),
[201 iters] min = 98.87ms max = 100.34ms median = 99.70ms mean = 99.72ms
Creating MNN Interpreter: SwiftFormer_S
(index: 985, score: 13.011803), (index: 720, score: 4.258768), (index: 89, score: 4.246983),
[146 iters] min = 136.87ms max = 138.32ms median = 137.85ms mean = 137.83ms
Creating MNN Interpreter: SwiftFormer_L1
(index: 985, score: 13.598293), (index: 310, score: 4.220455), (index: 309, score: 3.997333),
[100 iters] min = 198.23ms max = 200.84ms median = 200.16ms mean = 200.16ms
Creating MNN Interpreter: EMO_1M
(index: 985, score: 9.830594), (index: 309, score: 4.371325), (index: 310, score: 3.886370),
[488 iters] min = 40.68ms max = 45.53ms median = 41.02ms mean = 41.05ms
Creating MNN Interpreter: EMO_2M
(index: 985, score: 9.485299), (index: 309, score: 3.385174), (index: 308, score: 3.217845),
[328 iters] min = 60.78ms max = 61.57ms median = 61.02ms mean = 61.03ms
Creating MNN Interpreter: EMO_5M
(index: 985, score: 9.178064), (index: 883, score: 2.810544), (index: 872, score: 2.548738),
[182 iters] min = 109.50ms max = 111.09ms median = 109.92ms mean = 109.99ms
Creating MNN Interpreter: EMO_6M
(index: 985, score: 9.283857), (index: 309, score: 2.281762), (index: 308, score: 2.275626),
[170 iters] min = 117.72ms max = 119.00ms median = 118.06ms mean = 118.09ms
Creating MNN Interpreter: edgenext_xx_small
(index: 985, score: 10.566173), (index: 309, score: 5.252524), (index: 310, score: 4.913792),
[401 iters] min = 49.46ms max = 50.37ms median = 49.94ms mean = 49.96ms
Creating MNN Interpreter: edgenext_x_small
(index: 985, score: 9.699040), (index: 309, score: 4.417048), (index: 308, score: 3.542260),
[201 iters] min = 99.30ms max = 100.30ms median = 99.82ms mean = 99.83ms
Creating MNN Interpreter: edgenext_small
(index: 985, score: 12.120678), (index: 309, score: 4.450402), (index: 308, score: 3.965264),
[112 iters] min = 177.46ms max = 180.01ms median = 179.21ms mean = 179.23ms
Creating MNN Interpreter: mobilevitv2_050
(index: 985, score: 8.414263), (index: 309, score: 2.655451), (index: 89, score: 2.475034),
[392 iters] min = 50.65ms max = 51.76ms median = 51.04ms mean = 51.05ms
Creating MNN Interpreter: mobilevitv2_075
(index: 985, score: 8.283899), (index: 309, score: 2.720386), (index: 308, score: 2.143100),
[207 iters] min = 96.14ms max = 97.64ms median = 96.63ms mean = 96.68ms
Creating MNN Interpreter: mobilevitv2_100
(index: 985, score: 8.259032), (index: 557, score: 2.323329), (index: 309, score: 2.103631),
[128 iters] min = 153.92ms max = 158.24ms median = 156.86ms mean = 156.89ms
Creating MNN Interpreter: mobilevitv2_125
(index: 985, score: 8.478145), (index: 309, score: 2.082687), (index: 113, score: 1.427791),
[88 iters] min = 227.73ms max = 230.29ms median = 228.36ms mean = 228.47ms
Creating MNN Interpreter: mobilevitv2_150
(index: 985, score: 9.081184), (index: 308, score: 2.288956), (index: 301, score: 2.262746),
[65 iters] min = 305.85ms max = 313.83ms median = 312.23ms mean = 312.19ms
Creating MNN Interpreter: mobilevitv2_175
(index: 985, score: 8.934433), (index: 494, score: 2.101466), (index: 309, score: 1.884507),
[49 iters] min = 408.15ms max = 411.20ms median = 409.12ms mean = 409.19ms
Creating MNN Interpreter: mobilevitv2_200
(index: 985, score: 8.606405), (index: 309, score: 2.243204), (index: 308, score: 2.195781),
[39 iters] min = 510.52ms max = 522.30ms median = 521.04ms mean = 520.82ms
Creating MNN Interpreter: mobilevit_xx_small
(index: 985, score: 12.430620), (index: 309, score: 6.490894), (index: 308, score: 6.247855),
[427 iters] min = 46.59ms max = 47.63ms median = 46.87ms mean = 46.89ms
Creating MNN Interpreter: mobilevit_x_small
(index: 985, score: 13.045833), (index: 89, score: 6.823301), (index: 309, score: 5.870656),
[188 iters] min = 105.82ms max = 108.24ms median = 106.73ms mean = 106.76ms
Creating MNN Interpreter: mobilevit_small
(index: 985, score: 10.438315), (index: 309, score: 3.712325), (index: 838, score: 3.708171),
[121 iters] min = 160.17ms max = 167.25ms median = 165.95ms mean = 165.98ms
Creating MNN Interpreter: LeViT_128S
(index: 985, score: 11.709266), (index: 308, score: 3.568025), (index: 309, score: 3.375846),
[771 iters] min = 25.76ms max = 26.27ms median = 25.93ms mean = 25.94ms
Creating MNN Interpreter: LeViT_128
(index: 985, score: 11.346602), (index: 309, score: 3.408503), (index: 113, score: 3.297331),
[570 iters] min = 34.02ms max = 35.45ms median = 35.16ms mean = 35.13ms
Creating MNN Interpreter: LeViT_192
(index: 985, score: 11.811327), (index: 324, score: 3.396983), (index: 326, score: 3.303845),
[390 iters] min = 51.14ms max = 51.65ms median = 51.40ms mean = 51.41ms
Creating MNN Interpreter: LeViT_256
(index: 985, score: 11.188661), (index: 108, score: 3.035138), (index: 309, score: 2.935835),
[237 iters] min = 84.30ms max = 85.13ms median = 84.46ms mean = 84.48ms
Creating MNN Interpreter: resnet50
(index: 985, score: 7.986818), (index: 113, score: -5.246407), (index: 310, score: -5.445824),
[97 iters] min = 207.26ms max = 208.85ms median = 207.59ms mean = 207.69ms
Creating MNN Interpreter: mobilenetv3_large_100
(index: 985, score: 9.726589), (index: 310, score: 2.717164), (index: 308, score: 2.388678),
[963 iters] min = 20.60ms max = 21.11ms median = 20.76ms mean = 20.77ms
Creating MNN Interpreter: tf_efficientnetv2_b0
(index: 985, score: 9.735810), (index: 309, score: 2.588191), (index: 310, score: 2.398264),
[394 iters] min = 49.53ms max = 51.21ms median = 50.78ms mean = 50.80ms
Creating MNN Interpreter: tf_efficientnetv2_b1
(index: 985, score: 9.687206), (index: 309, score: 2.282777), (index: 310, score: 2.219707),
[261 iters] min = 76.44ms max = 77.43ms median = 76.77ms mean = 76.79ms
Creating MNN Interpreter: tf_efficientnetv2_b2
(index: 985, score: 10.035254), (index: 883, score: 2.634550), (index: 309, score: 2.177393),
[181 iters] min = 107.74ms max = 111.66ms median = 110.81ms mean = 110.82ms
Creating MNN Interpreter: tf_efficientnetv2_b3
(index: 985, score: 9.174591), (index: 955, score: 2.843929), (index: 310, score: 2.220167),
[105 iters] min = 190.84ms max = 192.60ms median = 191.46ms mean = 191.52mscortex-A78 @ 1 thread @ 2.2GHz w/ fp16 (can use --fp16 storage to half model size)
-
--weightQuantBits 8 --weightQuantAsymmetricperformance is the same.
INFO: Using CPU backend
INFO: Using num_threads == 1
The device support i8sdot:1, support fp16:1, support i8mm: 0
Creating MNN Interpreter: efficientformerv2_s0
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[953 iters] min = 20.57ms max = 30.35ms median = 20.96ms mean = 21.00ms
Creating MNN Interpreter: efficientformerv2_s1
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[624 iters] min = 31.58ms max = 38.92ms median = 32.11ms mean = 32.10ms
Creating MNN Interpreter: efficientformerv2_s2
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[354 iters] min = 55.72ms max = 59.22ms median = 56.57ms mean = 56.62ms
Creating MNN Interpreter: SwiftFormer_XS
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[687 iters] min = 28.45ms max = 33.06ms median = 29.08ms mean = 29.12ms
Creating MNN Interpreter: SwiftFormer_S
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[475 iters] min = 41.45ms max = 42.61ms median = 42.21ms mean = 42.16ms
Creating MNN Interpreter: SwiftFormer_L1
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[315 iters] min = 62.73ms max = 64.32ms median = 63.74ms mean = 63.68ms
Creating MNN Interpreter: EMO_1M
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[749 iters] min = 26.10ms max = 27.16ms median = 26.74ms mean = 26.72ms
Creating MNN Interpreter: EMO_2M
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[520 iters] min = 37.51ms max = 44.80ms median = 38.49ms mean = 38.46ms
Creating MNN Interpreter: EMO_5M
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[325 iters] min = 60.24ms max = 69.30ms median = 61.68ms mean = 61.68ms
Creating MNN Interpreter: EMO_6M
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[306 iters] min = 64.01ms max = 70.23ms median = 65.38ms mean = 65.56ms
Creating MNN Interpreter: edgenext_xx_small
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[1279 iters] min = 15.19ms max = 19.02ms median = 15.60ms mean = 15.64ms
Creating MNN Interpreter: edgenext_x_small
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[682 iters] min = 28.83ms max = 29.79ms median = 29.40ms mean = 29.36ms
Creating MNN Interpreter: edgenext_small
(index: 999, score: nan), (index: 998, score: nan), (index: 997, score: nan),
[351 iters] min = 56.36ms max = 57.64ms median = 57.17ms mean = 57.11ms
Creating MNN Interpreter: mobilevitv2_050
(index: 985, score: 8.265625), (index: 309, score: 2.619141), (index: 584, score: 2.359375),
[514 iters] min = 38.00ms max = 42.36ms median = 38.91ms mean = 38.97ms
Creating MNN Interpreter: mobilevitv2_075
(index: 985, score: 8.171875), (index: 309, score: 2.402344), (index: 308, score: 1.894531),
[293 iters] min = 66.79ms max = 71.10ms median = 68.32ms mean = 68.36ms
Creating MNN Interpreter: mobilevitv2_100
(index: 985, score: 8.234375), (index: 557, score: 2.238281), (index: 309, score: 1.943359),
[192 iters] min = 101.93ms max = 108.67ms median = 103.96ms mean = 104.29ms
Creating MNN Interpreter: mobilevitv2_125
(index: 985, score: 8.296875), (index: 309, score: 1.988281), (index: 883, score: 1.286133),
[138 iters] min = 142.87ms max = 146.51ms median = 145.68ms mean = 145.48ms
Creating MNN Interpreter: mobilevitv2_150
(index: 985, score: 8.992188), (index: 308, score: 2.257812), (index: 301, score: 2.146484),
[104 iters] min = 189.62ms max = 196.12ms median = 193.91ms mean = 193.58ms
Creating MNN Interpreter: mobilevitv2_175
(index: 985, score: 8.890625), (index: 494, score: 2.093750), (index: 309, score: 1.869141),
[81 iters] min = 242.85ms max = 249.15ms median = 247.81ms mean = 247.38ms
Creating MNN Interpreter: mobilevitv2_200
(index: 985, score: 8.531250), (index: 883, score: 2.261719), (index: 309, score: 2.226562),
[66 iters] min = 302.29ms max = 308.83ms median = 307.58ms mean = 307.12ms
Creating MNN Interpreter: mobilevit_xx_small
(index: 985, score: 12.679688), (index: 309, score: 6.367188), (index: 308, score: 6.210938),
[536 iters] min = 36.68ms max = 37.92ms median = 37.42ms mean = 37.36ms
Creating MNN Interpreter: mobilevit_x_small
(index: 985, score: 13.031250), (index: 89, score: 6.449219), (index: 951, score: 5.757812),
[250 iters] min = 78.50ms max = 80.79ms median = 80.21ms mean = 80.08ms
Creating MNN Interpreter: mobilevit_small
(index: 985, score: 10.585938), (index: 838, score: 4.250000), (index: 309, score: 4.039062),
[174 iters] min = 112.81ms max = 115.94ms median = 115.15ms mean = 114.98ms
Creating MNN Interpreter: LeViT_128S
(index: 985, score: 11.468750), (index: 308, score: 3.488281), (index: 309, score: 3.349609),
[1527 iters] min = 12.65ms max = 13.30ms median = 13.15ms mean = 13.10ms
Creating MNN Interpreter: LeViT_128
(index: 985, score: 11.132812), (index: 309, score: 3.425781), (index: 113, score: 3.382812),
[1096 iters] min = 17.50ms max = 29.42ms median = 18.14ms mean = 18.25ms
Creating MNN Interpreter: LeViT_192
(index: 985, score: 11.578125), (index: 644, score: 3.214844), (index: 326, score: 3.183594),
[807 iters] min = 24.25ms max = 25.08ms median = 24.86ms mean = 24.81ms
Creating MNN Interpreter: LeViT_256
(index: 985, score: 11.375000), (index: 108, score: 3.238281), (index: 309, score: 2.873047),
[490 iters] min = 39.83ms max = 43.18ms median = 40.94ms mean = 40.87ms
Creating MNN Interpreter: resnet50
(index: 985, score: 7.496094), (index: 113, score: -4.964844), (index: 310, score: -5.296875),
[197 iters] min = 98.14ms max = 109.60ms median = 101.67ms mean = 101.80ms
Creating MNN Interpreter: mobilenetv3_large_100
(index: 985, score: 9.578125), (index: 308, score: 2.345703), (index: 310, score: 2.337891),
[1764 iters] min = 10.85ms max = 13.73ms median = 11.30ms mean = 11.34ms
Creating MNN Interpreter: tf_efficientnetv2_b0
(index: 985, score: 9.507812), (index: 309, score: 2.367188), (index: 108, score: 2.281250),
[572 iters] min = 34.47ms max = 37.20ms median = 34.97ms mean = 35.01ms
Creating MNN Interpreter: tf_efficientnetv2_b1
(index: 985, score: 9.484375), (index: 861, score: 2.271484), (index: 309, score: 2.140625),
[379 iters] min = 52.19ms max = 56.58ms median = 52.85ms mean = 52.87ms
Creating MNN Interpreter: tf_efficientnetv2_b2
(index: 985, score: 9.875000), (index: 883, score: 2.527344), (index: 113, score: 2.044922),
[268 iters] min = 73.53ms max = 78.85ms median = 74.58ms mean = 74.68ms
Creating MNN Interpreter: tf_efficientnetv2_b3
(index: 985, score: 9.109375), (index: 955, score: 2.865234), (index: 947, score: 2.171875),
[159 iters] min = 123.70ms max = 129.68ms median = 125.65ms mean = 125.99msGPU Vulkan @ 1.3GHz
$ BACK=v MODEL=ALL make run-mnn-perf
INFO: Using Vulkan backend
INFO: Using num_threads == 1
The device support i8sdot:1, support fp16:1, support i8mm: 0
<<<<<<<<<
Creating MNN Interpreter: efficientformerv2_s0
(index: 985, score: 9.117188), (index: 309, score: 4.136719), (index: 644, score: 4.027344),
[179 iters] min = 110.92ms max = 115.87ms median = 111.88ms mean = 112.07ms
Creating MNN Interpreter: efficientformerv2_s1
(index: 985, score: 10.796875), (index: 308, score: 4.867188), (index: 309, score: 4.363281),
[127 iters] min = 156.31ms max = 161.35ms median = 157.50ms mean = 157.59ms
Creating MNN Interpreter: efficientformerv2_s2
(index: 985, score: 11.992188), (index: 22, score: 3.699219), (index: 309, score: 3.423828),
[84 iters] min = 236.00ms max = 243.11ms median = 237.78ms mean = 238.19ms
>>>>>>>>>
Creating MNN Interpreter: efficientformerv2_s0
(index: 985, score: 11.734375), (index: 644, score: 4.859375), (index: 309, score: 3.833984),
[173 iters] min = 113.67ms max = 119.33ms median = 115.60ms mean = 115.63ms
Creating MNN Interpreter: efficientformerv2_s1
(index: 985, score: 13.312500), (index: 984, score: 4.417969), (index: 308, score: 4.378906),
[124 iters] min = 159.88ms max = 166.59ms median = 161.23ms mean = 161.67ms
Creating MNN Interpreter: efficientformerv2_s2
(index: 985, score: 12.609375), (index: 22, score: 3.974609), (index: 309, score: 3.585938),
[82 iters] min = 242.32ms max = 250.37ms median = 244.14ms mean = 244.47ms
Creating MNN Interpreter: SwiftFormer_XS
(index: 985, score: 11.757812), (index: 883, score: 4.910156), (index: 309, score: 4.730469),
[239 iters] min = 83.57ms max = 85.19ms median = 83.89ms mean = 83.95ms
Creating MNN Interpreter: SwiftFormer_S
(index: 985, score: 13.101562), (index: 89, score: 4.292969), (index: 720, score: 4.285156),
[192 iters] min = 104.09ms max = 108.60ms median = 104.40ms mean = 104.54ms
Creating MNN Interpreter: SwiftFormer_L1
(index: 985, score: 13.867188), (index: 310, score: 4.140625), (index: 309, score: 3.996094),
[147 iters] min = 135.99ms max = 138.52ms median = 136.83ms mean = 136.90ms
Creating MNN Interpreter: EMO_1M
(index: 985, score: 9.835938), (index: 309, score: 4.378906), (index: 310, score: 3.888672),
[387 iters] min = 49.67ms max = 59.22ms median = 51.51ms mean = 51.69ms
Creating MNN Interpreter: EMO_2M
(index: 985, score: 9.492188), (index: 309, score: 3.394531), (index: 308, score: 3.222656),
[302 iters] min = 64.44ms max = 71.66ms median = 66.32ms mean = 66.44ms
Creating MNN Interpreter: EMO_5M
(index: 985, score: 9.171875), (index: 883, score: 2.806641), (index: 872, score: 2.548828),
[239 iters] min = 81.96ms max = 88.99ms median = 83.53ms mean = 83.79ms
Creating MNN Interpreter: EMO_6M
(index: 985, score: 9.273438), (index: 309, score: 2.283203), (index: 308, score: 2.277344),
[225 iters] min = 87.04ms max = 94.01ms median = 88.65ms mean = 88.90ms
Creating MNN Interpreter: edgenext_xx_small
(index: 985, score: 10.546875), (index: 309, score: 5.246094), (index: 310, score: 4.902344),
[284 iters] min = 68.50ms max = 76.24ms median = 70.24ms mean = 70.52ms
Creating MNN Interpreter: edgenext_x_small
(index: 985, score: 9.687500), (index: 309, score: 4.414062), (index: 308, score: 3.535156),
[171 iters] min = 115.76ms max = 121.82ms median = 117.34ms mean = 117.48ms
Creating MNN Interpreter: edgenext_small
(index: 985, score: 12.406250), (index: 309, score: 4.640625), (index: 308, score: 4.371094),
[122 iters] min = 157.62ms max = 175.14ms median = 162.80ms mean = 163.96ms
Creating MNN Interpreter: mobilevitv2_050
(index: 985, score: 8.398438), (index: 309, score: 2.654297), (index: 89, score: 2.539062),
[496 iters] min = 33.90ms max = 51.00ms median = 38.67ms mean = 40.34ms
Creating MNN Interpreter: mobilevitv2_075
(index: 985, score: 8.257812), (index: 309, score: 2.718750), (index: 308, score: 2.146484),
[394 iters] min = 45.89ms max = 62.30ms median = 49.22ms mean = 50.81ms
Creating MNN Interpreter: mobilevitv2_100
(index: 985, score: 8.226562), (index: 557, score: 2.337891), (index: 309, score: 2.095703),
[327 iters] min = 57.19ms max = 66.93ms median = 60.99ms mean = 61.18ms
Creating MNN Interpreter: mobilevitv2_125
(index: 985, score: 8.453125), (index: 309, score: 2.078125), (index: 113, score: 1.408203),
[269 iters] min = 70.42ms max = 85.41ms median = 73.16ms mean = 74.41ms
Creating MNN Interpreter: mobilevitv2_150
(index: 985, score: 9.039062), (index: 308, score: 2.255859), (index: 301, score: 2.226562),
[231 iters] min = 80.24ms max = 98.53ms median = 85.65ms mean = 86.73ms
Creating MNN Interpreter: mobilevitv2_175
(index: 985, score: 8.945312), (index: 494, score: 2.107422), (index: 309, score: 1.891602),
[203 iters] min = 89.87ms max = 109.84ms median = 98.23ms mean = 98.67ms
Creating MNN Interpreter: mobilevitv2_200
(index: 985, score: 8.578125), (index: 309, score: 2.218750), (index: 308, score: 2.191406),
[142 iters] min = 131.16ms max = 163.60ms median = 141.97ms mean = 141.60ms
Creating MNN Interpreter: mobilevit_xx_small
(index: 985, score: 12.406250), (index: 309, score: 6.507812), (index: 308, score: 6.250000),
[135 iters] min = 116.13ms max = 221.18ms median = 123.97ms mean = 148.91ms
Creating MNN Interpreter: mobilevit_x_small
(index: 985, score: 13.023438), (index: 89, score: 6.863281), (index: 309, score: 5.894531),
[141 iters] min = 123.32ms max = 206.79ms median = 131.99ms mean = 142.50ms
Creating MNN Interpreter: mobilevit_small
(index: 985, score: 10.476562), (index: 309, score: 3.781250), (index: 838, score: 3.751953),
[125 iters] min = 138.60ms max = 205.27ms median = 157.19ms mean = 161.37ms
<<<<<<<<<<<<<<<<<<<<<
Creating MNN Interpreter: LeViT_128S
(index: 999, score: 6.093750), (index: 985, score: 5.539062), (index: 574, score: 5.121094),
[438 iters] min = 41.74ms max = 50.39ms median = 45.73ms mean = 45.70ms
Creating MNN Interpreter: LeViT_128
(index: 985, score: 7.890625), (index: 465, score: 6.074219), (index: 968, score: 5.867188),
[344 iters] min = 49.98ms max = 82.04ms median = 57.85ms mean = 58.27ms
Creating MNN Interpreter: LeViT_192
(index: 985, score: 8.789062), (index: 947, score: 6.441406), (index: 992, score: 5.175781),
[328 iters] min = 54.57ms max = 89.29ms median = 59.55ms mean = 61.05ms
Creating MNN Interpreter: LeViT_256
(index: 879, score: 8.812500), (index: 112, score: 7.667969), (index: 999, score: 7.066406),
[309 iters] min = 60.97ms max = 69.00ms median = 64.65ms mean = 64.87ms
>>>>>>>>>>>>>>>>>>>>>
Creating MNN Interpreter: LeViT_128S
(index: 985, score: 11.687500), (index: 308, score: 3.576172), (index: 309, score: 3.359375),
[461 iters] min = 34.84ms max = 74.11ms median = 38.83ms mean = 43.39ms
Creating MNN Interpreter: LeViT_128
(index: 985, score: 11.296875), (index: 309, score: 3.382812), (index: 113, score: 3.230469),
[379 iters] min = 48.90ms max = 70.43ms median = 52.41ms mean = 52.87ms
Creating MNN Interpreter: LeViT_192
(index: 985, score: 11.796875), (index: 324, score: 3.390625), (index: 326, score: 3.285156),
[327 iters] min = 49.94ms max = 96.41ms median = 54.76ms mean = 61.19ms
Creating MNN Interpreter: LeViT_256
(index: 985, score: 11.187500), (index: 108, score: 3.029297), (index: 309, score: 2.933594),
[327 iters] min = 58.28ms max = 71.09ms median = 60.76ms mean = 61.36ms
Creating MNN Interpreter: resnet50
(index: 985, score: 7.988281), (index: 113, score: -5.246094), (index: 310, score: -5.441406),
[1947 iters] min = 10.01ms max = 11.13ms median = 10.24ms mean = 10.27ms
Creating MNN Interpreter: mobilenetv3_large_100
(index: 985, score: 9.718750), (index: 310, score: 2.730469), (index: 308, score: 2.384766),
[2779 iters] min = 6.72ms max = 9.05ms median = 7.01ms mean = 7.20ms
Creating MNN Interpreter: tf_efficientnetv2_b0
(index: 985, score: 9.695312), (index: 309, score: 2.580078), (index: 310, score: 2.396484),
[1636 iters] min = 11.97ms max = 13.31ms median = 12.16ms mean = 12.23ms
Creating MNN Interpreter: tf_efficientnetv2_b1
(index: 985, score: 9.671875), (index: 309, score: 2.291016), (index: 310, score: 2.207031),
[1340 iters] min = 14.65ms max = 16.07ms median = 14.88ms mean = 14.93ms
Creating MNN Interpreter: tf_efficientnetv2_b2
(index: 985, score: 10.007812), (index: 883, score: 2.626953), (index: 309, score: 2.167969),
[1148 iters] min = 17.13ms max = 18.60ms median = 17.36ms mean = 17.44ms
Creating MNN Interpreter: tf_efficientnetv2_b3
(index: 985, score: 9.179688), (index: 955, score: 2.839844), (index: 310, score: 2.199219),
[859 iters] min = 22.92ms max = 24.96ms median = 23.27ms mean = 23.30msGPU Vulkan @ 0.61GHz
$ BACK=v MODEL=ALL make run-mnn-perf
INFO: Using Vulkan backend
INFO: Using num_threads == 1
The device support i8sdot:1, support fp16:1, support i8mm: 0
<<<<<<<<<<<<<<<<<
Creating MNN Interpreter: efficientformerv2_s0
(index: 985, score: 9.117188), (index: 309, score: 4.136719), (index: 644, score: 4.027344),
[156 iters] min = 126.73ms max = 134.99ms median = 128.45ms mean = 128.75ms
Creating MNN Interpreter: efficientformerv2_s1
(index: 985, score: 10.796875), (index: 308, score: 4.867188), (index: 309, score: 4.363281),
[112 iters] min = 175.67ms max = 185.72ms median = 178.31ms mean = 178.73ms
Creating MNN Interpreter: efficientformerv2_s2
(index: 985, score: 11.992188), (index: 22, score: 3.699219), (index: 309, score: 3.423828),
[74 iters] min = 266.09ms max = 290.87ms median = 272.02ms mean = 272.82ms
>>>>>>>>>>>>>>>>>
Creating MNN Interpreter: efficientformerv2_s0
(index: 985, score: 11.734375), (index: 644, score: 4.859375), (index: 309, score: 3.833984),
[143 iters] min = 136.85ms max = 147.87ms median = 139.47ms mean = 140.10ms
Creating MNN Interpreter: efficientformerv2_s1
(index: 985, score: 13.312500), (index: 984, score: 4.417969), (index: 308, score: 4.378906),
[105 iters] min = 187.65ms max = 199.54ms median = 190.41ms mean = 190.89ms
Creating MNN Interpreter: efficientformerv2_s2
(index: 985, score: 12.609375), (index: 22, score: 3.974609), (index: 309, score: 3.585938),
[69 iters] min = 286.85ms max = 297.49ms median = 289.92ms mean = 290.82ms
Creating MNN Interpreter: SwiftFormer_XS
(index: 985, score: 11.757812), (index: 883, score: 4.910156), (index: 309, score: 4.730469),
[206 iters] min = 96.10ms max = 108.22ms median = 96.64ms mean = 97.53ms
Creating MNN Interpreter: SwiftFormer_S
(index: 985, score: 13.101562), (index: 89, score: 4.292969), (index: 720, score: 4.285156),
[167 iters] min = 118.83ms max = 127.10ms median = 119.76ms mean = 119.95ms
Creating MNN Interpreter: SwiftFormer_L1
(index: 985, score: 13.867188), (index: 310, score: 4.140625), (index: 309, score: 3.996094),
[128 iters] min = 153.45ms max = 170.10ms median = 155.03ms mean = 156.81ms
Creating MNN Interpreter: EMO_1M
(index: 985, score: 9.835938), (index: 309, score: 4.378906), (index: 310, score: 3.888672),
[250 iters] min = 67.80ms max = 128.93ms median = 77.09ms mean = 80.08ms
Creating MNN Interpreter: EMO_2M
(index: 985, score: 9.492188), (index: 309, score: 3.394531), (index: 308, score: 3.222656),
[208 iters] min = 82.73ms max = 143.41ms median = 87.29ms mean = 96.27ms
Creating MNN Interpreter: EMO_5M
(index: 985, score: 9.171875), (index: 883, score: 2.806641), (index: 872, score: 2.548828),
[186 iters] min = 101.14ms max = 133.53ms median = 105.66ms mean = 108.13ms
Creating MNN Interpreter: EMO_6M
(index: 985, score: 9.273438), (index: 309, score: 2.283203), (index: 308, score: 2.277344),
[182 iters] min = 105.95ms max = 123.27ms median = 109.69ms mean = 110.12ms
Creating MNN Interpreter: edgenext_xx_small
(index: 985, score: 10.546875), (index: 309, score: 5.246094), (index: 310, score: 4.902344),
[235 iters] min = 81.34ms max = 95.19ms median = 84.72ms mean = 85.29ms
Creating MNN Interpreter: edgenext_x_small
(index: 985, score: 9.687500), (index: 309, score: 4.414062), (index: 308, score: 3.535156),
[144 iters] min = 133.16ms max = 152.52ms median = 137.95ms mean = 139.01ms
Creating MNN Interpreter: edgenext_small
(index: 985, score: 12.406250), (index: 309, score: 4.640625), (index: 308, score: 4.371094),
[108 iters] min = 180.82ms max = 201.55ms median = 185.21ms mean = 186.70ms
Creating MNN Interpreter: mobilevitv2_050
(index: 985, score: 8.398438), (index: 309, score: 2.654297), (index: 89, score: 2.539062),
[327 iters] min = 56.81ms max = 68.77ms median = 60.77ms mean = 61.19ms
Creating MNN Interpreter: mobilevitv2_075
(index: 985, score: 8.257812), (index: 309, score: 2.718750), (index: 308, score: 2.146484),
[250 iters] min = 75.62ms max = 85.16ms median = 79.70ms mean = 80.00ms
Creating MNN Interpreter: mobilevitv2_100
(index: 985, score: 8.226562), (index: 557, score: 2.337891), (index: 309, score: 2.095703),
[200 iters] min = 93.73ms max = 110.34ms median = 99.70ms mean = 100.23ms
Creating MNN Interpreter: mobilevitv2_125
(index: 985, score: 8.453125), (index: 309, score: 2.078125), (index: 113, score: 1.408203),
[167 iters] min = 114.47ms max = 132.93ms median = 119.09ms mean = 119.85ms
Creating MNN Interpreter: mobilevitv2_150
(index: 985, score: 9.039062), (index: 308, score: 2.255859), (index: 301, score: 2.226562),
[145 iters] min = 133.62ms max = 144.00ms median = 138.50ms mean = 138.66ms
Creating MNN Interpreter: mobilevitv2_175
(index: 985, score: 8.945312), (index: 494, score: 2.107422), (index: 309, score: 1.891602),
[124 iters] min = 153.27ms max = 175.51ms median = 161.29ms mean = 162.10ms
Creating MNN Interpreter: mobilevitv2_200
(index: 985, score: 8.578125), (index: 309, score: 2.218750), (index: 308, score: 2.191406),
[90 iters] min = 205.48ms max = 261.42ms median = 224.03ms mean = 223.98ms
Creating MNN Interpreter: mobilevit_xx_small
(index: 985, score: 12.406250), (index: 309, score: 6.507812), (index: 308, score: 6.250000),
[102 iters] min = 184.41ms max = 207.50ms median = 197.42ms mean = 197.72ms
Creating MNN Interpreter: mobilevit_x_small
(index: 985, score: 13.023438), (index: 89, score: 6.863281), (index: 309, score: 5.894531),
[96 iters] min = 190.55ms max = 221.82ms median = 209.26ms mean = 209.54ms
Creating MNN Interpreter: mobilevit_small
(index: 985, score: 10.476562), (index: 309, score: 3.781250), (index: 838, score: 3.751953),
[85 iters] min = 227.10ms max = 334.69ms median = 232.88ms mean = 239.03ms
<<<<<<<<<<<<<<<<<<
Creating MNN Interpreter: LeViT_128S
(index: 999, score: 6.093750), (index: 985, score: 5.539062), (index: 574, score: 5.121094),
[282 iters] min = 61.19ms max = 78.26ms median = 71.54ms mean = 71.14ms
Creating MNN Interpreter: LeViT_128
(index: 985, score: 7.890625), (index: 465, score: 6.074219), (index: 968, score: 5.867188),
[219 iters] min = 75.51ms max = 103.14ms median = 91.57ms mean = 91.39ms
Creating MNN Interpreter: LeViT_192
(index: 985, score: 8.789062), (index: 947, score: 6.441406), (index: 992, score: 5.175781),
[210 iters] min = 81.58ms max = 113.64ms median = 95.53ms mean = 95.44ms
Creating MNN Interpreter: LeViT_256
(index: 879, score: 8.812500), (index: 112, score: 7.667969), (index: 999, score: 7.066406),
[195 iters] min = 95.83ms max = 107.51ms median = 103.32ms mean = 103.02ms
>>>>>>>>>>>>>>>>>>
Creating MNN Interpreter: LeViT_128S
(index: 985, score: 11.687500), (index: 308, score: 3.576172), (index: 309, score: 3.359375),
[303 iters] min = 54.08ms max = 94.00ms median = 66.33ms mean = 66.21ms
Creating MNN Interpreter: LeViT_128
(index: 985, score: 11.296875), (index: 309, score: 3.382812), (index: 113, score: 3.230469),
[215 iters] min = 70.95ms max = 133.15ms median = 88.02ms mean = 93.22ms
Creating MNN Interpreter: LeViT_192
(index: 985, score: 11.796875), (index: 324, score: 3.390625), (index: 326, score: 3.285156),
[204 iters] min = 82.21ms max = 135.07ms median = 91.74ms mean = 98.32ms
Creating MNN Interpreter: LeViT_256
(index: 985, score: 11.187500), (index: 108, score: 3.029297), (index: 309, score: 2.933594),
[200 iters] min = 88.61ms max = 107.25ms median = 100.74ms mean = 100.36ms
Creating MNN Interpreter: resnet50
(index: 985, score: 7.988281), (index: 113, score: -5.246094), (index: 310, score: -5.441406),
[1116 iters] min = 17.63ms max = 20.70ms median = 17.87ms mean = 17.93ms
Creating MNN Interpreter: mobilenetv3_large_100
(index: 985, score: 9.718750), (index: 310, score: 2.730469), (index: 308, score: 2.384766),
[1726 iters] min = 11.32ms max = 12.66ms median = 11.54ms mean = 11.59ms
Creating MNN Interpreter: tf_efficientnetv2_b0
(index: 985, score: 9.695312), (index: 309, score: 2.580078), (index: 310, score: 2.396484),
[980 iters] min = 20.00ms max = 21.66ms median = 20.42ms mean = 20.41ms
Creating MNN Interpreter: tf_efficientnetv2_b1
(index: 985, score: 9.671875), (index: 309, score: 2.291016), (index: 310, score: 2.207031),
[797 iters] min = 24.74ms max = 28.12ms median = 25.07ms mean = 25.09ms
Creating MNN Interpreter: tf_efficientnetv2_b2
(index: 985, score: 10.007812), (index: 883, score: 2.626953), (index: 309, score: 2.167969),
[685 iters] min = 28.80ms max = 30.68ms median = 29.19ms mean = 29.20ms
Creating MNN Interpreter: tf_efficientnetv2_b3
(index: 985, score: 9.179688), (index: 955, score: 2.839844), (index: 310, score: 2.199219),
[506 iters] min = 39.04ms max = 43.04ms median = 39.56ms mean = 39.56mscortex-A78 @ 1 thread @ 2.2GHz w/ 2.5.0 conversion+quantization int8 + latest(10/13) runtime fake performance
INFO: Using CPU backend
INFO: Using num_threads == 1
The device support i8sdot:1, support fp16:1, support i8mm: 0
Creating MNN Interpreter: efficientformerv2_s0
efficientformerv2_s0 model doesn't exist!!!
Creating MNN Interpreter: efficientformerv2_s1
efficientformerv2_s1 model doesn't exist!!!
Creating MNN Interpreter: efficientformerv2_s2
efficientformerv2_s2 model doesn't exist!!!
[ - ] efficientformerv2_s0.mnn max = 29.616 ms min = 29.181 ms avg = 29.420 ms
[ - ] efficientformerv2_s1.mnn max = 42.532 ms min = 42.056 ms avg = 42.373 ms
[ - ] efficientformerv2_s2.mnn max = 69.602 ms min = 68.106 ms avg = 69.256 ms
Creating MNN Interpreter: SwiftFormer_XS
(index: 990, score: 19.464653), (index: 988, score: 19.464653), (index: 986, score: 19.464653),
[581 iters] min = 34.12ms max = 38.63ms median = 34.45ms mean = 34.46ms
Creating MNN Interpreter: SwiftFormer_S
(index: 990, score: 15.304254), (index: 988, score: 15.304254), (index: 986, score: 15.304254),
[443 iters] min = 44.49ms max = 48.43ms median = 45.17ms mean = 45.18ms
Creating MNN Interpreter: SwiftFormer_L1
(index: 990, score: 15.005185), (index: 988, score: 15.005185), (index: 986, score: 15.005185),
[319 iters] min = 62.53ms max = 67.46ms median = 62.77ms mean = 62.84ms
Creating MNN Interpreter: EMO_1M
(index: 969, score: 11.736819), (index: 862, score: 9.862873), (index: 800, score: 9.566987),
[556 iters] min = 35.80ms max = 36.20ms median = 36.00ms mean = 36.01ms
Creating MNN Interpreter: EMO_2M
(index: 796, score: 6.245563), (index: 753, score: 5.189974), (index: 639, score: 5.189974),
[395 iters] min = 50.34ms max = 54.41ms median = 50.65ms mean = 50.66ms
Creating MNN Interpreter: EMO_5M
(index: 676, score: 8.427950), (index: 474, score: 7.853317), (index: 806, score: 7.661772),
[277 iters] min = 71.80ms max = 76.91ms median = 72.28ms mean = 72.29ms
Creating MNN Interpreter: EMO_6M
(index: 115, score: 6.597718), (index: 517, score: 6.503464), (index: 611, score: 6.126452),
[257 iters] min = 77.13ms max = 78.43ms median = 78.10ms mean = 78.07ms
Creating MNN Interpreter: edgenext_xx_small
(index: 714, score: 4.785457), (index: 743, score: 4.350416), (index: 755, score: 3.915374),
[829 iters] min = 23.87ms max = 24.35ms median = 24.15ms mean = 24.14ms
Creating MNN Interpreter: edgenext_x_small
(index: 691, score: 4.221617), (index: 655, score: 4.127803), (index: 712, score: 3.846362),
[471 iters] min = 41.87ms max = 42.72ms median = 42.51ms mean = 42.50ms
Creating MNN Interpreter: edgenext_small
(index: 831, score: 3.955293), (index: 624, score: 3.729276), (index: 868, score: 3.390251),
[281 iters] min = 70.97ms max = 71.59ms median = 71.33ms mean = 71.33ms
Creating MNN Interpreter: mobilevitv2_050
(index: 843, score: 11.173376), (index: 158, score: 10.733479), (index: 128, score: 10.205604),
[465 iters] min = 42.82ms max = 43.32ms median = 43.09ms mean = 43.09ms
Creating MNN Interpreter: mobilevitv2_075
(index: 91, score: 8.383656), (index: 749, score: 6.529578), (index: 21, score: 6.448966),
[289 iters] min = 68.11ms max = 73.89ms median = 69.22ms mean = 69.26ms
Creating MNN Interpreter: mobilevitv2_100
(index: 141, score: 9.741990), (index: 273, score: 9.417257), (index: 459, score: 9.092525),
[202 iters] min = 98.36ms max = 100.52ms median = 99.09ms mean = 99.09ms
Creating MNN Interpreter: mobilevitv2_125
(index: 868, score: 10.849152), (index: 808, score: 10.849152), (index: 778, score: 10.849152),
[152 iters] min = 129.87ms max = 132.88ms median = 131.81ms mean = 131.78ms
Creating MNN Interpreter: mobilevitv2_150
(index: 783, score: 13.499125), (index: 899, score: 11.585863), (index: 733, score: 11.266986),
[120 iters] min = 166.82ms max = 170.66ms median = 167.97ms mean = 167.99ms
Creating MNN Interpreter: mobilevitv2_175
(index: 742, score: 10.263430), (index: 544, score: 10.004688), (index: 613, score: 9.918441),
[97 iters] min = 206.49ms max = 208.94ms median = 207.49ms mean = 207.56ms
Creating MNN Interpreter: mobilevitv2_200
(index: 808, score: 7.822464), (index: 861, score: 7.356842), (index: 840, score: 7.263717),
[81 iters] min = 248.90ms max = 251.68ms median = 249.97ms mean = 249.99ms
Creating MNN Interpreter: mobilevit_xx_small
(index: 822, score: 10.542213), (index: 955, score: 10.439862), (index: 921, score: 1.944680),
[283 iters] min = 70.33ms max = 71.90ms median = 70.75ms mean = 70.77ms
Creating MNN Interpreter: mobilevit_x_small
(index: 763, score: 8.259170), (index: 428, score: 7.589508), (index: 437, score: 6.585014),
[171 iters] min = 116.81ms max = 120.65ms median = 117.69ms mean = 117.63ms
Creating MNN Interpreter: mobilevit_small
(index: 599, score: 8.018028), (index: 747, score: 1.030889), (index: 537, score: -2.290865),
[134 iters] min = 149.60ms max = 151.46ms median = 150.37ms mean = 150.37ms
Creating MNN Interpreter: LeViT_128S
(index: 990, score: 14.561517), (index: 988, score: 14.561517), (index: 986, score: 14.561517),
[1093 iters] min = 18.13ms max = 21.66ms median = 18.26ms mean = 18.31ms
Creating MNN Interpreter: LeViT_128
(index: 990, score: 13.826856), (index: 988, score: 13.826856), (index: 986, score: 13.826856),
[719 iters] min = 27.75ms max = 27.99ms median = 27.85ms mean = 27.85ms
Creating MNN Interpreter: LeViT_192
(index: 990, score: 13.753602), (index: 988, score: 13.753602), (index: 986, score: 13.753602),
[650 iters] min = 30.62ms max = 31.02ms median = 30.79ms mean = 30.79ms
Creating MNN Interpreter: LeViT_256
(index: 990, score: 13.123007), (index: 988, score: 13.123007), (index: 986, score: 13.123007),
[450 iters] min = 44.35ms max = 44.70ms median = 44.51ms mean = 44.52ms
Creating MNN Interpreter: resnet50
(index: 999, score: 19.008062), (index: 998, score: 19.008062), (index: 997, score: 19.008062),
[265 iters] min = 75.50ms max = 75.83ms median = 75.66ms mean = 75.66ms
Creating MNN Interpreter: mobilenetv3_large_100
(index: 868, score: 2.960339), (index: 730, score: 2.552016), (index: 843, score: 2.143693),
[1499 iters] min = 13.02ms max = 13.58ms median = 13.35ms mean = 13.35ms
Creating MNN Interpreter: tf_efficientnetv2_b0
(index: 929, score: 11.324560), (index: 632, score: 11.324560), (index: 548, score: 11.324560),
[547 iters] min = 36.51ms max = 36.70ms median = 36.60ms mean = 36.60ms
Creating MNN Interpreter: tf_efficientnetv2_b1
(index: 701, score: 6.156499), (index: 177, score: 6.156499), (index: 871, score: 6.067274),
[352 iters] min = 55.93ms max = 60.37ms median = 56.92ms mean = 56.89ms
Creating MNN Interpreter: tf_efficientnetv2_b2
(index: 59, score: 6.689412), (index: 737, score: 6.410686), (index: 688, score: 6.317778),
[259 iters] min = 77.26ms max = 77.79ms median = 77.41ms mean = 77.41ms
Creating MNN Interpreter: tf_efficientnetv2_b3
(index: 678, score: 1.706579), (index: 584, score: 1.616759), (index: 904, score: 1.437119),
[157 iters] min = 126.77ms max = 129.96ms median = 127.43ms mean = 127.41mscommid id:
0afdc3b3ad1f5b3bea205ed3426ed2235481a3a7
cortex-A78 @ 1 thread @ 2.2GHz
$ MODEL=ALL make run-tnn-perf
INFO: Using CPU backend
INFO: Using num_threads == 1
Creating TNN net: efficientformerv2_s0
(index: 985, score: 11.718507), (index: 644, score: 4.943151), (index: 309, score: 3.836673),
[445 iters] min = 44.56ms max = 55.45ms median = 44.92ms mean = 45.01ms
Creating TNN net: efficientformerv2_s1
(index: 985, score: 13.267982), (index: 984, score: 4.345347), (index: 308, score: 4.289361),
[293 iters] min = 67.87ms max = 69.30ms median = 68.41ms mean = 68.40ms
Creating TNN net: efficientformerv2_s2
(index: 985, score: 12.624017), (index: 22, score: 3.935431), (index: 309, score: 3.621786),
[167 iters] min = 118.82ms max = 121.19ms median = 120.41ms mean = 120.40ms
SwiftFormer_XS model doesn't exist!!!
SwiftFormer_S model doesn't exist!!!
SwiftFormer_L1 model doesn't exist!!!
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
edgenext_xx_small model doesn't exist!!!
edgenext_x_small model doesn't exist!!!
edgenext_small model doesn't exist!!!
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
mobilevit_xx_small model doesn't exist!!!
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
Creating TNN net: LeViT_128S
(index: 985, score: 11.709329), (index: 308, score: 3.567969), (index: 309, score: 3.375843),
[571 iters] min = 34.71ms max = 35.40ms median = 35.04ms mean = 35.04ms
Creating TNN net: LeViT_128
(index: 985, score: 11.346685), (index: 309, score: 3.408514), (index: 113, score: 3.297297),
[416 iters] min = 46.24ms max = 48.65ms median = 48.20ms mean = 48.19ms
Creating TNN net: LeViT_192
(index: 985, score: 11.811436), (index: 324, score: 3.397015), (index: 326, score: 3.303873),
[308 iters] min = 64.83ms max = 65.39ms median = 65.02ms mean = 65.04ms
Creating TNN net: LeViT_256
(index: 985, score: 11.188828), (index: 108, score: 3.035164), (index: 309, score: 2.935831),
[198 iters] min = 100.88ms max = 102.12ms median = 101.21ms mean = 101.26ms
Creating TNN net: resnet50
(index: 985, score: 7.986876), (index: 113, score: -5.246382), (index: 310, score: -5.445831),
[93 iters] min = 212.65ms max = 217.18ms median = 216.20ms mean = 216.21ms
Creating TNN net: mobilenetv3_large_100
(index: 985, score: 9.726579), (index: 310, score: 2.717163), (index: 308, score: 2.388682),
[1042 iters] min = 19.11ms max = 19.48ms median = 19.19ms mean = 19.20ms
tf_efficientnetv2_b0 model doesn't exist!!!
tf_efficientnetv2_b1 model doesn't exist!!!
tf_efficientnetv2_b2 model doesn't exist!!!
tf_efficientnetv2_b3 model doesn't exist!!!commid id:
edea477553f595512fcbcaf9faf977494129f3d9
cortex-A78 @ 1 thread @ 2.2GHz
$ MODEL=ALL make run-pdlite-perf
INFO: Using CPU backend
INFO: Using num_threads == 1
Creating PaddlePredictor: efficientformerv2_s0
[I 9/20 16:25:39.297 ...ork/Paddle-Lite/lite/core/device_info.cc:283 get_cpu_arch] Unknow cpu arch: 3394
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1330 Setup] ARM multiprocessors name: MODEL NAME : ARMV8 PROCESSOR REV 1 (V8L)
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1331 Setup] ARM multiprocessors number: 12
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1333 Setup] ARM multiprocessors ID: 0, max freq: 2201, min freq: 2201, cluster ID: 0, CPU ARCH: A-1
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1339 Setup] L1 DataCache size is:
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1341 Setup] 64 KB
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1343 Setup] L2 Cache size is:
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1345 Setup] 256 KB
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1347 Setup] L3 Cache size is:
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1349 Setup] 2048 KB
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1351 Setup] Total memory: 31322864KB
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1352 Setup] SVE2 support: 0
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1353 Setup] SVE2 f32mm support: 0
[I 9/20 16:25:39.299 ...ork/Paddle-Lite/lite/core/device_info.cc:1354 Setup] SVE2 i8mm support: 0
(index: 985, score: 11.863880), (index: 644, score: 5.181920), (index: 309, score: 3.783359),
[382 iters] min = 51.10ms max = 52.86ms median = 52.46ms mean = 52.44ms
Creating PaddlePredictor: efficientformerv2_s1
(index: 985, score: 13.485666), (index: 984, score: 4.418849), (index: 308, score: 4.411816),
[252 iters] min = 79.01ms max = 80.02ms median = 79.63ms mean = 79.60ms
Creating PaddlePredictor: efficientformerv2_s2
(index: 985, score: 12.741438), (index: 22, score: 3.938634), (index: 80, score: 3.552274),
[147 iters] min = 135.97ms max = 137.32ms median = 136.88ms mean = 136.84ms
SwiftFormer_XS model doesn't exist!!!
SwiftFormer_S model doesn't exist!!!
SwiftFormer_L1 model doesn't exist!!!
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
Creating PaddlePredictor: edgenext_xx_small
[I 9/20 16:26:54.847 ...Paddle-Lite/lite/operators/squeeze_op.cc:124 AttachImpl] PaddleLiteV2.12 remove XShape OutputTensor for SqueezeOp.
(index: 985, score: 10.566197), (index: 309, score: 5.252448), (index: 310, score: 4.913684),
[328 iters] min = 60.09ms max = 61.70ms median = 61.14ms mean = 61.14ms
Creating PaddlePredictor: edgenext_x_small
[I 9/20 16:27:19.956 ...Paddle-Lite/lite/operators/squeeze_op.cc:124 AttachImpl] PaddleLiteV2.12 remove XShape OutputTensor for SqueezeOp.
(index: 985, score: 9.699077), (index: 309, score: 4.416987), (index: 308, score: 3.542241),
[187 iters] min = 106.66ms max = 108.06ms median = 107.48ms mean = 107.47ms
Creating PaddlePredictor: edgenext_small
[I 9/20 16:27:45.155 ...Paddle-Lite/lite/operators/squeeze_op.cc:124 AttachImpl] PaddleLiteV2.12 remove XShape OutputTensor for SqueezeOp.
(index: 985, score: 12.120754), (index: 309, score: 4.450460), (index: 308, score: 3.965276),
[103 iters] min = 190.68ms max = 196.24ms median = 195.26ms mean = 195.18ms
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
Creating PaddlePredictor: mobilevit_xx_small
(index: 985, score: 12.435376), (index: 309, score: 6.497385), (index: 308, score: 6.236424),
[339 iters] min = 58.55ms max = 59.35ms median = 59.02ms mean = 59.02ms
Creating PaddlePredictor: mobilevit_x_small
(index: 985, score: 13.047843), (index: 89, score: 6.821341), (index: 309, score: 5.869913),
[161 iters] min = 121.19ms max = 125.37ms median = 124.65ms mean = 124.64ms
Creating PaddlePredictor: mobilevit_small
(index: 985, score: 10.445730), (index: 309, score: 3.723504), (index: 838, score: 3.721817),
[104 iters] min = 191.60ms max = 193.43ms median = 192.65ms mean = 192.68ms
Creating PaddlePredictor: LeViT_128S
(index: 985, score: 11.709342), (index: 308, score: 3.568015), (index: 309, score: 3.375860),
[578 iters] min = 34.32ms max = 34.95ms median = 34.67ms mean = 34.66ms
Creating PaddlePredictor: LeViT_128
(index: 985, score: 11.346714), (index: 309, score: 3.408517), (index: 113, score: 3.297331),
[425 iters] min = 45.43ms max = 47.43ms median = 47.16ms mean = 47.15ms
Creating PaddlePredictor: LeViT_192
(index: 985, score: 11.811453), (index: 324, score: 3.397026), (index: 326, score: 3.303875),
[312 iters] min = 63.81ms max = 64.49ms median = 64.15ms mean = 64.15ms
Creating PaddlePredictor: LeViT_256
(index: 985, score: 11.188838), (index: 108, score: 3.035203), (index: 309, score: 2.935844),
[190 iters] min = 101.78ms max = 106.04ms median = 105.57ms mean = 105.53ms
Creating PaddlePredictor: resnet50
(index: 985, score: 7.986873), (index: 113, score: -5.246380), (index: 310, score: -5.445833),
[95 iters] min = 211.39ms max = 213.49ms median = 212.77ms mean = 212.75ms
Creating PaddlePredictor: mobilenetv3_large_100
(index: 985, score: 9.726584), (index: 310, score: 2.717168), (index: 308, score: 2.388681),
[933 iters] min = 21.24ms max = 21.82ms median = 21.45ms mean = 21.45ms
tf_efficientnetv2_b0 model doesn't exist!!!
tf_efficientnetv2_b1 model doesn't exist!!!
tf_efficientnetv2_b2 model doesn't exist!!!
tf_efficientnetv2_b3 model doesn't exist!!!commit id:
f12ce2113e361dac536d95d6f1b396048259a3c3
cortex-A78 @ 1 thread @ 2.2GHz tinynn fp32
Creating tflite runtime interpreter: efficientformerv2_s0
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
(index: 985, score: 13.800571), (index: 644, score: 6.719913), (index: 662, score: 4.357441),
[496 iters] min = 39.26ms max = 45.93ms median = 40.38ms mean = 40.32ms
Creating tflite runtime interpreter: efficientformerv2_s1
(index: 985, score: 11.002513), (index: 892, score: 5.913536), (index: 794, score: 5.847847),
[318 iters] min = 61.40ms max = 67.91ms median = 62.92ms mean = 62.91ms
Creating tflite runtime interpreter: efficientformerv2_s2
(index: 985, score: 14.202505), (index: 574, score: 5.209904), (index: 650, score: 4.950992),
[178 iters] min = 109.78ms max = 116.03ms median = 112.93ms mean = 112.67ms
Creating tflite runtime interpreter: SwiftFormer_XS
(index: 985, score: 16.348614), (index: 107, score: 8.219887), (index: 308, score: 7.765065),
[388 iters] min = 50.24ms max = 52.28ms median = 51.76ms mean = 51.61ms
Creating tflite runtime interpreter: SwiftFormer_S
(index: 985, score: 10.332458), (index: 309, score: 4.304403), (index: 507, score: 3.876854),
[259 iters] min = 75.85ms max = 79.64ms median = 77.50ms mean = 77.33ms
Creating tflite runtime interpreter: SwiftFormer_L1
(index: 985, score: 13.199683), (index: 310, score: 4.649525), (index: 309, score: 4.089820),
[165 iters] min = 118.61ms max = 131.96ms median = 122.02ms mean = 121.91ms
Creating tflite runtime interpreter: EMO_1M
(index: 985, score: 9.241662), (index: 328, score: 7.229661), (index: 619, score: 6.890592),
[605 iters] min = 32.05ms max = 33.61ms median = 33.16ms mean = 33.07ms
Creating tflite runtime interpreter: EMO_2M
(index: 985, score: 9.554611), (index: 493, score: 6.152699), (index: 310, score: 3.873620),
[398 iters] min = 48.84ms max = 55.44ms median = 50.32ms mean = 50.28ms
Creating tflite runtime interpreter: EMO_5M
(index: 985, score: 8.625879), (index: 794, score: 5.527351), (index: 108, score: 4.698321),
[223 iters] min = 87.47ms max = 94.14ms median = 89.88ms mean = 89.99ms
Creating tflite runtime interpreter: EMO_6M
(index: 985, score: 9.466664), (index: 446, score: 5.847961), (index: 885, score: 4.761618),
[209 iters] min = 93.31ms max = 98.58ms median = 96.13ms mean = 96.10ms
Creating tflite runtime interpreter: edgenext_xx_small
(index: 144, score: 5.642828), (index: 858, score: 5.068350), (index: 132, score: 5.017061),
[828 iters] min = 23.63ms max = 26.44ms median = 24.21ms mean = 24.18ms
Creating tflite runtime interpreter: edgenext_x_small
(index: 904, score: 9.758960), (index: 905, score: 8.679010), (index: 828, score: 7.538647),
[431 iters] min = 45.31ms max = 47.13ms median = 46.59ms mean = 46.48ms
Creating tflite runtime interpreter: edgenext_small
(index: 904, score: 6.315926), (index: 753, score: 5.907308), (index: 905, score: 5.188027),
[219 iters] min = 88.99ms max = 98.84ms median = 91.88ms mean = 91.74ms
Creating tflite runtime interpreter: mobilevitv2_050
(index: 905, score: 7.073186), (index: 688, score: 5.798759), (index: 530, score: 4.821216),
[511 iters] min = 38.22ms max = 43.16ms median = 39.19ms mean = 39.16ms
Creating tflite runtime interpreter: mobilevitv2_075
(index: 904, score: 6.283208), (index: 753, score: 6.140928), (index: 905, score: 5.674746),
[273 iters] min = 71.81ms max = 74.34ms median = 73.47ms mean = 73.33ms
Creating tflite runtime interpreter: mobilevitv2_100
(index: 904, score: 6.422704), (index: 753, score: 4.768656), (index: 905, score: 3.757758),
[168 iters] min = 116.33ms max = 122.07ms median = 119.08ms mean = 119.06ms
Creating tflite runtime interpreter: mobilevitv2_125
(index: 549, score: 4.407668), (index: 905, score: 4.018123), (index: 753, score: 3.660830),
[115 iters] min = 170.95ms max = 179.04ms median = 174.88ms mean = 174.86ms
Creating tflite runtime interpreter: mobilevitv2_150
(index: 904, score: 6.959464), (index: 905, score: 5.279493), (index: 556, score: 4.383082),
[83 iters] min = 237.53ms max = 245.24ms median = 242.62ms mean = 242.09ms
Creating tflite runtime interpreter: mobilevitv2_175
(index: 905, score: 7.720383), (index: 904, score: 6.793842), (index: 753, score: 6.216552),
[63 iters] min = 314.47ms max = 330.20ms median = 320.40ms mean = 319.85ms
Creating tflite runtime interpreter: mobilevitv2_200
(index: 904, score: 7.456795), (index: 905, score: 5.135217), (index: 556, score: 4.294895),
[49 iters] min = 401.54ms max = 415.42ms median = 409.83ms mean = 409.58ms
Creating tflite runtime interpreter: mobilevit_xx_small
(index: 905, score: 9.074959), (index: 581, score: 7.366666), (index: 530, score: 6.987810),
[594 iters] min = 32.76ms max = 36.20ms median = 33.74ms mean = 33.69ms
Creating tflite runtime interpreter: mobilevit_x_small
(index: 905, score: 8.960419), (index: 904, score: 7.904361), (index: 753, score: 6.592516),
[253 iters] min = 76.78ms max = 80.49ms median = 79.50ms mean = 79.22ms
Creating tflite runtime interpreter: mobilevit_small
(index: 904, score: 6.754923), (index: 905, score: 6.661246), (index: 858, score: 5.174719),
[152 iters] min = 127.86ms max = 132.81ms median = 131.96ms mean = 131.58ms
Creating tflite runtime interpreter: LeViT_128S
(index: 985, score: 8.677929), (index: 868, score: 8.497841), (index: 446, score: 7.851535),
[835 iters] min = 22.59ms max = 24.50ms median = 24.15ms mean = 23.96ms
Creating tflite runtime interpreter: LeViT_128
(index: 985, score: 9.898071), (index: 619, score: 5.420589), (index: 539, score: 4.751773),
[620 iters] min = 30.58ms max = 32.84ms median = 32.47ms mean = 32.28ms
Creating tflite runtime interpreter: LeViT_192
(index: 985, score: 10.134396), (index: 328, score: 7.682946), (index: 619, score: 6.307396),
[421 iters] min = 45.62ms max = 48.19ms median = 47.76ms mean = 47.51ms
Creating tflite runtime interpreter: LeViT_256
(index: 985, score: 8.615749), (index: 818, score: 6.255535), (index: 619, score: 6.188113),
[250 iters] min = 76.56ms max = 81.44ms median = 80.58ms mean = 80.10ms
Creating tflite runtime interpreter: resnet50
(index: 985, score: 7.842050), (index: 652, score: -3.425980), (index: 439, score: -4.438686),
[76 iters] min = 253.76ms max = 267.03ms median = 265.73ms mean = 264.38ms
Creating tflite runtime interpreter: mobilenetv3_large_100
(index: 112, score: 6.790292), (index: 985, score: 5.795063), (index: 591, score: 5.150109),
[1100 iters] min = 17.54ms max = 18.58ms median = 18.28ms mean = 18.20ms
tf_efficientnetv2_b0 model doesn't exist!!!
tf_efficientnetv2_b1 model doesn't exist!!!
tf_efficientnetv2_b2 model doesn't exist!!!
tf_efficientnetv2_b3 model doesn't exist!!!cortex-A78 @ 1 thread @ 2.2GHz tinynn dynamic int8
INFO: Using num_threads == 1
Creating tflite runtime interpreter: efficientformerv2_s0
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
(index: 985, score: 10.414447), (index: 473, score: 7.323678), (index: 243, score: 5.948439),
[593 iters] min = 32.53ms max = 44.28ms median = 33.72ms mean = 33.76ms
Creating tflite runtime interpreter: efficientformerv2_s1
(index: 985, score: 17.765656), (index: 89, score: 8.259610), (index: 512, score: 7.648966),
[410 iters] min = 47.47ms max = 59.42ms median = 48.97ms mean = 48.88ms
Creating tflite runtime interpreter: efficientformerv2_s2
(index: 985, score: 10.992670), (index: 335, score: 6.229464), (index: 451, score: 6.014560),
[255 iters] min = 76.14ms max = 79.42ms median = 78.72ms mean = 78.48ms
Creating tflite runtime interpreter: SwiftFormer_XS
(index: 985, score: 12.597744), (index: 883, score: 6.429352), (index: 584, score: 6.104802),
[515 iters] min = 37.38ms max = 39.57ms median = 39.04ms mean = 38.87ms
Creating tflite runtime interpreter: SwiftFormer_S
(index: 985, score: 13.702253), (index: 632, score: 7.228092), (index: 904, score: 5.929401),
[397 iters] min = 48.46ms max = 54.11ms median = 50.48ms mean = 50.42ms
Creating tflite runtime interpreter: SwiftFormer_L1
(index: 985, score: 13.422840), (index: 904, score: 10.359863), (index: 556, score: 5.901742),
[286 iters] min = 67.46ms max = 75.08ms median = 70.10ms mean = 69.98ms
Creating tflite runtime interpreter: EMO_1M
(index: 985, score: 10.772147), (index: 310, score: 4.265167), (index: 309, score: 4.034026),
[603 iters] min = 31.85ms max = 39.58ms median = 33.28ms mean = 33.21ms
Creating tflite runtime interpreter: EMO_2M
(index: 985, score: 9.784461), (index: 309, score: 3.289332), (index: 310, score: 3.090404),
[444 iters] min = 43.71ms max = 45.78ms median = 45.20ms mean = 45.08ms
Creating tflite runtime interpreter: EMO_5M
(index: 985, score: 8.009010), (index: 310, score: 2.295238), (index: 311, score: 2.265032),
[294 iters] min = 65.96ms max = 69.25ms median = 68.38ms mean = 68.16ms
Creating tflite runtime interpreter: EMO_6M
(index: 985, score: 9.536195), (index: 883, score: 2.864874), (index: 968, score: 2.779290),
[280 iters] min = 69.10ms max = 73.25ms median = 71.72ms mean = 71.44ms
Creating tflite runtime interpreter: edgenext_xx_small
(index: 144, score: 5.726498), (index: 132, score: 5.228430), (index: 858, score: 4.910030),
[1122 iters] min = 17.39ms max = 18.34ms median = 17.86ms mean = 17.84ms
Creating tflite runtime interpreter: edgenext_x_small
(index: 905, score: 7.151570), (index: 904, score: 5.593434), (index: 539, score: 5.189555),
[637 iters] min = 30.58ms max = 32.23ms median = 31.48ms mean = 31.40ms
Creating tflite runtime interpreter: edgenext_small
(index: 905, score: 7.833123), (index: 753, score: 5.813052), (index: 904, score: 4.813413),
[373 iters] min = 52.18ms max = 54.80ms median = 53.81ms mean = 53.64ms
Creating tflite runtime interpreter: mobilevitv2_050
(index: 905, score: 6.351662), (index: 688, score: 6.100896), (index: 811, score: 5.178001),
[566 iters] min = 34.10ms max = 36.31ms median = 35.51ms mean = 35.39ms
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
Creating tflite runtime interpreter: mobilevitv2_150
(index: 885, score: 4.623362), (index: 905, score: 3.750659), (index: 148, score: 3.582521),
[144 iters] min = 132.90ms max = 143.84ms median = 139.30ms mean = 139.02ms
Creating tflite runtime interpreter: mobilevitv2_175
(index: 885, score: 5.663316), (index: 854, score: 5.009308), (index: 144, score: 4.522904),
[117 iters] min = 165.05ms max = 174.03ms median = 172.78ms mean = 172.03ms
Creating tflite runtime interpreter: mobilevitv2_200
(index: 905, score: 6.786233), (index: 650, score: 5.424253), (index: 904, score: 4.502124),
[96 iters] min = 200.93ms max = 213.88ms median = 209.24ms mean = 208.78ms
Creating tflite runtime interpreter: mobilevit_xx_small
(index: 898, score: 7.621788), (index: 611, score: 7.552627), (index: 782, score: 7.434354),
[619 iters] min = 31.30ms max = 35.23ms median = 32.34ms mean = 32.34ms
Creating tflite runtime interpreter: mobilevit_x_small
(index: 905, score: 10.061533), (index: 904, score: 8.190913), (index: 818, score: 6.948528),
[285 iters] min = 66.80ms max = 76.21ms median = 70.54ms mean = 70.36ms
Creating tflite runtime interpreter: mobilevit_small
(index: 905, score: 8.444880), (index: 904, score: 3.538346), (index: 794, score: 3.359672),
[208 iters] min = 92.33ms max = 98.49ms median = 97.00ms mean = 96.50ms
Creating tflite runtime interpreter: LeViT_128S
(index: 985, score: 11.338488), (index: 744, score: 3.550334), (index: 309, score: 3.416615),
[1318 iters] min = 14.55ms max = 15.57ms median = 15.26ms mean = 15.18ms
Creating tflite runtime interpreter: LeViT_128
(index: 985, score: 11.076033), (index: 309, score: 3.194498), (index: 113, score: 3.029531),
[970 iters] min = 19.80ms max = 21.00ms median = 20.72ms mean = 20.62ms
Creating tflite runtime interpreter: LeViT_192
(index: 985, score: 11.501357), (index: 326, score: 3.302804), (index: 644, score: 3.170937),
[728 iters] min = 26.60ms max = 27.88ms median = 27.62ms mean = 27.50ms
Creating tflite runtime interpreter: LeViT_256
(index: 985, score: 11.314730), (index: 108, score: 3.116097), (index: 309, score: 3.107525),
[491 iters] min = 39.17ms max = 43.04ms median = 40.85ms mean = 40.73ms
Creating tflite runtime interpreter: resnet50
(index: 985, score: 7.214152), (index: 310, score: -4.907701), (index: 113, score: -4.919555),
[210 iters] min = 91.08ms max = 100.03ms median = 95.38ms mean = 95.24ms
Creating tflite runtime interpreter: mobilenetv3_large_100
(index: 985, score: 9.674332), (index: 308, score: 2.503336), (index: 883, score: 2.451423),
[1192 iters] min = 16.07ms max = 23.55ms median = 16.80ms mean = 16.79ms
tf_efficientnetv2_b0 model doesn't exist!!!
tf_efficientnetv2_b1 model doesn't exist!!!
tf_efficientnetv2_b2 model doesn't exist!!!
tf_efficientnetv2_b3 model doesn't exist!!!cortex-A78 @ 1 thread @ 2.2GHz tinynn ptq int8 *fake*
INFO: Using num_threads == 1
Creating tflite runtime interpreter: efficientformerv2_s0
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
(index: 990, score: 15.554351), (index: 957, score: 15.554351), (index: 956, score: 15.554351),
[634 iters] min = 30.69ms max = 32.47ms median = 31.60ms mean = 31.55ms
Creating tflite runtime interpreter: efficientformerv2_s1
(index: 910, score: 14.773200), (index: 892, score: 14.773200), (index: 641, score: 14.773200),
[427 iters] min = 45.55ms max = 48.61ms median = 46.99ms mean = 46.90ms
Creating tflite runtime interpreter: efficientformerv2_s2
(index: 819, score: 13.587024), (index: 818, score: 13.587024), (index: 785, score: 12.763567),
[265 iters] min = 73.84ms max = 77.41ms median = 75.73ms mean = 75.60ms
SwiftFormer_XS model doesn't exist!!!
SwiftFormer_S model doesn't exist!!!
SwiftFormer_L1 model doesn't exist!!!
Creating tflite runtime interpreter: EMO_1M
(index: 985, score: 11.310929), (index: 108, score: 6.861964), (index: 310, score: 5.504652),
[968 iters] min = 20.40ms max = 20.92ms median = 20.68ms mean = 20.66ms
Creating tflite runtime interpreter: EMO_2M
(index: 985, score: 9.640327), (index: 883, score: 3.666040), (index: 712, score: 3.530261),
[664 iters] min = 29.69ms max = 30.48ms median = 30.18ms mean = 30.15ms
Creating tflite runtime interpreter: EMO_5M
(index: 985, score: 5.184752), (index: 506, score: 3.923597), (index: 644, score: 3.573275),
[419 iters] min = 46.99ms max = 48.21ms median = 47.82ms mean = 47.75ms
Creating tflite runtime interpreter: EMO_6M
(index: 985, score: 6.443181), (index: 905, score: 3.692160), (index: 971, score: 3.474974),
[393 iters] min = 50.04ms max = 54.44ms median = 50.96ms mean = 50.97ms
edgenext_xx_small model doesn't exist!!!
edgenext_x_small model doesn't exist!!!
edgenext_small model doesn't exist!!!
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
mobilevit_xx_small model doesn't exist!!!
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
LeViT_128S model doesn't exist!!!
LeViT_128 model doesn't exist!!!
LeViT_192 model doesn't exist!!!
LeViT_256 model doesn't exist!!!
Creating tflite runtime interpreter: resnet50
(index: 985, score: 5.906343), (index: 310, score: -4.200066), (index: 308, score: -4.725074),
[257 iters] min = 76.13ms max = 80.59ms median = 78.01ms mean = 77.97ms
mobilenetv3_large_100 model doesn't exist!!!
tf_efficientnetv2_b0 model doesn't exist!!!
tf_efficientnetv2_b1 model doesn't exist!!!
tf_efficientnetv2_b2 model doesn't exist!!!
tf_efficientnetv2_b3 model doesn't exist!!!cortex-A78 @ 1 thread @ 2.2GHz onnx-tf dynamic int8, which will use QSymmS8
$ MODEL=ALL make run-tflite-perf
INFO: Using num_threads == 1
Creating tflite runtime interpreter: efficientformerv2_s0
INFO: Created TensorFlow Lite delegate for select TF ops.
INFO: TfLiteFlexDelegate delegate: 35 nodes delegated out of 831 nodes with 35 partitions.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
(index: 985, score: 12.656642), (index: 108, score: 5.208982), (index: 984, score: 4.614052),
[393 iters] min = 47.14ms max = 52.47ms median = 50.95ms mean = 50.95ms
Creating tflite runtime interpreter: efficientformerv2_s1
(index: 985, score: 14.401400), (index: 512, score: 7.133657), (index: 326, score: 6.300667),
[266 iters] min = 74.48ms max = 86.93ms median = 75.07ms mean = 75.41ms
Creating tflite runtime interpreter: efficientformerv2_s2
(index: 985, score: 12.294003), (index: 108, score: 4.097857), (index: 644, score: 4.059546),
[165 iters] min = 112.97ms max = 125.55ms median = 121.02ms mean = 121.25ms
Creating tflite runtime interpreter: SwiftFormer_XS
(index: 985, score: 11.656082), (index: 309, score: 4.865403), (index: 883, score: 4.670465),
[392 iters] min = 50.57ms max = 53.85ms median = 50.91ms mean = 51.04ms
Creating tflite runtime interpreter: SwiftFormer_S
(index: 985, score: 14.615710), (index: 720, score: 5.028010), (index: 89, score: 4.631391),
[305 iters] min = 65.05ms max = 69.67ms median = 65.52ms mean = 65.74ms
Creating tflite runtime interpreter: SwiftFormer_L1
(index: 985, score: 14.904846), (index: 310, score: 4.374226), (index: 309, score: 4.088094),
[219 iters] min = 85.55ms max = 92.39ms median = 91.46ms mean = 91.45ms
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
Creating tflite runtime interpreter: edgenext_xx_small
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#996 is a dynamic-sized tensor).
(index: 985, score: 10.574258), (index: 310, score: 5.127322), (index: 309, score: 4.813240),
[350 iters] min = 56.69ms max = 57.53ms median = 57.16ms mean = 57.15ms
Creating tflite runtime interpreter: edgenext_x_small
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#1489 is a dynamic-sized tensor).
(index: 985, score: 9.733109), (index: 309, score: 4.522947), (index: 308, score: 3.611678),
[190 iters] min = 101.79ms max = 114.17ms median = 105.26ms mean = 105.32ms
Creating tflite runtime interpreter: edgenext_small
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#1489 is a dynamic-sized tensor).
(index: 985, score: 12.168671), (index: 309, score: 4.481769), (index: 308, score: 3.972034),
[141 iters] min = 142.12ms max = 143.33ms median = 142.72ms mean = 142.74ms
Creating tflite runtime interpreter: mobilevitv2_050
(index: 985, score: 8.845340), (index: 309, score: 3.012393), (index: 89, score: 2.800360),
[368 iters] min = 53.70ms max = 57.60ms median = 54.28ms mean = 54.49ms
Creating tflite runtime interpreter: mobilevitv2_075
(index: 985, score: 8.063389), (index: 309, score: 2.647400), (index: 308, score: 2.150668),
[220 iters] min = 89.54ms max = 96.99ms median = 90.90ms mean = 91.03ms
Creating tflite runtime interpreter: mobilevitv2_100
(index: 985, score: 8.255550), (index: 557, score: 2.322341), (index: 309, score: 2.029139),
[153 iters] min = 130.78ms max = 133.82ms median = 131.38ms mean = 131.44ms
Creating tflite runtime interpreter: mobilevitv2_125
(index: 985, score: 8.210611), (index: 309, score: 2.192470), (index: 132, score: 1.299099),
[115 iters] min = 161.27ms max = 180.04ms median = 174.38ms mean = 174.96ms
Creating tflite runtime interpreter: mobilevitv2_150
(index: 985, score: 8.838170), (index: 308, score: 2.144078), (index: 301, score: 2.063063),
[89 iters] min = 222.64ms max = 229.95ms median = 225.57ms mean = 225.70ms
Creating tflite runtime interpreter: mobilevitv2_175
(index: 985, score: 8.673342), (index: 309, score: 2.117110), (index: 494, score: 1.783873),
[73 iters] min = 268.11ms max = 280.99ms median = 277.46ms mean = 277.76ms
Creating tflite runtime interpreter: mobilevitv2_200
(index: 985, score: 8.597152), (index: 309, score: 2.462746), (index: 883, score: 2.262056),
[60 iters] min = 334.46ms max = 337.80ms median = 336.17ms mean = 336.13ms
Creating tflite runtime interpreter: mobilevit_xx_small
(index: 985, score: 12.612990), (index: 309, score: 6.408935), (index: 883, score: 6.122792),
[386 iters] min = 51.66ms max = 53.13ms median = 51.89ms mean = 51.92ms
Creating tflite runtime interpreter: mobilevit_x_small
(index: 985, score: 11.732212), (index: 951, score: 6.617014), (index: 723, score: 5.806361),
[159 iters] min = 116.51ms max = 127.74ms median = 126.55ms mean = 126.43ms
Creating tflite runtime interpreter: mobilevit_small
(index: 985, score: 10.606143), (index: 838, score: 4.171906), (index: 309, score: 4.077430),
[118 iters] min = 168.88ms max = 171.00ms median = 169.47ms mean = 169.53ms
Creating tflite runtime interpreter: LeViT_128S
(index: 985, score: 11.371873), (index: 949, score: 4.906947), (index: 904, score: 4.606572),
[1086 iters] min = 17.45ms max = 18.73ms median = 18.42ms mean = 18.42ms
Creating tflite runtime interpreter: LeViT_128
(index: 985, score: 11.173519), (index: 111, score: 3.877256), (index: 783, score: 3.428816),
[800 iters] min = 24.69ms max = 36.96ms median = 24.93ms mean = 25.01ms
Creating tflite runtime interpreter: LeViT_192
(index: 985, score: 11.933888), (index: 326, score: 3.711475), (index: 324, score: 3.631554),
[597 iters] min = 33.30ms max = 36.03ms median = 33.46ms mean = 33.55ms
Creating tflite runtime interpreter: LeViT_256
(index: 985, score: 11.975584), (index: 309, score: 3.963905), (index: 310, score: 3.277127),
[398 iters] min = 50.03ms max = 61.86ms median = 50.22ms mean = 50.36ms
Creating tflite runtime interpreter: resnet50
(index: 985, score: 8.152997), (index: 113, score: -5.376539), (index: 310, score: -5.619974),
[196 iters] min = 102.10ms max = 103.59ms median = 102.43ms mean = 102.50ms
Creating tflite runtime interpreter: mobilenetv3_large_100
(index: 985, score: 9.779805), (index: 310, score: 2.848756), (index: 308, score: 2.550093),
[912 iters] min = 20.52ms max = 22.66ms median = 21.94ms mean = 21.94ms
Creating tflite runtime interpreter: tf_efficientnetv2_b0
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#649 is a dynamic-sized tensor).
(index: 985, score: 9.519145), (index: 309, score: 2.452715), (index: 310, score: 2.307111),
[418 iters] min = 47.34ms max = 51.05ms median = 47.67ms mean = 47.87ms
Creating tflite runtime interpreter: tf_efficientnetv2_b1
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#788 is a dynamic-sized tensor).
(index: 985, score: 9.741409), (index: 309, score: 2.442716), (index: 310, score: 2.206350),
[271 iters] min = 71.02ms max = 75.32ms median = 73.93ms mean = 73.96ms
Creating tflite runtime interpreter: tf_efficientnetv2_b2
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#822 is a dynamic-sized tensor).
(index: 985, score: 10.074444), (index: 883, score: 2.494984), (index: 309, score: 2.232294),
[196 iters] min = 101.63ms max = 103.59ms median = 102.02ms mean = 102.07ms
Creating tflite runtime interpreter: tf_efficientnetv2_b3
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#958 is a dynamic-sized tensor).
(index: 985, score: 8.897607), (index: 955, score: 2.766511), (index: 310, score: 2.308449),
[119 iters] min = 167.57ms max = 169.85ms median = 168.07ms mean = 168.17mscortex-A78 @ 1 thread @ 2.2GHz onnx-tf fp32 & fp16 & bf16(a bit slower)
$ MODEL=ALL make run-tflite-perf
INFO: Using num_threads == 1
Creating tflite runtime interpreter: efficientformerv2_s0
INFO: Created TensorFlow Lite delegate for select TF ops.
INFO: TfLiteFlexDelegate delegate: 35 nodes delegated out of 831 nodes with 35 partitions.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
(index: 985, score: 11.767029), (index: 644, score: 4.848304), (index: 108, score: 3.925714),
[337 iters] min = 59.06ms max = 59.97ms median = 59.45ms mean = 59.46ms
Creating tflite runtime interpreter: efficientformerv2_s1
(index: 985, score: 13.112433), (index: 89, score: 4.162668), (index: 984, score: 4.077538),
[220 iters] min = 90.74ms max = 91.55ms median = 91.18ms mean = 91.18ms
Creating tflite runtime interpreter: efficientformerv2_s2
(index: 985, score: 12.485476), (index: 22, score: 3.693241), (index: 309, score: 3.691998),
[128 iters] min = 156.56ms max = 158.40ms median = 157.20ms mean = 157.22ms
Creating tflite runtime interpreter: SwiftFormer_XS
(index: 985, score: 11.914167), (index: 883, score: 5.001735), (index: 310, score: 4.622922),
[298 iters] min = 62.93ms max = 67.47ms median = 67.22ms mean = 67.15ms
Creating tflite runtime interpreter: SwiftFormer_S
(index: 985, score: 12.528475), (index: 89, score: 4.334187), (index: 720, score: 4.178124),
[208 iters] min = 95.89ms max = 97.41ms median = 96.46ms mean = 96.46ms
Creating tflite runtime interpreter: SwiftFormer_L1
(index: 985, score: 13.233635), (index: 309, score: 3.921286), (index: 310, score: 3.807556),
[135 iters] min = 140.53ms max = 149.77ms median = 149.26ms mean = 149.07ms
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
Creating tflite runtime interpreter: edgenext_xx_small
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#996 is a dynamic-sized tensor).
(index: 985, score: 10.885461), (index: 309, score: 4.954113), (index: 310, score: 4.638608),
[394 iters] min = 50.38ms max = 51.01ms median = 50.77ms mean = 50.77ms
Creating tflite runtime interpreter: edgenext_x_small
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#1489 is a dynamic-sized tensor).
(index: 985, score: 9.799909), (index: 309, score: 4.595185), (index: 308, score: 3.817010),
[214 iters] min = 92.79ms max = 94.07ms median = 93.51ms mean = 93.49ms
Creating tflite runtime interpreter: edgenext_small
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#1489 is a dynamic-sized tensor).
(index: 985, score: 12.156299), (index: 309, score: 4.532577), (index: 308, score: 4.049805),
[114 iters] min = 174.73ms max = 176.32ms median = 175.62ms mean = 175.62ms
Creating tflite runtime interpreter: mobilevitv2_050
(index: 985, score: 8.315767), (index: 309, score: 2.612426), (index: 584, score: 2.352666),
[310 iters] min = 63.88ms max = 65.83ms median = 64.51ms mean = 64.52ms
Creating tflite runtime interpreter: mobilevitv2_075
(index: 985, score: 8.129782), (index: 309, score: 2.389380), (index: 308, score: 1.880313),
[167 iters] min = 112.20ms max = 121.03ms median = 119.99ms mean = 119.80ms
Creating tflite runtime interpreter: mobilevitv2_100
(index: 985, score: 8.256241), (index: 557, score: 2.220457), (index: 309, score: 1.944935),
[106 iters] min = 189.06ms max = 190.89ms median = 189.81ms mean = 189.80ms
Creating tflite runtime interpreter: mobilevitv2_125
(index: 985, score: 8.282048), (index: 309, score: 1.962256), (index: 883, score: 1.285460),
[74 iters] min = 256.42ms max = 272.96ms median = 272.14ms mean = 271.79ms
Creating tflite runtime interpreter: mobilevitv2_150
(index: 985, score: 9.099127), (index: 308, score: 2.259560), (index: 301, score: 2.159019),
[54 iters] min = 369.93ms max = 371.42ms median = 370.57ms mean = 370.62ms
Creating tflite runtime interpreter: mobilevitv2_175
(index: 985, score: 8.888693), (index: 494, score: 2.104596), (index: 309, score: 1.869223),
[42 iters] min = 479.46ms max = 480.57ms median = 480.01ms mean = 480.02ms
Creating tflite runtime interpreter: mobilevitv2_200
(index: 985, score: 8.531492), (index: 883, score: 2.249018), (index: 309, score: 2.237880),
[33 iters] min = 605.61ms max = 612.58ms median = 610.26ms mean = 610.19ms
Creating tflite runtime interpreter: mobilevit_xx_small
(index: 985, score: 12.652470), (index: 309, score: 6.357603), (index: 308, score: 6.236125),
[344 iters] min = 57.92ms max = 58.76ms median = 58.25ms mean = 58.27ms
Creating tflite runtime interpreter: mobilevit_x_small
(index: 985, score: 12.998843), (index: 89, score: 6.411968), (index: 308, score: 5.775461),
[138 iters] min = 133.80ms max = 145.94ms median = 145.41ms mean = 144.97ms
Creating tflite runtime interpreter: mobilevit_small
(index: 985, score: 10.661427), (index: 838, score: 4.319447), (index: 309, score: 4.076353),
[90 iters] min = 222.47ms max = 224.21ms median = 223.57ms mean = 223.56ms
Creating tflite runtime interpreter: LeViT_128S
(index: 985, score: 11.427817), (index: 308, score: 3.451130), (index: 309, score: 3.319763),
[623 iters] min = 31.95ms max = 32.36ms median = 32.14ms mean = 32.14ms
Creating tflite runtime interpreter: LeViT_128
(index: 985, score: 11.089764), (index: 309, score: 3.409034), (index: 113, score: 3.385414),
[473 iters] min = 42.03ms max = 42.64ms median = 42.32ms mean = 42.32ms
Creating tflite runtime interpreter: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186352), (index: 644, score: 3.177924),
[327 iters] min = 60.98ms max = 61.75ms median = 61.33ms mean = 61.33ms
Creating tflite runtime interpreter: LeViT_256
(index: 985, score: 11.363824), (index: 108, score: 3.341186), (index: 310, score: 2.929489),
[194 iters] min = 98.58ms max = 103.79ms median = 103.27ms mean = 103.22ms
Creating tflite runtime interpreter: resnet50
(index: 985, score: 7.495987), (index: 113, score: -4.947908), (index: 310, score: -5.267951),
[74 iters] min = 272.02ms max = 273.67ms median = 272.55ms mean = 272.56ms
Creating tflite runtime interpreter: mobilenetv3_large_100
(index: 985, score: 9.592711), (index: 308, score: 2.354277), (index: 310, score: 2.337051),
[766 iters] min = 24.62ms max = 27.07ms median = 26.16ms mean = 26.13ms
Creating tflite runtime interpreter: tf_efficientnetv2_b0
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#649 is a dynamic-sized tensor).
(index: 985, score: 9.554760), (index: 309, score: 2.378345), (index: 108, score: 2.289133),
[237 iters] min = 84.00ms max = 84.86ms median = 84.39ms mean = 84.40ms
Creating tflite runtime interpreter: tf_efficientnetv2_b1
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#788 is a dynamic-sized tensor).
(index: 985, score: 9.484579), (index: 861, score: 2.258523), (index: 309, score: 2.134490),
[150 iters] min = 132.98ms max = 134.18ms median = 133.44ms mean = 133.45ms
Creating tflite runtime interpreter: tf_efficientnetv2_b2
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#822 is a dynamic-sized tensor).
(index: 985, score: 9.816823), (index: 883, score: 2.518672), (index: 113, score: 2.046143),
[108 iters] min = 185.07ms max = 186.02ms median = 185.50ms mean = 185.51ms
Creating tflite runtime interpreter: tf_efficientnetv2_b3
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#958 is a dynamic-sized tensor).
(index: 985, score: 9.089396), (index: 955, score: 2.892823), (index: 947, score: 2.188147),
[63 iters] min = 319.86ms max = 322.55ms median = 320.89ms mean = 320.94mscommit id:
ccb73fd827d29f69b1b5dfcbd4c27188b2364f0d
cortex-A78 @ 1 thread @ 2.2GHz fp32
$ MODEL=ALL make run-onnxruntime-perf
INFO: Using num_threads == 1
INFO: Using CPU backend
Creating onnx runtime session: efficientformerv2_s0
(index: 985, score: 11.767027), (index: 644, score: 4.848296), (index: 108, score: 3.925725),
[315 iters] min = 63.01ms max = 64.71ms median = 63.52ms mean = 63.54ms
Creating onnx runtime session: efficientformerv2_s1
(index: 985, score: 13.112438), (index: 89, score: 4.162661), (index: 984, score: 4.077526),
[205 iters] min = 97.14ms max = 99.73ms median = 97.87ms mean = 97.90ms
Creating onnx runtime session: efficientformerv2_s2
(index: 985, score: 12.485476), (index: 22, score: 3.693230), (index: 309, score: 3.692003),
[120 iters] min = 166.99ms max = 169.06ms median = 167.91ms mean = 167.90ms
Creating onnx runtime session: SwiftFormer_XS
(index: 985, score: 11.914163), (index: 883, score: 5.001734), (index: 310, score: 4.622918),
[282 iters] min = 70.27ms max = 73.70ms median = 70.89ms mean = 70.93ms
Creating onnx runtime session: SwiftFormer_S
(index: 985, score: 12.528481), (index: 89, score: 4.334188), (index: 720, score: 4.178124),
[194 iters] min = 101.17ms max = 106.83ms median = 103.18ms mean = 103.26ms
Creating onnx runtime session: SwiftFormer_L1
(index: 985, score: 13.233635), (index: 309, score: 3.921291), (index: 310, score: 3.807557),
[129 iters] min = 154.54ms max = 158.43ms median = 155.85ms mean = 155.88ms
Creating onnx runtime session: EMO_1M
(index: 985, score: 10.011187), (index: 309, score: 4.270288), (index: 310, score: 3.913450),
[403 iters] min = 49.11ms max = 52.89ms median = 49.59ms mean = 49.75ms
Creating onnx runtime session: EMO_2M
(index: 985, score: 9.367957), (index: 309, score: 3.259868), (index: 308, score: 3.008149),
[272 iters] min = 72.87ms max = 76.64ms median = 73.36ms mean = 73.58ms
Creating onnx runtime session: EMO_5M
(index: 985, score: 9.141464), (index: 883, score: 2.990551), (index: 308, score: 2.454388),
[157 iters] min = 126.94ms max = 130.64ms median = 127.59ms mean = 127.69ms
Creating onnx runtime session: EMO_6M
(index: 985, score: 9.396774), (index: 883, score: 2.240933), (index: 309, score: 2.083860),
[148 iters] min = 133.66ms max = 137.31ms median = 135.28ms mean = 135.39ms
Creating onnx runtime session: edgenext_xx_small
(index: 985, score: 10.885463), (index: 309, score: 4.954113), (index: 310, score: 4.638608),
[493 iters] min = 40.35ms max = 41.12ms median = 40.63ms mean = 40.65ms
Creating onnx runtime session: edgenext_x_small
(index: 985, score: 9.799908), (index: 309, score: 4.595183), (index: 308, score: 3.817008),
[262 iters] min = 75.08ms max = 77.39ms median = 76.60ms mean = 76.62ms
Creating onnx runtime session: edgenext_small
(index: 985, score: 12.156298), (index: 309, score: 4.532578), (index: 308, score: 4.049806),
[139 iters] min = 142.70ms max = 148.03ms median = 143.79ms mean = 144.01ms
Creating onnx runtime session: mobilevitv2_050
(index: 985, score: 8.315781), (index: 309, score: 2.612400), (index: 584, score: 2.352643),
[358 iters] min = 54.95ms max = 59.93ms median = 55.75ms mean = 55.99ms
Creating onnx runtime session: mobilevitv2_075
(index: 985, score: 8.129787), (index: 309, score: 2.389381), (index: 308, score: 1.880314),
[190 iters] min = 104.20ms max = 112.60ms median = 104.82ms mean = 105.36ms
Creating onnx runtime session: mobilevitv2_100
(index: 985, score: 8.256272), (index: 557, score: 2.220435), (index: 309, score: 1.944910),
[117 iters] min = 170.18ms max = 178.25ms median = 171.52ms mean = 172.08ms
Creating onnx runtime session: mobilevitv2_125
(index: 985, score: 8.281981), (index: 309, score: 1.962245), (index: 883, score: 1.285464),
[79 iters] min = 242.18ms max = 264.05ms median = 253.34ms mean = 253.41ms
Creating onnx runtime session: mobilevitv2_150
(index: 985, score: 9.098927), (index: 308, score: 2.259604), (index: 301, score: 2.159042),
[58 iters] min = 345.23ms max = 355.02ms median = 347.09ms mean = 347.29ms
Creating onnx runtime session: mobilevitv2_175
(index: 985, score: 8.888681), (index: 494, score: 2.104774), (index: 309, score: 1.869403),
[44 iters] min = 457.30ms max = 464.03ms median = 459.28ms mean = 459.70ms
Creating onnx runtime session: mobilevitv2_200
(index: 985, score: 8.531374), (index: 883, score: 2.248779), (index: 309, score: 2.237854),
[35 iters] min = 580.38ms max = 606.12ms median = 582.96ms mean = 585.05ms
Creating onnx runtime session: mobilevit_xx_small
(index: 985, score: 12.652473), (index: 309, score: 6.357603), (index: 308, score: 6.236127),
[380 iters] min = 51.82ms max = 56.44ms median = 52.52ms mean = 52.65ms
Creating onnx runtime session: mobilevit_x_small
(index: 985, score: 12.998842), (index: 89, score: 6.411969), (index: 308, score: 5.775460),
[161 iters] min = 122.51ms max = 131.09ms median = 124.18ms mean = 124.48ms
Creating onnx runtime session: mobilevit_small
(index: 985, score: 10.661427), (index: 838, score: 4.319448), (index: 309, score: 4.076354),
[101 iters] min = 197.84ms max = 206.19ms median = 198.82ms mean = 199.37ms
Creating onnx runtime session: LeViT_128S
(index: 985, score: 11.427822), (index: 308, score: 3.451136), (index: 309, score: 3.319764),
[734 iters] min = 25.11ms max = 30.29ms median = 27.15ms mean = 27.28ms
Creating onnx runtime session: LeViT_128
(index: 985, score: 11.089764), (index: 309, score: 3.409033), (index: 113, score: 3.385417),
[561 iters] min = 35.42ms max = 38.11ms median = 35.66ms mean = 35.69ms
Creating onnx runtime session: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186356), (index: 644, score: 3.177926),
[380 iters] min = 49.51ms max = 63.31ms median = 52.58ms mean = 52.75ms
Creating onnx runtime session: LeViT_256
(index: 985, score: 11.363824), (index: 108, score: 3.341184), (index: 310, score: 2.929484),
[232 iters] min = 86.16ms max = 87.40ms median = 86.52ms mean = 86.55ms
Creating onnx runtime session: resnet50
(index: 985, score: 7.495985), (index: 113, score: -4.947910), (index: 310, score: -5.267947),
[69 iters] min = 292.98ms max = 303.70ms median = 293.68ms mean = 294.00ms
Creating onnx runtime session: mobilenetv3_large_100
(index: 985, score: 9.592711), (index: 308, score: 2.354275), (index: 310, score: 2.337051),
[641 iters] min = 31.04ms max = 32.13ms median = 31.22ms mean = 31.23ms
Creating onnx runtime session: tf_efficientnetv2_b0
(index: 985, score: 9.554757), (index: 309, score: 2.378345), (index: 108, score: 2.289132),
[273 iters] min = 73.20ms max = 75.04ms median = 73.50ms mean = 73.53ms
Creating onnx runtime session: tf_efficientnetv2_b1
(index: 985, score: 9.484577), (index: 861, score: 2.258525), (index: 309, score: 2.134489),
[176 iters] min = 111.23ms max = 118.58ms median = 113.62ms mean = 113.69ms
Creating onnx runtime session: tf_efficientnetv2_b2
(index: 985, score: 9.816820), (index: 883, score: 2.518670), (index: 113, score: 2.046142),
[126 iters] min = 158.95ms max = 164.34ms median = 159.52ms mean = 159.68ms
Creating onnx runtime session: tf_efficientnetv2_b3
(index: 985, score: 9.089397), (index: 955, score: 2.892823), (index: 947, score: 2.188145),
[74 iters] min = 266.32ms max = 275.08ms median = 270.98ms mean = 271.07mscortex-A78 @ 1 thread @ 2.2GHz fp16?!
只有LeViT/resnet50是正常的，但是LeViT系列性能大降，只有resnet50出现了性能提升
INFO: Using num_threads == 1
INFO: Using CPU backend
Creating onnx runtime session: efficientformerv2_s0
(index: 375, score: 312.645630), (index: 270, score: 310.344360), (index: 625, score: 308.174011),
[319 iters] min = 60.57ms max = 63.47ms median = 62.93ms mean = 62.79ms
Creating onnx runtime session: efficientformerv2_s1
(index: 110, score: 3713.058594), (index: 519, score: 3310.613281), (index: 798, score: 3293.118652),
[215 iters] min = 90.24ms max = 94.11ms median = 93.60ms mean = 93.41ms
Creating onnx runtime session: efficientformerv2_s2
(index: 68, score: 1192.941772), (index: 238, score: 1122.094238), (index: 542, score: 1113.852417),
[132 iters] min = 147.54ms max = 154.09ms median = 152.59ms mean = 152.21ms
Creating onnx runtime session: SwiftFormer_XS
(index: 723, score: 6.027040), (index: 721, score: 5.818944), (index: 879, score: 5.662380),
[304 iters] min = 63.47ms max = 66.52ms median = 66.00ms mean = 65.83ms
Creating onnx runtime session: SwiftFormer_S
(index: 408, score: 896.605103), (index: 207, score: 866.244507), (index: 205, score: 807.019104),
[228 iters] min = 85.17ms max = 88.95ms median = 88.38ms mean = 88.10ms
Creating onnx runtime session: SwiftFormer_L1
(index: 904, score: 8.103521), (index: 828, score: 6.649522), (index: 555, score: 6.419710),
[152 iters] min = 127.93ms max = 133.21ms median = 132.31ms mean = 132.02ms
Creating onnx runtime session: EMO_1M
(index: 363, score: 7.078125), (index: 328, score: 5.394531), (index: 769, score: 5.242188),
[403 iters] min = 48.11ms max = 50.42ms median = 49.84ms mean = 49.74ms
Creating onnx runtime session: EMO_2M
(index: 699, score: 3.312500), (index: 520, score: 3.267578), (index: 488, score: 3.097656),
[282 iters] min = 68.79ms max = 72.36ms median = 71.24ms mean = 71.13ms
Creating onnx runtime session: EMO_5M
(index: 741, score: 5.875000), (index: 846, score: 5.621094), (index: 885, score: 5.074219),
[174 iters] min = 111.14ms max = 116.15ms median = 115.46ms mean = 115.16ms
Creating onnx runtime session: EMO_6M
(index: 983, score: 2.837891), (index: 2, score: 2.804688), (index: 147, score: 2.763672),
[164 iters] min = 117.81ms max = 123.16ms median = 122.49ms mean = 122.22ms
Creating onnx runtime session: edgenext_xx_small
2023-10-28 11:38:31.240536104 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.240936373 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.260037294 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.260121968 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.267220135 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.267296138 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.274348799 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.274425410 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.281498616 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.281570138 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.288618063 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.288694930 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.295689317 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.295761512 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.302783516 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.302856190 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.309880530 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.309949429 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
2023-10-28 11:38:31.316873030 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add_1'
2023-10-28 11:38:31.316972681 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.1/pos_embd/Add'
(index: 818, score: 6.308594), (index: 819, score: 6.007812), (index: 898, score: 5.937500),
[419 iters] min = 46.43ms max = 48.45ms median = 47.86ms mean = 47.78ms
Creating onnx runtime session: edgenext_x_small
2023-10-28 11:38:56.593773458 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.594107998 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.617526908 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.617604927 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.626604754 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.626685620 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.635602212 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.635680679 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.644585623 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.644662969 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.653569833 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.653643179 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.662542587 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.662617341 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.671522733 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.671598672 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.680509631 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.680588930 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:38:56.689508818 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:38:56.689582101 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
(index: 349, score: 5.171875), (index: 851, score: 4.894531), (index: 473, score: 4.738281),
[233 iters] min = 83.38ms max = 86.64ms median = 85.96ms mean = 85.84ms
Creating onnx runtime session: edgenext_small
2023-10-28 11:39:22.034059509 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.034410208 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.057924402 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.058011829 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.066811867 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.066890142 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.075668611 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.075745702 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.084541580 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.084617423 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.093414805 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.093490455 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.102308254 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.102390177 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.111261065 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.111336396 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.120235381 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.120313400 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
2023-10-28 11:39:22.129164223 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add_1'
2023-10-28 11:39:22.129236130 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/stages.1/stages.1.2/pos_embd/Add'
(index: 968, score: 3.546875), (index: 103, score: 2.960938), (index: 68, score: 2.880859),
[128 iters] min = 152.10ms max = 158.37ms median = 157.26ms mean = 156.89ms
Creating onnx runtime session: mobilevitv2_050
(index: 971, score: 5.484375), (index: 111, score: 4.992188), (index: 892, score: 4.632812),
[367 iters] min = 52.86ms max = 58.67ms median = 54.52ms mean = 54.51ms
Creating onnx runtime session: mobilevitv2_075
(index: 559, score: 8.578125), (index: 646, score: 5.570312), (index: 506, score: 4.902344),
[219 iters] min = 89.01ms max = 92.89ms median = 91.82ms mean = 91.67ms
Creating onnx runtime session: mobilevitv2_100
(index: 677, score: 4.464844), (index: 733, score: 4.078125), (index: 765, score: 3.970703),
[145 iters] min = 134.11ms max = 140.87ms median = 138.30ms mean = 138.10ms
Creating onnx runtime session: mobilevitv2_125
(index: 646, score: 6.011719), (index: 506, score: 5.980469), (index: 346, score: 3.451172),
[105 iters] min = 187.20ms max = 192.35ms median = 191.31ms mean = 190.96ms
Creating onnx runtime session: mobilevitv2_150
(index: 271, score: 3.404297), (index: 269, score: 3.142578), (index: 990, score: 3.023438),
[80 iters] min = 248.14ms max = 253.32ms median = 252.25ms mean = 251.88ms
Creating onnx runtime session: mobilevitv2_175
(index: 271, score: 4.156250), (index: 194, score: 3.779297), (index: 369, score: 3.632812),
[63 iters] min = 315.05ms max = 320.30ms median = 319.05ms mean = 318.52ms
Creating onnx runtime session: mobilevitv2_200
(index: 855, score: 5.050781), (index: 508, score: 4.976562), (index: 689, score: 4.855469),
[49 iters] min = 398.93ms max = 412.54ms median = 411.21ms mean = 410.29ms
Creating onnx runtime session: mobilevit_xx_small
(index: 813, score: 152.250000), (index: 662, score: 100.187500), (index: 401, score: 61.343750),
[316 iters] min = 61.56ms max = 64.40ms median = 63.58ms mean = 63.46ms
Creating onnx runtime session: mobilevit_x_small
(index: 779, score: 35.718750), (index: 687, score: 32.218750), (index: 771, score: 24.000000),
[159 iters] min = 121.95ms max = 131.49ms median = 126.15ms mean = 125.99ms
Creating onnx runtime session: mobilevit_small
(index: 806, score: 28.187500), (index: 411, score: 25.578125), (index: 778, score: 22.906250),
[103 iters] min = 191.30ms max = 206.44ms median = 196.11ms mean = 195.87ms
Creating onnx runtime session: LeViT_128S
(index: 985, score: 11.452311), (index: 308, score: 3.453630), (index: 309, score: 3.344767),
[400 iters] min = 43.41ms max = 52.68ms median = 50.60ms mean = 50.08ms
Creating onnx runtime session: LeViT_128
(index: 985, score: 11.091269), (index: 309, score: 3.424088), (index: 113, score: 3.393584),
[304 iters] min = 58.26ms max = 68.92ms median = 66.82ms mean = 65.95ms
Creating onnx runtime session: LeViT_192
(index: 985, score: 11.600651), (index: 644, score: 3.213440), (index: 308, score: 3.186548),
[246 iters] min = 73.24ms max = 83.79ms median = 82.22ms mean = 81.48ms
Creating onnx runtime session: LeViT_256
(index: 985, score: 11.398668), (index: 108, score: 3.284300), (index: 309, score: 2.865213),
[139 iters] min = 127.59ms max = 149.03ms median = 145.98ms mean = 144.26ms
Creating onnx runtime session: resnet50
(index: 985, score: 7.519531), (index: 113, score: -4.972656), (index: 310, score: -5.296875),
[93 iters] min = 202.89ms max = 223.76ms median = 218.29ms mean = 217.32ms
Creating onnx runtime session: mobilenetv3_large_100
(index: 925, score: 4.050781), (index: 409, score: 3.324219), (index: 635, score: 3.097656),
[629 iters] min = 30.84ms max = 32.18ms median = 31.87ms mean = 31.80ms
Creating onnx runtime session: tf_efficientnetv2_b0
2023-10-28 11:46:34.589473728 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.602116522 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.608306380 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.614467245 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.620581293 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.626663211 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.632687977 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.638748327 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.644780005 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
2023-10-28 11:46:34.650823715 [W:onnxruntime:, constant_folding.cc:214 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Ceil node '/conv_stem/Ceil'
Got dynamic batch size. Setting output batch size to 1.
(index: 953, score: 4.953125), (index: 470, score: 4.746094), (index: 489, score: 4.230469),
[301 iters] min = 64.38ms max = 67.21ms median = 66.80ms mean = 66.64mscortex-A78 @ 1 thread @ 2.2GHz static int8
INFO: Using num_threads == 1
INFO: Using CPU backend
Creating onnx runtime session: efficientformerv2_s0
(index: 904, score: 9.301314), (index: 905, score: 8.267836), (index: 794, score: 8.038173),
[362 iters] min = 53.88ms max = 58.71ms median = 55.28ms mean = 55.31ms
Creating onnx runtime session: efficientformerv2_s1
(index: 985, score: 8.869938), (index: 644, score: 6.363216), (index: 108, score: 6.266804),
[246 iters] min = 79.51ms max = 87.16ms median = 81.25ms mean = 81.32ms
Creating onnx runtime session: efficientformerv2_s2
(index: 641, score: 6.804350), (index: 906, score: 6.651443), (index: 810, score: 6.039816),
[157 iters] min = 125.35ms max = 131.91ms median = 128.01ms mean = 128.07ms
Creating onnx runtime session: SwiftFormer_XS
(index: 985, score: 10.305935), (index: 308, score: 4.524557), (index: 883, score: 3.896146),
[358 iters] min = 54.80ms max = 56.37ms median = 56.04ms mean = 55.93ms
Creating onnx runtime session: SwiftFormer_S
(index: 985, score: 16.637768), (index: 309, score: 6.884593), (index: 723, score: 6.081391),
[280 iters] min = 69.98ms max = 71.96ms median = 71.65ms mean = 71.48ms
Creating onnx runtime session: SwiftFormer_L1
(index: 985, score: 14.329005), (index: 904, score: 8.160891), (index: 905, score: 4.365128),
[208 iters] min = 94.18ms max = 96.93ms median = 96.45ms mean = 96.22ms
Creating onnx runtime session: EMO_1M
(index: 949, score: 6.828766), (index: 533, score: 6.736485), (index: 109, score: 6.644205),
[599 iters] min = 32.40ms max = 33.88ms median = 33.54ms mean = 33.41ms
Creating onnx runtime session: EMO_2M
(index: 985, score: 9.498310), (index: 309, score: 3.749333), (index: 308, score: 3.582696),
[425 iters] min = 45.65ms max = 47.76ms median = 47.21ms mean = 47.09ms
Creating onnx runtime session: EMO_5M
(index: 553, score: 6.486908), (index: 885, score: 6.240570), (index: 977, score: 5.255217),
[284 iters] min = 68.93ms max = 71.35ms median = 70.81ms mean = 70.64ms
Creating onnx runtime session: EMO_6M
(index: 722, score: 3.215647), (index: 624, score: 2.756269), (index: 368, score: 2.572518),
[268 iters] min = 73.07ms max = 75.97ms median = 75.04ms mean = 74.86ms
Creating onnx runtime session: edgenext_xx_small
(index: 985, score: 10.135961), (index: 883, score: 4.106812), (index: 308, score: 4.019433),
[457 iters] min = 42.69ms max = 44.26ms median = 43.94ms mean = 43.81ms
Creating onnx runtime session: edgenext_x_small
(index: 985, score: 7.823123), (index: 318, score: 3.871236), (index: 452, score: 3.387332),
[271 iters] min = 72.17ms max = 74.58ms median = 74.16ms mean = 73.97ms
Creating onnx runtime session: edgenext_small
(index: 985, score: 10.629969), (index: 108, score: 5.656313), (index: 883, score: 4.681087),
[179 iters] min = 110.11ms max = 113.29ms median = 112.61ms mean = 112.35ms
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
Creating onnx runtime session: mobilevit_xx_small
(index: 530, score: 8.487267), (index: 646, score: 7.858580), (index: 688, score: 6.391645),
[463 iters] min = 42.04ms max = 43.86ms median = 43.35ms mean = 43.23ms
Creating onnx runtime session: mobilevit_x_small
(index: 985, score: 11.108162), (index: 951, score: 6.572329), (index: 310, score: 6.294625),
[279 iters] min = 70.36ms max = 72.55ms median = 72.09ms mean = 71.91ms
Creating onnx runtime session: mobilevit_small
(index: 985, score: 8.687240), (index: 626, score: 5.894913), (index: 619, score: 5.894913),
[199 iters] min = 98.51ms max = 101.73ms median = 101.14ms mean = 100.88ms
Creating onnx runtime session: LeViT_128S
(index: 985, score: 10.046530), (index: 720, score: 4.465124), (index: 605, score: 3.906984),
[1363 iters] min = 14.00ms max = 15.05ms median = 14.74ms mean = 14.68ms
Creating onnx runtime session: LeViT_128
(index: 985, score: 11.506492), (index: 794, score: 4.728695), (index: 854, score: 4.255826),
[969 iters] min = 19.76ms max = 21.04ms median = 20.75ms mean = 20.65ms
Creating onnx runtime session: LeViT_192
(index: 985, score: 11.009346), (index: 644, score: 3.878292), (index: 949, score: 3.628080),
[770 iters] min = 24.85ms max = 30.03ms median = 26.07ms mean = 25.99ms
Creating onnx runtime session: LeViT_256
(index: 985, score: 11.014970), (index: 310, score: 3.697883), (index: 309, score: 3.697883),
[530 iters] min = 36.20ms max = 39.45ms median = 37.98ms mean = 37.78ms
Creating onnx runtime session: resnet50
(index: 985, score: 7.083403), (index: 310, score: -4.857191), (index: 113, score: -5.261957),
[260 iters] min = 74.92ms max = 81.04ms median = 77.17ms mean = 77.03ms
Creating onnx runtime session: mobilenetv3_large_100
(index: 985, score: 10.199331), (index: 883, score: 2.781636), (index: 310, score: 2.613052),
[1329 iters] min = 14.53ms max = 17.50ms median = 15.07ms mean = 15.05ms
Creating onnx runtime session: tf_efficientnetv2_b0
(index: 985, score: 8.849307), (index: 108, score: 2.374204), (index: 309, score: 2.230313),
[690 iters] min = 27.74ms max = 31.44ms median = 29.11ms mean = 29.02ms
Creating onnx runtime session: tf_efficientnetv2_b1
(index: 985, score: 9.477756), (index: 309, score: 2.707930), (index: 949, score: 1.998710),
[452 iters] min = 42.57ms max = 54.47ms median = 44.36ms mean = 44.33ms
Creating onnx runtime session: tf_efficientnetv2_b2
(index: 985, score: 6.901396), (index: 108, score: 1.757903), (index: 309, score: 1.562580),
[332 iters] min = 58.18ms max = 62.94ms median = 60.51ms mean = 60.29ms
Creating onnx runtime session: tf_efficientnetv2_b3
(index: 985, score: 8.195366), (index: 955, score: 2.280450), (index: 946, score: 2.066658),
[198 iters] min = 97.86ms max = 104.19ms median = 101.74ms mean = 101.34mscortex-A78 @ 1 thread @ 2.2GHz dynamic int8 (disable conv quant, which onnx doesn't support !!!!)
INFO: Using num_threads == 1
INFO: Using CPU backend
Creating onnx runtime session: efficientformerv2_s0
(index: 985, score: 11.767601), (index: 644, score: 4.856317), (index: 108, score: 3.926015),
[320 iters] min = 61.36ms max = 64.42ms median = 62.57ms mean = 62.50ms
Creating onnx runtime session: efficientformerv2_s1
(index: 985, score: 13.111914), (index: 89, score: 4.162173), (index: 984, score: 4.069082),
[208 iters] min = 94.66ms max = 98.08ms median = 96.68ms mean = 96.54ms
Creating onnx runtime session: efficientformerv2_s2
(index: 985, score: 12.490580), (index: 309, score: 3.688048), (index: 22, score: 3.679213),
[122 iters] min = 161.80ms max = 166.13ms median = 165.37ms mean = 165.08ms
Creating onnx runtime session: SwiftFormer_XS
(index: 985, score: 11.847700), (index: 883, score: 4.940231), (index: 308, score: 4.753491),
[286 iters] min = 68.24ms max = 72.31ms median = 70.23ms mean = 70.05ms
Creating onnx runtime session: SwiftFormer_S
(index: 985, score: 13.930529), (index: 720, score: 4.841800), (index: 89, score: 4.634744),
[198 iters] min = 99.11ms max = 102.40ms median = 101.76ms mean = 101.51ms
Creating onnx runtime session: SwiftFormer_L1
(index: 985, score: 14.378620), (index: 309, score: 3.903602), (index: 310, score: 3.784205),
[133 iters] min = 147.20ms max = 152.36ms median = 151.33ms mean = 150.94ms
Creating onnx runtime session: EMO_1M
(index: 985, score: 10.008471), (index: 309, score: 4.282920), (index: 310, score: 3.920927),
[405 iters] min = 48.02ms max = 50.06ms median = 49.51ms mean = 49.38ms
Creating onnx runtime session: EMO_2M
(index: 985, score: 9.369873), (index: 309, score: 3.270487), (index: 308, score: 2.998578),
[274 iters] min = 71.24ms max = 74.19ms median = 73.31ms mean = 73.14ms
Creating onnx runtime session: EMO_5M
(index: 985, score: 9.149606), (index: 883, score: 2.984873), (index: 308, score: 2.455936),
[159 iters] min = 123.25ms max = 127.28ms median = 126.30ms mean = 126.04ms
Creating onnx runtime session: EMO_6M
(index: 985, score: 9.386406), (index: 883, score: 2.231557), (index: 309, score: 2.076946),
[150 iters] min = 130.41ms max = 137.70ms median = 133.88ms mean = 133.86ms
Creating onnx runtime session: edgenext_xx_small
(index: 985, score: 10.854851), (index: 309, score: 4.818978), (index: 310, score: 4.660226),
[596 iters] min = 32.77ms max = 36.87ms median = 33.63ms mean = 33.58ms
Creating onnx runtime session: edgenext_x_small
(index: 985, score: 9.754297), (index: 309, score: 4.585029), (index: 308, score: 3.785746),
[329 iters] min = 59.35ms max = 64.00ms median = 60.85ms mean = 60.85ms
Creating onnx runtime session: edgenext_small
(index: 985, score: 12.141153), (index: 309, score: 4.538023), (index: 308, score: 4.138398),
[201 iters] min = 97.02ms max = 100.69ms median = 99.91ms mean = 99.64ms
Creating onnx runtime session: mobilevitv2_050
(index: 985, score: 8.300854), (index: 309, score: 2.613225), (index: 584, score: 2.343000),
[360 iters] min = 53.88ms max = 56.96ms median = 55.71ms mean = 55.58ms
Creating onnx runtime session: mobilevitv2_075
(index: 985, score: 8.126451), (index: 309, score: 2.387401), (index: 308, score: 1.869111),
[192 iters] min = 101.05ms max = 105.72ms median = 104.59ms mean = 104.31ms
Creating onnx runtime session: mobilevitv2_100
(index: 985, score: 8.271072), (index: 557, score: 2.219984), (index: 309, score: 1.957172),
[120 iters] min = 162.70ms max = 169.87ms median = 168.17ms mean = 167.70ms
Creating onnx runtime session: mobilevitv2_125
(index: 985, score: 8.280787), (index: 309, score: 1.957819), (index: 883, score: 1.288820),
[82 iters] min = 240.67ms max = 248.51ms median = 247.27ms mean = 246.58ms
Creating onnx runtime session: mobilevitv2_150
(index: 985, score: 9.107194), (index: 308, score: 2.253815), (index: 301, score: 2.142919),
[59 iters] min = 333.29ms max = 344.71ms median = 341.62ms mean = 340.65ms
Creating onnx runtime session: mobilevitv2_175
(index: 985, score: 8.886470), (index: 494, score: 2.094772), (index: 309, score: 1.872693),
[45 iters] min = 441.51ms max = 457.33ms median = 450.62ms mean = 449.47ms
Creating onnx runtime session: mobilevitv2_200
(index: 985, score: 8.529298), (index: 883, score: 2.249700), (index: 309, score: 2.229221),
[36 iters] min = 563.65ms max = 577.27ms median = 572.16ms mean = 570.89ms
Creating onnx runtime session: mobilevit_xx_small
(index: 985, score: 12.656101), (index: 309, score: 6.405189), (index: 308, score: 6.261666),
[420 iters] min = 46.49ms max = 48.67ms median = 47.73ms mean = 47.64ms
Creating onnx runtime session: mobilevit_x_small
(index: 985, score: 12.965905), (index: 89, score: 6.380670), (index: 308, score: 5.773078),
[182 iters] min = 106.70ms max = 123.53ms median = 110.27ms mean = 110.19ms
Creating onnx runtime session: mobilevit_small
(index: 985, score: 10.726731), (index: 838, score: 4.307947), (index: 309, score: 4.066022),
[121 iters] min = 160.98ms max = 170.36ms median = 166.39ms mean = 166.17ms
Creating onnx runtime session: LeViT_128S
(index: 985, score: 10.759948), (index: 949, score: 3.963227), (index: 781, score: 3.439202),
[1351 iters] min = 14.24ms max = 15.12ms median = 14.87ms mean = 14.81ms
Creating onnx runtime session: LeViT_128
(index: 985, score: 10.447086), (index: 113, score: 3.491381), (index: 322, score: 3.252628),
[1022 iters] min = 18.82ms max = 23.98ms median = 19.66ms mean = 19.58ms
Creating onnx runtime session: LeViT_192
(index: 985, score: 11.600488), (index: 326, score: 3.805164), (index: 324, score: 3.421825),
[706 iters] min = 27.37ms max = 28.89ms median = 28.46ms mean = 28.36ms
Creating onnx runtime session: LeViT_256
(index: 985, score: 12.019890), (index: 309, score: 3.603356), (index: 310, score: 3.501025),
[456 iters] min = 42.35ms max = 44.70ms median = 44.10ms mean = 43.94ms
Creating onnx runtime session: resnet50
(index: 985, score: 7.540756), (index: 113, score: -4.925373), (index: 310, score: -5.233062),
[69 iters] min = 285.69ms max = 294.71ms median = 292.92ms mean = 292.21ms
Creating onnx runtime session: mobilenetv3_large_100
(index: 985, score: 9.583168), (index: 308, score: 2.351349), (index: 310, score: 2.336620),
[652 iters] min = 29.40ms max = 37.62ms median = 30.71ms mean = 30.72ms
Creating onnx runtime session: tf_efficientnetv2_b0
(index: 985, score: 9.558366), (index: 309, score: 2.373356), (index: 108, score: 2.280105),
[276 iters] min = 71.15ms max = 76.24ms median = 72.71ms mean = 72.60ms
Creating onnx runtime session: tf_efficientnetv2_b1
(index: 985, score: 9.488265), (index: 861, score: 2.265718), (index: 309, score: 2.139990),
[178 iters] min = 110.09ms max = 115.09ms median = 112.56ms mean = 112.59ms
Creating onnx runtime session: tf_efficientnetv2_b2
(index: 985, score: 9.823639), (index: 883, score: 2.515293), (index: 113, score: 2.049522),
[126 iters] min = 156.08ms max = 162.54ms median = 159.35ms mean = 159.58ms
Creating onnx runtime session: tf_efficientnetv2_b3
(index: 985, score: 9.088434), (index: 955, score: 2.879115), (index: 310, score: 2.179863),
[75 iters] min = 264.99ms max = 283.07ms median = 269.84ms mean = 269.90mscommit id
b18e1b684a7673daa3a51128aae4e75ed7aa7cbc
- arm_compute-v23.05-bin-linux-arm64-v8.2-a-neon.tar.gz
- torch-2.1.0.dev20230825-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- libopenblas.so.0
- libgfortran-42309ea2.so.5.0.0
cortex-A78 @ 1 thread @ 2.2GHz trace w/ onednn+acl by gcc-10
$ OMP_NUM_THREADS=1 MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using trace CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767033), (index: 644, score: 4.848289), (index: 108, score: 3.925720),
[118 iters] min = 169.74ms max = 171.59ms median = 170.67ms mean = 170.69ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112442), (index: 89, score: 4.162664), (index: 984, score: 4.077511),
[80 iters] min = 250.77ms max = 254.01ms median = 253.22ms mean = 253.17ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485486), (index: 22, score: 3.693231), (index: 309, score: 3.692009),
[50 iters] min = 403.84ms max = 407.07ms median = 405.16ms mean = 405.21ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914165), (index: 883, score: 5.001728), (index: 310, score: 4.622917),
[130 iters] min = 153.29ms max = 154.60ms median = 153.82ms mean = 153.85ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528474), (index: 89, score: 4.334189), (index: 720, score: 4.178121),
[97 iters] min = 205.07ms max = 207.26ms median = 206.32ms mean = 206.38ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233633), (index: 309, score: 3.921279), (index: 310, score: 3.807570),
[68 iters] min = 292.75ms max = 297.65ms median = 294.09ms mean = 294.20ms
Creating pytorch module: EMO_1M
(index: 985, score: 10.011185), (index: 309, score: 4.270287), (index: 310, score: 3.913450),
[169 iters] min = 116.04ms max = 119.50ms median = 118.41ms mean = 118.41ms
Creating pytorch module: EMO_2M
(index: 985, score: 9.367957), (index: 309, score: 3.259868), (index: 308, score: 3.008149),
[120 iters] min = 167.19ms max = 169.09ms median = 167.91ms mean = 167.96ms
Creating pytorch module: EMO_5M
(index: 985, score: 9.141463), (index: 883, score: 2.990551), (index: 308, score: 2.454388),
[78 iters] min = 255.51ms max = 259.00ms median = 257.98ms mean = 257.98ms
Creating pytorch module: EMO_6M
(index: 985, score: 9.396775), (index: 883, score: 2.240934), (index: 309, score: 2.083860),
[72 iters] min = 278.13ms max = 281.19ms median = 280.00ms mean = 279.97ms
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885459), (index: 309, score: 4.954109), (index: 310, score: 4.638605),
[200 iters] min = 98.96ms max = 101.23ms median = 100.08ms mean = 100.12ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799909), (index: 309, score: 4.595184), (index: 308, score: 3.817010),
[106 iters] min = 187.55ms max = 190.51ms median = 188.93ms mean = 188.97ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156299), (index: 309, score: 4.532576), (index: 308, score: 4.049803),
[64 iters] min = 312.43ms max = 315.11ms median = 313.88ms mean = 313.92ms
Creating pytorch module: mobilevitv2_050
(index: 985, score: 8.315772), (index: 309, score: 2.612400), (index: 584, score: 2.352643),
[187 iters] min = 102.85ms max = 108.02ms median = 106.97ms mean = 106.98ms
Creating pytorch module: mobilevitv2_075
(index: 985, score: 8.129788), (index: 309, score: 2.389378), (index: 308, score: 1.880310),
[109 iters] min = 178.39ms max = 190.03ms median = 184.64ms mean = 184.18ms
Creating pytorch module: mobilevitv2_100
(index: 985, score: 8.256273), (index: 557, score: 2.220434), (index: 309, score: 1.944912),
[74 iters] min = 263.31ms max = 279.21ms median = 272.14ms mean = 272.14ms
Creating pytorch module: mobilevitv2_125
(index: 985, score: 8.281982), (index: 309, score: 1.962245), (index: 883, score: 1.285464),
[54 iters] min = 368.82ms max = 379.29ms median = 374.07ms mean = 373.96ms
Creating pytorch module: mobilevitv2_150
(index: 985, score: 9.098927), (index: 308, score: 2.259606), (index: 301, score: 2.159039),
[41 iters] min = 486.54ms max = 494.03ms median = 489.25ms mean = 489.48ms
Creating pytorch module: mobilevitv2_175
(index: 985, score: 8.888678), (index: 494, score: 2.104781), (index: 309, score: 1.869408),
[33 iters] min = 609.74ms max = 621.30ms median = 615.93ms mean = 615.97ms
Creating pytorch module: mobilevitv2_200
(index: 985, score: 8.531363), (index: 883, score: 2.248764), (index: 309, score: 2.237853),
[27 iters] min = 755.23ms max = 772.08ms median = 765.00ms mean = 764.46ms
Creating pytorch module: mobilevit_xx_small
(index: 985, score: 12.652477), (index: 309, score: 6.357602), (index: 308, score: 6.236127),
[156 iters] min = 124.97ms max = 131.10ms median = 128.07ms mean = 128.34ms
Creating pytorch module: mobilevit_x_small
(index: 985, score: 12.998842), (index: 89, score: 6.411968), (index: 308, score: 5.775460),
[78 iters] min = 257.19ms max = 264.45ms median = 258.68ms mean = 259.00ms
Creating pytorch module: mobilevit_small
(index: 985, score: 10.661425), (index: 838, score: 4.319453), (index: 309, score: 4.076357),
[56 iters] min = 344.86ms max = 373.12ms median = 362.88ms mean = 363.26ms
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427817), (index: 308, score: 3.451130), (index: 309, score: 3.319760),
[312 iters] min = 63.29ms max = 74.55ms median = 63.99ms mean = 64.27ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089767), (index: 309, score: 3.409031), (index: 113, score: 3.385418),
[245 iters] min = 81.31ms max = 83.03ms median = 81.67ms mean = 81.74ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186359), (index: 644, score: 3.177923),
[207 iters] min = 96.00ms max = 103.45ms median = 96.49ms mean = 96.72ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363824), (index: 108, score: 3.341188), (index: 310, score: 2.929487),
[128 iters] min = 154.82ms max = 159.48ms median = 155.97ms mean = 156.39ms
Creating pytorch module: resnet50
(index: 985, score: 7.495995), (index: 113, score: -4.947914), (index: 310, score: -5.267949),
[43 iters] min = 455.58ms max = 466.64ms median = 465.50ms mean = 465.26ms
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592707), (index: 308, score: 2.354278), (index: 310, score: 2.337049),
[313 iters] min = 63.57ms max = 65.53ms median = 63.92ms mean = 63.98ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554752), (index: 309, score: 2.378345), (index: 108, score: 2.289131),
[151 iters] min = 124.39ms max = 134.48ms median = 133.11ms mean = 132.90ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484587), (index: 861, score: 2.258526), (index: 309, score: 2.134490),
[99 iters] min = 201.87ms max = 205.13ms median = 203.06ms mean = 203.19ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816826), (index: 883, score: 2.518669), (index: 113, score: 2.046141),
[73 iters] min = 272.58ms max = 276.61ms median = 274.43ms mean = 274.32ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089396), (index: 955, score: 2.892824), (index: 947, score: 2.188146),
[45 iters] min = 447.64ms max = 451.66ms median = 449.39ms mean = 449.48mscortex-A78 @ 1 thread @ 2.2GHz trace w/ onednn+acl by llvm-14
$ OMP_NUM_THREADS=1 MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using trace CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767036), (index: 644, score: 4.848290), (index: 108, score: 3.925720),
[107 iters] min = 186.40ms max = 188.32ms median = 187.52ms mean = 187.53ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112440), (index: 89, score: 4.162663), (index: 984, score: 4.077516),
[72 iters] min = 277.68ms max = 279.49ms median = 278.98ms mean = 278.96ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485483), (index: 22, score: 3.693227), (index: 309, score: 3.692008),
[46 iters] min = 441.89ms max = 444.90ms median = 443.29ms mean = 443.29ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914167), (index: 883, score: 5.001725), (index: 310, score: 4.622917),
[123 iters] min = 162.65ms max = 164.36ms median = 163.48ms mean = 163.48ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528477), (index: 89, score: 4.334195), (index: 720, score: 4.178129),
[92 iters] min = 212.45ms max = 219.58ms median = 218.83ms mean = 218.63ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233637), (index: 309, score: 3.921281), (index: 310, score: 3.807555),
[65 iters] min = 309.34ms max = 311.45ms median = 310.66ms mean = 310.62ms
Creating pytorch module: EMO_1M
(index: 985, score: 10.011186), (index: 309, score: 4.270287), (index: 310, score: 3.913450),
[156 iters] min = 124.74ms max = 129.06ms median = 128.49ms mean = 128.46ms
Creating pytorch module: EMO_2M
(index: 985, score: 9.367956), (index: 309, score: 3.259869), (index: 308, score: 3.008148),
[110 iters] min = 181.64ms max = 183.94ms median = 183.02ms mean = 183.01ms
Creating pytorch module: EMO_5M
(index: 985, score: 9.141463), (index: 883, score: 2.990551), (index: 308, score: 2.454389),
[71 iters] min = 281.63ms max = 283.30ms median = 282.29ms mean = 282.34ms
Creating pytorch module: EMO_6M
(index: 985, score: 9.396775), (index: 883, score: 2.240934), (index: 309, score: 2.083860),
[66 iters] min = 301.70ms max = 305.57ms median = 304.63ms mean = 304.62ms
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885462), (index: 309, score: 4.954109), (index: 310, score: 4.638606),
[187 iters] min = 106.73ms max = 107.82ms median = 107.25ms mean = 107.25ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799911), (index: 309, score: 4.595183), (index: 308, score: 3.817008),
[98 iters] min = 199.13ms max = 205.03ms median = 204.35ms mean = 204.27ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156299), (index: 309, score: 4.532576), (index: 308, score: 4.049803),
[60 iters] min = 336.78ms max = 338.58ms median = 337.58ms mean = 337.55ms
Creating pytorch module: mobilevitv2_050
(index: 985, score: 8.315772), (index: 309, score: 2.612401), (index: 584, score: 2.352643),
[173 iters] min = 113.83ms max = 117.74ms median = 115.82ms mean = 115.63ms
Creating pytorch module: mobilevitv2_075
(index: 985, score: 8.129788), (index: 309, score: 2.389379), (index: 308, score: 1.880310),
[104 iters] min = 189.90ms max = 196.72ms median = 193.52ms mean = 192.99ms
Creating pytorch module: mobilevitv2_100
(index: 985, score: 8.256272), (index: 557, score: 2.220434), (index: 309, score: 1.944911),
[72 iters] min = 278.47ms max = 283.83ms median = 281.93ms mean = 281.60ms
Creating pytorch module: mobilevitv2_125
(index: 985, score: 8.281981), (index: 309, score: 1.962247), (index: 883, score: 1.285465),
[52 iters] min = 377.34ms max = 392.03ms median = 388.37ms mean = 388.11ms
Creating pytorch module: mobilevitv2_150
(index: 985, score: 9.098927), (index: 308, score: 2.259606), (index: 301, score: 2.159039),
[40 iters] min = 506.48ms max = 514.94ms median = 510.52ms mean = 510.54ms
Creating pytorch module: mobilevitv2_175
(index: 985, score: 8.888678), (index: 494, score: 2.104781), (index: 309, score: 1.869407),
[32 iters] min = 619.52ms max = 642.13ms median = 635.30ms mean = 634.70ms
Creating pytorch module: mobilevitv2_200
(index: 985, score: 8.531360), (index: 883, score: 2.248765), (index: 309, score: 2.237852),
[26 iters] min = 784.59ms max = 798.40ms median = 791.67ms mean = 790.78ms
Creating pytorch module: mobilevit_xx_small
(index: 985, score: 12.652478), (index: 309, score: 6.357604), (index: 308, score: 6.236127),
[149 iters] min = 133.65ms max = 134.93ms median = 134.26ms mean = 134.26ms
Creating pytorch module: mobilevit_x_small
(index: 985, score: 12.998840), (index: 89, score: 6.411969), (index: 308, score: 5.775461),
[74 iters] min = 271.85ms max = 273.72ms median = 272.79ms mean = 272.75ms
Creating pytorch module: mobilevit_small
(index: 985, score: 10.661426), (index: 838, score: 4.319452), (index: 309, score: 4.076355),
[53 iters] min = 378.86ms max = 383.65ms median = 379.77ms mean = 379.93ms
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427817), (index: 308, score: 3.451130), (index: 309, score: 3.319760),
[310 iters] min = 59.26ms max = 65.33ms median = 64.60ms mean = 64.58ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089767), (index: 309, score: 3.409031), (index: 113, score: 3.385418),
[246 iters] min = 80.87ms max = 82.21ms median = 81.46ms mean = 81.47ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186359), (index: 644, score: 3.177923),
[207 iters] min = 89.67ms max = 97.49ms median = 96.71ms mean = 96.65ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363824), (index: 108, score: 3.341188), (index: 310, score: 2.929487),
[129 iters] min = 155.64ms max = 156.74ms median = 156.03ms mean = 156.06ms
Creating pytorch module: resnet50
(index: 985, score: 7.495994), (index: 113, score: -4.947915), (index: 310, score: -5.267948),
[44 iters] min = 455.63ms max = 462.39ms median = 456.46ms mean = 456.71ms
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592705), (index: 308, score: 2.354278), (index: 310, score: 2.337049),
[283 iters] min = 70.57ms max = 71.25ms median = 70.85ms mean = 70.85ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554752), (index: 309, score: 2.378345), (index: 108, score: 2.289131),
[146 iters] min = 136.92ms max = 138.09ms median = 137.46ms mean = 137.49ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484588), (index: 861, score: 2.258526), (index: 309, score: 2.134490),
[97 iters] min = 199.50ms max = 208.17ms median = 207.41ms mean = 207.32ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816827), (index: 883, score: 2.518669), (index: 113, score: 2.046142),
[72 iters] min = 277.36ms max = 279.63ms median = 278.80ms mean = 278.82ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089393), (index: 955, score: 2.892824), (index: 947, score: 2.188145),
[44 iters] min = 456.29ms max = 465.19ms median = 464.33ms mean = 464.03mscortex-A78 @ 1 thread @ 2.2GHz mobile w/ onednn+acl by gcc-10
$ OMP_NUM_THREADS=1 BACK=c MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using mobile CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767030), (index: 644, score: 4.848297), (index: 108, score: 3.925721),
[139 iters] min = 144.14ms max = 147.54ms median = 144.58ms mean = 144.66ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112427), (index: 89, score: 4.162661), (index: 984, score: 4.077535),
[94 iters] min = 208.93ms max = 214.23ms median = 213.15ms mean = 213.14ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485474), (index: 22, score: 3.693241), (index: 309, score: 3.691998),
[59 iters] min = 341.99ms max = 343.72ms median = 342.52ms mean = 342.65ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914167), (index: 883, score: 5.001735), (index: 310, score: 4.622921),
[140 iters] min = 142.90ms max = 144.13ms median = 143.52ms mean = 143.54ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528477), (index: 89, score: 4.334185), (index: 720, score: 4.178122),
[106 iters] min = 187.87ms max = 190.86ms median = 189.95ms mean = 189.98ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233639), (index: 309, score: 3.921288), (index: 310, score: 3.807554),
[74 iters] min = 272.82ms max = 274.80ms median = 273.49ms mean = 273.61ms
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885463), (index: 309, score: 4.954110), (index: 310, score: 4.638607),
[253 iters] min = 77.91ms max = 82.82ms median = 79.13ms mean = 79.27ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799910), (index: 309, score: 4.595183), (index: 308, score: 3.817009),
[129 iters] min = 153.78ms max = 161.20ms median = 154.93ms mean = 155.29ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156300), (index: 309, score: 4.532576), (index: 308, score: 4.049804),
[77 iters] min = 259.26ms max = 269.29ms median = 260.32ms mean = 260.50ms
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
mobilevit_xx_small model doesn't exist!!!
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427816), (index: 308, score: 3.451128), (index: 309, score: 3.319762),
[513 iters] min = 38.59ms max = 41.36ms median = 39.01ms mean = 39.05ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089766), (index: 309, score: 3.409033), (index: 113, score: 3.385415),
[375 iters] min = 53.03ms max = 55.67ms median = 53.43ms mean = 53.47ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186354), (index: 644, score: 3.177923),
[307 iters] min = 61.81ms max = 67.46ms median = 65.31ms mean = 65.36ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363821), (index: 108, score: 3.341193), (index: 310, score: 2.929493),
[188 iters] min = 105.76ms max = 108.15ms median = 106.46ms mean = 106.52ms
resnet50 model doesn't exist!!!
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592701), (index: 308, score: 2.354276), (index: 310, score: 2.337050),
[500 iters] min = 39.25ms max = 42.54ms median = 39.87ms mean = 40.01ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554751), (index: 309, score: 2.378344), (index: 108, score: 2.289130),
[210 iters] min = 94.33ms max = 98.57ms median = 95.00ms mean = 95.28ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484585), (index: 861, score: 2.258525), (index: 309, score: 2.134489),
[133 iters] min = 149.45ms max = 154.03ms median = 150.28ms mean = 150.73ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816826), (index: 883, score: 2.518668), (index: 113, score: 2.046140),
[98 iters] min = 204.04ms max = 212.16ms median = 205.46ms mean = 205.89ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089395), (index: 955, score: 2.892825), (index: 947, score: 2.188146),
[57 iters] min = 342.86ms max = 356.00ms median = 352.00ms mean = 352.06mscortex-A78 @ 1 thread @ 2.2GHz mobile w/ onednn+acl by llvm-14
$ OMP_NUM_THREADS=1 BACK=c MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using mobile CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767035), (index: 644, score: 4.848297), (index: 108, score: 3.925725),
[131 iters] min = 153.26ms max = 154.11ms median = 153.64ms mean = 153.65ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112425), (index: 89, score: 4.162658), (index: 984, score: 4.077536),
[89 iters] min = 222.46ms max = 226.33ms median = 225.39ms mean = 225.45ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485474), (index: 22, score: 3.693239), (index: 309, score: 3.691997),
[56 iters] min = 360.51ms max = 361.95ms median = 361.49ms mean = 361.49ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914165), (index: 883, score: 5.001735), (index: 310, score: 4.622919),
[135 iters] min = 148.44ms max = 149.62ms median = 149.14ms mean = 149.13ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528477), (index: 89, score: 4.334186), (index: 720, score: 4.178123),
[101 iters] min = 198.81ms max = 200.55ms median = 199.52ms mean = 199.55ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233641), (index: 309, score: 3.921288), (index: 310, score: 3.807556),
[71 iters] min = 281.75ms max = 283.94ms median = 282.60ms mean = 282.63ms
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885462), (index: 309, score: 4.954110), (index: 310, score: 4.638605),
[242 iters] min = 81.33ms max = 83.18ms median = 82.77ms mean = 82.76ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799909), (index: 309, score: 4.595184), (index: 308, score: 3.817008),
[124 iters] min = 161.50ms max = 162.97ms median = 162.24ms mean = 162.27ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156298), (index: 309, score: 4.532575), (index: 308, score: 4.049804),
[74 iters] min = 267.18ms max = 272.12ms median = 271.49ms mean = 271.42ms
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
mobilevit_xx_small model doesn't exist!!!
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427816), (index: 308, score: 3.451128), (index: 309, score: 3.319762),
[517 iters] min = 38.52ms max = 39.87ms median = 38.74ms mean = 38.75ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089766), (index: 309, score: 3.409033), (index: 113, score: 3.385415),
[378 iters] min = 52.68ms max = 53.95ms median = 52.99ms mean = 52.99ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186354), (index: 644, score: 3.177923),
[311 iters] min = 61.21ms max = 65.42ms median = 64.50ms mean = 64.44ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363821), (index: 108, score: 3.341193), (index: 310, score: 2.929493),
[191 iters] min = 103.36ms max = 107.97ms median = 105.29ms mean = 105.16ms
resnet50 model doesn't exist!!!
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592703), (index: 308, score: 2.354278), (index: 310, score: 2.337049),
[502 iters] min = 37.30ms max = 43.18ms median = 39.85ms mean = 39.86ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554752), (index: 309, score: 2.378344), (index: 108, score: 2.289130),
[210 iters] min = 94.49ms max = 95.74ms median = 95.26ms mean = 95.24ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484585), (index: 861, score: 2.258525), (index: 309, score: 2.134490),
[133 iters] min = 147.74ms max = 151.75ms median = 150.98ms mean = 150.95ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816826), (index: 883, score: 2.518668), (index: 113, score: 2.046140),
[97 iters] min = 205.90ms max = 208.26ms median = 207.23ms mean = 207.25ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089395), (index: 955, score: 2.892825), (index: 947, score: 2.188146),
[57 iters] min = 352.16ms max = 355.89ms median = 353.55ms mean = 353.59mscortex-A78 @ 1 thread @ 2.2GHz trace w/ openblas by gcc-10
$ OMP_NUM_THREADS=1 MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using trace CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767040), (index: 644, score: 4.848301), (index: 108, score: 3.925720),
[103 iters] min = 193.40ms max = 197.48ms median = 195.92ms mean = 195.92ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112434), (index: 89, score: 4.162667), (index: 984, score: 4.077518),
[69 iters] min = 287.46ms max = 296.45ms median = 289.54ms mean = 289.89ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485489), (index: 22, score: 3.693230), (index: 309, score: 3.692008),
[41 iters] min = 480.26ms max = 492.59ms median = 488.87ms mean = 488.35ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914165), (index: 883, score: 5.001730), (index: 310, score: 4.622920),
[186 iters] min = 106.90ms max = 109.52ms median = 107.58ms mean = 107.65ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528473), (index: 89, score: 4.334188), (index: 720, score: 4.178120),
[129 iters] min = 155.07ms max = 157.26ms median = 155.81ms mean = 155.84ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233627), (index: 309, score: 3.921280), (index: 310, score: 3.807575),
[88 iters] min = 225.97ms max = 234.25ms median = 229.67ms mean = 229.65ms
Creating pytorch module: EMO_1M
(index: 985, score: 10.011186), (index: 309, score: 4.270289), (index: 310, score: 3.913450),
[168 iters] min = 118.16ms max = 127.59ms median = 118.97ms mean = 119.20ms
Creating pytorch module: EMO_2M
(index: 985, score: 9.367956), (index: 309, score: 3.259869), (index: 308, score: 3.008148),
[122 iters] min = 163.13ms max = 173.86ms median = 165.02ms mean = 165.15ms
Creating pytorch module: EMO_5M
(index: 985, score: 9.141462), (index: 883, score: 2.990551), (index: 308, score: 2.454388),
[76 iters] min = 263.27ms max = 273.44ms median = 265.93ms mean = 265.88ms
Creating pytorch module: EMO_6M
(index: 985, score: 9.396774), (index: 883, score: 2.240934), (index: 309, score: 2.083858),
[70 iters] min = 285.29ms max = 288.14ms median = 286.27ms mean = 286.32ms
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885462), (index: 309, score: 4.954112), (index: 310, score: 4.638609),
[330 iters] min = 59.02ms max = 62.28ms median = 60.71ms mean = 60.71ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799910), (index: 309, score: 4.595184), (index: 308, score: 3.817011),
[171 iters] min = 115.54ms max = 120.34ms median = 117.16ms mean = 117.22ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156297), (index: 309, score: 4.532578), (index: 308, score: 4.049804),
[97 iters] min = 201.03ms max = 208.43ms median = 206.82ms mean = 206.76ms
Creating pytorch module: mobilevitv2_050
(index: 985, score: 8.315773), (index: 309, score: 2.612401), (index: 584, score: 2.352643),
[181 iters] min = 109.79ms max = 112.34ms median = 110.86ms mean = 110.88ms
Creating pytorch module: mobilevitv2_075
(index: 985, score: 8.129786), (index: 309, score: 2.389379), (index: 308, score: 1.880310),
[107 iters] min = 181.13ms max = 192.89ms median = 186.55ms mean = 187.48ms
Creating pytorch module: mobilevitv2_100
(index: 985, score: 8.256278), (index: 557, score: 2.220436), (index: 309, score: 1.944911),
[72 iters] min = 274.13ms max = 284.53ms median = 279.14ms mean = 278.66ms
Creating pytorch module: mobilevitv2_125
(index: 985, score: 8.281981), (index: 309, score: 1.962245), (index: 883, score: 1.285464),
[53 iters] min = 379.07ms max = 389.75ms median = 384.36ms mean = 384.40ms
Creating pytorch module: mobilevitv2_150
(index: 985, score: 9.098922), (index: 308, score: 2.259605), (index: 301, score: 2.159040),
[40 iters] min = 494.87ms max = 504.51ms median = 500.04ms mean = 500.18ms
Creating pytorch module: mobilevitv2_175
(index: 985, score: 8.888676), (index: 494, score: 2.104782), (index: 309, score: 1.869408),
[32 iters] min = 624.12ms max = 634.28ms median = 630.08ms mean = 630.43ms
Creating pytorch module: mobilevitv2_200
(index: 985, score: 8.531368), (index: 883, score: 2.248763), (index: 309, score: 2.237854),
[26 iters] min = 773.25ms max = 788.49ms median = 780.03ms mean = 779.84ms
Creating pytorch module: mobilevit_xx_small
(index: 985, score: 12.652478), (index: 309, score: 6.357600), (index: 308, score: 6.236126),
[195 iters] min = 102.10ms max = 103.82ms median = 102.98ms mean = 102.99ms
Creating pytorch module: mobilevit_x_small
(index: 985, score: 12.998841), (index: 89, score: 6.411970), (index: 308, score: 5.775462),
[87 iters] min = 221.17ms max = 232.35ms median = 230.93ms mean = 230.74ms
Creating pytorch module: mobilevit_small
(index: 985, score: 10.661433), (index: 838, score: 4.319452), (index: 309, score: 4.076357),
[62 iters] min = 323.17ms max = 325.00ms median = 324.05ms mean = 324.00ms
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427824), (index: 308, score: 3.451133), (index: 309, score: 3.319760),
[523 iters] min = 38.01ms max = 38.85ms median = 38.26ms mean = 38.27ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089764), (index: 309, score: 3.409033), (index: 113, score: 3.385417),
[401 iters] min = 47.24ms max = 50.71ms median = 50.00ms mean = 49.93ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594853), (index: 308, score: 3.186353), (index: 644, score: 3.177923),
[303 iters] min = 65.71ms max = 66.91ms median = 66.20ms mean = 66.22ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363823), (index: 108, score: 3.341187), (index: 310, score: 2.929486),
[187 iters] min = 103.69ms max = 111.44ms median = 107.10ms mean = 107.32ms
Creating pytorch module: resnet50
(index: 985, score: 7.495989), (index: 113, score: -4.947912), (index: 310, score: -5.267945),
[59 iters] min = 343.28ms max = 345.66ms median = 344.29ms mean = 344.38ms
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592708), (index: 308, score: 2.354277), (index: 310, score: 2.337050),
[226 iters] min = 86.97ms max = 90.62ms median = 88.51ms mean = 88.54ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554757), (index: 309, score: 2.378345), (index: 108, score: 2.289133),
[105 iters] min = 189.86ms max = 194.44ms median = 191.26ms mean = 191.33ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484581), (index: 861, score: 2.258524), (index: 309, score: 2.134490),
[75 iters] min = 264.78ms max = 270.31ms median = 266.96ms mean = 267.02ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816823), (index: 883, score: 2.518671), (index: 113, score: 2.046143),
[60 iters] min = 334.38ms max = 342.40ms median = 337.99ms mean = 338.42ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089396), (index: 955, score: 2.892823), (index: 947, score: 2.188144),
[38 iters] min = 534.78ms max = 544.43ms median = 539.04ms mean = 539.32mscortex-A78 @ 1 thread @ 2.2GHz trace w/ openblas by llvm-14
$ OMP_NUM_THREADS=1 MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using trace CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767040), (index: 644, score: 4.848290), (index: 108, score: 3.925722),
[113 iters] min = 176.81ms max = 179.70ms median = 178.45ms mean = 178.44ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112445), (index: 89, score: 4.162668), (index: 984, score: 4.077518),
[77 iters] min = 260.08ms max = 263.29ms median = 261.70ms mean = 261.72ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485485), (index: 22, score: 3.693229), (index: 309, score: 3.692007),
[46 iters] min = 435.45ms max = 441.38ms median = 437.41ms mean = 437.82ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914165), (index: 883, score: 5.001730), (index: 310, score: 4.622921),
[192 iters] min = 103.55ms max = 106.57ms median = 104.49ms mean = 104.67ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528475), (index: 89, score: 4.334196), (index: 720, score: 4.178128),
[134 iters] min = 148.01ms max = 151.07ms median = 150.27ms mean = 150.25ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233629), (index: 309, score: 3.921285), (index: 310, score: 3.807566),
[90 iters] min = 221.63ms max = 224.13ms median = 223.00ms mean = 222.98ms
Creating pytorch module: EMO_1M
(index: 985, score: 10.011185), (index: 309, score: 4.270289), (index: 310, score: 3.913450),
[176 iters] min = 110.46ms max = 115.12ms median = 113.89ms mean = 113.83ms
Creating pytorch module: EMO_2M
(index: 985, score: 9.367955), (index: 309, score: 3.259869), (index: 308, score: 3.008149),
[126 iters] min = 158.16ms max = 160.79ms median = 159.86ms mean = 159.84ms
Creating pytorch module: EMO_5M
(index: 985, score: 9.141462), (index: 883, score: 2.990552), (index: 308, score: 2.454388),
[79 iters] min = 254.16ms max = 256.14ms median = 254.98ms mean = 255.04ms
Creating pytorch module: EMO_6M
(index: 985, score: 9.396772), (index: 883, score: 2.240934), (index: 309, score: 2.083858),
[74 iters] min = 272.59ms max = 274.69ms median = 273.85ms mean = 273.76ms
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885461), (index: 309, score: 4.954110), (index: 310, score: 4.638609),
[335 iters] min = 59.11ms max = 61.63ms median = 59.66ms mean = 59.80ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799911), (index: 309, score: 4.595184), (index: 308, score: 3.817010),
[176 iters] min = 110.72ms max = 119.15ms median = 114.14ms mean = 114.08ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156297), (index: 309, score: 4.532577), (index: 308, score: 4.049804),
[101 iters] min = 199.14ms max = 200.60ms median = 199.64ms mean = 199.71ms
Creating pytorch module: mobilevitv2_050
(index: 985, score: 8.315772), (index: 309, score: 2.612400), (index: 584, score: 2.352643),
[183 iters] min = 106.78ms max = 110.57ms median = 109.47ms mean = 109.51ms
Creating pytorch module: mobilevitv2_075
(index: 985, score: 8.129786), (index: 309, score: 2.389378), (index: 308, score: 1.880310),
[109 iters] min = 182.85ms max = 188.14ms median = 184.45ms mean = 185.17ms
Creating pytorch module: mobilevitv2_100
(index: 985, score: 8.256277), (index: 557, score: 2.220437), (index: 309, score: 1.944912),
[73 iters] min = 270.67ms max = 278.37ms median = 274.31ms mean = 274.15ms
Creating pytorch module: mobilevitv2_125
(index: 985, score: 8.281981), (index: 309, score: 1.962245), (index: 883, score: 1.285464),
[53 iters] min = 374.41ms max = 381.08ms median = 377.27ms mean = 377.47ms
Creating pytorch module: mobilevitv2_150
(index: 985, score: 9.098921), (index: 308, score: 2.259605), (index: 301, score: 2.159040),
[41 iters] min = 492.50ms max = 500.31ms median = 495.64ms mean = 495.88ms
Creating pytorch module: mobilevitv2_175
(index: 985, score: 8.888677), (index: 494, score: 2.104781), (index: 309, score: 1.869408),
[33 iters] min = 618.49ms max = 626.80ms median = 622.91ms mean = 623.02ms
Creating pytorch module: mobilevitv2_200
(index: 985, score: 8.531368), (index: 883, score: 2.248762), (index: 309, score: 2.237853),
[27 iters] min = 762.30ms max = 770.45ms median = 766.59ms mean = 766.92ms
Creating pytorch module: mobilevit_xx_small
(index: 985, score: 12.652476), (index: 309, score: 6.357601), (index: 308, score: 6.236126),
[194 iters] min = 100.03ms max = 104.00ms median = 103.54ms mean = 103.45ms
Creating pytorch module: mobilevit_x_small
(index: 985, score: 12.998842), (index: 89, score: 6.411971), (index: 308, score: 5.775463),
[87 iters] min = 230.05ms max = 231.68ms median = 230.90ms mean = 230.87ms
Creating pytorch module: mobilevit_small
(index: 985, score: 10.661433), (index: 838, score: 4.319451), (index: 309, score: 4.076356),
[62 iters] min = 322.91ms max = 324.86ms median = 323.92ms mean = 323.94ms
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427824), (index: 308, score: 3.451133), (index: 309, score: 3.319760),
[527 iters] min = 37.81ms max = 38.38ms median = 37.99ms mean = 37.99ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089764), (index: 309, score: 3.409033), (index: 113, score: 3.385417),
[401 iters] min = 49.50ms max = 50.28ms median = 49.88ms mean = 49.89ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594853), (index: 308, score: 3.186353), (index: 644, score: 3.177923),
[303 iters] min = 63.09ms max = 66.35ms median = 66.12ms mean = 66.04ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363823), (index: 108, score: 3.341187), (index: 310, score: 2.929486),
[188 iters] min = 106.14ms max = 107.43ms median = 106.53ms mean = 106.54ms
Creating pytorch module: resnet50
(index: 985, score: 7.495990), (index: 113, score: -4.947911), (index: 310, score: -5.267944),
[59 iters] min = 322.77ms max = 342.36ms median = 341.34ms mean = 340.91ms
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592709), (index: 308, score: 2.354278), (index: 310, score: 2.337051),
[249 iters] min = 78.84ms max = 81.62ms median = 80.48ms mean = 80.50ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554757), (index: 309, score: 2.378345), (index: 108, score: 2.289133),
[114 iters] min = 175.42ms max = 186.83ms median = 176.44ms mean = 176.50ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484580), (index: 861, score: 2.258524), (index: 309, score: 2.134490),
[81 iters] min = 247.21ms max = 249.92ms median = 248.21ms mean = 248.24ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816823), (index: 883, score: 2.518671), (index: 113, score: 2.046143),
[63 iters] min = 318.70ms max = 323.30ms median = 320.32ms mean = 320.31ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089397), (index: 955, score: 2.892823), (index: 947, score: 2.188145),
[40 iters] min = 501.14ms max = 513.98ms median = 511.05ms mean = 510.87mscortex-A78 @ 1 thread @ 2.2GHz mobile w/ openblas by gcc-10
$ OMP_NUM_THREADS=1 BACK=c MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using mobile CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767035), (index: 644, score: 4.848308), (index: 108, score: 3.925720),
[246 iters] min = 80.81ms max = 86.74ms median = 81.51ms mean = 81.58ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112425), (index: 89, score: 4.162664), (index: 984, score: 4.077536),
[164 iters] min = 121.28ms max = 125.59ms median = 121.93ms mean = 122.00ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485470), (index: 22, score: 3.693244), (index: 309, score: 3.692000),
[99 iters] min = 202.89ms max = 205.36ms median = 203.56ms mean = 203.62ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914167), (index: 883, score: 5.001737), (index: 310, score: 4.622924),
[225 iters] min = 86.40ms max = 90.57ms median = 88.91ms mean = 88.91ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528477), (index: 89, score: 4.334188), (index: 720, score: 4.178122),
[164 iters] min = 121.46ms max = 123.66ms median = 122.01ms mean = 122.04ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233635), (index: 309, score: 3.921287), (index: 310, score: 3.807558),
[109 iters] min = 182.30ms max = 187.76ms median = 183.48ms mean = 183.90ms
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885465), (index: 309, score: 4.954110), (index: 310, score: 4.638604),
[422 iters] min = 46.97ms max = 49.69ms median = 47.40ms mean = 47.40ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799911), (index: 309, score: 4.595185), (index: 308, score: 3.817010),
[222 iters] min = 89.72ms max = 91.05ms median = 90.24ms mean = 90.25ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156294), (index: 309, score: 4.532573), (index: 308, score: 4.049802),
[125 iters] min = 157.43ms max = 161.50ms median = 160.45ms mean = 160.42ms
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
mobilevit_xx_small model doesn't exist!!!
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427816), (index: 308, score: 3.451128), (index: 309, score: 3.319762),
[643 iters] min = 30.86ms max = 31.60ms median = 31.10ms mean = 31.11ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089766), (index: 309, score: 3.409033), (index: 113, score: 3.385415),
[476 iters] min = 39.86ms max = 42.66ms median = 42.10ms mean = 42.08ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186354), (index: 644, score: 3.177923),
[352 iters] min = 56.52ms max = 57.61ms median = 56.88ms mean = 56.89ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363821), (index: 108, score: 3.341193), (index: 310, score: 2.929493),
[212 iters] min = 93.98ms max = 95.46ms median = 94.62ms mean = 94.66ms
resnet50 model doesn't exist!!!
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592701), (index: 308, score: 2.354276), (index: 310, score: 2.337050),
[524 iters] min = 37.85ms max = 38.77ms median = 38.20ms mean = 38.22ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554751), (index: 309, score: 2.378344), (index: 108, score: 2.289130),
[212 iters] min = 92.22ms max = 95.43ms median = 94.42ms mean = 94.48ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484585), (index: 861, score: 2.258525), (index: 309, score: 2.134489),
[135 iters] min = 148.14ms max = 149.99ms median = 148.86ms mean = 148.91ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816826), (index: 883, score: 2.518668), (index: 113, score: 2.046140),
[98 iters] min = 201.64ms max = 205.55ms median = 204.62ms mean = 204.56ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089395), (index: 955, score: 2.892825), (index: 947, score: 2.188146),
[58 iters] min = 347.59ms max = 352.72ms median = 349.53ms mean = 349.64mscortex-A78 @ 1 thread @ 2.2GHz mobile w/ openblas by llvm-14
$ OMP_NUM_THREADS=1 BACK=c MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using mobile CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767038), (index: 644, score: 4.848302), (index: 108, score: 3.925722),
[242 iters] min = 80.70ms max = 83.36ms median = 82.98ms mean = 82.94ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112428), (index: 89, score: 4.162664), (index: 984, score: 4.077536),
[163 iters] min = 121.95ms max = 123.61ms median = 123.10ms mean = 122.86ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485475), (index: 22, score: 3.693243), (index: 309, score: 3.691999),
[98 iters] min = 204.75ms max = 207.10ms median = 205.82ms mean = 205.79ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914165), (index: 883, score: 5.001738), (index: 310, score: 4.622924),
[223 iters] min = 87.24ms max = 90.33ms median = 89.91ms mean = 89.87ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528475), (index: 89, score: 4.334188), (index: 720, score: 4.178121),
[163 iters] min = 122.60ms max = 123.69ms median = 123.15ms mean = 123.10ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233637), (index: 309, score: 3.921288), (index: 310, score: 3.807558),
[109 iters] min = 180.96ms max = 185.32ms median = 183.94ms mean = 183.86ms
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885464), (index: 309, score: 4.954108), (index: 310, score: 4.638605),
[425 iters] min = 46.86ms max = 64.11ms median = 47.11ms mean = 47.15ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799910), (index: 309, score: 4.595186), (index: 308, score: 3.817011),
[221 iters] min = 90.07ms max = 96.27ms median = 90.74ms mean = 90.78ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156299), (index: 309, score: 4.532575), (index: 308, score: 4.049802),
[125 iters] min = 159.56ms max = 160.93ms median = 160.51ms mean = 160.51ms
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
mobilevit_xx_small model doesn't exist!!!
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427816), (index: 308, score: 3.451128), (index: 309, score: 3.319762),
[659 iters] min = 30.14ms max = 30.74ms median = 30.36ms mean = 30.36ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089766), (index: 309, score: 3.409033), (index: 113, score: 3.385415),
[486 iters] min = 39.21ms max = 41.70ms median = 41.22ms mean = 41.19ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186354), (index: 644, score: 3.177923),
[358 iters] min = 55.51ms max = 56.35ms median = 55.91ms mean = 55.91ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363821), (index: 108, score: 3.341193), (index: 310, score: 2.929493),
[214 iters] min = 88.65ms max = 94.15ms median = 93.59ms mean = 93.48ms
Creating pytorch module: resnet50
(index: 227, score: 26.693110), (index: 334, score: 20.228640), (index: 278, score: 17.633595),
[63 iters] min = 319.68ms max = 321.48ms median = 320.10ms mean = 320.12ms
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592703), (index: 308, score: 2.354278), (index: 310, score: 2.337049),
[527 iters] min = 37.41ms max = 38.43ms median = 37.99ms mean = 37.98ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554752), (index: 309, score: 2.378344), (index: 108, score: 2.289130),
[216 iters] min = 90.73ms max = 93.29ms median = 92.95ms mean = 92.91ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484585), (index: 861, score: 2.258525), (index: 309, score: 2.134490),
[137 iters] min = 146.35ms max = 147.87ms median = 146.84ms mean = 146.94ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816826), (index: 883, score: 2.518668), (index: 113, score: 2.046140),
[99 iters] min = 199.65ms max = 204.26ms median = 202.37ms mean = 202.39ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089395), (index: 955, score: 2.892825), (index: 947, score: 2.188146),
[57 iters] min = 349.01ms max = 352.13ms median = 351.07ms mean = 350.95msversion: 2.0.1 by pip install w/ onednn+acl
cortex-A78 @ 1 thread @ 2.2GHz trace w/ python
$ python python/pytorch_perf.py --use-trace
nb processors 12
model name : ARMv8 Processor rev 1 (v8l)
Using 1 cpu thread
Creating model: efficientformerv2_s0
[(985, 11.767034530639648), (644, 4.848289966583252), (108, 3.925722122192383)]
min = 166.02ms max = 167.90ms mean = 166.86ms, median = 166.83ms
Creating model: efficientformerv2_s1
[(985, 13.112439155578613), (89, 4.162661552429199), (984, 4.077512264251709)]
min = 245.23ms max = 249.38ms mean = 247.71ms, median = 247.67ms
Creating model: efficientformerv2_s2
[(985, 12.485483169555664), (22, 3.6932296752929688), (309, 3.692007303237915)]
min = 394.45ms max = 397.61ms mean = 396.03ms, median = 396.00ms
Creating model: SwiftFormer_XS
[(985, 11.914167404174805), (883, 5.0017242431640625), (310, 4.622915744781494)]
min = 149.97ms max = 153.15ms mean = 151.86ms, median = 151.85ms
Creating model: SwiftFormer_S
[(985, 12.528473854064941), (89, 4.3341875076293945), (720, 4.178118705749512)]
min = 203.74ms max = 205.36ms mean = 204.36ms, median = 204.31ms
Creating model: SwiftFormer_L1
[(985, 13.233625411987305), (309, 3.9212796688079834), (310, 3.807574510574341)]
min = 291.11ms max = 296.47ms mean = 292.02ms, median = 291.78ms
Creating model: EMO_1M
[(985, 10.011184692382812), (309, 4.270286560058594), (310, 3.913449287414551)]
min = 116.32ms max = 122.68ms mean = 117.33ms, median = 116.98ms
Creating model: EMO_2M
[(985, 9.367955207824707), (309, 3.2598681449890137), (308, 3.0081465244293213)]
min = 163.07ms max = 173.73ms mean = 164.14ms, median = 163.75ms
Creating model: EMO_5M
[(985, 9.141463279724121), (883, 2.990551471710205), (308, 2.4543871879577637)]
min = 252.52ms max = 260.39ms mean = 258.85ms, median = 259.00ms
Creating model: EMO_6M
[(985, 9.396775245666504), (883, 2.240933895111084), (309, 2.083859443664551)]
min = 274.68ms max = 276.56ms mean = 275.44ms, median = 275.37ms
Creating model: edgenext_xx_small
[(985, 10.885461807250977), (309, 4.954111099243164), (310, 4.638607978820801)]
min = 91.56ms max = 98.06ms mean = 93.97ms, median = 93.70ms
Creating model: edgenext_x_small
[(985, 9.799908638000488), (309, 4.595181465148926), (308, 3.8170080184936523)]
min = 179.30ms max = 184.00ms mean = 180.66ms, median = 180.22ms
Creating model: edgenext_small
[(985, 12.156298637390137), (309, 4.532576560974121), (308, 4.049804210662842)]
min = 298.61ms max = 303.57ms mean = 299.78ms, median = 299.68ms
Creating model: mobilevitv2_050
[(985, 8.349032402038574), (309, 2.584130048751831), (584, 2.319410800933838)]
min = 115.52ms max = 117.59ms mean = 116.44ms, median = 116.40ms
Creating model: mobilevitv2_075
[(985, 8.16434383392334), (309, 2.3874454498291016), (308, 1.8584961891174316)]
min = 192.97ms max = 195.01ms mean = 194.14ms, median = 194.13ms
Creating model: mobilevitv2_100
[(985, 8.236356735229492), (557, 2.2240982055664062), (309, 1.8535703420639038)]
min = 280.24ms max = 286.70ms mean = 285.54ms, median = 285.51ms
Creating model: mobilevitv2_125
[(985, 8.272746086120605), (309, 2.0136659145355225), (883, 1.3231419324874878)]
min = 389.17ms max = 392.26ms mean = 390.54ms, median = 390.51ms
Creating model: mobilevitv2_150
[(985, 9.097070693969727), (308, 2.224851608276367), (301, 2.1440443992614746)]
min = 498.76ms max = 511.29ms mean = 509.42ms, median = 509.82ms
Creating model: mobilevitv2_175
[(985, 8.880781173706055), (494, 2.0807058811187744), (968, 1.8858556747436523)]
min = 657.37ms max = 676.08ms mean = 666.47ms, median = 666.27ms
Creating model: mobilevitv2_200
[(985, 8.548127174377441), (309, 2.226964235305786), (883, 2.2137231826782227)]
min = 807.79ms max = 815.96ms mean = 813.77ms, median = 814.32ms
Creating model: mobilevit_xx_small
[(985, 12.629860877990723), (309, 6.416606426239014), (308, 6.263427734375)]
min = 120.40ms max = 122.26ms mean = 121.19ms, median = 121.14ms
Creating model: mobilevit_x_small
[(985, 13.033150672912598), (89, 6.419534683227539), (308, 5.793033599853516)]
min = 263.66ms max = 267.87ms mean = 265.09ms, median = 264.66ms
Creating model: mobilevit_small
[(985, 10.672835350036621), (838, 4.352145671844482), (309, 4.135124206542969)]
min = 364.37ms max = 369.40ms mean = 366.32ms, median = 365.74ms
Creating model: LeViT_128S
[(985, 11.427818298339844), (308, 3.451131820678711), (309, 3.319760322570801)]
min = 42.68ms max = 45.90ms mean = 43.08ms, median = 42.99ms
Creating model: LeViT_128
[(985, 11.089767456054688), (309, 3.4090335369110107), (113, 3.385415554046631)]
min = 52.56ms max = 56.68ms mean = 55.93ms, median = 55.97ms
Creating model: LeViT_192
[(985, 11.594854354858398), (308, 3.1863551139831543), (644, 3.1779229640960693)]
min = 72.21ms max = 73.51ms mean = 72.81ms, median = 72.79ms
Creating model: LeViT_256
[(985, 11.363821983337402), (108, 3.3411974906921387), (310, 2.929494619369507)]
min = 109.99ms max = 117.34ms mean = 116.19ms, median = 116.23ms
Creating model: resnet50
[(985, 7.4433512687683105), (113, -5.0514445304870605), (310, -5.506593227386475)]
min = 472.29ms max = 484.23ms mean = 474.29ms, median = 473.57ms
Creating model: mobilenetv3_large_100
[(985, 9.592708587646484), (308, 2.354276180267334), (310, 2.337049722671509)]
min = 61.33ms max = 74.30ms mean = 61.95ms, median = 61.69ms
Creating model: tf_efficientnetv2_b0
[(985, 9.554756164550781), (309, 2.3783445358276367), (108, 2.289132595062256)]
min = 138.34ms max = 140.11ms mean = 139.03ms, median = 138.98ms
Creating model: tf_efficientnetv2_b1
[(985, 9.484580039978027), (861, 2.2585256099700928), (309, 2.1344892978668213)]
min = 211.04ms max = 214.46ms mean = 212.59ms, median = 212.51ms
Creating model: tf_efficientnetv2_b2
[(985, 9.816822052001953), (883, 2.5186715126037598), (113, 2.046143054962158)]
min = 284.76ms max = 291.15ms mean = 287.62ms, median = 287.07ms
Creating model: tf_efficientnetv2_b3
[(985, 9.089397430419922), (955, 2.892822265625), (947, 2.1881449222564697)]
min = 473.44ms max = 479.38ms mean = 476.42ms, median = 476.54mscortex-A78 @ 1 thread @ 2.2GHz mobile w/ python
$ python python/pytorch_perf.py --use-mobile
nb processors 12
model name : ARMv8 Processor rev 1 (v8l)
Using 1 cpu thread
Creating model: efficientformerv2_s0
[(985, 11.767034530639648), (644, 4.8482985496521), (108, 3.925720691680908)]
min = 139.63ms max = 144.78ms mean = 141.39ms, median = 141.07ms
Creating model: efficientformerv2_s1
[(985, 13.112424850463867), (89, 4.162662506103516), (984, 4.077541828155518)]
min = 208.07ms max = 214.96ms mean = 209.95ms, median = 209.47ms
Creating model: efficientformerv2_s2
[(985, 12.485469818115234), (22, 3.6932387351989746), (309, 3.6919991970062256)]
min = 332.43ms max = 337.24ms mean = 333.87ms, median = 333.32ms
Creating model: SwiftFormer_XS
[(985, 11.914167404174805), (883, 5.001735210418701), (310, 4.622920036315918)]
min = 141.26ms max = 148.91ms mean = 143.21ms, median = 142.86ms
Creating model: SwiftFormer_S
[(985, 12.52847671508789), (89, 4.334184646606445), (720, 4.178120136260986)]
min = 188.78ms max = 190.36ms mean = 189.37ms, median = 189.34ms
Creating model: SwiftFormer_L1
[(985, 13.233636856079102), (309, 3.921288251876831), (310, 3.8075551986694336)]
min = 270.77ms max = 276.59ms mean = 272.08ms, median = 271.51ms
Creating model: EMO_1M
EMO_1M model doesn't exist!!!
Creating model: EMO_2M
EMO_2M model doesn't exist!!!
Creating model: EMO_5M
EMO_5M model doesn't exist!!!
Creating model: EMO_6M
EMO_6M model doesn't exist!!!
Creating model: edgenext_xx_small
[(985, 10.885459899902344), (309, 4.954109191894531), (310, 4.63860559463501)]
min = 78.27ms max = 82.68ms mean = 78.91ms, median = 78.64ms
Creating model: edgenext_x_small
[(985, 9.799910545349121), (309, 4.595183372497559), (308, 3.817009449005127)]
min = 152.27ms max = 155.67ms mean = 154.24ms, median = 154.18ms
Creating model: edgenext_small
[(985, 12.156299591064453), (309, 4.532577037811279), (308, 4.0498046875)]
min = 258.18ms max = 260.88ms mean = 259.02ms, median = 258.92ms
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
mobilevit_xx_small model doesn't exist!!!
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
Creating model: LeViT_128S
[(985, 11.427816390991211), (308, 3.451131582260132), (309, 3.319758176803589)]
min = 33.06ms max = 36.39ms mean = 33.36ms, median = 33.32ms
Creating model: LeViT_128
[(985, 11.089767456054688), (309, 3.40903377532959), (113, 3.3854129314422607)]
min = 44.99ms max = 53.04ms mean = 45.32ms, median = 45.27ms
Creating model: LeViT_192
[(985, 11.594850540161133), (308, 3.1863536834716797), (644, 3.177922248840332)]
min = 59.14ms max = 60.33ms mean = 59.62ms, median = 59.60ms
Creating model: LeViT_256
[(985, 11.363815307617188), (108, 3.341197967529297), (310, 2.929499387741089)]
min = 92.87ms max = 98.64ms mean = 97.72ms, median = 97.73ms
resnet50 model doesn't exist!!!
Creating model: mobilenetv3_large_100
[(985, 9.59270191192627), (308, 2.354276418685913), (310, 2.3370494842529297)]
min = 39.37ms max = 41.29ms mean = 39.85ms, median = 39.80ms
Creating model: tf_efficientnetv2_b0
[(985, 9.554752349853516), (309, 2.3783442974090576), (108, 2.289130449295044)]
min = 101.98ms max = 107.25ms mean = 104.74ms, median = 104.66ms
Creating model: tf_efficientnetv2_b1
[(985, 9.48458480834961), (861, 2.25852632522583), (309, 2.1344892978668213)]
min = 162.56ms max = 170.07ms mean = 164.28ms, median = 163.77ms
Creating model: tf_efficientnetv2_b2
[(985, 9.816827774047852), (883, 2.518669605255127), (113, 2.0461411476135254)]
min = 221.56ms max = 228.30ms mean = 223.40ms, median = 222.92ms
Creating model: tf_efficientnetv2_b3
[(985, 9.08939266204834), (955, 2.8928256034851074), (947, 2.188145399093628)]
min = 375.23ms max = 379.12ms mean = 377.04ms, median = 377.10mscortex-A78 @ 1 thread @ 2.2GHz trace
$ OMP_NUM_THREADS=1 MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using trace CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767033), (index: 644, score: 4.848289), (index: 108, score: 3.925720),
[121 iters] min = 165.37ms max = 167.15ms median = 165.61ms mean = 165.72ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112442), (index: 89, score: 4.162664), (index: 984, score: 4.077511),
[81 iters] min = 247.34ms max = 249.54ms median = 247.98ms mean = 248.06ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485486), (index: 22, score: 3.693231), (index: 309, score: 3.692009),
[51 iters] min = 396.71ms max = 398.54ms median = 397.64ms mean = 397.59ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914165), (index: 883, score: 5.001728), (index: 310, score: 4.622917),
[132 iters] min = 150.56ms max = 160.16ms median = 151.26ms mean = 151.52ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528474), (index: 89, score: 4.334189), (index: 720, score: 4.178121),
[99 iters] min = 200.23ms max = 205.77ms median = 203.55ms mean = 203.62ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233625), (index: 309, score: 3.921280), (index: 310, score: 3.807572),
[69 iters] min = 288.56ms max = 293.66ms median = 290.31ms mean = 290.59ms
Creating pytorch module: EMO_1M
(index: 985, score: 10.011185), (index: 309, score: 4.270287), (index: 310, score: 3.913450),
[173 iters] min = 112.83ms max = 126.63ms median = 115.99ms mean = 116.12ms
Creating pytorch module: EMO_2M
(index: 985, score: 9.367957), (index: 309, score: 3.259868), (index: 308, score: 3.008149),
[125 iters] min = 160.38ms max = 164.09ms median = 160.81ms mean = 161.19ms
Creating pytorch module: EMO_5M
(index: 985, score: 9.141463), (index: 883, score: 2.990551), (index: 308, score: 2.454388),
[79 iters] min = 254.82ms max = 267.10ms median = 255.47ms mean = 256.05ms
Creating pytorch module: EMO_6M
(index: 985, score: 9.396775), (index: 883, score: 2.240934), (index: 309, score: 2.083860),
[74 iters] min = 270.30ms max = 272.76ms median = 271.56ms mean = 271.58ms
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885459), (index: 309, score: 4.954109), (index: 310, score: 4.638605),
[218 iters] min = 91.59ms max = 93.52ms median = 91.92ms mean = 91.96ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799909), (index: 309, score: 4.595184), (index: 308, score: 3.817008),
[114 iters] min = 174.25ms max = 177.54ms median = 176.33ms mean = 176.38ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156297), (index: 309, score: 4.532575), (index: 308, score: 4.049803),
[68 iters] min = 296.56ms max = 298.85ms median = 297.46ms mean = 297.48ms
Creating pytorch module: mobilevitv2_050
(index: 985, score: 8.315772), (index: 309, score: 2.612400), (index: 584, score: 2.352643),
[172 iters] min = 112.86ms max = 117.57ms median = 116.74ms mean = 116.72ms
Creating pytorch module: mobilevitv2_075
(index: 985, score: 8.129784), (index: 309, score: 2.389378), (index: 308, score: 1.880310),
[102 iters] min = 193.56ms max = 202.30ms median = 198.39ms mean = 197.31ms
Creating pytorch module: mobilevitv2_100
(index: 985, score: 8.256277), (index: 557, score: 2.220435), (index: 309, score: 1.944912),
[70 iters] min = 285.51ms max = 294.48ms median = 286.27ms mean = 287.78ms
Creating pytorch module: mobilevitv2_125
(index: 985, score: 8.281982), (index: 309, score: 1.962244), (index: 883, score: 1.285465),
[51 iters] min = 391.75ms max = 402.74ms median = 399.40ms mean = 399.33ms
Creating pytorch module: mobilevitv2_150
(index: 985, score: 9.098922), (index: 308, score: 2.259605), (index: 301, score: 2.159039),
[39 iters] min = 513.04ms max = 524.21ms median = 519.64ms mean = 519.52ms
Creating pytorch module: mobilevitv2_175
(index: 985, score: 8.888678), (index: 494, score: 2.104781), (index: 309, score: 1.869409),
[31 iters] min = 643.05ms max = 656.05ms median = 652.33ms mean = 651.91ms
Creating pytorch module: mobilevitv2_200
(index: 985, score: 8.531369), (index: 883, score: 2.248763), (index: 309, score: 2.237854),
[25 iters] min = 799.28ms max = 806.07ms median = 802.87ms mean = 802.74ms
Creating pytorch module: mobilevit_xx_small
(index: 985, score: 12.652477), (index: 309, score: 6.357602), (index: 308, score: 6.236127),
[167 iters] min = 117.28ms max = 120.77ms median = 119.76ms mean = 119.82ms
Creating pytorch module: mobilevit_x_small
(index: 985, score: 12.998840), (index: 89, score: 6.411970), (index: 308, score: 5.775462),
[77 iters] min = 260.51ms max = 262.95ms median = 261.21ms mean = 261.27ms
Creating pytorch module: mobilevit_small
(index: 985, score: 10.661434), (index: 838, score: 4.319452), (index: 309, score: 4.076357),
[55 iters] min = 363.30ms max = 365.36ms median = 364.04ms mean = 364.09ms
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427818), (index: 308, score: 3.451128), (index: 309, score: 3.319760),
[477 iters] min = 38.99ms max = 45.49ms median = 41.84ms mean = 41.94ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089766), (index: 309, score: 3.409032), (index: 113, score: 3.385416),
[368 iters] min = 54.26ms max = 54.90ms median = 54.46ms mean = 54.48ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186357), (index: 644, score: 3.177924),
[280 iters] min = 67.60ms max = 72.25ms median = 71.64ms mean = 71.57ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363825), (index: 108, score: 3.341189), (index: 310, score: 2.929488),
[175 iters] min = 114.56ms max = 115.63ms median = 114.85ms mean = 114.91ms
Creating pytorch module: resnet50
(index: 985, score: 7.495994), (index: 113, score: -4.947914), (index: 310, score: -5.267949),
[42 iters] min = 474.12ms max = 502.11ms median = 484.36ms mean = 485.96ms
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592712), (index: 308, score: 2.354277), (index: 310, score: 2.337051),
[322 iters] min = 61.88ms max = 64.80ms median = 62.19ms mean = 62.28ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554757), (index: 309, score: 2.378345), (index: 108, score: 2.289133),
[144 iters] min = 138.37ms max = 142.09ms median = 138.90ms mean = 139.25ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484579), (index: 861, score: 2.258524), (index: 309, score: 2.134490),
[95 iters] min = 207.48ms max = 223.66ms median = 212.30ms mean = 212.54ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816822), (index: 883, score: 2.518670), (index: 113, score: 2.046143),
[71 iters] min = 283.27ms max = 290.88ms median = 285.11ms mean = 285.18ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089396), (index: 955, score: 2.892823), (index: 947, score: 2.188146),
[43 iters] min = 448.62ms max = 473.90ms median = 472.82ms mean = 471.88mscortex-A78 @ 1 thread @ 2.2GHz mobile
$ BACK=c MODEL=ALL make run-torch-perf
INFO: Using num_threads == 1
INFO: Using mobile CPU backend
Creating pytorch module: efficientformerv2_s0
(index: 985, score: 11.767030), (index: 644, score: 4.848297), (index: 108, score: 3.925721),
[144 iters] min = 136.59ms max = 143.64ms median = 139.23ms mean = 139.38ms
Creating pytorch module: efficientformerv2_s1
(index: 985, score: 13.112427), (index: 89, score: 4.162661), (index: 984, score: 4.077535),
[98 iters] min = 205.03ms max = 209.48ms median = 205.74ms mean = 206.07ms
Creating pytorch module: efficientformerv2_s2
(index: 985, score: 12.485474), (index: 22, score: 3.693241), (index: 309, score: 3.691998),
[61 iters] min = 330.26ms max = 332.20ms median = 330.83ms mean = 330.99ms
Creating pytorch module: SwiftFormer_XS
(index: 985, score: 11.914167), (index: 883, score: 5.001735), (index: 310, score: 4.622921),
[143 iters] min = 139.71ms max = 146.37ms median = 140.21ms mean = 140.56ms
Creating pytorch module: SwiftFormer_S
(index: 985, score: 12.528477), (index: 89, score: 4.334185), (index: 720, score: 4.178122),
[107 iters] min = 186.75ms max = 193.17ms median = 187.02ms mean = 187.57ms
Creating pytorch module: SwiftFormer_L1
(index: 985, score: 13.233639), (index: 309, score: 3.921288), (index: 310, score: 3.807554),
[75 iters] min = 266.26ms max = 271.24ms median = 268.94ms mean = 268.98ms
EMO_1M model doesn't exist!!!
EMO_2M model doesn't exist!!!
EMO_5M model doesn't exist!!!
EMO_6M model doesn't exist!!!
Creating pytorch module: edgenext_xx_small
(index: 985, score: 10.885463), (index: 309, score: 4.954110), (index: 310, score: 4.638607),
[261 iters] min = 76.37ms max = 77.60ms median = 76.70ms mean = 76.74ms
Creating pytorch module: edgenext_x_small
(index: 985, score: 9.799910), (index: 309, score: 4.595183), (index: 308, score: 3.817009),
[132 iters] min = 149.59ms max = 152.76ms median = 151.90ms mean = 151.90ms
Creating pytorch module: edgenext_small
(index: 985, score: 12.156300), (index: 309, score: 4.532576), (index: 308, score: 4.049804),
[79 iters] min = 255.37ms max = 258.33ms median = 255.93ms mean = 256.12ms
mobilevitv2_050 model doesn't exist!!!
mobilevitv2_075 model doesn't exist!!!
mobilevitv2_100 model doesn't exist!!!
mobilevitv2_125 model doesn't exist!!!
mobilevitv2_150 model doesn't exist!!!
mobilevitv2_175 model doesn't exist!!!
mobilevitv2_200 model doesn't exist!!!
mobilevit_xx_small model doesn't exist!!!
mobilevit_x_small model doesn't exist!!!
mobilevit_small model doesn't exist!!!
Creating pytorch module: LeViT_128S
(index: 985, score: 11.427816), (index: 308, score: 3.451128), (index: 309, score: 3.319762),
[634 iters] min = 31.30ms max = 34.89ms median = 31.50ms mean = 31.59ms
Creating pytorch module: LeViT_128
(index: 985, score: 11.089766), (index: 309, score: 3.409033), (index: 113, score: 3.385415),
[462 iters] min = 42.89ms max = 52.76ms median = 43.19ms mean = 43.32ms
Creating pytorch module: LeViT_192
(index: 985, score: 11.594851), (index: 308, score: 3.186354), (index: 644, score: 3.177923),
[347 iters] min = 57.29ms max = 62.44ms median = 57.63ms mean = 57.73ms
Creating pytorch module: LeViT_256
(index: 985, score: 11.363821), (index: 108, score: 3.341193), (index: 310, score: 2.929493),
[210 iters] min = 90.09ms max = 98.32ms median = 95.11ms mean = 95.25ms
resnet50 model doesn't exist!!!
Creating pytorch module: mobilenetv3_large_100
(index: 985, score: 9.592701), (index: 308, score: 2.354276), (index: 310, score: 2.337050),
[542 iters] min = 34.99ms max = 46.17ms median = 36.92ms mean = 36.96ms
Creating pytorch module: tf_efficientnetv2_b0
(index: 985, score: 9.554751), (index: 309, score: 2.378344), (index: 108, score: 2.289130),
[198 iters] min = 101.07ms max = 101.89ms median = 101.33ms mean = 101.34ms
Creating pytorch module: tf_efficientnetv2_b1
(index: 985, score: 9.484585), (index: 861, score: 2.258525), (index: 309, score: 2.134489),
[125 iters] min = 158.93ms max = 161.54ms median = 160.29ms mean = 160.30ms
Creating pytorch module: tf_efficientnetv2_b2
(index: 985, score: 9.816826), (index: 883, score: 2.518668), (index: 113, score: 2.046140),
[92 iters] min = 215.94ms max = 219.25ms median = 218.15ms mean = 218.19ms
Creating pytorch module: tf_efficientnetv2_b3
(index: 985, score: 9.089395), (index: 955, score: 2.892825), (index: 947, score: 2.188146),
[54 iters] min = 369.63ms max = 372.97ms median = 371.07ms mean = 371.07ms