Logs
MAXN synchronized execution
- patch for synchronized execution
diff --git a/closed/NVIDIA/code/harness/lwis/include/lwis.hpp b/closed/NVIDIA/code/harness/lwis/include/lwis.hpp
index 69979077..2158326f 100755
--- a/closed/NVIDIA/code/harness/lwis/include/lwis.hpp
+++ b/closed/NVIDIA/code/harness/lwis/include/lwis.hpp
@@ -73,7 +73,7 @@ namespace lwis {
struct ServerSettings {
bool EnableCudaGraphs{false};
- bool EnableSyncOnEvent{false};
+ bool EnableSyncOnEvent{true};
bool EnableSpinWait{false};
bool EnableDeviceScheduleSpin{false};
bool EnableDma{true};
---------------------
{'action': 'run_harness', 'benchmarks': 'ssd-small', 'configs': '', 'scenarios': 'SingleStream', 'no_gpu': False, 'gpu_only': False}
---------------------
[2020-01-06 11:35:22,329 main.py:294 INFO] Using config files: measurements/Xavier/ssd-small/SingleStream/config.json
[2020-01-06 11:35:22,330 __init__.py:144 INFO] Parsing config file measurements/Xavier/ssd-small/SingleStream/config.json ...
-------------------------
[{'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'scenario': 'SingleStream', 'ssd-small': {'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False}, 'system_id': 'Xavier'}]
-------------------------
[2020-01-06 11:35:22,330 main.py:301 INFO] Processing config "Xavier_ssd-small_SingleStream"
[2020-01-06 11:35:22,330 main.py:111 INFO] Running harness for ssd-small benchmark in SingleStream scenario...
{'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False, 'system_id': 'Xavier', 'scenario': 'SingleStream', 'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'test_mode': 'PerformanceOnly', 'log_dir': '/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2020.01.06-11.35.22'}
[2020-01-06 11:35:22,339 __init__.py:42 INFO] Running command: ./build/bin/harness_default --plugins="build/plugins/NMSOptPlugin/libnmsoptplugin.so" --logfile_outdir="/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2020.01.06-11.35.22/Xavier/ssd-small/SingleStream" --logfile_prefix="mlperf_log_" --test_mode="PerformanceOnly" --use_graphs=false --gpu_batch_size=1 --map_path="data_maps/coco/val_map.txt" --tensor_path="${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4" --gpu_engines="./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan" --performance_sample_count=256 --max_dlas=0 --single_stream_expected_latency_ns=1621000 --mlperf_conf_path="measurements/Xavier/ssd-small/SingleStream/mlperf.conf" --user_conf_path="measurements/Xavier/ssd-small/SingleStream/user.conf" --scenario SingleStream --model ssd-small --response_postprocess coco
&&&& RUNNING Default_Harness # ./build/bin/harness_default
[I] mlperf.conf path: measurements/Xavier/ssd-small/SingleStream/mlperf.conf
[I] user.conf path: measurements/Xavier/ssd-small/SingleStream/user.conf
[I] Device:0: ./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan has been successfully loaded.
[I] Creating batcher thread: 0 EnableBatcherThreadPerDevice: false
Starting warmup. Running for a minimum of 5 seconds.
Finished warmup. Ran for 5.01295s.
================================================
MLPerf Results Summary
================================================
SUT name : LWIS_Server
Scenario : Single Stream
Mode : Performance
90th percentile latency (ns) : 1544633
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
================================================
Additional Stats
================================================
QPS w/ loadgen overhead : 653.29
QPS w/o loadgen overhead : 661.35
Min latency (ns) : 1435380
Max latency (ns) : 5293859
Mean latency (ns) : 1512068
50.00 percentile latency (ns) : 1494518
90.00 percentile latency (ns) : 1544633
95.00 percentile latency (ns) : 1631996
97.00 percentile latency (ns) : 1696670
99.00 percentile latency (ns) : 1803842
99.90 percentile latency (ns) : 2282356
================================================
Test Parameters Used
================================================
samples_per_query : 1
target_qps : 616.903
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 60000
max_duration (ms): 0
min_query_count : 1024
max_query_count : 0
qsl_rng_seed : 3133965575612453542
sample_index_rng_seed : 665484352860916858
schedule_rng_seed : 3622009729038561421
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
print_timestamps : false
performance_issue_unique : false
performance_issue_same : false
performance_issue_same_index : 0
performance_sample_count : 256
No warnings encountered during test.
No errors encountered during test.
Device Device:0 processed:
39199 batches of size 1
Memcpy Calls: 0
PerSampleCudaMemcpy Calls: 0
BatchedCudaMemcpy Calls: 39199
&&&& PASSED Default_Harness # ./build/bin/harness_default
[2020-01-06 11:36:31,109 main.py:142 INFO] Result: 90th percentile latency (ns) : 1544633 and Result is : VALID
======================= Perf harness results: =======================
Xavier-SingleStream:
ssd-small: 90th percentile latency (ns) : 1544633 and Result is : VALID
======================= Accuracy results: =======================
Xavier-SingleStream:
ssd-small: No accuracy results in PerformanceOnly mode.
MODE_10W
---------------------
{'action': 'run_harness', 'benchmarks': 'ssd-small', 'configs': '', 'scenarios': 'SingleStream', 'no_gpu': False, 'gpu_only': False}
---------------------
[2019-12-17 15:56:01,516 main.py:294 INFO] Using config files: measurements/Xavier/ssd-small/SingleStream/config.json
[2019-12-17 15:56:01,517 __init__.py:142 INFO] Parsing config file measurements/Xavier/ssd-small/SingleStream/config.json ...
-------------------------
[{'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'scenario': 'SingleStream', 'ssd-small': {'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False}, 'system_id': 'Xavier'}]
-------------------------
[2019-12-17 15:56:01,518 main.py:301 INFO] Processing config "Xavier_ssd-small_SingleStream"
[2019-12-17 15:56:01,519 main.py:111 INFO] Running harness for ssd-small benchmark in SingleStream scenario...
{'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False, 'system_id': 'Xavier', 'scenario': 'SingleStream', 'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'test_mode': 'PerformanceOnly', 'log_dir': '/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2019.12.17-15.56.01'}
[2019-12-17 15:56:01,536 __init__.py:42 INFO] Running command: ./build/bin/harness_default --plugins="build/plugins/NMSOptPlugin/libnmsoptplugin.so" --logfile_outdir="/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2019.12.17-15.56.01/Xavier/ssd-small/SingleStream" --logfile_prefix="mlperf_log_" --test_mode="PerformanceOnly" --use_graphs=false --gpu_batch_size=1 --map_path="data_maps/coco/val_map.txt" --tensor_path="${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4" --gpu_engines="./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan" --performance_sample_count=256 --max_dlas=0 --single_stream_expected_latency_ns=1621000 --mlperf_conf_path="measurements/Xavier/ssd-small/SingleStream/mlperf.conf" --user_conf_path="measurements/Xavier/ssd-small/SingleStream/user.conf" --scenario SingleStream --model ssd-small --response_postprocess coco
&&&& RUNNING Default_Harness # ./build/bin/harness_default
[I] mlperf.conf path: measurements/Xavier/ssd-small/SingleStream/mlperf.conf
[I] user.conf path: measurements/Xavier/ssd-small/SingleStream/user.conf
[W] [TRT] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[I] Device:0: ./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan has been successfully loaded.
[I] Creating batcher thread: 0 EnableBatcherThreadPerDevice: false
Starting warmup. Running for a minimum of 5 seconds.
Finished warmup. Ran for 5.04909s.
================================================
MLPerf Results Summary
================================================
SUT name : LWIS_Server
Scenario : Single Stream
Mode : Performance
90th percentile latency (ns) : 4750275
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
================================================
Additional Stats
================================================
QPS w/ loadgen overhead : 211.59
QPS w/o loadgen overhead : 212.80
Min latency (ns) : 4617990
Max latency (ns) : 5809088
Mean latency (ns) : 4699234
50.00 percentile latency (ns) : 4691721
90.00 percentile latency (ns) : 4750275
95.00 percentile latency (ns) : 4772020
97.00 percentile latency (ns) : 4786384
99.00 percentile latency (ns) : 4827168
99.90 percentile latency (ns) : 5023592
================================================
Test Parameters Used
================================================
samples_per_query : 1
target_qps : 616.903
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 60000
max_duration (ms): 0
min_query_count : 1024
max_query_count : 0
qsl_rng_seed : 3133965575612453542
sample_index_rng_seed : 665484352860916858
schedule_rng_seed : 3622009729038561421
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
print_timestamps : false
performance_issue_unique : false
performance_issue_same : false
performance_issue_same_index : 0
performance_sample_count : 256
No warnings encountered during test.
No errors encountered during test.
Device Device:0 processed:
12697 batches of size 1
Memcpy Calls: 0
PerSampleCudaMemcpy Calls: 0
BatchedCudaMemcpy Calls: 12697
&&&& PASSED Default_Harness # ./build/bin/harness_default
[2019-12-17 15:57:17,204 main.py:142 INFO] Result: 90th percentile latency (ns) : 4750275 and Result is : VALID
======================= Perf harness results: =======================
Xavier-SingleStream:
ssd-small: 90th percentile latency (ns) : 4750275 and Result is : VALID
======================= Accuracy results: =======================
Xavier-SingleStream:
ssd-small: No accuracy results in PerformanceOnly mode.
MODE_15W
---------------------
{'action': 'run_harness', 'benchmarks': 'ssd-small', 'configs': '', 'scenarios': 'SingleStream', 'no_gpu': False, 'gpu_only': False}
---------------------
[2019-12-17 15:40:06,320 main.py:294 INFO] Using config files: measurements/Xavier/ssd-small/SingleStream/config.json
[2019-12-17 15:40:06,322 __init__.py:142 INFO] Parsing config file measurements/Xavier/ssd-small/SingleStream/config.json ...
-------------------------
[{'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'scenario': 'SingleStream', 'ssd-small': {'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False}, 'system_id': 'Xavier'}]
-------------------------
[2019-12-17 15:40:06,323 main.py:301 INFO] Processing config "Xavier_ssd-small_SingleStream"
[2019-12-17 15:40:06,323 main.py:111 INFO] Running harness for ssd-small benchmark in SingleStream scenario...
{'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False, 'system_id': 'Xavier', 'scenario': 'SingleStream', 'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'test_mode': 'PerformanceOnly', 'log_dir': '/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2019.12.17-15.40.06'}
[2019-12-17 15:40:06,338 __init__.py:42 INFO] Running command: ./build/bin/harness_default --plugins="build/plugins/NMSOptPlugin/libnmsoptplugin.so" --logfile_outdir="/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2019.12.17-15.40.06/Xavier/ssd-small/SingleStream" --logfile_prefix="mlperf_log_" --test_mode="PerformanceOnly" --use_graphs=false --gpu_batch_size=1 --map_path="data_maps/coco/val_map.txt" --tensor_path="${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4" --gpu_engines="./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan" --performance_sample_count=256 --max_dlas=0 --single_stream_expected_latency_ns=1621000 --mlperf_conf_path="measurements/Xavier/ssd-small/SingleStream/mlperf.conf" --user_conf_path="measurements/Xavier/ssd-small/SingleStream/user.conf" --scenario SingleStream --model ssd-small --response_postprocess coco
&&&& RUNNING Default_Harness # ./build/bin/harness_default
[I] mlperf.conf path: measurements/Xavier/ssd-small/SingleStream/mlperf.conf
[I] user.conf path: measurements/Xavier/ssd-small/SingleStream/user.conf
[I] Device:0: ./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan has been successfully loaded.
[I] Creating batcher thread: 0 EnableBatcherThreadPerDevice: false
Starting warmup. Running for a minimum of 5 seconds.
Finished warmup. Ran for 5.02971s.
================================================
MLPerf Results Summary
================================================
SUT name : LWIS_Server
Scenario : Single Stream
Mode : Performance
90th percentile latency (ns) : 2711245
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
================================================
Additional Stats
================================================
QPS w/ loadgen overhead : 368.99
QPS w/o loadgen overhead : 371.46
Min latency (ns) : 2634089
Max latency (ns) : 4594070
Mean latency (ns) : 2692104
50.00 percentile latency (ns) : 2688907
90.00 percentile latency (ns) : 2711245
95.00 percentile latency (ns) : 2721837
97.00 percentile latency (ns) : 2728493
99.00 percentile latency (ns) : 2746509
99.90 percentile latency (ns) : 2802929
================================================
Test Parameters Used
================================================
samples_per_query : 1
target_qps : 616.903
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 60000
max_duration (ms): 0
min_query_count : 1024
max_query_count : 0
qsl_rng_seed : 3133965575612453542
sample_index_rng_seed : 665484352860916858
schedule_rng_seed : 3622009729038561421
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
print_timestamps : false
performance_issue_unique : false
performance_issue_same : false
performance_issue_same_index : 0
performance_sample_count : 256
No warnings encountered during test.
No errors encountered during test.
Device Device:0 processed:
22141 batches of size 1
Memcpy Calls: 0
PerSampleCudaMemcpy Calls: 0
BatchedCudaMemcpy Calls: 22141
&&&& PASSED Default_Harness # ./build/bin/harness_default
[2019-12-17 15:41:17,750 main.py:142 INFO] Result: 90th percentile latency (ns) : 2711245 and Result is : VALID
======================= Perf harness results: =======================
Xavier-SingleStream:
ssd-small: 90th percentile latency (ns) : 2711245 and Result is : VALID
======================= Accuracy results: =======================
Xavier-SingleStream:
ssd-small: No accuracy results in PerformanceOnly mode.
MODE_30W_6CORE
[2019-12-16 17:32:29,163 main.py:291 INFO] Using config files: measurements/Xavier/ssd-small/SingleStream/config.json
[2019-12-16 17:32:29,165 __init__.py:142 INFO] Parsing config file measurements/Xavier/ssd-small/SingleStream/config.json ...
[2019-12-16 17:32:29,165 main.py:295 INFO] Processing config "Xavier_ssd-small_SingleStream"
[2019-12-16 17:32:29,165 main.py:111 INFO] Running harness for ssd-small benchmark in SingleStream scenario...
{'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCE
SSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False, 'system_id': 'Xavier', 'scenario': 'SingleStream', 'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'test_mod
e': 'PerformanceOnly', 'log_dir': '/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2019.12.16-17.32.28'}
[2019-12-16 17:32:29,180 __init__.py:42 INFO] Running command: ./build/bin/harness_default --plugins="build/plugins/NMSOptPlugin/libnmsoptplugin.so" --logfile_outdir="/home/nvidia/data/inference_results_v0.5/
closed/NVIDIA/build/logs/2019.12.16-17.32.28/Xavier/ssd-small/SingleStream" --logfile_prefix="mlperf_log_" --test_mode="PerformanceOnly" --use_graphs=false --gpu_batch_size=1 --map_path="data_maps/coco/val_ma
p.txt" --tensor_path="${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4" --gpu_engines="./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan" --performance_sample_c
ount=256 --max_dlas=0 --single_stream_expected_latency_ns=1621000 --mlperf_conf_path="measurements/Xavier/ssd-small/SingleStream/mlperf.conf" --user_conf_path="measurements/Xavier/ssd-small/SingleStream/user.
conf" --scenario SingleStream --model ssd-small --response_postprocess coco
&&&& RUNNING Default_Harness # ./build/bin/harness_default
[I] mlperf.conf path: measurements/Xavier/ssd-small/SingleStream/mlperf.conf
[I] user.conf path: measurements/Xavier/ssd-small/SingleStream/user.conf
[I] Device:0: ./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan has been successfully loaded.
[I] Creating batcher thread: 0 EnableBatcherThreadPerDevice: false
Starting warmup. Running for a minimum of 5 seconds.
Finished warmup. Ran for 5.02025s.
================================================
MLPerf Results Summary
================================================
SUT name : LWIS_Server
Scenario : Single Stream
Mode : Performance
90th percentile latency (ns) : 2110080
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
================================================
Additional Stats
================================================
QPS w/ loadgen overhead : 473.68
QPS w/o loadgen overhead : 477.80
Min latency (ns) : 2025051
Max latency (ns) : 5350963
Mean latency (ns) : 2092922
50.00 percentile latency (ns) : 2087806
90.00 percentile latency (ns) : 2110080
95.00 percentile latency (ns) : 2124064
97.00 percentile latency (ns) : 2134913
99.00 percentile latency (ns) : 2167362
99.90 percentile latency (ns) : 2458191
================================================
Test Parameters Used
================================================
samples_per_query : 1
target_qps : 616.903
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 60000
max_duration (ms): 0
min_query_count : 1024
max_query_count : 0
qsl_rng_seed : 3133965575612453542
sample_index_rng_seed : 665484352860916858
schedule_rng_seed : 3622009729038561421
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
print_timestamps : false
performance_issue_unique : false
performance_issue_same : false
performance_issue_same_index : 0
performance_sample_count : 256
No warnings encountered during test.
No errors encountered during test.
Device Device:0 processed:
28423 batches of size 1
Memcpy Calls: 0
PerSampleCudaMemcpy Calls: 0
BatchedCudaMemcpy Calls: 28423
&&&& PASSED Default_Harness # ./build/bin/harness_default
[2019-12-16 17:33:40,328 main.py:142 INFO] Result: 90th percentile latency (ns) : 2110080 and Result is : VALID
======================= Perf harness results: =======================
Xavier-SingleStream:
ssd-small: 90th percentile latency (ns) : 2110080 and Result is : VALID
======================= Accuracy results: =======================
Xavier-SingleStream:
ssd-small: No accuracy results in PerformanceOnly mode.
MODE_30W_2CORE
---------------------
{'action': 'run_harness', 'benchmarks': 'ssd-small', 'configs': '', 'scenarios': 'SingleStream', 'no_gpu': False, 'gpu_only': False}
---------------------
[2019-12-17 15:47:02,024 main.py:294 INFO] Using config files: measurements/Xavier/ssd-small/SingleStream/config.json
[2019-12-17 15:47:02,024 __init__.py:142 INFO] Parsing config file measurements/Xavier/ssd-small/SingleStream/config.json ...
-------------------------
[{'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'scenario': 'SingleStream', 'ssd-small': {'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False}, 'system_id': 'Xavier'}]
-------------------------
[2019-12-17 15:47:02,025 main.py:301 INFO] Processing config "Xavier_ssd-small_SingleStream"
[2019-12-17 15:47:02,025 main.py:111 INFO] Running harness for ssd-small benchmark in SingleStream scenario...
{'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False, 'system_id': 'Xavier', 'scenario': 'SingleStream', 'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'test_mode': 'PerformanceOnly', 'log_dir': '/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2019.12.17-15.47.01'}
[2019-12-17 15:47:02,035 __init__.py:42 INFO] Running command: ./build/bin/harness_default --plugins="build/plugins/NMSOptPlugin/libnmsoptplugin.so" --logfile_outdir="/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2019.12.17-15.47.01/Xavier/ssd-small/SingleStream" --logfile_prefix="mlperf_log_" --test_mode="PerformanceOnly" --use_graphs=false --gpu_batch_size=1 --map_path="data_maps/coco/val_map.txt" --tensor_path="${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4" --gpu_engines="./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan" --performance_sample_count=256 --max_dlas=0 --single_stream_expected_latency_ns=1621000 --mlperf_conf_path="measurements/Xavier/ssd-small/SingleStream/mlperf.conf" --user_conf_path="measurements/Xavier/ssd-small/SingleStream/user.conf" --scenario SingleStream --model ssd-small --response_postprocess coco
&&&& RUNNING Default_Harness # ./build/bin/harness_default
[I] mlperf.conf path: measurements/Xavier/ssd-small/SingleStream/mlperf.conf
[I] user.conf path: measurements/Xavier/ssd-small/SingleStream/user.conf
[I] Device:0: ./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan has been successfully loaded.
[I] Creating batcher thread: 0 EnableBatcherThreadPerDevice: false
Starting warmup. Running for a minimum of 5 seconds.
Finished warmup. Ran for 5.01878s.
================================================
MLPerf Results Summary
================================================
SUT name : LWIS_Server
Scenario : Single Stream
Mode : Performance
90th percentile latency (ns) : 2122587
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
================================================
Additional Stats
================================================
QPS w/ loadgen overhead : 473.93
QPS w/o loadgen overhead : 477.82
Min latency (ns) : 2013110
Max latency (ns) : 10534273
Mean latency (ns) : 2092846
50.00 percentile latency (ns) : 2087033
90.00 percentile latency (ns) : 2122587
95.00 percentile latency (ns) : 2134555
97.00 percentile latency (ns) : 2143419
99.00 percentile latency (ns) : 2164700
99.90 percentile latency (ns) : 2390246
================================================
Test Parameters Used
================================================
samples_per_query : 1
target_qps : 616.903
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 60000
max_duration (ms): 0
min_query_count : 1024
max_query_count : 0
qsl_rng_seed : 3133965575612453542
sample_index_rng_seed : 665484352860916858
schedule_rng_seed : 3622009729038561421
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
print_timestamps : false
performance_issue_unique : false
performance_issue_same : false
performance_issue_same_index : 0
performance_sample_count : 256
No warnings encountered during test.
No errors encountered during test.
Device Device:0 processed:
28437 batches of size 1
Memcpy Calls: 0
PerSampleCudaMemcpy Calls: 0
BatchedCudaMemcpy Calls: 28437
&&&& PASSED Default_Harness # ./build/bin/harness_default
[2019-12-17 15:48:10,974 main.py:142 INFO] Result: 90th percentile latency (ns) : 2122587 and Result is : VALID
======================= Perf harness results: =======================
Xavier-SingleStream:
ssd-small: 90th percentile latency (ns) : 2122587 and Result is : VALID
======================= Accuracy results: =======================
Xavier-SingleStream:
ssd-small: No accuracy results in PerformanceOnly mode.
MAXN
[2019-12-16 17:34:03,008 main.py:291 INFO] Using config files: measurements/Xavier/ssd-small/SingleStream/config.json
[2019-12-16 17:34:03,009 __init__.py:142 INFO] Parsing config file measurements/Xavier/ssd-small/SingleStream/config.json ...
[2019-12-16 17:34:03,009 main.py:295 INFO] Processing config "Xavier_ssd-small_SingleStream"
[2019-12-16 17:34:03,010 main.py:111 INFO] Running harness for ssd-small benchmark in SingleStream scenario...
{'gpu_batch_size': 1, 'gpu_single_stream_expected_latency_ns': 1621000, 'input_dtype': 'int8', 'input_format': 'chw4', 'map_path': 'data_maps/coco/val_map.txt', 'precision': 'int8', 'tensor_path': '${PREPROCE
SSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4', 'use_graphs': False, 'system_id': 'Xavier', 'scenario': 'SingleStream', 'benchmark': 'ssd-small', 'config_name': 'Xavier_ssd-small_SingleStream', 'test_mod
e': 'PerformanceOnly', 'log_dir': '/home/nvidia/data/inference_results_v0.5/closed/NVIDIA/build/logs/2019.12.16-17.34.02'}
[2019-12-16 17:34:03,017 __init__.py:42 INFO] Running command: ./build/bin/harness_default --plugins="build/plugins/NMSOptPlugin/libnmsoptplugin.so" --logfile_outdir="/home/nvidia/data/inference_results_v0.5/
closed/NVIDIA/build/logs/2019.12.16-17.34.02/Xavier/ssd-small/SingleStream" --logfile_prefix="mlperf_log_" --test_mode="PerformanceOnly" --use_graphs=false --gpu_batch_size=1 --map_path="data_maps/coco/val_ma
p.txt" --tensor_path="${PREPROCESSED_DATA_DIR}/coco/val2017/SSDMobileNet/int8_chw4" --gpu_engines="./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan" --performance_sample_c
ount=256 --max_dlas=0 --single_stream_expected_latency_ns=1621000 --mlperf_conf_path="measurements/Xavier/ssd-small/SingleStream/mlperf.conf" --user_conf_path="measurements/Xavier/ssd-small/SingleStream/user.
conf" --scenario SingleStream --model ssd-small --response_postprocess coco
&&&& RUNNING Default_Harness # ./build/bin/harness_default
[I] mlperf.conf path: measurements/Xavier/ssd-small/SingleStream/mlperf.conf
[I] user.conf path: measurements/Xavier/ssd-small/SingleStream/user.conf
[I] Device:0: ./build/engines/Xavier/ssd-small/SingleStream/ssd-small-SingleStream-gpu-b1-int8.plan has been successfully loaded.
[I] Creating batcher thread: 0 EnableBatcherThreadPerDevice: false
Starting warmup. Running for a minimum of 5 seconds.
Finished warmup. Ran for 5.0146s.
================================================
MLPerf Results Summary
================================================
SUT name : LWIS_Server
Scenario : Single Stream
Mode : Performance
90th percentile latency (ns) : 1500900
Result is : VALID
Min duration satisfied : Yes
Min queries satisfied : Yes
================================================
Additional Stats
================================================
QPS w/ loadgen overhead : 662.56
QPS w/o loadgen overhead : 669.99
Min latency (ns) : 1394879
Max latency (ns) : 4722358
Mean latency (ns) : 1492557
50.00 percentile latency (ns) : 1480643
90.00 percentile latency (ns) : 1500900
95.00 percentile latency (ns) : 1517989
97.00 percentile latency (ns) : 1722830
99.00 percentile latency (ns) : 1809554
99.90 percentile latency (ns) : 1912375
================================================
Test Parameters Used
================================================
samples_per_query : 1
target_qps : 616.903
target_latency (ns): 0
max_async_queries : 1
min_duration (ms): 60000
max_duration (ms): 0
min_query_count : 1024
max_query_count : 0
qsl_rng_seed : 3133965575612453542
sample_index_rng_seed : 665484352860916858
schedule_rng_seed : 3622009729038561421
accuracy_log_rng_seed : 0
accuracy_log_probability : 0
print_timestamps : false
performance_issue_unique : false
performance_issue_same : false
performance_issue_same_index : 0
performance_sample_count : 256
No warnings encountered during test.
No errors encountered during test.
Device Device:0 processed:
39755 batches of size 1
Memcpy Calls: 0
PerSampleCudaMemcpy Calls: 0
BatchedCudaMemcpy Calls: 39755
&&&& PASSED Default_Harness # ./build/bin/harness_default
[2019-12-16 17:35:11,520 main.py:142 INFO] Result: 90th percentile latency (ns) : 1500900 and Result is : VALID
======================= Perf harness results: =======================
Xavier-SingleStream:
ssd-small: 90th percentile latency (ns) : 1500900 and Result is : VALID
======================= Accuracy results: =======================
Xavier-SingleStream:
ssd-small: No accuracy results in PerformanceOnly mode.