Performance Data Parsing Scripts - crt26/PQC-LEO GitHub Wiki

The project includes several Python scripts that handle automatic parsing of benchmarking results:

• parse_results.py
• performance_data_parse.py
• tls_performance_data_parse.py
• energy_data_parse.py
• results_averager.py

These scripts support automated invocation (triggered by the automated test scripts) and manual execution via terminal input or command-line flags. Parsing is currently supported only on Linux systems. Windows environments are not supported due to the inability to create the environment needed to parse the raw performance results.

By default, parsing is triggered automatically after a supported test suite completes. Computational performance results are parsed on the testing machine, TLS performance results on the client machine, and energy usage results on the collection machine. The applicable automation scripts directly pass the necessary parameters (Machine-ID, number of runs, and test type) to the parsing system. TLS handshake transmission cost testing is excluded because its script writes structured CSV files directly. When TLS handshake performance results are stored during energy usage testing, the TLS controller also instructs the parser to skip TLS speed results because those tests are not performed in this mode.

While several scripts are utilised for the result parsing process, only the parse_results.py is intended to be run directly. The main parsing script calls the remaining scripts depending on which parameters the user supplies to the script when prompted. The main parsing script is stored in the scripts/parsing_scripts directory, whilst internal scripts are stored in the scripts/parsing_scripts/internal_scripts directory.

For full documentation on how the parsing system works, including usage instructions and a breakdown of the performance metrics collected, please refer to the following documentation:

• Parsing Performance Results Usage Guide
• Performance Metrics Guide

This script acts as the main controller for the result-parsing processes. It supports two modes of operation:

• Interactive Mode: Prompts the user to select a result type (computational performance, TLS performance, or energy consumption) and to enter parsing parameters such as Machine-ID and number of test runs. If TLS performance parsing is selected, the script asks whether TLS speed results should be skipped. If energy parsing is selected, the script also prompts for the energy test type.

• Command-Line Mode: Accepts the same parameters via flags. The automated test scripts use this mode and can also be called manually for scripting purposes.

In both modes, the script identifies the relevant raw test results in the test_data/up_results directory and invokes the appropriate parsing routines to generate structured CSV output. The results are then saved to the test_data/results directory, organised by test type and Machine-ID.

Usage Examples:

Interactive Mode:

python3 parse_results.py

Command-Line Mode:

python3 parse_results.py --parse-mode=computational --machine-id=2 --total-runs=10

The table below outlines each of the accepted commands that are required for operation:

Note: The script parses one result type per run in both command-line and interactive modes. Run it again to parse a different result type.

This script contains functions for parsing raw computational benchmarking data, transforming unstructured speed and memory test data into clean, structured CSV files. It processes CPU performance results and peak memory usage metrics gathered from Liboqs for each algorithm and operation across multiple test runs and machines. This script is not to be called manually and is only invoked by the parse_results.py script.

This script processes TLS performance data collected from handshake and OpenSSL speed benchmarking using PQC, Hybrid-PQC, and classical algorithms. It parses the classical OpenSSL speed output into separate RSA, EC/Ed, ECDH, and XDH result files in addition to the PQC and Hybrid-PQC outputs, then generates per-run CSVs for analysis. This script is not to be called manually and is only invoked by the parse_results.py script.

This script processes the PQC computational and TLS energy usage testing results produced by the automated test scripts. Unlike other parsing scripts in PQC-LEO, the energy usage parsing script extracts all of the test metadata from the filename of the un-parsed result files, rather than a mix of filename and algorithm list text files, as seen in the other parsing scripts.

For TLS operations energy results, the parser classifies algorithms into pqc, hybrid_pqc, and classic groups. It writes the group CSVs to pqc, hybrid, and classic sub-directories beneath the machine results directory. The PQC groups contain KEM and signature CSVs, while the classical group contains signature and key-exchange CSVs, with a condensed per-run summary for each output. The TLS operations energy baseline CSV remains directly in the machine results directory.

The file naming format expected for each of the different types of energy usage testing results is as follows:

• Computational Energy Usage Result - comp_(alg-type)_(operation)_(alg-name)_(run-number).txt
• PQC/Hybrid-PQC TLS Handshake Energy Result - tls_handshake_(session-id-type)_(signing-alg)@(kem-alg)_(run-number).txt
• Classical TLS Handshake Energy Result - tls_handshake_(session-id-type)_(signing-alg)@(key-exchange-group)@(ciphersuite)_(run-number).txt
• TLS Operations Energy Usage Result - tls_operations_(alg-type)_(operation)_(alg-name)_(run-number).txt

This script provides the internal classes used to aggregate and condense parsed benchmarking results. It is used by performance_data_parse.py and tls_performance_data_parse.py to generate per-algorithm averages across multiple test runs, and is also used by energy_data_parse.py to create condensed per-run energy result sheets (min/max/avg/std for polled metrics and max-aggregated totals for cumulative metrics). For computational performance tests, it handles CPU speed and memory profiling metrics collected using Liboqs. For TLS performance tests, it calculates average handshake durations and cryptographic operation timings gathered from OpenSSL and OQS-Provider. For energy usage parsing, it groups each run by algorithm/operation context and computes structured summary metrics for easier analysis. This script is not to be called manually and only executes internally by the result parsing scripts.