Outreachy: Performance Testing Enhancements: Project Perforama - adoptium/aqa-tests GitHub Wiki

Project description

We currently run many different open-source performance benchmarks at the project.  There are many different ways we can improve the automation for running these benchmarks, as well as improving the data collection and visualization of the results.  This "Perforama" project proposal is for enhancing our ability to run performance tests that require multiple machines to run.  In some cases, certain benchmarks require a machine for the workload plus 1 or more support machines responsible for running a database or other supporting software required by the benchmark.  Our automation tools need to be extended to be able to support this multi-machine configuration.  This would be the main aspect of the project work. 

Following that, we currently have developed ways of parsing various benchmark results and use a 3rd party charting library to visualize the results.  There are several enhancements that we would like to add to these existing interactive charts to make it easier to determine root cause of performance regressions.  This would be the second aspect of the project proposal, if time allows. The codebase for this portion of the project is in the aqa-test-tools repository.

Some learning materials that will be useful

Adoptium Performance Testing README:

• https://github.com/adoptium/aqa-tests/tree/master/perf#adoptium-performance-testing

Information about some benchmark suites we are currently running:

• Dacapo benchmark information: http://dacapobench.sourceforge.net/
• Renaissance benchmark information: https://github.com/renaissance-benchmarks/renaissance#renaissance-benchmark-suite

Information about adding parsers for new types of benchmarks in Test Results Summary Service (TRSS):

• Code for parsing and displaying benchmark results in TRSS / https://github.com/adoptium/aqa-test-tools/blob/master/TestResultSummaryService/parsers/BenchmarkMetric.js

Project Perforama Q&A session

• Q&A recording link

Participants

Intern: Azah Norbline

Mentors: Stewart Addison, Awsaf Arefin Sakif, Lan Xia, Joey Li and Shelley Lambert