Introduction

This JMeter plugin calculates both the Apdex Scores and the Coefficient of Variation of samplers from a JMeter test results file. It lets you specifiy thresholds that will be used for a pass/fail test. It presents the results in a table so that this helps you in deciding how close or far your application is from your SLAs.

Apdex

Given a Tolerated threshold that you specify, the Application Performance Index Apdex returns a score between 0 and 1. It defines ratings from "Unacceptable" up to "Excellent" which will help you find out where your application stands out depending on its score:

Coefficient of Variation

The Coefficient of Variation (CoV) will give you the variability of your test results. A low CoV indicates a consistent and predictable application (good). The following table may serve as a guideline in your decision-making process whether the amount of variation is within a reasonable range or if it might be considered excessive:

Recommendation

The plugin assumes that the results to analyze are more or less "normal". However, usually bulk test results contain outliers in the right tail which may bias the analysis. So it is recommended to remove the outliers from the bulk results with the Outlier Detector plugin and and then run the Apdex and Cov quality analysis on the cleansed results.

Usage

Via the GUI

Adding the calculator to a Test Plan

Add the plugin to a Test Plan via the "Add > Non-Test Elements" menu and select then "Apdex & Coefficient of Variation".

Specify the input fields

• Apdex Target Threshold T (in seconds): default value is 4s. If your target threshold is in milliseconds, then convert it to seconds before entering T. For instance, if you want to check APIs against a threshold of 500ms, enter T as "0.5".
• Filename: JMeter results file you want to analyze.

Specify your pass/fail criteria

• Apdex Acceptable Quality Level: minimum score to pass (default is 0.85).
• Coefficient of Variation Acceptable Limit: maximum amount of variation to pass (default is 30%).

Any samplers where their Apdex or CoV scores do not meet these criteria shall be marked as "failed" in the results table. By changing the default values, you can be more or less forgiving.

Press the Calculate button

The plugin parses the data and shows the results in a grid, together with their Apdex and CoV ratings. When the number of samples is less than 100, the plugin will mark them as "Small Group" as a precaution.

Save the calculation results in a CSV file

By clicking on the "Save" button, the plugin will copy the results into a csv file with a suffix of "_ApdexCoVScores.csv" to the input filename which you can then import into your final test report.

Via Command line

In a script or in a Windows Powershell window, you can call the tool with the following command:

java -jar $env:JMETER_HOME\lib\cmdrunner-x.y.jar --tool ApdexCoV --input-file {filenameIn} --apdex-tgt-thold-secs {T} --apdex-aql {ApdexMin} --cov-alim-pct {CoVMax}

where:

• cmdrunner-x.y.jar: version of the cmdrunner you have in your jmeter/lib directory;
• --input-file: full path to your JMeter results file;
• --apdex-tgt-thold-secs: your Apdex Satisfied Threshold T in seconds;
• --apdex-aql: your minimum Apdex score to pass (any number between 0 and 1);
• --cov-alim-pct: your maximum variation to pass (e.g., 0.4 for 40%)

The tool will save the results of the computation in an HTML file with a suffix of "_ApdexCoVScores.html" to the input filename which can be imported into CI/CD tools. It will return the number of failed samplers in the exit code, allowing to fail the build step in the CI/CD tool if the build process of the tool uses a non zero exit code as a failed indicator.

See also

• Apdex in Apache JMeter Dashboard.
• Apdex Ratings