This page describes planned and unimplemented features.

We have two versions of zpaq (7.05 and 7.07) and want to know if their performance is different.

Steps

You can use the measure command to measure the performance of one version:

pilot measure zpaq7.05 add /tmp/tmp_archive.zpaq /tmp/sample_input_files

But that is actually not needed because we can just use the compare command to compare the two versions directly. This is better than running two commands separately because pilot will try to alternate the execution of the two commands to minimize temporal correlation between test runs. We use the following command:

pilot compare "zpaq7.05 add /tmp/tmp_archive.zpaq /tmp/sample_input_files" "zpaq7.06 add /tmp/tmp_archive.zpaq /tmp/sample_input_files" --clean-up "rm /tmp/tmp_archive.zpaq"

The command after --clean-up will be executed between runs to do clean up jobs.

Sample output

Pilot version xxx (Mar. 19, 2016)
Tasks: compare
Settings: min_sample_size 200, desired_p_value 0.05
[Command A][  1] duration 12.3271s
[Command B][  1] duration 11.2911s
[Command A][  1] duration 12.2181s
[Command B][  1] duration 13.8110s
Running command A: zpaq7.05 add /tmp/tmp_archive.zpaq /tmp/sample_input_files

What Pilot Does For You

1. Bottleneck detection: make sure CPU is the bottleneck, not the disk