Comparing Benchmarks - prestodb/pbench GitHub Wiki

Use pbench cmp to compare the files of two directories and generate unified diffs for matching files.

Run ./pbench cmp --help to see the online help for pbench cmp.

Compare two query result directories

Usage:
  pbench cmp [flags] [directory 1] [directory 2]

Flags:
  -r, --file-id-regex string   regex to extract file id from file names in two directories to find matching files to compare (default ".*(query_\\d+)(?:_c0)?(?:_ordered)?\\.output")
  -h, --help                   help for cmp
  -o, --output-path string     diff output path (default "./diff")

-r, --file-id-regex string

When specifying a custom regex with -r, you must include one capture group that is written to match the files in the directories that you are comparing.

For more information, visit Using pbench cmp.

-h, --help

Display the help for pbench cmp.

-o, --output-path string

The output directory to write to, relative to the current directory that pbench cmp is run from.

If -o is not used, the default output directory is ./diff.

Using pbench cmp

Use pbench cmp to compare the files of two directories and generate unified diffs for matching files, based on a regex expression and a capture group.

  • The regex expression is used to filter and extract relevant parts of the filenames.

  • A capture group is a part of a regular expression that is enclosed in parentheses, and is used in pbench cmp to match files.

Default Regex and Capture Group

The default regex expression is .*(query_\d+)(?:_c0)?(?:_ordered)?\.output.

The default capture group is (query_\d+).

(?:_c0) and (?:_ordered) are non-capturing groups, and are not used to match files. The ? makes each group optional, so they match zero or one occurrence of _c0 or _ordered respectively.

Example

The process follows these steps:

  1. In directory1, if the file name matches the regex, the capture group is extracted and stored.
  2. In directory2, if the file name matches the regex, the capture group is extracted and is used to search for a match in directory1.
  3. If there is a match, the two files are compared with a diff.

For example, ds_power_query_01_c0.output matches the default regex expression.

query_01 is extracted, and is used to look in the other directory to match files who also have query_01 in the capture group and match the regex expression.