{{toc}}

There is currently no particular order to these tests, the numbers are provided for easy reference to a particular test or stage of testing.
Note that the metrics reported here are for comparing artdaq versions only. They are not indicative of artdaq performance on any given machine, and should not be used as either guarantees or limitations.
The tests are predicated on a version of artdaq_demo being available from SciSoft.

1. Make sure that artdaq builds without errors or warnings
2. Make sure that all artdaq packages pass their built-in test suites
3. Check that quick-mrb-start.sh functions properly — run without parameters
4. Perform transfer_driver tests (See [[Artdaq_Release_testing_documentation_and_results#transfer_driver tests|transfer_driver tests]] below):
   1. Large fragments (100 MB) x 10000, record rate for Shmem, TCP, MPI
   2. Small fragments (5 KB) x 1000000, record rate for Shmem, TCP, MPI (Originally 1K fragments)
5. Perform artdaqDriver tests:
   1. test1: 10,000 1 MB events, record time
   2. test2: 1,000,000 1 KB events, record time
   3. test3: 10,000 1 MB events without disk-writing, record time
   4. test4: 10,000 1 MB events with Binary disk-writing to /dev/null, record time (new for v2_03_00, run for v2_02_01)
6. Run quick-mrb-start.sh —run-demo
   1. Make sure the demo runs as expected
   2. Make sure that the output data file is created
      1. Run rawEventDump.fcl over the data file
      2. Run toyDump.fcl over the data file
   3. Store data file in Redmine as version reference
7. Run the [[Artdaq_Release_testing_documentation_and_results#DAQInterface example configurations|DAQInterface example configurations]] on a low-performance machine
   1. Make sure each example runs as expected for 200s
      1. Make sure the output data file is created
      2. Run verification FCL jobs on data file
   2. Make sure each example runs as expected for 3 runs of 60s each
      1. Make sure the output data file is created
      2. Run verification FCL jobs on data file
8. Run the [[Artdaq_Release_testing_documentation_and_results#DAQInterface example configurations|DAQInterface example configurations]] on a high-performance machine New as of v3_03_01
   1. Make sure each example runs as expected for 200s
      1. Make sure the output data file is created
      2. Run verification FCL jobs on data file
   2. Make sure each example runs as expected for 3 runs of 60s each
      1. Make sure the output data file is created
      2. Run verification FCL jobs on data file
9. Run the [[Artdaq_Release_testing_documentation_and_results#DAQInterface example configurations|DAQInterface example configurations]] on multiple machines New as of v3_03_01
   1. Make sure each example runs as expected for 200s
      1. Make sure the output data file is created
      2. Run verification FCL jobs on data file
   2. Make sure each example runs as expected for 3 runs of 60s each
      1. Make sure the output data file is created
      2. Run verification FCL jobs on data file
10. Test version reference data files from Redmine — note if version incompatibility exists
11. Test previous version of artdaq with current reference data files — note if data files are not backwards-compatible
    1. Run quick-mrb-start.sh --tag [previous version tag] in new directory
    2. See compatibility test [[Artdaq_Release_testing_documentation_and_results#Testing file compatibility|notes]]

ironwork retired, ironvirt7 metrics begin

ironwork retired, ironvirt7 metrics begin

Run via command line:
$ run_demo.sh --config <configuration> --comps "<components>" [--no_om]
or, more robustly:
$ ipcrm -a;. ./run_demo.sh --config <configuration> --comps "<components>" --no_om --runduration 20
For example:
$ ipcrm -a;. ./run_demo.sh --config request_based_dataflow_example --comps component{01..03} --no_om --runduration 20
Note: These examples will (should) run easily on single node; extending/modifying these examples to run on multiple nodes may
requiring node and/or network administration (to enable multicast, for example).

*: Run during quick-mrb-start —run-demo test

(See page v3_13_00) https://docs.google.com/spreadsheets/d/1SVk6x3UPBdlg2RQwi2GXtU5outvHHGRf_VctrguIGL4/edit?usp=sharing

(See page v3_12_07) https://docs.google.com/spreadsheets/d/1SVk6x3UPBdlg2RQwi2GXtU5outvHHGRf_VctrguIGL4/edit?usp=sharing

https://docs.google.com/spreadsheets/d/1SVk6x3UPBdlg2RQwi2GXtU5outvHHGRf_VctrguIGL4/edit?usp=sharing

1. Builds using Jenkins
2. Tests run on Jenkins
3. quick-mrb-start.sh downloads and installs artdaq_demo v3_12_02 with no options. (From develop only)
4. Results posted above for s112:e20 for performance comparison
5. Results posted above for s112:e20 for performance comparison
6. See below
   1. Example runs
   2. Output data file is created
      1. Event dump succeeds
      2. toyDump succeeds
7. See below for run numbers of verified runs
8. See below for run numbers of verified runs
9. See below for run numbers of verified runs

1. Builds using Jenkins
2. Tests run on Jenkins
3. quick-mrb-start.sh downloads and installs artdaq_demo v3_12_01 with no options. (From develop only)
4. Results posted above for s112:e20 for performance comparison
5. Results posted above for s112:e20 for performance comparison
6. See below
   1. Example runs
   2. Output data file is created
      1. Event dump succeeds
      2. toyDump succeeds
7. See below for run numbers of verified runs
8. See below for run numbers of verified runs
9. See below for run numbers of verified runs

1. Builds using Jenkins
2. Tests run on Jenkins
3. quick-mrb-start.sh downloads and installs artdaq_demo v3_11_02 with no options.
4. Results posted above for s110:e20 for performance comparison
5. Results posted above for s110:e20 for performance comparison
6. See below
   1. Example runs
   2. Output data file is created
      1. Event dump succeeds
      2. toyDump succeeds
   3. e20:s110:prof: https://cdcvs.fnal.gov/redmine/attachments/67591/artdaqdemo_r000007_sr01_20220513T142243_1\_dl1.root
7. See below for run numbers of verified runs
8. See below for run numbers of verified runs
9. See below for run numbers of verified runs
10. artdaq v3_11_02 correctly reads reference data files from v3_11_01
11. artdaq v3_11_01 can read files from v3_11_02

1. Builds using Jenkins
2. Tests run on Jenkins
3. quick-mrb-start.sh downloads and installs artdaq_demo v3_11_01 with no options.
4. Results posted above for s110:e20 for performance comparison
5. Results posted above for s110:e20 for performance comparison
6. See below
   1. Example runs
   2. Output data file is created
      1. Event dump succeeds
      2. toyDump succeeds
   3. e20:s110:prof: https://cdcvs.fnal.gov/redmine/attachments/66546/artdaqdemo_r000007_sr01_20220216T163749_1\_dl1.root
7. See below for run numbers of verified runs
8. See below for run numbers of verified runs
9. See below for run numbers of verified runs
10. artdaq v3_11_01 correctly reads reference data files from v3_11_00
11. artdaq v3_11_00 can read files from v3_11_01

1. Builds using Jenkins
2. Tests run on Jenkins
3. quick-mrb-start.sh downloads and installs artdaq_demo v3_11_00 with no options. (after some post-tag modifications)
4. Results posted above for s110:e20 for performance comparison
5. Results posted above for s110:e20 for performance comparison
6. See below
   1. Example runs
   2. Output data file is created
      1. Event dump succeeds
      2. toyDump succeeds
   3. e20:s110:prof: https://cdcvs.fnal.gov/redmine/attachments/65356/artdaqdemo_r000001_sr01_20210915T133042_1\_dl1.root
7. See below for run numbers of verified runs
8. See below for run numbers of verified runs
9. See below for run numbers of verified runs
10. artdaq v3_11_00 correctly reads reference data files from v3_10_03
11. artdaq v3_10_03 can read files from v3_11_00

. products/setup
. setupARTDAQDEMO
for test in 5K 100M;do
  for type in shmem tcp;do
    runTransferTest.sh srcs/artdaq/test/TransferPlugins/fcl/transfer_driver_${type}_${test}.fcl 2 >|transfer_driver_${type}_${test}.log 2>&1
  done
done
# Newline for copy/paste execution

. products/setup
. setupARTDAQDEMO
olddir=$PWD
cd /tmp
for test in 1 2 3 4;do
  time artdaqDriver -c $olddir/srcs/artdaq_demo/tools/fcl/driver_test${test}.fcl >driver_test${test}.log 2>&1
done
rm driver_test*.root
mv driver_tets*.log $olddir
cd $olddir
# Newline for copy/paste execution

Make sure that port 3001/udp is open so that the data requests get through
On SL7:

firewall-cmd --add-port 3001/udp

source setupARTDAQDEMO
cd daqdata
for file in *.root;do echo $file;rm out.bin;art -c toyDump.fcl $file >$file.toydump 2>&1 && hexdump -C out.bin >$file.hexdump;done

Examine output from toydump to determine that events contain expected Fragments.

Use asciiDump.fcl and toyDump.fcl over reference source file:
(in a directory with the version being tested set up, i.e. use ./quick-mrb-start.sh —tag )

art -c $ARTDAQDEMO_REPO/artdaq-demo/ArtModules/fcl/toyDump.fcl /path/to/reference/file.root
art -c $ARTDAQDEMO_REPO/artdaq-demo/ArtModules/fcl/asciiDump.fcl /path/to/ASCII/reference/file.root

Make sure that ports 35555/tcp, 35556/udp, 35557/udp, 36555/tcp, 36556/udp, and 36557/udp are open so that routing tokens get through