This page contains instructions for performing Acceptance Testing for the DSWx-HLS beta delivery from the OPERA-ADT team. These instructions assume the user has access to the JPL FN-Artifactory, and has Docker installed on their local machine.

The image is currently hosted on JPL FN-Artifactory, which requires JPL VPN access and JPL credentials. You may also need to be added to the gov.nasa.jpl.opera.adt organization.

Once you have access, the container tarball delivery is available under general/gov/nasa/jpl/opera/adt/r2/beta/dockerimg_proteus_beta.tar. Sample inputs and outputs are also available under general/gov/nasa/jpl/opera/adt/r2/beta/delivery_2_beta.zip.

Download both images to a location on your local machine. This location will be referred to throughout this instructions as <DSWx_DIR>

The first step in running the DSWx-HLS image is to load it into Docker via the following command:

docker load -i <DSWx_DIR>/dockerimg_proteus_beta.tar

This should add the Docker image to your local repository with the name opera/proteus and the tag beta.

Once the delivery_2_beta.zip file is downloaded to your local machine, unpack it to <DSWx_DIR>:

unzip delivery_2_beta.zip

This will create a delivery_2_beta directory within <DSWx_DIR> containing two test datasets, l30_greenland and s30_louisiana. Within each dataset are the following files/directories:

• expected_output_dir/
• input_dir/
• output_dir/
• scratch_dir/
• dswx_hls.yaml
• README.txt

In order to execute the SAS, the input file directory, runconfig and an output location will be mounted into container instance as Docker Volumes. To help streamline this process, we recommend making the following changes to the delivery_2_beta directory:

1. Rename the delivery_2_beta directory to test_datasets

   mv <DSWx_DIR>/delivery_2_beta <DSWx_DIR>/test_datasets

2. Create a directory named runconfig within each dataset directory, and move the existing runconfig YAML file into them:

   mkdir -p <DSWx_DIR>/test_datasets/{l30_greenland|s30_louisiana}/runconfig

   mv <DSWx_DIR>/test_datasets/{l30_greenland|s30_louisiana}/dswx_hls.yaml <DSWx_DIR>/test_datasets/{l30_greenland|s30_louisiana}/runconfig

We're now ready to execute the DSWx-HLS Beta. Run the following the command to kick off execution for each dataset:

docker run --rm -u $UID:$(id -g) \
  -v <DSWx_DIR>/test_datasets/<DATASET>/runconfig:/home/conda/runconfig:ro \
  -v <DSWx_DIR>/test_datasets/<DATASET>/input_dir:/home/conda/input_dir:ro \
  -v <DSWx_DIR>/test_datasets/<DATASET>/output_dir:/home/conda/output_dir \
  -i --tty opera/proteus:beta \
  sh -ci "python3 proteus-0.1/bin/dswx_hls.py runconfig/dswx_hls.yaml --log output_dir/<DATASET>.log"

Where <DATASET> is one of l30_greenland or s30_louisiana.

You should see console output from the executing SAS container. Execution should only take roughly 10-20 minutes per dataset. Once execution is complete, you should see a number of dswx_hls_v0.1_B##_<BAND>.tif files and a <DATASET>.log file.

Now that we've successfully executed the SAS container and generated outputs, the last step is to perform a QA check against the expected outputs.

The first step is to create a script that will compute difference images between the output and expected TIF files using the dswx_compare.py utility included with the SAS container delivery. Within the <DSWx_DIR> directory, create a script named compare_results.sh with the following contents:

#!/bin/bash

WORKING_DIR=`pwd`
DATASET=$1

declare -a dswx_bands=( "dswx_hls_v0.1_B01_WTR.tif"
                        "dswx_hls_v0.1_B02_BWTR.tif"
                        "dswx_hls_v0.1_B03_CONF.tif"
                        "dswx_hls_v0.1_B04_DIAG.tif"
                        "dswx_hls_v0.1_B05_WTR-1.tif"
                        "dswx_hls_v0.1_B06_WTR-2.tif"
                        "dswx_hls_v0.1_B08_SHAD.tif"
                        "dswx_hls_v0.1_B09_CLOUD.tif"
                        "dswx_hls_v0.1_B10_DEM.tif" )

for dswx_band in "${dswx_bands[@]}"; do
    echo "Comparing results for $DATASET:$dswx_band"
    docker run --rm -u $UID:$(id -g) \
      -v $WORKING_DIR/$DATASET/output_dir:/home/conda/output_dir:ro \
      -v $WORKING_DIR/$DATASET/expected_output_dir:/home/conda/expected_output_dir:ro \
      -i --tty opera/proteus:beta \
      sh -ci "python3 proteus-0.1/bin/dswx_compare.py output_dir/$dswx_band expected_output_dir/$dswx_band"
done

Make sure the script has execute permissions:

chmod 775 <DSWx_DIR>/compare_results.sh

Now execute the QA script on each dataset from within <DSWx_DIR>:

./compare_results.sh l30_greenland
./compare_results.sh s30_louisiana

The QA console output should appear on screen, with comparisons made for each output tif band:

Comparing results for l30_greenland:dswx_hls_v0.1_B01_WTR.tif
[OK]   Comparing number of bands
Comparing DSWx bands...
[OK]        Band 1 - WTR: "Water classification (WTR)"
[OK]   Comparing geotransform
[OK]   Comparing metadata
...
Comparing results for s30_louisiana:dswx_hls_v0.1_B01_WTR.tif
[OK]   Comparing number of bands
Comparing DSWx bands...
[OK]        Band 1 - WTR: "Water classification (WTR)"
[OK]   Comparing geotransform
[OK]   Comparing metadata
...

The expected results for this Acceptance Test are for all bands, excepting the dswx_hls_v0.1_B10_DEM.tif file (see Known Issues), to return values of [OK] for all tests.

• For dswx_hls_v0.1_B10_DEM.tif, a value of [FAIL] is expected for the band comparison test. This is due to floating point precision differences and can be safely ignored.
• There is a bug currently in dswx_compare.py where all band types are described as "Band 1 - WTR: "Water classification (WTR)".