This page contains instructions for performing Acceptance Testing for the DSWx-S1 Final delivery from the OPERA-ADT team. These instructions pertain to the latest version of the Final release, currently v5.1. These instructions assume the user has access to the JPL FN-Artifactory, and Docker is installed on their local machine. Optionally, access to the opera-dev-pge AWS machine allows for significantly faster downloads from Artifactory.

The image is currently hosted on JPL FN-Artifactory, which requires JPL VPN access and JPL credentials. You may also need to be added to the gov.nasa.jpl.opera.adt organization.

Once you have access, the container tarball delivery is available under general/gov/nasa/jpl/opera/adt/dswx_s1/r5.1/final/dockerimg_dswx_s1_final_patch_1.1.tar. Sample inputs and outputs are available under general/gov/nasa/jpl/opera/adt/dswx_s1/r5.1/final/dswx_s1_sample_product.zip.

Download both images to a location on your local machine. This location will be referred to throughout this instructions as <DSWX-S1_DIR> Note that the sample data is quite large, so the download from Artifactory can take some time.

The first step in running the DSWx-S1 image is to load it into Docker via the following command:

    docker load -i <DSWX-S1_DIR>/dockerimg_dswx_s1_final_patch_1.1.tar

Use the following command to verify the Docker image with the name opera/dswx-s1 and the tag final_patch_1.1 has been added to your Docker repository:

    docker image ls 

Once the dswx_s1_sample_product.zip file is downloaded to your local machine, unpack it to <DSWX-S1_DIR>:

    cd <DSWX-S1_DIR>; unzip dswx_s1_sample_product.zip

This will create a the following directories within <DSWX-S1_DIR>:

    expected_output/
    input_dir/
    output_dir/
    scratch_dir/
    dswx_s1.yaml

In order to execute the SAS, the input file directory, runconfig, scratch and output locations will be mounted into the container instance as Docker Volumes. To help streamline this process, we recommend making the following changes within the shared directory:

• Create a directory named runconfig, and copy the existing runconfig YAML file into it:

    `mkdir -p <DSWX-S1_DIR>/runconfig`
  
    `cp <DSWX-S1_DIR>/dswx_s1.yaml <DSWX-S1_DIR>/runconfig/dswx_s1.yaml`
  

• Modify the copied version of the dswx_s1.yaml runconfig such that all paths within are absolute with respect to the container's file system. For example, the path input_dir/RTC/T114-243003-IW1 should become /home/dswx_user/input_dir/RTC/T114-243003-IW1 and so on...

• Make sure the output and scratch directories have write permissions set

    `chmod ga+w output_dir/ scratch_dir/`
  

We're now ready to execute the DSWX-S1 Interface. Run the following the command to kick off execution with the test assets:

    cd <DSWX-S1_DIR>

    docker run --rm -u $(id -u):$(id -g) \
      -w /home/dswx_user \
      -v $(pwd)/runconfig:/home/dswx_user/runconfig:ro \
      -v $(pwd)/input_dir:/home/dswx_user/input_dir:ro \
      -v $(pwd)/output_dir:/home/dswx_user/output_dir \
      -v $(pwd)/scratch_dir:/home/dswx_user/scratch_dir \
      -i --tty opera/dswx-s1:final_patch_1.1 sh -ci "dswx_s1.py /home/dswx_user/runconfig/dswx_s1.yaml \
         --log /home/dswx_user/output_dir/dswx_s1_1.1.log"

Execution should complete in about 30 minutes. Once complete, you should see ~45 files in <DSWX-S1_DIR>/output_dir. Files can be categorized into sets of four GeoTIFF files sharing the same time stamps, plus a .png "browse" image:

      OPERA_L3_DSWx-S1_T44UNA_20231213T121200Z_20240719T190825Z_S1A_30_v1.0_B01_WTR.tif
      OPERA_L3_DSWx-S1_T44UNA_20231213T121200Z_20240719T190825Z_S1A_30_v1.0_B02_BWTR.tif
      OPERA_L3_DSWx-S1_T44UNA_20231213T121200Z_20240719T190825Z_S1A_30_v1.0_B03_CONF.tif
      OPERA_L3_DSWx-S1_T44UNA_20231213T121200Z_20240719T190825Z_S1A_30_v1.0_B04_DIAG.tif
      OPERA_L3_DSWx-S1_T44UNA_20231213T121200Z_20240719T190825Z_S1A_30_v1.0_BROWSE.png

With this version a log file (dswx_s1_1.1.log) is also written to the output_dir.

There are comparison scripts available within the OPERA PGE repository. From a cloned copy of the repo, copy the following files to <DSWX-S1_DIR>:

cp <REPO DIR>/opera-sds-pge/.ci/scripts/dswx_s1/diff_dswx_files.py <DSWX-S1_DIR>
cp <REPO DIR>/opera-sds-pge/.ci/scripts/dswx_s1/dswx_comparison.py <DSWX-S1_DIR>

Where <REPO DIR> is the location containing the cloned copy of the OPERA PGE repository.

You can now perform the test by executing:

$ python3 <DSWX-S1_DIR>/diff_dswx_files.py <DSWX-S1_DIR>/expected_output <DSWX-S1_DIR>/output_dir

Or if you want the results written to a text file called result.log:

$ python3 <DSWX-S1_DIR>/diff_dswx_files.py <DSWX-S1_DIR>/expected_output <DSWX-S1_DIR>/output_dir 2>&1 | tee dswx_comparsion_results.log

The QA output should look similar to the following, but for each pair of output products compared:

Comparing files:
    file 1: expected_output/OPERA_L3_DSWx-S1_T44UNA_20231213T121200Z_20240717T194004Z_S1A_30_v1.0_B01_WTR.tif
    file 2: output_dir/OPERA_L3_DSWx-S1_T44UNA_20231213T121200Z_20240719T190825Z_S1A_30_v1.0_B01_WTR.tif
[OK]Comparing number of bands
Comparing DSWx bands...
[OK]     Band 1 - Water classification (WTR)"
[OK]Comparing geotransform
[OK]Comparing projection
[OK]Comparing metadata
...

This acceptance test must be run on an Intel family instance, such as c6i.2xlarge. If an AMD family instance is utilized, the output products are not expected to match correctly with the set of expected products.