Adding and Modifying ADCIRC Patch Sets - StormSurgeLive/asgs GitHub Wiki

Introduction

ASGS has developed a necessarily sophisticated capability for managing the building and patching of ADCIRC. This makes is an attractive system to use just for managing different versions of ADCIRC for a variety of activities; including: research, model development, operational forecasting, or really anything that might benefit from the ability to quickly switch among versions of ADCIRC - some of which might intended to only be customized locally.

$SCRIPTDIR/patches/ADCIRC

This directory is simply laid out, but contains all the necessary information needed to drive the build adcirc command; this is the command that when run presents the menu listing the different versions of ADCIRC to build.

Inspection of the $SCRIPTDIR/patches/ADCIRC reveals that each directory in it corresponds to an entry in the build adcirc menu (except for the custom) option that is always last.

Each directory / menu item represents a patch set. And each patch set directory contains minimally 2 files:

  • about.txt - description used by the right side of the build adcirc menu * info.sh - defines important variables required by the build adcirc command

These 2 files are required. The rest of the file in this directory are optional, but where the power to patch ADCIRC is brought to bear. The remainder of this document will focus on describing:

  • the variables needed in info.sh * examples of creating, maintaining, and testing ADCIRC patch sets

about.txt

This file is literally just a text string describing the patch set for the build adcirc menu. Please see the many examples in $SCRIPTDIR/patches/ADCIRC.

info.sh

There are plenty of examples of info.sh in $SCRIPTDIR/patches/ADCIRC. Here is an example for the basic v53release patch set:

export ADCIRC_SRC_TYPE=git          # tells "build adcirc" to look at the upstream git repo
export ADCIRC_GIT_BRANCH=v53release # defines the branch, tag, or SHA
export SWANDIR=swan                 # points at where the SWAN source is

By comparison, the one vor v55.01:

export ADCIRC_SRC_TYPE=git
export ADCIRC_GIT_BRANCH=v55.01     # again, the branch, tag, or SHA of the git repo
export SWANDIR=thirdparty/swan      # where SWAN's source is (for adcswan, padcswan)

SWANDIR is needed because the ADCIRC source was reorganized between the v54 release and the v55 release.

Patch Set Application and Methodology

The whole point of a patch set is to take ADCIRC code as it exists and patch it into a state that compiles on the targeted machine environment and operates with the desired functionality. Given a set of patches in a patch set, each one is applied using the patch utility in an order based on the numerical sort of the file names. Patches are applied at the top level of the ADCIRC source directory, e.g., where the work/ directory exists.

If the source code is in a git repository and it has been clone'd, then git is used to create a checkpoint in the code right before changes are made via patch. Then for patch, a git commit is made. This is extremely useful for maintaining and editing patch sets. Git supports patch management via git am see and git format-patch see.

The psuedo code for the application of patches in build adcirc is described below:

  cd ADCIRC_SRC_DIR
  (if git repo) git checkout -b ADCIRC_PROFILE_NAME          # creates a code checkpoint
  foreach PATCH_FILE in SORTED_LIST_OF_PATCHES:
    patch -p 1 < PATCH_FILE 
    (if git rep) git commit -a -m "ASGS applied a patch ..." # save each successive patch application as a commit
  end

This implies a couple of things; namely:

  1. patch files should be named with a leading number (e.g., 01-some-patch-name.diff
  2. patches are generated while in ADCIRC_SRC_DIR, then saved in patches/ADCIRC/patchset-dir
  3. we are operating entirely under the ASGS Shell Environment (asgsh)
  4. git can be very helpful for creating, maintaining, and troubleshooting patch sets

Creating a New ADCIRC Patch Set

Creating a new patch set is pretty straightforward when using git. Assuming a locally cloned ADCIRC source directory from the official upstream repo, the steps are basically:

Starting in `$SCRIPTDIR/patches/ADCIRC

  1. pick a base branch to start - it should be suffucient to pick an existing patch set; (e.g., v53release or v55.02)
  cp -r ./v53release ./my-new-patchset
  1. update ./about.txt with an accurate short description
  2. inspect ./info.sh, but just to see what's in there; there are more options available, but they're for more advanced patch set support that is out of scope here
  3. run build adcirc
  4. select the new patch set (it will be listed)
  5. follow the wizard but do not build it (last step); remember what you named it
  6. it's okay if you build it
  7. cd to the directory in ASGS where the source was downloaded and prepared
  cd $SCRIPTDIR/opt/models/adcircs/YOUR-VERSION
  1. look at contents of ./asgs-build.sh - it can be used to rebuild this source directory
  2. run git log and see the commits from the patches already supported in the starting patch set; run also git branch to see that you are in a new branch named after the ADCIRC profile name you specified during build adcirc's wizard
  3. update remote - since this code was probably git clone'd from the upstream ADCIRC repo, be sure to change the URL that is associated with the origin remote
  git remote set-url origin git@.... # specify the new "origin" here
  git pull fetch --all               # for good measure
  1. push the local branch to origin to initiate the development branch you're going to be working from:
  git push origin YOUR-VERSION
  1. now you may proceed to hack on ADCIRC, using ./asgs-build.sh to do test builds along the way
  2. when done, a new patch with changes since the last applied patch may be generated and stored for testing using the following command:
  git diff > $SCRIPTDIR/patches/ADCIRC/patchset-dir/NN-issue-XYZ.patch

Note: this might work exactly for your situtation; please ask for help when making this taking this step because it'd out of scope to cover patch generation via git here.

Maintaining or Updating an ADCIRC Patch Set

Usually, it is preferred to have 1 patch per logical change. Let's assume there is only one patch, and it's for work/cmplrflags.mk, which is currently the typical case. However, it is no problem to have more than 1 patch as long as they are ordered properly and are logically divided by file or change unit.

The goal is to get a local version of the ADCIRC repo that has all the current patches applied (as commits) and in your own branch.

As a result of the following methodology, there will be an additional patch created on top of the current patch set.

  1. while in asgsh, run build adcirc
  2. select the patch set to be updated
  3. run through the wizard, through the current patch application process
  4. select NO when prompted to build ADCIRC
  5. cd into the directory with the source, most likely $SCRIPTDIR/opt/models/adcircs/adcirc-cg-patchset-dir
  6. change the "origin" to the one you want, (see step 12 above)
  7. make the necessary modifications, testing, etc - be sure to treat this as a real development branch on your own remote; in the meantime do not worry about the logistics of generating a patch. Focus on implementing your changes as you would normally.
  8. for testing builds, take advantage of the ./asgs-build.sh script that is in this directory
  9. when done and ready to make the patch; as for help, git has many options to do this.

Maintaining Local or Private ADCIRC Patch Sets

Many sites prefer to maintain their own patches to ADCIRC for one reason or another.

Currently the best way to have local patch sets show up in the build adcirc menu is simply to copy the patch set directories into $SCRIPTDIR/patches/ADCIRC. This is unelegant, but effective for immediate use.

For longer term use of local patch sets, please see the page on managing local resources supported by ASGS.

Appendix I - Supported ADCIRC Code Sources

ASGS currently supports ADCIRC code provided from the following sources,

  • The official upstream repository (access required)
  • .zip file support (given a web URL)
  • custom source code that is already available locally and assumed working (no patch support provided)

Appendix II - Troubleshooting an ADCIRC Patch Set

The steps here are the same as updating or adding a patch, described in the section above.

Here, however the activity will likely include trying to get ADCIRC to build. Use the asgs-build.sh file for building, this contains the build commands that build adcirc uses ultimately

The best position to troubleshooting a broken patch starts with the ones outlined above for create a new patchset or updating an existing patch set.

Once you set up your development environment in this way, it is best to focus on just making the code compile and ensure it is correct. Once this is the case, push the dev branch up to your git remote see above of choice. Once it is up there and known to be working, ask for help to produce the patch set for testing within the build adcirc wizard.