• Introduction
• FATES User
• CTSM/FATES Developer

The FATES code is an external to CTSM and an optional configuration. The version of FATES that CTSM points to is a version under the standard NGEET repository. FATES users and developers however, may have need to update the default version of FATES pointed to in CTSM tags and will need to update their FATES external pointed to in CTSM to a different tag or branch, as well as their own fork of FATES. Some of the core CTSM developers are also responsible to update the version of FATES pointed to by default CTSM as well as do the needed testing with the new version of FATES for CTSM. So this guide is divided into those two functions. Updating the FATES version for one's own use, and update FATES for CTSM tags.

NOTE REGARDING MANAGE_EXTERNALS: The philosophy of the manage_externals tool is to provide an easy way to checkout all of the externals needed that go along with a CTSM tag, and to checkout specific versions that a given tag was tested with. It's expected that your real version control work will be done using git. With that philosophy manage_externals tries to stay out of your way and not do too much. It does exactly what you ask it to do, and then stops, and that means pointing to a known frozen version. So it might not do what you want it to do, and in that case you should do what you are actually trying to do in git rather than with manage_externals. In general when you run manage_externals, you want to have "clean" directories in your clone, where any changes you have made have been committed. And you want the externals that are pointed to in Externals_CLM.cfg for manage_externals to be the repository and tag or branch to be in agreement with what the FATES git clone is pointing to. You don't want manage_externals to needlessly clobber over your work.

What we will show here treats FATES differently from other externals in CTSM, because in this case FATES users are very likely to mostly do all of their work within FATES. You could use manage_extnerals to point to the FATES branch you are working on, but in general this workflow is easier and more intuitive to most FATES users.

In general we are keeping the version of FATES that CTSM points to close to the latest on FATES main-development. But, sometimes you will need to point to a different version or to a branch on your own fork.

When a FATES user wants to point to a different version of FATES than CTSM is pointing to. The following steps are taken. The first list are the steps you do to update your CTSM clone to point to a different NGEET FATES version.

1. ./manage_externals/checkout_externals -S # Just to check that your externals are updated (Should show that externals listed are correct without a "s", "M" or "e" prefix in front of them, nor does it call the externals "dirty".) (If externals are NOT correct run manage_externals/checkout_externals to get the correct versions of everything)
2. cd src/fates
3. git checkout <fates_tag>

(Optionally you could edit the Externals_CLM.cfg file so that manage-externals will know about the update. We show the above since it's simpler and useful for most FATES developers doing work on just FATES.)

These next steps are what you do to create a new FATES branch from your fork, to checkin changes that you are working on. It does assume you did all of the previous steps. This includes the step of creating a new branch, if you already have a branch on your fork, you can skip that step. To create a branch to work on...

1. cd src/fates
2. git remote add $GITUSER [email protected]:$GITUSER/fates.git # Add a remote for your fork
3. Create branch from origin remote (NGEET) branch or tag: i.e. git checkout -b <newbranch> origin/master
4. Send the new branch to your fork i.e. git push -u $GITUSER <newbranch> # Note, assumes step 2 above
5. cd ../..
6. ./manage_externals/checkout_externals -S # Run status on manage-externals and it will show that fates is modified (i.e. M src/fates)

Then make changes and commit to your clone as normal.

To push commits on your clone to your fork.

1. cd src/fates
2. git push -u $GITUSER <newbranch>

1. cd src/fates
2. Make sure everything you have in your fates branch is checked in i.e. git status # make sure the branch is up to date and there are no modified files or files to be committed
3. cd ../..
4. git fetch --all
5. git checkout <ctsm-tag>
6. Check if externals were updated: i.e. manage_externals/checkout_externals -S You will only need to proceed if externals outside of FATES were updated If FATES was updated you'll need to check if your branch is compatible with the new version of FATES that CTSM is using. If not you'll need to either update your branch or go back to a previous version of CTSM.
7. ./manage_externals/checkout_externals
8. cd src/fates
9. git checkout -b <yourbranch> $GITUSER/<yourbranch> NOTE: In some cases you might have to delete your src/fates version (with \rm -rf src/fates) in order to make manage_externals happy. In that case do the above steps and then repeat the steps above to point to your branch.

When new FATES tags have gone into NGEET and we want to bring those changes into CTSM, the following steps need to happen. For most of this the same process for creating CTSM tags is done, mostly below are outlined the additional steps that need to happen. Note, also that in contrast to FATES Users manage_externals is used to point to the FATES version.

The fates test list is run on every FATES tag as it is merged to the main FATES branch in NGEET. This testing is done by checking out the new FATES branch in the context of the last CTSM tag where there was a FATES update, and comparing against baselines that used that same CTSM tag.

For example, if ctsm5.1.dev061 involved a FATES update to fates-sci.1.50.0_api.17.0.0, then the fates test suite would have been run with this tag, creating baselines fates-sci.1.50.0_api.17.0.0-ctsm5.1.dev061. Then, for subsequent FATES changes, each new FATES tag will be tested in the context of ctsm5.1.dev061. So the next few baselines might be fates-sci.1.51.0_api.17.0.0-ctsm5.1.dev061, fates-sci.1.52.0_api.17.0.0-ctsm5.1.dev061, fates-sci.1.52.0_api.17.0.0-ctsm5.1.dev061, etc. (noting that all of them list ctsm5.1.dev061).

1. Create a branch for CTSM that includes the FATES update
2. Update Externals_CLM.cfg to point to the latest FATES tag (see README_EXTERNALS.rst at the top level of CTSM for more details), then commit that change to the CTSM branch (NOTE: If the FATES tag is not made because it needs to be coordinated with the CTSM testing see the section below)
3. Run the aux_clm testing (https://github.com/ESCOMP/CTSM/wiki/System-Testing-Guide#pre-merge-system-testing ), comparing against the previous CTSM tag
4. Run the fates test list, comparing against the latest existing FATES baselines (these should have been created with the FATES tag you are pulling in, but in the context of an earlier CTSM tag). These should generate baselines with the FATES suite baseline naming convention (e.g., fates-sci.1.50.0_api.17.0.0-ctsm5.1.dev061)
5. The changes are presented as a Pull Request to CTSM and approved as a normal CTSM tag (see CTSM checklist)

In this case the FATES tag isn't complete, because it has to be coordinated with the CTSM PR and tag. Most of the above steps still apply, but there are some changes or additional steps that are added.

1. Create a branch in FATES that includes the updates
2. Create a branch in CTSM that includes any updates within the CTSM repository
3. Update Externals_CLM.cfg to point to the FATES branch (or latest hash on that branch), then commit that change to the CTSM branch
   • Using a hash ensures that you will use the same, fixed version for all clones; to do this, change the tag = ... line to hash = ....
   • Using a branch allows you to fetch the latest version of the branch each time you run manage_externals/checkout_externals; to do this, change the tag = ... line to branch = ...
   • In either case, change the repo_url to point to your fork
   • See README_EXTERNALS.rst at the top level of CTSM for more details
4. Create a PR in FATES for the update
5. Create a PR in CTSM for the update
6. Get approvals for the FATES PR and get it scheduled for next in line
7. Get the CTSM PR scheduled for next in line
8. Merge the latest version of FATES master into your FATES branch
9. Merge the latest version of CTSM master into your CTSM branch
10. Create the baselines for the fates test lists for the previous CTSM tag
    • To create the baselines, check out the baseline CTSM tag (i.e., the latest CTSM tag, which your CTSM branch should now be up-to-date with) and update the FATES external to the baseline FATES tag (i.e., the latest FATES tag, which your FATES branch should now be up-to-date with); create the baselines from there.
    • Note that this step is optional, but recommended. You can compare against the existing FATES baselines from the last run of the FATES test suite (i.e., with the latest FATES tag in the context of (generally) some earlier CTSM tag). There are two situations where this can make sense:
      • If you expect answer changes in all FATES tests anyway (so there's no point in making new baselines since you'll have baseline failures in any case)
      • If you are confident enough that there have only been bit-for-bit changes on the CTSM side since the last time FATES baselines were generated (so new baselines, i.e., in the context of the latest CTSM tag, would be bit-for-bit with already-existing baselines).
11. As in the above section (step 3.) run the standard testing for CTSM
12. As in the above section (step 4.) also run the fates test list, but now compare against the baselines you just created
13. Once the testing is complete and satisfactory, have the FATES admin create the new FATES tag
14. Update the Externals_CLM.cfg to point to the new tag, commit and push that change to your branch in the CTSM PR.
15. Run manage_externals/checkout_externals to get the new FATES tag
16. Make sure the version of FATES you actually ran with (in src/fates) and the new tag are identical (by doing a git diff of the new tag against the hash you used for testing)
17. Complete the CTSM tag as for other CTSM tags (see CTSM checklist)