access_APS2_ANCIL_LSM - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki

HOW TO FIX APS2 CITY LAND-SEA_MASK AND PRODUCE ALL ANCILLARIES

Please note readers can skip to Guide to Fix Land-Sea-Mask and Produce All Ancillaries for technical details on producing ancillaries. Readers can go straight to Guide to Fix Land-Sea-Mask and Produce All Ancillaries -> View Land-Sea-Mask for just reviewing and checking the product of Land-Sea-Mask.

Suite

There are two suites in MO share repository,

u-ae663: Martin developed this suite for producing ancillaries using CAP vn9.1.
u-ai370: Chun-hsu has been experimenting u-ae663 for producing ancillaries for the reanalysis project. Chun-hsu has made some changes and Wenming commit his changes to make this suite.

Wiki, Ticket and Tool

Martin has a ticket https://accessdev.nci.org.au/trac/ticket/258 which details the background of the issues and history of efforts to fix it.
The guide for the manual edit tool at https://accessdev.nci.org.au/trac/wiki/access/AncillaryFileModification

Sample Output

All the sample output and relevant scripts etc. are archived ar raijin:/g/data/dp9/access_c2_ancil/.

README: direct users to this wiki
ancil: all the sample ancillaries
SY: various sample ancillaries
SY.cap77: qrclim.murk70 and qrclim.ozone_L70_O7070 produced by CAP vn7.7. We will not use these two ancillaries produced by CAP vn9.1.
SY.cap91.lakes: ancillaries including qrparm.mask produced by CAP vn9.1 with lakes switched on
SY.cap91.nolake: ancillaries including qrparm.mask produced by CAP vn9.1 with lakes switched off
SY.cap91.autoedit: sample output for auto edit tools
SY.cap91.final : output for review and operational; not finalised yet at moment
coast_ga: GA coastline data
script: tool used in fixing C2 Land-Sea-Mask

Relevant Docs

Relevant the docs in communications will be attached in this section.

Gary Dietachmayer: APS2_ACCESS_C_LSM
Chun-hsu Su: Land-Sea-Mask of Sydney and Tansmania

Guide to Fix Land-Sea-Mask and Produce All Ancillaries= Create Ancillary

Check out the suite u-ai370 on accessdev: rosie checkout u-ai370
Go to the folder accessdev: $HOME/roses/u-ai370 and run the suite, rose suite-run; Note
- The domain is specified in rose-suite.conf as the variable HORIZ. In our case, we have HORIZ='sydney'.
- The grid of each domain id defined in etc/$DOMAIN.grid, in our case, etc/sydney.grid; the file is as
```
  &GRID
  POINTS_LAMBDA_TARG=594,POINTS_PHI_TARG=742,
  LAMBDA_ORIGIN_TARG=147.0,PHI_ORIGIN_TARG=-38.0,
  INWSW=1,DELTA_LAMBDA_TARG=0.0135,DELTA_PHI_TARG=0.0135,
  PHI_POLE=90.0,LAMBDA_POLE=180.0,ROTATED=F,
  GLOBAL=F,IGRID_TARG=1
  /
```
- The configuration for Land-Sea-Mask is app/ancilMask/opt/rose-app-C2.conf. Few points here
```
  1. [namelist:mask]
     You need to set min_lake_frac=2 to remove all inland lakes; if min_lake_frac=0.5, the lakes are kept in Land-Sea_mask.
     You need to set loverride=.true. to be able to feed CAP the extra namelist to override certain points.
  2. [env]
     You need point mask_changes_file to the namelist for overriding points.
  3. [file:mask_changes.nl]
     You need to set source to be the namelist for overriding.
```
- Once the suite completes successfully, you will find all outputs on raijin:$HOME/cylc-run/u-ai370/share/data/sydney. Particularly, you should be able to find qrparm.mask at land_sea_mask/igbp.
- You should also find $HOME/cylc-run/u-ai370/work/1/ancilMask/README which tells you the total land points in this Land-Sea-Mask.
- You may go to /g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.lakes to check the ancillaries with lakes switched on and /g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.nolake for ancillaries with lake switched off.

View Land-Sea-Mask

Normally, you may use xconv to view the Land-Sea-Mask. However, this does not give a clear view on the coastlines. To be able to check out the coast closely, follow the steps,

  module use ~access/modules
  module load iris
  python ~access/apps/pythonlib/umfile_utils/mask_edit.py -m a qrparm.mask
  Make sure that the GA datasets cstauscd_l.mid and cstauscd_l.mif are in the same folder as qrparm.mask (you may create a soft link).

Once the Land-Sea-Mask is loaded as in the following figure

manual_edit1.PNG, 35%

you can zoom in to certain locations for details as shown in the next figure,

manual_edit2.PNG, 35%

Manual Eidt Land-Sea-Mask

Martin has developed a python app to rectify land points manually. The detailed instruction is at https://accessdev.nci.org.au/trac/wiki/access/AncillaryFileModification.

The instruction is simplified in here,

 on raijin, go to the directory where qrparm.mask is located,
 module use ~access/modules
 module load iris
 python ~access/apps/pythonlib/umfile_utils/mask_edit.py -m a -o sy_ed1.nl qrparm.mask

You can further zoom in as shown in this figure

manual_edit3.PNG, 35%

Click the zoom button again, you are able to update the points, green for existing land points, dark green for updated land points and blue for updated sea points as shown,

manual_edit4.PNG, 35%

The app will output on terminal when it is closed

Number of points changed 14

And the sy_ed1.nl is as

&DATAC FIELD_NO=1, POINT_NO=219530, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=219531, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=221318, DATA_NEW=.FALSE. /
&DATAC FIELD_NO=1, POINT_NO=224275, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=224869, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=226651, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=226652, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=226653, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=227246, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=227247, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=227248, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=229624, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=229627, DATA_NEW=.FALSE. /
&DATAC FIELD_NO=1, POINT_NO=230218, DATA_NEW=.TRUE. /

Wenming has some sample namelist for reference,

Location: /g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.nolake/manual_edit_namelist

There are four namelists, sy_edit0.nl  sy_edit1.nl  sy_edit2.nl  sy_edit3.nl.
Since the app may stop unexpectedly, it is better to do the manual editing few times by covering different areas. That is why we end up with 4 namelists.

We then produce

cat sy_edit0.nl  sy_edit1.nl  sy_edit2.nl  sy_edit3.nl | sort > sy_duplication.nl          
#contains duplicated points because we may do some adjacent areas twice

cat sy_edit0.nl  sy_edit1.nl  sy_edit2.nl  sy_edit3.nl | sort | uniq > sy_noduplication.nl 
#remove all duplicated points; simple solution from Shaun, many thanks! I was thinking to code a python

Actually, CAP will ignore the duplicated points so using sy_duplication.nl or sy_noduplication.nl will result in same Land-Sea-Mask.

Auto Edit Land-Sea-Mask

Martin has also developed a tool which will auto edit the Land-Sea-Mask. The method is summarised,

 module load pythonlib/masktools
 python ~access/apps/pythonlib/mask_tools/orog_to_mask.py --source=GA qrparm.mask
  
 This produces a file newmask.nc. Then,

 module load pythonlib/umfile_utils
 python ~access/apps/pythonlib/umfile_utils/um_replace_field.py -v 30 -n newmask.nc -V height qrparm.mask

 This will rectify most points according to GA coast but not perfect.

Please go to raijin:/g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.autoedit. There are

 qrparm.mask.orig     #original Land-Sea-Mask
 qrparm.mask.autoedit #Land-Sea-Mask auto edited

The second step is to create a namelist based on changes done on qrparm.mask.orig to produce qrparm.mask.autoedit. We can then feed this namelist to CAP for producing Land-Sea-Mask close to qrparm.mask.autoedit.

Wenming has coded a simple python program to do this task.

 module use ~access/modules
 module load iris
 python /g/data/dp9/access_c2_ancil/script/mask_auto_edit.py -i qrparm.mask.orig -t qrparm.mask.autoedit -o sy_autoedit.nl

 The namelist sy_autoedit.nl contains all changes from qrparm.mask.orig to qrparm.mask.autoedit

In the same folder,

 sy_autoedit.nl               #namelsit for auto edit
 qrparm.mask.autoedit_by_cap  #Land-Sea-Mask produced by CAP vn9.1 using suite u-ai370 by deeding the namelist sy_autoedit.nl

Edit Land-Sea-Mask with Both Auto and Manual Edit

We have explained that how the manual and auto edit are done in previous sections. As mentioned earlier, auto edit will rectify the majority of the points but manual edit is still needed to rectify some points after the auto rectification. Please go to

 raijin:/g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.final
 
 qrparm.mask.1st  # original Land-Sea-Mask
 qrparm.mask.2nd  # Land-Sea-Mask with auto edit
 qrparm.mask.3rd  # Land-Sea-Mask with auto edit and manual edit by setting all points to land if the point contains any land part
 qrparm.mask.4th  # Land-Sea-Mask with auto edit and manual edit by setting points to land if the point contains more land than water; also, coastlines are made smoother for NWP in some places