access_APS2_ANCIL_LSM - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki
HOW TO FIX APS2 CITY LAND-SEA_MASK AND PRODUCE ALL ANCILLARIES
Please note readers can skip to Guide to Fix Land-Sea-Mask and Produce All Ancillaries for technical details on producing ancillaries. Readers can go straight to Guide to Fix Land-Sea-Mask and Produce All Ancillaries -> View Land-Sea-Mask for just reviewing and checking the product of Land-Sea-Mask.
Suite
There are two suites in MO share repository,
- u-ae663: Martin developed this suite for producing ancillaries using CAP vn9.1.
- u-ai370: Chun-hsu has been experimenting u-ae663 for producing ancillaries for the reanalysis project. Chun-hsu has made some changes and Wenming commit his changes to make this suite.
Wiki, Ticket and Tool
- Martin has a ticket https://accessdev.nci.org.au/trac/ticket/258 which details the background of the issues and history of efforts to fix it.
- The guide for the manual edit tool at https://accessdev.nci.org.au/trac/wiki/access/AncillaryFileModification
Sample Output
All the sample output and relevant scripts etc. are archived ar raijin:/g/data/dp9/access_c2_ancil/.
- README: direct users to this wiki
- ancil: all the sample ancillaries
- SY: various sample ancillaries
- SY.cap77: qrclim.murk70 and qrclim.ozone_L70_O7070 produced by CAP vn7.7. We will not use these two ancillaries produced by CAP vn9.1.
- SY.cap91.lakes: ancillaries including qrparm.mask produced by CAP vn9.1 with lakes switched on
- SY.cap91.nolake: ancillaries including qrparm.mask produced by CAP vn9.1 with lakes switched off
- SY.cap91.autoedit: sample output for auto edit tools
- SY.cap91.final : output for review and operational; not finalised yet at moment
- coast_ga: GA coastline data
- script: tool used in fixing C2 Land-Sea-Mask
Relevant Docs
Relevant the docs in communications will be attached in this section.
- Gary Dietachmayer: APS2_ACCESS_C_LSM
- Chun-hsu Su: Land-Sea-Mask of Sydney and Tansmania
Guide to Fix Land-Sea-Mask and Produce All Ancillaries= Create Ancillary
- Check out the suite u-ai370 on accessdev:
rosie checkout u-ai370
- Go to the folder accessdev: $HOME/roses/u-ai370 and run the suite,
rose suite-run
; Note- The domain is specified in
rose-suite.conf
as the variableHORIZ
. In our case, we haveHORIZ='sydney'
. - The grid of each domain id defined in
etc/$DOMAIN.grid
, in our case,etc/sydney.grid
; the file is as
&GRID POINTS_LAMBDA_TARG=594,POINTS_PHI_TARG=742, LAMBDA_ORIGIN_TARG=147.0,PHI_ORIGIN_TARG=-38.0, INWSW=1,DELTA_LAMBDA_TARG=0.0135,DELTA_PHI_TARG=0.0135, PHI_POLE=90.0,LAMBDA_POLE=180.0,ROTATED=F, GLOBAL=F,IGRID_TARG=1 /
- The configuration for Land-Sea-Mask is
app/ancilMask/opt/rose-app-C2.conf
. Few points here
1. [namelist:mask] You need to set min_lake_frac=2 to remove all inland lakes; if min_lake_frac=0.5, the lakes are kept in Land-Sea_mask. You need to set loverride=.true. to be able to feed CAP the extra namelist to override certain points. 2. [env] You need point mask_changes_file to the namelist for overriding points. 3. [file:mask_changes.nl] You need to set source to be the namelist for overriding.
- Once the suite completes successfully, you will find all outputs on raijin:$HOME/cylc-run/u-ai370/share/data/sydney. Particularly, you should be able to find
qrparm.mask
atland_sea_mask/igbp
. - You should also find
$HOME/cylc-run/u-ai370/work/1/ancilMask/README
which tells you the total land points in this Land-Sea-Mask. - You may go to
/g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.lakes
to check the ancillaries with lakes switched on and/g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.nolake
for ancillaries with lake switched off.
- The domain is specified in
View Land-Sea-Mask
Normally, you may use xconv
to view the Land-Sea-Mask. However, this does not give a clear view on the coastlines. To be able to check out the coast closely, follow the steps,
module use ~access/modules
module load iris
python ~access/apps/pythonlib/umfile_utils/mask_edit.py -m a qrparm.mask
Make sure that the GA datasets cstauscd_l.mid and cstauscd_l.mif are in the same folder as qrparm.mask (you may create a soft link).
Once the Land-Sea-Mask is loaded as in the following figure
you can zoom in to certain locations for details as shown in the next figure,
Manual Eidt Land-Sea-Mask
Martin has developed a python app to rectify land points manually. The detailed instruction is at https://accessdev.nci.org.au/trac/wiki/access/AncillaryFileModification.
The instruction is simplified in here,
on raijin, go to the directory where qrparm.mask is located,
module use ~access/modules
module load iris
python ~access/apps/pythonlib/umfile_utils/mask_edit.py -m a -o sy_ed1.nl qrparm.mask
You can further zoom in as shown in this figure
Click the zoom button again, you are able to update the points, green for existing land points, dark green for updated land points and blue for updated sea points as shown,
The app will output on terminal when it is closed
Number of points changed 14
And the sy_ed1.nl
is as
&DATAC FIELD_NO=1, POINT_NO=219530, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=219531, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=221318, DATA_NEW=.FALSE. /
&DATAC FIELD_NO=1, POINT_NO=224275, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=224869, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=226651, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=226652, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=226653, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=227246, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=227247, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=227248, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=229624, DATA_NEW=.TRUE. /
&DATAC FIELD_NO=1, POINT_NO=229627, DATA_NEW=.FALSE. /
&DATAC FIELD_NO=1, POINT_NO=230218, DATA_NEW=.TRUE. /
Wenming has some sample namelist for reference,
Location: /g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.nolake/manual_edit_namelist
There are four namelists, sy_edit0.nl sy_edit1.nl sy_edit2.nl sy_edit3.nl.
Since the app may stop unexpectedly, it is better to do the manual editing few times by covering different areas. That is why we end up with 4 namelists.
We then produce
cat sy_edit0.nl sy_edit1.nl sy_edit2.nl sy_edit3.nl | sort > sy_duplication.nl
#contains duplicated points because we may do some adjacent areas twice
cat sy_edit0.nl sy_edit1.nl sy_edit2.nl sy_edit3.nl | sort | uniq > sy_noduplication.nl
#remove all duplicated points; simple solution from Shaun, many thanks! I was thinking to code a python
Actually, CAP will ignore the duplicated points so using sy_duplication.nl or sy_noduplication.nl will result in same Land-Sea-Mask.
Auto Edit Land-Sea-Mask
Martin has also developed a tool which will auto edit the Land-Sea-Mask. The method is summarised,
module load pythonlib/masktools
python ~access/apps/pythonlib/mask_tools/orog_to_mask.py --source=GA qrparm.mask
This produces a file newmask.nc. Then,
module load pythonlib/umfile_utils
python ~access/apps/pythonlib/umfile_utils/um_replace_field.py -v 30 -n newmask.nc -V height qrparm.mask
This will rectify most points according to GA coast but not perfect.
Please go to raijin:/g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.autoedit
. There are
qrparm.mask.orig #original Land-Sea-Mask
qrparm.mask.autoedit #Land-Sea-Mask auto edited
The second step is to create a namelist based on changes done on qrparm.mask.orig
to produce qrparm.mask.autoedit
. We can then feed this namelist to CAP for producing Land-Sea-Mask close to qrparm.mask.autoedit
.
Wenming has coded a simple python program to do this task.
module use ~access/modules
module load iris
python /g/data/dp9/access_c2_ancil/script/mask_auto_edit.py -i qrparm.mask.orig -t qrparm.mask.autoedit -o sy_autoedit.nl
The namelist sy_autoedit.nl contains all changes from qrparm.mask.orig to qrparm.mask.autoedit
In the same folder,
sy_autoedit.nl #namelsit for auto edit
qrparm.mask.autoedit_by_cap #Land-Sea-Mask produced by CAP vn9.1 using suite u-ai370 by deeding the namelist sy_autoedit.nl
Edit Land-Sea-Mask with Both Auto and Manual Edit
We have explained that how the manual and auto edit are done in previous sections. As mentioned earlier, auto edit will rectify the majority of the points but manual edit is still needed to rectify some points after the auto rectification. Please go to
raijin:/g/data/dp9/access_c2_ancil/ancil/SY/SY.cap91.final
qrparm.mask.1st # original Land-Sea-Mask
qrparm.mask.2nd # Land-Sea-Mask with auto edit
qrparm.mask.3rd # Land-Sea-Mask with auto edit and manual edit by setting all points to land if the point contains any land part
qrparm.mask.4th # Land-Sea-Mask with auto edit and manual edit by setting points to land if the point contains more land than water; also, coastlines are made smoother for NWP in some places