access_AG2_ERA_IC - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki

A procedure for creating ACCESS Global N512L70 AG2 ic from ERA-Interim grib fields

This procedure demonstrates how to obrain ERA-Interim data and then use this to create UM dump files suitable for running forecasts (or more correcltly hindcasts) from the ACCESS Global model AG2, with N512L70 resolution. For this demonstration we will assume we want to run 5 day forecasts from AG2 starting at 00 UTC for the 5 dates 2006-07-12:2006-07-16. These dates cover the extreme rain event following the landfall of typhoon Bilis (2006).

All this work is done on raijin and example outputs and copies of all the scripts used below have been saved to sam:/samcrc/gen/glr/2014/cyclone as well as on raijin in dir /g/data1/p66/glr548/tamip_sam/cyclone and /projects/access/AMEL/TransposeAMIPII/cyclone.

Copies of the resultant ACCESS Global N512L70 AG2 ic dump files created from ERA-Interim grib fields can be found in /g/data1/rr4/transpose-amip/n512l70nd

Get ERA-Interim required grib fields

The ERA-Interim fields we require are:

Invariant: Land-sea mask, Geopotential
Surface fields: Skin Temperature and Surface Pressure
3D fields: U, V, T and Q
surface forcing fields: sea surface temperature and sea ice fraction

Anyone can obtain such fields by registering on the ERA-Interim data portal http://apps.ecmwf.int/datasets/data/interim_full_daily .

The invariant fields are dateless; the Surface and 3D fields are downloaded for the dates 2006-07-12:16 at 00hr; and the surface forcing fields are downloaded for the dates 2006-07-12:26 at 00:06:12:18hr. The extra times in the latter fields are required in case we want to run with changing surface forcings. Download these to your system eg

cd /g/data1/p66/glr548/tamip_sam/cyclone
ls -l mars*
-rwxr----- 1 glr bmrc 139173600 Dec 19 11:08 mars-atls05-20141218235322-3007-15958_T_Q_jul06.grib
-rwxr----- 1 glr bmrc    462936 Dec 15 10:46 mars-web223-20141214234616-9916-12113_geop_mask.grib
-rwxr----- 1 glr bmrc  20125200 Dec 19 10:36 mars-web224-20141218233459-17191-25960_sst_sice_jul06.grib
-rwxr----- 1 glr bmrc   2314680 Dec 19 10:26 mars-web226-20141218232605-16125-26641_surfp_skinTjul06.grib
-rwxr----- 1 glr bmrc 139173600 Dec 19 10:52 mars-web229-20141218233853-17511-26710_U_V_jul06.grib

Convert these grib files into single date era initial condition grib files

Using cdo commands we can extract fields for each of the required dates and save in a single grib file for each date. First check we have the correct dates by running xconv on the grib files, in this case 00UTC for dates 2006-07-12 2006-07-13 2006-07-14 2006-07-15 2006-07-16, then create the following script:

cd /g/data1/p66/glr548/tamip_sam/cyclone
module load cdo nco
for date in 2006-07-12 2006-07-13 2006-07-14 2006-07-15 2006-07-16; do
  echo ${date}
  cdo seldate,${date}T00:00:00 \
    mars-web226-20141218232605-16125-26641_surfp_skinTjul06.grib \
    pT1D${date}T00:00:00.grib
  cdo seldate,${date}T00:00:00 \
    mars-atls05-20141218235322-3007-15958_T_Q_jul06.grib \
    TQ3D${date}T00:00:00.grib
  cdo seldate,${date}T00:00:00 \
    mars-web229-20141218233853-17511-26710_U_V_jul06.grib \
    UV3D${date}T00:00:00.grib
  cdo setdate,${date} \
    mars-web223-20141214234616-9916-12113_geop_mask.grib dum0
  cdo settime,00:00:00 dum0 INV1D${date}T00:00:00.grib
  cat INV1D${date}T00:00:00.grib \
      pT1D${date}T00:00:00.grib  \
      TQ3D${date}T00:00:00.grib  \
      UV3D${date}T00:00:00.grib  >  cyclone_${date}T00.grib
  \rm INV1D${date}T00:00:00.grib pT1D${date}T00:00:00.grib \
      TQ3D${date}T00:00:00.grib  UV3D${date}T00:00:00.grib dum0
done

This creates ERA-Interim ic grib files: cyclone_2006-07-12T00.grib cyclone_2006-07-13T00.grib cyclone_2006-07-14T00.grib cyclone_2006-07-15T00.grib cyclone_2006-07-16T00.grib

Create sst+sice ancillary files from era grib data

Convert the sst+sice grib file to separate netcdf files

Here we first convert the sst+sice grib file to netcdf using cdo commands; then split this into an sst field file and a sea-ice fraction file using nco commands - note that the sst and ice fields are labelled as var34 and var31, respectively, in the grib files.

cd /g/data1/p66/glr548/tamip_sam/cyclone
cdo -f nc copy mars-web224-20141218233459-17191-25960_sst_sice_jul06.grib cyclone_sst_sice.nc
ncks -v var31 cyclone_sst_sice.nc sice_var31.nc
ncks -v var34 cyclone_sst_sice.nc sst_var34.nc

Extrapolate SST field over land

We need to extrapolate the SST field over land as the mask from ERA-Interim will not be the same as out mask so we want SST values to cover any missing grid points. The tcl script we need, convEXTRAP.tcl, can be found on sam at /samcrc/gen/glr/2014/cyclone and raijin at /g/data1/p66/glr548/tamip_sam/cyclone

cd /g/data1/p66/glr548/tamip_sam/cyclone
convEXTRAP.tcl sst_var34.nc

This creates sst_var34x.nc where the sst values have been extrapolated over land points. Now we use the netcdf files sice_var31.nc and sst_var34x.nc to create the ancillary files we need.

Use Xancil to create ancillary files on the ERA-Interim grid

cd /g/data1/p66/glr548/tamip_sam/cyclone
xancil -jobfile sice0_cycloneOK.job
xancil -jobfile sst0_cycloneOK.job

Once Xancil opens its gui, you will have to edit the Atmospheric Ancillary Files 'Sea Ice' or 'Sea Surface Temperature' section, respectively, to point to your sice_var31.nc or sst_var34x.nc file, respectively. Then press the create button in each case and you will get the ERA-Interim grid ancillary files ice0 and sst0, respectively.

Create the N512L70 ancillaries from the ice0 and sst0 ERA-Interim ancillaries generated above

Here we use the Central Ancillary Programs (CAP) routines to generate the N512L70 ancillaries from the ice0 and sst0 ERA-Interim ancillaries generated above. The script we use is run.sh and it refers to an xml file, n512.xml, which describes the N512 configuration - both are in the /g/data1/p66/glr548/tamip_sam/cyclone directory.

cd /short/dp9/glr548/ancil_n512
./run.sh

This script uses n512.xml to create another script AncilScr_sstice and it submits this to the raijin queue.

However the memory is hardwired to 2GB and this is too small for an n512 grid, so you must qdel the submitted job.

Then edit the AncilScr_sstice and change mem=2GB to mem=6GB, then "qsub AncilScr_sstice".

This will then create the ancillary files qrclim.seaice qrclim.sst.

However - the dates will be rubbish - the CAP routines muck then up - so we need to now run a python script to copy the original dates from the ERA-iInterim ancillary files we create - ice0 and sst0 - to these new N512 ancillary files ie

cd /short/dp9/glr548/ancil_n512
module load python/2.7.3-matplotlib
module load pythonlib/umfile_utils/latest
module load pythonlib/cdat-lite/6.0rc2-fixed
python copy_dates.py ice0 qrclim.seaice
python copy_dates.py sst0 qrclim.sst
mv qrclim.seaice /g/data1/p66/glr548/tamip_sam/cyclone/qrclim.seaice_cyclone
mv qrclim.sst    /g/data1/p66/glr548/tamip_sam/cyclone/qrclim.sst_cyclone

and here we have relabelled and moved the N512L70 sst and sice ancillary files to /g/data1/p66/glr548/tamip_sam/cyclone.

Reconfig era grib ic file to um dump file format using era sst+sice anc and the accessdev umui job: vaceo

Make a copy of my accessdev job vaceo, then go through the following processes on it.

Check era reconfig job vaceo is pointing to correct ic and sst+sice

Initial Conditions: select reconfig file created above and make a soft link to it accessdev: umuix/vaceo input grib ic = $TAMIP_DATA + tamipicGRIBag2 $TAMIP_DATA = /short/dp9/glr548/tamip_ic = /short/dp9/glr548/tamip_ic/tamipicGRIBag2 output um ic = $DATAW + $RUNID.astart $DATAW = /short/dp9/glr548/vaceo = /short/dp9/glr548/vaceo/vaceo.astart raijin: cd /short/dp9/glr548/tamip_ic \rm tamipicGRIBag2 ln -s /g/data1/p66/glr548/tamip_sam/cyclone/cyclone_2006-07-12T00.grib =>ic = /short/dp9/glr548/tamip_ic/tamipicGRIBag2 = /g/data1/p66/glr548/tamip_sam/cyclone/cyclone_2006-07-12T00.grib
SST and Seaice accessdev umuix/vaceo/Mod/Atm/Anc/Clim/ SST $UM_ANCIL_SST_DIR $UM_ANCIL_SST_FILE Y-SST-c N-ANOM Y-AMIP2 ICE $UM_ANCIL_SEAICE_DIR $UM_ANCIL_SEAICE_FILE Y-Frac-c Y-Thick-c Y-AMIP2 umuix/vaceo/Mod/Atm/Anc/InFileRelatedOption/AncilVers Filenames = /g/data1/p66/glr548/tamip_sam/ag2/ag2ancfilnames_N512L70np Versions = /g/data1/p66/glr548/tamip_sam/ag2/ag2ancdirnames_N512L70np raijin cd /g/data1/p66/glr548/tamip_sam/ag2 emacs ag2ancdirnames_N512L70np& and we see export UM_ANCIL_N512cyclone=/g/data1/p66/glr548/tamip_sam/cyclone #n512 ag2 grid for cyclone sst+sice export UM_ANCIL_SST_DIR=$UM_ANCIL_N512cyclone export UM_ANCIL_SEAICE_DIR=$UM_ANCIL_N512cyclone emacs ag2ancfilnames_N512L70np& and we see #cyclone ag2.grid export UM_ANCIL_SEAICE_FILE=qrclim.seaice_cyclone #cyclone ag2.grid export UM_ANCIL_SST_FILE=qrclim.sst_cyclone =>sst anc = /g/data1/p66/glr548/tamip_sam/cyclone/qrclim.sst_cyclone ice anc = /g/data1/p66/glr548/tamip_sam/cyclone/qrclim.seaice_cyclone

Run vaceo reconfiguration job

Run accessdev reconfiguration job vaceo to convert the ERA-Interim grib file created above into UM format accessdev: umuix/vaceo/CSP note: - ignore single stash error in Check Setup section - Only do the Check/Save/Process steps first Now we need to change the emacs /home/548/glr548/umui_jobs/vaceo/SHARED& change L_endgame=.TRUE., -> L_endgame=.FALSE., (note: otherwise we get reconfig to EndGame grid) umuix/vaceo/Submit=ok, outputs /home/548/glr548/um_output/vaceo000.vaceo.d14353.t130103.rcf.leave /short/dp9/glr548/vaceo/vaceo.astart ==produces the UM format dump file /short/dp9/glr548/vaceo/vaceo.astart
One final step is to remove the extra GA6 fields produced by the reconfiguration job

   cd /short/dp9/glr548/vaceo/
   module load python/2.7.3-matplotlib
   module load pythonlib/umfile_utils/latest
   module load pythonlib/cdat-lite/6.0rc2-fixed
   cp /g/data1/p66/glr548/tamip_sam/cyclone/um_fields_subset.py .
   python um_fields_subset.py \
   -i vaceo.astart \
   -o cyclone_2006-07-12T00ss \
   -x 243,387,388,389,390,391,392,393,394,395,396,397,398 
   ==Resetting no of prognostic fields from 3500 to 3499

This creates cyclone_2006-07-12T00ss which is the UM-formatted dump file corresponding to the ERA-Interim grib data we first downloaded and is suitable for running in an ACCESS Global AG2 N512L70 forecast job eg accessdev/raijin job APS2 ACCESS Global Domain (access_g_vn8.2_25km) or ngamai job azuje.

Back to ACCESS Model Experiment Library

Back to Main ACCESS page

[12] Review and updates by greg Roff

For further support on this suite, please contact [email protected]