ticket_266_TicketSummary - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki

Importing VerPy beta version (version 0.8) into the access-svn utils repository

Chris Bridge has put VerPy version 0.8 in Bureau's gitlab repository,

[email protected]:cbridge/VerPy.git

Unfortunately the Bureau's gitlab repository is not accessible from outside of Bureau's firewall. As the bulk of R&D work is done at NCI the VerPy source was imported into the following location in the access-svn utils repository,

https://access-svn.nci.org.au/svn/utils/trunk/VerPy

Update. VerPy version 0.8 seems to have problems and I received a copy of version 1.1 from Adam Clayton. This was imported into the following location of access-svn utils repository,

https://access-svn.nci.org.au/trac/utils/browser/trunk/verpy/verpy1.1

Installing VerPy on Raijin

  • Pre-requisite Python modules needed on Raijin,

    1. docutils

  • Location of temporary installation,

    1. /g/data/dp9/jtl548/source/verpy/verpy1.1

    ToDo VerPy should be installed under a more general location: preferably under raijin:~access

Testing VerPy on Raijin

Environment needed by VerPy

  • To use the VerPy module set PYTHONPATH to point to the VerPy installation. E.g.
export PYTHONPATH=/g/data/dp9/jtl548/source/verpy/verpy1.1:$PYTHONPATH

  • VerPy makes a call to the VER script, "VerScr_GeneralPlotting". But it looks like in VER17.2 on the shared repository this script is missing. Rob Darvell sent me a copy and I placed under VER 'build/bin' subdirectory.

    . Q. Does VerPy use IDL for plotting?

  • Need to set the environment variable, GEN_BUILD_DIR which should point to the VER 'build/bin' subdirectory. E.g.

export GEN_BUILD_DIR=/g/data/dp9/jtl548/source/ver/r84_ver17.2_nci_mirror/build/bin

Plotting data in ARDFiles using VerPy

In order to test whether the ARDFiles produced by VER are correct or not I am using VerPy to read data from ARDFiles and plot the data. The generated ARDFiles are from the deterministic cycles of the Rose suite, "u-ac651" and are under,

raijin6:/g/data1/dp9/jtl548/cylc-run/u-ac651/share/data/ver/ARD_GM

Following is the commands I used to plot errors vs. forecast lead times,

import VerPy as ver
opts = {
      'start'   : 20150615,
      'end'     : 20150616,
      'times'   : [0,600,1200,1800],
      'stat'    : [6051,6752],
      'output'  : 'timeseries',
      'fcrs'    : [1200],
      'system'  : 'ABV',
      'type'    : 'continuous',
      'source'  : '/g/data/dp9/jtl548/cylc-run/u-ac651/share/data/ver/ARD_GM',
      'expids'  : ['u-ac651-GM'],
      'names'   : ['ARD_GM'],
      'area'    : [101,102,103],
      'grid'    : 2,
      'param'   : [(16,8,1.0)],
      'jobid'   : 'Jin',
      'confopts': 'stderr',
      'dataopts': ['equalize','K to C', 'm/s to kt', 'Pa to hPa'],
      'truth'   : 0}
ver.job.run('.',opts)

Or to set the key/value pairs of the dictionary variable, 'opts' in a single line to be copy-and-paste friendly,

opts={'start':20150615, 'end':20150616, 'times':[0,600,1200,1800], 'stat':[6051,6752], 'output':'timeseries', 'fcrs': [1200], 'system':'ABV', 'type':'continuous', 'source':'/g/data/dp9/jtl548/cylc-run/u-ac651/share/data/ver/ARD_GM', 'expids':['u-ac651-GM'], 'names':['ARD_GM'], 'area':[101,102,103], 'grid': 2, 'param':[(16,8,1.0)], 'jobid':'Jin', 'confopts':'stderr', 'dataopts':['equalize','K to C', 'm/s to kt', 'Pa to hPa'], 'truth': 0}

Documentation

  1. UKMO made VerPy sources available through the VER MOSRS shared repository, https://code.metoffice.gov.uk/trac/ver/browser/verpy. Various versions of VerPy documentation are viewable through https://code.metoffice.gov.uk/doc/verpy

  2. From time to time you may need to refer to the VER documentation. Please use the MOSRS VER documentation page (https://code.metoffice.gov.uk/doc/ver).

Sample VerPy scripts

Adam Clayton (the person who works on Hybrid VAR for UKMO but has been in KMA for a number of years) sent me a tarball with a newer version of VerPy (version 1.1). In that tarball there are some example VerPy scripts for generating different types of plots. I untar'ed the tarball and placed all the files under,

raijin1:/g/data/dp9/jtl548/source/verpy/verification_package/

Xudong Sun tested various Verpy plots at Raijin: /g/data/dp9/xds548/verpy for APS3 Presentation and some plot in http://logan:8011/nwp-wg/wiki/NWPTNotes_20160818#no1 Discussions and plots at (APS3) Suite Working Group Wiki