access_YiXiaoEmail22Mar2012 - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki

#!html
<h1 style="text-align: center; color: blue"> ACCESS NWP cylc prototype and discussion </h1>

From: Yi Xiao

Sent: Thursday, 22 March 2012 10:12 AM

To: Kamal Puri; Martin Dix; Peter Steinle; Gary Dietachmayer; Jim Fraser

Cc: Michael Naughton; Yi Xiao

Subject: ACCESS NWP [SEC=UNCLASSIFIED]

Hello, Leaders

Here are some thoughts from me about ACCESS NWP systems.

After developed access nwp suites from global to srep high resolution suites, I was trying to improve their efficiency. This is particular a problem for us, when we are moving to higher resolution. For example SREP-1.5km, it takes about 16 hours to run entire one's 8 cycles -- when everything is running smoothly.

About mid of last year, Hilary Oliver from NIWA introduced cylc to me I was attract to its unusual algorithm, and the maximum efficiency that cylc can achieve. After spent some time on it, I am very pleased that not only it offers the efficiency improvement that we need, it is also easy to use. In addition, under cylc, a research suite can naturally become an operational suite.

My suggestion to you is simple, we should use cylc. Reasons being:

  1. Efficiency Using cylc can greatly improve the efficiency of ACCESS NWP systems. As a result, the usage of the machine will be improved -- so we can do more with the existing computing power.
  2. Usability Cylc offers a lot of command line helps and a rich set of utilities which make development easier. Suite under cylc are more transparent, debug will be a lot less stressful. The building in event hooks make the monitoring more effective.
  3. Portability The namespace hierarchy that cylc offers make it very simple to run a suite across a network. For example, some tasks run on machine A at BOM, and others on Machine B at NCI.
  4. Availability and Dependency Cylc is on public domain, it is free to use. Cylc does not depend on ROSE, although ROSE will make things easier for developing ACCESS nwp. Cylc is currently installed on flurry (yix) and solar (azs).
  5. Experiences a. I have a very small suite under cylc that has been running daily since late last year. It produces radar obs for SREP Sydney system. b. I have a fully functional ACCESS-G suite under cylc, as well as a small relocatable forecast only suite. c. I also have LAM and SREP suites under cylc, they are not fully functional yet.

In addition to all the advantages listed above, cylc can also produce very nice graphs which are very effective for users to understand the suites. Those graphs are very nice for publications too. I attached a few just for illustration.

Steps we should be taken, I think, are as follows:

  1. Continue the support for all current ACCESS NWP systems on solar

  2. Gradually introduce ACCESS NWP suites under cylc to CAWCR users

  3. Port ACCESS-NWP-cylc to NCI --- we do not port ACCESS-NWP-SCS to NCI

I can give a demo if interested.

I am please that I can escape from the reality sometimes. Xiao

⚠️ **GitHub.com Fallback** ⚠️