Using cylc version 6

Differences

Cylc6 introduces a new format for dates based on the ISO 8601 standard for date-time instants, intervals, and sequences.

See Appendix J of the user guide for information on migration http://cylc.github.io/cylc/html/single/cug-html.html#J

There is also a change in the directory structure for log and work directories, https://github.com/cylc/cylc/pull/1069. This requires a newer version of rose. Now uses

    log/job/CYCLE/TASK/{01,02,...,NN}/{job,job.out,job.err,job.status}
    work/CYCLE/TASK/

rather than

    log/job/{job.CYCLE.TASK,job.CYCLE.TASK.out,...}

See the full list of changes at http://cylc.github.io/cylc/changes.html#6.0.0.

Many suites should work unchanged with cylc6. In other cases cylc6 tries to upgrade times specified in the older format automatically when processing the suite, though this doesn't always work and manual changes may be required.

PBS directives

With cylc 5 all the -l directives had to be on one line, e.g.,

  [[directives](/ACCESS-NRI/accessdev-Trac-archive/wiki/[directives)]
    -l = 'walltime=1000,ncpus=64,mem=64gb'

With cylc6 (actually introduced in 6.1.0), these can be separated

  [directives](/ACCESS-NRI/accessdev-Trac-archive/wiki/directives)
    -l walltime = 1000
    -l ncpus = 64
    -l mem = 64gb

This makes inheritance more useful because directives can be overridden individually.

Setup on accessdev

The current default versions have not changed, rose 2014-05 and cylc 5.4.14. New versions installed alongside are rose 2015.02.0 and cylc 6.3.0. accessdev doesn't support the same module environment as on raijin so these are selected via environment variables rather than loading modules.

On raijin, there are wrapper scripts ~access/bin/rose and ~access/bin/cylc. With cylc6 this path is specified in the configuration on accessdev, but for cylc5 this isn't possible. To run cylc5 suites either include ~access/bin in your default PATH on raijin or load the cylc module in the root section of your suite, e.g.

[runtime]
    [root](/ACCESS-NRI/accessdev-Trac-archive/wiki/root)
        initial scripting = """
 module use ~access/modules
 module load cylc
"""

However if you want to test a suite with both cylc5 and cylc6, you must use

module load cylc/wrapper

instead.

It's no longer necessary to load rose or cylc modules in your raijin environment (e.g. in .bashrc, .profile, .bash_profile). Doing so may cause conflicts with the cylc6 wrappers.

If you were an early adopter of cylc you may have a $HOME/.cylc/user.rc configuration file. If there is a raijin section in this it might cause job submission problems by overriding the path specification. If you get problems with job submission try running

% CYLC_VERSION=6.3.0 cylc get-site-config

Output from this should include

[hosts]
  ... 
  [raijin.*](/ACCESS-NRI/accessdev-Trac-archive/wiki/raijin.*)
     .... 
     cylc executable = /projects/access/bin/cylc

Running a suite with cylc6

Specify the rose and cylc versions with environment variables, either persistently or on the command line as

CYLC_VERSION=6.3.0 ROSE_VERSION=2015.02.0 rose suite-run

Alternately, you can specify the cylc version in the suite rose-suite.conf file by adding

[env]
CYLC_VERSION=6.3.0

(a value set in rose-suite.conf overrides a shell environment variable). There's no equivalent way to set the rose version.

The version environment variables need to be passed to raijin so that matching versions are used there. With cylc 6.3.0 this doesn't quite work automatically so your suite should have a section

[runtime]
   [root](/ACCESS-NRI/accessdev-Trac-archive/wiki/root)
       initial scripting = export ROSE_VERSION=[ ROSE_VERSION ]

Trying to run with the old rose and new cylc or vice-versa will cause problems due to the different log file paths if nothing else.

Upgrading suites

Running

CYLC_VERSION=6.3.0 cylc 5to6 suite.rc

will give suggested changes. See the manual for more details http://cylc.github.io/cylc/html/single/cug-html.html#D.2.1

Viewing the Shared Repository Rose jobs

Currently the Rose repository on code.metoffice.gov.uk is not configured centrally. To view the Met Office Rose repository you'll need to add a custom configuration to your personal Rose config file.

Your personal Rose config file is located at ~/.metomi/rose.conf. To see the shared repository add the following text to this file, replacing {METO_USER} with your Shared Repository username:

[rosie-id]
prefix-username.u = {METO_USER}
prefix-location.u = https://code.metoffice.gov.uk/svn/roses-u
prefix-web.u      = https://code.metoffice.gov.uk/trac/intertrac/source:
prefix-ws.u       = https://code.metoffice.gov.uk/rosie/u

This will be added to the central config file when we update the default Rose.

Issues

rose-bush with cylc6

rose-bush uses the default version of rose and so doesn't understand the new cylc6 log structure. It shows the cycle list properly but not the job list. This will be fixed when we make the new versions the default.

New default versions

After the default versions are updated to rose 2015.04.1 and cylc 6.4.1 it's possible to run older versions by specifying environment variables, e.g.

CYLC_VERSION=5.4.14 ROSE_VERSION=2014-05 rose suite-run

Beware the change in format of the rose version numbers between 2014 and 2015.