#!html
<h1 style="text-align: center; color: blue"> Rose Tutorial Based on MET's Tutorial: A Brief Tour </h1>

This wiki page provides a brief guide to several aspects of Rose. It is mainly based on the MET Office Tutorial: A Brief Tour, but includes several modifications for NCI's VM accessdev.

Rose can either be run on the command line or through the its graphical user interface. This tutorial will focus on the command line and is aimed at people learning about Rose and how to write suites from scratch.

To create an empty new suite, run the command for the prompt:

rosie create --prefix=test # test is the identifier for the test repositories

This command will prompt you to edit the suite discovery information file. Put some meaningful values in for the project (e.g. project=rose-user-guide) and the title settings (e.g. title=Brief tour). Save the file and exit the editor.

A new suite will have been created in the version control system, with a local copy created (i.e. checked out) at $HOME/roses/$ID/ where $ID is a unique ID for the new suite. The $ID is made up of two components. The first part is the prefix (test in this case) followed by a unique id made up of two letters and three numbers (eg. aa001). Your $ID should look similar to test-aa001.

Change directory to the local copy of the suite and populate it by running the following commands:

cd $HOME/roses/$ID  # Replace $ID with your suite ID
rose rug-brief-tour # Rose command to set up the tour suite

The rose rug-brief-tour command should create some content for this guide within the current working directory. The following files and sub-directories will have been created:

• A rose-suite.conf file. This is used to define suite configurations outside of cylc. It contains variables that can be inserted into suite.rc file. In this example the file is left empty. See http://metomi.github.io/rose/doc/rose-rug-advanced-tutorials-jinja2.html for more details.
• A rose-suite.info file. This is the suite discovery information file which you saw when you created the suite.
• A suite.rc file. This is used to define configurations in a cylc suite. In this example, we are running:
  • fcm_make: a special Rose application to build an executable from a source file containing a hello world program
  • fred_hello_world, my_hello_mars, my_hello_world: Rose applications that invoke the hello world executable using different configurations.
  • locate_fred: A preparation task run each time before fred_hello_world.
• Sub-directories
  • app/. This directory contains the configurations of the applications run by the tasks of the suite. In this example suite, the directory contains sets of application configurations for running the hello world program in different ways. app/fcm_make contains the configuration for a special Rose application called fcm make. Calls of fcm make can be used for extracting and compiling source code.
  • src/. This directory contains the source file of a simple program that we shall run in this suite. Note: This directory will not normally be present in a real scientific suite, as source files are typically held in separate locations.

At this point, you may want to have a look at the files before moving on to the next section. You may also want to commit your suite - if so, change directory to the working copy of your suite ($HOME/roses/$ID). To add new files for version control, run:

svn add *
svn commit -m "Add files from Brief Tour"
svn update

Committing files into the repository will also update the Rose database. The Rose database contains basic information about each suite, including owner, project name, suite title, etc.

Install and run the suite by using the following command:

cd $HOME/roses/$ID # replace $ID with your suite ID
rose suite-run

Whilst the suite is running you can explore the cylc gui. Try experimenting with different viewing options using the toolbar.

The suite should run successfully to completion and shutdown (the status bar should indicate stopped with 'held'). You can now quit the cylc gui.

Note: If you quit cylc gui by mistake, you can restart it by typing rose suite-gcontrol --name=$ID on the command line.

The output of your suite can be found at $HOME/cylc-run/$ID/.

Important sub directories are in log/job/. These directories contains the STDOUT and STDERR from each task alongside their respective script used to run the task.

Alternatively, you can type rose suite-log-view --name=$ID to launch your browser and view the output of your suite without having to navigate to its folder.

The share/ sub-directory is used by the tasks in the suite to share files. In this example suite, the build created by the fcm_make task can be found here.

The work/ sub-directory contains the working directory for the tasks at their run times. It can be used by the tasks to create files that are only relevant to the tasks.

The second part to this tutorial will show how a suite is run on the compute node rather than the local machine. To run the suite on the compute node, you will need to edit the files fcm-make.cfg and suite.rc in order to specify tasks run on the compute nodes rather than the local machine.

Your environment on Accessdev needs to be configured to allow job submission to the computes nodes. If you haven't set up job submission beforehand, run the following command (You will be asked for your password several times);

remote-job-submission

The FCM make file can modified to specify where the compilation should happen. To specify compilation should occur on vayu.nci.org.au, edit the fcm-make.cfg file in app/fcm_make/file/ by adding the following lines:

extract.ns = hello
extract.location[hello] = $ROSE_SUITE_DIR/src
mirror.target = vayu.nci.org.au:/short/$PROJECT/$USER/cylc-run/$ROSE_SUITE_NAME/share/fcm_make
mirror.prop{config-file.steps} = build

And replace the line:

steps = build

with

steps = extract mirror

This will tell fcm make to copy the source code onto the compute node and do the build there instead on Accessdev.

The suite.rc also needs to be edited to specify that jobs should be submitted on the compute nodes rather than Accessdev. To do this, edit suite.rc by:

Adding a new task called fcm_make2 under the fcm_make task. So it looks like:

    [[fcm_make]]
        pre-command scripting = "sleep 10"  # Slow down fcm make so we can see it run
    [[fcm_make2]]
      [[[job submission]]]
         method = pbs
      [[[remote]]]
        host = vayu.nci.org.au
      [[[directives]]]
        -l = 'walltime=600, ncpus=1, vmem=100mb'

This tells cylc to run the task fcm_make2 on vayu.nci.org.au via pbs.

We also want to tell other tasks to run on the compute node. So for each task locate_fred, my_hello_world, fred_hellow_world and my_hello_mars add the following lines.

    [[[job submission]]]
         method = pbs
      [[[remote]]]
        host = vayu.nci.org.au
      [[[directives]]]
        -l = 'walltime=1800, ncpus=1, vmem=100mb'

We need to specify that fcm_make2 will run only once at the beginning. To do this, change the special tasks to include fcm_make2.

   [[special tasks]]
        cold-start = fcm_make, fcm_make2 # These tasks are only run at the beginning of the suite

We also need to include fcm_make2 in the task tree. To do this, change the dependencies in suite.rc to include calls to fcm_make2. This tells cylc to run the tasks, my_hellow_world, locate_fred after fcm_make2 is called on the compute node rather than fcm_make.

    [[dependencies]]
        [[[ 0, 6, 12, 18 ]]] # Each day at 00:00, 06:00, 12:00, 18:00:
            graph = """
                fcm_make => fcm_make2
                # my_hello_world is run after fcm_make or after the previous one finishes
                fcm_make2 | my_hello_world[T-6] => my_hello_world

                # locate_fred is run after fcm_make or after the previous one finishes
                # fred_hello_world is run after locate_fred
                fcm_make2 | locate_fred[T-6] => locate_fred => fred_hello_world
            """

        [[[ 0, 12 ]]] # Each day at 00:00 and 12:00:
            graph = """
                # my_hello_mars is run after fcm_make or after the previous one finishes
                fcm_make2 | my_hello_mars[T-12] => my_hello_mars
            """

Save the file and run the suite as before. The suite should run as before, except the tasks will be submitted to vayu.nci.org.au.

Log files are viewed in same way as before. However, output from running the suite is now located on the compute node under /short/$PROJECT/$USER/cylc-run/$ID/, where $ID was the name of your suite.