#!html
<h1 style="text-align: center; color: blue"> Rose Stem FAQ </h1>

PageOutline(2-8,Contents,inline)

A: A means to test code changes against standard tests. Each source code also contains a rose-stem directory which contains the tests. These are run as cylc suites and tested against known good output (KGO) or control files.

Rose Stem Documentation

The UM rose-stem Suite for External Users

VAR & OPS Rose Stem Documentation

A: Under rose-stem/site// are a series of directories containing suite*.rc files which contain site-specific configurations, environment variables, suite families and groups of tasks to run.

A: The rose-stem/app directory contains fcm_make_extract and fcm_make tasks that extract code from the various repositories, and compile the code. Note that the extract and build config is part of the source code under src/config.

To run, the command is:

rose stem -vv --task=fcm_make:ops_x86_64_ifort_opt_mirror

This specifies the configuration to use (the platform, compiler and optimization level) and in this case extract from a mirror. fcm_make is dependent on fcm_make_extract, so fcm_make_extract will run first, if it hasn't been run yet. Executing this command will result in a gcylc window opening and the fcm_make family of tasks running.

A: Like the rose-stem command used to do a build the command option, '--task' is used to specify the ID of the application to run and the FCM make config file to be used in the build.

To run (without building the code again, assuming it is already built), the command is:

rose stem -vv --task=<task_name>:<config_version> --define-suite=DEP_TASKS_COMPILE=false
e.g.
rose stem -vv --task=process_atms_glb:ops_x86_64_ifort_opt_mirror --define-suite=DEP_TASKS_COMPILE=false

To avoid running rose-ana tasks, you can use the following

rose stem --task=... -S DEP_TASKS_COMPARE_EXTERNAL=false

A: Stem testing is intended to make sure you haven't broken anything when you modify the source code. You should always run all the relevant stem tests after any code changes, and check that the outputs haven't changed (unless they are supposed to). The outputs are tested against KGOs using rose-ana tasks, which exist along side the stem tests.

For UM testing, see The UM rose-stem Suite for External Users

For OPS stem testing, there are a range of apps to test ODB creation, and OPS processing of all the various observation types that are in use. These stem tests are run by the Met Office as well as by NCI, and include almost all stem tests with bom in the name, as well as tasks with access_c in the name. You can run the full array of OPS stem tests by typing rose stem --task=bomall This will compile the code using the default config ops_x86_64_ifort_opt_mirror and run all the stem tests, comparing against KGOs.

For SURF and VAR you can run the full set of stem tests using rose stem --task=R_ALL. These cover the types of tasks typically run by us.

N.B. the KGOs are not always updated in sync with the trunk, so it's a good idea to check if the tests fail before making any code changes.

A: When you add to the source code, sometimes you need to create a stem test to exercise that new part of the code. In that case, there should be a stem test to go with your code changes, when you submit them for review. Here's how to create a stem test.

1. You should already have a ticket for your change, and a branch with the change.
2. Take a look at the stem tests in the <your_branch>/rose-stem/app directory and see if there are any similar tasks (e.g. a createodb task or a process task)
3. Create your test (maybe by copying from another test)

• You should aim to have the input files in the common location under $DIR/Acceptance_Tests where $DIR may be $OPSIR or $VARDIR or $SURF_HOME. Check in rose-stem/site/nci//suite-env.rc
• (optional) For portability, in the suite-app.rc, set the config_default with jinja thus {% set config_default = variables['default_<component>_config'] %} and make sure the default config is set in the suite-variables.rc file in rose-stem/site/nci/<common|component>/suite-variables.rc. This is standard in OPS but not SURF.
• Additionally, you can add your test to suite-apps.rc (to give it an abbreviation), suite-groups.rc (to include it in a batch of tests)

4. Run your test
5. Copy results to a control dir and set up suite-controls.rc
6. Set up a rose_ana task for your test (easiest is to copy another test)
7. Retest your branch and make sure your new test succeeds and the rose-ana test succeeds (can find the controls)
8. If necessary, get an ITO to copy the start files and controls to the appropriate directories.
9. Put your start files on jasmin in the TIDS workspace, for the code owners to access
10. Update your ticket with the test info and inputs location, changesets
11. Submit your branch to review, assigning your ticket to a code owner

A: Sometimes code changes will cause changes to the output. This may cause rose-ana to fail. If this happens, make sure your results are correct (i.e. the differences are intended.) Once you understand why the outputs have changed, update your ticket with this information.

You may need to update the controls for the stem tests, along with your code changes. In this case, get your new KGOs (normally the log and work dirs of your stem test) and copy them to the KGO location, into a new directory named appropriately (e.g. after your ticket number). Update the suite-controls.rc file to point to your new control.

When your code is reviewed and approved, the other partner(s) will also update their controls for that test.

A: Often, code changes are made without information us that they will alter the KGOs for some stem tests. If it is known that they are tests run by specific partners, ideally the person making the changes will notify the partners, and a new control can be organized to be added at the same time as the changes are committed to the trunk. However for OPS the stem tests were only recently introduced and VAR and SURF are not set up, so this hasn't been happening. The end result is that our KGOs become out of date, and need to be updated. This is a description of how to update the controls once they start to fail.

1. Create a ticket and cc at least one code owner, plus appropriate ITO (if you aren't one).
2. Create a branch off the trunk. (fcm bc -k <ticket_number> <branch name, e.g. update_kgo> <URL to trunk, e.g. https://code.metoffice.gov.uk/svn/ops/main/trunk or fcm:ops.x/trunk>)
3. Compile and run the stem tests to see which ones fail rose-ana. (rose stem --task=bomall for OPS or rose stem --task=R_ALL for VAR/SURF)
4. At this point you might want to check why the tests are failing.

• Most changes that affect the results are listed on the LoseBitComparison page, which is linked on the front wiki page of the relevant code component. From here, you can see which tickets affected which tests.
• Alternatively, if you go to <your_branch>/rose-stem/site/meto//suite-controls.rc you can see which controls have been updated since the last release.
• You may need to flick back through previous revisions to find all relevant changes
• The Met Office controls include the ticket number for which the controls were updated, and you can check the ticket to see why the controls changed.
• Alternatively, if you click on 'blame' at the top of the page, you can see the revision when the control changed. If you click on the revision number (right columm) you will get the change set, which usually includes the ticket number as well.

5. Once you're satisfied which controls need to be updated, you need to copy the control files to the shared location. For OPS this is "/g/data/access/OPS/KGO/"

• Create a new directory for the controls, which for example is named "control-t<ticket_number>", either the ticket number which altered the code, or your ticket to update the controls. If several tickets have contributed to changes, you should instead name the new controls for the revision number of the trunk for which your controls are valid, i.e. "control-r<revision_number>.
• Into this directory copy the log and work directories, i.e. cp $CYLC_SUITE_DIR/log <your_control_dir>; cp $CYLC_SUITE_DIR/work <your_control_dir>

6. Modify the <your_branch>/rose-stem/site/nci//suite-controls.rc to point to your control dir for the relevant tasks. There is a script to do this automatically, which is useful if many controls are being updated. It is held in one of the repositories (var_trunk/rose-stem/bin/update_controls).
7. Rerun the stem tests and check that no rose-anas fail.
8. If necessary, get an ITO to copy the new controls to the correct directory, then repeat the above two steps, with the correct path.
9. Commit your changes to the repository (fcm ci or fcm commit)
10. Update your ticket with changeset info and anything else
11. Submit your ticket for review and assign to a code owner.

A: Sometimes during the development of rose-stem suites and tasks for our site it's useful to examine processed rose-stem suite definitions ('suite.rc.processed') which are produced as if 'meto' was used for SITE.

Use following rose-stem arguments:

rose stem --group=ops_createbufrdirodb_kma_iasi_global: -S SITE='"meto"' -- --hold

This runs the group/task, "ops_createbufrdirodb_kma_iasi_global" using a default configuration. SITE is 'meto'.

A: This usually means that the configuration was not defined correctly.