Cromwell

This guide assumes that you have read through the shared file systems page and the slurm page. You may wish to also read through the official cromwell for more information.

• Create toil directories on the sfs
• Activate env
• Running TOIL

Starting the cromwell server

This must be done before launching any workflows.

A simple shell script to start cromwell on the master node has been created in the post-install.sh.
This can be found at "${HOME}/bin/start-cromwell-server.sh".

Launching a workflow

We should first set up cromwell, which will allow us to easily launch workflows to the server.
This saves us having to create POST requests through curl and instead lets us stick to commandline variables.

In order to save the variables initialised below throughout your shell, you should complete the following through a screen

Set up cromshell

cromshell is an easy way to talk to the cromwell server.
We will go through the very basic commands in the next step.

cromshell is installed in the cromwell_tools conda env.
Please activate this environment before continuing.

conda activate cromwell_tools

mkdir -p .cromshell
cromshell_config_file=".cromshell/cromwell_server.config"
cromwell_port="8000"

if [ ! -f .cromshell/cromwell_server.config ](/umccr/aws_parallel_cluster/wiki/-!--f-.cromshell/cromwell_server.config-); then
  echo "http://localhost:${cromwell_port}" > "${cromshell_config_file}"
fi

Launching a job through cromshell.

Cromshell takes in three positional arguments

1. /path/to/wdl-workflow - this is the path to the wdl workflow
2. /path/to/inputs.json - this is the path to the input json file
3. /path/to/options.json - these are options that can override the configuration file.
   • Set the options file to /opt/cromwell/configs/options.json.

Metadata

We can also check if a workflow is running through cromshell using the status subcommand.

cromshell status

Check metadata of a job

Useful for debugging a workflow that has failed

cromshell metadata <WORKFLOW_ID>

Finding my outputs

All files should be under ${SHARED_DIR}/cromwell/outputs.
This may be different if you have specified a different place in the options.json