5. How to create a Submission Script - bregord/emcee-on-calcul-quebec GitHub Wiki

In order to tell Torque, the task manager, what resources your job requires, you will need to write a submission script.

A script has the following structure: A head, involving the various directives to tell Torque how to process your job and what resources to give to it, followed by the body which contains the sequence of commands required to run your job.

###Head The first line of your script must always be a shebang:

  #!/bin/bash

Basically, this line just tells Torque which shell it should use to run your script. If a user was so inclined, "bash" could be replaced by any other shell name.

The next part of the head is the list of directives to be given to torque. They are often of the format: #!/bin/bash

#PBS -l nodes=4:ppn=4  

#PBS -l walltime=00:30:00  

#PBS -A your-compute-canada-project-id  

#Your id should be of the form abc-123-aa  

#PBS -V  

#PBS -N JobName  

#PBS -l pmem=5700m  

#PBS -m bae  

#PBS -M [email protected]  

The usage of these and other arguments is explained in depth on the calcul quebec wiki here

It is also important to note, that certain arguments differ accross different clusters. For instance, the pmem flag above is unique to guillimin, in that it specifies the minimimum memory required per core.

IMPORTANT NODE When choosing a memory allocation, make sure that you choose an amount that is never equal to or greater than the amount allowed by the amount . Your job will fail. Above, I allocate 5.7 GB per core, even though each node

###Body

The body of the script simply involves whatever sequence of commands you would require to run your program normally if you were to start from the moment the session began. Often, this involves loading the modules you wish to use, changing to your directory of choice, the activating of your virtual environment, and the running of the command.

Here is an example:

cd ~/workspace/program

module load ifort_icc/15.0 
module load python/2.7.3-MKL
module load iomkl/2015b
module unload OpenMPI/1.8.8
module load openmpi/1.6.3-gcc

source env/bin/activate
 
#this specifies the program's output
program.py > output.txt