2. Running jobs on Biowulf - AmyStonelake/Biowulf GitHub Wiki

See hpc.nih.gov to get a Biowulf account.
https://hpc.nih.gov/docs/accounts.html

Get a Globus account for transferring large files.
https://hpc.nih.gov/storage/globus.html

To run jobs on Biowulf, you must designate them as interactive, batch or swarm. Failure to do this may result in termination of your account.

1. Running interactive jobs

For most of what we do in this tutorial, it is suitable to use an interactive node. To do this, use the "sinteractive" command. You will see something like this on your screen. It may take a minute or two for the command to finish. You'll know it's done when you get your command line ($) back. You only need to use the "sinteractive" command once per session. If you try to start an interactive node on top of another interactive node, you will get a message asking why you want to start another node.

[username@biowulf ]$ sinteractive    
salloc.exe: Pending job allocation 34516111    
salloc.exe: job 34516111 queued and waiting for resources    
salloc.exe: job 34516111 has been allocated resources    
salloc.exe: Granted job allocation 34516111    
salloc.exe: Waiting for resource configuration    
salloc.exe: Nodes cn3317 are ready for job    
srun: error: x11: no local DISPLAY defined, skipping    
[username@cn3317 ]$     

If you exit the Biowulf system while running an interactive node, your job will be terminated. To keep jobs running after you log out you will need to run a batch or swarm job.

2. Batch jobs

Most jobs on Biowulf should be run as batch jobs using the "sbatch" command.

3. Swarm-ing on Biowulf

Swarm is a script for running a group of commands on Biowulf. Swarm reads a list of command lines and automatically submits them to the system. To create a swarm file, you can use "nano" or another text editor and put all of your command lines in a file called "file.swarm". Then you will use the "swarm" command to execute.

swarm -f file.swarm

Swarm creates two output files for each command line, one each for STDOUT (file.o) and STDERR (file.e). You can look into these files with the "less"command to see any important messages. We will create a swarm file later in the tutorial.