PBS Job Array - shawfdong/hyades GitHub Wiki

Here is the source code (jobarray_hello.c) for a sample serial program that takes an integer argument:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main (int argc, char *argv[])
{
  int id;
  char hostname[80];

  if ( argc == 2 ) {
    id = atoi(argv[1]);
    gethostname(hostname, 80);
    printf("Hello master, I am slave no. %d running on %s!\n", id, hostname);
    exit(0);
  }
  else {
    printf("Usage: %s id\n", argv[0]);
    printf("where id is an integer\n");
    exit(1);
  }
}

We compile it as usual using Intel C compiler:

icc jobarray_hello.c -o jobarray_hello.x

The executable takes an integer argument. For example, if we run it on the master node:

./jobarray_hello.x 15

And it says:

Hello master, I am slave no. 15 running on hyades.ucsc.edu!

If we want to run a lot of instances of the program, with different arguments, for example:

./jobarray_hello.x 101
./jobarray_hello.x 102
./jobarray_hello.x 103
./jobarray_hello.x 104

instead of submitting 4 serial jobs, we can submit one job array (jobarray.pbs):

#!/bin/bash

#PBS -N jobarray
#PBS -l ncpus=1
#PBS -t 101-104
#PBS -j oe
#PBS -l walltime=0:20:00

cd $PBS_O_WORKDIR
./jobarray_hello.x $PBS_ARRAYID

I submitted it to the gpu queue:

qsub -q gpu jobarray.pbs

When the jobs completed, I got 4 files:

jobarray.o54654-101  jobarray.o54654-102  jobarray.o54654-103  jobarray.o54654-104

each for a member of the job array. For example:

# cat jobarray.o54654-102
Hello master, I am slave no. 102 running on gpu-8.local!