strigger - hokiegeek2/slurm-cloud-integration GitHub Wiki

Requirements

Slurm User

The strigger command can only be executed by the slurm user. Otherwise, a permissions error is thrown:

root@einhorn:/home/kjyost# sbatch simple.batch 
Submitted batch job 37
root@einhorn:/home/kjyost# squeue
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
                37     debug     test     root  R       0:03      1 einhorn
root@einhorn:/home/kjyost# strigger --set --jobid=37 --fini --program='/tmp/trigger.sh' -v
slurm_set_trigger: Access/permission denied
root@einhorn:/home/kjyost# 

Executable File

The program file must be executable and be executable by the slurm user. Otherwise, an Access/permission denied error is thrown

Scripts

A script defined as the program in an strigger must have the shebang at the top (example below):

#! /bin/sh
echo 'job complete' >> /tmp/slurm-echo.txt

Setting strigger for a job

slurm@einhorn:/tmp$ sbatch simple.slurm
Submitted batch job 40
slurm@einhorn:/tmp$ squeue
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
                40     debug     test    slurm  R       0:03      1 einhorn
slurm@einhorn:/tmp$ strigger --set --jobid=40 --fini --program='/tmp/trigger.sh' -v
strigger: trigger set

Getting striggers for a job

slurm@einhorn:/tmp$ strigger --get --jobid=40
TRIG_ID RES_TYPE   RES_ID TYPE                                OFFSET USER     FLAGS PROGRAM
      2 job            40 fini                                     0 slurm          /tmp/trigger.sh
slurm@einhorn:/tmp$ 

Confirming Example strigger Worked

slurm@einhorn:/tmp$ strigger --set --jobid=61 --fini --program='/tmp/trigger.sh' -v
strigger: trigger set
slurm@einhorn:/tmp$ squeue
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
                61     debug     test    slurm  R       0:21      5 ace,einhorn,finkel,robinson,shickadance
slurm@einhorn:/tmp$ ls -ltr
total 36
drwx------ 3 root   root   4096 Jan 23 15:33 systemd-private-2aaa74dd1b324f74816dad6298886d59-systemd-timesyncd.service-p16EJI
drwx------ 3 root   root   4096 Jan 23 15:33 systemd-private-2aaa74dd1b324f74816dad6298886d59-systemd-resolved.service-ZxVcvH
drwx------ 3 root   root   4096 Jan 23 15:34 systemd-private-2aaa74dd1b324f74816dad6298886d59-ModemManager.service-GR5OeO
drwx------ 3 root   root   4096 Jan 23 15:34 systemd-private-2aaa74dd1b324f74816dad6298886d59-colord.service-CI7hxD
drwx------ 3 root   root   4096 Jan 23 15:35 systemd-private-2aaa74dd1b324f74816dad6298886d59-rtkit-daemon.service-6TWRBa
drwx------ 3 root   root   4096 Jan 23 15:35 systemd-private-2aaa74dd1b324f74816dad6298886d59-bolt.service-NeFokW
-rw-r--r-- 1 kjyost kjyost  208 Jan 23 15:48 simple.slurm
-rwxrwxr-x 1 slurm  slurm    54 Jan 23 16:04 trigger.sh
-rw-r--r-- 1 slurm  slurm    26 Jan 23 16:08 slurm-echo.txt
slurm@einhorn:/tmp$ cat slurm-echo.txt 
job complete
job complete
slurm@einhorn:/tmp$