Cluster setup - imanlab/iml_wiki GitHub Wiki

Steps to setup the cluster

Guide on how to use LCAS/CLAS cluster

1- Ask xxx to create an account.

2- Make a UoL lincoln VPN by this repository "https://github.com/marc-hanheide/openconnect_cookie_extractor"

3- open "remote.lincoln.ac.uk" by google chrome and log in

4- click on VPN extension sign and copy the command

5- open a terminal and paste the copied command

6- open a new terminal and enter:

     > sudo ip route add 10.5.39.159 dev tun0

(tun0 name can be found by "ip addr" or "ifconfig", it's the device name of the VPN)

7- To access the file system in Ubuntu file system click on connect to server and type "sftp://username@lch01/home/username" to get connected.

8- To connect in the terminal use:

     > ssh [email protected]

and enter password.

9- Slurm is used to resource allocation. To submit a job use "sbatch fileName.sh" in the terminal. The .sh script to run script.py looks like:

     #!/bin/bash
     # file submit.sh
     #SBATCH --job-name=RCNLSTM
     #SBATCH -N 1
     #SBATCH --output=test.out




     ## Run two jobs in parallel on the SAME node (e.g. if each job requires a single GPU)
     ## Currently, creates an additional empty output file "slurm-jobid.out", not sure
     ## how to avoid.

     module add cuda/9.0

     srun -wLCH02 python3 script.py

10- To verify that your job has been submitted, use:

     > squeue
     
     > watch --i 0.5 squeue

11- You can cancel a job by "scancel job_id"

     > scancel job_id

12- To check if any gpu is empty on a given node:

     > srun -wLCH02 nvidia-smi

13- To copy files with large volume try to use:

     > scp -r Music username@lch01:~