Cluster setup - imanlab/iml_wiki GitHub Wiki
Steps to setup the cluster
Guide on how to use LCAS/CLAS cluster
1- Ask xxx to create an account.
2- Make a UoL lincoln VPN by this repository "https://github.com/marc-hanheide/openconnect_cookie_extractor"
3- open "remote.lincoln.ac.uk" by google chrome and log in
4- click on VPN extension sign and copy the command
5- open a terminal and paste the copied command
6- open a new terminal and enter:
> sudo ip route add 10.5.39.159 dev tun0
(tun0 name can be found by "ip addr" or "ifconfig", it's the device name of the VPN)
7- To access the file system in Ubuntu file system click on connect to server and type "sftp://username@lch01/home/username" to get connected.
8- To connect in the terminal use:
> ssh [email protected]
and enter password.
9- Slurm is used to resource allocation. To submit a job use "sbatch fileName.sh" in the terminal. The .sh script to run script.py looks like:
#!/bin/bash
# file submit.sh
#SBATCH --job-name=RCNLSTM
#SBATCH -N 1
#SBATCH --output=test.out
## Run two jobs in parallel on the SAME node (e.g. if each job requires a single GPU)
## Currently, creates an additional empty output file "slurm-jobid.out", not sure
## how to avoid.
module add cuda/9.0
srun -wLCH02 python3 script.py
10- To verify that your job has been submitted, use:
> squeue
> watch --i 0.5 squeue
11- You can cancel a job by "scancel job_id"
> scancel job_id
12- To check if any gpu is empty on a given node:
> srun -wLCH02 nvidia-smi
13- To copy files with large volume try to use:
> scp -r Music username@lch01:~