CCR - UB-BiomedicalInformatics/BMI_p2p_sessions GitHub Wiki
Introduction to the University at Buffalo Center for Computational Research (CCR).
secure socket shell - used to securely access a remote computer
$ ssh [email protected]
- hostname: vortex.ccr.buffalo.edu
- username: UB username
- port: 22
Important Tags to take note of:
- Cluster: Group of partitions on a head node
- Example: mae, industry, ub-hpc
- Partition: Group of nodes within a cluster
- Example: compbio, scavenger, gpu, general-compute
- Nodes: The official names of the computing nodes (only used when logging directly onto a node, can only be done when the user has a job running on that node)
- Processors: Number of computers on each node
- Memory: The amount of RAM on a node in MB
- Example: 23 GB = 23000 MB
- Constraints: IMPORTANT!!! These must be used to force your jobs onto the node you want!
What the constraint tags mean:
- IB : Infiniband, Nodes with infiniband allow for multi-node jobs. Nodes without them will not run properly for multi-node jobs
- The other constraint gives the head node directions about which nodes are allowed to run the given job
- Example: CPU-E5-2430 means the job can only run on the 12 processor 48 GB nodes.
According to one CCR update (June 25-29, 2018), a valid QOS is required in slurmscripts, eq:
#SBATCH --partition=general-compute #SBATCH --qos=general-computeUsually QOS is the same as the partition name specified. The command "slimits" will show the QOS settings you have access to.
If you have a question please ask before you submit.
Example of environmental variables to set for any SLURM submit script:
#!/bin/sh #SBATCH --nodes= (number of nodes required: 1) #SBATCH --ntasks-per-node= (number of processors on node: 12) #SBATCH --cluster= (Name of cluster: ub-hpc) #SBATCH --partition= (Name of partition: general-compute) #SBATCH --qos= (general-compute, compbio, etc.) #SBATCH --job-name= (job name: test) #SBATCH --output= (output file name: test.out) #SBATCH --error= (error file name: test.err) #SBATCH --mail-user= (email address of user: [email protected]) #SBATCH --mail-type= (when to email: end, all) #SBATCH --mem= (Memory allocation in MB: 23000) #SBATCH --constraint= (constraint tags: IB, CPN-L5520)
module list echo "The number of nodes is $SLURM_NNODES" echo "The number of processors per node $SLURM_NTASKS_PER_NODE" echo "The number of processors is $NPROCS" echo "END"
To display this from the terminal use the command:
$ sinfo -el -o "%16P %48N %8c %8D %8m %16f" -M 'cluster' -p 'partition'
For example:
someone@rush:~$ > sinfo -el -o "%16P %48N %8c %8D %8m %16f" -M ub-hpc -p general-compute Fri Nov 5 23:55:00 1605 CLUSTER: ub-hpc PARTITION NODELIST CPUS NODES MEMORY AVAIL_FEATURES general-compute* cpn-d09-34-02 8 1 19000 IB,CPU-L5630,INTEL general-compute* cpn-f16-[03-19,23-39] 16 34 128000 IB,CPU-E5-2660,INTEL general-compute* cpn-d14-[02-04,06-20,22,26-39],cpn-d15-[01-02,04 8 107 23000 IB,CPU-L5520,INTEL general-compute* cpn-k07-01-[01-02],cpn-k07-02-[01-02],cpn-k07-03 12 356 48000 IB,CPU-E5645,INTEL general-compute* cpn-k13-27-01,cpn-k14-07-01,cpn-k14-16-02,cpn-k1 12 5 48000 CPU-E5645,INTEL general-compute* cpn-d07-04-[01-02],cpn-d07-05-[01-02],cpn-d07-06 8 123 23000 IB,CPU-L5630,INTEL general-compute* cpn-d09-14-02 8 1 23000 CPU-L5630,INTEL general-compute* cpn-d15-33,cpn-d16-[15-16] 8 3 23000 CPU-L5520,INTEL
$ sbatch (enter slurm script name here)
You should get a confirmation with a job number if everything is set up properly
For example:
someone@rush:~$ > sbatch slurmscript Submitted batch job 9999 on cluster general-compute
$ scancel –M (cluster name) (job number)
For example:
someone@rush:~$ > scancel –M ub-hpc 9999
If you would like to cancel all of your jobs
$ scancel –M (cluster name) –u (user name)
For example:
someone@rush:~$ > scancel –M ub-hpc –u someone
$ squeue –M (cluster name) –p (partition name)
Shows queued and running jobs on given cluster and partition
For example:
someone@rush:~$ > squeue –M ub-hpc –p general-compute
CLUSTER: chemistry
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
13 general-compute ph14-C2P jane_doe PD 0:00 1 (Resources)
12 general-compute XeCl someone PD 0:00 1 (Priority)
11 general-compute XeCl2 someone PD 0:00 1 (Priority)
8 general-compute Rb-III_1 mr_nonsense R 23:00:57 1 cpn-p26-26
7 general-compute Au-CA-PZ miss_leading R 23:00:57 1 cpn-p26-27
6 general-compute ph1-U jane_doe R 23:00:57 1 cpn-p26-29
5 general-compute PH2-from jane_doe R 7:44:18 1 cpn-p26-33
secure copy - opens an ssh connection to transfer files between local and remote compputers
scp [email protected]:pathname/filename filename
secure file transfer protocol
$ sftp vortex.ccr.buffalo.edu
Connected to vortex.ccr.buffalo.edu
sftp> put filename (Use to upload a file.)
sftp> get filename (Use to download a file.)
For Windows users (and Linux and Mac OS X if the command line isn't for you), use FileZilla.
- hostname: vortex.ccr.buffalo.edu
- username: UB username
- port: 22
cd projects/academic
Find your PI's directory:
ls
Initialize your usage:
kinit
- then enter password
Find out how much space available for your group:
iquota -g
Make your own directory in your PI's directory:
cd your-PI-directory
mkdir your-directory-name
To find what modules are in your environment:
module list
To find available modules for python:
module avail python
To load module:
module load python
To run on the loaded python:
python -your.py file-
To edit in vim:
vim -file name-
To save in vim: :w
To exit in vim: :q