Our lab uses the Peloton2 cluster for jobs with big computational demands. We have two nodes on Peloton (head node scom-marr). Each node has 48 cores/96 threads and 256GB of RAM. This node is dedicated for lab use and we have first priority when submitting jobs to this node.

There are other nodes on Peloton, and we can access them if there is a need (we might have to wait for other people's jobs to finish), see here for more information on job queue priority.

Peloton is a High-Performance Computing (HPC) environment at UC Davis. This guide does not detail how to use Peloton (in specific) or HPC (in general). The best starting point would be taking some of the classes in the DSI or Data Lab. Instead, this provides some quick guidelines on how to schedule jobs on Peloton. It uses FSL jobs as an example, but other jobs using different programs (e.g., Matlab, R, Python) are also possible.

But, before you begin, you will need to make an account on Peloton. To do that, follow these steps:

1. In a browser, navigate to the Account Request Form and fill in the required information. Select Letters and Sciences as your sponsor.

2. You will need to generate an SSH key (if you haven't done this already). Click here to learn how to generate a SSH key pair.

3. SUPER IMPORTANT: Upload your PUBLIC key (and NOT your private key). Running ssh-keygen (discussed in "How to generate a key pair on your machine" FAQ heading) will generate two files the private key (usually named identity or id_rsa or id_dsa) in your your home in the .ssh directory (this is a hidden directory, you may need to enable "show hidden directories" to find it). The file with the .pub extension is your public key and the one without an extension is your private key.

4. Save the confirmation page.

5. For more guidance on using the HPC, see the HPC Core Facility Cluster Documentation.

If you are having trouble, email [email protected]. There is an #hpc channel on the UC Davis Slack. This is a great channel and you should be on it.

Once your Peloton account has been created and approved, you can ssh into Peloton once you are logged into the VPN.

Once logged into the VPN, use the following command:

$ ssh [email protected]

Alternatively, you can add a hostname to /etc/hosts by copying the following information to a new line at the bottom of the hosts file:

peloton.hpc.ucdavis.edu	marr

Once you've down that, you can ssh into Peloton as follows

$ ssh username@marr

Our lab's account name is rwhuskeygrp.

When you connect to Peloton, you'll land in your home directory. Data for our lab live in /group/rwhuskeygrp. To access simply:

$ cd /group/rwhuskeygrp

You may need to specify our account name with the -A [ACCOUNT] name flag for sbatch and srun, or by modifying the corresponding line in your job submission scripts.

If you do an rsync to/from Peloton, you need to correctly specify your syntax. This changes from time to time. If your syntax is not working, check the latest HPC documentation. Your rsync command should look something like:

$ rsync -aP -e ssh path/to/directory/i/want/copied/to/peloton [email protected]:/group/rwhuskeygrp

This will transfer data from your local workstation to the lab directory on Peloton.

Peloton only supports one SSH Key. If you want to ssh into Peloton from multiple systems, you need to use the same keypair. To do this, follow these steps:

1. Copy id_rsa from your old machine onto a thumb drive or something like that (don't send over the internet)

2. On your new machine, paste id_rsa into ~/.ssh/

3. Change the permissions

   $ chmod 700 ~/.ssh

   $ chmod 600 ~/.ssh/id_rsa

4. Add the private key to the list maintained by ssh-agent:

   $ ssh-add

5. That should do the trick!

Having trouble? Read the How to move and copy SSH keys FAQ

$ sbatch -p high2 script.sh

1. How to use SLURM
2. Intro to HPC
3. HPC FAQ