AIRE - BNNLab/BN_Group_Wiki GitHub Wiki

1. Introduction to AIRE

AIRE is the new University of Leeds High Powered Computing (HPC) service from 2025. It replaces ARC3 and ARC4, with additional resources and capabilities for AI and machine learning.

Other than its much improved capabilities, the most important changes from [ARC] are

  • AIRE uses Slurm queuing system, which is much easier to manage.
  • AIRE has separate modules for cuda and gcc, which can be used with module load ###. The list of available module can be found with 'module avail'. Lastly, attempt to install packages which overlap with existing modules will fail.
  • The current modules are:
    • cuda
    • gcc
    • openblas
    • openmpi (relevant to ORCA!)
    • apptainer
    • miniforge (no need to install anaconda, which save space; you may want to install miniconda if the packages you need are not available through miniforge).
    • python 3.13
    • imb

The main documentation for users is at https://arcdocs.leeds.ac.uk/aire/welcome.html and all users are recommended to consult it. Only the most essential information to begin using AIRE is included here.

2. Accessing AIRE

To access AIRE, you will need to first request access to AIRE at this link. Then, you need to raise another IT request for access to rash.leeds.ac.uk, which is the bridge to connect to all university computer, using this link.

Currently, the best way to access AIRE is by ssh, VPN is not needed. To transfer files to and from AIRE, the protocol is scp instead of sftp. On Windows, MobaXterm (portable version is available to download) is the simplest tool for the job. On MacOS and Linux, the terminal is the simplest way to connect to AIRE. The instructions are below.

2.1 Connecting through a terminal

To connect, open the terminal and use the command below:

ssh <username>@aire.leeds.ac.uk -J <username>@rash.leeds.ac.uk

You will be prompted with DUO authentication, selection option 1, which will push the request to your phone with the DUO app. Approve the request and the terminal will prompt you to enter your password. This needs to be done every time you login. You may want to create an alias for the command to simplify the process.

To upload a file, open the terminal and use the command below:

scp -oProxyCommand="ssh -W %h:%p <username>@rash.leeds.ac.uk" /location/filename <username>@aire.leeds.ac.uk:/mnt/scratch/<username>/filename

To download:

scp -oProxyCommand="ssh -W %h:%p <username>@rash.leeds.ac.uk" <username>@aire.leeds.ac.uk:/mnt/scratch/<username>/filename /location/filename

The authentication is similar to with ssh. The variations to download or upload files, folders are already well documented elsewhere. The command get and put after connecting via scp seem to work just as well. When uploading and downloading folders or multiple files, it's much simpler to zip them up first and unzip at the destination.

A more convenient way to upload and download is to use the two script files (username needs to be changed): aire_up.sh and aire_down.sh. You can create your own alias inside .bashrc to make this even easier in the terminal.

2.2 Connecting through MobaXterm

Connecting through MobaXterm is more or less the same process, but with a nice graphic interface.

  • Step 1: Start a new session with the Session button in the top left corner.

Step 1 with MobaXterm

  • Step 2: Use aire.leeds.ac.uk as the host and specify your username.

Step 2 with MobaXterm

  • Step 3: Click on the Advanced SSH settings tab, and change the SSH-browser type to SCP, either enhanced or normal speed.

Step 3 with MobaXterm

  • Step 4: Now we set up the SSH gateway by clicking on the Network settings tab and then the SSH gateway (jump host) button

Step 4 with MobaXterm

  • Step 5: Input rash.leeds.ac.uk for the remote host and your username. Click OK twice to finish setting up the connection.

Step 5 with MobaXterm

  • Step 6: Double click on the connection to start it, you will be prompted for the DUO authentication, input 1 then OK. The request for approval will be sent to your phone. Approve it and then you will be prompted for your passwords. MobaXterm will remember these, so you will not have to type your password the next time (unlike with the terminal approach), but the DUO authentication will happen every time.

Step 6 with MobaXterm

2.2 Connecting via WinSCP

3. Files and folders

When you first login to AIRE, you'll be at your home folder: /users/<username>/. You have a standard allocation of 30GB. This will just be enough for 3-4 conda environments. So you have to be careful on what you use this space for. You can check your remaining quota with the command below:

quota -s

For high throughpbut calculations and molecular modelling, the jobs can be upload to /mnt/scratch/<username>, you may need to go to mnt/scratch and use the command mkdir to make your own username folder the first time. There is much more space here (TBs), but this is a scratch folder and will be cleared out regularly. So run your jobs and download the results. Do not just leave them there.

Sometimes a completely full scratch folder will not let new jobs run as there is no space to write to them. Navigate into your scratch folder with cd $SCRATCH then delete all the files with rm -r *. Afterwards navigate back to your $HOME directory.

4. Typical job setup

All users are strongly recommended to read the documentation on job types and variables here: https://arcdocs.leeds.ac.uk/aire/usage/job_type.html

4.1 A standard job

A job on AIRE will, like on ARC4, consists of a .sh file for queuing, and the job files (Gaussian/ORCA/MOPAC input file or a python file). The content of a typical .sh file is below:

#!/bin/bash -l              
#SBATCH --job-name=ALTRCA_5_cpu			# specify job name, which will be used as the filename for output file
#SBATCH --output=%x.out			        # specify output filename, all output to the terminal will be saved here
#SBATCH -p himem 				# request high memory node (only needed for CCSD DFT calculations)
#SBATCH --cpus-per-task=8			# ask for 8 CPUs
#SBATCH --mem-per-cpu=16GB			# ask for 16GB per CPU
#SBATCH -t 48:00:00				# set maximum length of the job
#SBATCH --mail-type=begin			# ask Slurm to send an email when the job starts
#SBATCH --mail-type=end 			# ask Slurm to send an email when the job ends
#SBATCH [email protected]	        # specify the email address to send to

module load openmpi				# load the modules needed for the job
python /mnt/scratch/chmbnn/ALTRCA_5_cpu.py      # Run job, may need conda activate environment

The output file .out will capture all the output to the terminal, if the job is run directly with out the queue system. The default name is slurm-<job_number>.out, which is not helpful afterward. So the first 3 lines are very important to get right.

Once you have the .sh and job files set up, you can queue the job with:

sbatch job.sh

To check your job status, use the command:

squeue -u <username>

This will only list your jobs. The job ID number obtained with this is useful if you need to kill off a job. If that's the case, use the command:

scancel <job_number>

4.2 A job using GPUs

To request GPUs, the .sh file can be used as shown below:

#!/bin/bash -l              
#SBATCH --job-name=ALTRCA_5
#SBATCH --output=%x.out
#SBATCH -p gpu --gres=gpu:2                           # Requesting 2 GPUs
#SBATCH --mem=128GB
#SBATCH -t 48:00:00
#SBATCH --mail-type=BEGIN,END
#SBATCH [email protected]

module load cuda                                      # Loading cuda is essential if you plan to use the GPUs
python /mnt/scratch/chmbnn/ALTRCA_5.py                # Run job
⚠️ **GitHub.com Fallback** ⚠️