How to access aspera onsite - TGAC/knowledge_base GitHub Wiki

A quickly thrown together recipe for grabbing downloads using aspera.

Ideally, large downloads would be handled via globus. Less ideally, we can use wget/curl/aria2 to retrieve files using the HPC download partition (if they are hosted on an allowed address) or by mounting the HPC storage directly and tunneling the download through our local machine directly onto the HPC storage.

In this case, neither approach was possible. The Chinese National Genomics Data Center (NGDC) did not offer a globus option, and wget was returning extremely slow transfer speeds (10-50KB/s). However, they did offer an aspera option.

Aspera is a proprietary system for large, fast data transfers which is no longer supported on the NRP. However, we can use a dedicated machine in EI for the purposes of running this software if there is no alternative. Here are the steps to get it running:

  1. Log into the remote machine (149.155.215.82) with your NBI credentials
ssh <USERNAME>@149.155.215.82

  1. Request sudo access on the remote machine from a data champion (or MA)

  2. Mount the HPC storage

mkdir ~/hpc_projects/ ~/hpc_scratch/ 
sudo mount.cifs //ei-hpc-data.nbi.ac.uk/projects/ ~/hpc_projects/ -o domain=nr4,user=aylingm,uid=$(id -u),vers=3.0
sudo mount.cifs //ei-hpc-data.nbi.ac.uk/projects-scratch/ ~/hpc_scratch/ -o domain=nr4,user=aylingm,uid=$(id -u),vers=3.0
  1. Locally install asperaconnect (3.9.6 has been confirmed to work, more recent versions have failed), including any ssh keys required by the host
#The installation file is in /srv/ on the machine
cp /srv/ibm-aspera-cli-3.9.6.1467.159c5b1-linux-64-release.sh ~/

#Then extract and install the asperaconnect
./ibm-aspera-connect_4.2.14.855-HEAD_linux_x86_64.sh

#Add any required ssh keys to ~/.ssh/

#Add aspera path to $PATH
export PATH="/usr/users/ga002/<USER>/.aspera/connect/bin/:$PATH"
  1. Start a persistent session (long downloads will require this step)
#Check that tmux is available
which tmux

#Start a named session
tmux new-session -s aspera

#To detach from the current session hold 'ctrl+B' and press 'D'

#To check your session is still active
aylingm@N82102:~$ tmux ls
aspera: 1 windows (created Mon Jul  7 10:58:15 2025)

#Attach an existing session
tmux attach-session -t aspera

#Kill the current session
exit
  1. Retrieve the files (example here is from the NGDC).
target_path=<DESTINATION>
ascp -v -P33001  -i .ssh/aspera01.openssh -QT -l100m -k1 -d [email protected]:gsa4/CRA013551 ${target_path}
⚠️ **GitHub.com Fallback** ⚠️