Interactive Job for File Transfer Using rsync

This guide walks through setting up an interactive job to transfer files using rsync in a detached screen session, allowing the process to run independently without interruption.

Steps

Step 1: Submit an Interactive Job with sbatch

To create a job that remains active for 3 days, use the following command. This keeps the session open to allow rsync to complete without timing out prematurely:

sbatch --part=ncpu --time=3-00:00:00 --ntasks=6 --mem=32G <(echo -e '#!/bin/bash\nsleep 3d')

Step 2: Start a screen Session

To keep the rsync process running independently, start a screen session:

screen -S copying

Step 3: Login to node

ssh cnΧΥΖ

Step 4: Run rsync command with Timeout

rsync -vr --progress --partial --append --timeout=120 /nemo/project/proj-tracerx-lung/tracerx/_RELEASE/my_folder [email protected]:/SAN/my_new_folder/