Rclone is a command line program that allows users to connect their server to a cloud service, such as Google Drive or Dropbox. As mentioned in our Data Management documentation, UTK provides all users with an unlimited Google Drive; this will allow you to store projects in the cloud once they have been completed. This page will serve as a guide on how to use Rclone effectively.

Rclone is available through Spack, so to activate Rclone in your current session run spack load rclone. You will need to rerun this everytime you login to a new session; however, you can edit your ~/.bashrc to include the command so that it is run everytime you login.

UPDATE 02/15/23 - The Spack version of Rclone is out of date, and you are not able to configure a drive with Google Drives with this version of Rclone. From now on, use a local installation in /pickett_centaur/software/rclone/ or /sphinx_local/software/rclone/. All following commands will remain the same, though you will need to provide the full path to the rclone software.

Rclone is configured by running the command rclone config. When you first run this command, because there are no remotes at present, you will be guided through setting up a remote. First, select the n option for the following:

No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> 

You are then prompted to give your remote a name. I suggest using something you will easily remember, such as your UTK username or a variation of it.

Next, you are asked to specify what type of Storage you will be configuring. This guide will cover configuring Google Drive, so type 13.

The next two options, client_id and client_secret, can be left blank. For scope, select 1 to allow access to all files in your drive. Once again, leave the root_folder_id and service_account_file options blank. Then, you will be asked if you want to enter advanced config. Type n here.

The next question is in regards to auto_config, which would allow you to automatically connect to your account from the server. We don't have that option, so select n. This gives you a link; copy this link and paste it in a web-browser. Login using your UTK credentials and grant Rclone access to your Google Drive; once complete, you are given a code to paste into the command line to continue configuring Rclone.

If you plan to use a team drive, select y for the next option, otherwise select n. Finally, select y to finish configuration, and then select q to return to your session.

Once you have established your first remote, using rclone listremote will print the name of your remote. To list the current contents of your remote, run rclone ls <remotename>:. This works like the ls function on command line and lists the current files and directories located in your Google Drive account. If you need to look at what has been shared with you, add the --drive-shared-with-me tag to rclone. Note that this will only show items shared with you.

To copy files from your drive to your current location, or vice versa, run the copy command:

rclone copy <remotename>:example/directory/file.fasta .
rclone copy --drive-shared-with-me <remotename>:project/raw_data/sample1.fastq .
rclone copy /sphinx_local/projects/example/analysis/3_STAR/example.bam <remotename>:example/results/

You can also completely sync a directory on the server to a backup on your drive:

rclone sync /sphinx_local/projects/example <remotename>:example_backup