Installing on Digital Research Alliance of Canada's infrastructure - McIntosh-Lab/tvb-ukbb GitHub Wiki

This page contains instructions for installing the TVB-UKBB pipeline for users using the Advanced Research Computing (ARC) infrastructure from the Digital Research Alliance of Canada (formerly Compute Canada). Note that you will likely need to customize the pipeline to your dataset.

Notes

All of the dependencies we need are provided by ARC's module system. To load Singularity for installation, simply type:

module load apptainer

and Singularity will be added to your environment. We'll be using this later in the installation for the dependencies not included in the Singularity container.

Installation Steps

Download the TVB-UKBB Singularity Container

There are currently two versions of the container. If you are using the Cam-CAN_dev_CPU or ADNI3_dev_CPU branches, you can simply download the Singularity container from here to your desired location.

wget 'https://public.boxcloud.com/d/1/b1!K3QLzPHRpJdCqRrpMfUFzjEfIZKlgkzrfEansIMWCESHYZM5XNgAIr-VhAsXKIXHYPC5qFvhOlW34Dw9s-uvD8qTR_06oZMpKN9zbCXz1nRK0nKT5eMLiMgt3h_YAh-8gIXgjFDGwzDm3a0AZkUqJd-V0uAuoRU6kgp-LoUsG-WT1z11fMehUOnvDd_Tqr8lLKkUtEACCxuPBzkq21rm2lWwdcurpptpjgD1aggJC2zC4j1ljNylF9VbY-AagS4ws-_69rQJfmWrfUflqNRReiVLf9djgH4k8H-e1Zwazn5T7YiqIhLYPEJquC28Xnc9D3lr6Tg5YzQGdlK60OGI69B84DjJp6tNk4tNl9o7YPZJALdvjI0aLusZEBk2O6X6mVCyKaXaNU99v7VJyPj--X9aASbJPRrBUV2vwhElF1mVzZAL3IX_7HWt9jHvopYgx87LjnX8r00IeZxhnnQVgYMOasiKQp3JyD4IqRJtrd3LwaPqNfPJqSZX6sl0MNsul8rWcwkyhW3llIVqPGvcZY7xgv7q1u194xqifPOa-ueJAUXOxFfXLYqB8rQX6rgcG57TxUom7pRV0vI6hjCuNHEW0FHSUiB1P-NKnD1QkzB73Vl0Lbc33oRN1149stEoFzHw2eOG3gLPnJIYLyvjTcJHBQjzfgxheEp4g-Tde37Dn9fLTALHOFzhyBxPTDNHt-igAZrwI2FJeGFtU1_3QOtIn66q_l8R81bcfwqD_QHsxG1CLumTENjnZUUMYJA3U-DVNTZ5OTk_QwC2WZWglibGXppBwrC1toSwf0TtL5BTzEsB2zk_Tlct6pJXjFhek1_4pEQQHjZt4Xb-OPEWu57IMOIZs19cc_d9wsXOo2-tX6m-ypB6gYHNNnWF8dk-LwWWRc2MprPvgIc5xrnFcJyTnprJ8XAiBg7F7rgOCvNhUxmzpmzV1awAK09J8w9kTLS2UnnnMr-PuNUePbSt3upWJs0u3AMkkqLhR_-0FUxKPoQC6IghKpdL2NJ9EnMyzgC09uHTW9_OkplCbDr0zYCKiVZkscgoqDh_rra6co0A51L6LbAU8e2iyBk3oIVqE4E2SezqBanyWqS1_GzCDO9N95W_1gmQgZtwmppWk08TFL8E9_-DCejIInWAOxO9MqSXQDxWSkYJmb1ZBxthnF3txn035psQLqaURYVEOsXFzkT8YBIXkKZXADdKYnnAQ_r39ST-EV5s0dKNsPNTM3U3rpg-XeQKLWVbNfNmYf3TvZCawnXj017lb9DwjbjoY2GR4DV5zvXkiTNzRl6DlDA2uZd4DUzEacfpBE9eQA../download'

Otherwise, you should also pull the Singularity container from the Sylabs Cloud library to wherever convenient. You can check out the webpage for it here.

apptainer remote add --no-login SylabsCloud cloud.sycloud.io
apptainer remote use SylabsCloud
apptainer pull library://noahfl/tvb/tvb-ukbb:latest

Install git-lfs

Install git-lfs so large files from the repository can be downloaded properly. This only needs to be run once for your user account:

module load git-lfs
git lfs install

module load git-lfs should be run every time you start a new session and want to pull changes from the repository.

Download the TVB-UKBB pipeline

Note: the container doesn't have the repository installed into it since the container is read-only, so you have to have your own copy of the pipeline installed. Clone the git repository to wherever convenient:

git clone https://github.com/McIntosh-Lab/tvb-ukbb.git

We release bugfixes and introduce new features on a rolling basis. If you have been working on a local version or a fork of the pipeline, please be sure to pull the most recent version of the pipeline before you run your data.

Modify init_vars

The environment variables in init_vars are set to the ARC locations for AFNI, FreeSurfer, and ANTS.

Other variables you might want change include:

SUBJECT_AGE_LIST

AGE_SPECIFIC_TEMPLATE_LIST

PARC_IMG

PARC_LUT

SUSCEPT_ROIS

SUSCEPT_PARC_IMG

TRAINING_FILE

Configure FreeSurfer

Note: installation steps for McIntosh Lab members and general users are different. See below.

General Users

See the the Digital Research Alliance of Canada's Wiki entry for FreeSurfer for information on how to configure a single-user or shared installation for FreeSurfer.

Note: the .lua file users need to use the shared installation is generated by the steps from the the Digital Research Alliance of Canada's Wiki entry for FreeSurfer's shared installation instructions. Be sure to put this file in a directory that's readable by all users who will be using your shared installation so they can copy it. Read the below steps for the McIntosh Lab installation as an example for how to execute this on a Compute Canada node.

McIntosh Lab

FreeSurfer has a shared installation for McIntosh Lab members. If you're not a McIntosh lab member but want to use FreeSurfer 7.1 on ARC, see the above links in General Users. If you're a McIntosh Lab member, just follow these easy instructions to configure FreeSurfer for your account. In your home directory, copy and paste these commands to run:

mkdir -p /home/$USER/.local/easybuild/modules/2020/Core/freesurfer
cp /project/def-rmcintos/noahfl/7.1.0.lua /home/$USER/.local/easybuild/modules/2020/Core/freesurfer/

And that's it.

Next Steps

Visit this page for instructions for running a subject on ARC.

Details and instructions for customizing the pipeline to acquisitions can be found in Customizing the Pipeline.