3. CONN Preprocessing on Discovery - mind-lab-bos/MBI_Project_MRI_Analysis GitHub Wiki

Move files to Scratch

Generate a directory in your scratch workspace named 'RAW_BIDS'. Copy new participant data from NUBIC/MCI_Study/raw_bids folder into this directory. Only add raw bids for new subjects that are being added to the project.

It has been shown that we cannot run more than 6 subjects at a time. While unlikely, if you have more than 6 new subjects, please add the first 6 to the RAW_BIDS folder first. After the following steps has been done, repeat for the next 6 subjects.

Copy the contents of /work/mindlab/Projects/GammaMBI/CONN_Project into your scratch workspace, including the Current_Subs folder, the MBI_Gamma_Conn folder, and the MBI_Gamma_Conn.mat, MBI_Gamma_Generate_Batch.m, MBI_Gamma_Batch_submit.sh, and MBI_GAMMA_CONN_Submit.m files. You could either click the copy and paste buttons or use the following code in terminal.

rsync -rltDvu /work/mindlab/Projects/GammaMBI/CONN_Project/ /scratch/<yourusername>/

Important: Edit file path in first line of MBI_GAMMA_CONN_Submit.m to match the path to your scratch workspace (/scratch/<yourusername>).

Please check each participant's fMRI data in the /RAW_BIDS/sub-*/func folder. Each subject should have 2 runs for facename, 1 run for musbid, and 1 run for rest. If they have more than that, check the notes on the runlog to see what happened during the MRI session and delete all the unwanted scans. Most likely, the last run should be the one to keep, but it is always good practice to check before deleting.

Run Preprocessing Batch

SSH into your scratch workspace. (Go to your scratch workspace on OOD, and click on "Open in Terminal", type in your password as prompted).

Go to compute node by pasting the following code on Terminal: srun --partition=short --nodes=1 --cpus-per-task=1 --pty /bin/bash

Paste the following code into command line: sbatch MBI_Gamma_Batch_submit.sh

Use scontrol show jobid -d "job number" or squeue -u xi.wu or click on Jobs -> Active Jobs on OOD to check on job status, or check matching output file (slurm-<jobnumber>.out) in your scratch workspace. This job will take several hours (23h max). It is OK to log out of OOD or let your computer go to sleep while the job is running. After the job is completed, the .out file should end like this:

Screen Shot 2023-11-01 at 10 53 26 AM

Update Project folder in /work/mindlab

Once the job completes, check the /RAW_BIDS/sub-*/func/ folder. It should have more than what you started with (should have 78 in total). Move the new participants' preprocessed data from the RAW_BIDS folder in your scratch space into the Current_Subs folder in your scratch. Then, start an interactive Matlab session, checking boxes to use GPU and Gurobi, and allocating 10 GB of memory.

Change the working directory to /scratch/yourusername on the top. Then, set path and open conn toolbox, using the following code:

addpath('/work/mindlab/Programs/conn')

addpath('/work/mindlab/Programs/spm12')

conn

In conn, open the project file (MBI_Gamma_Conn.mat) from your scratch workspace. If you are prompted to select a location for a specific file in a pop-up window. Locate this within the Current_Subs folder of your scratch, select the appropriate file of that exact name, and click "Open". Do not end this interactive session.

Ensure that existing participants are present and new ones have been added. You could do that by clicking on each tab on the left and hovering your cursor above each scan on the right. The file names for the scan in each of the tabs should be as follows:

  • Structural (T1): wc0c*
  • Functional: swau* (e.g. swausub-094_task-musbid_bold.nii; session 1 = rest, session 2 = mus-bid, session 3 = face name study (run 1), session 4 = face name test (run 2))
  • ROIs: wc1c* (Grey Matter), wc2c* (White Matter), wc3c (CSF)
  • Covariates 1st level: rp_*
  • Covariates 2nd level: no regular names, but make sure each sub has a distinct pattern

Once the project is confirmed to be accurate, run the following codes on Terminal to sync new subjects' data to the /work/mindlab/ directory.

rsync -rltDvu /scratch/xi.wu/MBI_Gamma_Conn.mat /work/mindlab/Projects/GammaMBI/CONN_Project/

rsync -rltDvu --ignore-existing /scratch/xi.wu/MBI_Gamma_Conn/ /work/mindlab/Projects/GammaMBI/CONN_Project/MBI_Gamma_Conn/

After moving RAW_BIDS data to the Current_Subs folder in your scratch manually, rsync -rltDvu --ignore-existing /scratch/xi.wu/Current_Subs/ /work/mindlab/Projects/GammaMBI/CONN_Project/Current_Subs/

Returning to your interactive Matlab session, open the updated conn project .mat file located in /work/mindlab/Projects/GammaMBI/CONN_Project/. You will likely be prompted to select a location for a specific file in a pop-up window. Locate this within /work/mindlab/Projects/GammaMBI/CONN_Project/Current_Subs/, and select the appropriate file of that exact name. Additional filepaths should then be updated automatically. If you can't find the file in the pop-up window, that mostly likely means preprocessing wasn't done correctly. If that's the case, you should look at the slurm-*.out file to troubleshoot. Ask Alex or Ben if you need help.

Save and exit the project, terminate the interactive session, and the project should now be fully updated.

If you have more than 5 subjects, repeat the above steps for new subjects.

⚠️ **GitHub.com Fallback** ⚠️