2. Dicom to Bids on Discovery - mind-lab-bos/MBI_Project_MRI_Analysis GitHub Wiki

If you are doing this for the first time, please see this page to install and activate miniconda before getting started with the instructions below. You only have to install it the first time you do the bids conversion.

Uploading the zip folder to the lab computer

  1. Put the Thing2/GRACE USB onto the lab computer and copy the latest participant's zipped folder to /Volumes/Promise_Pegasus/MCI_Study/Raw_data_zipped/.

  2. Unzip the folder by double clicking it, and move the unzipped participant folder to /Volumes/Promise_Pegasus/MCI_Study/dicom/ folder.

  3. You will notice that the participant's folder has the file structure below:

MCI_Study/dicom/MCIMBI_210719BFOR1/Mcimbi_210719Bfor1/Loui_Goutama_1009_Mcimbi - 1 The space between the "Mcimbi" and the "-" and the "1" are problematic for the dicom to bids conversion on discovery, so please delete those two spaces so the folder structure looks like this:

MCI_Study/dicom/MCIMBI_210719BFOR1/Mcimbi_210719Bfor1/Loui_Goutama_1009_Mcimbi-1

  1. Open terminal. Use the rsync command below to rsync the dicom folder to your scratch workspace on Discovery.

rsync -rltDvu --ignore-existing /Volume/Promise_Pegasus/MCI_Study/dicom/ <yourusername>@xfer.discovery.neu.edu:/scratch/<yourusername>/MCI_Study/dicom/

(More rsync commands can be found in /Volumes/Promise_Pegasus/rsync_backup_codes.txt.)

Dicom to Bids on Discovery

!!!(skip the following steps if you plan to preprocessing data using fmriprep and go to page: Fmriprep preprocessing)

  1. Open a terminal on discovery and go to the compute node by using the command below:

srun --partition=short --nodes=1 --cpus-per-task=1 --pty /bin/bash

  1. rsync the MCI_Study folder from /work/mindlab/NUBIC/ to scratch (/scratch/<yourusername>). This will take some time as the files that are copying are quite large, so please plan for that, especially if you are doing this for the first time. This step can take about 2.5 hours. You can use the command below for this:

rsync -rltDvu --exclude 'ignore' /work/mindlab/NUBIC/MCI_Study /scratch/<yourusername>/

You will have to do this every time you do this conversion in order to ensure that you are working with the most up to date data.

  1. In the file explorer, find the makebids_withfacename_discovery.py script in the MCI_Study folder on your scratch. Click on the 3 dots to the right and then choose "Edit". Scroll to line 48. This line defines the base path, so make sure you change that path to /scratch/<yourusername>/. You might have to do this every time you run the script.

  2. In the discovery File Explorer, change the name of the dicom folder you would like to convert to include a "z" at the beginning, so 210719BFOR1 would become z210719BFOR1. Make sure that only one folder that has not been put through the dicom to bids script is in the dicom folder at a time. If you are trying to process two participants at a time, remove the second one from the dicom folder temporarily

  3. Delete all files containing ant at the beginning EXCEPT the following:

anatT1w_acqmprageNav4e_RMS_XX

antaT1w_acqmprageNav4e_RMS_XX

  1. Now we are ready to run the script!

Running the Script (replace "xi.wu" with your own username)

  1. Load anaconda and activate your miniconda environment

module load anaconda3/2022.01

source activate my_dcm2bids_env

  1. Load python

module load python/3.8.1

  1. cd into your scratch folder

cd /scratch/xi.wu/

  1. Run the script.

/scratch/xi.wu/MCI_Study/makebids_withfacename_discovery.py --project "MCI_Study" --all

  1. Check the data. By running the script, each participant will have a folder made in the raw_bids folder that has been assigned a numeric ID. Make sure that within the participants folder (raw_bids/sub-040), you see the following folders: fmap, func, anat, dwi.

  2. Add the participant ID to the README.txt file in /scratch/<yourusername>/MCI_Study/raw_bids/. In the first column, assign the same number to each participant (i.e., BFOR1 and BFOR3 should both be 23). In the third column, ensure to include the numeric ID the participant was given in the bids conversion. Note: Please add a dash in front of the numeric ID if this is a post intervention dataset. Please also add information about the participant's ID, sex, age, and type (control, MCI, AD, etc.) to the participant.tsv in the /raw_bids folder.

  3. Using the File Explorer, remove the "z" at the beginning of the participant's data in the dicom folder.

Repeat the above "Dicom to Bids on Discovery" and "Running the Script" steps (no need to repeat steps 1-3 for both sections if you did not exit or log out) until you finish converting all new participants' data (one by one). After that, continue to the following steps.

  1. In terminal, rsync the raw bids folder in your scratch with the one in the /work/mindlab/ location.

rsync -rltDvu /scratch/xi.wu/MCI_Study/raw_bids /work/mindlab/NUBIC/MCI_Study/

  1. Run the following command to change the permissions on the most recent participant's folder to ensure everyone in /work/mindlab/ can access the folder:

chmod -R 770 /work/mindlab/NUBIC/MCI_Study/raw_bids/<latestparticipant>

In the long run, Raw_data_zipped and dicom should be stored on Promise_Pegasus, and raw_bids onward should be stored on the /work/mindlab/ directory on Discovery. You can keep the existing files on your scratch for ease of future preprocessing, but do not rely on your scratch to store data since it will be purged once in a while and your data will be lost.

Make sure no more than one person preprocesses each participant's data. Make sure one participant's preprocessing is finished and synced before the next one starts. Coordinate before you start. (It would be easiest to manage if one person takes care of all the preprocessing.)