Word to the wise: In case some terms/jargon are not clear, please refer to the glossary

1. Prepare:
   1. Check data for movement correction
   2. Create folder for movement correction related files
   3. Configure background flags
2. Create configuration files for movement correction experiments
3. Run movement correction experiments
4. Examine results of movement correction experiments
   1. Examine Shift correction report
   2. Examine Shift+Movement correction report
   3. Examine Crispness scores
   4. Jointly load and examine data before and after correction
5. Decide on the best movement correction parameters and finalize them

Load data of one or more measurements ▶️ Transfer data to ILTIS ▶️ Check if movement artifacts exist within and between measurements. Additionally, try to gauge the kind of movement artifacts present, for example, translations, rotations, slow/fast movements, rigid/non-rigid movements

Create a folder for storing files related to movement correction. It is recommended that this folder is within the "STG_MotherOfAllFolders" folder. Set the flag STG_MoveCorrPath in your project YAML file to indicate this folder (absolute path or path of the folder relative to "STG_MotherOfAllFolders"). This folder will be referred to as the "MoveCorr" folder in the rest of this guide.

Shift correction tries to correct movements between measurements and does this using a background image created from each measurement. This background image is created in one of two ways:

1. If atleast one stimulus is specified for this animal in the measurement list, then CTV303.
2. If no stimulus is specified in the measurement list file, CTV301 is used

If stimuli are specified for the animals in question in the measurement list, then make sure that the flags LE_StartBackground and LE_PrestimEndBackground are properly set.

For each animal that requires movement correction create a movement-correction configuration file with the name "<animal_tag>.yml" in the "MoveCorr" folder created in Step 2. Here is a template for the configuration and explanation follows after the template.

# this will contain the name of the experiment to use finally. To be added after running 
# movement-correction experiments and examining their results
to_use: none 

# Specification of Experiment 1
exp1: # arbitrary experiment name
    shift_corr: # section for specifying plugin and parameters for shift correction
        plugin: <plugin name> 
        plugin_params: 
            <param1>: <value1>
            <param2>: <value2>
            ..................
    move_corr: # section for specifying plugin and parameters for movement correction
        plugin: <plugin name> # possibly different from that for shift correction
        plugin_params: 
            <param3>: <value3> # possibly different from those for shift correction
            <param3>: <value4>
            ..................

# Specification of Experiment 2
exp2: # possibly different plugins and plugin params
........
........
........
.......

1. Make sure you have created a movement-correction-configuration-file for each animal that needs movement correction in the "MoveCorr" folder.
2. A templated script for running movement correction experiments is available at "VIEWTemplates/run_move_corr_experiments_template.py" of the repository. Copy/download this file to your "Progs" folder and remove the suffix "template" from the filename.
3. Open the file in a text editor.
4. Specify the path to your project YAML file in the variable "yml_file".
5. Specify the list of animals that require movement correction in the variable "animals".
6. Open a terminal/Anaconda Powershell and activate the environment created for view (Step 4 here).
7. Run the script. You can drag and drop the script from your file explorer to the terminal/powershell to get its full path.

python <full path of the script>

The script above creates one folder per animal in the "MoveCorr" folder and within each of these, a folder called "Reports" to store the results of movement correction experiments. The following is the recommended order of examining these results.

This is a tapestry-style HTML, is named "shift_corr_report.html" and is created in the "Reports" folder for each animal. If an animal has N measurements, and M experiments were specified for this animal in its movement-correction-configuration-file, then this report is a tabular arrangement of images/plots with (N+2) rows and (M+2) columns. Here is a guide to help you understand this report:

• The first two panels of the first column are empty by design.
• The first panel of the last column is empty by design.

• All columns except the last column contain images.
• The method by which the image was calculated is indicated on the bottom to the right of the image.
• The data from which an image was calculated is encoded in the label below the image. This label contains several tags separated by underscores.
• All tags except the last one contain metadata about the measurement: animal, measu and any other metadata specified in the flag LE_labelColumns.
• The last tag is either "Original" or the name of an experiment. This indicates the data from which the image was derived.
• Note that the first column contains images from original data. Columns other than the first and last column each contain images calculated from data resulting from the application of shift correction as specified by an experiment.

• Non-empty panels in the last column contain plots. The quantity on the Y axis in the correlation/similarity of a frame with the mean image across all frames of the measurement in question. A dip in this plot indicates a movement artifact.
• The plot on the second row of the last column has "Measu" on the X axis. For this plot, correlation is calculated using background images of the measurements. Hence, this plot can be used to judge the strength of movement artifacts between measurements.
• Plots on all other rows of the last column has "Frame number" on the X axis. Hence, this plot can be used to judge the strength of movement artifacts within each measurement.
• Plots in row three onwards can be used to understand the effect of shift correction on frame-wise correlation. This is relevant in relation to similar plots in Shift+Movement Correction reports.

• The first row contains the initial template used for shift correction.
• This is presented here as it is automatically chosen and can strongly affect shift correction.
• The initial template is chosen to be the background image of the measurement with highest average correlation, when calculated across frames within one measurement.

• The second row contains the final template of the shift-correction process for each movement-correction experiment
• This is presented here as it will be used as the initial template for movement-correction that follows shift-correction.

This is a tapestry-style HTML, is named "shift_move_corr_report.html" and is created in the "Reports" folder for each animal. If an animal has N measurements, and M experiments were specified for this animal in its movement-correction-configuration-file, then this report is a tabular arrangement of images/plots with (N+1) rows and (M+2) columns. Here is a guide to help you understand this report:

• The first and last panels of the first column are empty by design.

• All columns except the last column contain images.
• The method by which the image was calculated is indicated on the bottom to the right of the image.
• The data from which an image was calculated is encoded in the label below the image. This label contains several tags separated by underscores.
• All tags except the last one contain metadata about the measurement: animal, measu and any other metadata specified in the flag LE_labelColumns.
• The last tag is either "Original" or the name of an experiment. This indicates the data from which the image was derived.
• Note that the first column contains images from original data. Columns other than the first and last column each contain images calculated from data resulting from the application of shift and movement correction as specified by an experiment.
• More crisp the image is, i.e., more sharp the borders between brightly lit spots and dark areas are, weaker is the movement artifact present. This should also be reflected in the plots in the last column.

• Non-empty panels in the last column contain plots. The quantity on the Y axis in the correlation/similarity of a frame with the mean image across all frames of the measurement in question. A dip in this plot indicates a movement artifact.
• These plots can be used to judge the strength of movement artifacts within each measurement and the extent to which they are corrected by each experiment.

• The first row contains initial templates used for movement correction.
• They are also the final template of shift correction.
• They have a strong effect on the output of movement correction.

Running movement correction experiments also creates a table of crispness scores in the "Reports" folder, which can be used to compare the performance for experiments. Crispness, or the sharpness of borders between brightly lit spots and surrounding dark areas, can be qualitatively assessed by visually inspecting mean images in Shift and Shift+Move Reports. This is quantified using a Crispness metric (see "Crispness and focus measures" here).

Running movement correction experiments also creates a measurement list file in the "Reports" folder. This file has the same file name as the original measurement list file for the animal in question. It can be used to jointly load original data as well and as movement corrected data into VIEW:

1. Select the project YAML file that was specified when running movement correction experiments
2. Select the newly created measurement list in the "Reports" folder and select data based on the column "Label". Original data have labels as in the original measurement list file, while shift and shift+move corrected data have suffixes indicating the extent of correction and the experiment name.
3. Transfer data to ILTIS for detailed inspection and comparison.

If the results of none of the experiments are acceptable, choose a new sets of parameters and run more experiments. If an experiment/set of parameters can be deemed to be the most satisfactory among all experiments, movement correction can be finalized as follows:

1. For each animal, open the movement-correction-configuration-file in a text editor and enter the name of the most satisfactory experiment next to the key "to_use: ".
2. A templated script for finalizing movement correction parameters is available at "VIEWTemplates/finalize_move_corr_template.py" of the repository. Copy this template file to your "Progs" folder, remove "template" from the name.
3. Enter the YAML file used when running movement correction experiments in the variable "yml_file".
4. Enter the tags of all animals that you want to finalize in the variable "animals". Make sure that Step 1 above has been done for all these animals.
5. Open a terminal/Anaconda Powershell and activate the environment created for view (Step 4 here).
6. Run the script. You can drag and drop the script from your file explorer to the terminal/powershell to get its full path.

python <full path of the script>

Finalization step does the following:

1. Creates a new folder called "Corrected Data" next to your "Reports" folder and copies all movement-corrected files into this folder.
2. Backs up old measurement list file. Adds measurements representing movement-corrected data to the original measurement list file with modified labels. Entries in the column "Analyze" are set to 2 for original uncorrected data and to 3 for movement-corrected data.