Preprocessing From The Command Line - nimwegenLab/moma GitHub Wiki

The following instructions describe how to automate the preprocessing of raw microscopy data for MoMA from the command line. As it uses the tools introduced in the previous sections (Image Registration and Growth Channel Separation) and produces the same output as manual preprocessing, it is probably easier to start by analysing a few simple datasets by hand.

For a given experiment, each position is analysed independently. At first, an ImageJ-macro is used to import the raw data, register all frames with HyperStackReg. In a second step, MMPreprocess is called to produce the splitted files which can be analysed using MoMA. For standardisation issues, it is recommended to create three subfolders inside the raw image data directory:

  • "1_registered": to store the result of the image registration step,
  • "2_split": with subfolders containing the individual growth channel image sequences
  • "3_analysed": with the same subfolders containing the analysis results files

A bash script is executed with the relevant arguments to run all preprocessing steps.

Installation

  • Follow the instructions here: Installation guide, also follow the instructions on that page for installation of the command line tools
  • Download the ImageJ macro moma_pre_cli.ijm.
  • Download the bash script for automated preprocessing moma_pre_cli.
  • We recommend to put all jar files, ImageJ macros and bash scripts in the same folder and make this folder part of the systems PATH variable. Furthermore, the bash script need to be executable. Use chmod u+x moma_pre_cli to make the script executable.

Further prerequisites

  • A link named fiji, pointing to the ImageJ executable (e.g. ImageJ-macosx), must be found in your PATH. You can create it using ln -s ... and need to ensure that the link file can be executed (chmod +x ...).
  • xvfb-run installed (this is used to run Fiji from the command line). \

Remarks

  • this workflow loads the entire dataset to RAM. Although this limits the size of datasets that can be analysed on personal computers, it is usually not an issue on servers...
  • we recommend using a dedicated Fiji version and update it with caution (breaking changes used to be regularly introduced in automatic Fiji updates).
  • normally Xvfb itself is installed as part of X11; the utility script xvfb-run should come along with it but can also be downloaded.

Step by step

  1. If necessary (in case other objects than growth channels are present in the field of view), open the images in Fiji, manually create an ROI including only the growth channels area for each position, save each ROI in a dedicated file. This will be used to cut out the portion of the picture outside the ROI once registration is done.
  2. run the moma_pre_cli bash script from the command line with the appropriate arguments, for instance:
moma_pre_cli -i raw_image_path -o output_folder -p pos_name -x roi_path -r 180

Some technical details

Because neither ImageJ nor Bio-Formats handle multi-position datasets, they are usually loaded as several "series" of a given dataset. Hence the path to the dataset is the same for all positions even if it's split over several files (usually the path to the first file), the name of the position is passed as another argument.

Data are imported in Fiji using the BioFormats library which supports over 140 file formats. The script provided has been designed to import Micro-Manager datasets and will certainly need to be slightly modified to work with other formats.

The ImageJ-macro moma_pre_cli.ijm takes arguments with the syntax Arg_flag1__value1__ArgSep__Arg_flag2__value2__ArgSep__…; this somewhat unsual syntax circumvents the fact that ImageJ accepts only one argument when called from the command line with a macro. It recognizes the following flags:

  • i: path of raw image folder to process
  • o: path of folder where the splitted growth channels should be saved
  • p: name of the position to be loaded
  • x: path of cropping ROI for this position (optional but recommended)
  • r: rotation angle (in [180, 90, -90, 270]; optional)
  • c: new order of channels (e.g. 21; optional)

Corresponding arguments are available in the bash script moma_pre_cli (with the syntax -x value), along with:

  • m: path to fiji macro
  • tmin: first time-point to be processed
  • tmax: last time-point to be processed