Pixel classification is a method that assigns each pixel in an image into classes that you specify (e.g., background, stroma, tumor), based on textural and intensity information. This can be useful especially for image types like brightfield, histochemical stains, or other non-fluorescent images, where often traditional intensity-based thresholding methods do not perform well. The researcher teaches the classifier to recognize which pixels belong in which class by marking a small number of pixels across a few images (“the training set”), which is used to build a classifier that can then be applied to a larger batch of images (“the test set”).

Table of Contents CellProfiler 3.0 and beyond Procedure CellProfiler 2.3 and earlier Windows Mac OS X, Linux Abandoned work

Our current recommendation is to download ilastik from http://ilastik.org and create a pixel classification or autocontext project following the workflows outlined there. Once your classifier is trained to your satisfaction, you can export your probability map images as TIFs as described below.

In future work, we aim to restore a direct connection between ilastik and CellProfiler that would make the intermediate export of the probability maps obsolete; you can follow progress here: http://github.com/CellProfiler/CellProfiler-plugins/pull/13

• In ilastik, assign and train classes; it is best to train across a wide variety of images and pixels. For more guidance, see our blog post at http://blog.cellprofiler.org/2017/01/19/cellprofiler-ilastik-superpowered-segmentation/
• From ilastik, export probability maps
  • In the "Prediction export" under "Choose Export Image Settings", you must check all 3 boxes under "Transformations"
    • Set "Convert to Data Type" to "Unsigned 16-bit"
    • Set "Renormalize [min,max]" to "from: 0.00,1.00 to: 0, 65535"
    • Set "Transpose to Axis Order" to "cyx"
  • Also in the "Choose Export Image Settings" module, set the Output File Info "Format" to "tif sequence"
• In CellProfiler, load the probability maps (and original image(s) if desired) using the Images module
  • Assign each a name in NamesAndTypes, and use “Image Type” = Grayscale
• In CellProfiler, segment objects from the probability maps using IdentifyPrimaryObjects
  • Particularly if you only have two classes, rather than allowing CP to choose a threshold you one option is to set a manual threshold of 0.5. This threshold guarantees that only the most likely pixel class is chosen, though a lower value could be used if applied judiciously (an argmax operation would be more general, though this cannot currently be done in CP)

Historically, pixel classification was done in CellProfiler by first using the open-source software ilastik to create the training set and classifier, then instructing CellProfiler to apply that classifier to images in a pipeline. Unfortunately, maintaining the necessary connection between CellProfiler and ilastik became impossible given our team’s resources, so this workflow only functioned with a very old version of ilastik (v 0.5) that was bundled with CellProfiler v 2.1.1 - 2.3, and even then only on Windows.

• Use ilastik bundled with CellProfiler (ilastik version 0.5) and the ClassifyPixels Module in CellProfiler
  • This is the most tightly integrated approach between ilastik and CellProfiler (version 2.1.1 - 2.3 only)
  • Procedure
    • Train with ilastik 0.5 in a small number of images
    • Export the classifier h5 file from within ilastik
    • In CellProfiler (version 2.1.1 - 2.3 only)
      • Load the raw images
      • Use the ClassifyPixels module to select the classifier file, which will be used to classify all pixels in all loaded images

• Not applicable: ClassifyPixels in CellProfiler only works in Windows, and not in OS X or Linux. This incompatibility (at least the short version) is due to the complex nature of building both ilastik and CellProfiler from source, both of which have a large number of dependencies (and sometimes in conflict).

• We created a prototype new pixel classifier fully embedded within CellProfiler which would allow classification on Windows, Mac OS X, and Linux and access advanced machine learning algorithms in scikit-image.
• If you are comfortable setting up source code on your computer, you can try it out by using this Github Pull Request.
• Procedure:
  • Step 1: Create a training set on a subset of pixels in a small number of images and save images showing those labels
    • You can use any version of ilastik to train and save the labels
    • You can also use any other labeling software that generates “label matrices” or binary images (in the case of two-class classification, foreground/background)
  • Step 2: Train the classifier in CellProfiler
    • In CP's Classify module, choose train, choose ilastik as ground truth input and point to labels tiff file; upon running the pipeline, this will create a classifier
  • Step 3: Apply the classifier to your images using CellProfiler:
    • Build a separate pipeline where in the ClassifyPixels module, you choose classify (instead of train) and run the module
  • Optional: identify objects guided by classified pixels
• The number of steps needed to create and use the classifier, as described above, is non-optimal.
• The speed and accuracy of the pixel-based classifier is not yet ideal.