MAGeT Brain Quality Control (QC) Guide - CobraLab/documentation GitHub Wiki

For information on how to set up/run MAGeT, see the MAGeTBrain Wiki.

Overview

1. Instructions
2. Rating scales (3-pt and 5-pt)
3. Examples of 0, 0.5, 0.75, and 1 labels
4. Common segmentation errors to look out for

Instructions

To QC MAGeT outputs, you need to open the subject brain with the label generated by MAGeT overlaid on it, and determine how accurately the label demarcates (segments) the underlying structures.

To display one image with its corresponding label:

  > Display -gray $file.mnc -label $file_label.mnc 

-gray is an optional flag that displays images in grayscale

To display multiple images with each image's corresponding label, use a for loop:

> for file in /path/to/subjects/*mnc; do Display -gray $file -label /path/to/labels/$(basename $file .mnc)_labels.mnc ; done 

Note: this command assumes that your image and label file names differ only by the addition of “_labels” to label files; you may have to adjust the basename function to match your files if they do not fit this convention.

To adjust the viewing contrast of images to view subcortical structures more clearly, adjust the slider in the left corner of the main viewing window in Display.

To toggle labels on and off on Display, press “f” and then “b” on your keyboard or “F - Segmenting” and then “B - Show Labels: On/Off” on the popup keyboard inferface. Press "space" at any time to return to the top level of this menu.

Rating Scales

When rating a label for structure, be sure to judge the label's segmentation accuracy in all views (i.e. coronal, sagittal, and horizontal). Sometimes over/undersegmentations are only visible in one view.

Three-point rating scale

0 : Obvious fails

Either the label is completely in the wrong place (which means there was likely a registration error), or (less obvious), decent sized errors, or many small errors Eg. large parts, or many small parts, of a structure not labeled, or label spilling out into ventricle or other brain areastriatum

0.5 : Small errors

Things like slight over or under segmentation, usually just in a few slices and confined to a small area, or a few voxels mislabeled or missing label These might be quite common depending on your data set (in some data sets you will often see the same minor error happening over and over again)

1 : Perfect (or near-perfect) outputs!

Label accurately covers the structure and nothing else, and looks good in all slices - these are usually fairly obvious.

Back to Main Menu

Five-point rating scale

If you want a rating scale that doesn't lump all less-than-perfect outputs into one bin (0.5), then this scale is for you! This rating scale helps to distinguish outputs that are imperfect but may be good enough to include in your analysis if there are not enough perfect (1) outputs, from those that are too poor to include in your analysis.

Note: When the criteria below specifies "over/undersegmented by x voxels," this is referring to the number of voxels that comprise the peak orthogonal distance from edge of the anatomical structure to the edge of the region that MAGeT has erroneously assigned or not assigned to the structure (not the total area of over/undersegmentation).

For example:

0 : Totally misses the mark

Part of the structure over/undersegmented dramatically (> 8 voxels), even just for a few slices.

or

Over/undersegmentation by 6-8 voxels for 4+ slices in more than one area.

0.25 : Large and expansive errors

Over/undersegmentation by 6-8 voxels for 4+ slices.

0.5 : Moderate or broad errors

Over/undersegmentation by 4-6 voxels that is confined to 2-4 slices.

or

Over/undersegmentation by 2-4 voxels that is confined to 2-4 slices for one (relatively) large area or in more than one area.

or

Over/undersegmentation by 2-4 voxels that is confined to 4-6 slices.

or

Over/undersegmentation by 1-2 voxels for 6+ slices.

0.75 : Small, contained errors

Over/undersegmentation by 2-4 voxels that is confined to 2-4 slices.

or

Over/undersegmentation by 1-2 voxels for 1-2 slices in more than one area.

1 : Perfect (or near-perfect) outputs!

Perfect outputs.

or

Over/undersegmentation by 1-2 voxels for 1-2 slices.

Subjectivity is unavoidable:

Ultimately QC scores are a way for you to divide your data into outputs you want to include and outputs you want to exclude from your analysis. You can always make a subjective judgement based on the relative size of the error compared to the size of the structure. For example, a label that oversegments by 1-3 voxels for 4 slices meets the criteria for a score of 0.5. This degree of oversegmentation may be relatively small for the cerebellum, but represent a considerable proportion of the label for the hippocampus. If you choose to incorporate this subjective judgment-calling in your QCing, make sure that you're consistent.

Examples of ratings for three- and five-point scales

Jump to:

• Striatum

• Thalamus

• Cerebellum

• Hippocampus

• Amygdala

• White Matter

• Globus Pallidus

Striatum:

Score of 0:

Score of 0.5:

Score of 0.75:

Score of 1:

Back to Examples menu

Thalamus:

Score of 0.5:

Score of 0.75:

Score of 1:

Back to Examples menu

Cerebellum:

Score of 0:

Score of 0.5:

Score of 0.75:

Score of 1:

Back to Examples menu

Hippocampus:

Score of 0 (1/2):

Score of 0 (2/2):

Score of 0.5:

Score of 0.75 (1/2):

Score of 0.75 (2/2):

Score of 1:

Back to Examples menu

Amygdala

Score of 1:

Back to Examples menu

White Matter

Score of 0:

Score of 0.5:

Score of 0.75:

Score of 1:

Back to Examples menu

Globus Pallidus

Score of 0.5:

Score of 1:

Back to Examples menu

Back to Main Menu

Common MAGeT Errors:

Note that this is not an all-encompassing list of MAGeT errors.

Hippocampus:

a) Hippocampus hole: Significant looking hole in middle of hippocampus for a few slices. This is actually OK! The hole here is a blood vessel that is anatomically accurate.

b) Hippocampus head oversegmentation:

Often visible in more lateral sagittal sections. Label often spills into the inferior horn of the lateral ventricle or into the white matter subjacent to the head (pictured below - green label).

c) Hippocampus head undersegmentation: Often visible in lateral sagittal views.

d) Hippocampus tail oversegmentation: Also usually seen in lateral sections, seen spilling into the lateral ventricle. See 1.g) for an example of what looks like an oversegmentation of the tail but is actually a correct label of the white tracts

e) Hippocampus tail undersegmentation:

f) Hippocampus patchiness; a few voxels missing outside of hole area

g) Hippocampus left posterior region looks oversegmented (blue label), but label is actually for white matter tract.

Thalamus

a) Thalamus undersegmentation:

Cerebellum

a) Inferior undersegmentation:

b) Cerebellum oversegmentation:

c) Cerebellum cut off from image:

Striatum

a) Oversegmentation of head/body; note that oversegmentation of the left striatum into the ventricles is a problem in multiple atlases so if you're QCing to build a template library, you may just need to find the least oversegmented outputs.

b) Oversegmentation of head between internal capsule; seen best in coronal slices

Amygdala:

a) Spilling into the hippocampus:

Back to Main Menu