---

The second of two examples to demonstrate the application of the workflow to a backscattered energy image is presented. Similar to the example presented in section 3, a dual-modality breast imaging approach is taken by combining ultrasound and microwaves. The image has been reconstructed by an enhanced variant of the time-shift-and-sum approach described in [19] that is applied to numerical ultra-wideband (UWB) microwave radar data. Prior structural information acquired from ultrasound imagery is incorporated into time-delay computations that are used by the reconstruction operator.

The case presented is more challenging, relative to the first case, as the data are generated from a numerical model of a dense breast with one tumor embedded in the glandular tissue that dominates the breast interior and is described in [9]. Once again, the researcher uses the workflow as a tool to help support or refute the hypothesis that the use of prior information to refine the segmentation of the path ray improves the accuracy of the time delays that are calculated. However, the researcher extends the hypothesis to evaluate if a refinement in the reconstruction algorithm leads to improvement in the quality of reconstructions over a broad range of scenarios. The first example implemented an experiment using a numerical model of a fatty breast; this example implements an experiment using a numerical model of a dense breast.

The relative permittivity component of the electromagnetic model that was used to generate the numerical data serves as ground truth. For the analysis, it is used as the reference image while the backscatter energy image serves as the test image. The reference and test images that are input to the workflow are shown in Figure 1.

Figure 1. Reference image (left) set to numerical model of dense breast that is used to generate data. Test image (right) reconstructed from backscattered fields generated by numerical model [9].

The reference and test images shown in Figure 1 are input to a workflow shown in Figure 2. The workflow is comprised of segmentation and analysis steps. In the first step of the segmentation process, the reference image is partitioned into immersion medium, breast interior, and a region-of-interest. The region-of-interest mask constructed from this initial segmentation process is used to further to refine the segmentation of the reference and test images into malignant and non-malignant tissue regions. The region-of-interest of the reference image is partitioned with a thresholding segmentation technique; an unsupervised machine learning technique is applied to the region-of-interest of the test image. The segmentation steps are described in more detail in section 4.1. The analysis steps are described in more detail in sections 4.2-4.3.

Figure 2. Segmentation and image analysis workflow.

4.1 Segmentation of images

---

4.1.1 Segmentation of reference image

The first step of the workflow is to delineate the breast interior with a skin surface contour. The closed contour is used to partition the reference image into immersion medium, breast interior, and a region-of-interest and is described in section 3.1.1. Similar to the fatty breast example, the reference and test images share the same immersion medium, breast interior, and region-of-interest masks. For both the reference and test images, only the region of the breast that is within the region-of-interest (i.e., interior to the skin) is partitioned into tissue types.

The region-of-interest mask constructed in the first step is applied to the reference image. A simple thresholding segmentation [12] operates on the pixels extracted from the reference image to delineate the region-of-interest into malignant and non-malignant tissues. The resulting malignant and non-malignant tissue masks are shown in the left column of Figure 3. The breast interior mask formed in the first step is also shown in the top row of Figure 3.

A skin tissue mask is also formed by subtracting the region-of-interest mask from the breast interior mask. The malignant, non-malignant, and skin masks are mapped to a tissue type, and, the immersion mask is mapped to background. The disjoint union of the tissue type regions leads to a tissue type image shown in the left panel of the middle row of Figure 4. The segmentation of the reference image and the mapping of the masks to tissue types is discussed in more detail for the fatty breast example presented in section 3.1.2.

Figure 3. Masks constructed by segmentation techniques applied to the region-of-interest within the image. The reference masks (left column) are constructed with a thresholding technique and the test masks (right column) are constructed with an unsupervised machine learning technique.

Figure 4. Reference and test images for a fatty breast (first row) with the corresponding tissue types identified via segmentation (middle row). Cluster map for final iteration of segmentation of region-of-interest of test image (bottom row).

4.1.2 Segmentation of test image

---

An automatic segmentation technique is applied to the region-of-interest within the test image to partition the region based on the intensity of the reconstructed backscatter energy. It is comprised of an iterative unsupervised machine learning approach and is described in [17, 18]. First, the region-of-interest mask is applied to the test image and the automatic segmentation algorithm applies the $k$-means clustering algorithm to the extracted region in an iterative fashion. The segmentation results of the region-of-interest within the test image after each iteration of algorithm are shown in Figure 5. For this example, when the segmentation algorithm is terminated, the region-of-interest within the test image is partitioned into $7$ clusters. The cluster map for the final iteration of the segmentation algorithm is shown in the right hand side of the bottom row of Figure 4.

Next, a mapping function assigns the clusters to binary images that have the same dimensions as the test image and are referred to as masks. The highest valued cluster corresponds to the dominant scatterer (i.e., highest backscatter energy), so it is mapped to the malignant tissue mask. The remaining clusters are mapped to the non-malignant tissue mask. Similar to what was described for the reference image, a skin tissue mask is formed by subtracting the region-of-interest mask from the breast interior mask. Furthermore, each tissue mask is mapped to a tissue type and the immersion mask is mapped to background. The segmentation of the region-of-interest within the test image and the mapping of the masks to tissue types is presented in section 3.1.3, but refer to [17, 18] for a more detailed description.

The disjoint union of the tissue type regions leads to a tissue type image shown in middle of the right column of Figure 4. The tissue type images constructed from the reference and test images are used for qualitative purposes and can assist with the interpretation of the backscatter energy images. Conversely, performance metrics are applied to the tissue masks and are used for the quantitative analysis of the backscatter energy images. Note that the immersion medium and skin masks are only used to construct the tissue type image; metrics are not applied to these masks in the quantitative analysis steps.

Figure 5. Segmentation results of the test image after each iteration of algorithm. Region-of-interest within test image is partitioned into $7$ clusters when convergence is sensed.

4.2 Interface analysis of dense breast images

---

The edge points that are sampled uniformly from the boundary of the test masks constructed by the segmentation task collectively represent estimated points on interfaces of regions of the significant scattering region. A contour is fitted to the estimated points and superimposed onto the ground truth model along with the corresponding contour of the reference mask. The contours are also superimposed onto the backscatter energy map. The ground truth model and the backscatter energy map adds context to the test and reference contours, so the images are helpful to visually evaluate the quality of the interface extracted from the data. These images are shown in the top row of Figure 6. The reference contour of the tumor region is shown in red while the test contour of the significant scattering region is represented with the blue line.

The estimated points sampled from the boundary of a test mask are compared with the corresponding boundary of the reference mask to measure how accurately the interface of the significant scattering region is extracted by the reconstruction algorithm. More specifically, the ability of the reconstruction algorithm to accurately extract a tumor interface is evaluated by measuring the distance each edge point on a reconstructed interface is to the nearest point on the corresponding reference surface (refer to section 1 description of metrics). A negative distance implies that the estimated point is within the ground truth tumor region, while a positive distance implies the opposite.

In the bottom center panels of Figure 6, points of the estimated interface along with the errors are superimposed onto the reference contour to help visualize how the errors are spatially distributed along the interface. A quantitative analysis of the errors is provided by plotting the error distribution along with an estimate of the mean and variance (or spread) of the errors. This information is shown in the bottom right hand panel of Figure 6.

Figure 6. Interface analysis results of interface associated with dominant scattering region.

3.2 Geometric property and backscatter energy analysis of dense breast images

---

The significant scattering region is examined to determine if it is connected to (i.e., share the same pixels) and there is overlap with the reference tumor region. If the test and reference masks are connected, then a comprehensive geometric property analysis is performed to assess the overlap between the reconstructed significant scattering region using metrics 4-8 and 16 described in the Description of Metrics. The overlap metrics 4-8 are primarily used to measure how accurately the scattering region is reconstructed relative to the reference region, so the measured values are most meaningful when the ground truth is used as the reference image (as in this case).

A suite of metrics is added to the workflow to support backscatter energy analysis, and is described in more detail in the Analysis of backscatter-energy properties. To analyze backscatter energy, the test masks are applied to the image to extract the energy bounded by the contour of the mask. The resulting energy is assessed with metrics 20-22 (localization error, FWHM, and SMR). The results of the analysis when the metrics are applied to the reconstructed significant scattering region are shown in the table within Figure 7.

Figure 7. Dominant scattering region of backscatter energy image compared with tumor tissue region within reference image.