---

The first of two examples of the application of the workflow to images reconstructed from numerical ultrasound backscattered fields (or reflection data) is presented. Knowledge related to the breast’s internal structure is extracted from the ultrasound imagery. This prior structural information is then incorporated into the reconstruction algorithm implemented for microwave imaging. Refer to the examples presented in sections 2-4 for multi-modality imaging examples that demonstrate improvements to microwave tomography and radar images when the reconstruction algorithms are furnished with ultrasound derived structural information. Additional examples, a detailed explanation of the multi-modality approach, and a description of the reconstruction algorithms are reported in [9, 12, 19].

As shown in [9, 19], both the breast exterior as well as the interior fat–fibroglandular interfaces are extracted from a qualitative ultrasound reconstruction. The aim of this first example is to demonstrate the use of the segmentation and analysis workflow to evaluate how effective the ultrasound reconstruction algorithm is to identify the breast surface, and to delineate dense tissue regions. The dense tissue regions are comprised of glandular, transition, and for these examples, malignant tissues; the non-dense tissue regions are comprised of fatty (or adipose) tissue. The reconstruction algorithm is not used for tumor detection. Therefore, the workflow is adapted for the analysis of the breast surface and the delineation of the dense tissue regions rather than tumor detection.

For this first example, a numerical ultrasound model of a fatty breast with one tumor embedded in the glandular tissue described in [9] is illuminated with a circular array of equally spaced ultrasound transducers. For this multi-static scenario, the backscattered field at each transducer location is extracted by subtracting the incident field from the total field. Note that the total and incident fields correspond to the fields measured in the presence and absence of the numerical breast model, respectively. An ultrasound image is reconstructed by applying a multi-static delay-and-sum method to the backscattered fields over all transducer locations. Refer to [9] for details that include a description of the forward model and how it was assembled, the placement of the ultrasound sensors in the immersion medium and the manner in which the they were simulated, the bandwidth of the pulse used to illuminate the model, and the ultrasound solver used to generate the numerical data.

The ultrasound propagation speed (or velocity) component of the forward model that was used to generate the numerical data serves as ground truth. For the analysis, it is used as the reference image while the ultrasound image reconstructed from the scattered fields serves as the test image. The reference and test images that are input to the workflow are shown in Figure 1.

Figure 1 Velocity model of fatty breast used as reference (left) and test image (right) reconstructed from numerical reflection data generated from the numerical model [9].

The reference and test images displayed in Figure 1 are input to a workflow shown in Figure 2. The workflow is comprised of segmentation and analysis steps. In the first step of the segmentation process, the reference and test images are partitioned into immersion medium, breast interior, and a region-of-interest. An approximation of the skin region is also formed.

The region-of-interest is partitioned further into dense and non-dense (or fatty) tissue regions. The region-of-interest of the reference image is partitioned with a thresholding segmentation technique; an unsupervised machine learning technique is applied to the region-of-interest of the test image. The segmentation steps of the reference and test images are described in more detail in sections 7.1 and 7.2, respectively. The interface and geometric property analysis steps are described in more detail in sections 7.3 and 7.4, respectively.

Figure 2a. Segmentation and image analysis workflow.

Figure 2b. Workflow applied to test-reference pair. Images are segmented into regions dominated by increased scattering to construct masks. Masks are used to measure discrepancies between regions within test and reference images.

7.1 Segmentation of reference image

---

As presented in Figure 2, the first step of the segmentation procedure is to delineate the breast interior. This is accomplished using a thresholding technique [12] to identify the region comprised of the immersion medium. Once this region has been identified, a closed contour is fitted along the skin surface to delineate the breast interior. This partitions the image into two regions: the breast interior bound by the skin surface contour and the immersion medium that is external to this region. Masks are constructed from these two regions.

A region-of-interest (ROI) mask is constructed by uniformly contracting the skin surface contour by $3.0$ $mm$ towards the center of the interior using the morphological contraction method described in [13, 14]. A skin mask is formed by subtracting the region-of-interest mask from the breast interior mask. Next, the region-of-interest mask is applied to the reference image to extract pixels within the region-of-interest.

The tissues represented in the forward model are constructed using literature values for the speed of propagation of sound through various tissue types (see Table I of [27], for example). Using this information, the regions containing fatty and dense tissues are segmented with a simple thresholding procedure [12]. Namely, the upper bound of the speed of propagation of sound through fat tissue establishes the threshold to segment the fatty tissue within the region-of-interest. Hence, all pixel values less than the threshold value within the region-of-interest are classified as non-dense (or fatty) tissue. These pixels are used to construct the Fatty tissue mask. The remaining pixels are classified as dense tissues and are used to construct the Dense tissue mask. Contained within the dense tissues are fibroglandular, transition, and in this case, malignant tissues. The Fatty and Dense tissue masks that are constructed with this method are shown in the left column of Figure 3, while the breast interior mask formed in the first step is shown in the left panel of the top row. The Interior and Dense tissue masks are used by the analysis component of the workflow.

A tissue type image is formed by mapping each mask to a tissue type. The mask corresponding to dense tissues is mapped to the Dense label, the non-dense tissue (or Fatty) mask is mapped to the fat label, and the Skin mask is mapped to the skin label. Finally, the immersion mask is mapped to background. The disjoint union of the tissue type regions leads to a tissue type image shown in the left panel of the middle row of Figure 4.

Figure 3. Segmentation masks constructed from reference (left column) and test images (right column).

Figure 4. Reference image representing forward model (top left) and test image representing reconstructed ultrasound image (top right) of a fatty breast with corresponding tissue types identified via segmentation (middle row). Cluster map of final iteration (bottom row) of unsupervised machine learning segmentation algorithm that is used to segment test image. The reference image is segmented using a thresholding technique.

7.2 Segmentation of test image

---

Similar to the preprocessing step used for the reference image, the test image is preprocessed to construct masks of the breast interior and immersion medium. This is accomplished by applying a k-means clustering technique [28] to the test image in order to partition the image into two clusters. The highest valued cluster corresponds to the skin layer, therefore, a closed contour is fitted along the outer boundary of the cluster to delineate the breast interior. A Breast Interior mask is constructed from the region bound by the contour, while an Immersion medium mask is constructed from the all pixels outside the interior.

The closed contour that bounds the interior represents the skin surface. A region-on-interest (ROI) mask is constructed by uniformly contracting the skin surface contour toward the center of the breast interior by $3.0$ $mm$ using the morphological contraction method described in [13, 14]. A skin mask is formed by subtracting the region-of-interest mask from the breast interior mask.

The region-of-interest mask is applied to the test image. The aim is to partition the region-of-interest into two general groups of tissues: dense tissues, and non-dense (or fatty) tissues. This is achieved by applying an automatic segmentation technique that partitions the region based on the intensity of the reconstructed backscatter energy. The rationale for using this approach is that the increase scattering that arises along the skin surface and the interface between dense and fatty tissues manifests as an increase in intensity of the backscattered energy observed in the ultrasound image.

The automatic segmentation algorithm presented in [17] is comprised of an unsupervised machine learning approach that iteratively refines the partitioning of the region. After each iteration, a statistical test evaluates the hypothesis that the cumulative distribution function of the pixel values within clusters at the previous and current iterations originate from different distributions. If the hypothesis does not fail, then the number of clusters used by the algorithm is incremented, otherwise convergence is implied and the image is not partitioned further.

For this example, when the segmentation algorithm is terminated, the region-of-interest within the test image is partitioned into $7$ clusters. The cluster map for the final iteration of the segmentation algorithm is shown in the right hand side of the bottom row of Figure 4. Note that each cluster within the image can be identified using the provided color bar.

Next, a mapping function assigns the clusters to binary images that have the same dimensions as the test image and are referred to as masks. The mapping function described by equation (7) in [17] has been adapted to accommodate ultrasound images. Given the region-of-interest has been partitioned into $k$ clusters, the two highest valued clusters, $c_{max(k)}$ and $c_{max(k-1)}$, correlate to the dominant scatterer (i.e., highest backscattered energy), so they are mapped to the Dense tissue mask. Likewise, cluster $c_{1}$ identifies the background and correlates to pixels outside the region-of-interest. The remaining clusters $c_{2}$ to $c_{max(k)-2}$ are mapped to the Fatty tissue mask. The cluster-to-mask mapping function that maps clusters, $c_{k}$ where $k = 1, 2, …, max(k)$, to the corresponding mask is expressed mathematically with

Likewise, a tissue type image is formed by mapping the tissue masks to labels as described for the reference image in the previous section (i.e., section 7.1). The disjoint union of the tissue type regions leads to a tissue type image shown in the middle of the right column of Figure 4. The tissue type images constructed from the reference and test images serve as a visualization tool that may assist with the interpretation of the ultrasound images. The analysis component of the workflow applies metrics to the Interior mask, the Dense tissue masks, and the closed contour that bounds the interior. The metrics measure discrepancies between regions within the test and reference images and is described next.

Figure 5. Estimated (blue line) and actual (red line) interface that delineates the breast interior (left) and dense tissue regions (right). The interfaces are superimposed onto the test image.

7.3 Interface analysis

---

The boundary of each Dense tissue mask constructed from the test image is sampled uniformly. This leads to a set of edge points that collectively represent estimated points along each interface that delineates the Dense tissue. A closed contour is fitted to the points and is superimposed onto the test image along with the corresponding contour of the reference mask.

An important aspect of the modality is the ability to extract structural information of the breast exterior. Consequently, the skin surface contour that was formulated in the preprocessing step to delineate the breast interior is also superimposed onto the test image. The test image adds context to the test and reference contours, and so the image is helpful to visually evaluate the quality of the interface extracted from the data. Contours of the reference and test interfaces extracted from the skin surface and Dense tissue masks are shown in Figure 5. The reference contour is shown in red while the test (referred in the figures as reconstructed) contour is represented with the blue line.

Figure 6. Analysis of skin surface contour that delineates the interior.

The estimated points sampled from the boundary of a test mask are compared with the corresponding boundary of the reference mask to measure how accurately the interface is extracted by the reconstruction algorithm. More specifically, the ability of the reconstruction algorithm to accurately extract the skin surface or a tissue interface is evaluated by measuring the distance each edge point on a reconstructed interface is to the nearest point on the corresponding reference contour (refer to section 1). A negative distance implies that the estimated point is within the reconstructed region, while a positive distance implies the opposite.

In Figures 6-7, selected points of the estimated reconstructed interface along with the errors are superimposed onto the reference contour. The images are used to visualize the spatial distribution of errors along an interface. A quantitative analysis of the errors is provided by plotting the error distribution along with an estimate of the mean and variance (or spread) of the errors. This information is shown in the right hand panels of Figures 6-7.

Figure 7. Analysis of interfaces associated with each of the reconstructed dense tissue regions.

7.4 Geometric property analysis

---

The interior region bound by the skin surface and the regions bound by each of the dense tissue interfaces are compared with the corresponding region within the reference image. A comprehensive geometric property analysis is performed on each of the reconstructed regions using metrics 1, 2, 3, 4, 5, 6, 7, 8, 16 described in section 1. The subset of overlap metrics are primarily used to measure how accurately the skin surface and dense tissue are reconstructed relative to the reference region. Accordingly, the measured values are most meaningful when the ground truth is used as the reference image (as in this case). Results of the analysis are shown in Figures 8 and 9.

Figure 8. Geometric property analysis of skin surface and the region bound by this closed contour.

Figure 9. Geometric property analysis of each region of dense tissue extracted from test image.