---

The first of two examples is presented to demonstrate the application of the workflow to three-dimension (3D) backscattered energy images; the previous examples applied the segmentation and analysis workflow to 2D scenarios. Moreover, for the examples presented in sections 2-4, a dual-modality breast imaging approach was taken by combining ultrasound and microwaves. The workflow was used as a tool to test a more general hypothesis that incorporating prior information extracted from ultrasound imagery into a reconstruction algorithm improves the quality of the images reconstructed relative to present state-of-art variants. In section 2 the refinements were related to the use of prior information to construct an inhomogeneous numerical background that is used by the reconstruction algorithm. In sections 3 and 4, the change was related to a refinement of the path ray model used for the time-delay calculations.

For the examples presented in sections 5 and 6, the reconstruction algorithm is fixed over the numerical experiments that are conducted. The objective is to demonstrate how the workflow may be used assist a researcher who is conducting an experiment to investigate how a change to an independent variable used in the reconstruction process impacts a dependent variable associated with a response that is reconstructed within the image. Accordingly, the quantitative results may be used to support or challenge an hypothesis held by the researcher.

The image processing and analysis workflow presented in sections 2 – 4 has been adapted to operate on volumes and 3D representations of masks (or objects). Furthermore, these 3D microwave radar examples differ from the 2D examples in that the aim is to compare and quantify differences between two reconstructions. For this scenario, the parameters that are initially implemented by the reconstruction operator are set to base case values. A reference image is formed when the operator is applied to microwave reflection data.

The parameter values used by the operator are subsequently modified, and the operator is applied to the same microwave reflection data. This leads to a reconstruction that is referred to as a test image. Application of the segmentation and analysis workflow to the reference and test images is used to measure changes within the test image that arise due to a variation of a reconstruction operator parameter.

Figure 1. Image reconstruction workflow [8, 23, 24].

A general overview of the 3D reconstruction workflow and the reconstruction operator that is applied to the backscattered fields is presented in section 5.1. The overview provides key background information and identifies important parameters used in the image reconstruction process. The numerical experiment used to demonstrate how the segmentation and analysis workflow may be used to analyse results is described in section 5.2. The segmentation methodology that is applied to the reference and test images is presented in section 5.3. The aim of the methodology is to delineate regions associated with reflections from significant scatterers. A general backscatter energy analysis of these dominant responses present within the test and reference images is carried out and the results are shown in section 5.4. The metrics described in section 1.4.b(1) have been adapted for the application to volumes and 3D objects, and so are applied to the test and reference 3D masks in order to evaluate the geometric properties of the dominant scattering regions within the reconstructions. The geometric analysis results are presented in section 5.5. Measurable changes in the response of a dominant scatterer, smearing of a response, changes in the intensity and focus of the response, and changes to the extent of a response are examined in section 5.6. Finally, in order to provide an understanding beyond the dominant responses, the structural similarity index measure (SSIM) and normalized root mean square error (NRMSE) metric are applied over 2D slices along each of the coordinate axes. The results are presented in section 5.7.

Figure 2. Scattered density breast model constructed from an MRI scan of a patient. Gland consists of three different types of tissues indicated by different colors. A $15$ $mm$ tumor (red) is embedded in the tissue. Tissue properties are shown in Table I [21].

Table I. Scattered density breast model Debye parameters [21]

---

A general overview of the workflow implemented to reconstruct 3D images from microwave radar data is provided. The workflow is summarized in Figure 1.

The first step of the workflow is to generate a set of data. For these examples, data are generated by simulating a radar system operating in the monostatic mode. In this mode, a single balanced antipodal Vivaldi antenna with a director included in the aperture (BAVA-D) [20] illuminates a breast model with an ultra-wideband pulse. The 3D numerical model of a scattered density breast is described in [21] and is constructed from an MRI scan acquired from a patient study reported in [22]. The model is covered with a $2$ $mm$ thick skin layer and a $15$ $mm$ diameter tumor is embedded in the breast tissue as shown in Figure 2. Properties assigned to each tissue type used by the model are shown in Table I and is discussed in more detail in [21].

The model and antenna are immersed in Canola oil and are constructed with SEMCAD X (SPEAG, Zurich). The setup is simulated with a finite difference time-domain (FDTD) solver which computes the electromagnetic fields. The single antenna is moved to $M$ positions around the breast model and the setup is simulated at each new position to create an $M$ element synthetic patient-specific array [21].

The backscattered field computed at each antenna position is extracted by subtracting the incident field from the total field. The total and incident fields correspond to the fields computed in the presence and absence of the numerical breast model, respectively. Once the data are generated, an adaptive filtering technique is applied to each backscattered field to remove the dominant response corresponding to the skin reflection [23]. This results in the set of preprocessed time-domain signals, { s₁(t), s₂(t) , ..., s_M(t) }.

To form an image, the set of preprocessed signals are synthetically focused to a specific location, or focal point, in the imaging domain that represents the breast interior. A backscattered signal that originates from a focal point propagates along a path ray to arrive at the synthetic array with different times-of-arrival at each antenna location. The delay-and-sum image reconstruction operator compensates for these delays by applying a reverse delay to each preprocessed signal. This has the effect of time shifting the signals to align the reflections from the focal point. Accordingly, if the time delays are accurately computed, the data are focused so that the signals from each antenna add constructively at the focal point.

The time delays are computed with two preprocessing steps. First, the breast surface is reconstructed. For the numerical examples, a tessellated representation of the breast surface is extracted from the numerical model. The surface is used to construct the imaging domain that is comprised of voxels bound by the surface [24].

For the second step, the path ray that extend from a focal point at $\vec{r}$ within the imaging domain to each antenna position at $\vec{r_m}$ within the synthetic array is identified. A path ray is defined as the path or direction along which wave energy propagates through the breast tissues. The time for the signal to propagate over each path ray is computed with a three step procedure [24]. First, the reconstructed breast surface is used to delineate the breast interior. By identifying the point where the path ray intersects the breast surface, the distance along the path through the immersion medium from the antenna to the skin surface is assessed. Starting from the point of intersection and by assuming a skin thickness of $2$ $mm$, a segment corresponding to the skin layer is also defined. With the outer two segments revealed, this permits the decomposition of the path ray into three disjoint segments corresponding to the immersion medium, skin, and breast interior.

Next, real valued permittivities are assigned to each segment of the path ray; the material within each partition is assumed to be homogeneous, lossless, and constant across the band of frequencies that comprise the ultra-wideband signal (i.e., non-dispersive). This information is used to evaluate the velocity of the signal over the path length. Finally, the round trip travel time of the signal over the path is computed using the segment velocities and lengths. For each focal point, a set of $M$ round trip time delays, { $\tau_1(\vec{r}), \tau_2(\vec{r}), ..., \tau_M(\vec{r})$ }, is computed as the synthetic array is comprised of $M$ array elements; a time-delay is computed for each propagation path.

During focusing, the reconstruction operator applies the corresponding time-delay, $\tau_m(\vec{r})$, to the preprocessed signal, $s_m(t)$, associated with the $m^{th}$ antenna position [8]. Consequently, each preprocessed signal is time-shifted by the amount specified by the time-delay. The operator then combines the time-shifted signals over all $M$ antenna positions of the synthetic array. This has the effect of time shifting the signals to align the reflections from the focal point. The value computed by the reconstruction operator is assigned to the voxel at the focal point and represents the intensity of the backscatter energy.

The focal point iteratively moves to each reconstruction location within the breast, so the process is repeated over all voxels within the imaging domain. The result is a 3D backscattered energy image of the breast interior. Importantly, all time-shifted responses that arise from significant scatterers, such as regions of malignant tissue, are coherently summed; reflections from healthy tissues and any artifacts from preprocessing are assumed to add incoherently. Consequently, regions within the image where the backscatter energy intensity is elevated may be interpreted as responses corresponding to dominant scattering regions. These responses may, in turn, be used to detect the presence of a malignancy.

Figure 3. Reference image. A permittivity of $\epsilon_{r,int} = 4.47$ is assigned to all interior segments of the path rays when computing the corresponding time-delays. The reconstruction operator applies these time-delays to the preprocessed backscatter signals when forming an image. The reconstruction is sliced along each of the coordinate axes at the location of maximum intensity.

Figure 4. Test image. A permittivity of $\epsilon_{r,int} = 7.0$ is assigned to all interior segments of the path rays when computing the corresponding time-delays. These values deviate from the values assigned to the interior path ray segments used to compute the time-delays for the reference image by $57$ %. The reconstruction operator applies these time-delays to the backscatter signals when forming an image. The image is sliced along each of the coordinate axes at the location of maximum intensity.

---

The objective is to demonstrate how the segmentation and analysis framework may assist a researcher who is investigating how a change to an independent variable such as a reconstruction operator parameter impacts a dependent variable (the intensity of a response, for example). A key step in microwave radar image reconstruction is the time delay calculation. The calculation is dependent on a number of variables that may impact the accuracy of the calculation. An experiment is conducted in which an independent variable used in the time delay calculation is changed relative to a reference value. The segmentation and analysis workflow is applied to the test and reference images to evaluate the impact the change has on a dependent variable, such as the intensity of a response.

As described in section 5.1, an image is formed by synthetically focusing signals to focal points within the imaging domain. A signal that originates from a focal point, travels along a path ray and arrives at the synthetic array with different times-of-arrival at each antenna location. A path ray is modeled with three constituent segments: immersion medium, skin, and breast interior. The length of each segment varies and is dependent on the focal point location and direction (or angle) of the ray [24].

Real valued permittivities are assigned to each segment to evaluate the velocity of the signal over the path ray. The round trip travel time of the signal over the path is computed using the segment velocities and lengths. These travel times, or time delays, are computed by assigning $\epsilon_{r,ext}=2.5$, and $\epsilon_{r,skin}=36.4$, to the immersion medium and skin components of the ray path, respectively. The same immersion medium and skin segment value assignment is implemented for the reconstruction of both the reference and test images.

However, to form the reference image, a value of $\epsilon_{r, int}=4.47$, is assigned to the breast interior segments. This represents the mean value of the relative permittivity that is computed over all breast interior voxels (i.e., interior to the skin layer) of the forward model shown in Figure 1. Conversely, to form the test image, a value of $\epsilon_{r, int}=7.0$ is assigned to the interior segments. This means that the value assigned to all breast interior segments to form the test image is $57$ % higher compared to the value used to form the reference image.

The relative permittivity values are used in the preprocessing step described in section 5.1 to compute the time-delays for all path rays. During focusing, the reconstruction operator applies these time-delays to the preprocessed signal to time-shift each signal by the amount specified by the time-delay. The operator then combines the time-shifted signals over all $M$ antenna positions of the synthetic array and the resulting value is assigned to the voxel. The value assigned to the voxel represents the intensity of the backscatter energy at the focal point within the imaging domain. The reference and test images are shown in Figures 3 and 4, respectively.

In the context of a controlled experiment, the independent variable is the relative permittivity assigned to the interior segments of the path rays that are used to compute the time delays. Namely, the value assigned to all breast interior segments to form the test image is increased by $57$ % relative to the base line case (i.e., the value used to form the reference image). The segmentation algorithm delineates dominant scattering regions that are possibly associated with malignant tissue. The analysis framework is applied to the segmented regions to measure the affect that the change of the independent variable has on a dominant scattering region. Measurable changes of a dependent variable include shifts in the response of the dominant scatterer, smearing of a response, changes in the intensity and focus of the response, and changes in the extent of a response.

Figure 5a. Segmentation and analysis workflow.

Figure 5b. Overview of automated workflow to segment and evaluate 3D microwave radar breast images

---

The reference and test images shown in Figures 3 and 4, respectively, are input to the workflow represented in Figure 5. As described in section 5.2, the images are comprised of voxels that are interior to the breast surface. The entire image corresponds to the region-of-interest, so no pre-processing of the image is carried out prior to segmentation.

The unsupervised machine learning based segmentation algorithm is applied to the reference and test 3D images (not individual slices within the 3D images) to partition the volumes based on the intensity of the reconstructed backscatter energy. The algorithm applies the $k$-means clustering algorithm to the image in an iterative fashion. After each iteration, a statistical test evaluates the hypothesis that the cumulative distribution function of the voxel values within clusters at the previous and current iterations originate from different distributions. If the hypothesis does not fail, then the number of clusters used by the algorithm is incremented, otherwise convergence is implied and the volume is not partitioned further. The iterative procedure is summarized by the flow diagram presented in Figure 5 of 2.1.3.

When the segmentation algorithm is terminated, the reference and test volumes are partitioned into $12$ and $17$ clusters, respectively. The segmentation results where the 3D image is sliced at the location of maximum intensity sliced along each of the coordinate axes are shown for the reference and test volumes in Figures 6 and 7, respectively.

Figure 6. Reference image segmentation results. Reference image (left column) is input to segmentation workflow and partitioned into $12$ clusters (right column). The clusters are mapped to a tissue type image (center column).

Figure 7. Test image segmentation results. Test image (left column) is input to segmentation workflow and partitioned into $17$ clusters (right column). The clusters are mapped to a tissue type image (center column).

The segmentation algorithm partitions the 3D image based on the value of the intensity of the reconstructed backscatter energy. The cluster-to-mask mapping function is simplified to suit the functional interpretation of the backscatter energy image. Given the region-of-interest has been partitioned into $k$ clusters, the highest valued clusters, $c_{max(k)}$, are mapped to voxels within a binary image comprised of Dominant scatterer masks. Cluster $c_{1}$ identifies voxels external to the region-of-interest and are associated with the Background (i.e., skin and immersion medium). The remaining clusters represent the union of $c_{2}$ to $c_{max(k)-1}$ which correlates to the region within the region-of-interest that is external to the Dominant scatterer for which there is not significant scattering. Accordingly, these clusters are mapped to voxels within another binary image and are simply referred to as Non-dominant scatterers masks. The cluster-to-mask mapping function that is applied to the partitioned region-of-interest to assign clusters, $c_{k}$ where $k = 1, 2, …, max(k)$, to masks is expressed mathematically as,

The construction of masks leads to a set of test and reference objects, where object refers to a region of connected voxels within the mask. For some scenarios, the presence of a large number of significant scattering regions related to glandular/fatty tissue interfaces may occur. This complicates the application of the 3D metrics, as some of these regions may be considered as image noise (e.g. small relative to the breast interior). Accordingly, the post-processing technique developed for the microwave tomography images has been adapted to filter out these small isolated regions or artefacts. The volume enclosed by each closed surface is evaluated, and the ratio of each of the volumes to the largest volume is computed. Ratios below a user-defined extraction threshold are removed from the analysis.

In order to enhance the flexibility to identify artefacts and regions of interest, all significant scattering regions are automatically labeled as part of the mask construction process. This allows the user to specify which of the labeled scattering regions to remove from further analysis (small objects near the skin surface or chest wall, for example).

An analysis framework comprised of metrics described in section 1 is applied to the final set of test and reference objects. The aim of the analysis is to measure the affect that the change of the reconstruction operator parameter has on the response of the dominant scatterer that has been segmented from the reference and test images. The analysis is described next.

---

Figure 8. Dominant scattering regions segmented from test image that result in a set of test objects. The objects are displayed within the tessellated breast surface that bounds the imaging domain. The results of a general analysis of the backscatter energy associated with each of the detected dominant scattering regions is shown in the table.

Figure 9. Dominant scattering regions segmented from reference image that result in a set of reference objects. The objects are displayed within the tessellated breast surface that bounds the imaging domain. The results of a general analysis of the backscatter energy associated with each of the detected dominant scattering regions is shown in the table.

---

A general analysis is carried out by applying backscatter energy analysis metrics, described in section 1.4.b(1), to the test and reference objects. Each test object is applied to the test image to extract voxels with values corresponding to the backscattered energy intensity within a region associated with a dominant scatterer. The maximum intensity, $I_{max,test(i)}$, over all voxels within the test object is determined with (23). The coordinates of the location of this value are determined and used to calculate the FWHM and the SMR of the response using (21) and (22), respectively. Finally, the volume of each test object is evaluated with (24). The metrics that are applied to each object are described in detail in section 1.

The maximum backscattered energy intensity of each test object is scaled to the maximum value over the imaging domain of the test image to assist with the interpretation of the results. The value, along with the other values calculated for the general analysis are presented in a table. The objects are displayed within the imaging domain that is bound by the tessellated breast surface. The objects and tables are shown in Figure 8.

The analysis is repeated for each of the reference objects. That is, each reference object is applied to the reference image to extract voxels with values corresponding to the backscatter energy intensity within a region associated with a dominant scatterer. The maximum intensity, $I_{ref(j)}$ over all voxels within the reference object is determined with (25), and is used to compute the FWHM and SMR associated with the response. The volume of each reference object is calculated with (26).

To assist with a qualitative comparison of results, a similar table and display are provided for the reference objects. Note that the maximum backscattered energy intensity that is displayed for reference object is scaled to the maximum value over the imaging domain of the reference image. The objects and tables are shown in Figure 9.

Figure 10. Detailed geometric analysis to evaluate the similarity between $test(1)$ and $ref(2)$.

---

As described in section 5.2, the reconstruction operator used to form the reference and test images is applied to the same backscattered fields calculated from the equivalent forward model. The backscatter energy intensities are reconstructed onto voxels within an imaging domain with the same coordinates. Therefore, as a preprocessing step, the reference objects are transformed to a space shared by the test object.

The test object under investigation is then examined to determine if it is connected to any of the reference objects. The objects are connected if they share any of the same voxels. If this is the case, then the test and connected reference objects are evaluated within a region of interest defined as the box that bounds the union between the reference and test objects.

A comprehensive geometric property analysis is performed to assess the overlap between a test object under investigation and the connected reference object using the metrics described in section 1 that include metrics 1-8 (i.e., Dice Coefficient, Jaccard, Ratio-of-detection, Artefact-rejection). The similarity in shape is assessed with metric 16, the average Hausdorff distance. If the test object is not connected with any of the other reference objects, then the analysis is not conducted.

For the set of segmented test objects, test object $1$ is connected to reference object $2$. Therefore, a detailed geometric analysis is applied to compare the shape and overlap between the two objects; the results are shown in Figure 10. The metric values that are computed indicate that the degree of overlap between the two responses is not significant. The reconstruction operator parameter change led to a significant shift in the response in the test image relative to the corresponding response in the reference image. None of the other test objects are connected to a reference object, so a detailed geometric analysis is not conducted on these objects.

Recall from section 5.2 that the reference image is set to an image reconstructed by an operator with a baseline set of parameter values. The parameter values assigned to the reconstruction operator that was applied to the backscattered fields to reconstruct the test image differ from the baseline values by $57$%. Consequently, the time-delays values that were computed diverged from those values computed for the reference case. The operator applied the altered time-delays to time-shift the preprocessed signals. The resulting perturbation of the time-shifts led to variations in the constructive interference of signals at locations that differed from the corresponding locations within the reference image. The shift in the test response was significant enough that there was very little overlap with the corresponding reference response. This is reflected in the metric values computed by the similarity in shape and overlap metrics shown in Figure 10. A more detailed discussion on the impact that changes to the relative permittivity, $\epsilon_{r, int}$, of the interior path ray segment have on reconstructed responses of dominant scatterers is reported in [25].

The results are not included in the submitted manuscript [11], as the overlap between the test and reference object is not significant. As pointed out in section 1.4, the metric values are most meaningful when the ground truth is used as the reference image. That is, when the responses reconstructed within the test image are compared with malignant tissue regions within the forward model. For scenarios where differences between reconstructions are being evaluated, the geometric analysis results may be more relevant for scenarios where a controlled variable is perturbed by a small amount.

---

Figure 11. Comparative backscattered energy analysis of $test(1)$ which corresponds to the dominant scatterer within the test image. It is compared with the nearest reference object, $ref(2)$ which corresponds to the dominant scatterer within the reference image. The dominant response within the test image has been shifted by $9$ $mm$ relative to the reference image.

---

The backscatter energy intensities within the test object and the nearest reference object are compared. Similar to the preprocessing step described in section 5.5, the reference objects are transformed to a space shared by the test object. For each test object, the nearest reference object is identified (i.e., shortest distance between centroids). For example, for the $i^{th}$ test object, $test(i)$, the nearest reference object is $ref(j)$.

Starting with the first test object, $test(1)$, the nearest reference object is $ref(2)$. The maximum intensity, $I_{max,test(1)}$, over all voxels within the test object is determined with (23). The coordinates of the location of maximum value within the test object are determined to be < $-20, -6, -32$ > $mm$ and are used to calculate the FWHM and the SMR of the response using (21) and (22), respectively.

Likewise, the maximum intensity, $I_{max,ref(2)}$, over all voxels within reference object $ref(2)$ is determined with (25). The coordinates of $I_{max,ref(2)}$ are < $-20, 2, -28$ > $mm$ and are used to evaluate the shift in the test response relative to the reference image with (27). The calculated value indicates that the test response has shifted by $9$ $mm$, relative to the reference response. As already noted, the centroid of the forward model tumor is < $-20, 0, -28$ > $mm$. Therefore, the maximum intensity, $I_{max,test(1)}$ is displaced from the centroid of the actual tumor implemented by the forward model by $7$ $mm$, while $I_{max,ref(2)}$ deviates from the ground truth tumor centroid by $2$ $mm$.

The test image is formed with a reconstruction algorithm that uses permittivity values $57$ % higher than those values used to form the reference image. As part of the reconstruction workflow (see Figure 1 as part of section 5.1), the algorithm assigns the values to interior segments of each propagation path connecting an array element to a focal point. The interior segment path values were used to calculate the time-delays implemented by the reconstruction operator to form the test image. Time-delay values that deviate from actual values perturb the amount of time-shift applied to each preprocessed signal.

When accurate time-delay values are used, the reconstruction operator is applied to signals that are correctly time-shifted. The time-shifted signals are combined to form an image, and corresponding peaks in each signal interfere constructively where scatterers are located; the signals combine destructively in other regions of the image.

Conversely, when the reconstruction operator is applied to signals with perturbed time-shifts that arise from inaccurate time-delays, signal peaks no longer center on the scatterers. Consequently, constructive interference of signals occurs in locations that are offset from the actual scatterer locations. This offset in constructive interference manifests as a $9$ $mm$ shift in the test response relative to the reference response, and a $7$ $mm$ shift relative to the centroid of the actual tumor.

Differences between the test and reference response are evaluated with metrics $28-31$. First, the ratio between the maximum intensity within $test(1)$ and the maximum intensity within $ref(2)$ and is evaluated with (28). The resulting intensity ratio is $0.64$ which implies that the response reconstructed in the test image is diminished, relative to the response in the reference image. Perturbations of time-shifted signals due to the use of inaccurate time-delay leads to a misalignments of signal peaks that arise from tissue interfaces. When the signals are combined, this misalignment manifests as a weaker response relative to a response reconstructed from accurate time-delays.

Next, the volume ratio given by (29) measures the ratio between volumes $test(1)$ and the nearest reference object $ref(2)$. The volume ratio is intended to complement the $FWHM$ metric. The volume ratio is $0.52$, which implies that the test response is more compact, focused, and sharper relative to the reference image. This is also observed when qualitatively comparing $test(1)$ and $ref(2)$ shown in Figure 11.

The change in FWHM, $\Delta FWHM$, calculates the difference in FWHM between the $test(1)$ and nearest reference object $ref(2)$, and is given by (30). A negative distance of $−4$ $mm$ is measured which suggests the extent of the test response is reduced and more compact relative to the extent of the reference response. The result supports the reduced volume ratio value.

Finally, the change in SMR relative to the reference, $\Delta SMR$ %, is calculated with (31) to be $−32.2$ %. The negative $\Delta SMR$ % is primarily attributed to the receding response intensity. However, one third of the change relative to the reference is traced to elevated levels of clutter, noise, and artefacts external to the response.

Collectively, the comparative analysis infers that the increase in independent variable value leads to a decrease in accuracy of the calculated time-delays which has a negative impact on image quality. Although the response is sharper and more focused, the reconstructed response has shifted, is weaker, and there is an increase in clutter within the imaging domain. The results are summarized in Figure 11.

The comparative analysis is repeated for the remaining test objects. The results for $test(2)$ and $test(3)$ are shown in Figures 12 and 13, respectively. Qualitatively, the figures demonstrate that the nearest reference object used for the comparative analysis is a significant distance from the test object under investigation. This is quantified by the shift-in-response values measured for $test(2)$ and $test(3)$, which are $48$ $mm$ and $61$ $mm$, respectively. For these test objects, the nearest reference object is not in close proximity, and so the comparative analysis results are not as meaningful as the results for the first test object. Namely, the distance between the test and reference object suggests that the response do not arise from the same scatterer. Furthermore, it is possible that these responses are associated with artefacts.

Figure 12. Comparative backscattered energy analysis of $test(2)$. It is compared with the nearest reference object, $ref(1)$. Deviation between location of responses is $48$ $mm$.

Figure 13. Comparative backscattered energy analysis of $test(3)$. It is compared with the nearest reference object, $ref(2)$. Deviation between location of responses is $61$ $mm$.

---

---

Two-dimensional slices of the test and reference imaging domains are compared along each of the coordinate axes with the Normalized Root Mean Square Error (NRMSE). The metric is given by (17) except $ref_{mask}$ and $test_{mask}$ are replaced with two-dimensional slices of the reference and test imaging domains, respectively, at a point along a coordinate axis. As described in section 1.4b(4), each slice is preprocessed by removing pixels not contained in the imaging domain. This step is taken, as inclusion of these background pixels may reduce the accuracy of the metric. For example, if both the test and reference slices contain a large number of background pixels, then a low metric value (i.e., high degree of similarity) may be calculated. This may give a researcher a misleading indication that the slices are closely matched.

The metric is then applied to vectorized versions of the preprocessed slices to measure the normalized distance between the test and reference vectors. Hence, it heuristically informs a researcher how close the test image is to the reference image. A value of zero indicates a perfect match between images; an increasing value indicates that the difference between test and reference images is increasing. For the 3D examples, the NRMSE implies how much the test image has changed relative to the reference image due to the perturbation of the reconstruction operator parameters.

The metric is applied to slices along each point along a coordinate axis resulting in a set of values that are then plotted. The process is repeated for each coordinate axis. The plots are shown along the bottom row of Figure 14, and may be used as a tool to evaluate how the test and reference images deviate from each other along each of the coordinate axes.

To assist with the interpretation of the plots, note that the coordinates of the location of maximum value within the test and reference images is < $-20, -6, -32$ > $mm$ and < $-20, 2, -28$ > $mm$, respectively. The centroid of the forward model tumor is < $-20, 0, -28$ > $mm$.

The forward model tumor is located at $y=0$ $mm$, and the axial plot shows the NRMSE value is $0.3$ in this region, indicating some difference in these images (i.e. a shift in the location of the response, as expected from Figures 3, 4 and 10). In the sagittal and coronal planes, the NRMSE is lowest at $x = -20$ $mm$ and $z = -28$ $mm$, respectively, which corresponds to the physical tumor location. While an overall NRMSE value may suggest a high degree of similarity, the 2D slices provide granular information leading to additional insights that are useful when comparing results of different algorithms.

To complement the analysis using the NRMSE metric, the Structural Similarity Index (SSIM) presented in [10] is applied to two-dimensional slices of the test and reference imaging domains along each of the coordinate axes. It measures perceptual differences between two similar images. The computed values range from $0$ to $1$, where $1$ indicates that there are no perceptual difference between the test and the reference images.

Similar to the NRMSE metric, each slice is preprocessed by removing pixels not contained in the imaging domain. The SSIM Structural Similarity Index operator given by (32) is then applied to the preprocessed slices along each of the coordinate axes. A more detailed description is provided in section 1.4b(4).

The plots are presented in top row of Figure 14 and can be compared with the corresponding NRMSE plot shown in the bottom row. The SSIM plots do not appear to furnish meaningful information related to changes within the test image compared to the NMRSE plots. As presented in [10], the motivation for the development of the metric is to quantify subtle changes to image quality caused by processing such as data compression or by losses in data transmission.

For the 3D reconstructions presented in sections 5 and 6, the changes in the dependent variable are large relative to the base line case. These perturbations lead to significant spatial shifts, variations of the extent, and distortions of shape and intensity of the reconstructed responses. The dependent variable changes also manifest as image artefacts. These reconstructed response changes differ from the small subtle changes that the metric was intended to quantify. Hence, other metrics such as the NRMSE appear to be better suited for evaluating image changes compared to the SSIM. Nevertheless, it has been as a tool for assessing image differences by microwave imaging researchers (see [7, 26], for examples).

Figure 14. SSIM applied to series of 2D slices (bottom row) along axial (left), coronal (center), and sagittal direction (right) for comparison between test and reference images. Application of NRSME to series of 2D slices shown on top row.

---