🧵 How to plot a time‐series of tongue movement (M‐Mode Analysis) - articulateinstruments/AAA-DeepLabCut-Resources GitHub Wiki

AAA provides many tools to perform analysis through a slice of the ultrasound image along a single scan line, even though it does not directly have M-Mode. Most research methods which use M-Mode can be emulated using AAA.

When you have a spline that follows the anatomy you want to measure, then there are 3 main approaches to emulate M-mode type analysis:

• Where does the spline (eg. the tongue surface) cross a fiducial line?
• How does a spline point (eg. the tongue tip) move within an arbitrary axis?
• Where does the spline (eg. the tongue surface) cross an ultrasound scanline?

For these examples, we will investigate the tongue tip touching the palate. We can begin with a recording for which we have used DeepLabCut to spline the tongue surface.

First, create a Fiducial Spline.

You can do this from the Edit Splines dialog, which you can open by right-clicking on the ultrasound image and selecting Edit Splines.

Then, click on New Spline and choose to create a Fiducial.

You can change the name of the Fiducial you have created by editing its name at the top of the dialog. For this example we will leave it as "Fid 1".

You can now see your new Fiducial on the ultrasound image behind the Edit Splines Dialog. When we prepare the analysis later, we will record the position along the fiducial at which the tongue surface spline intersects with the fiducial.

You can click and drag the points at each end of the Fiducial to move it. Please move it into the position you want for your analysis.
For this M-mode style analysis, please ensure that the 1^st fiducial point (the one with the dotted line coming out of it perpendicular) is below and the other point is above, as shown in the animation below.

Save and Close the Edit Splines dialog. Then, open the Analysis Values dialog by using the menu bar at the top of AAA to click Options→Analysis Values→Edit...

Analysis Values are time-series measurements and calculations for your recording. They can be used to see how things change over time, such as the tongue shape or the position of an anatomical feature.

The Analysis Values dialog shows a list of existing Analyses on the left, and a set of buttons to create new Analyses on the right.
For this tutorial, please click the Spline button on the right to create a new Spline based Analysis. The name you choose for it is important because you will refer back to this name later, so please choose an informative name. In this example, let's call it Palate_Distance

Use the controls in the centre of the Analysis Values dialog to specify what kind of analysis you want to perform. For this analysis, please choose the following:

• 2D Spline tab because we're working with 2D splines (which includes all DeepLabCut splines).
• Pos A analysis, which measures where a spline intersects with a Fiducial.
• Now use the Spline A and Fiducial N drop-down boxes to choose your tongue surface spline (DLC_Tongue if you used DeepLabCut) and the Fiducial you just created.

You've now created the Analysis Value.
Close the Analysis Values dialog.

If you are not currently using a Task Window which shows an Analysis Value Chart, please switch to one now. The "Analyse" task windows such as Analyse Ultrasonic contain Analysis Value Charts.

Your new Analysis Value will be automatically plotted on the Analysis Value chart. It is a graph over time of the position at which the tongue surface intersects with the fiducial. If you placed the 1^st fiducial point lower down and the 2^nd point higher up then a high value on the graph means a high intersection point closer to the palate.

If you see a sudden spike to zero on the graph then that means the spline no longer intersects with the fiducial, so the Analysis Value defaults to a value of zero. This occurs in the example above near the end of the recording, where the tongue tip retraction in the word "rain" moves the tongue outside the fiducial.

If you want to change the fiducial position, you can do this just by opening the Edit Splines dialog again, selecting the fiducial, and clicking-and-dragging the fiducial points. You don't need to re-make the Analysis Value.

Please note, however, that if you are using a Maths Analysis Value then if you change anything such as a fiducial position you will need to use the menu at the top of AAA to select Options→Analysis Values→Re-Calc Current Recording.

If you want to change the colours of your Analysis Values or smooth them, please read the Basics of Analysis tutorial.

This is very similar to the approach described above, but uses a different type of Analysis Value.

Please perform the steps in the approach described above, but when placing the Fiducial we will instead be using it as a set of axes used as the reference frame for our analysis.

Fiducials have a dotted line that comes out from the 1^st point perpendicular to the fiducial. This dotted line represents a y-axis line, and the solid line of the fiducial represents an x-axis.
Consider it like a graph, and consider how the point you want to measure will move within this coordinate space.

You can move the points of the fiducial to rotate this coordinate space as well, which is useful if you don't want the x and y axes to be in-line with the x and y axes of the screen.

Moving the 2 points of the fiducial closer together or further apart adjusts the scale of this coordinate space. The distance from the 1^st point of the fiducial to the far ends of its x-axis and y-axis will be 1.0 for the purposes of analysis.

In this example, let's suppose that we want to measure the vertical movement of the tongue tip. Because of this, let's place the fiducial horizontally so the dotted y-axis is vertical, as shown in the image above. We can adjust how far the 2^nd fiducial point is from the 1^st point so that the movement of the tongue tip will move within the area of the coordinate space we have created.

Next, when we create the Analysis Value we make a Spline analysis value that a 2D Spline y co-ord analysis.

This analysis measures how a single point moves within the 2D area defined by the Fiducial.

• Set Spline A to the spline you want to measure, for example DLC_Tongue for the DeepLabCut splined tongue surface.
• Set Knot A to the index of the knot you want to measure the movement of. The DLC_Tongue spline has 11 knots in it, which begins at index 0 for the most posterior knot at the vallecula, through to index 10 for the tongue tip.
• Set Fiducial N to the fiducial which you want to define the coordinate space.

In the resulting graph, a high value means the spline knot is positioned high in the coordinate space's y-axis. A low value means the knot is positioned low in the coordinate space.

You can also make an Analysis Value that uses x co-ord to measure the movement of the point within the x-axis of the coordinates defined by a fiducial.

The best way to do this is to use the same approach as in the first method described above, and place the fiducial so that one end of it is at the probe origin and the other end is at the far side of the ultrasound image. This most closely emulates M-mode based analysis of how the tongue surface moves along a single ultrasound scanline.

There is another way to achieve this, but the method below is not recommended if you are using 2D splines (such as DeepLabCut splines) because it is more difficult to do and cannot correctly measure retroflex tongue movements. If you still want to do it anyway, then instructions are as follows:

• Right-click on the ultrasound image and open the Edit Splines dialog.
• Click on the Batch tab at the top, then click on the Change Splines sub-tab below it.
• On the left, use the checkboxes to tick the spline you want to measure, for example the DLC_Tongue spline is the DeepLabCut tongue surface spline.
• On the right, click Convert to Fan Spline and select the ultrasound system you're using. AAA suggests the ultrasound system it thinks you have by highlighting it with a colored tick.

• Then, click the Change all button. This creates a new spline that's a copy of your current spline with the suffix _Fan added to its name. This leaves your previous spline unchanged so your data is safe.
  For example, if you were using the DLC_Tongue spline then the new copy is called DLC_Tongue_Fan.
• Click Save and Close.

The new spline is a Fan Spline instead of a 2D Spline. The ultrasound image is made up of 42 scan-lines, and a Fan Spline spans all 42 of them. For each scan-line, the Fan Spline has a knot which is positioned at a certain distance along the scan-line. The image below shows the positioning of the 42 knots for the converted DLC_Tongue in this example.

Next, create an Analysis Value in the same way as is done in the first method described above.

• Set the Spline A to the _Fan version of the spline you want to measure. In this example that would be DLC_Tongue_Fan.
• Set Fan N to the N^th scan-line of the 42 scan-lines in the ultrasound image. This is like choosing which scan-line in ultrasound M-Mode you want to measure. By default, the scan-line 0 is the leftmost, and scan-line 41 is the rightmost. You can define the fan and its orientation by right-clicking the ultrasound image and selecting Fan Setups...

If the your Analysis Value Chart is showing a flat line with an apparently zero value throughout, try the following:

• Use the menu at the top of AAA to select Options→Analysis Values...→Re-Calc Current Recording.
• Right-click the Analysis Value Chart and select Zoom→Zoom each to fit.
• Open the Analysis Value dialog using the menu at the top of AAA to select Options→Analysis Values...→Edit..., and then check all the names of the splines and fiducials you have specified in each Analysis Value. Check that their spelling is correct and they have the correct number of spaces in the names (eg. Fid1 and Fid 1 are different).