🧵 Basics of Recording, Analysis and Data Export - articulateinstruments/AAA-DeepLabCut-Resources GitHub Wiki

Contents

1. Welcome to AAA
2. Understanding the AAA user interface
3. How to create a new project
4. How to record new data
5. How to synchronise audio, ultrasound and video
6. How to annotate regions of data with labels
7. How to fit splines to data
   • Different types of Spline
   • How to prepare video data for splining
   • Fan Splines in polar coordinates
   • Tracking the Hyoid and Mandible
   • 2D Splines in euclidean coordinates
   • Batch splining many recordings at once
   • Editing many existing splines at once
8. How to analyse and measure data, and create live graphs
9. How to export your data and analyses, and create data visualisations
10. How does AAA manage data?

This guide will walk you through making your first recording and analysing it. For examples, we will follow Alice as she uses AAA with an ultrasound system, camera and microphone to record a participant speaking some sentences, then analyses her data using splines and exports her results.

When you first start AAA, it will probably look something like this:

AAA uses a simple drag-and-drop system to clip together useful tools and displays into one big window we can work with easily. In the above image there are 5 tools present:

• Prompt List (Used to create and curate prompts and load recordings)
• Ultrasonic Display (Shows a live readout of the ultrasound or the recorded ultrasound data)
• Audio Timeline (Shows a live readout of the audio waveform or the recorded audio data, which you can click in to navigate)
• Ultrasonic Timeline (As above, you can click and drag in this timeline to navigate a recording)
• Menu Bar (provides access to many functions and tools, and shows a list of task-windows)

You can click on the image above to see labels showing each tool.

^{The ultrasonic data recorded by EchoB, Micro, Ultrasonix and Art systems is held as raw scanline data in AAA and is distinct from the video stream which may contain camera images or images imported from other ultrasound systems.}

AAA always starts in a demonstration project. It is not possible to record or import data into a demonstration project. A new project must be created (or a previously created project must be opened).

Alice is just starting her new research, so she wants to create a AAA project for it. She uses the menu at the top to select File → Create/Copy Project and then fills in the project details and the details of her participants. When she creates the project it is empty and contains no prompts. In AAA you must have a prompt to record. To create some prompts, Alice right-clicks the empty prompts-list and selects Edit → Edit Prompt List.... This process is shown below.

Alice is ready to record her participant. Using the prompt-list she clicks on the prompt she wishes to record and tells her participant to start speaking when they hear a beep-sound; she then presses the button at the bottom of the screen, and AAA plays a beep as it begins. The recording can then be stopped by pressing the same button again, or it will stop automatically after a time has elapsed: this automatic timeout can be set in the menu bar at the top, Options → Settings → Utterances.

^{You can set the maximum duration of individual prompts by adding a special suffix to the prompt. Just add a space and then the time in seconds in square brackets after the prompt. For example, the prompt "Hello world." could be set to a 3 second limit by changing it to "Hello world. [3]". This takes priority over the maximum duration set in the AAA options.}

If Alice does not select a different prompt, she can press the record button again immediately to begin a second recording which will be stored in addition to previous recordings. All recordings she has made are displayed as filled checkboxes with an to the left of each prompt. She can click on each of them to load the corresponding recording for review or for analysis. After Alice has shown the recordings to her participant, she clicks on the text of a prompt to prepare AAA for a new recording.

Alice decides to move on to her next plan, which is to record simultaneous ultrasound, audio and camera data; she has all 3 recording devices correctly set up but AAA is not showing any camera data. This is because AAA does not currently have any camera video related tools and displays visible, so Alice changes to a different task-window by using the horizontal row of Task Windows at the top of the screen and choosing Record Ultrasonic plus Video.

This Record Ultrasonic plus Video task-window contains tools and displays we are familiar with from before: the prompt list, prompt display, ultrasound display and timelines are present as before, but now in addition we have a video display and a video timeline too. AAA could show every available tool and display simultaneously but there are so many that it would be very cluttered and hard to use, so by using task-windows we can show only the tools and displays we need for the task we want to do. If you ever want to make your own task-window because you're not satisfied with the existing ones, or if you just want to modify an existing one to suit your needs better, you can do so easily using the task-window designer.

Alice's recordings from earlier are still present in the prompt list. To keep things organised, Alice decides to make a separate prompt list for her simultaneous ultrasound, audio and video data, so she edits the prompt-list again by right-clicking it and this time she uses the menu at the top to select List→New... and creates a new empty list. She then uses the text box to the right to type in her new prompts, with each prompt on a new line, and presses OK when done. The recordings from earlier are still present, because those recordings are attached to this client (participant). Alice now selects a new prompt in the list by clicking on its text, and presses .

Alice has finished some recordings of simultaneous ultrasound, audio and camera video data, and now she wants to synchronise them exactly. Because she has been using the Micro research system to record the data, she has her recording hardware wired up with a SyncBrightup and a PStretch: these send signals to AAA which allow it to respectively synchronise video and ultrasound data to the audio recording. You can read more about how to set these up in the first time setup guide.

Right-click on the ultrasound image and then click Ultrasonic Setup → Sync Recording...

The Sync Recording dialog provides options that control how AAA synchronises new recordings and existing recordings.

Synchronisation relies on the PStretch pulses being present and easily visible on an audio channel. If you're using a 2^nd or 3^rd generation Focusrite Scarlett Solo then the sync signal should be on audio channel 2. If they are on a different channel or if the recording gain was low and the pulses are barely visible in the waveform display, then the channel setting may need to be changed or the threshold value lowered. Try clicking the Synchronise the current recording button and see if the ultrasound frames line up with the audio sync pulses.

Alice wants to export the ultrasonic data without loading each recording manually. So to ensure all the recordings are synchronised she uses the batch synchronisation process in the synchronisation dialog. She selects the date or range of dates of the recordings to be synchronised. She then selects either the current client or all clients in the current project and clicks the Synchronise all recordings that match these criteria button.

If the video sync signal is also recorded at the start of channel 2 then the ignore the first X ms of the sync signal should be set to X = 800.

If an ultrasonic sync signal is not present at the time of recording, the Synchronisation signal is missing checkbox can be checked and AAA will assign the ultrasound frames times according to when they are acquired. Note that soundcards usually introduce a lag which can be as much as 100ms so ideally this lag should be estimated by a tap test and then the Sync Parameters - Offset set accordingly.

To synchronise video, Alice selects the first recording that she wishes to sync to load it, then she right-clicks on the video display and chooses Video Setup. In the tab Sync Hardware, she ensures it is set to Sync-Bright-up and the channel number is set to the audio channel with the video sync pulses. Then in the Sync New Recording tab she sets the rate to 29.97 and then clicks Sync and offset. She closes the dialog, and clicks and drags within one of the three timelines (ultrasound / audio / video) to observe that the audio beep occurs at the exact same time as the white square appears in the corner of the video. The audio timeline may only be displaying one of multiple audio channels, so you can switch the visible audio channel using the buttons on the status bar at the very bottom of the screen.

A synchronised video should briefly show a white square in the top-left corner of the video for the duration of the large pulse on the audio sync channel (one of the two sound channels), as shown below:

^{How does it work? One of the two audio channels records the microphone, and the other has an electronic signal on it. The signal starts with a beep, then after a pause it starts a tone. The video contains a small white square that appears in a corner for a few frames. The synchronisation matches the starting beep to the white square, and then matches the ultrasound to the tone.}

Most often, Alice will use the Batch sync option to synchronise all the video recordings with a single click of the sync/offset/set rate button. Before she clicks this button she sets the date of the recording session or range of dates of a set of sessions and whether to process the current client only or all clients in the project. She selects the De-interlace option to separate each frame into two separate images, doubling the frame rate from 29.97 to 59.94. She sets the rate to 59.94 in accordance with this choice. Then she clicks the sync/offset/set rate button. If she chooses not to de-interlace then she leaves the rate at 29.97.

Occasionally, the synchronisation will not align the white square with the audio pulses. This may be because the sync process is restricted to looking in only the first 500ms for the sync signal. This limit can be extended by clicking the Advanced checkbox in the Sync Hardware tab and editing the Flash must be within X ms of the start setting X to perhaps 1000

If you want more information about synchronisation you can read more about it here.

Alice wants to investigate phonemes for her research, so she wishes to annotate regions of her data with labels showing the phonemes. For this, she uses the annotation tool in AAA, which is a powerful tool because it allows for many operations to be done on specific annotations, for example exporting splined videos and other subsets of data for all instances of a retroflex ɻ.

First, Alice chooses a task-window which contains the Annotation tool, such as Analyse Ultrasonic plus Video. The annotation tool lists all annotations for your current project, which you can filter and edit.

To add an annotation, click and drag within the audio timeline to select a region, then either right-click within the audio timeline and select Add Annotation (and either select an existing annotation or choose <new annotation>) or click on the Add button in the Annotation tool. Next, type the label for the annotation in and then press the enter-key.

To zoom into a region of the recording to allow more precision, you can click and drag in the audio timeline and then right-click the selected region and click Zoom; to return to seeing the full recording, click anywhere in the audio timeline once so there is no region selected, then right-click and click Zoom similarly to before.

Alice wants to investigate tongue shape during specific phonemes, so she needs a way to measure the shape and position of the tongue surface. In AAA you can create splines to follow the contour of the tongue surface or the positions and movement of other anatomy, and you can either place them manually or fit them automatically.

AAA offers 3 different kinds of spline:

• 2D Splines use euclidean coordinates (measurements in x, y and pythagorean distances) and are easy to fit to the tongue automatically using machine learning. They are appropriate for any kind of data and most kinds of analysis.
• Fan Splines use polar coordinates originating from the ultrasound probe: measurements are in angle and distance from the probe. They are appropriate only for ultrasound data, not video camera data.
  Fan Splines are an old style of analysis which we no longer recommend unless you have specific need of them: most analysis can be done more easily with 2D Splines.
• Fiducials are special 2D splines consisting only of a straight line. They are used as references for measurements involving other splines, for example acting as a ruler against which to measure something, a boundary to be crossed, or to define a frame of reference. For example, you can use one to represent a bite-plane.

Both types of spline can be used for shape-based analysis, for example calculating the dorsum excursion index (DEI) or the number of inflections (NINFL), and both types of spline can be manually adjusted at any time.

^{You can easily convert 2D Splines into Fan Splines, and vice-versa.}

You can convert Fan Splines into 2D Splines, and convert 2D splines into Fan Splines, but doing can be partially destructive to your data.
2D splines can double-back on themselves but Fan splines cannot: converting 2D to Fan will preserve only the closest spline surface to the probe origin. Furthermore, there cannot be more or less knots than fan lines, so additional knots may need to be created at arbitrary radii, and knots beyond the range of the fan will be removed. Fan splines always have a number of knots equal to the number of fan lines, which is a fixed value of 42, but 2D splines can have any number of knots from 2 to 50. When converting Fan to 2D you will be prompted to choose the number of knots: if the number of knots you choose differs from the number of fan lines, the knot positions will be calculated as intelligently as possible, but choosing fewer knots than fan lines will still be fundamentally destructive to the resolution of your data. Because of all these reasons, it is recommended that you carefully consider the data analysis you wish to perform before choosing whether to use Fan Splines or 2D Splines.

To convert, right-click on the ultrasound or video image which has the spline you want to convert, then click Edit Splines....
Use the tabs at the bottom of the dialog to select the spline you want to convert.
Then, click the Convert button.

In AAA you can automatically fit splines using powerful tools.
AAA natively supports DeepLabCut, a free open-source state-of-the-art machine learning tool, which can fit 2D splines to your data.

You can also fit 2D splines using any other external tool of your choice and easily import them into your AAA project.

We recommend using 2D Splines for most kinds of analysis.
If you want to use Fan Splines anyway, you can click here to learn how to use them.

Alice has finished batch processing many recordings with a spline fitting algorithm and now she wants to make the Hyoid and Mandible splines invisible in every recording without having to do it manually. This is easy to do.

In the Edit Splines dialog, in the Batch tab at the top there is a Change Splines sub-tab near the top. This lets you change many properties of splines, including their appearance, visibility and name, or delete them.

The list on the left shows every spline for every client in your currently loaded project. You can choose to only affect a subset of your project's splines by using the Choose Recordings sub-tab above and proceeding to Filter the data, for example only affecting splines in recordings between certain dates. In the Change Splines sub-tab you can select splines on the left by clicking on their individual checkboxes, then on the right choose what you would like to change about them. You can select multiple things on the right and they will all be applied to all the selected splines on the left when you click Change All. Be careful because you cannot undo any batch changes you make; it's always good practice to keep incremental backups of your work.

The Analysis Values system is a powerful suite of data analysis tools in AAA. It allows you to create many different measurements of recorded data and their relationships to each other. You can create and format graphs of Analysis Values and create your own mathematical formulae to modify Analysis Values or combine them.

Task Windows each contain only some of the tools and displays available in AAA. Before you start working with Analysis Values, it is recommended that you move to a Task Window that contains Analysis Value displays and graphs, by clicking on one in the menu bar at the top of the screen. All of the following Task Windows contain Analysis Value tools and displays by default and can be used to analyse any type of data:

• Analyse Ultrasonic (also contains Audio and Ultrasound displays)
• Analyse Video Ultrasound (also contains Audio and Video displays)
• Analyse (also contains an Audio display)
• Analyse Ultrasonic plus Video (also contains Audio, Ultrasound and Video displays)
• 2-Screen Ultrasound and Video (also contains Audio, Ultrasound and Video displays)

Next, in the menu bar at the top of the screen, click Options → Analysis Values... → Edit...

The Analysis Value dialog might look complicated when you first see it, but it's quite straightforward.

• On the left is a list of all Analysis Values that you have created in the current project. (Analysis Values are shared across all clients within a single project. If you've created a new project then it will have automatically copied all Analysis Values from the previous project you had loaded. You can easily delete a selected Analysis Value using the Delete Analysis button in the bottom right).
• The buttons on the right side of the dialog let you create new Analysis Values: Each button represents a different type of data and each contains many different Analysis Values that can be used with that data type.
• The main area in the middle lets you learn about and choose the type of data analysis to perform.

Alice is interested in the contraction of Genioglossus muscles attached to different points along the tongue's surface during production of particular phonemes. She has recorded mid-sagittal ultrasound data and has chosen to use DeepLabCut to fit 2D splines to her data. Thus, certain analyses will suit her research intents and her methods more than others.

There are many Analysis Values you can create to measure and compare different things. Every one provides its own explanation and diagram showing what it does. You can see this in the area below each Analysis Value's settings. (For example in the image above you can see the 2D Spline Area analysis is selected, and so a diagram is visible explaining the functionality of this selected Analysis Value).

When Alice fit 2D splines to her data using DeepLabCut, she splined the tongue surface and also the hyoid, the mandible and the short-tendon. In order to measure the Genioglossus muscle contractions, she wants to measure the distance from the short-tendon to various points on the tongue surface. Because this is a measurement of spline data, she creates a Spline Analysis Value.

Alice clicks on the Spline button on the right to create a new Spline Analysis Value and she gives it an appropriate name. (If you wish to rename an Analysis Value, you need to delete it and remake it). She ensures that she has her new Analysis Value selected in the list on the left, then because she's working with 2D splines she selects the 2D Spline tab at the top, and because she wants to measure the distance between two knots (one on the tongue and one on a spline that defines the short-tendon), she clicks on Dist AB Knot to measure the distance between two spline knots. She then uses the drop-down boxes to specify the splines, and inputs the index of the specific knots (the first knot is index zero).

Alice repeats this process to create a total of 5 Analysis Values corresponding to her chosen tongue regions: Root, Body, Dorsum, Blade and Tip. Each Analysis Value she has created now measures the distance from a tongue region to the short-tendon.

After closing the dialog, the new Analysis Values become automatically visible overlaid together on line graph in AAA, showing the change in each Analysis Value over time. However, at the moment the formatting is very simple, homogeneous and hard to read. To the left of the graph, the Analysis Values present in the chart are listed.

You can easily improve the appearance of the chart. Right-click on the Analysis Value names to the left and click Edit Chart.... You will see a dialog showing all Analysis Values in your project with a checkbox to the left of each. You can tick or untick them to show or hide them on the graph. To the right is the settings for the currently selected Analysis Value.

• The Axis tab at the bottom allows you to fine-tune the vertical axis. (The horizontal axis is always tied to the visible region of the recording, which you can change by clicking and dragging the mouse cursor in the audio timeline, then right-clicking within your selection and clicking Zoom; you can then drag the sides of the selected region around to expand or contract them).
• The Line tab at the bottom lets you change the appearance of the selected Analysis Value in the chart, for example changing its colour, style and thickness.
• The Chart tab lets you name the current chart, and also create a new chart with the Add button. New charts will be visible adjacent to your other chart(s), which allows you to split your Analysis Values across multiple charts for clearer visualisation and comparison.

When Alice finishes making changes, she closes the dialog then refreshes the chart to ensure her latest changes are visible: You can click Options → Analysis Values... → Re-Calc Current Recording to make AAA recalculate every Analysis Value to incorporate any changes that's been made. Then, right-click on the Analysis Value chart and click Zoom and choose an option. Alice is more interested in relative comparisons than absolute ones, so she clicks Zoom → Zoom each to fit which normalises every line on the graph show its minimum as the lowest value within the visible graph area, and show its maximum as the highest value within the visible graph area.

It's good practice to make a habit of recalculating and zooming any time you make a significant change to an Analysis Value.

Alice has formatted the different Analysis Values in her chart to look clearer and more readable, but she's worried that the estimation error noise in the DeepLabCut splining is making it hard to see the trends. She decides to create a smoothing filter: this can be done using Maths Analysis Values.

Alice re-opens the Analysis Values dialog from the menu at the top (Options → Analysis Values... → Edit...). She creates a new Maths Analysis Value by clicking the Maths button on the right and gives it a descriptive name for its intended purpose. A Maths Analysis Value consists of a free text field where you can type in a formula.

All Analysis Values of any type must ultimately evaluate to a scalar. For Maths Analysis Values, you can see a list of all operators and functions you can use by selecting a Maths Analysis Value and clicking the question-mark (?) button near the top.

Maths Analysis Values also include an optional smoothing filter (which is applied after your specified equation resolves but before any differentiation). You can click on the Edit Filter button below your equation and choose a smoothing filter using the tabs available at the top of the new dialog.

By creating a Maths Analysis Value it will be automatically added to your existing charts, and it exists separately from any Analysis Values that are used in its equation (even if it only exists to smooth another Analysis Value). You may wish to format your new Maths Analysis Value to make it more presentable on the chart.

After trying the different smoothing filters, Alice settles on a SavGol (m=2) filter operating on a sample window of 40ms, and as such she creates a Maths Analysis Value to act as a smoothed version of each of her Spline Analysis Values. She edits the chart to show her new Analysis Values instead of her original Spline Analysis Values, and formats them to be visually easier to read.

Next, Alice wishes to create a metric that describes the degree of retroflection in the tongue. This is what Maths Analysis Values are really useful for. She re-opens the Analysis Value dialog, creates a new Maths Analysis Value which she names descriptively, ands then she writes an equation for it.

In the chart editor, she uses the Chart tab at the bottom to Add a new chart, and checks the box for her new Analysis Value "Retroflection" to load it into the chart. By using multiple charts and distinct readable formatting, Alice can keep specific Analysis Values overlaid for visual comparison while keeping others separate.

Alice has finished analysing her data and wishes to export her results to put in a paper. AAA provides multiple tools for exporting data in useful formats:

• "Publisher": Make visualisations of your data, such as combining labelled graphs, timelines and other information directly from your project into a single image.
• "Make Movie": Design animated videos that show multiple types of data concurrently changing over time in a clear, readable layout.
• "Export Data": Get tables of Analysis Values and raw data values as comma-separated-variable (.CSV) files for putting in an appendix or for further analysis (eg. R, SPSS, Excel).
• "Export Files": Export your raw recorded data in its entirety out of AAA for backup or for importing into other software.

So far, Alice has used many tools in AAA to help her find interesting patterns in her data, and now she wants to present some examples of her findings. The Publisher in AAA is a fast, easy way to make plots of data and automatically keep data tied to common axes for comparison. You can arrange and format data visualisations and then export them as images to put in a document or import into other software.

You can open the Publisher from the menu bar at the top of the screen by clicking File → Publisher...

When you first open the publisher it might initially look confusing, but its controls are simple and consistently laid out.

• At the top is a row of menus you can use to load in new Elements such as data plots, labels, etc.
• On the left of the Publisher is a list of all Elements you've added to your canvas. When you first start you won't have any Elements yet so the list will be empty.
• At the bottom of the Publisher is a grey region where information and settings for your currently selected Element will be displayed. This will also be blank while you have no Elements.

The publisher lets you plot data and analyses over time, such as audio spectrograms and Analysis Values. For these plots, you must first select a region of interest in your recording's audio timeline (outside the Publisher) by clicking-and-dragging within the timeline with your mouse cursor. You can then add the appropriate element (eg. Analysis plot) from the Publisher menu.

Alice selects a region of interest in her timeline corresponding to her participant's spoken word "orthodontist", and creates a plot of three of her Analysis Values. She creates an axis from the Axes menu and then ties the three Analysis plots to it together (using control-click, select both the axis and each plot to tie, then click Axes → Tie Plots to Axes → x-y Axis : this means that they share the same X and Y axes, so are directly compared).

Alice also adds an audio spectrogram, and ties it to the same axis as the other Elements, however, she only ties it to the x-axis: this means that it shares the same x-axis (the time axis) with the three Analysis plots and so can be compared temporally, but can be freely moved and resized in the y-axis.

For plots of data over time it's useful to tie only the x-axis of a group because that's the time axis, so you can freely arrange and scale the plots vertically to your taste and they'll be guaranteed to be lined up in the time axis using the same scale.

Axes can be rescaled and renamed by selecting them and using the settings panel at the bottom of the Publisher window. Any changes you make to an axis will affect all Elements tied to it (eg. Analysis plots and audio waveforms) since they share the same axis.

You can also format many elements such as the appearance of lines in a line graph (eg. colour, font, text size, or rename then Element) by selecting the Element in the list on the left and then using the Plot menu at the top.

Alice is satisfied with her plot and wants to import it into other software outside AAA. She clicks File → Save Bitmap... and chooses a resolution to save her canvas to as a raster image. If you want to copy your canvas as an image to your Windows clipboard, you can click Edit → Copy and choose a resolution; you can then paste it into 3rd-party software such as a Microsoft Word document or an image editing program.

You can access this by right-clicking on an ultrasound or video display and selecting Make Movie...

You can design a video of your data by arranging several different types of data visualisation into a layout of your choice. You can click-and-drag elements from the list on the right to arrange in the window display in the left. Each element is a visualisation of data that changes over time in your recording: in your movie, they will always be synchronised as long as your data is synchronised, and as such will be shown changing concurrently.

Click here to see how to easily synchronise data.

You can choose the location to save your movie by clicking Browse at the top-right of the dialog, or by typing a file-path at the top. You can also select playback speed, and video dimensions, and frame-rate (higher values result in smoother looking video but also much larger file-size).

The Background colour is used in areas where the aspect ratio of images (eg. your recorded video) differs from the space you've given it in your chosen layout.

Here is an example movie made using the layout and settings shown above:

You can access this from the menu bar at the top of the screen by clicking File → Export → Data...

This dialogue creates a tab separated ascii file where each row is a time point and each column contains a data value.

The data export dialog is divided into 4 main tabs at the top. First, use the Export File tab to choose the name and location of the .csv file to create which will contain the exported data and analysis values. For .csv export, ensure that the checkbox Export to file is ticked.

Note : the Columns tab will only appear if the Export to file checkbox is ticked.

Next, go to the Filter tab: you can choose to either Export Recordings for whole recordings or Export Annotations to export only the data within the timespan of specified annotations, then click Change Filter and specify the conditions a recording or annotation must meet to be exported.

Next, in the Rows tab you must specify which time points should be included within each of the previously specified recordings or annotations. Recorded data are sampled at discrete timepoints referred to as samples or frames. In between the frame sample times, values are interpolated linearly by the software. By selecting to export every sample of your primary data type (ultrasonic, video, EPG etc) you avoid interpolated values. It is often desired to export the midpoint or N equally spaced points derived from an annotated segment. It is also possible to export the timepoint within each annotated segment that has the maximum value of a specified variable (e.g. a distance value derived from tongue splines or palatal contact derived from EPG).

If you wanted to export a row for every frame of a particular type, you would choose All Video Frames or All Ultrasonic Frames. If you wish to export only the data from your spline-based Analysis Values, you may wish to select All Spline Keyframes to avoid data frames which have no corresponding spline or which contain interpolated values.

The Data Summary sub-tab lets you choose if you want to add some simple statistics to your exported data file. If you are planning to use tools external to AAA to analyse your data after export, it is probably better to choose Just write data.

You can access this from the menu bar at the top of the screen by clicking File → Export → Files...

This dialog lets you choose one or more types of data stored in AAA to be saved as files on your computer in common standard file formats, making them easy to import into other software or backup. Any type of data you can record using AAA can be exported like this.

Note: This method exports the entire recording. To export only annotated regions, use the Export Data dialogue.

The tabs along the top of the dialog each correspond to a type of data, and you can visit each tab to choose whether or not to export that type of data. Each tab also has settings you can use specify data formatting, or choose a subset to export, if desired.

Files

To export, you must choose a folder as the location to save your exported data to by using the Browse button or typing in a file path. You must also click the button `Select Clients/Sessions to Export' and tick the checkboxes for every client you want to export the data for. (If you select the current client and that client has more than one recording associated with it then you can also specify a subset of those recordings to export).

For naming your exported files, you have two options:

1. You can type a file name 'stem' which will prefix all exported file names, and will automatically increment a counter for every recording exported. You can choose the number it will start at and how many digits should be used. eg. Stem File, start 1 and numerals 3 would export the first 9 files as something similar to the following:

   File001.avi
   File001.spl
   File001_Track0.wav
   File002.avi
   File002.spl
   File002_Track0.wav
   File003.avi
   File003.spl
   File003_Track0.wav
   

2. You can check the box Use Prompt to prefix each file with the full prompt text for its recording, and increment the numbers for each recording within the same prompt. If you have very long prompts this will create very long filenames. eg. Use Prompt, start 1 and numerals 3 would export the first 9 files as something similar to the following:

   Lorem ipsum001.avi
   Lorem ipsum001.spl
   Lorem ipsum001_Track0.wav
   Lorem ipsum002.avi
   Lorem ipsum002.spl
   Lorem ipsum002_Track0.wav
   Dolor sit amet001.avi
   Dolor sit amet001.spl
   Dolor sit amet001_Track0.wav
   

What order are files exported in, though? Unfortunately it's complicated. If you are exporting only one client, and have chosen a subset of sessions and recordings to export, then it will export it in the following order: For each repetition, for each prompt: export. (Where repetition here means a recording you make of the same prompt on the same client on the same day)

Thus it would be ordered like:

Prompt1-Rep1
Prompt2-Rep1
Prompt1-Rep2
Prompt2-Rep2

If, alternatively, you want to export multiple clients at once, then the order the files will be exported in will just be whatever order they were recorded in chronologically. For this reason it is advisable to check the box Use Prompt to name each exported file according to its prompt.

To cope with many different data streams (audio, ultrasound, video, EPG, EMA, MRI, VICON, Glottograph) recorded at different sample rates and to permit fast annotation queries and data analysis, a proprietary database forms the foundation of the software. Data can be analysed within AAA or exported in many common file formats for analysis by other software.

AAA can load any number of databases (referred to hereafter as projects), one at a time. Each project is located in a separate folder and consists of a core database file (Data.aa0) and a sequence of Chunk files (Chunk_1.aa0, Chunk_2.aa0, etc.). The Data.aa0 file contains most (but not all) of the data associated with the project. Some data such as audio, video and EPG data is too large for the computer to handle as one big file, so it is instead stored separately as Chunk files. Chunks must NOT be separated from their Data.aa0 file! The Data.aa0 file keeps a list of all the Chunk files it expects to see, so if you rename, delete or move Chunk files it will corrupt your entire project and make it unusable.

WARNING: Don't copy parts of projects! If you want to move or back-up your work, you must back-up everything in the project folder together at the same time. You must NOT modify or back-up a subset of the files in a project folder or it will corrupt your project and you won't be able to use it. When you use AAA, all the Chunk files get regularly modified and reassigned with new names. Microsoft Windows can sometimes fail to correctly show the date a Chunk file has been modified, and this can mislead people into thinking they can back up only a subset of their data.

Version-control software such as Git is a much safer and more effective way to back-up incremental changes to AAA projects, but you must always commit every change that AAA makes to the files. If you only commit a subset of changed files then you will create a corrupt commit.

Every frame or sample of data is time stamped to the nearest 100,000th of a second (10 microseconds). AAA can even cope with data that does not have a regular sample rate. The only exception to this is audio data; audio data is the ground truth for all other data streams. Audio data must have regular sample intervals and has a sample rate associated with it which is taken from the recording device. Even it this rate is inaccurate, it becomes the clock to which all the other streams are synchronised. A recording must have an audio track: other data streams cannot be recorded by AAA without audio.

The AAA software allows you to

• Create as many projects as you wish.
• Transfer data between projects.
• Import data in standard file formats. (*.wav, *.epg, PRAAT textgrid, *.avi, etc)
• Export data to standard file formats.
• Calculate values from data and export them and/or plot them.
• Create publishable quality graphics from the data.

All data within a project can be compared and analysed as a group. Each recorded data channel can be filtered, smoothed, differentiated, combined with other data channels, etc. Key features like peaks, valleys, midpoints etc can be found automatically and labelled.

Data values, durations of labelled regions and many other pieces of information can be exported to Excel or R for charting and statistical analysis. Sophisticated queries can be carried out within AAA on labelled data to extract only the information that is required.

Movies of articulatory data can be created for use in presentations and played in slow motion if required. High quality charts suitable for 600dpi journal and book publication can be generated from the various displays.

AAA provides a powerful tool for multichannel speech analysis but we are always on the lookout to improve the utility of the software. So if you have ideas for new features please get in touch and we'll see what we can do.