🧵 How to synchronise ultrasonic data recorded with Micro, EchoB or Art system - articulateinstruments/AAA-DeepLabCut-Resources GitHub Wiki

Throughout AAA and this documentation, the term "sync" is used as a short way of writing "synchronisation".

If you have a research ultrasound system, then AAA synchronises recordings using an electronic signal that is created by your ultrasound system which is received by your computer as sound on one of your microphone audio channels. AAA listens to this electronic audio signal and uses it to place each ultrasound frame at the exact time the ultrasound machine recorded it. This process requires the Pulse Stretch ("PStretch"), which is a device made by us at Articulate Instruments that converts the ultrasound machine's signal into a format that AAA can understand.

If you have a visual feedback ultrasound system, then AAA approximately synchronises your data from the timings of when AAA receives ultrasound images, and you can adjust it manually to improve the accuracy.

^{The sync signal consists of many tiny pulses. Each time the ultrasound system records an ultrasound image, it produces one pulse. AAA places one ultrasound image at each pulse. Like all electronic signals which are transmitted through wires, the sync signal can experience noise, which can look like tiny pulses. To stop AAA accidentally interpreting noise as a real sync pulse, AAA expects each sync pulse to be of a certain height and length.}

---

1. First, please ensure that you have successfully completed all parts of the First Time Setup tutorials.
   Please pay particular attention to the hardware setup, as synchronisation cannot work if there are any mistakes in how your equipment is connected together.

---

2. Run AAA.

---

3. Right-click on the ultrasound image and then click Ultrasonic Setup → Sync Recording...

   

   The Sync Recording dialog provides options that control how AAA synchronises new recordings and existing recordings.

   

The process of synchronising is easy. Please follow through each of the questions listed below.

---

Articulate Instruments provides 2 different kinds of ultrasound systems.

• Research systems provide a synchronisation signal which comes out of a "Frame Strobe" port on the ultrasound scanner. If you have this then you also need a PStretch to convert the signal into a format that AAA can understand.
• Visual feedback systems do not provide a synchronisation signal and so cannot be automatically synchronised. If this is the case, then AAA can estimate the synchronisation instead.

If you have a research system then please use the controls shown below to say that you have a synchronisation signal.
If you have a visual feedback system, please use the controls shown below to say that you don't have a synchronisation signal, then skip to the section at the bottom of this page.

In AAA version 221.5.1 to version 221.3.9:

In AAA version 221.3.8 and earlier:

---

The audio channel that the sync signal appears on is determined by how you connected your hardware. If you did it correctly according to the hardware setup guide, then it should be on Channel 2.

In AAA version 221.5.1 to version 221.3.9:

In AAA version 221.3.8 and earlier:

---

Any pulse smaller than this height is assumed to be noise and is discarded. The height of your sync pulses is controlled by the Gain dial on your Focusrite soundcard for the audio channel that the PStretch is connected to. Please turn this dial clockwise as far as possible without it turning orange or red; it should be the highest value possible while still being green. In other words, it should be as high as possible without clipping.

The height of a sync pulse in AAA is not measured with any units, so please experiment with different values to find what works for you. If your Gain is as high as possible without clipping then a threshold of 4000 should work well.

In AAA version 221.5.1 to version 221.3.9:

In AAA version 221.3.8 and earlier:

---

Video synchronisation also uses pulses on an audio channel. The most common hardware setup puts the video sync pulses on the same audio channel as the ultrasound sync pulses. This means that AAA needs to know where to expect video sync pulses so it doesn't misinterpret them as ultrasound sync pulses. The video sync pulses happen before the ultrasound, and AAA waits for the video sync to happen before starting the ultrasound recording. You can control how long AAA waits to start recording ultrasound using the ultrasound settings.

This setting tells AAA what is the earliest time it should expect to see the ultrasound sync signal.

If you're not recording any video at the same time as ultrasound, then set this to zero.
If you are recording video at the same time as ultrasound, set this to 800, and then use your ultrasound settings to set an acquisition delay of 1000 ms.

In AAA version 221.5.1 to version 221.3.9:

In AAA version 221.3.8 and earlier:

---

To synchronise the recording you have currently selected, you can click the button shown below:

In AAA version 221.5.1 to version 221.3.9:

In AAA version 221.3.8 and earlier:

---

If you want to synchronise many recordings as a single batch process, you can choose which recordings you want to be synchronised:

• You must specify a range of dates. Every recording that is on one of these dates or in between them will be synchronised.
• You must choose if you want only the recordings that were made for your currently selected client to be synchronised (the current client only box is ticked) or if you want the recordings for all clients between the dates to be synchronised (the current client only box is not ticked).
• You must choose if you want the ultrasound frames to be moved to the right (further into the future) relative to the audio by an extra amount, using the shift/offset number. If you don't want them to be moved by an extra amount, set the shift/offset to zero.

In AAA version 221.5.1 to version 221.3.9:

In AAA version 221.3.8 and earlier:

---

If you have selected the option that you don't have a sync signal, then AAA will automatically try to synchronise your data by estimating the timings. This will only happen for new recordings that you make after you have changed the setting: old recordings will not be automatically fixed. For new recordings, you don't have to change these settings unless you think AAA's estimations are bad.

^{An easy way to know how synchronised audio and ultrasound are is to look at lingual plosives such as /k/, /t/ and /q/. At the release, does the tongue movement coincide with the sound?}

Fixing a recording that is not properly synchronised involves 2 steps:

• Tell AAA to evenly space the ultrasound images at equal intervals so it matches the rate at which the ultrasound system recorded them.
• Move all ultrasound images left or right so they align with the audio.

First, tell AAA what at what frequency your ultrasound system produces images. This lets AAA know how far apart to space the ultrasound images.

You can change the frame rate number to the number of frames per second that the ultrasound system records (ie. the sample rate in hertz). You can also click Detect rate from current recording to set the frame rate number to the average frame rate in your current recording.

In AAA version 221.5.1 to version 221.3.9:

In AAA version 221.3.8 and earlier:

Next, shift/offset all the ultrasound frames left or right relative to the start of the audio.
The offset might need to be individually adjusted for each recording you make. Without a synchronisation signal, it's possible for the computer to cause slight delays in when each kind of data gets recorded. This will be different for different computers, but it's likely to be approximately between 100 to 200 milliseconds.

It is recommended that you test it at the start of every day that you plan to record, and set the shift/offset accordingly. Every new recording you make for the rest of the day will then use that offset.

In AAA version 221.5.1 to version 221.3.9:

In AAA version 221.3.8 and earlier: