KAT7_Transient_Detector - david-macmahon/wiki_convert_test GitHub Wiki

There is currently an effort underway to equip the KAT-7 telescope with a transient detector. We aim to do this entirely in software.

System Architecture

The following diagram illustrates the proposed interconnect: Kat7_transient_detection.png

KAT-7 interface considerations

KAT-7 uses SPEAD (UDP-based) for data interchange and KATCP (TCP-based) for control and monitoring.

ROACH emits a SPEAD stream that goes into GPU at 800Msps, 8 bits. This is written into circular buffer in RAM. GPU's do PFB on this (already written). Trigger mechanism selects a window of time-domain data and dumps it to disk, then frees up RAM buffer space.

Data packet sizes

Suggest 4096 bytes of content (8 bit samples) plus headers. More requires additional buffer space in FPGA (currently at a premium).

Required Metadata

What is the frequency required for this metadata? Broadcast on change (good for slowly changing stuff). On connect, will give current sensor level.

Antenna ID

A globally-unique antenna identifier.

Antenna mode

Is the antenna tracking the target? Which target? Wind stow?

Antenna pointing information

If this information is available.

Antenna locations

Current antenna locations on the ground.

System delays

What are the current system delays? With this metadata, this software will need to realign the incomming datastreams from each antenna. Coarse delays (1.25ns ADC sample clocks) ok for this work. Timestamps are provided so that you can realign the data streams.

Timestamps

Correlator tracks ADC sample counts since last reset and puts this counter in each packet. 1.25ns resolution. Reset (ie absolute time) accurate to 15ns. Need to know when reset occured and sample rate. -> SPEAD metadata provides a timestamp scale factor and offset.

RF analogue attenuator levels

Can change while recording?

LO frequencies

Can change while recording?

RFI flagging

Is there a system on site that tells us about RFI? RFI is generally not such a problem. Currently detect and flag correlator output data. Not particularly useful for transient events. Will be per-frequency channel. Long term, will have satellite lookout too. Keep above 1705MHz and there's little to no RFI. Hopefully this system will detect transient events and can actually feed these back into rest of KAT-7 system. It will also be useful to have a "bad data flag" of some kind in the datastream. This could be used to flag RFI but also periods when the data is known to be uninterpretable (for example, during slewing operations)

Dedispersed Data Protocol

The default scheme produces ~1ms time series dedispersed to various candidate DMs. It yields a 2D matrix of samples [frequency x time] with the following metadata. I envision the entire chunk (with the results of all DM trials) is broadcast to all machines on the subscribing network. This could enable maximum flexibility with peer-to-peer or spoke-and-hub detection systems, and whatever subsequent processing you'd want.

However, the network might not support these data rates, and It's an open question of whether we'll want to transmit the "most significant subset" of DMs instead. For example, we could have matched filter processes that live on each machine and transform this data into a more parsimonious form for sharing across the network. They'd only communicate the result of the most promising few filter matches at each timestep. This becomes increasingly insensitive for large-n arrays, where we might see quite deep by adding many antennas.

Antenna ID

Starting absolute timestamp

Time sample duration

List of DMS

Antenna ID

Antenna mode

Antenna pointing information

Antenna locations

Bad data flags

Trigger message

Anyone can send a trigger message; it's up to the client (antenna) to decide what to do with it, Normally, the machine is expected to simply stop recording and dump the relevant portion of its buffer of data to the SSD.

Event-start (absolute time)

Event end (absolute time)

Priority

A scalar magnitude which may or may not be interpretable as a detection sigma level but should indicate the "interestingness" of a signal.

Antennas participating in the detection

Although we might set up other antennas to dump their buffers, too.