Last updated: 29/09/2022

Latest protocol version: 1.0.5 (see the MaRCoS server’s version.hpp in case this is out of date)

The communication between client and server uses the MessagePack protocol for encoding/decoding data. On the server it is built on the MPack C library, on the client it uses Python msgpack (NOT python-msgpack).

MessagePack is similar to JSON or XML: it’s a way of encoding a nested hierarchy of data, whether it be strings, arrays of numbers, or general binary data. MaRCoS uses a subset of MessagePack.

There are two basic containers that the MaRCoS protocol uses. One is the array: each element can have a different type, like a list in Python and unlike a C/C++ array, and the elements have a numerical index. The other is the map: each element is made up of a key-value pair, like a Python dictionary or a C++ map, and each element is indexed by a descriptive string.

MessagePack allows you to nest these; e.g. one element of an array or a map might point to a deeper array/map.

Each packet consists of an array as follows:

Index	Element	Type	Purpose
0	message type	unsigned int	identify the message source and purpose
1	packet index	unsigned int	keep track of the number of packets arriving, and whether any packets have been dropped etc
2	unused	unsigned int	perhaps the size? Reserved for later use
3	version	unsigned int	3 bytes (major, minor, debug) with bitmaps of ff0000, ff00 and ff within an unsigned int. [TODO: shift to chars to avoid bitbanging?]
4	requests or replies	map	Depending on the message type, the map contains a nested series of data. Each element corresponds to a particular request/reply (see below)
5	status (only in packets coming from the server)	map	Three elements in the map, with arrays containing strings: errors, warnings and infos from the server to the client

The first unsigned int can take the following values:

Value	Source	Meaning	Behaviour
0	client	request	normal conditions; a command to carry out some standard operation
1	client	emergency stop	tries to halt whatever the server is currently attempting, turn off the RF and zero the gradient voltages
2	client	exit server	shuts the server down gracefully
128	server	reply	normal conditions; a reply to a client command containing data in most cases as well as infos/warnings/errors
129	server	reply	error conditions; describing error state

This list is growing as functionality is added to the server. Each key is optional. The server interprets keys in the order listed in this table; i.e. if a packet has multiple keys, those from the top of this table are processed and executed first.

For each key present in the request that the server understood, the server will reply with the same key whose value contains reply-specific data.

To find some examples, see the self.assertEqual() comparisons in the server test script.

Key	Argument/s	Interpretation	Reply contains:	Description
fpga_clk	array of three unsigned ints	clock words	0 if okay, -1 if there was a problem	[ADVANCED] Control the FPGA clock PLL settings
lo_freq	unsigned int	Local-oscillator frequency for both RX and TX	0 if okay, -1 if it’s outside ~ [0.01, 50] MHz	RX sampling frequency, in units of (either 122.88 or 125 MHz, depending on STEMlab version) / (2**30)
tx_div	unsigned int	Clock divider for RF TX samples	0 if okay, -1 if it’s outside [1, 10000]	TX clock divider; controls how often TX samples in the TX RAM are played out to produce pulses. [TODO: units]
rx_div	unsigned int	Clock divider for RX acquisition	0 if okay, -1 if it’s outside [25, 8192]	RX clock divider; controls how often RX samples are saved. [TODO: units]
tx_size	unsigned 16-bit int	TODO, maybe legacy only	0 if okay, -1 if it’s out-of-range	TODO
raw_tx_data	binary byte array	Directly write the RF waveforms	0 if okay, -1 if more than 64 KiB of data was sent	Binary 16-bit unsigned integer samples to be saved in the RF pulse envelope memory contents. Generate using the desktop APIs in marcos_client.
seq_data	binary byte array	Pulse sequence instructions	0 if okay, -1 if more than 64 KiB of data was sent	Binary machine code run by the pulse processor, generated either from an assembly text file, or using the ocra-pulseq library from a pulseq file.
grad_div	array of two unsigned ints	Gradient update interval and SPI clock dividers	0 if okay, -1 if either divider is out of range	First argument: update interval, determines how often samples are sent to the gradient boards. Second argument: SPI clock, determines how quickly the SPI runs.
grad_ser	unsigned int	Gradient serialiser select	0 if okay, -1 if it’s above 0xf	Enables/disables the gradient serialisers. 0x1 enables the OCRA1, 0x2 enables the GPA-FHDO. 0x3 enables both (for debugging only; binary data formats differ)
grad_dir	unsigned int	Gradient direct update	0 (all 32-bit ints valid)	Sends data directly to the gradient serialiser/s for immediate transmission. Used for setting the gradient DAC registers without running a sequence or reading the ADC.
gadc	none	Read the current gradient ADC register	32-bit ADC data word	For the GPA-FHDO board, stores the latest ADC data word received.
grad_mem	binary byte array	Directly write the gradient memory	0 if okay, -1 if more than 32 KiB of data was sent	Organised as a stream of 32-bit words. Each word maps to a single gradient DAC BRAM address. Generate using the desktop APIs in marcos_client.
acq	unsigned int samples	Run sequence and acquire data	Array of 64-bit complex floats, samples long	Runs the sequence loaded using seq_data, and acquires data continuously into one buffer until it finishes. Acquisition rate is controlled using rx_div .
test_throughput	unsigned int	Test the server-client throughput	A map of the form {'array1': arr1, 'array2': arr2}	arr1, arr2 are of length specified by the argument, containing doubles computed according to a simple formula (WARNING: this can take a long time!). Useful for testing/debugging the server-side performance without involving the Zynq PL.
test_bus	TODO	Test the PS-PL AXI bus performance	TODO	TODO
state	double	Calculate and return various system parameters	Multiple info messages in the standard infos array	The argument is the RP NCO clock frequency, 122.88 MHz for the RP-122 and 125 MHz for the RP-125. It is used to calculate the LO frequency, TX/RX/gradient sample durations, gradient SPI transmission duration, etc.

I chose the top level to be an array rather than a map to avoid repetitive overhead for the map keys.

Each packet carries a version with it to avoid any potentially dangerous behaviour in the case of misinterpreted packets. It also adds future-proofing in the case where there are multiple clients/servers on a single network in the future, with disparate operations; the version might be useful to handle subsets of this protocol.

I used an int with three bytes rather than a map simply for expediency. This decision might end up being limiting, since it only permits a max version of 255.255.255, however for now I think it’s safe. (After all, IPv4 has lasted for decades!)

Packet indices are provided to deal with high-throughput situations where packets might arrive out-of-order, causing unforeseen consequences since the server has a lot of state. They’re not used yet.

There is a map for the requests/replies to be more flexible; I’m writing this at a very early stage in development and can’t foresee yet what kinds of operation may end up being needed. A future version might replace the map with an array for performance, however it may be too small an overhead to bother with.

The gradient DAC commands are all identical, except for the channel. I could have instead created a single command that would take data as well as a channel argument, however the downside of this is that only a single channel could be updated per packet (otherwise there would be multiple values for a single key). A future version of the protocol should have an array argument, with each element in the array fully specifying a channel to update; this would reduce code duplication but make the packet structure slightly more nested and thus marginally harder to debug.