Realtime protocol for remote control

The realtime protocol is designed for low-latency, high-frequency links such as ESPnow, XBee, or Bluetooth. It is stateless, mostly unidirectional, and has only one fixed packet frame size of 10 bytes, with a 9-byte packet, of which 8 bytes are the payload.

There are four different packet types:

• Control packets (PACKET_TYPE_CONTROL): These are designed to be sent continuously from remote controls to droids, at high rates of 25-100 frames per second. They contain information about 5 axes and 5 buttons each. Typically sent without requesting a receive confirmation from the protocol they are sent over, and no receive confirmation supported by the application protocol.
• State packets (PACKET_TYPE_STATE): These are designed to be sent continuously from droids to remote controls, also at high rates of 25-100 frames per second. They contain information about the droid's state ("telemetry"). Typically sent without requesting a receive confirmation from the protocol they are sent over, and no receive confirmation supported by the application protocol.
• Configuration packets (PACKET_TYPE_CONFIG): These are designed to set various parameters in both the droid and the remote controls. As such, they are not sent continuously but rather for example as a reaction of some interface event the user triggers on the remote. Typically sent with requesting a receive confirmation from the protocol they are sent over, and the application protocol supports requesting a reply.
• Pairing packets (PACKET_TYPE_PAIRING): Currently the protocol does not have an option to safely pair remotes to droids, which is a (small) safety risk. In the future, this packet type will be used to implement a safe pairing mechanism.

Packet frame

The packet frame contains the packet itself plus an 8-bit CRC calculated using the bb::Packet::calculateCRC() method.

Packet

The packet contains an 8-bit flags field and the 8-byte payload. The flags field is interpreted as follows:

• Bit 7 and 6 encode the packet type (PACKET_TYPE_CONTROL: 0, PACKET_TYPE_STATE: 1, PACKET_TYPE_CONFIG: 2, PACKET_TYPE_PAIRING: 3).
• Bit 5 and 4 encode the packet source (PACKET_SOURCE_LEFT_REMOTE: 0, PACKET_SOURCE_RIGHT_REMOTE: 1, PACKET_SOURCE_DROID: 2, PACKET_SOURCE_TEST_ONLY: 3). The value of PACKET_SOURCE_TEST_ONLY can be used for protocol testing from an unspecified other source; reception of such packets will be announced but no action will be taken on their reception.
• Bit 3, 2, and 1 encode a sequence number rising from 0 to 7, then wrapping. This number can be evaluated to diagnose packet loss through protocol or reception errors or timing problems. The sequence number field is filled automatically upon send with the Runloop sequence number.
• Bit 0 is reserved.

The payload is a union field to be interpreted depending on bit 7 and 6 of the flags field, as follows:

Control payload (PACKET_TYPE_CONTROL)

Control packets encode the following information:

• Five axes with a 10-bit resolution, values ranging from 0 to 1023.
• Five axes with an 8-bit resolution, values ranging from 0 to 255.
• Eight boolean fields encoding button press states on the remote control.
• One battery status value with a 6-bit resolution, values ranging from 0 to 63.

The ControlPacket struct contains setAxis() and getAxis() convenience methods to set and retrieve axis information in the form of floats, that will be converted to use the full value range of the respective axis. Those methods take a Unit parameter that can be one of the following:

• UNIT_DEGREES: 0.0 .. 360.0
• UNIT_DEGREES_CENTERED: -180.0 .. 180.0
• UNIT_UNITY: 0.0 .. 1.0
• UNIT_UNITY_CENTERED: -1.0 .. 1.0
• UNIT_RAW: Raw value, will be constrained to a value between 0 and the maximum value the axis supports.

On the Bavarian Builders remotes, the ControlPacket structure is filled with the following information:

• Axis 0: Horizontal joystick, increasing values mean joystick motion to the front of the remote (away from the LEDs).
• Axis 1: Vertical joystick, increasing values mean joystick motion to the right of the remote.
• Axis 2: Gyro roll position, 511 denoting zero roll
• Axis 3: Gyro pitch position, 511 denoting zero pitch
• Axis 4: Gyro heading position, 511 denoting zero heading (note this axis may drift because there is no magnetometer in the remotes)
• Axis 5: Acceleration along X axis, in g
• Axis 6: Acceleration along Y axis, in g
• axis 7: Acceleration along Z (down) axis, in g
• Button 0: Side / Back ("Pinky")
• Button 1: Side / Front ("Index")
• Button 2: Front / Left
• Button 3: Front / Right
• Button 4: Joystick push button
• Button 5: Top / Left
• Button 6: Top / Right
• Button 7: Top / Confirm
• Battery status

State payload (PACKET_TYPE_STATE)

Note: This is Work in Progress

StatePacket payloads encode the status of several core components, as well as the way several droid subsystems are currently controlled.

Component status can be one of STATUS_OK (all clear), STATUS_DEGRADED (workable but something is wrong), STATUS_ERROR (there is a problem which causes this component not to work), and STATUS_CRITICAL (take action now or something will break). The packet encodes the status of

• The battery or batteries: STATUS_DEGRADED means the battery is slowly being depleted, STATUS_ERROR means battery power is so low that all electromechanical components are switched off, STATUS_CRITICAL means power is so low that the battery will be damaged if not charged immediately.
• The drive system: STATUS_DEGRADED means speed is reduced, STATUS_ERROR means that the drive system is disabled, STATUS_CRITICAL means some component is damaged, or blocked and damage is imminent.
• The servo system: STATUS_DEGRADED means that not all servos have been found but the ones critical to system operation are there, STATUS_ERROR means that servos critical to system operation have not been found, STATUS_CRITICAL means some component is damaged, or it is blocked and damage is imminent.

Droid subsystem control can be one of CONTROL_OFF (not used at all), CONTROL_RC (fully controlled by remote control), CONTROL_RC_FREE_ANIM (controlled by a mix of remote control and free animation), and CONTROL_AUTOMATIC (fully controlled by some internal system input). Subsystems for which control is encoded this way are

• the main drive,
• the dome or head,
• the arms or other appendages,
• the sound output.

This subsystem control status finds its parallel in the CONFIG_SET_*_CONTROL_MODE Config packet, that sets the control mode for the respective subsystem. With this scheme, it is e.g. possible to switch the dome to completely automatic control (steered by a camera in the dome or similar, and looking around), while the drive system is controlled by the remote and the arms by a mix of remote and free animation.

Config payload (PACKET_TYPE_CONFIG)

Note: This is Work in Progress

Pairing payload (PACKET_TYPE_PAIRING)

Note: This is Work in Progress; the following describes a PLAN that has not been implemented yet.

Paired And Unpaired Stations

A station (right remote or droid) can be Paired or Unpaired.

• Paired stations do not accept messages from other communication partners than the ones they are pared with, including Pairing messages.
• Unpaired stations accept Pairing messages (and only those) from any left remote. If a well-formed Pairing message is received, the station stores the information contained and transitions to Paired mode. Note that there is currently no way to protect against someone hijacking the pairing process by pairing from a different left remote than the intended one, or another device masquerading as a left remote. This is the same process as with e.g. the Spektrum remotes so Be aware of this, and test that pairing was successful and was done with the remote you intended...
• To pair the left remote and the right remote, the left remote sends a PAIR_REMOTES message to the discovered right remote, containing its own address.
• To pair the left/right remote set with a droid, the left remote sends a PAIR_DROID message to the discovered droid, containing the addresses of the left and right remotes. The left remote does not offer the option to pair with a droid if it is not paired with a right remote.
• At this point, the remotes know each other and the droid, and the droid knows both remotes. From now on, they will only accept messages from each other.

To unpair a right remote or droid, do one of the following:

• Select Unpair on the left remote's menu, which sends an UNPAIR config packet to the corresponding partner.
• Select Factory Reset on the left remote's menu, which sends a FACTORY_RESET config packet to the corresponding partner. This loses all parameter customization, so it's a destructive option.
• Issue the command xbee unpair or esp32 unpair (or factory_reset) via the serial terminal.

Note: There is no explicit Paired/Unpaired status; rather, the right remote considers itself unpaired if the left remote address it knows is 0, and the droid considers itself unpaired if both the left and right remote addresses it knows are 0. As a side effect therefore, using the set_left_address console command in the right remote or the set_left_address or set_right_address commands in the droid can be used to move the remote or droid into Paired or Unpaired status. Careful with these commands though, they can be used to create inconsistent configurations.

Changing Droids

Changing droids does not require re-pairing, and does not go through the pairing message scheme. Rather, if the user selects a different droid in the left remote (that the remote set has been paired with),

1. the left remote sends a SELECT_DROID message to the right remote, which the right remote reacts to by blinking twice;
2. the left remote starts sending control messages to the newly selected droid;
3. the left remote only reacts to state messages coming from the newly selected droid;
4. the right remote starts sending control messages to the newly selected droid.

The droid that has been deselected does not receive control messages anymore. It will continue sending state messages to the left remote, but those will not be processed anymore since they are coming from the wrong address.

If the right remote was not switched on while the droid was changed, and gets switched on later while the left remote is still on, the right remote's COME_ONLINE message sent to the left remote will trigger the SELECT_DROID message being sent.

Caveat / Note: If the right remote is switched off, the left remote is used to switch droids, then the left remote is switched off and the right remote is switched on, the right remote has no way of knowing about the droid change. It will still talk to the previously selected droid.

XBee Discovery

XBee can do node discovery, and it has the option of setting 16-bit station addresses. We are basing the pairing process on this. Unpaired stations (remotes or droids) set their station address based on the XBee station encoding described at the end of the document. Paired stations set their address to 0xffff. A Paired station can be set to Unpaired by one of the following:

ESP32 Discovery

ESP32s cannot do node discovery. Instead, unpaired ESP32 remotes and droids broadcast NODE_PUBLISH messages every second containing their 16-bit station addresses and names. Every ESP32-based node collects a list of these stations. As soon as a station has been paired, it broadcasts a NODE_UNPUBLISH message, which causes every receiving ESP32-based node to remove it from its broadcast list. Since there is no transmission guarantee, this may lead to rare situations where the NODE_UNPUBLISH message is not received and a paired node still appears in a discovery list. Since it will not react to pairing messages anymore, this can cause confusion but no harm.

---

Safety and Security

Nothing is worse than having a droid run haywire in a crowd, or off a stage, because it lost connection to its remote control and doesn't know it, or because someone (inadvertently or intentionally) hijacked the communication to your droid. Therefore the RC protocol has a couple of features baked in that help make communication reliable and safely stop operation if it isn't.

Feature #1: Open Source

Droid building is a niche activity, and a very complicated one, with many moving parts, communication partners, and a lot of DIY components. We are a small group, so we are limited in testing breadth we can do. All code has bugs. The most important safety feature is therefore opening up the source code, so that any bugs or shortcomings can be immediately found and fixed.

Safety Concept

Safety is protection against bugs and random influences, e.g. congested wireless spectrum causing packet drops, or memory bugs wiping packet structure in-code.

CRC Protection - Reaction: Automatic by API

All packets feature an 8-bit CRC checksum. This is in addition to the protocol checksum of any underlying stack, and doesn't just add an extra layer of safety to the communication itself but also to any memory issues in the droid - stray pointers into the packet data may cause corruption and invalid commands. The checksum is checked upon arrival; packets with broken checksum are dropped by the API (with a console warning).

Sequence Counter - Reaction: Must be implemented by client code

All packets feature a 3-bit sequence counter filled by the runloop when the packets are sent out. This allows for in-sequence check of up to 8 packets, after which the sequence counter wraps, giving immediate drop recognition. The only case that cannot be detected is the extremely unlikely one of regular drops of 8 packets. Note Checksum errors, per the last paragraph, manifest as packet drops. Since we do not want to prescribe system reaction to packet drops, it needs to be implemented by the client code.

Security Concept - NOTE: Not implemented yet

Security is protection against (intentional or accidental) spoofing, e.g. two remotes accidentally sending messages to the same droid. The concepts in this section are not yet implemented but will be in milestone 0.2.

Addressing / pairing to hardware addresses

The Station Encoding described below is a filtering mechanism that allows simpler pairing in crowded situations, but it is not secure - if you neglect setting up your information in the code, or set it up incorrectly, it is possible that at a convention you find several droids with the same ID. It is therefore a good idea to do your pairing in private. After pairing is done, remotes and droids refer to each other using their hardware addresses, which are globally unique.

Encryption

Even with hardware addressing, spoofing is still theoretically possible. Therefore upon pairing, the left remote generates a random encryption key that is stored by both partners. Encryption is very simple - just XORing the packet payload with the encryption key - so it does not provide hard protection, especially as it can be reconstructed from zero fields in packets by statistic analysis, but it does require a bit of criminal energy. If harder protection is required, better encryption schemes may be implemented in the future.

---

XBee remote control

Simple XBee Station Encoding

Every XBee station has a 16 bit unique address (the „MY“), apart from the channel and PAN parameters. As a default, we use 0x3333 as a PAN. But it is highly encouraged, at conventions or similar, to go into your droid’s Wifi and use the console to configure for an unique PAN - the Builder ID field for the station is not enough to differentiate world wide, and you really don’t want to connect to / steer another person’s droid by accident. This is unlikely to happen but not impossible at large conventions.

Bits 14, 15: Station type

Encoded by bits 14 and 15 (the highest bits) of the MY. Two bits give 4 values:

• 0: This is a droid (i.e. a receiver).
• 1: This is a remote control (i.e. a sender).
• 2: This is a PC or other diagnostic device.
• 3: This is a device in pairing mode (reserved / not implemented for now).

Bits 11, 12, 13: Droid type / Remote type

These 3 bits give an indication, depending on whether the station is a droid or a remote, what basic type we are looking at.

For droids: This encodes only the most basic kinematics of course, more detailed settings are done either in the droid or the remote.

• 0: Static droids without a drive system.
• 1: Differential, dynamically stable kinematics, like R2 units (things that don’t wobble).
• 2: Differential, dynamically unstable kinematics along one axis, like D-O (things that wobble only in one direction).
• 3: Drive/tilt dynamically unstable kinematics along two axes, like BB-8 (things that wobble in two directions).
• 4: Holonomous drives, like B2EMO.
• 5: 4 legged walking droids, like AT-ATs.
• 6: 2 legged walking droids, like Ducklings.
• 7: Reserved.

For remotes:

• 0: Bavarian Builders left remote (2 axis plus button joystick, 3 buttons, 6DOF IMU)
• 1: Bavarian Builders right remote
• 2-7: Reserved.

Bits 4, 5, 6, 7, 8, 9, 10: Builder ID; Bits 0, 1, 2, 3: Station ID

This allows distinguishing up to 128 different builders and up to 16 droids / remotes / other stations per builder. Please note that this is absolutely not a world-wide unique identifier - we have thousands of members right now, we would need a separate 16bit address to identify each one of us. And there is no central registry. In fact the distinction between Builder ID and Station ID is somewhat arbitrary, if you have more than 16 droids to control you can of course wrap into the Builder ID field.

XBee Pairing Protocol

It is clear that the simple XBee addressing scheme is not safe, e.g. for large conventions where many people drive many droids. For example, if two builders at a convention have only an R2-D2 each, both having Droid ID 0, and they both use the same builder ID (of which there are only 128), their droids will have the same station ID, so one remote will talk to two droids at the same time, creating confusion that may be very hard to debug. This can be worked around by using a different PAN, but it's hardly safe.

However each XBee has a 64 bit address that gives unique addressing. A future extension will support pairing remotes to droids, so that this kind of uncertainty can be excluded.