Rallypoint Link Options - rallytac/pub GitHub Wiki
> WORK IN PROGRESS <
Rallypoint Link Options
TCP
Rallypoint links always use TCP for some or all of their communications. This means that control-plane messaging - which is generally transactional (i.e. bi-directional) in nature - can be reliably delivered between the endpoints because TCP ensures that those messages are delivered - be they requests or responses to those requests.
So, TCP is great for control-plane traffic.
These same TCP links are also used for packets that are not transactional in nature, and are generally uni-directional. A great example of this kind of traffic is streamed media such as voice (the media-plane). Now, while its great that TCP takes care of ensuring packet delivery; that sort of thing is not ideal for uni-directional traffic. In fact, we don't really mind too much if a few packets of a media stream are lost, corrupted, duplicated, or reordered - primarily because a loss of a media packet here and there may not even be detected by someone listening to that media stream or, even if they do detect it, that lost packet forms only a tiny part of the spoken transmission - typically way less that 1/10 of a second.
But, when we use TCP for the media-plane, we then run the risk of TCP actually making the audio worse by trying its best to deliver packets who's loss we're not going to get heartbroken about.
The diagram below shows a typical setup where we have 3 Engage clients (C1, C2, and C3) communicating with each other via a Rallypoint (RP). Each client nails up a secured TLS connection over TCP with the Rallypoint, using that connection for control-plane and media-plane traffic. Most of the time this is a pretty good setup - giving us a single, secure connection for all our traffic. That security is ensured by TLS working hand-in-hand with TCP to get packets where they need to be and not being susceptible to attacks (at least not any we know of).
------
| C1 | <---------- TCP ------------+
------ |
v
-----------
| |
------ | |
| C2 | <--------- TCP ------> | RP |
------ | |
| |
-----------
^
------ |
| C3 | <---------- TCP ------------+
------
All this security and reliability comes at a cost - that media traffic can be interrupted and/or delayed because of how TCP works. Those hiccups in TCP generally happen due to network issues. So, if your network is pretty good (reasonably error-free, sufficient bandwidth, and pretty fast) you shouldn't have any problems.
But not all networks are perfect. While they may have been engineered to be perfect (and are much of the time) sometimes bad things happen and your network sucks. And when you network sucks, TCP starts sucking a lot. And when you're using sucky TCP on a sucky network - your audio sucks. And that's no good!
Ideally, what we'd have is reliable message delivery for control-plane traffic and best-effort delivery for the media-plane.
That's where UDP comes in.
UDP
UDP is a far more efficient method of delivering media-plane traffic. It doesn't have the per-packet overhead that TCP imposes nor does it have the back-and-forth messaging that TCP uses to manage the connection. Basically, whereas TCP is a "fire-and-hit-the-target" strategy; UDP is a "fire-and-forget" strategy. Of course, most of the time UDP packets do get to where they're supposed to go - but just not guaranteed like with TCP.
Now, let's say we need to be more efficient with our bandwidth utilization (remember, TLS over TCP eats up some of your bandwidth). Also, we want to get our network-imposed latency down as low as we can go. Our control-plane messages are few and far between enough not to really be a concern so we'll leave control plane traffic on TCP. But our path forward for our media packets is to use UDP in some form.
Something like this would be great:
------
| C1 | <---------- TCP ------------+
| | <---------- UDP ---------+ |
------ | |
v v
-----------
| |
------ | |
| C2 | <--------- TCP ------> | RP |
| | <--------- UDP ------> | |
______ | |
-----------
^ ^
------ | |
| | <---------- UDP ---------+ |
| C3 | <---------- TCP ------------+
------
What we'd do is have clients setup TCP connections as they do in the first example and use that TCP connection for all control-plane traffic. Then, we'd also have each client establish a UDP connection with the Rallypoint for media packets to traverse. Fabulous!
But, "why not just use UDP for everything" you say? Well, UDP is unreliable and we'd have to do a lot of work to guarantee control-plane traffic delivery - something TCP already does rather well. In fact, we'd basically just be reinventing the TCP wheel - and that's kind of silly. Also ... UDP has a glaring issue that we need to take into account - it doesn't always work!
We don't mean UDP doesn't work as a general rule, but there are situations where UDP may not be able to cross organizational boundaries, where firewalls with some NAT configurations don't allow UDP traffic in both directions, and so on. So, if we always use TCP then we can use it as a backup if UDP flat-out fails, fails during a connection, or goes all wonky on us and we have traffic flowing only in one direction. (In fact, Engage and Rallypoints support "asymmetric streaming" which means that we can have a link's media streaming in one direction over UDP and over TCP in reverse. Cool huh!?)
Pros
- Uses TLS for encryption
- Client/RP session uses just a single port for control and media plane traffic
- Packet delivery is guaranteed
Cons
- TCP management adds bandwidth overhead
- TLS management adds bandwidth overhead
- Head-of-line blocking introduces delay
- RP-side requires single decrypt (from source) + per-target encrypt
DTLS over UDP
Shared-Key over UDP
No-Key over UDP
In this scenario, stream packets produced by the source, while potentially encrypted with the group-specific key, are NOT encrypted for transmission over the Rallypoint link. Of course it may seem like this option is a non-starter but it certainly has its place depending on your environment.
Here's an example:
Let's assume that you're not using Rallypoints but are, instead, using multicast for packet streaming. Well, in that case, your packets are likely going to be traversing a network infrastructure that you have control over and are able to lock down to prevent unwanted access. It also assumes that you trust that people and entities with access to the network are vetted and trusted not to do nasty things like packet replay attacks, man-in-the-middle attacks, packet spoofing, and so on. In short, we're assuming that the network is already secured.
Fun fact: Oftentimes people don't even encrypt their groups in this kind of situation because their network is already locked down and there's no need to further lock things down at an application level.
Now, what if your entire network - including your Wide Area Network (WAN) - is secured? And, let's assume we're going to use Rallypoints because this network of yours, while secured, does not support multicast across the WAN, or even just on some LANs. Well, that's where you're going to want to use Rallypoints to get around this multicast shortcoming. In this scenario, you'd of course want to minimize the amount of traffic you put on the network (isn't that always a goal?) as well as reduce the amount of CPU and memory usage on each endpoint and RP (also an obvious desire).
So, given that there's no concerns about attackers on the network, there's really no need to add all the overhead of encryption on these Rallypoint links. And for this you'd get the best bang for your buck by doing RP packet streaming with no encryption.
Pros
- Negligible per-packet bandwidth overhead introduced by packet's stream header
- Negligible CPU and memory overhead on RP (no source decryption or target-specific encryption)
- Negligible CPU and memory overhead on client
- Single UDP port for all streams and all clients on the RP
- Single UDP port for all streams on each client
Cons
- No encryption
- Almost no protection against replay, man-in-the-middle, or other attacks