5.3 FEC Components

In this section, we discuss the components of our FEC extension of Live555 Streaming Media. In Live555, a FramedSource is requested by a sink or source to fetch data from some resource and deliver frames to a correspond- ing filter or sink. In our scenario, we envision every RTP packet we wish to protect as aframe. We, therefore, use our custom FEC sources to push RTP packets, packed into FEC packets, as frames to an RTP sink.

The receiver side of the connection uses several Live555 provided classes and some custom classes. We use pre build BasicUDPSourceclasses to receive source and repair packets. Because the entire RTP packet is needed to rebuild lost packets, we cannot remove any RTP header at this point; and thus no RTPSource object. Instead, we use custom FEC sinks connected to the UDP sources and push the packets into a custom FEC multiplexor. In the multiplexor we aligning packets in clusters and repair lost packets, if any. The computed source packets are at last forwarded to a local address.

5.3.1 Sender Side

FEC Groupsock

As mentioned previously, we need complete RTP source packets to generate appropriate FEC repair packets. In our scenario, we use a H264VideoStreamFramer to encapsulate H.264 frames in RTP packets and aH264VideoRTPSinkto send source packets over the network using an instance of the Groupsock class. All of these classes are included in the Live555 framework. The last chain in creation of RTP packets is a sink. Therefore, we cannot use this sinks source to request frames with an additional sink, because the packets are not complete at this point in time. Instead, we extend theGroupsock class and make copies of the packets that are passed through. We push these packets to custom FEC sources which we discuss in below. Because every source uses an instance of the Groupsock class to transfer data over a network, this extension works regardless of the media type.

FECSource

We use two FEC source classes to create non interleaved and interleaved FEC repair packets. These classes are called FECNonInterleavedSource andFECInterleavedSource, respectively. In the most recent FEC draft it is stated that the non-interleaved and interleaved source packets should have different payload formats to distinguish them from another. In Live555, the payload format is set in the sink, and there is only sink per pipeline endpoint. Therefore, we cannot use a single source to generate both non interleaved and interleaved FEC repair packets.

The main responsibility of these classes is to receive source packets and place them in a packet array, with the intention of generating repair packets. When a sufficient amount of source packets have been placed in the array, we generate repair packets for that sequence of source packets. When the repair packets are complete, they are placed in a queue, and the temporary source packets are deleted. When these operations have concluded, we perform an event-trigger. When this event-trigger, triggers, we deliver a repair packet to the connected sink.

There are a few differences betweenFECNonInterleavedSourceandFECIn- terleavedSource. InFECNonInterleavedSourcethe array of source packets is the size of D. Note that the constants D and L are the row and column length of the source packets, respectively. Since the packets are generated for each row, there is no need to hold more than D source packets. In FECInterleavedSourcewe maintain an array of D * L size. This is because the packets are protected by column and we must therefore hold all packets until the array is filled. Additionally, when we generate repair packets, we only do this once forFECNonInterleavedSource, because only one row is done at once. InFECInterleavedSourcewe perform the operation L times because we need to protect each column.

For bothFECNonInterleavedSourceandFECInterleavedSourcewe use the sameFECEncoderfunction:repairRow. This function returns a new FEC repair packet generated by a row of source packets sent in as a parameter. Therefore, inFECInterleavedSource, as we have columns not rows, we ex- tract a column of packets and convert them to a row. We send this row into therepairRow procedure. Next, we push this FEC packet into the FEC Packet queue and clear out the temporary RTP array. At last, we trigger an event via the event-trigger. This event signals to the task scheduler that there is data ready in the FEC packet queue.

5.3.2 Receiver Side

FEC Sink

We receive the source and repair packets in multiple sources, and therefore, we use several FECSinks to request these packets from the correspond- ing source and forward them to ourFEC2DParityMultiplexor. We were not able to connect ourFEC2DParityMultiplexordirectly to the receiving sources, because multiple calls to the getNextFrame method in Framed- Source are not allowed in Live555. Multiple calls to getNextFrame are not allowed because it causes race conditions. However, connecting the FEC2DParityMultiplexor and the receiving sources might be possible in future work.

FEC2DParityMultiplexor

FEC FEC2DParityMultiplexor is a class which attempts to repair RTP source packets, if any are lost. This implementation is specifically for repairing matrixes of RTP source packets, protected by non-interleaved and interleaved FEC repair packets. We apply some of the base ideas as we did in Section 5.3.1. We use a queue of RTP packets to hold our outgoing repaired packets. When there are available repaired packets, we use an event-trigger to signal to the task scheduler that there are available pack- ets. When the task scheduler executes this task, the packets are delivered to the appropriate sink.

As mentioned in Section 5.3.2, packets are pushed into this class using a callback procedure. Within this procedure, we forward both source and repair packets to into appropriateFECClusters. We discussFECClusterin detail in Section 5.3.2. In short terms,FECClusteris a matrix containing the source and repair packets for interleaved and non interleaved protection. InFEC FEC2DParityMultiplexor we maintain a vector of these clusters. When packets arrive, we search through this vector for a corresponding cluster for the current packet. If no cluster is found, we create a new one and insert the packet. Additionally, in order to know which cluster we should insert an packet into, we hold a variable with the current sequence number base. We update this base when a new cluster is created.

When FEC2DParityMultiplexor is instantiated we start a delayed task which executes every 20ms. Note that this number is completely arbi- trary, as we do not know if we should schedule it more or less frequent. It is, nonetheless a low cost operation. In this task we loop through the vector of FECClusters to determine if either all source packets have arrived intact, or the repair window of a cluster has expired. If a repair window has expired and not all source packets have arrived, we attempt to repair the cluster. We use therepairCluster procedure, defined in Section 5.4.2. When we have completed repairing the packets, or all the packets did arrive intact, we push the packets into the aforementioned queue of ready RTP packets. When we push ready packets into the queue, we also trigger the event-trigger to signal to the task scheduler that the packets are ready for delivery.

The RTP sequence number is chosen at random and is an unsigned 16 bit integer. When the sequence number reaches the maximum number for 16 bits (65535) the sequence number overflows and reverts back to 0. This causes problems if the sequence number overflows in the middle of our cluster. Therefore, we compute the potential next sequence number by adding the cluster size to the current sequence number. If the potential new sequence number is smaller than the current sequence number, we have a special case. Figure 5.4 shows how we handle this scenario. Note that we also use an arbitrary failsafe. We do this in case a packet arrives late within the sequence number wraparound at 65535.

FECCluster The FECCluster class holds an array of RTP packets (source and repair), the size of this array and a timestamp of when the cluster was created, e.g. when the first packet was inserted into the cluster. As mentioned previously, when we repair the clusters in FEC2DParityMultiplexorwe check wether the repair window has expired or not. We calculate this by comparing the difference between the times- tamp of the cluster and the repair window. We also include a procedure to insert a packet at the appropriate index within the cluster. In this procedure, we differentiate between source and different repair packets by checking the payload type. We also account for the previously mentioned overflow by using 16 bit unsigned variables and whole number division. Figure 5.5 shows this scenario in detail.

RTPPacket and FECPacket

We use two container classes for holding RTP and FEC packets. These classes only hold a size and a buffer. The buffer contains the entire packet as bytes and the size is the volume of the buffer. While it is technically unnecessary to separate this logic into two classes, we still do it to make the code more readable.