Normalized OPRA Datafeed: Bandwidth Reduction Algorithm - SpiderRock/SRDataFeed GitHub Wiki

Tech Notes on Options Feed Bandwidth Reduction Algorithm

November 2019

The OPRA direct high bandwidth datafeed bursts greater than 10 Gbps on a typical trading day. This makes the networking and host processing resource requirements a challenge. Most of the data published on the OPRA feed are exchange quote updates that do not affect the NBBO, or size changes that have minimal impact on trading decisions.

The SpiderRock normalized options feed was designed with a reduced bandwidth profile that is more manageable for firms utilizing options data. Our feed is typically 1/10 the size of the OPRA feed. We accomplish this normalization in two manners:

1. We reduce the feed from a 16 exchange + NBBO quote messaging format, into a flat market wide NBBO feed with extra details.

As such, our number of quote messages are reduced to NBBO quote messages containing a consolidated view of the market as follows:

• NBBO bid/ask price/size/largest exchange

• Cumulative bid/ask size (total NBBO size)

• Bid/ask exchange masks (of all exchanges participating in the NBBO)

• 2nd level best bid/ask price, along with 2nd level cumulative bid/ask size.

2. We shape the traffic we publish as follows:

• We immediately publish prints and NBBO price level changes with very little processing time (min: 4us, avg: 100us, max: 50ms). Note that max delays are rare (1 or 2 per minute) and almost all (99.9%) packets are handled between 50us and 500us.

• We hold back “size only changes” for slightly longer periods of time. The reason for this is that more than 90% of the actual OPRA quote messages are size only changes, and size only changes are generally not as time sensitive in typical end application use cases.

• The size only holdback algorithm is a variable delay based on looking back at the publishing rate for the trailing 10,000 packets per SpiderRock multicast channel. This variable delay can shift between 10ms and 250ms depending on traffic levels (larger delay holdbacks for high traffic levels). This also helps shape traffic levels during short bursts. Also note that the maximum delay level is typically only hit during the opening minute of the day (if at all), with the rest of the day running at the 10ms level.

• Our OptionNBBOQuote message has an updateType field, which will be either [PrcChange or SizeOnly] to indicate the type of message being published.

Please see our multicast datafeed specs for full details on the fields within our published messages.