The rise of electronic and algorithmic trading has led to larger volumes of market data during a trading session. Fast asset classes such as equities and FX often challenge the network bandwidth during busy market conditions.
FIX has become the prominent protocol for market data. FIX is however very verbose and its performance limitations have become apparent as market data volumes have increased. There have been attempts to make FIX a more efficient protocol such as Fast FIX. According to Paul Heffernan, principal consultant at Brickendon: “Fast FIX is a complicated addendum to an already complicated protocol. Perhaps it’s time for a rethink.”
Notably, the industry seems to be moving in this direction. Some exchanges have developed their own proprietary protocols for market data. Euronext has UTP Market Data Protocol and Nasdaq has ITCH. The FIX standard is considering new protocols such as ITCH for market data.
Market Data Best Practices
A market data protocol should be simple. The cost to develop an adapter should be low to facilitate widespread adaption. Simple suggests a protocol addressing a specific use rather than a protocol intended for multiple use.
The message size should be compact to facilitate low latency and high throughput requirements, and busy market conditions. This suggests a binary protocol rather than text. Most data transmitted is numerical and binary data takes approximately one-half the space of text. Boilerplate and redundant information should not be present in the message.
There are two protocols for network data transmission – TCP/IP and UDP. UDP is more lightweight than TCP and can multicast (send messages to multiple destinations simultaneously), but it is also more complicated to work with. Data packets may get corrupted and messages may not arrive in sequence. An application supporting UDP needs to be able to handle these scenarios using check sums, sequence numbers and retransmission requests. TCP handles message sequencing, retransmission and checksums for the user. Ideally messages are small enough to be transmitted over UDP if desired. UDP IpV6 has a typical max size of 1,500 bytes; that’s not too much room.
Many protocols follow the market snapshot followed by incremental update approach. A snapshot provides a full view of the market for an instrument. Then incremental updates that should be applied to the snapshot are sent. The consumer must maintain the depth of book for the market when applying these incremental updates. The intention is to save space by only supplying updates. If however, there’s the need to retrospectively find the state of the market it is necessary to apply all updates from the snapshot to the time in question – potentially millions of incrementals. If the snapshot was requested at 07.00 and you want to view the market at 16.00 that day then you’ve got many incrementals to apply.
Heffernan believes that a stateless protocol is better. A single message would contain complete depth of book for that instrument. It is straightforward to view the market at any period of time because all the data is there. It simplifies the logic required to consume the market data feed. It’s easy to run calculations on the depth of book. It works well within trading applications that typically have complex throttling and threading logic, and it is easier to pass an atomic data structure around the trading application then a delta batch.
A market data message should have a source timestamp. This can be used to determine the latency before the source and the destination.
The protocol should be universally fair and easy to apply for all market participants whether price makers, price takers or an exchange.
An FX Example
For brevity, we will concentrate on the market data message. There will be additional messages to logon/logoff, subscribe/unsubscribe to instruments, request a retransmission, and a heartbeat message for times of inactivity.
The market data message will have a header containing Source Timestamp, Instrument ID, CheckSum, and Counter. The timestamp is for measuring latency. The CheckSum is to verify the contents of the message. The Counter is for sequencing messages on the consumer end should they arrive in a different order than that in which they were sent. The TCP protocol will checksum and sequence packets for the user, but UDP will not. Therefore a TCP consumer can ignore these fields.
All fields are 32-bit unsigned integers. These support a number range between 0 and just above 4 trillion. To keep below the 1,500 byte capacity for UDP we can have 46 data fields in the message. Prices will have a scale of 6, so to specify the price 1.23456 you send 1234560, 100.50 becomes 100500000.
The market data will consist of a number of bids fields and a number of offers fields. Counting these and the four header fields there are 40 remaining fields for market data. We supply price and quantity for each level in the depth of book. That’s room for 20 prices, or 10 levels of bid-offer depth. If UDP is not required then there is no limit to the depth of book.
This should be suitable for most FX use-cases, but the protocol may be adjusted as required
