Binary Market Data (UDP)
This section describes the implementation and protocol of the UDP Broadcast functionality present in the Binary Market Data Gateway.
The Binary Market Data Gateway can be configured to broadcast Market Data updates to a UDP Endpoint. This Endpoint can be configured to be either a Standard UDP Address or a UDP Multicast Address. Market Data updates received from the Cluster will be sent to this configured endpoint.
Protocol Definition
Header
All messages broadcast from the Binary Market Data Gateway use a shared header. This header is defined using a pure binary format and is encoded in BIG_ENDIAN form. The current size of the header is in total 20 bytes, reading implementations are encouraged to always read the version to determine the size of the header and following content.
| Offset | Length | Description |
|---|---|---|
| 0 | 1 | Message Type |
| 1 | 1 | Version and Flags |
| 2 | 8 | Current Session Id |
| 10 | 8 | Sequence Number |
| 18 | 2 | Message Count |
The Binary Market Data Gateway will ensure that a single Header is broadcast per UDP Datagram emitted to the network from its host address. The underlying Data Link Layer may split this packet into multiple fragments, but they will be reassembled before being delivered to a receiving application by this layer.
Version and Flags
The second byte of the header encodes information about the version and optional bits for flags presented by the broadcast endpoint.
| Bit Offset | Length (in bits) | Description |
|---|---|---|
| 0 | 4 | Protocol Version |
| 4 | 1 | Reserved Flag |
| 5 | 1 | Reserved Flag |
| 6 | 1 | Reserved Flag |
| 7 | 1 | Reserved Flag |
Message Types
The Binary Market Data Gateway supports the following types of messages.
| Value | Message Type |
|---|---|
| 0 | Heartbeat |
| 1 | Reserved |
| 2 | Market Data Message |
| 3-7 | Not Used |
In its current implementation only the values 0 and 2 will be broadcast from the Binary Market Data Gateway. Value 1 is reserved as it was used by earlier versions of the application. The remainder are currently available for future use.
Current Session ID
The Current Session ID is an identifier for the current session of the Binary Market Data Gateway. This Session ID changes whenever the Binary Market Data Gateway loses connectivity from upstream components or restarts.
In the event of an unscheduled broadcast Session ID change when the gateway goes down, a receiving client must request a new snapshot of Market Data from the Gateway, to ensure it has the correct view of the current market data.
When Session ID changes as scheduled, receiving clients will carry on as normal.
Sequence Number
The Header Value contains the current Sequence Number that the Binary Market Data Gateway has sent up to. Each Market Data message that is sent will increase the Sequence Number by the number of messages contained within the Datagram.
As part of consumption of UDP Market Data it is expected that some loss may occur on the network. In situations where a client detects loss through the Sequence Number incrementing by a larger than expected value between messages it is required to request a new snapshot of Market Data from the Gateway to ensure that it has a correct view of the current market data.
Heartbeat Message
In the absence of other messages, a heartbeat message will be sent every 15 seconds, indicating the current Session ID and Sequence Number. A heartbeat will not increment the sequence number.
| Offset | Length | Description |
|---|---|---|
| 0 | 20 | Common Header (Message Type 0) |
| 20 | 0 | Payload |
Example
| Offset | Length | Description | Example | Notes |
|---|---|---|---|---|
| 0 | 1 | Message Type | 0 (Heartbeat Message) | |
| 1 | 1 | Protocol Version and Framing | 16 | Protocol Version 1, No Flags Set |
| 2 | 8 | Session ID | 17065462840000000 | The current Session ID |
| 10 | 8 | Sequence Number | 5 | The current Sequence Number |
| 18 | 2 | Message Count | 0 |
Market Data Message
Each Market Data Message datagram contains one or more messages to be processed by listening clients. The first message within the datagram has a sequence number equal to the field found in the Header. Subsequent messages increment this value by one, but these sequence numbers are not encoded as part of the datagram itself and are instead implicit. Clients should expect the Sequence Number in a subsequent message to be the previous message Sequence Number plus the number of messages within the Datagram.
The Header portion of a Sequenced Message is encoded using a pure binary format in BIG_ENDIAN format.
| Offset | Length | Description |
|---|---|---|
| 0 | 20 | Common Header (Message Type 2) |
| 20 | n | Payload |
The Payload portion of the Market Data Message is then divided into sections for each Message containing the length of the message and payload bytes. This is again encoded using a pure binary format in BIG_ENDIAN.
| Length | Description |
|---|---|
| 2 | Payload Length |
| n | Payload |
Example Datagram 1
| Offset | Length | Description | Example | Notes |
|---|---|---|---|---|
| 0 | 1 | Message Type | 2 (Market Data Message) | |
| 1 | 1 | Protocol Version and Framing | 16 | Protocol Version 1, No Flags Set |
| 2 | 8 | Session ID | 17065462840000000 | |
| 10 | 8 | Sequence Number | 1 | |
| 18 | 2 | Message Count | 2 | |
| 20 | 2 | Message 1 Length | 16 | |
| 22 | 16 | Message 1 Payload | SBE OrderAdded | (Message Sequence Number 1) |
| 38 | 2 | Message 2 Length | 16 | |
| 40 | 16 | Message 2 Payload | SBE OrderDeleted | (Message Sequence Number 2) |
Example Datagram 2
| Offset | Length | Description | Example | Notes |
|---|---|---|---|---|
| 0 | 1 | Message Type | 2 (Market Data Message) | |
| 1 | 1 | Protocol Version and Framing | 16 | Protocol Version 1, No Flags Set |
| 2 | 8 | Session ID | 17065462840000000 | |
| 10 | 8 | Sequence Number | 3 | |
| 18 | 2 | Message Count | 2 | |
| 20 | 2 | Message 1 Length | 16 | |
| 22 | 16 | Message 1 Payload | SBE OrderAdded | (Message Sequence Number 3) |
| 38 | 2 | Message 2 Length | 16 | |
| 40 | 16 | Message 2 Payload | SBE OrderDeleted | (Message Sequence Number 4) |