Overview

The Exchange Streaming API provides the ability to subscribe to market changes (both price and definitions) and to your orders.  The protocol is based on ssl sockets (normal) with a CRLF json protocol.

We publish a definition of the schema of the json messages in the Swagger format.

Swagger Definition

We provide a Swagger schema to allow specification browsing & code generation for various languages; please use:

Swaggerhttp://editor.swagger.io/#/

Market Stream: Swagger DefinitionESASwaggerSchema.json 

To generate a client for the programming language of your choice:

  1. Navigate to http://editor.swagger.io/#/
  2. Selection File > Import File
  3. Choose the "ESASwaggerSchema.json" file > Generate Client > Select your required language to generate a language binding.

  • It's cross platform and we can't control how it works / behaves - but it does save a lot of error prone typing.
  • Enums and Inheritance are a little flaky:
    • Enums for error codes / filters etc. are defined but are treated as strings in c# (so you will need to copy definitions from the swagger spec until this is fixed by swagger).
    • Inheritance is defined but not generated correctly - you will have to manually manipulate the op=<type> field
      • In c# JsonCreationConverter is the typical way to model inheritance
      • In java look at JsonSubTypes
  • We are not a REST service - so only the swagger generated model package is relevant

Typical Interactions with Stream API:

The typical API interactions are documented below (detail is below this).

 

Market Stream:


Order Stream:

 


Connection

Protocol

Every message is in json & terminated with a line feed (CRLF):

{json message}\r\n

 

As the protocol is CRLF delimited don't forget to turn-off Json pretty printing (C# has this on by default)

TCP / SSL Connection

Connection is established with an SSL socket to the following address:

stream-api.betfair.com:443

 

For pre-production (beta) release the following URL should be used for integration testing:

stream-api-integration.betfair.com

Basic Message Protocol

Two base message classes exist:

Every child message type has:


Note: Any fields representing time and having a long type will represent the UNIX Timestamps (See https://currentmillis.com/ for conversions) 

RequestMessage

RequestMessage is the base class for requests from the client; the discriminator is op=<message type>

Key fields:

  • Remember to set op=<message type> - otherwise we can't decode the request
  • Remember to set id=<unique sequence> - this will let you link requests with responses (these should be logged and provided on support calls)
  • Every RequestMessage will receive a StatusMessage with the status of the call (linked by the id that you send).
    • All errors apart from SUBSCRIPTION_LIMIT_EXCEEDED close the connection

ResponseMessage

ResponseMessage is the base class for responses back to the client; the discriminator is op=<message type>

Key fields:

As mentioned earlier the id=<request id> and links your request with your response.

ChangeMessages carry the id of the original request that established the subscription

Status / StatusMessage

Every request receives a status response with a matching id.

Key fields:

ErrorCode

This categorizes the various error codes that could be expected (these are subject to change and extension)

CategoryErrorCodeDescription
Protocol   General errors not sent with id linking to specific request (as no request context)
INVALID_INPUT
Failure code returned when an invalid input is provided (could not deserialize the message)
TIMEOUT
Failure code when a client times out (i.e. too slow sending data)
  
Authentication Specific to authentication
NO_APP_KEY
Failure code returned when an application key is not found in the message
INVALID_APP_KEY
Failure code returned when an invalid application key is received
NO_SESSION
Failure code returned when a session token is not found in the message
INVALID_SESSION_INFORMATION
Failure code returned when an invalid session token is received
NOT_AUTHORIZED
Failure code returned when client is not authorized to perform the operation
MAX_CONNECTION_LIMIT_EXCEEDED
Failure code returned when a client tries to create more connections than allowed to
  
Subscription   Specific to subscription requests
SUBSCRIPTION_LIMIT_EXCEEDED
Customer tried to subscribe to more markets than allowed to
INVALID_CLOCK
Failure code returned when an invalid clock is provided on re-subscription (check initialClk / clk supplied)
  
 General  General errors which may or may not be linked to specific request id
UNEXPECTED_ERROR
Failure code returned when an internal error occurred on the server
CONNECTION_FAILED
Failure code used when the client / server connection is terminated

Connection / ConnectionMessage

This is received by the client when it successfully opens a connection to the server

Key fields:

On establishing a connection a client receives a ConnectionMessage - the connectionId must be logged & supplied on any support queries:

{"op":"connection","connectionId":"002-230915140112-174"}

Authentication / AuthenticationMessage

This message is the first message that the client must send on connecting to the server - you must be authenticated before any other request is processed

Key fields:

Some common authentication errors that you should handle - these are defined on ErrorCodes enum (these will all close your connection):

  • NO_APP_KEY / INVALID_APP_KEY - Check you are using the correct app key
  • NO_SESSION / INVALID_SESSION_INFORMATION - Check the session is current
  • NOT_AUTHORIZED - Check that you are using the correct appkey / session and that it has been setup by BDP

  • MAX_CONNECTION_LIMIT_EXCEEDED - Check that you are not creating too many connections / are closing connections properly.

Subscription / SubscriptionMessage

This message changes the client's subscription - there are currently two subscription message types:

On creating a subscription you will receive:

It is possible to subscribe multiple times - each replaces the previous (each will send a new initial image and deltas) - they are not additive.

Key fields on a SubscriptionMessage:

ChangeMessage

This message is the payload that delivers changes (both initial image & updates) to a client - there are currently two change message types:

Key fields on a ChangeMessage:

heartbeatMs is a guarantee of how often (even with no changes) you will receive a ChangeMessage; i.e.:

If heartbeatMs= 500 and your subscription has not changed in 500ms then we will send an empty ChangeMessage with ct=HEARTBEAT

(this verifies your connection is live and processing data)


Change Message Segmentation

The below shows the key interactions for subscription & changes with segmentation applied:

Typically on changing your subscription you will want to clear any local cache you maintain.

  • How can I detect the start of an initial image & clear my cache?
    • ct=ChangeType.SUB_IMAGE and segmentType=null or SegmentType.SEG_START indicates the start of a new image
  • How can I detect the end of an initial image?
    • ct=ChangeType.SUB_IMAGE and segmentType=null or SegmentType.SEG_END indicates the end of a new image
  • When I change Subscription how do I safely ignore messages for a previous subscription?
    • All ChangeMessages carry have id=<request id> this allows safe disposal during subscription change

MarketSubscriptionMessage

This subscription type is used to receive price changes for one or more markets; your subscription criteria determine what you see.

 

It is preferable to use coarse grain subscriptions (subscribe to a super-set) rather than fine grain (specific market ids).

  • If you find yourself frequently changing subscriptions you probably want to find a wider super-set to subscribe to

 

A MarketSubscription has two types of filter:

Limiting the amount of data that you consume will make your initial image much smaller (and faster) & suppress changes that are uninteresting to you.

Market Filtering / MarketFilter

As with the APING API users have the ability to filter the market data they get from the new Exchange Stream API (ESA).

All subscriptions are evaluated with a few default criteria:

 Users can then specify the following filters when they subscribe to ESA:

Filter nameTypeMandatoryDescription

marketIds

Set<String>

No

If no marketIds passed user will be subscribed to all markets

bspMarket

Boolean

No

Restrict to bsp markets only, if True or non-bsp markets if False. If not specified then returns both BSP and non-BSP markets

bettingTypes

Set<BettingType>

No

Restrict to markets that match the betting type of the market (i.e. Odds, Asian Handicap Singles, or Asian Handicap Doubles)

eventTypeIds

Set<String>

No

Restrict markets by event type associated with the market. (i.e., "1" for Football, "7" for Horse Racing, etc)

eventIds

Set<String>

No

Restrict markets by the event id associated with the market.

turnInPlayEnabled

Boolean

No

Restrict to markets that will turn in play if True or will not turn in play if false. If not specified, returns both.

marketTypes

Set<String>

No

Restrict to markets that match the type of the market (i.e., MATCH_ODDS, HALF_TIME_SCORE). You should use this instead of relying on the market name as the market type codes are the same in all locales

venues

Set<String>

No

Restrict markets by the venue associated with the market. Currently only Horse Racing markets have venues.

countryCodes

Set<String>

No

Restrict to markets that are in the specified country or countries


For example a subscription message with almost all filters enabled will look something like this:  

{"op":"marketSubscription","id":2,"marketFilter":{"marketIds":["1.120684740"],"bspMarket":true,"bettingTypes":["ODDS"],"eventTypeIds":["1"],"eventIds":["27540841"],"turnInPlayEnabled":true,"marketTypes":["MATCH_ODDS"],"countryCodes":["ES"]},"marketDataFilter":{}}

 

We don't verify your subscription criteria as you could potentially subscribe to either a wild card (which would include future markets) or a future marketid which we do not have yet but would send on arrival

Market data field filtering / MarketDataFilter

A market data filter restricts the fields that you get back (and only if the fields have changed).

Key fields:


The field filter flags are defined as:
Filter nameFields:TypeDescription

EX_BEST_OFFERS_DISP

bdatb, bdatl

level, price, size

Best prices including virtual prices - depth is controlled by ladderLevels (1 to 6)

EX_BEST_OFFERS

batb, batl

level, price, sizeBest prices not including virtual prices - depth is controlled by ladderLevels (1 to 6)

EX_ALL_OFFERS

atb, atl

price, sizeFull available to BACK/LAY ladder

EX_TRADED

trd

price, sizeFull traded ladder

EX_TRADED_VOL

tv

sizeMarket and runner level traded volume

EX_LTP

ltp

priceLast traded price

EX_MARKET_DEF

marketDefinition

MarketDefinitionSend market definitions.

SP_TRADED

spb, spl

price, sizeStarting price ladder

SP_PROJECTED

spn, spf

priceStarting price projection prices


Multiple field filters may be combined; a subscription message that contains data fields should look like the following: 

{"op":"marketSubscription","id":2,"marketFilter":{"marketIds":["1.120684740"]},"marketDataFilter":{"fields":["EX_BEST_OFFERS_DISP","EX_BEST_OFFERS","EX_ALL_OFFERS","EX_TRADED","EX_TRADED_VOL","EX_LTP","EX_MARKET_DEF","SP_TRADED","SP_PROJECTED"]}} 


Correctly configuring field filters can help by:

  • Reducing the size (and time) of initial images
  • Reducing the rate of change (as only changes matching your field filter are sent)

 

MC / MarketChangeMessage

This is the ChangeMessage stream of data we send back to you once you subscribe to the market stream. 

Key fields:

Building a price cache

Most of the change based data (RunnerChange) is delta based - this means a few rules:

OrderSubscriptionMessage

This subscription type is used to receive order changes; there are no additional filter criteria (you will see the orders for the account you authenticated with).

OCM / OrderChangeMessage

This is the ChangeMessage stream of data we send back to you once you subscribe to the order stream. 

Key fields:

Building an order cache

An order cache is somewhat simpler as orders are sent in full (on change) and only matches need delta merging

 

Market subscriptions are always in underlying exchange currency - GBP

New subscriptions: Will receive an initial image with only E - Executable orders (unmatched).

Live subscriptions: Will receive a transient of the order to EC - Execution Complete as the order transits into that state (allowing you to remove the order from your cache).

 

Heartbeat / HearbeatMessage

This is an explicit heartbeat request (in addition to server heartbeat interval which is automatic).

This functionality should not normally be necessary unless you need to keep a firewall open.


No - under normal circumstances the subscription level ChangeType.HEARTBEAT is an acceptable guarantee of connection health.

Use the HeartbeatMessage only if you need to keep a firewall open - as it will incur some performance penalty (as a response will block your connection)

Re-connection / Re-subscription

If a client is disconnected a client may connect, authenticate and re-subscribe.

Prerequisite steps:

Connection is broken.

  • Store any new subscription message you send as a "pending subscription"
  • Store this as a "active subscription" once you get your initial image
  • Update the initialClk & clk on the subscription message with any non-null values
  • Resend this message after re-connecting

Performance Considerations

Here are a few tips on performance which are worth bearing in mind:

 

  • A single market subscription & a subscription to all markets have an identical latency:
    • Cost is identical as the two subscriptions above would evaluate in sequence and thus with the same average latency.
    • Initial image is more costly to send than extra updates.
    • Limiting data with appropriate filters reduces initial image time
  • Segmented data will always out perform non-segmented data:
    • You will be processing a buffer while another is in-flight and another is being prepared to send
  • Writes to your connection are directly effected by how quickly you consume data & clear your socket's buffer
    • Consuming data slowly is effectively identical to setting conflation.
    • If you receive conf=true flag on a market - then you are consuming data slower than the rate of deliver

Currency Support

The Exchange Stream API supports GBP currency only.

Those looking to convert data from GBP to a different currency should use listCurrencyRates to do so.

Australian Exchange 

Support for Australian Exchange markets isnt provided via the Exchange Stream API.

Runner Removals on the Order Stream

When a Rule 4 Runner Removal occurs in a Horse Race the price of matched bets on remaining runners are reduced by a Reduction Factor.

For these matched bets, you will receive on the Order Stream both a uo for the affected bet and the relevant updates to mb or ml (reducing the matched volume at the original matched price and adding volume at the new reduced price).

 

Initial bet placement at price 12

 

{"op":"ocm","id":2,"clk":"AK0CAPsBALEC","pt":1467219304831,"oc":[{"id":"1.102151675","orc":[{"fullImage":true,"id":6113662,"uo":[{"id":"10822867886","p":12,"s":2,"side":"B","status":"E","pt":"L","ot":"L","pd":1467219304000,"sm":0,"sr":2,"sl":0,"sc":0,"sv":0,"rac":"","rc":"REG_GGC"}]}]}]}

 

Bet fully matched at price 12

 

{"op":"ocm","id":2,"clk":"AK0CAPsBALMC","pt":1467219316709,"oc":[{"id":"1.102151675","orc":[{"id":6113662,"uo":[{"id":"10822867886","p":12,"s":2,"side":"B","status":"EC","pt":"L","ot":"L","pd":1467219304000,"md":1467219316000,"avp":12,"sm":2,"sr":0,"sl":0,"sc":0,"sv":0}],"mb":[[12,2]]}]}]}

 

Runner removed (and so bet reduced in price to 9.47)

 

{"op":"ocm","id":2,"clk":"AK0CAJACALsC","pt":1467219376611,"oc":[{"id":"1.102151675","orc":[{"id":6113662,"uo":[{"id":"10822867886","p":12,"s":2,"side":"B","status":"EC","pt":"L","ot":"L","pd":1467219304000,"md":1467219316000,"avp":9.47,"sm":2,"sr":0,"sl":0,"sc":0,"sv":0}],"mb":[[9.47,2],[12,0]]}]}]}

 

See the avp in the uo record showing the new price of 9.47 and see the two entries in mb, one to remove the previously added size of 2 at price point 12 and one to add the size of 2 into the new price point of size 9.47.

 

Bets placed on the actual removed runner will be voided/lapsed (for matched/unmatched bets respectively), these updates will not be sent through the order stream. The advice to consumers should be to also consume the market stream, detect that the runner has been marked as REMOVED in an update to the Market Definition and to therefore consider all bets held on that runner to be Void (or Lapsed for unmatched bets).

Sample Applications - C# & Java

A console based C# & Java sample application is available for the Market & Order Streaming API and is available via https://github.com/betfair/stream-api-sample-code