Replaying messages in FIX protocol
In FIX Protocol, two FIX engines communicate with each other using FIX messages and every FIX messages is assign with unique sequence number denoted by tag 34. Apparently every FIX engine has two sequence numbers Incoming Sequence Number (which FIX engine is expecting from a counter party) and Outgoing Sequence Number (which FIX engine is sending to counterparty). This sequence numbers along with rules specified in FIX protocol technical specification ensures that no FIX engine should lose any FIX messages in the event of any disconnect.
In FIX Protocol, two FIX engines communicate with each other using FIX messages and every FIX messages is assign with unique sequence number denoted by tag 34. Apparently every FIX engine has two sequence numbers Incoming Sequence Number (which FIX engine is expecting from a counter party) and Outgoing Sequence Number (which FIX engine is sending to counterparty). This sequence numbers along with rules specified in FIX protocol technical specification ensures that no FIX engine should lose any FIX messages in the event of any disconnect.
In this FIX Protocol Tutorial, we will discuss some scenarios where the disconnect between two FIX Engine occurs and how they recover from that situation. Normally disconnect and reconnect can cause a replay of messages which would be required from either party e.g. either client or broker based upon who has a higher sequence number.
This FIX Protocol Tutorial is in continuation of my earlier tutorial FIX Protocol Session or Admin messages tutorial and Difference between FIX 4.2 vs FIX 4.4 in FIX connectivity.
Now let’s see few scenarios where replay of FIX messages can occur:
1) If End of Day (EOD) didn’t run for any of FIX engine:
Every FIX session has some EOD or End of day timings usually sometimes after market close, on which they reset sequence number both incoming and outgoing to 1/1 and starts a fresh day.
If by any chance FIX EOD doesn't happen on either side then both incoming and outgoing sequence number doesn’t reset to 1/1 on that side and both FIX engine will be out of sequence and when next day when they start connecting to each other login will not be successful, then client FIX engine keeps sending logon message every time increasing sequence number until the broker (acceptor) fix engine accepts the connection. once broker fix engine accept the connection it will reply back with its outgoing sequence number which is not 1 and could be 400 based on a number of messages yesterday.
Since client fix engine incoming sequence number is still 1, it would send a resend request for messages from 1 to 400 and broker will replay those messages. These replay messages could be the trades of yesterday. So to avoid such a scenario its best to have your End of Day job runs on time agreed between broker and client and you should have an alert when it failed.
If you fix the engine provides supports to manual change of sequence number then you can manually adjust those as well. Normally commercial FIX engines like NYFIX Appia supports EOD operation in the process itself but if your FIX engine doesn't support EOD then you could set up an Autosys or Unix Cron job to delete the persistence files related to sequence number so that when process comes up again next day it will create new persistence file with sequence number 1/1.
2) Due to network issues, the connection between Sender FIX Engine and Receiver FIX Engine has been lost and after issue is resolved when they try to reconnect they might be Out of sequence.
3) The sender or Receiver FIX Engine goes down due to Host Failure, In this event also when they try to reconnect each other validation of the Sequence number will be done and a replay will happen if any of having dropped any messages.
4) Due to the restart of either Sender or Receiver FIX Engine.
Also, it's worth noting that to handle replay of FIX messages properly FIX protocol provides two special tags PossDup (fix tag 43) and PossResend (tag 97) which suggest the receiving fix engine that these message has been sent earlier. you can read more about Difference between PossDup and PossResed in FIX Protocol here.
2) Due to network issues, the connection between Sender FIX Engine and Receiver FIX Engine has been lost and after issue is resolved when they try to reconnect they might be Out of sequence.
3) The sender or Receiver FIX Engine goes down due to Host Failure, In this event also when they try to reconnect each other validation of the Sequence number will be done and a replay will happen if any of having dropped any messages.
4) Due to the restart of either Sender or Receiver FIX Engine.
Also, it's worth noting that to handle replay of FIX messages properly FIX protocol provides two special tags PossDup (fix tag 43) and PossResend (tag 97) which suggest the receiving fix engine that these message has been sent earlier. you can read more about Difference between PossDup and PossResed in FIX Protocol here.
Hi, Do you know how to handle out of sequence messages in FIX protocol e.g. Cancel coming before NEW or, Cancel coming before AMEND etc, Please share code to do that in FIX protocol?
ReplyDeleteHello there, what should be the general gap between incoming and outgoing FIX sequence numbers? I believe they should be close to same e.g. if IN=600 then OUT should also be somewhere close to 600? I may be wrong but I need your help, as in our application our OUT sequence is keep falling behind exchange incoming sequence number and we have an issue with Failover and recovery, I am suspecting this may be related to each other, please help.
ReplyDeleteI got asked in interview - if exchange last received sequence number 99 from a client and client FIX engine goes down, recovers corrupt with sequence number 90- what will happen in this case?
ReplyDelete