Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Last night one of our fix connections failed to successfully reconnect, looking the previous days message log I see almost identical messages (diffs being the timestamps…) and slightly different log ordering

 

Successful

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfix.j.event: Initiated logon request

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

 

Failed

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

quickfix.j.event: Initiated logon request

quickfixj.msg.outgoing: 49=me,56=them,35=5,58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

 

I can’t see why quickfix would legitimately reject the incoming message unless the expected seq num is not updated atomically or there’s a race condition – has anyone else seen this behaviour?

 

Cheers

 

Jon




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi,

which QFJ version are you using?

Chris.


On 20/06/17 12:04, Freedman, Jon wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/




Last night one of our fix connections failed to successfully reconnect, looking the previous days message log I see almost identical messages (diffs being the timestamps…) and slightly different log ordering

 

Successful

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfix.j.event: Initiated logon request

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

 

Failed

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

quickfix.j.event: Initiated logon request

quickfixj.msg.outgoing: 49=me,56=them,35=5,58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

 

I can’t see why quickfix would legitimately reject the incoming message unless the expected seq num is not updated atomically or there’s a race condition – has anyone else seen this behaviour?

 

Cheers

 

Jon




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



1.6.3 which I believe is the latest?

 

From: Christoph John [mailto:[hidden email]]
Sent: 20 June 2017 12:13
To: [hidden email]; Freedman, Jon
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

which QFJ version are you using?

Chris.

On 20/06/17 12:04, Freedman, Jon wrote:

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
 
 




Last night one of our fix connections failed to successfully reconnect, looking the previous days message log I see almost identical messages (diffs being the timestamps…) and slightly different log ordering

 

Successful

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfix.j.event: Initiated logon request

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

 

Failed

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

quickfix.j.event: Initiated logon request

quickfixj.msg.outgoing: 49=me,56=them,35=5,58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

 

I can’t see why quickfix would legitimately reject the incoming message unless the expected seq num is not updated atomically or there’s a race condition – has anyone else seen this behaviour?

 

Cheers

 

Jon




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot




_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Yes, it is the latest released version. You could also try the 1.6.4-SNAPSHOT but I don't think it will change something as there were no changes around this.
Are you able to reliably reproduce this (maybe even in a unit test)?

However, there is one issue which also deals with the session reset on startup of the connection. It might be related. http://www.quickfixj.org/jira/browse/QFJ-926

Cheers,
Chris.


On 20/06/17 15:09, Freedman, Jon wrote:

1.6.3 which I believe is the latest?

 

From: Christoph John [[hidden email]]
Sent: 20 June 2017 12:13
To: [hidden email]; Freedman, Jon
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

which QFJ version are you using?

Chris.

On 20/06/17 12:04, Freedman, Jon wrote:

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
 
 




Last night one of our fix connections failed to successfully reconnect, looking the previous days message log I see almost identical messages (diffs being the timestamps…) and slightly different log ordering

 

Successful

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfix.j.event: Initiated logon request

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

 

Failed

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

quickfix.j.event: Initiated logon request

quickfixj.msg.outgoing: 49=me,56=them,35=5,58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

 

I can’t see why quickfix would legitimately reject the incoming message unless the expected seq num is not updated atomically or there’s a race condition – has anyone else seen this behaviour?

 

Cheers

 

Jon




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot




_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary


--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



I’ve only seen this once – I will take a look at the JIRA issue, do you have a good example test case that I could start with?

 

From: Christoph John [mailto:[hidden email]]
Sent: 20 June 2017 14:53
To: Freedman, Jon; [hidden email]
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Yes, it is the latest released version. You could also try the 1.6.4-SNAPSHOT but I don't think it will change something as there were no changes around this.
Are you able to reliably reproduce this (maybe even in a unit test)?

However, there is one issue which also deals with the session reset on startup of the connection. It might be related. http://www.quickfixj.org/jira/browse/QFJ-926

Cheers,
Chris.

On 20/06/17 15:09, Freedman, Jon wrote:

1.6.3 which I believe is the latest?

 

From: Christoph John [[hidden email]]
Sent: 20 June 2017 12:13
To: [hidden email]; Freedman, Jon
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

which QFJ version are you using?

Chris.


On 20/06/17 12:04, Freedman, Jon wrote:

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
 
 





Last night one of our fix connections failed to successfully reconnect, looking the previous days message log I see almost identical messages (diffs being the timestamps…) and slightly different log ordering

 

Successful

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfix.j.event: Initiated logon request

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

 

Failed

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

quickfix.j.event: Initiated logon request

quickfixj.msg.outgoing: 49=me,56=them,35=5,58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

 

I can’t see why quickfix would legitimately reject the incoming message unless the expected seq num is not updated atomically or there’s a race condition – has anyone else seen this behaviour?

 

Cheers

 

Jon




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot





_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Looks like this happened again – could you point me at an existing unit test I could use as a basis for my testing?

 

From: Freedman, Jon
Sent: 20 June 2017 14:58
To: '[hidden email]'; [hidden email]
Subject: RE: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

I’ve only seen this once – I will take a look at the JIRA issue, do you have a good example test case that I could start with?

 

From: Christoph John [[hidden email]]
Sent: 20 June 2017 14:53
To: Freedman, Jon; [hidden email]
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Yes, it is the latest released version. You could also try the 1.6.4-SNAPSHOT but I don't think it will change something as there were no changes around this.
Are you able to reliably reproduce this (maybe even in a unit test)?

However, there is one issue which also deals with the session reset on startup of the connection. It might be related. http://www.quickfixj.org/jira/browse/QFJ-926

Cheers,
Chris.

On 20/06/17 15:09, Freedman, Jon wrote:

1.6.3 which I believe is the latest?

 

From: Christoph John [[hidden email]]
Sent: 20 June 2017 12:13
To: [hidden email]; Freedman, Jon
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

which QFJ version are you using?

Chris.

On 20/06/17 12:04, Freedman, Jon wrote:

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
 
 




Last night one of our fix connections failed to successfully reconnect, looking the previous days message log I see almost identical messages (diffs being the timestamps…) and slightly different log ordering

 

Successful

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfix.j.event: Initiated logon request

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

 

Failed

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

quickfix.j.event: Initiated logon request

quickfixj.msg.outgoing: 49=me,56=them,35=5,58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

 

I can’t see why quickfix would legitimately reject the incoming message unless the expected seq num is not updated atomically or there’s a race condition – has anyone else seen this behaviour?

 

Cheers

 

Jon




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot




_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/





On 26/06/17 10:44, Christoph John wrote:
Hi,

sorry, am quite busy. Here are some tests that you could use as a starting point.

This one tests quite basic behaviour: https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionTest.java

These were created for specific problems, so maybe they are a better starting point. They both use multi-threading to provoke said problems.
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionResetTest.java
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectConcurrentlyTest.java

If you need any assistance just ask, but might take a while to answer. ;)

Thanks,
Chris.


On 26/06/17 09:04, Freedman, Jon wrote:

Looks like this happened again – could you point me at an existing unit test I could use as a basis for my testing?

 

From: Freedman, Jon
Sent: 20 June 2017 14:58
To: '[hidden email]'; [hidden email]
Subject: RE: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

I’ve only seen this once – I will take a look at the JIRA issue, do you have a good example test case that I could start with?

 

From: Christoph John [[hidden email]]
Sent: 20 June 2017 14:53
To: Freedman, Jon; [hidden email]
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Yes, it is the latest released version. You could also try the 1.6.4-SNAPSHOT but I don't think it will change something as there were no changes around this.
Are you able to reliably reproduce this (maybe even in a unit test)?

However, there is one issue which also deals with the session reset on startup of the connection. It might be related. http://www.quickfixj.org/jira/browse/QFJ-926

Cheers,
Chris.

On 20/06/17 15:09, Freedman, Jon wrote:

1.6.3 which I believe is the latest?

 

From: Christoph John [[hidden email]]
Sent: 20 June 2017 12:13
To: [hidden email]; Freedman, Jon
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

which QFJ version are you using?

Chris.

On 20/06/17 12:04, Freedman, Jon wrote:

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
 
 




Last night one of our fix connections failed to successfully reconnect, looking the previous days message log I see almost identical messages (diffs being the timestamps…) and slightly different log ordering

 

Successful

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfix.j.event: Initiated logon request

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

 

Failed

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

quickfix.j.event: Initiated logon request

quickfixj.msg.outgoing: 49=me,56=them,35=5,58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

 

I can’t see why quickfix would legitimately reject the incoming message unless the expected seq num is not updated atomically or there’s a race condition – has anyone else seen this behaviour?

 

Cheers

 

Jon




--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
In reply to this post by Freedman, Jon
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Thanks Chris, looking at this now

 

From: Christoph John [mailto:[hidden email]]
Sent: 26 June 2017 09:44
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

sorry, am quite busy. Here are some tests that you could use as a starting point.

This one tests quite basic behaviour: https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionTest.java

These were created for specific problems, so maybe they are a better starting point. They both use multi-threading to provoke said problems.
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionResetTest.java
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectConcurrentlyTest.java

If you need any assistance just ask, but might take a while to answer. ;)

Thanks,
Chris.

On 26/06/17 09:04, Freedman, Jon wrote:

Looks like this happened again – could you point me at an existing unit test I could use as a basis for my testing?

 

From: Freedman, Jon
Sent: 20 June 2017 14:58
To: '[hidden email]'; [hidden email]
Subject: RE: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

I’ve only seen this once – I will take a look at the JIRA issue, do you have a good example test case that I could start with?

 

From: Christoph John [[hidden email]]
Sent: 20 June 2017 14:53
To: Freedman, Jon; [hidden email]
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Yes, it is the latest released version. You could also try the 1.6.4-SNAPSHOT but I don't think it will change something as there were no changes around this.
Are you able to reliably reproduce this (maybe even in a unit test)?

However, there is one issue which also deals with the session reset on startup of the connection. It might be related. http://www.quickfixj.org/jira/browse/QFJ-926

Cheers,
Chris.

On 20/06/17 15:09, Freedman, Jon wrote:

1.6.3 which I believe is the latest?

 

From: Christoph John [[hidden email]]
Sent: 20 June 2017 12:13
To: [hidden email]; Freedman, Jon
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

which QFJ version are you using?

Chris.


On 20/06/17 12:04, Freedman, Jon wrote:

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/
 
 





Last night one of our fix connections failed to successfully reconnect, looking the previous days message log I see almost identical messages (diffs being the timestamps…) and slightly different log ordering

 

Successful

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfix.j.event: Initiated logon request

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

 

Failed

quickfixj.event: Session state is not current; resetting

quickfixj.msg.outgoing: 49=me,56=them,35=A,34=1,789=1

quickfixj.msg.incoming: 49=them,56=me,35=A,34=1,789=2

quickfix.j.event: Initiated logon request

quickfixj.msg.outgoing: 49=me,56=them,35=5,58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

 

I can’t see why quickfix would legitimately reject the incoming message unless the expected seq num is not updated atomically or there’s a race condition – has anyone else seen this behaviour?

 

Cheers

 

Jon




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot





_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
In reply to this post by Freedman, Jon
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Ok I believe I have found the source of the issue, is it possible there is a race condition between persisting a message & incrementing the next sender seq num in Session#sendRaw and receiving a logon response on another thread?

 

I have a test which is replicating the behaviour observed in logs in a sensible manner (we are using JdbcStoreFactory) here: https://github.com/jonfreedman/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectTest.java

 

When running I get the following:

<20170626-13:47:12, FIX.4.4:US->THEM, event> (Session FIX.4.4:US->THEM schedule is null)

<20170626-13:47:13, FIX.4.4:US->THEM, event> (Created session: FIX.4.4:US->THEM)

to admin [FIX.4.4:US->THEM] 8=FIX.4.49=6735=A34=149=US52=20170626-13:47:13.06856=THEM98=0108=60789=110=252

<20170626-13:47:13, FIX.4.4:US->THEM, outgoing> (8=FIX.4.49=6735=A34=149=US52=20170626-13:47:13.06856=THEM98=0108=60789=110=252)

from admin [FIX.4.4:US->THEM] 8=FIX.4.49=6735=A34=149=THEM52=20170626-13:47:13.07356=US98=0108=60789=210=249

<20170626-13:47:13, FIX.4.4:US->THEM, event> (Initiated logon request)

to admin [FIX.4.4:US->THEM] 8=FIX.4.49=13235=534=249=US52=20170626-13:47:13.07556=THEM58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 210=156

<20170626-13:47:13, FIX.4.4:US->THEM, outgoing> (8=FIX.4.49=13235=534=249=US52=20170626-13:47:13.07556=THEM58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 210=156)

<20170626-13:47:13, FIX.4.4:US->THEM, error> (Disconnecting: Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2)

 

Let me know if this sounds feasible and if you’d like me to tidy the test up / send a PR.  I’d also be willing to look at fixing if my hypothesis is correct with a few pointers…

 

Cheers

 

Jon

 

From: Christoph John [mailto:[hidden email]]
Sent: 26 June 2017 09:44
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

sorry, am quite busy. Here are some tests that you could use as a starting point.

This one tests quite basic behaviour: https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionTest.java

These were created for specific problems, so maybe they are a better starting point. They both use multi-threading to provoke said problems.
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionResetTest.java
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectConcurrentlyTest.java

If you need any assistance just ask, but might take a while to answer. ;)

Thanks,
Chris.




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi Jon,

somehow your mail got through twice but with different size?! Maybe the first one was held back due to size restrictions. In that case please make sure to cancel the old posting to prevent double posts. Thank you.

I've looked at your test and the QFJ code and I think you are right about the race condition. In the case where 789/NextExpectedMsgSeqNum is used on an incoming Logon message, QFJ checks if the next outgoing sequence number is the same as on the incoming Logon. But it could happen that the sequence number has not been incremented yet although the message has been sent out already.
This is odd, since for years I've thought that the message is first put to the store and sent out afterwards. Quickly checked the code and that was actually the case in QuickFIX for C++. Why it is the other way round in QFJ is beyond me.
However, this has no effect on the problem since there is a lock around the sequence number which is not released until after the message has been sent out and persisted. I.e. the correct value can only be read after Session.sendRaw() returns.

So one could simply wait for the lock to be released to read the correct value. Since this is only done rarely (only on Logon and only when tag 789 is used) it might be a feasible solution. But I'll think a while more about it.
On the other hand: what happens when this problem occurs? I assume the next Logon is successful then? (but since it is a race condition it is not guaranteed, though ;))

Cheers,
Chris.



On 26/06/17 18:04, Freedman, Jon wrote:

Ok I believe I have found the source of the issue, is it possible there is a race condition between persisting a message & incrementing the next sender seq num in Session#sendRaw and receiving a logon response on another thread?

 

I have a test which is replicating the behaviour observed in logs in a sensible manner (we are using JdbcStoreFactory) here: https://github.com/jonfreedman/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectTest.java

 

When running I get the following:

<20170626-13:47:12, FIX.4.4:US->THEM, event> (Session FIX.4.4:US->THEM schedule is null)

<20170626-13:47:13, FIX.4.4:US->THEM, event> (Created session: FIX.4.4:US->THEM)

to admin [FIX.4.4:US->THEM] 8=FIX.4.49=6735=A34=149=US52=20170626-13:47:13.06856=THEM98=0108=60789=110=252

<20170626-13:47:13, FIX.4.4:US->THEM, outgoing> (8=FIX.4.49=6735=A34=149=US52=20170626-13:47:13.06856=THEM98=0108=60789=110=252)

from admin [FIX.4.4:US->THEM] 8=FIX.4.49=6735=A34=149=THEM52=20170626-13:47:13.07356=US98=0108=60789=210=249

<20170626-13:47:13, FIX.4.4:US->THEM, event> (Initiated logon request)

to admin [FIX.4.4:US->THEM] 8=FIX.4.49=13235=534=249=US52=20170626-13:47:13.07556=THEM58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 210=156

<20170626-13:47:13, FIX.4.4:US->THEM, outgoing> (8=FIX.4.49=13235=534=249=US52=20170626-13:47:13.07556=THEM58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 210=156)

<20170626-13:47:13, FIX.4.4:US->THEM, error> (Disconnecting: Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2)

 

Let me know if this sounds feasible and if you’d like me to tidy the test up / send a PR.  I’d also be willing to look at fixing if my hypothesis is correct with a few pointers…

 

Cheers

 

Jon

 

From: Christoph John [[hidden email]]
Sent: 26 June 2017 09:44
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

sorry, am quite busy. Here are some tests that you could use as a starting point.

This one tests quite basic behaviour: https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionTest.java

These were created for specific problems, so maybe they are a better starting point. They both use multi-threading to provoke said problems.
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionResetTest.java
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectConcurrentlyTest.java

If you need any assistance just ask, but might take a while to answer. ;)

Thanks,
Chris.




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).

--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



I did attempt to remove one of my two replies, I guess that didn’t work…

 

When we see this occur we get into a state which has to be recovered manually with a combination of resend & seq num reset requests, on Saturday we had an engine running for 8 hours logging the following every 60 secs (when a heartbeat is received)

 

FIX.4.4:US->THEM: Enqueued at pos 477: 8=FIX.4.4|9=69|35=0|34=477|49=THEM|52=20170624-06:54:17.949|56=US|10=248

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 477: 8=FIX.4.4|9=69|35=0|34=477|49=THEM|52=20170624-06:54:17.949|56=US|10=248

FIX.4.4:US->THEM: 8=FIX.4.4|9=75|35=0|34=477|49=US|52=20170624-06:54:16.942|56=THEM|369=0|10=253

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

 

If it auto-recovered I probably wouldn’t even have noticed.

 

Cheers

 

Jon

 

From: Christoph John [mailto:[hidden email]]
Sent: 27 June 2017 09:19
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

somehow your mail got through twice but with different size?! Maybe the first one was held back due to size restrictions. In that case please make sure to cancel the old posting to prevent double posts. Thank you.

I've looked at your test and the QFJ code and I think you are right about the race condition. In the case where 789/NextExpectedMsgSeqNum is used on an incoming Logon message, QFJ checks if the next outgoing sequence number is the same as on the incoming Logon. But it could happen that the sequence number has not been incremented yet although the message has been sent out already.
This is odd, since for years I've thought that the message is first put to the store and sent out afterwards. Quickly checked the code and that was actually the case in QuickFIX for C++. Why it is the other way round in QFJ is beyond me.
However, this has no effect on the problem since there is a lock around the sequence number which is not released until after the message has been sent out and persisted. I.e. the correct value can only be read after Session.sendRaw() returns.

So one could simply wait for the lock to be released to read the correct value. Since this is only done rarely (only on Logon and only when tag 789 is used) it might be a feasible solution. But I'll think a while more about it.
On the other hand: what happens when this problem occurs? I assume the next Logon is successful then? (but since it is a race condition it is not guaranteed, though ;))

Cheers,
Chris.


On 26/06/17 18:04, Freedman, Jon wrote:

Ok I believe I have found the source of the issue, is it possible there is a race condition between persisting a message & incrementing the next sender seq num in Session#sendRaw and receiving a logon response on another thread?

 

I have a test which is replicating the behaviour observed in logs in a sensible manner (we are using JdbcStoreFactory) here: https://github.com/jonfreedman/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectTest.java

 

When running I get the following:

<20170626-13:47:12, FIX.4.4:US->THEM, event> (Session FIX.4.4:US->THEM schedule is null)

<20170626-13:47:13, FIX.4.4:US->THEM, event> (Created session: FIX.4.4:US->THEM)

to admin [FIX.4.4:US->THEM] 8=FIX.4.49=6735=A34=149=US52=20170626-13:47:13.06856=THEM98=0108=60789=110=252

<20170626-13:47:13, FIX.4.4:US->THEM, outgoing> (8=FIX.4.49=6735=A34=149=US52=20170626-13:47:13.06856=THEM98=0108=60789=110=252)

from admin [FIX.4.4:US->THEM] 8=FIX.4.49=6735=A34=149=THEM52=20170626-13:47:13.07356=US98=0108=60789=210=249

<20170626-13:47:13, FIX.4.4:US->THEM, event> (Initiated logon request)

to admin [FIX.4.4:US->THEM] 8=FIX.4.49=13235=534=249=US52=20170626-13:47:13.07556=THEM58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 210=156

<20170626-13:47:13, FIX.4.4:US->THEM, outgoing> (8=FIX.4.49=13235=534=249=US52=20170626-13:47:13.07556=THEM58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 210=156)

<20170626-13:47:13, FIX.4.4:US->THEM, error> (Disconnecting: Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2)

 

Let me know if this sounds feasible and if you’d like me to tidy the test up / send a PR.  I’d also be willing to look at fixing if my hypothesis is correct with a few pointers…

 

Cheers

 

Jon

 

From: Christoph John [[hidden email]]
Sent: 26 June 2017 09:44
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi,

sorry, am quite busy. Here are some tests that you could use as a starting point.

This one tests quite basic behaviour: https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionTest.java

These were created for specific problems, so maybe they are a better starting point. They both use multi-threading to provoke said problems.
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionResetTest.java
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectConcurrentlyTest.java

If you need any assistance just ask, but might take a while to answer. ;)

Thanks,
Chris.




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



OK, but what about the ResendRequest? Didn't the other side resend the asked messages?

Chris.


On 27/06/17 10:44, Freedman, Jon wrote:

I did attempt to remove one of my two replies, I guess that didn’t work…

 

When we see this occur we get into a state which has to be recovered manually with a combination of resend & seq num reset requests, on Saturday we had an engine running for 8 hours logging the following every 60 secs (when a heartbeat is received)

 

FIX.4.4:US->THEM: Enqueued at pos 477: 8=FIX.4.4|9=69|35=0|34=477|49=THEM|52=20170624-06:54:17.949|56=US|10=248

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 477: 8=FIX.4.4|9=69|35=0|34=477|49=THEM|52=20170624-06:54:17.949|56=US|10=248

FIX.4.4:US->THEM: 8=FIX.4.4|9=75|35=0|34=477|49=US|52=20170624-06:54:16.942|56=THEM|369=0|10=253

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

 

If it auto-recovered I probably wouldn’t even have noticed.

 

Cheers

 

Jon

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 09:19
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

somehow your mail got through twice but with different size?! Maybe the first one was held back due to size restrictions. In that case please make sure to cancel the old posting to prevent double posts. Thank you.

I've looked at your test and the QFJ code and I think you are right about the race condition. In the case where 789/NextExpectedMsgSeqNum is used on an incoming Logon message, QFJ checks if the next outgoing sequence number is the same as on the incoming Logon. But it could happen that the sequence number has not been incremented yet although the message has been sent out already.
This is odd, since for years I've thought that the message is first put to the store and sent out afterwards. Quickly checked the code and that was actually the case in QuickFIX for C++. Why it is the other way round in QFJ is beyond me.
However, this has no effect on the problem since there is a lock around the sequence number which is not released until after the message has been sent out and persisted. I.e. the correct value can only be read after Session.sendRaw() returns.

So one could simply wait for the lock to be released to read the correct value. Since this is only done rarely (only on Logon and only when tag 789 is used) it might be a feasible solution. But I'll think a while more about it.
On the other hand: what happens when this problem occurs? I assume the next Logon is successful then? (but since it is a race condition it is not guaranteed, though ;))

Cheers,
Chris.


On 26/06/17 18:04, Freedman, Jon wrote:

Ok I believe I have found the source of the issue, is it possible there is a race condition between persisting a message & incrementing the next sender seq num in Session#sendRaw and receiving a logon response on another thread?

 

I have a test which is replicating the behaviour observed in logs in a sensible manner (we are using JdbcStoreFactory) here: https://github.com/jonfreedman/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/SessionDisconnectTest.java

 

When running I get the following:

<20170626-13:47:12, FIX.4.4:US->THEM, event> (Session FIX.4.4:US->THEM schedule is null)

<20170626-13:47:13, FIX.4.4:US->THEM, event> (Created session: FIX.4.4:US->THEM)

to admin [FIX.4.4:US->THEM] 8=FIX.4.49=6735=A34=149=US52=20170626-13:47:13.06856=THEM98=0108=60789=110=252

<20170626-13:47:13, FIX.4.4:US->THEM, outgoing> (8=FIX.4.49=6735=A34=149=US52=20170626-13:47:13.06856=THEM98=0108=60789=110=252)

from admin [FIX.4.4:US->THEM] 8=FIX.4.49=6735=A34=149=THEM52=20170626-13:47:13.07356=US98=0108=60789=210=249

<20170626-13:47:13, FIX.4.4:US->THEM, event> (Initiated logon request)

to admin [FIX.4.4:US->THEM] 8=FIX.4.49=13235=534=249=US52=20170626-13:47:13.07556=THEM58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 210=156

<20170626-13:47:13, FIX.4.4:US->THEM, outgoing> (8=FIX.4.49=13235=534=249=US52=20170626-13:47:13.07556=THEM58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 210=156)

<20170626-13:47:13, FIX.4.4:US->THEM, error> (Disconnecting: Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2)

 

Let me know if this sounds feasible and if you’d like me to tidy the test up / send a PR.  I’d also be willing to look at fixing if my hypothesis is correct with a few pointers…

 

Cheers

 

Jon

 


--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi Jon,

fair enough. But I don't see that the counterparty really satisfies the ResendRequest, i.e. by resending the messages or sending a SequenceReset. Am I missing something?

Chris.


On 27/06/17 10:59, Freedman, Jon wrote:

Here’s the log for the first 7min from midnight Saturday onward – no interaction from us, this is all quickfix attempting and failing to recover

 

FIX.4.4:US->THEM: Enqueued at pos 10: 8=FIX.4.4|9=68|35=0|34=10|49=THEM|52=20170623-23:07:16.955|56=US|10=174

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

FIX.4.4:US->THEM: 8=FIX.4.4|9=74|35=0|34=10|49=US|52=20170623-23:07:16.943|56=THEM|369=0|10=184

FIX.4.4:US->THEM: Enqueued at pos 9: 8=FIX.4.4|9=67|35=0|34=9|49=THEM|52=20170623-23:06:16.955|56=US|10=132

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 9: 8=FIX.4.4|9=67|35=0|34=9|49=THEM|52=20170623-23:06:16.955|56=US|10=132

FIX.4.4:US->THEM: 8=FIX.4.4|9=67|35=0|34=9|49=THEM|52=20170623-23:06:16.955|56=US|10=132

FIX.4.4:US->THEM: 8=FIX.4.4|9=73|35=0|34=9|49=US|52=20170623-23:06:16.943|56=THEM|369=0|10=142

FIX.4.4:US->THEM: 8=FIX.4.4|9=67|35=0|34=8|49=THEM|52=20170623-23:05:16.955|56=US|10=130

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 8: 8=FIX.4.4|9=67|35=0|34=8|49=THEM|52=20170623-23:05:16.955|56=US|10=130

FIX.4.4:US->THEM: Enqueued at pos 8: 8=FIX.4.4|9=67|35=0|34=8|49=THEM|52=20170623-23:05:16.955|56=US|10=130

FIX.4.4:US->THEM: 8=FIX.4.4|9=73|35=0|34=8|49=US|52=20170623-23:05:16.943|56=THEM|369=0|10=140

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

FIX.4.4:US->THEM: Enqueued at pos 7: 8=FIX.4.4|9=67|35=0|34=7|49=THEM|52=20170623-23:04:16.955|56=US|10=128

FIX.4.4:US->THEM: 8=FIX.4.4|9=67|35=0|34=7|49=THEM|52=20170623-23:04:16.955|56=US|10=128

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 7: 8=FIX.4.4|9=67|35=0|34=7|49=THEM|52=20170623-23:04:16.955|56=US|10=128

FIX.4.4:US->THEM: 8=FIX.4.4|9=73|35=0|34=7|49=US|52=20170623-23:04:16.943|56=THEM|369=0|10=138

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 6: 8=FIX.4.4|9=67|35=0|34=6|49=THEM|52=20170623-23:03:16.955|56=US|10=126

FIX.4.4:US->THEM: Enqueued at pos 6: 8=FIX.4.4|9=67|35=0|34=6|49=THEM|52=20170623-23:03:16.955|56=US|10=126

FIX.4.4:US->THEM: 8=FIX.4.4|9=67|35=0|34=6|49=THEM|52=20170623-23:03:16.955|56=US|10=126

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

FIX.4.4:US->THEM: 8=FIX.4.4|9=73|35=0|34=6|49=US|52=20170623-23:03:16.943|56=THEM|369=0|10=136

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 5: 8=FIX.4.4|9=67|35=0|34=5|49=THEM|52=20170623-23:02:16.955|56=US|10=124

FIX.4.4:US->THEM: Enqueued at pos 5: 8=FIX.4.4|9=67|35=0|34=5|49=THEM|52=20170623-23:02:16.955|56=US|10=124

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

FIX.4.4:US->THEM: 8=FIX.4.4|9=67|35=0|34=5|49=THEM|52=20170623-23:02:16.955|56=US|10=124

FIX.4.4:US->THEM: 8=FIX.4.4|9=73|35=0|34=5|49=US|52=20170623-23:02:16.943|56=THEM|369=0|10=134

FIX.4.4:US->THEM: Enqueued at pos 4: 8=FIX.4.4|9=67|35=0|34=4|49=THEM|52=20170623-23:01:16.955|56=US|10=122

FIX.4.4:US->THEM: 8=FIX.4.4|9=67|35=0|34=4|49=THEM|52=20170623-23:01:16.955|56=US|10=122

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 4: 8=FIX.4.4|9=67|35=0|34=4|49=THEM|52=20170623-23:01:16.955|56=US|10=122

FIX.4.4:US->THEM: 8=FIX.4.4|9=73|35=0|34=4|49=US|52=20170623-23:01:16.943|56=THEM|369=0|10=132

FIX.4.4:US->THEM: Required resend will be suppressed as we are setting tag 789

FIX.4.4:US->THEM: Enqueued at pos 3: 8=FIX.4.4|9=96|35=A|34=3|49=THEM|52=20170623-23:00:16.955|56=US|98=0|108=60|383=262144|789=4|10=208

FIX.4.4:US->THEM: Received logon

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 3: 8=FIX.4.4|9=96|35=A|34=3|49=THEM|52=20170623-23:00:16.955|56=US|98=0|108=60|383=262144|789=4|10=208

SessionId: FIX.4.4:US->THEM logged on

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

FIX.4.4:US->THEM: Initiated logon request

FIX.4.4:US->THEM: 8=FIX.4.4|9=96|35=A|34=3|49=THEM|52=20170623-23:00:16.955|56=US|383=262144|789=4|98=0|108=60|10=208

Checking if FRIDAY 23:00:16 is valid for logon

FIX.4.4:US->THEM: 8=FIX.4.4|9=91|35=A|34=3|49=US|52=20170623-23:00:16.943|56=THEM|369=0|98=0|108=60|789=1|10=198

Connected

MINA session created for FIX.4.4:US->THEM: local=/[ip-removed], class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/[ip-removed]

[FIX.4.4:US->THEM] - reset IoConnector

FIX.4.4:US->THEM: Already disconnected: Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2

SessionId: FIX.4.4:US->THEM logged out

Disconnected

FIX.4.4:US->THEM: Disconnecting: Socket exception (/[ip-removed]): java.io.IOException: Connection reset by peer

FIX.4.4:US->THEM: Initiated logon request

FIX.4.4:US->THEM: 8=FIX.4.4|9=156|35=5|34=2|49=US|52=20170623-23:00:01.979|56=THEM|369=0|58=Tag 789 (NextExpectedMsgSeqNum) is higher than expected. Expected 1, Received 2|10=114

FIX.4.4:US->THEM: 8=FIX.4.4|9=96|35=A|34=1|49=THEM|52=20170623-23:00:01.955|56=US|383=262144|789=2|98=0|108=60|10=198

Checking if FRIDAY 23:00:01 is valid for logon

FIX.4.4:US->THEM: 8=FIX.4.4|9=91|35=A|34=1|49=US|52=20170623-23:00:01.943|56=THEM|369=0|98=0|108=60|789=1|10=190

Connected

MINA session created for FIX.4.4:US->THEM: local=/[ip-removed], class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/[ip-removed]

Reset

FIX.4.4:US->THEM: Session state is not current; resetting FIX.4.4:US->THEM

Reconnecting according to schedule

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 09:51
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

OK, but what about the ResendRequest? Didn't the other side resend the asked messages?

Chris.

On 27/06/17 10:44, Freedman, Jon wrote:

I did attempt to remove one of my two replies, I guess that didn’t work…

 

When we see this occur we get into a state which has to be recovered manually with a combination of resend & seq num reset requests, on Saturday we had an engine running for 8 hours logging the following every 60 secs (when a heartbeat is received)

 

FIX.4.4:US->THEM: Enqueued at pos 477: 8=FIX.4.4|9=69|35=0|34=477|49=THEM|52=20170624-06:54:17.949|56=US|10=248

FIX.4.4:US->THEM: MsgSeqNum too high, expecting 1 but received 477: 8=FIX.4.4|9=69|35=0|34=477|49=THEM|52=20170624-06:54:17.949|56=US|10=248

FIX.4.4:US->THEM: 8=FIX.4.4|9=75|35=0|34=477|49=US|52=20170624-06:54:16.942|56=THEM|369=0|10=253

FIX.4.4:US->THEM: Already sent ResendRequest FROM: 1 TO: 0. Not sending another.

 

If it auto-recovered I probably wouldn’t even have noticed.

 

Cheers

 

Jon

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 09:19
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 



--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



There was a resend request implicitly sent via the usage of tag 789 on the Logon message that QFJ sent (there is 789=1).
Now the counterparty should respond with either the resent messages or a SequenceReset message. I can see neither in the logs.

Chris.



On 27/06/17 12:02, Freedman, Jon wrote:

No I think you are correct – it looks to me that quickfix is supressing the resends because it somehow thinks one has already been sent

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 10:17
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

fair enough. But I don't see that the counterparty really satisfies the ResendRequest, i.e. by resending the messages or sending a SequenceReset. Am I missing something?

Chris.


--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



I will query this with the venue, their documentation states tag 789 is supported.

 

From: Christoph John [mailto:[hidden email]]
Sent: 27 June 2017 11:08
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

There was a resend request implicitly sent via the usage of tag 789 on the Logon message that QFJ sent (there is 789=1).
Now the counterparty should respond with either the resent messages or a SequenceReset message. I can see neither in the logs.

Chris.


On 27/06/17 12:02, Freedman, Jon wrote:

No I think you are correct – it looks to me that quickfix is supressing the resends because it somehow thinks one has already been sent

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 10:17
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

fair enough. But I don't see that the counterparty really satisfies the ResendRequest, i.e. by resending the messages or sending a SequenceReset. Am I missing something?

Chris.

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
In reply to this post by QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



The counterparty have confirmed they should have sent a gap fill but did not.  They’re looking into correcting that.

 

In the meantime is it possible to work on fixing the race condition?  I am happy to help with that.

 

From: Christoph John [mailto:[hidden email]]
Sent: 27 June 2017 11:08
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

There was a resend request implicitly sent via the usage of tag 789 on the Logon message that QFJ sent (there is 789=1).
Now the counterparty should respond with either the resent messages or a SequenceReset message. I can see neither in the logs.

Chris.


On 27/06/17 12:02, Freedman, Jon wrote:

No I think you are correct – it looks to me that quickfix is supressing the resends because it somehow thinks one has already been sent

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 10:17
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

fair enough. But I don't see that the counterparty really satisfies the ResendRequest, i.e. by resending the messages or sending a SequenceReset. Am I missing something?

Chris.

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi Jon,

what about my suggestion from an earlier mail:
So one could simply wait for the lock to be released to read the correct value. Since this is only done rarely (only on Logon and only when tag 789 is used) it might be a feasible solution. But I'll think a while more about it.

Do you have a more elegant solution? ;) Until now I don't have a more sophisticated solution. And it probably is the only thing you can do.
But I'd rather not wait indefinitely for the lock to be released. Question is what happens when after a given time (1 second??) the lock cannot be obtained? Abort the Logon or simply obtain the sequence number without the lock with knowing that it might be wrong?

Chris.



On 04/07/17 12:53, Freedman, Jon wrote:

The counterparty have confirmed they should have sent a gap fill but did not.  They’re looking into correcting that.

 

In the meantime is it possible to work on fixing the race condition?  I am happy to help with that.

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 11:08
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

There was a resend request implicitly sent via the usage of tag 789 on the Logon message that QFJ sent (there is 789=1).
Now the counterparty should respond with either the resent messages or a SequenceReset message. I can see neither in the logs.

Chris.


On 27/06/17 12:02, Freedman, Jon wrote:

No I think you are correct – it looks to me that quickfix is supressing the resends because it somehow thinks one has already been sent

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 10:17
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

fair enough. But I don't see that the counterparty really satisfies the ResendRequest, i.e. by resending the messages or sending a SequenceReset. Am I missing something?

Chris.

 


--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Guido Medina
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi all,

I wonder if this can simply be done with an AtomicInteger? for reset you can do something like compareAndSet, 1 or 2 threads accessing an AtomicInteger will cause a very tiny performance impact.
Though I'm not looking at the code so maybe what I'm proposing makes no sense.

Cheers,

Guido.

On Tue, Jul 4, 2017 at 12:32 PM, Christoph John via Quickfixj-users <[hidden email]> wrote:
QuickFIX/J Documentation: <a href="http://www.quickfixj.org/documentation/ QuickFIX/J" rel="noreferrer" target="_blank">http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi Jon,

what about my suggestion from an earlier mail:
So one could simply wait for the lock to be released to read the correct value. Since this is only done rarely (only on Logon and only when tag 789 is used) it might be a feasible solution. But I'll think a while more about it.

Do you have a more elegant solution? ;) Until now I don't have a more sophisticated solution. And it probably is the only thing you can do.
But I'd rather not wait indefinitely for the lock to be released. Question is what happens when after a given time (1 second??) the lock cannot be obtained? Abort the Logon or simply obtain the sequence number without the lock with knowing that it might be wrong?

Chris.



On 04/07/17 12:53, Freedman, Jon wrote:

The counterparty have confirmed they should have sent a gap fill but did not.  They’re looking into correcting that.

 

In the meantime is it possible to work on fixing the race condition?  I am happy to help with that.

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 11:08
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

There was a resend request implicitly sent via the usage of tag 789 on the Logon message that QFJ sent (there is 789=1).
Now the counterparty should respond with either the resent messages or a SequenceReset message. I can see neither in the logs.

Chris.


On 27/06/17 12:02, Freedman, Jon wrote:

No I think you are correct – it looks to me that quickfix is supressing the resends because it somehow thinks one has already been sent

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 10:17
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

fair enough. But I don't see that the counterparty really satisfies the ResendRequest, i.e. by resending the messages or sending a SequenceReset. Am I missing something?

Chris.

 


--
Christoph John
Development & Support
Direct: <a href="tel:+49%20241%2055708028" value="+4924155708028" target="_blank">+49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: <a href="tel:+49%20241%2055708010" value="+4924155708010" target="_blank">+49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi Guido,

the problem is that we want to read a value (outgoing seqnum) while it is changed. IMHO the performance impact is negligible here as it will only happen on Logon and only if tag 789 is used.

I've looked at your test and the QFJ code and I think you are right about the race condition. In the case where 789/NextExpectedMsgSeqNum is used on an incoming Logon message, QFJ checks if the next outgoing sequence number is the same as on the incoming Logon. But it could happen that the sequence number has not been incremented yet although the message has been sent out already.
So it might happen that we look at the value too early. But we want to look at the value directly after it has been incremented. So my idea was to wait for the lock to be released to get the sequence number right afterwards.

Or could this be better achieved with an AtomicInteger?

Thanks and cheers,
Chris.




On 04/07/17 13:46, Guido Medina wrote:
Hi all,

I wonder if this can simply be done with an AtomicInteger? for reset you can do something like compareAndSet, 1 or 2 threads accessing an AtomicInteger will cause a very tiny performance impact.
Though I'm not looking at the code so maybe what I'm proposing makes no sense.

Cheers,

Guido.

On Tue, Jul 4, 2017 at 12:32 PM, Christoph John via Quickfixj-users <[hidden email]> wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J
Support: http://www.quickfixj.org/support/



Hi Jon,

what about my suggestion from an earlier mail:
So one could simply wait for the lock to be released to read the correct value. Since this is only done rarely (only on Logon and only when tag 789 is used) it might be a feasible solution. But I'll think a while more about it.

Do you have a more elegant solution? ;) Until now I don't have a more sophisticated solution. And it probably is the only thing you can do.
But I'd rather not wait indefinitely for the lock to be released. Question is what happens when after a given time (1 second??) the lock cannot be obtained? Abort the Logon or simply obtain the sequence number without the lock with knowing that it might be wrong?

Chris.



On 04/07/17 12:53, Freedman, Jon wrote:

The counterparty have confirmed they should have sent a gap fill but did not.  They’re looking into correcting that.

 

In the meantime is it possible to work on fixing the race condition?  I am happy to help with that.

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 11:08
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

There was a resend request implicitly sent via the usage of tag 789 on the Logon message that QFJ sent (there is 789=1).
Now the counterparty should respond with either the resent messages or a SequenceReset message. I can see neither in the logs.

Chris.


On 27/06/17 12:02, Freedman, Jon wrote:

No I think you are correct – it looks to me that quickfix is supressing the resends because it somehow thinks one has already been sent

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 10:17
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

fair enough. But I don't see that the counterparty really satisfies the ResendRequest, i.e. by resending the messages or sending a SequenceReset. Am I missing something?

Chris.

 


--
Christoph John
Development & Support
Direct: <a href="tel:+49%20241%2055708028" value="+4924155708028" target="_blank" moz-do-not-send="true">+49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: <a href="tel:+49%20241%2055708010" value="+4924155708010" target="_blank" moz-do-not-send="true">+49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users



--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...



http://www.macd.com


MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald


take care of the environment - print only if necessary

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|

Re: Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

Freedman, Jon
In reply to this post by QuickFIX/J mailing list
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



I think the correct fix is to make sure that any change to SessionState is atomic, there appears to be an attempt to do this by surrounding Session#sendRaw with lockSenderMsgSeqNum

 

Are you suggesting that rather than change Session#nextLogon to:

 

            state.lockSenderMsgSeqNum();

            final int actualNextNum = state.getMessageStore().getNextSenderMsgSeqNum();

            state.unlockSenderMsgSeqNum();

 

Refactor SessionState to use Lock#tryLock instead of Lock#lock and add time/unit parameters to SessionState#lockSenderMsgSeqNum ?

 

See https://github.com/quickfix-j/quickfixj/compare/master...jonfreedman:master

 

From: Christoph John [mailto:[hidden email]]
Sent: 04 July 2017 12:32
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

what about my suggestion from an earlier mail:

So one could simply wait for the lock to be released to read the correct value. Since this is only done rarely (only on Logon and only when tag 789 is used) it might be a feasible solution. But I'll think a while more about it.


Do you have a more elegant solution? ;) Until now I don't have a more sophisticated solution. And it probably is the only thing you can do.
But I'd rather not wait indefinitely for the lock to be released. Question is what happens when after a given time (1 second??) the lock cannot be obtained? Abort the Logon or simply obtain the sequence number without the lock with knowing that it might be wrong?

Chris.


On 04/07/17 12:53, Freedman, Jon wrote:

The counterparty have confirmed they should have sent a gap fill but did not.  They’re looking into correcting that.

 

In the meantime is it possible to work on fixing the race condition?  I am happy to help with that.

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 11:08
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

There was a resend request implicitly sent via the usage of tag 789 on the Logon message that QFJ sent (there is 789=1).
Now the counterparty should respond with either the resent messages or a SequenceReset message. I can see neither in the logs.

Chris.



On 27/06/17 12:02, Freedman, Jon wrote:

No I think you are correct – it looks to me that quickfix is supressing the resends because it somehow thinks one has already been sent

 

From: Christoph John [[hidden email]]
Sent: 27 June 2017 10:17
To: Freedman, Jon; '[hidden email]'
Subject: Re: [Quickfixj-users] Issue with daily re-connect & NextExpectedMsgSeqNum off by 1

 

Hi Jon,

fair enough. But I don't see that the counterparty really satisfies the ResendRequest, i.e. by resending the messages or sending a SequenceReset. Am I missing something?

Chris.

 

 

--

Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:Christoph.John@...




http://www.macd.com



MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10

 Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald



take care of the environment - print only if necessary




This email, the information therein and any attached materials (collectively the "Email") are intended only for the addressee(s) and may contain confidential, proprietary, copyrighted and/or privileged material. If you have received this Email in error please delete it and notify the sender immediately. This Email remains the property of Brevan Howard, which reserves the right to require its return (together with any copies or extracts thereof) at any time upon request. Any unauthorised review, retransmission, dissemination, forwarding, printing, copying or other use of this Email is prohibited. Brevan Howard may be legally required to review and retain outgoing and incoming email and produce it to regulatory authorities and others with legal rights to the information. Internet communications cannot be guaranteed to be secure or error free as information could be intercepted, changed corrupted, lost, arrive late or contain viruses. Brevan Howard accepts no liability for any errors or omissions in this Email which arise as a result of internet transmission. This Email is not an official confirmation of any transaction. Any comments or statements made herein do not necessarily reflect the views of Brevan Howard. 
This Email is not an offer to sell or solicitation of an offer to buy any security or investment. It does not constitute or contain any investment advice and is being made without regard to the recipients investment objectives, financial situation or means. Past Performance is not an indicator of future results and Brevan Howard provides no assurance that future results will be consistent with any information provided herein or attached hereto. Brevan Howard and the sender make no warranties regarding the accuracy or completeness of the information in this Email and it should not be relied upon and is subject to change without notice. Brevan Howard and its representatives, officers and employees accept no responsibility for any losses suffered as a result of reliance on the information in this Email or the reliability, accuracy, or completeness thereof.
In this Email, "Brevan Howard" means Brevan Howard Asset Management LLP ("BHAM"), Brevan Howard Inc., Brevan Howard (Israel) Ltd and their respective affiliates. BHAM is a limited liability partnership authorised and regulated by the Financial Conduct Authority of the United Kingdom and registered in England & Wales (reg. no. OC302636).

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
12