Quantcast

OutOfMemoryError in FIX engine without stacktrace

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

OutOfMemoryError in FIX engine without stacktrace

girish
Hello Team,

I am seeing many 'OutOfMemoryError' in console causing my FIX engine to go down. I dont see whole stack to identify the reason though.

Memory arguments for my engines are - -Xms1024m -Xmx1024m -XX:MaxPermSize=1024m

Any help regarding this will be really helpful. Thanks in advance.

Here are some of the logs -



Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "server-timer"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Old I/O client worker ([id: 0x70c52298, /10.128.225.128:23135 => /10.128.225.128:8448])"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-17 (HornetQ-client-global-threads-1463889752)"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-178"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Old I/O client worker ([id: 0xbdcc9df0, /10.128.225.128:23225 => /10.128.225.128:8448])"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "QFJ Message Processor"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-23 (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
00:21:39,210 ERROR [org.hornetq.core.client] (Thread-12 (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)) HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Java heap space


Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "PositionEngineThread1"
00:21:38,899 ERROR [org.hornetq.core.client] (Thread-22 (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)) HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Java heap space

00:26:44,388 ERROR [stderr] (Periodic Recovery) Exception in thread "QFJ Message Processor" Exception in thread "Thread-23 (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)" Exception in thread "Thread-17 (HornetQ-client-global-threads-1463889752)" D=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"                                                                                                                                   Exception in thread "Thread-136" java.lang.OutOfMemoryError: Java heap space
.
.
.
.
00:26:33,871 ERROR [org.xnio.listener] (Remoting "EngineServer" read-1) A channel event listener threw an exception: java.lang.OutOfMemoryError: Java heap space

00:26:44,401 ERROR [stderr] (Thread-136) ava.lang.OutOfMemoryError: Java heap spaceent-global-threads-1463889752)" D=46812c95-c88<20160206-16:26:44, FIX.4.4:TICFIe-11e5-bb3c-a5759788dc45-586613861)"                                                                                                                                   .
.
.
.
.
00:26:44,416 ERROR [stderr] (Old I/O client worker ([id: 0x2db2c279, /10.128.225.128:22765 => /10.128.225.128:8448])) Exception in thread "QFJ Message Processor" java.lang.OutOfMemoryError: Java heap space
[LOG|FATAL|07 Feb 2016 00:26:08,628|com.calypso.engine.TwoPhasedEngine$TaskProcessorThread|TaskProcessorThread-1|-]
Exception while handling error in TaskProcessorThread, must exit
[END]
java.lang.OutOfMemoryError: Java heap space



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OutOfMemoryError in FIX engine without stacktrace

Kimpton, C (Chris)
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/


Hi,

Has anything changed recently or is this a new app?

We give ours just under 3g of heap and 300m PermSize - but I guess it depends on the app (we use JRuby that seems to use a lot too).

Cheers,
Chris

-----Original Message-----
From: girish [mailto:[hidden email]]
Sent: 22 February 2016 07:59
To: [hidden email]
Subject: [Quickfixj-users] OutOfMemoryError in FIX engine without stacktrace

QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/


Hello Team,

I am seeing many 'OutOfMemoryError' in console causing my FIX engine to go down. I dont see whole stack to identify the reason though.

Memory arguments for my engines are - *-Xms1024m -Xmx1024m
-XX:MaxPermSize=1024m*

Any help regarding this will be really helpful. Thanks in advance.

Here are some of the logs -



Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "server-timer"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Old I/O client worker ([id: 0x70c52298,
/10.128.225.128:23135 => /10.128.225.128:8448])"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-17 (HornetQ-client-global-threads-1463889752)"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-178"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Old I/O client worker ([id: 0xbdcc9df0,
/10.128.225.128:23225 => /10.128.225.128:8448])"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "QFJ Message Processor"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-23 (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
00:21:39,210 ERROR [org.hornetq.core.client] (Thread-12
(HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861))
HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Java heap space


Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "PositionEngineThread1"
00:21:38,899 ERROR [org.hornetq.core.client] (Thread-22
(HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861))
HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Java heap space

00:26:44,388 ERROR [stderr] (Periodic Recovery) Exception in thread "QFJ Message Processor" Exception in thread "Thread-23 (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
Exception in thread "Thread-17 (HornetQ-client-global-threads-1463889752)"
D=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"                                                                                                                                  
Exception in thread "Thread-136" java.lang.OutOfMemoryError: Java heap space .
.
.
.
00:26:33,871 ERROR [org.xnio.listener] (Remoting "EngineServer" read-1) A channel event listener threw an exception: java.lang.OutOfMemoryError: Java heap space

00:26:44,401 ERROR [stderr] (Thread-136) ava.lang.OutOfMemoryError: Java heap spaceent-global-threads-1463889752)" D=46812c95-c88<20160206-16:26:44,
FIX.4.4:TICFIe-11e5-bb3c-a5759788dc45-586613861)"                                                                                                                                  
.
.
.
.
.
00:26:44,416 ERROR [stderr] (Old I/O client worker ([id: 0x2db2c279,
/10.128.225.128:22765 => /10.128.225.128:8448])) Exception in thread "QFJ Message Processor" java.lang.OutOfMemoryError: Java heap space
[LOG|FATAL|07 Feb 2016
00:26:08,628|com.calypso.engine.TwoPhasedEngine$TaskProcessorThread|TaskProcessorThread-1|-]
Exception while handling error in TaskProcessorThread, must exit [END]
java.lang.OutOfMemoryError: Java heap space







--
View this message in context: http://quickfix-j.364392.n2.nabble.com/OutOfMemoryError-in-FIX-engine-without-stacktrace-tp7579392.html
Sent from the QuickFIX/J mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
_________________________________________________________________________________

This email (including any attachments to it) is confidential, legally privileged, subject to copyright and is sent for the personal attention of the intended recipient only. If you have received this email in error, please advise us immediately and delete it. You are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Although we have taken reasonable precautions to ensure no viruses are present in this email, we cannot accept responsibility for any loss or damage arising from the viruses in this email or attachments. We exclude any liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided in this email or its attachments, unless that information is subsequently confirmed in writing.
_________________________________________________________________________________

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OutOfMemoryError in FIX engine without stacktrace

Dariusz Nawrocki-2
In reply to this post by girish
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi,
Keep looking for the first event of the OutOfMemory Exception in the logs. Java hotspot has bogus optimazation that causes often exception occurrences to lost stack trace.

Regards
Darek Nawrocki

22 lut 2016 9:01 AM "girish" <[hidden email]> napisał(a):
QuickFIX/J Documentation: <a href="http://www.quickfixj.org/documentation/ QuickFIX/J" rel="noreferrer" target="_blank">http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/


Hello Team,

I am seeing many 'OutOfMemoryError' in console causing my FIX engine to go
down. I dont see whole stack to identify the reason though.

Memory arguments for my engines are - *-Xms1024m -Xmx1024m
-XX:MaxPermSize=1024m*

Any help regarding this will be really helpful. Thanks in advance.

Here are some of the logs -



Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "server-timer"

Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "Old I/O client worker ([id: 0x70c52298,
/10.128.225.128:23135 => /10.128.225.128:8448])"

Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "Thread-17
(HornetQ-client-global-threads-1463889752)"

Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "Thread-178"

Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "Old I/O client worker ([id: 0xbdcc9df0,
/10.128.225.128:23225 => /10.128.225.128:8448])"

Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "QFJ Message Processor"

Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "Thread-23
(HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
00:21:39,210 ERROR [org.hornetq.core.client] (Thread-12
(HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861))
HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Java heap
space


Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "PositionEngineThread1"
00:21:38,899 ERROR [org.hornetq.core.client] (Thread-22
(HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861))
HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Java heap
space

00:26:44,388 ERROR [stderr] (Periodic Recovery) Exception in thread "QFJ
Message Processor" Exception in thread "Thread-23
(HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
Exception in thread "Thread-17 (HornetQ-client-global-threads-1463889752)"
D=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
Exception in thread "Thread-136" java.lang.OutOfMemoryError: Java heap space
.
.
.
.
00:26:33,871 ERROR [org.xnio.listener] (Remoting "EngineServer" read-1) A
channel event listener threw an exception: java.lang.OutOfMemoryError: Java
heap space

00:26:44,401 ERROR [stderr] (Thread-136) ava.lang.OutOfMemoryError: Java
heap spaceent-global-threads-1463889752)" D=46812c95-c88<20160206-16:26:44,
FIX.4.4:TICFIe-11e5-bb3c-a5759788dc45-586613861)"
.
.
.
.
.
00:26:44,416 ERROR [stderr] (Old I/O client worker ([id: 0x2db2c279,
/10.128.225.128:22765 => /10.128.225.128:8448])) Exception in thread "QFJ
Message Processor" java.lang.OutOfMemoryError: Java heap space
[LOG|FATAL|07 Feb 2016
00:26:08,628|com.calypso.engine.TwoPhasedEngine$TaskProcessorThread|TaskProcessorThread-1|-]
Exception while handling error in TaskProcessorThread, must exit
[END]
java.lang.OutOfMemoryError: Java heap space







--
View this message in context: http://quickfix-j.364392.n2.nabble.com/OutOfMemoryError-in-FIX-engine-without-stacktrace-tp7579392.html
Sent from the QuickFIX/J mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OutOfMemoryError in FIX engine without stacktrace

girish
In reply to this post by Kimpton, C (Chris)
Hi Chris,

Thanks for reply.

No -  nothing has been changed on this app recently. We encountered this error message 4 times till date. My app keeps on running for few days without any issue and suddenly it gives this OOM message and dies.

I want to understand the root cause. I can increase memory to 3 GB but I want to understand which part of processing is using that much memory.

My engine is light weighted and ideally it should not need memory more than 1 GB.

Is there any way to identify the cause for this OOM.


Cheers,
Girish
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OutOfMemoryError in FIX engine without stacktrace

Øyvind Matheson Wergeland
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/




The memory leak could quite possibly be in any thread, not necessarily the thread where the OOM occurs.

I suggest you perform a heap dump of your app after it has run for some time and analyze it with a tool, e.g. YourKit. You can also configure the JVM to perform heap dump on OOM.


Best regards

Øyvind Matheson Wergeland
CTO


Mobile: (+47) 95 16 16 88
E-mail: [hidden email]

Oslo Market Solutions
PO Box 4, 0051 Oslo, Norway
Telephone: (+47) 40 00 23 13
www.oslomarketsolutions.no

On 02/22/2016 09:31 AM, girish wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/


Hi Chris,

Thanks for reply.

No -  nothing has been changed on this app recently. We encountered this
error message 4 times till date. My app keeps on running for few days
without any issue and suddenly it gives this OOM message and dies.

I want to understand the root cause. I can increase memory to 3 GB but I
want to understand which part of processing is using that much memory.

My engine is light weighted and ideally it should not need memory more than
1 GB.

Is there any way to identify the cause for this OOM. 


Cheers,
Girish



--
View this message in context: http://quickfix-j.364392.n2.nabble.com/OutOfMemoryError-in-FIX-engine-without-stacktrace-tp7579392p7579395.html
Sent from the QuickFIX/J mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OutOfMemoryError in FIX engine without stacktrace

Christoph John
In reply to this post by girish
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/


Hi,

how many FIX sessions does your application have? If it is only one session then 1GB should be more
than enough. We are running single session gateways with much lower heap sizes.
The easiest way to find out is to either take heap dumps every once in a while and check which
objects grow in size or to use the JVM option "-XX:+HeapDumpOnOutOfMemoryError".
https://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/clopts.html

Whatkind of message store do you use? What kind of messages do you send? In which intervals do you
reset your sequence numbers? E.g. if you only reset sequence numbers weekly and send out many
messages I could imagine that the message store grows quite large in size.

Christoph.


On 22/02/16 08:58, girish wrote:

> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> Hello Team,
>
> I am seeing many 'OutOfMemoryError' in console causing my FIX engine to go
> down. I dont see whole stack to identify the reason though.
>
> Memory arguments for my engines are - *-Xms1024m -Xmx1024m
> -XX:MaxPermSize=1024m*
>
> Any help regarding this will be really helpful. Thanks in advance.
>
> Here are some of the logs -
>
>
>
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "server-timer"
>
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Old I/O client worker ([id: 0x70c52298,
> /10.128.225.128:23135 => /10.128.225.128:8448])"
>
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Thread-17
> (HornetQ-client-global-threads-1463889752)"
>
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Thread-178"
>
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Old I/O client worker ([id: 0xbdcc9df0,
> /10.128.225.128:23225 => /10.128.225.128:8448])"
>
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "QFJ Message Processor"
>
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Thread-23
> (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
> 00:21:39,210 ERROR [org.hornetq.core.client] (Thread-12
> (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861))
> HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Java heap
> space
>
>
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "PositionEngineThread1"
> 00:21:38,899 ERROR [org.hornetq.core.client] (Thread-22
> (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861))
> HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Java heap
> space
>
> 00:26:44,388 ERROR [stderr] (Periodic Recovery) Exception in thread "QFJ
> Message Processor" Exception in thread "Thread-23
> (HornetQ-server-HornetQServerImpl::serverUUID=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
> Exception in thread "Thread-17 (HornetQ-client-global-threads-1463889752)"
> D=46812c95-c88e-11e5-bb3c-a5759788dc45-586613861)"
> Exception in thread "Thread-136" java.lang.OutOfMemoryError: Java heap space
> .
> .
> .
> .
> 00:26:33,871 ERROR [org.xnio.listener] (Remoting "EngineServer" read-1) A
> channel event listener threw an exception: java.lang.OutOfMemoryError: Java
> heap space
>
> 00:26:44,401 ERROR [stderr] (Thread-136) ava.lang.OutOfMemoryError: Java
> heap spaceent-global-threads-1463889752)" D=46812c95-c88<20160206-16:26:44,
> FIX.4.4:TICFIe-11e5-bb3c-a5759788dc45-586613861)"
> .
> .
> .
> .
> .
> 00:26:44,416 ERROR [stderr] (Old I/O client worker ([id: 0x2db2c279,
> /10.128.225.128:22765 => /10.128.225.128:8448])) Exception in thread "QFJ
> Message Processor" java.lang.OutOfMemoryError: Java heap space
> [LOG|FATAL|07 Feb 2016
> 00:26:08,628|com.calypso.engine.TwoPhasedEngine$TaskProcessorThread|TaskProcessorThread-1|-]
> Exception while handling error in TaskProcessorThread, must exit
> [END]
> java.lang.OutOfMemoryError: Java heap space
>
>
>
>
>
>
>
> --
> View this message in context: http://quickfix-j.364392.n2.nabble.com/OutOfMemoryError-in-FIX-engine-without-stacktrace-tp7579392.html
> Sent from the QuickFIX/J mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> Quickfixj-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:[hidden email]
       


http://www.macd.com <http://www.macd.com/>
----------------------------------------------------------------------------------------------------
       
----------------------------------------------------------------------------------------------------
MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
         Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald
----------------------------------------------------------------------------------------------------
       
----------------------------------------------------------------------------------------------------

take care of the environment - print only if necessary

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

OutOfMemoryError in FIX engine without stacktrace

girish
Hi Christoph,

I have only one FIX session. I have already added JVM option -XX:+HeapDumpOnOutOfMemoryError today.

I am using filestore. My messages are generally related to Bond trades. Intervals are reset on daily basis.

Thanks,
Girish
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OutOfMemoryError in FIX engine without stacktrace

Christoph John
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/


Hi Girish,

OK, so what is the outcome of the heap dump analysis? Or do you mean you have already added the
parameter but there was no OutOfMemoryError since?

Which version of QF/J are you using? IIRCthere were some memory leaks regarding resend processing in
versions earlier than 1.6.0.

Cheers

On 22/02/16 09:52, girish wrote:

> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> Hi Christoph,
>
> I have only one FIX session. I have already added JVM option
> -XX:+HeapDumpOnOutOfMemoryError today.
>
> I am using filestore. My messages are generally related to Bond trades.
> Intervals are reset on daily basis.
>
> Thanks,
> Girish
>
>
>
> --
> View this message in context: http://quickfix-j.364392.n2.nabble.com/OutOfMemoryError-in-FIX-engine-without-stacktrace-tp7579392p7579397.html
> Sent from the QuickFIX/J mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> Quickfixj-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:[hidden email]
       


http://www.macd.com <http://www.macd.com/>
----------------------------------------------------------------------------------------------------
       
----------------------------------------------------------------------------------------------------
MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
         Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663

Geschäftsführer: George Macdonald
----------------------------------------------------------------------------------------------------
       
----------------------------------------------------------------------------------------------------

take care of the environment - print only if necessary

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OutOfMemoryError in FIX engine without stacktrace

girish
Hi Chris,

yes, there were no OOM error since I added these attributes.

I am using QF/J - 1.5.3.

Thanks,
Girish
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OutOfMemoryError in FIX engine without stacktrace

ChristopherH
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi,

  I would also in general enable GC logging and plot the resulting file with GC viewer (free), you should be able  to see the issue happening i.e. growing to destruction or not.  You should be able to signal your app (OS dependent) to heap dump at any point if you can't get it to OOM for analysis.

When it does OOM or you get it to heap dump probably the easiest thing is to use EciplseMAT (fee) and run its "leak suspects report" before looking at Visual VM (free with the JDK) or any of the other (excellent) tools. I usually set up a script to automatically heap dump and analyse all our other application runs to catch stuff early though I only use QFJ as a test client.

Chris  

> Date: Mon, 22 Feb 2016 02:44:18 -0700

> From: [hidden email]
> To: [hidden email]
> Subject: Re: [Quickfixj-users] OutOfMemoryError in FIX engine without stacktrace
>
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> Hi Chris,
>
> yes, there were no OOM error since I added these attributes.
>
> I am using QF/J - 1.5.3.
>
> Thanks,
> Girish
>
>
>
> --
> View this message in context: http://quickfix-j.364392.n2.nabble.com/OutOfMemoryError-in-FIX-engine-without-stacktrace-tp7579392p7579400.html
> Sent from the QuickFIX/J mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> Quickfixj-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Loading...