rolling log files

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

rolling log files

Oded Kramer
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Hi, 

I have been using QFJ for quite a while now very successfully. I am looking into using rolling files for all Fix Messages log. 
Any suggestions how should i do this?


thanks,

Oded Kramer

------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rolling log files

Colin DuPlantis
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



We use SLF4J as our logging interface with QFJ combined with Log4J2. These two things combined provide log rolling out-of-the-box.

Create an instance of SLF4JLogFactory and pass it to your Initiator/Acceptor. At runtime, provide Log4j2 jars as the logging provider.

Here's a snippet from a log4j2.xml file that provides log rolling:

    <RollingRandomAccessFile name="FILE" fileName="target/logs/application${instanceName}.log" filePattern="target/logs/application${instanceName}-%i.log.gz" append="true">
      <PatternLayout pattern="%d{DATE} %5p [%t] %c{3} (%F:%L) - %m%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>

This appender will roll a log segment when it gets to be 100 MB, compress it with gzip, and rename it with an ordinal appended to the file name up 1000 segments.

On 11/12/15 5:20 AM, Oded Kramer wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/




Hi, 

I have been using QFJ for quite a while now very successfully. I am looking into using rolling files for all Fix Messages log. 
Any suggestions how should i do this?


thanks,

Oded Kramer


------------------------------------------------------------------------------


_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Colin DuPlantis
Chief Architect, Marketcetera
Download, Run, Trade
800.819.2928 x101 +1.541.306.6556
http://www.marketcetera.org

------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rolling log files

Oded Kramer
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



i use this line to create the initiator this.sessionConnector = new SocketInitiator(this, this.storeFactory, this.settings, this.logFactory, messageFactory); you can see this logFactory is passed to the constructor. 

the sesstionSettings are configurable using the .cfg file. The requirement is to have a rolling log file for each fix session (there are dozens of sessions). can i achieve this in the log4j2 file?

thanks,
Oded


On Thu, Nov 12, 2015 at 3:30 PM, Colin DuPlantis <[hidden email]> wrote:
QuickFIX/J Documentation: <a href="http://www.quickfixj.org/documentation/ QuickFIX/J" rel="noreferrer" target="_blank">http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



We use SLF4J as our logging interface with QFJ combined with Log4J2. These two things combined provide log rolling out-of-the-box.

Create an instance of SLF4JLogFactory and pass it to your Initiator/Acceptor. At runtime, provide Log4j2 jars as the logging provider.

Here's a snippet from a log4j2.xml file that provides log rolling:

    <RollingRandomAccessFile name="FILE" fileName="target/logs/application${instanceName}.log" filePattern="target/logs/application${instanceName}-%i.log.gz" append="true">
      <PatternLayout pattern="%d{DATE} %5p [%t] %c{3} (%F:%L) - %m%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>

This appender will roll a log segment when it gets to be 100 MB, compress it with gzip, and rename it with an ordinal appended to the file name up 1000 segments.

On 11/12/15 5:20 AM, Oded Kramer wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/




Hi, 

I have been using QFJ for quite a while now very successfully. I am looking into using rolling files for all Fix Messages log. 
Any suggestions how should i do this?


thanks,

Oded Kramer


------------------------------------------------------------------------------


_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Colin DuPlantis
Chief Architect, Marketcetera
Download, Run, Trade
<a href="tel:800.819.2928%20x101" value="+18008192928" target="_blank">800.819.2928 x101 <a href="tel:%2B1.541.306.6556" value="+15413066556" target="_blank">+1.541.306.6556
http://www.marketcetera.org

------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users



------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rolling log files

Colin DuPlantis
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Yes, you are passing the logFactory to the initiator. In order to use SLF4J, it has to be the correct log factory (SLF4JLogFactory). We can't tell that for sure from the code you've provided, but let's assume it is the correct one.

SLF4J is a log front end that allows you to use a number of different logger backends. Log4j2 is one of them, but not the only one you can use. Let's assume that you're using the correct log4j2 runtime jars to use log4j2.

If both those are true, then, yes, you can have a different log file for each fix session. In your configuration, set the SLF4JLogEventCategory option (see the QFJ doc) to use the variables of the session as part of the category. Set up different loggers for each session and different appenders.

Suppose I have two sessions: FIX.4.2:ME->SESSION1 and FIX.4.2:ME->SESSION2.

I can set up two different appenders in log4j2.xml:

    <RollingRandomAccessFile name="ME-SESSION1" fileName="logs/me-session1.log" filePattern="logs/me-session1-%i.log.gz" append="true">
      <PatternLayout pattern="%d{DATE} %m%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>


    <RollingRandomAccessFile name="ME-SESSION2" fileName="logs/me-session2.log" filePattern="logs/me-session2-%i.log.gz" append="true">
      <PatternLayout pattern="%d{DATE} %m%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>

then, loggers for each:

    <logger name="ME.SESSION1" level="info" additivity="false">
      <AppenderRef ref="ME-SESSION1"/>
    </logger>
    <logger name="ME.SESSION2" level="info" additivity="false">
      <AppenderRef ref="ME-SESSION2"/>
    </logger>

your QFJ config would want:

SLF4JLogEventCategory=${senderCompID}.${targetCompID}

You might have to play with this a little, but that's the generate idea.

On 11/12/15 6:23 AM, Oded Kramer wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/




i use this line to create the initiator this.sessionConnector = new SocketInitiator(this, this.storeFactory, this.settings, this.logFactory, messageFactory); you can see this logFactory is passed to the constructor. 

the sesstionSettings are configurable using the .cfg file. The requirement is to have a rolling log file for each fix session (there are dozens of sessions). can i achieve this in the log4j2 file?

thanks,
Oded


On Thu, Nov 12, 2015 at 3:30 PM, Colin DuPlantis <[hidden email]> wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J
Support: http://www.quickfixj.org/support/



We use SLF4J as our logging interface with QFJ combined with Log4J2. These two things combined provide log rolling out-of-the-box.

Create an instance of SLF4JLogFactory and pass it to your Initiator/Acceptor. At runtime, provide Log4j2 jars as the logging provider.

Here's a snippet from a log4j2.xml file that provides log rolling:

    <RollingRandomAccessFile name="FILE" fileName="target/logs/application${instanceName}.log" filePattern="target/logs/application${instanceName}-%i.log.gz" append="true">
      <PatternLayout pattern="%d{DATE} %5p [%t] %c{3} (%F:%L) - %m%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>

This appender will roll a log segment when it gets to be 100 MB, compress it with gzip, and rename it with an ordinal appended to the file name up 1000 segments.

On 11/12/15 5:20 AM, Oded Kramer wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/




Hi, 

I have been using QFJ for quite a while now very successfully. I am looking into using rolling files for all Fix Messages log. 
Any suggestions how should i do this?


thanks,

Oded Kramer


------------------------------------------------------------------------------


_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Colin DuPlantis
Chief Architect, Marketcetera
Download, Run, Trade
<a moz-do-not-send="true" href="tel:800.819.2928%20x101" value="+18008192928" target="_blank">800.819.2928 x101 <a moz-do-not-send="true" href="tel:%2B1.541.306.6556" value="+15413066556" target="_blank">+1.541.306.6556
http://www.marketcetera.org

------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users




------------------------------------------------------------------------------


_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Colin DuPlantis
Chief Architect, Marketcetera
Download, Run, Trade
800.819.2928 x101 +1.541.306.6556
http://www.marketcetera.org

------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rolling log files

Oded Kramer
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



I thought that for some reason i can have 1 logger and 1 appender for all files. thanks for clearing that up!
working fine now


Oded


On Thu, Nov 12, 2015 at 4:35 PM, Colin DuPlantis <[hidden email]> wrote:
QuickFIX/J Documentation: <a href="http://www.quickfixj.org/documentation/ QuickFIX/J" rel="noreferrer" target="_blank">http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/



Yes, you are passing the logFactory to the initiator. In order to use SLF4J, it has to be the correct log factory (SLF4JLogFactory). We can't tell that for sure from the code you've provided, but let's assume it is the correct one.

SLF4J is a log front end that allows you to use a number of different logger backends. Log4j2 is one of them, but not the only one you can use. Let's assume that you're using the correct log4j2 runtime jars to use log4j2.

If both those are true, then, yes, you can have a different log file for each fix session. In your configuration, set the SLF4JLogEventCategory option (see the QFJ doc) to use the variables of the session as part of the category. Set up different loggers for each session and different appenders.

Suppose I have two sessions: FIX.4.2:ME->SESSION1 and FIX.4.2:ME->SESSION2.

I can set up two different appenders in log4j2.xml:

    <RollingRandomAccessFile name="ME-SESSION1" fileName="logs/me-session1.log" filePattern="logs/me-session1-%i.log.gz" append="true">
      <PatternLayout pattern="%d{DATE} %m%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>


    <RollingRandomAccessFile name="ME-SESSION2" fileName="logs/me-session2.log" filePattern="logs/me-session2-%i.log.gz" append="true">
      <PatternLayout pattern="%d{DATE} %m%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>

then, loggers for each:

    <logger name="ME.SESSION1" level="info" additivity="false">
      <AppenderRef ref="ME-SESSION1"/>
    </logger>
    <logger name="ME.SESSION2" level="info" additivity="false">
      <AppenderRef ref="ME-SESSION2"/>
    </logger>

your QFJ config would want:

SLF4JLogEventCategory=${senderCompID}.${targetCompID}

You might have to play with this a little, but that's the generate idea.

On 11/12/15 6:23 AM, Oded Kramer wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/




i use this line to create the initiator this.sessionConnector = new SocketInitiator(this, this.storeFactory, this.settings, this.logFactory, messageFactory); you can see this logFactory is passed to the constructor. 

the sesstionSettings are configurable using the .cfg file. The requirement is to have a rolling log file for each fix session (there are dozens of sessions). can i achieve this in the log4j2 file?

thanks,
Oded


On Thu, Nov 12, 2015 at 3:30 PM, Colin DuPlantis <[hidden email]> wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J
Support: http://www.quickfixj.org/support/



We use SLF4J as our logging interface with QFJ combined with Log4J2. These two things combined provide log rolling out-of-the-box.

Create an instance of SLF4JLogFactory and pass it to your Initiator/Acceptor. At runtime, provide Log4j2 jars as the logging provider.

Here's a snippet from a log4j2.xml file that provides log rolling:

    <RollingRandomAccessFile name="FILE" fileName="target/logs/application${instanceName}.log" filePattern="target/logs/application${instanceName}-%i.log.gz" append="true">
      <PatternLayout pattern="%d{DATE} %5p [%t] %c{3} (%F:%L) - %m%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="1000"/>
    </RollingRandomAccessFile>

This appender will roll a log segment when it gets to be 100 MB, compress it with gzip, and rename it with an ordinal appended to the file name up 1000 segments.

On 11/12/15 5:20 AM, Oded Kramer wrote:
QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
QuickFIX/J Support: http://www.quickfixj.org/support/




Hi, 

I have been using QFJ for quite a while now very successfully. I am looking into using rolling files for all Fix Messages log. 
Any suggestions how should i do this?


thanks,

Oded Kramer


------------------------------------------------------------------------------


_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Colin DuPlantis
Chief Architect, Marketcetera
Download, Run, Trade
<a href="tel:800.819.2928%20x101" value="+18008192928" target="_blank">800.819.2928 x101 <a href="tel:%2B1.541.306.6556" value="+15413066556" target="_blank">+1.541.306.6556
http://www.marketcetera.org

------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users




------------------------------------------------------------------------------


_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Colin DuPlantis
Chief Architect, Marketcetera
Download, Run, Trade
<a href="tel:800.819.2928%20x101" value="+18008192928" target="_blank">800.819.2928 x101 <a href="tel:%2B1.541.306.6556" value="+15413066556" target="_blank">+1.541.306.6556
http://www.marketcetera.org

------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users



------------------------------------------------------------------------------

_______________________________________________
Quickfixj-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Loading...