Python Logging: RotatingFileHandler
Whenever you write a computer program that is going be a service is recommended that this program saves somewhere what is doing, most of the popular services do it (Apache, Cherokee, MySQL, Put your service here:___________________), or maybe you have a desktop application where you want to write what is doing maybe for debug purposes.Fortunately this is an easy task if you use Python's Logging module. This module have several handlers, some of them to write to the standardError, to a Socket, to the syslog and many others
One of my favorite handler is the RotatingFileHandler This module creates a log file, when this file is full (if you set the maximum bytes per file flag) it is renamed, appending the .1 and a new file is created with the same name as main log file. You can set up the max number of files to be created.
This module (logging) is good, easy to use, but you have to use the RotatingFileHandler with care, or you will have two, three or more files being filled and then, your logs will be in N files..
When you create your Logger Object you can set the log level, which can be INFO, WARNING, DEBUG, ERROR, CRITICAL and EXCEPTION, you already realized which level prints what. You can set the Handler, and this is where using the RotatingFileHandler becomes a bit trickier.
In your applications you may want to have several loggers, one for each module you have in your application. If you plan to use the RotatingFileHandler you have to use the same handler for every Logger you are creating if you plan this Logger to use the same file. If you don't do this then you will end with something saving logs in your main file, and every other data will be stored somewhere else.
This is because when you try to save a something in your log, the handler check the log size, and if the file size is >= maxbytes makes the rollover, if you are using several handlers related to the same file, then the handler that realize first that the size of the log file reach its max size will do the rollover but this will not take effect in the other handlers, so, one handler will be logging in the newly created file while the others will be logging to the old file and now you have two files growing.
To avoid this, just use the same fucking handler for every logger you are creating if you plan to use the same file.
Another issue I face with logger what the fact that if you call two time s to the same logger.. I mean:
a = logging.getLogger('chanchanchan')
b = logging.getLogger('chanchanchan')
b = logging.getLogger('chanchanchan')
When you make something like this:
a.info('This is a test')
You'll se this:
This is a test
This is a test
Yes, you are calling a to write "This is a test" but in your log file appears two times (or N times you have created a logger with the same name). So, its better for you to have something like a manager that gives you the logger already created with that name if exists or create it for you.
Let's do an example:
This is a small program that logs something in a endless cycle:
#!/usr/bin/env python
# -*- encoding: latin-1 -*-
import sys
import time
from logger1 import LoggerManager
a = LoggerManager().getLogger('a')
b = LoggerManager().getLogger('b')
c = LoggerManager().getLogger('c')
while 1:
t = time.time()
for index,i in enumerate((a,b,c)):
msg = str(index) + repr(t)
i.debug(msg)
time.sleep(0.005)
#!/usr/bin/env python
import logging
import logging.handlers
from Singleton import Singleton
import os
if os.name == 'nt':
LOGPATH = 'C:\\'
else:
LOGPATH = './'
class LoggerManager(Singleton):
def __init__(self):
self.loggers = {}
formatter = logging.Formatter('%(asctime)s:%(levelname)-8s:%(name)-10s:%(lineno)4s: %(message)-80s')
level = 'DEBUG'
nlevel = getattr(logging, level, None)
if nlevel != None:
self.LOGGING_MODE = nlevel
else:
self.LOGGING_MODE = logging.DEBUG
self.LOGGING_HANDLER = logging.handlers.RotatingFileHandler(
os.path.join(LOGPATH, 'log_event.log'),'a',524288, 10)
self.ERROR_HANDLER = logging.handlers.RotatingFileHandler(
os.path.join(LOGPATH,'log_error.log'),'a',524288, 10)
self.LOGGING_HANDLER.setFormatter(formatter)
self.LOGGING_HANDLER.setLevel(self.LOGGING_MODE)
def getLogger(self, loggername):
if not self.loggers.has_key(loggername):
logger = Logger(loggername,
logging_handler= self.LOGGING_HANDLER,
error_handler = self.ERROR_HANDLER,
logging_mode = self.LOGGING_MODE)
self.loggers[loggername] = logger
return self.loggers[loggername]
class Logger:
'''
Implements the christine logging facility.
'''
def __init__(self, loggername, type = 'event', logging_handler= '', error_handler = '', logging_mode = ''):
'''
Constructor, construye una clase de logger.
@param loggername: Nombre que el logger tendra.
@param type: Tipo de logger. Los valores disponibles son : event y error
por defecto apunta a event. En caso de utilizarse otro
que no sea event o error se apuntara a event.
'''
# Creating two logger, one for the info, debug and warnings and
#other for errors, criticals and exceptions
self.__Logger = logging.getLogger(loggername)
self.__ErrorLogger = logging.getLogger('Error'+ loggername)
# Setting Logger properties
self.__Logger.addHandler(logging_handler)
self.__Logger.setLevel(logging_mode)
self.__ErrorLogger.addHandler(error_handler)
self.__ErrorLogger.setLevel(logging_mode)
self.info = self.__Logger.info
self.debug = self.__Logger.debug
self.warning = self.__Logger.warning
self.critical = self.__ErrorLogger.critical
self.error = self.__ErrorLogger.error
self.exception = self.__ErrorLogger.exception
I hope this help you.




