Introduction to the logging module
Python's logging module provides a generalized logging system, which makes it easy for developers to develop third-party modules or their own Python applications. This module also provides different logging levels, and can be logged in different ways, such as file, HTTP, GET/POST, SMTP, Socket, etc., and even implement their own specific logging methods. In the following section I will focus on how to log using the file method.
The logging module includes four basic concepts: logger, handler, filter, and formatter.
logging module and log4j mechanism is the same , only the specific implementation details are different. Module provides logger, handler, filter, formatter.
logger：Provides a logging interface for application code. logger's longest running operations are of two types: configuring and sending log messages. You can get the logger object by (name), if you do not specify the name of the root object is returned , the same name multiple times using the same call to getLogger method returns the same logger object .
handler：Sends the log record to a suitable destination, such as a file, socket, etc. A logger object can add 0 to multiple handlers via the addHandler method. A logger object can be added through the addHandler method 0 to more than one handler, each handler can define a different level of logging, in order to realize the log hierarchical filtering display.
filter：Provides an elegant way to decide whether a log entry is sent to the handler.
formatter：Specifies the specific format of the logging output. formatter's constructor takes two arguments: the format string of the message and a date string, both of which are optional.
Similar to log4j, logger, handler and log message calls can have specific logging levels, only if the level of the log message is greater than the level of the logger and handler.
import logging
(level=,
format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s',
datefmt='%a, %d %b %Y %H:%M:%S',
filename='',
filemode='w')
#################################################################################################
# Define a StreamHandler that prints log messages at the INFO level or higher to the standard error and adds them to the current log handling object.
console = ()
()
formatter = ('%(name)-12s: %(levelname)-8s %(message)s')
(formatter)
('').addHandler(console)
#################################################################################################
('This is debug message')
('This is info message')
('This is warning message')
on-screen printing:
root : INFO This is info message
root : WARNING This is warning message
./The contents of the file are:
Sun, 24 May 2009 21:48:54 [line:11] DEBUG This is debug message
Sun, 24 May 2009 21:48:54 [line:12] INFO This is info message
Sun, 24 May 2009 21:48:54 [line:13] WARNING This is warning message
: Log output to a stream, which can be, or a file
: Log output to file
Log rollback methods, the actual use of RotatingFileHandler and TimedRotatingFileHandler
: Remote log output to TCP/IP sockets
: Remote log output to UDP sockets
: Remotely export logs to an e-mail address
: Log output to syslog
: Remotely export logs to Windows NT/2000/XP event logs
: Log output to a formulated buffer in memory
:: Remote output to HTTP servers via "GET" or "POST".
import logging
import sys
# Get the logger instance, or return the root logger if the parameter is empty.
logger = ("AppName")
# Specify the logger output format
formatter = ('%(asctime)s %(levelname)-8s: %(message)s')
# File logs
file_handler = ("")
file_handler.setFormatter(formatter) # You can specify the output format via setFormatter.
# Console log
console_handler = ()
console_handler.formatter = formatter # You can also assign a value to the formatter directly.
# Log processor added to the logger, you can customize the log processor to output to other places
(file_handler)
(console_handler)
# Specify the lowest output level for logs, default is WARN level
()
# Output different levels of logs
('this is debug info')
('this is information')
('this is warning message')
('this is error message')
('this is fatal message, it is same as ')
('this is critical message')
# 2016-10-08 21:59:19,493 INFO : this is information
# 2016-10-08 21:59:19,493 WARNING : this is warning message
# 2016-10-08 21:59:19,493 ERROR : this is error message
# 2016-10-08 21:59:19,493 CRITICAL: this is fatal message, it is same as
# 2016-10-08 21:59:19,493 CRITICAL: this is critical message
# Remove some log processors
(file_handler)
python: logging module
10 DECEMBER 2015
summarize
python's logging module (logging is thread-safe) provides applications with a standard interface for outputting log messages. logging not only supports outputting logs to files, but also to TCP/UDP servers, EMAIL servers, HTTP servers, UNIX syslog systems, and so on. There are four main concepts in logging : logger, handler, filter and formatter, will be introduced below.
logger
The Logger object plays a triple role:
It storms the application with several methods so that the application can log at runtime.
The Logger object determines which logs to log based on the level of the log or based on the Filter object.
The Logger object is responsible for passing log messages to the relevant handler.
The most commonly used methods in the Logger object fall into two categories: configuration, message sending. The configuration methods include:
setLevel(level)
The setLevel(level) method is used to set the logger's logging level. if the log's level is lower than the value set by the setLevel(level) method then the logger will not process it. the logging module's built-in logging levels are:
CRITICAL = 50 FATAL = CRITICAL ERROR = 40 WARNING = 30 WARN = WARNING INFO = 20 DEBUG = 10 NOTSET = 0 （The larger the value, the higher the level） addFilter(filter) removeFilter(filter) addHandler(handler) removeHandler(handler)
The message sending methods include:
debug(log_message, [*args[, **kwargs]])
Use the DEBUG level to log log_message % args.
In order to log exception information, the keyword parameter exc_info needs to be set to a true value.
For example:
("Houston, we have a %s", "thorny problem", exc_info=1)
info(log_message, [*args[, **kwargs]])
Use the INFO level to log log_message % args.
In order to log exception information, the keyword parameter exc_info needs to be set to a true value.
For example:
("Houston, we have a %s", "interesting problem", exc_info=1)
warning(log_message, [*args[, **kwargs]])
Use the WARNING level to log log_message % args.
In order to log exception information, the keyword parameter exc_info needs to be set to a true value.
For example:
("Houston, we have a %s", "bit of a problem", exc_info=1)
error(log_message, [*args[, **kwargs]])
Use Error level to log log_message % args.
In order to log exception information, the keyword parameter exc_info needs to be set to a true value.
For example:
("Houston, we have a %s", "major problem", exc_info=1)
critical(log_message, [*args[, **kwargs]])
Use CRITICAL level to log log_message % args.
In order to log exception information, the keyword parameter exc_info needs to be set to a true value.
For example:
("Houston, we have a %s", "major disaster", exc_info=1)
exception(message[, *args])
(*((msg,) + args), **{'exc_info': 1})
log(log_level, log_message, [*args[, **kwargs]])
Log log_message % args using the integer level level.
In order to log exception information, the keyword parameter exc_info needs to be set to a true value.
For example:
(level, "We have a %s", "mysterious problem", exc_info=1) ([name])
method returns a reference to a Logger instance, which is the name of the Logger instance if the name parameter is supplied, or root if the name parameter is not supplied.
The name attribute of the Logger instance can be used to see the name of the Logger instance.
Logger instances are named using a period (.) separated by a multilevel structure.
In this naming scheme, the later logger is the child of the earlier one (parent-child loggers are simply identified by naming), e.g., if there is a logger with the name foo, then loggers such as , and are all children of the logger foo.
The child logger automatically inherits the definition and configuration of the parent logger.
Calling the ([name]) method multiple times with the same name returns a reference to the same logger object.
This rule is valid not only for the same module, but also for multiple modules in the same Python interpreter process.
So an application can define a parent logger in one module and then inherit that logger in other modules without having to configure all the loggers all over again.
handler
The handler instance is responsible for distributing log events to specific destinations. logger objects can add zero or more handler objects to itself using the addHandler() method. A common scenario is that an application may want to log all logs to a single log file, all ERROR and higher level logs to stdout, and all CRITICAL level logs to an email address. This scenario requires three separate handlers, each of which is responsible for sending a specific level of logging to a specific place.
Here is the handler built into the logging module:
StreamHandler FileHandler RotatingFileHandler TimedRotatingFileHandler SocketHandler DatagramHandler SysLogHandler NTEventLogHandler SMTPHandler MemoryHandler HTTPHandler
The built-in handler provides the following configuration methods:
setLevel(level)
The setLevel() method of the handler object, like the setLevel() method of the logger object, is also used to set a log level if the level of the log is lower than the level of the
value set by the setLevel() method, then the handler will not process it.
setFormatter(formatter)
addFilter(filter)
removeFilter(filter)
Application code should not instantiate and use handlers directly. is a base class that defines the interfaces that all handlers should implement and establishes some default behaviors that subclasses can use (or override).
Custom Handler A custom handler must be inherited from and implement the following methods:
class Handler(Filterer):
def emit(self, record):
"""
Do whatever it takes to actually log the specified logging record.
This version is intended to be implemented by subclasses and so
raises a NotImplementedError.
"""
raise NotImplementedError, 'emit must be implemented '\
'by Handler subclasses'
def flush(self):
"""
Ensure all logging output has been flushed.
This version does nothing and is intended to be implemented by
subclasses.
"""
pass
def close(self):
"""
Tidy up any resources used by the handler.
This version does removes the handler from an internal list
of handlers which is closed when shutdown() is called. Subclasses
should ensure that this gets called from overridden close()
methods.
"""
#get the module data lock, as we're updating a shared structure.
_acquireLock()
try: #unlikely to raise an exception, but you never know...
if self in _handlers:
del _handlers[self]
if self in _handlerList:
_handlerList.remove(self)
finally:
_releaseLock()
The emit(record) method is responsible for performing all the things needed to actually record the log, and must be implemented in subclasses of . The close() method is responsible for cleaning up the resources used by the handler (when the Python interpreter exits, it calls the flush() and close() methods of all the handlers), and subclasses of . should make sure that when they override the close() method, they call that method of the parent class.
The source code analyzed below:
class StreamHandler(Handler):
def __init__(self, strm=None):
Handler.__init__(self)
if strm is None:
strm =
= strm
def flush(self):
if and hasattr(, "flush"):
()
def emit(self, record):
try:
msg = (record)
stream =
fs = "%s\n"
if not hasattr(types, "UnicodeType"): #if no unicode support...
(fs % msg)
else:
try:
if (isinstance(msg, unicode) and
getattr(stream, 'encoding', None)):
fs = ()
try:
(fs % msg)
except UnicodeEncodeError:
#Printing to terminals sometimes fails. For example,
#with an encoding of 'cp1251', the above write will
#work if written to a stream opened or wrapped by
#the codecs module, but fail when writing to a
#terminal even when the codepage is set to cp1251.
#An extra encoding step seems to be needed.
((fs % msg).encode())
else:
(fs % msg)
except UnicodeError:
(fs % ("UTF-8"))
()
except (KeyboardInterrupt, SystemExit):
raise
except:
(record)
In the constructor, if the strm parameter is supplied then it is the stream to output to, if it is not supplied then the log is output to the standard error output stream.
What the flush() method does: flushes the internal I/O buffer. This method is called after each emit log to sync the log from the I/O buffer.
The emit(record) method does the following: logs the LogRecord object (record) to. the emit(record) method first calls the format(record) method provided by the base class, which formats the record object according to the set Formatter object to get the string msg to be logged. Then it performs a series of encoding and decoding of fs (fs is actually adding a newline '\n' to the end of the msg) to write it to. Finally it is flushed. Exceptions that occur during the emit(record) call should be handled by calling the provided handleError(record) method.
filter
The Filter object is used to perform filtering on the LogRecord object. both logger and handler can use filter to filter the record. the following is a column to illustrate the role of the Filter base class:
If a filter is instantiated using, then it allows logs logged by loggers with names like,, to pass and does not allow logs logged by loggers with names like,, to pass.
If a filter is instantiated with an empty string, then it allows all events to pass.
The Filter base class has a method called filter(record), which is used to determine whether the specified record (LogRecord object) is logged. If the method returns 0, the record is not logged; a non-zero return logs the record.
Filterer (note: not Filter) is the base class for logger and handler. It provides methods to add and remove filters, and provides the filter(record) method for filtering records, which allows records to be logged by default, but any filter can override this default behavior, and the filter(record) method should return 0 if it wants to discard the record, otherwise it should return non-zero.
formatter
The Formatter object is used to convert a LogRecord object into text, which defines the format, structure of the log. Unlike classes, an application can instantiate the Formatter class directly, or subclass Formatter if needed to customize some of its behavior.
The constructor of Formatter accepts two arguments: the first argument is a formatted string for log messages; the second argument is a formatted string for dates. The second parameter is optional and defaults to %Y-%m-%d %H:%M:%S.
The formatting strings for log messages are replaced with %(<dictionary key>)s style strings.
Here are the replacement strings and what they represent:
%(name)s loggeryour name %(levelno)s Numerical presentation at the log level %(levelname)s Textual presentation at the log level %(pathname)s call (programming)loggingThe full pathname of the source file of the %(filename)s pathnamepart of the filename of the %(module)s module name（filenameyour name部分） %(lineno)d call (programming)loggingline number %(funcName)s function name %(created)f LogRecordCreation time of the（()return value of） %(asctime)s LogRecordCreation time of the的文本表现形式 %(msecs)d Millisecond portion of creation time %(relativeCreated)d LogRecordCreation time of the，In milliseconds。This time is relativelyloggingof the time the module was loaded（This is usually the time when the application is started）。 %(thread)d threadingID %(threadName)s threading名称 %(process)d stepID %(message)s ()The result of the return of the。
Configuring logging
Here is a simple example that will print the log to standard output:
import logging
import sys
logger = (__name__)
filter = (__name__)
formatter = ("%(asctime)s|%(name)-12s|%(message)s", "%F %T")
stream_handler = ()
stream_handler.addFilter(filter)
stream_handler.setLevel()
stream_handler.setFormatter(formatter)
()
(filter)
(stream_handler)
if __name__ == "__main__":
("info")
Running this script results in the following output.
2015-12-16 13:52:17|__main__ |info
Using the configuration file, configure logging
The following is an example of configuring logging, using a configuration file:
import logging
import
("")
if __name__ == "__main__":
logger = ("test_logging.sublogger")
("info")
Below:
[loggers] keys = root,logger [handlers] keys = stream_handler [formatters] keys = formatter [logger_root] handlers = stream_handler [logger_logger] handlers = stream_handler level = DEBUG propagate = 1 qualname = test_logging [handler_stream_handler] class = StreamHandler args = (,) formatter = formatter level = DEBUG [formatter_formatter] format = %(asctime)s|%(name)-12s|%(message)s datefmt = %F %T
There are two things that need to be explained: the first is the propagate option in the logger_xxxsection, which, when the logger object passes the record to all the relevant handlers, looks (cascading upwards) for all the handlers of this logger and all of its parent loggers. during the search, if the logger object's propagate property is set to 1, then the search continues upwards; if a logger's propagate property is set to 0, then the search stops.
The second is the qualname option in the logger_xxxsection, which is actually the name of the logger.
When using the configuration file, the root logger must be defined.
The coolest listen(port) function ever!
The (port) function allows an application to listen for new configuration information on a socket for the purpose of changing the configuration at runtime without restarting the application.
Listening program:
import
import logging
import time
("")
logger = ("test_logging.listen")
t = (9999)
(True)
()
try:
while True:
('running.')
(3)
except (KeyboardInterrupt, SystemExit, Exception):
()
Send a new configuration message program:
import socket
import struct
HOST = 'localhost'
PORT = 9999
s = (socket.AF_INET, socket.SOCK_STREAM)
((HOST, PORT))
print "connected..."
data_to_send = open("").read()
((">L", len(data_to_send)))
(data_to_send)
print "closing..."
()
The above this comprehensive explanation of the logger module in python is all that I have shared with you, I hope to give you a reference, and I hope you will support me more.