In addition to the parameters passed to the logging function (e.g. msg), sometimes we want to include some additional contextual information in the log output. For example, in a web application, it may be desirable to record client-specific information in the log, such as: the IP address and username of the remote client. Here we describe the following implementations:
- Introduce contextual information by passing an extra parameter to the logging function
- Using LoggerAdapters to introduce contextual information
- Using Filters to Introduce Contextual Information
I. Introduce contextual information by passing an extra parameter to the logging function
As we mentioned earlier, adding additional contextual information to the logging output can be accomplished by passing an extra parameter to the logging function, such as:
import logging
import sys
fmt = ("%(asctime)s - %(name)s - %(ip)s - %(username)s - %(message)s")
h_console = ()
h_console.setFormatter(fmt)
logger = ("myPro")
()
(h_console)
extra_dict = {"ip": "113.208.78.29", "username": "Petter"}
("User Login!", extra=extra_dict)
extra_dict = {"ip": "223.190.65.139", "username": "Jerry"}
("User Access!", extra=extra_dict)
Output:
2017-05-14 15:47:25,562 - myPro - 113.208.78.29 - Petter - User Login!
2017-05-14 15:47:25,562 - myPro - 223.190.65.139 - Jerry - User Access!
But passing information in this way is not so convenient, because each call to the logger method has to pass an extra keyword parameter. This is true even if there is no contextual information that needs to be inserted, because the fields specified in the formatter format set by that logger have to be present. So, we recommend using the following two ways to implement the introduction of context information.
It might be possible to try to solve this problem by creating a Logger instance every time a connection is created, but this is obviously not a good solution because these Logger instances are not garbage collected. While this is not a problem in practice, it is very difficult to manage when the number of Loggers becomes uncontrollable.
II. Using LoggerAdapters to introduce contextual information
Using the LoggerAdapter class to pass contextual information to the messages of a logging event is a very simple way to think of it as an optimized version of the first implementation - as it provides a default value for extra. This class is designed to be similar to Logger, so we can call the debug(), info(), warning(), error(), exception(), critical(), and log() methods as if we were using an instance of the Logger class. When creating an instance of LoggerAdapter, we need to pass a Logger instance and a class dictionary object containing contextual information to the class's instance construction method. When a logging method of a LoggerAdapter instance is called, the method will, after processing the logging log messages and the dictionary object, call the logging method with the same name of the logger object passed to the instance when it was built. The following are the definitions of several methods in the LoggerAdapter class:
class LoggerAdapter(object): """ An adapter for loggers which makes it easier to specify contextual information in logging output. """ def __init__(self, logger, extra): """ Initialize the adapter with a logger and a dict-like object which provides contextual information. This constructor signature allows easy stacking of LoggerAdapters, if so desired. You can effectively pass keyword arguments as shown in the following example: adapter = LoggerAdapter(someLogger, dict(p1=v1, p2="v2")) """ = logger = extra def process(self, msg, kwargs): """ Process the logging message and keyword arguments passed in to a logging call to insert contextual information. You can either manipulate the message itself, the keyword args or both. Return the message and kwargs modified (or not) to suit your needs. Normally, you'll only need to override this one method in a LoggerAdapter subclass for your specific needs. """ kwargs["extra"] = return msg, kwargs def debug(self, msg, *args, **kwargs): """ Delegate a debug call to the underlying logger, after adding contextual information from this adapter instance. """ msg, kwargs = (msg, kwargs) (msg, *args, **kwargs)
The following conclusions can be drawn by analyzing the above code:
- Context information is in the LoggerAdapter class process () method is added to the logging output message, if you want to achieve custom requirements only need to implement the LoggerAdapter subclass and override the process () method can be;
- The default implementation of the process() method does not modify the value of msg, but simply inserts a key named extra for the keyword argument, which is the value of the parameter passed to the LoggerAdapter class constructor;
- The extra parameter received by the LoggerAdapter class build method is actually the default context information provided to satisfy the logger's formatter format requirements.
Regarding the 3rd conclusion mentioned above, let's look at an example:
import logging
import sys
# Initialize a logger instance to be passed to the LoggerAdapter constructor method
fmt = ("%(asctime)s - %(name)s - %(ip)s - %(username)s - %(message)s")
h_console = ()
h_console.setFormatter(fmt)
init_logger = ("myPro")
init_logger.setLevel()
init_logger.addHandler(h_console)
# Initialize a context dictionary object to be passed to the LoggerAdapter constructor method
extra_dict = {"ip": "IP", "username": "USERNAME"}
# Get an instance of the LoggerAdapter class
logger = (init_logger, extra_dict)
# Logging method calls in the application
("User Login!")
("User Login!", extra={"ip": "113.208.78.29", "username": "Petter"})
= {"ip": "113.208.78.29", "username": "Petter"}
("User Login!")
("User Login!")
Output results:
# Use extra defaults: {"ip": "IP", "username": "USERNAME"}
2017-05-14 17:23:15,148 - myPro - IP - USERNAME - User Login!
# The extra method passed in the info(msg, extra) method does not override the default value
2017-05-14 17:23:15,148 - myPro - IP - USERNAME - User Login!
# extra defaults have been changed
2017-05-14 17:23:15,148 - myPro - 113.208.78.29 - Petter - User Login!
2017-05-14 17:23:15,148 - myPro - 113.208.78.29 - Petter - User Login!
According to the output of the above program, we will find a problem: the value of the extra parameter passed to the constructor method of the LoggerAdapter class cannot be overridden by the extra parameter in the logging function of the LoggerAdapter instance (e.g., the info() method called above), and the default value can only be modified by modifying the LoggerAdapter instance's The default value can only be modified by modifying the extra attribute of the LoggerAdapter instance (e.g., the =xxx used above), but this also means that the default value has been modified.
The idea to solve this problem should be: implement a subclass of LoggerAdapter, rewrite the process() method. Which for the operation of the kwargs parameter should be the first to determine whether it contains the extra keyword, if it contains the default value is not used for replacement; if the kwargs parameter does not contain the extra keyword then take the default value. Look at the specific implementation:
import logging
import sys
class MyLoggerAdapter():
def process(self, msg, kwargs):
if 'extra' not in kwargs:
kwargs["extra"] =
return msg, kwargs
if __name__ == '__main__':
# Initialize a logger instance to be passed to the LoggerAdapter constructor method
fmt = ("%(asctime)s - %(name)s - %(ip)s - %(username)s - %(message)s")
h_console = ()
h_console.setFormatter(fmt)
init_logger = ("myPro")
init_logger.setLevel()
init_logger.addHandler(h_console)
# Initialize a context dictionary object to be passed to the LoggerAdapter constructor method
extra_dict = {"ip": "IP", "username": "USERNAME"}
# Get an instance of a custom LoggerAdapter class
logger = MyLoggerAdapter(init_logger, extra_dict)
# Logging method calls in the application
("User Login!")
("User Login!", extra={"ip": "113.208.78.29", "username": "Petter"})
("User Login!")
("User Login!")
Output results:
# Use extra defaults: {"ip": "IP", "username": "USERNAME"}
2017-05-22 17:35:38,499 - myPro - IP - USERNAME - User Login!
# info(msg, extra) method passed in info(msg, extra) method has overridden defaults
2017-05-22 17:35:38,499 - myPro - 113.208.78.29 - Petter - User Login!
# extra defaults remain unchanged
2017-05-22 17:35:38,499 - myPro - IP - USERNAME - User Login!
2017-05-22 17:35:38,499 - myPro - IP - USERNAME - User Login!
Problem solved.
In fact, if we want to implement free field insertion in the log output without the restriction of the formatter, we can do so by splicing the keyword information from the dictionary parameter into the message of the log event in the process() method of the subclass of the custom LoggerAdapter. Obviously, there is a limit to where the field information in these contexts can be placed in the log output. The advantage of using 'extra' is that the values of this class dictionary object will be merged into the __dict__ of this LogRecord instance, which allows us to customize the formatting strings of the log output through the Formatter instance. This makes the placement of the field information in the log output as flexible as the built-in fields in the context information, but only if the key in the dictionary-like object passed to the constructor method is defined and explicit.
III. Using Filters to Introduce Contextual Information
Alternatively, we can use a customized instance approach by modifying the passed LogRecord instance in the filter(record) method, assigning the context information to be added as a new attribute to the instance, so that we can output this context information by specifying the string format of the formatter.
We mimic the above implementation by adding two pieces of information related to the current web request to the LogRecord instance passing the filter(record) method: ip and username.
import logging
from random import choice
class ContextFilter():
ip = 'IP'
username = 'USER'
def filter(self, record):
=
=
return True
if __name__ == '__main__':
levels = (, , , , )
users = ['Tom', 'Jerry', 'Peter']
ips = ['113.108.98.34', '219.238.78.91', '43.123.99.68']
(level=,
format='%(asctime)-15s %(name)-5s %(levelname)-8s %(ip)-15s %(username)-8s %(message)s')
logger = ('myLogger')
f = ContextFilter()
(f)
('A debug message')
('An info message with %s', 'some parameters')
for x in range(5):
lvl = choice(levels)
lvlname = (lvl)
= choice(ips)
= choice(users)
(lvl, 'A message at %s level with %d %s', lvlname, 2, 'parameters')
Output results:
2017-05-15 10:21:49,401 myLogger DEBUG IP USER A debug message
2017-05-15 10:21:49,401 myLogger INFO IP USER An info message with some parameters
2017-05-15 10:21:49,401 myLogger INFO 219.238.78.91 Tom A message at INFO level with 2 parameters
2017-05-15 10:21:49,401 myLogger INFO 219.238.78.91 Peter A message at INFO level with 2 parameters
2017-05-15 10:21:49,401 myLogger DEBUG 113.108.98.34 Jerry A message at DEBUG level with 2 parameters
2017-05-15 10:21:49,401 myLogger CRITICAL 43.123.99.68 Tom A message at CRITICAL level with 2 parameters
2017-05-15 10:21:49,401 myLogger INFO 43.123.99.68 Jerry A message at INFO level with 2 parameters
It should be noted that: the actual network application, you may also want to consider multi-thread concurrency thread safety issues, this time you can connect to the information or custom filter instance by saving to a threadlocal.
The above is a small introduction to Python to add contextual information to the log output, I hope to help you, if you have any questions please leave me a message, I will reply to you in a timely manner. I would also like to thank you very much for your support of my website!