SaltyCrane Blog — Notes on JavaScript and web development

Python logging filters do not propagate like handlers and levels do

Loggers are organized in a hierarchical fashion. A logger named 'foo.bar' is a child of a logger named 'foo'.

getLogger() returns a reference to a logger instance with the specified name if it is provided, or root if not. The names are period-separated hierarchical structures. Multiple calls to getLogger() with the same name will return a reference to the same logger object. Loggers that are further down in the hierarchical list are children of loggers higher up in the list. For example, given a logger with a name of foo, loggers with names of foo.bar, foo.bar.baz, and foo.bam are all descendants of foo. - Loggers documentation

If the level is not set on a logger, the level of the parent is used.

Loggers have a concept of effective level. If a level is not explicitly set on a logger, the level of its parent is used instead as its effective level. If the parent has no explicit level set, its parent is examined, and so on - all ancestors are searched until an explicitly set level is found. The root logger always has an explicit level set (WARNING by default). When deciding whether to process an event, the effective level of the logger is used to determine whether the event is passed to the logger’s handlers. - Loggers documentation

import logging

foo_logger = logging.getLogger('foo')
foo_logger.setLevel(20)

foo_bar_logger = logging.getLogger('foo.bar')

print foo_logger.getEffectiveLevel()
print foo_bar_logger.getEffectiveLevel()
20
20

Similarly, if a handler is not defined for a logger, the handler of the parent is used.

Child loggers propagate messages up to the handlers associated with their ancestor loggers. Because of this, it is unnecessary to define and configure handlers for all the loggers an application uses. It is sufficient to configure handlers for a top-level logger and create child loggers as needed. (You can, however, turn off propagation by setting the propagate attribute of a logger to False.) - Loggers documentation

import logging

myformatter = logging.Formatter("MY HANDLER: %(name)s - %(message)s")

myhandler = logging.StreamHandler()
myhandler.setFormatter(myformatter)

foo_logger = logging.getLogger('foo')
foo_logger.addHandler(myhandler)

foo_bar_logger = logging.getLogger('foo.bar')

foo_logger.error('asdfasdf')
foo_bar_logger.error('zxcvzxcv')
MY HANDLER: 40 foo - asdfasdf
MY HANDLER: 40 foo.bar - zxcvzxcv

However, filters, unlike levels and handlers, do not propagate. If a filter is not defined for a logger, the filter of the parent is NOT used.

Note that filters attached to handlers are consulted before an event is emitted by the handler, whereas filters attached to loggers are consulted whenever an event is logged (using debug(), info(), etc.), before sending an event to handlers. This means that events which have been generated by descendant loggers will not be filtered by a logger’s filter setting, unless the filter has also been applied to those descendant loggers. - Filter Objects documentation

See also the logging flowchart.

import logging

class MyFilter(logging.Filter):
    def filter(self, record):
        record.msg = 'MY FILTER: ' + record.msg
        return 1

myfilter = MyFilter()

myformatter = logging.Formatter("MY HANDLER: %(name)s - %(message)s")

myhandler = logging.StreamHandler()
myhandler.setFormatter(myformatter)

foo_logger = logging.getLogger('foo')
foo_logger.addFilter(myfilter)
foo_logger.addHandler(myhandler)

foo_bar_logger = logging.getLogger('foo.bar')

foo_logger.error('asdfasdf')
foo_bar_logger.error('zxcvzxcv')
MY HANDLER: foo - MY FILTER: asdfasdf
MY HANDLER: foo.bar - zxcvzxcv

I guess I'll attach it to the handler instead...

import logging

class MyFilter(logging.Filter):
    def filter(self, record):
        record.msg = 'MY FILTER: ' + record.msg
        return 1

myfilter = MyFilter()

myformatter = logging.Formatter("MY HANDLER: %(name)s - %(message)s")

myhandler = logging.StreamHandler()
myhandler.setFormatter(myformatter)
myhandler.addFilter(myfilter)

foo_logger = logging.getLogger('foo')
foo_logger.addHandler(myhandler)

foo_bar_logger = logging.getLogger('foo.bar')

foo_logger.error('asdfasdf')
foo_bar_logger.error('zxcvzxcv')
MY HANDLER: foo - MY FILTER: asdfasdf
MY HANDLER: foo.bar - MY FILTER: zxcvzxcv

Comments


#1 Vinay Sajip commented on :

Levels on handlers don't propagate, either. The concept of an effective level (use an ancestor's level if none is set for the current instance) applies only to loggers.


#2 Burrito commented on :

Thanks for the tip.

A suggestion for the styling on this website: Make your links actually look like links. I had to hover over "the logging flowchart" to know whether it is clickable.

disqus:2510386934


#3 Eliot commented on :

Thanks for bringing that to my attention. I fixed the styling for that link.

disqus:2532217740


#4 Aaron Ross commented on :

Whee, I was sooo confused! Thanks for posting. I wonder what the motivation was for the difference in propagation? Must be some interesting use case I'm missing.

disqus:2551167112