Python logging module: duplicated console output [IPython Notebook/Qtconsole] -

i trying play python logging module bit confused here. below standard script create logger first, , create , add file handler , console handler logger.

import logging  logger = logging.getlogger('logging_test') logger.setlevel(logging.debug)  print(len(logger.handlers))  # output: 0  # create file handler logs debug messages fh = logging.filehandler('/home/jian/downloads/spam.log', mode='w') fh.setlevel(logging.debug)  # create console handler higher log level ch = logging.streamhandler() ch.setlevel(logging.debug)  # create formatter , add handlers formatter = logging.formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') ch.setformatter(formatter) fh.setformatter(formatter)  # add handlers logger logger.addhandler(ch) logger.addhandler(fh)  print(len(logger.handlers))  # output: 2  # write log messages logger.debug('debug message') logger.info('info message') logger.warn('warn message') logger.error('error message') logger.critical('critical message') 

i run on newly started kernel. file handler works expected. in console output, i've got sort of duplicated messages:

2015-07-14 10:59:26,942 - logging_test - debug - debug message debug:logging_test:debug message 2015-07-14 10:59:26,944 - logging_test - info - info message info:logging_test:info message 2015-07-14 10:59:26,944 - logging_test - warning - warn message warning:logging_test:warn message 2015-07-14 10:59:26,945 - logging_test - error - error message error:logging_test:error message 2015-07-14 10:59:26,946 - logging_test - critical - critical message critical:logging_test:critical message 

i suppose log messages time stamps user-defined console handler, duplicated messages come from? can rid of them, say, keep every other line? appreciated.

the issue has been raised here.

the observation following: in normal python, or ipython console, there no handler root logger installed until root logger used issue log message:

in [1]: import logging  in [2]: logging.getlogger().handlers out[2]: []  in [3]: logging.warn('something happened!') warning:root:something happened!  in [4]: logging.getlogger().handlers out[4]: [<logging.streamhandler @ 0x42acef0>] 

however, in ipython notebook, there default stderr root logger installed right away:

in [1]: import logging  in [2]: logging.getlogger().handlers out[2]: [<logging.streamhandler @ 0x35eedd8>] 

maybe i'm missing something, think in notebook, there should no handler installed automatically, number of reasons:

1. it make default logging configuration consistent between standard python, ipython console , ipython notebook.
2. as user writes log message root logger, handler gets installed automatically, , therefore log messages not missed.
3. with current behaviour, library configures child logger, , handler child logger, might spam notebook debug messages supposed go in log file (or elsewhere). example, astropy seems have such issue, , i'm running same issue own library. problem such library, there no "clean" way around this. library remove handler of root logger when imported, hack-y. set own logger's propagate attribute false, log messages not propagated root logger, not disables debug output going notebook, more severe messages. also, prevents users capturing log output if want to.

an alternative might add configuration option specifies log level automatically added stream handler, becomes possible ignore less severe messages showing in notebook automatically. still make behaviour different between ipython console , ipython notebook.

the downside see making sure there no default handler set libraries/notebooks in use might rely on behaviour , actively work around it, example disabling own handlers if detect running in ipython notebook. such cases break such change.

so setting logger.propagate false or using reload(logging) prevent duplicate output depending on have side effects.

note reload not available in newer version of python (3.4, maybe earlier). 3.1 onwards see importlib.reload