Configure logging for a project¶
The question. Your project has several modules and you want one place that decides what gets logged, where it goes, and at what level. You want the console to stay terse while a file captures everything; you want __name__-based loggers in each module to inherit from a project-level logger; and you don't want to hand-write handler wiring in every module.
The answer: use logging.config.dictConfig(...) at your application entry point. One dictionary describes formatters, handlers, and loggers; every module calls logging.getLogger(__name__) and inherits from the hierarchy.
# One-stop configuration: formatters + handlers + a project logger.
# Console is brief and WARNING+; file captures everything at DEBUG.
import logging
import logging.config
import tempfile
from pathlib import Path
log_path = Path(tempfile.mkdtemp()) / 'app.log'
LOGGING_CONFIG = {
'version': 1, # always 1; only valid value
'disable_existing_loggers': False, # keep loggers created before now
'formatters': {
'brief': {'format': '%(levelname)s: %(message)s'},
'detailed': {'format': '%(asctime)s - %(name)s - %(levelname)s - %(message)s'},
},
'handlers': {
'console': {'class': 'logging.StreamHandler',
'level': 'WARNING', 'formatter': 'brief'},
'file': {'class': 'logging.FileHandler',
'level': 'DEBUG', 'formatter': 'detailed',
'filename': str(log_path)},
},
'loggers': {
'my_project': {'level': 'DEBUG',
'handlers': ['console', 'file'],
'propagate': False},
},
'root': {'level': 'WARNING', 'handlers': ['console']},
}
logging.config.dictConfig(LOGGING_CONFIG)
# In each module: logger = logging.getLogger(__name__)
# Children of 'my_project' (e.g. 'my_project.database') inherit automatically.
log = logging.getLogger('my_project.demo')
log.debug('verbose trace — file only')
log.info('routine progress — file only')
log.warning('something to notice — file AND console')
for handler in logging.getLogger('my_project').handlers:
handler.flush()
print('--- file contents ---')
print(log_path.read_text())
Variant: load config from a JSON file¶
Pull the config out of code so it can be tweaked by ops without a code deploy. JSON works out of the box; YAML is nicer to read if you're willing to take the pyyaml dependency.
import json
import logging
import logging.config
import tempfile
from pathlib import Path
tmp = Path(tempfile.mkdtemp())
config_file = tmp / 'logging_config.json'
log_file = tmp / 'app.log'
config = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'standard': {'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'}
},
'handlers': {
'file': {'class': 'logging.FileHandler', 'level': 'DEBUG',
'formatter': 'standard', 'filename': str(log_file)},
},
'root': {'level': 'DEBUG', 'handlers': ['file']},
}
config_file.write_text(json.dumps(config, indent=2))
# At program start:
logging.config.dictConfig(json.loads(config_file.read_text()))
logging.getLogger('json_config_demo').info('loaded config from JSON')
for handler in logging.getLogger().handlers:
handler.flush()
print(log_file.read_text())
Variant: hierarchy in action¶
Children inherit from their dotted-name parent. You can set a child's level higher than its parent's to quieten a noisy module without losing the rest.
import logging
import logging.config
logging.config.dictConfig({
'version': 1,
'disable_existing_loggers': False,
'formatters': {'standard': {'format': '%(name)s - %(levelname)s - %(message)s'}},
'handlers': {'console': {'class': 'logging.StreamHandler', 'formatter': 'standard'}},
'loggers': {
'my_project': {'level': 'DEBUG', 'handlers': ['console'], 'propagate': False},
'my_project.database': {'level': 'WARNING'}, # quieter child
},
})
logging.getLogger('my_project.app').debug('inherits DEBUG from parent')
logging.getLogger('my_project.database').debug('SUPPRESSED — child is WARNING')
logging.getLogger('my_project.database').warning('slow database query')
Why this works¶
dictConfig is the declarative setup path: formatters, handlers, and loggers are all described in one dict rather than wired up with individual calls. You configure once at program start and every subsequent getLogger(__name__) call inherits the result.
Logger names are dot-separated and form a hierarchy. my_project.database is a child of my_project, which is a child of the root logger. A child without its own handlers propagates records to its parent — that's how modules across your codebase all end up writing through the same handlers without each one configuring them.
Each handler has its own level filter, which is how 'console is terse, file is verbose' works. A record passes the logger's level first, then the handler's; both gates have to open. propagate: False on the project logger stops records from bubbling up to the root (where you'd get duplicates if the root had its own handlers for the same destinations).
Trade-offs¶
Configure logging once, at the application entry point (inside your main() or if __name__ == '__main__':). Library code must not call dictConfig or basicConfig — configuration is the application's decision, not the library's. A library should at most add a logging.NullHandler to its top-level logger to silence the 'no handlers found' warning.
For more complex setups, pull the config dict out to a JSON or YAML file and load it at startup — that lets ops tweak levels and destinations without editing code. See the extra cells for the JSON variant.
For dev-vs-prod differences, feed the environment into a small function that returns the right dict (get_logging_config(env)). Keeps the branching in one place rather than scattered across handler definitions.
Related reading¶
- Create custom log handlers — for destinations not covered by the built-ins.
- Avoid common logging mistakes — duplicate handlers, f-strings in log calls, library-side configuration.
- Understanding log levels — what DEBUG/INFO/WARNING/ERROR actually mean.
- Logging module quick reference — the dictConfig schema in full.