{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configure logging for a project\n",
    "\n",
    "**The question.** Your project has several modules and you want one place that decides what gets logged, where it goes, and at what level. You want the console to stay terse while a file captures everything; you want `__name__`-based loggers in each module to inherit from a project-level logger; and you don't want to hand-write handler wiring in every module.\n",
    "\n",
    "The answer: use `logging.config.dictConfig(...)` at your application entry point. One dictionary describes formatters, handlers, and loggers; every module calls `logging.getLogger(__name__)` and inherits from the hierarchy.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# One-stop configuration: formatters + handlers + a project logger.\n",
    "# Console is brief and WARNING+; file captures everything at DEBUG.\n",
    "import logging\n",
    "import logging.config\n",
    "import tempfile\n",
    "from pathlib import Path\n",
    "\n",
    "log_path = Path(tempfile.mkdtemp()) / 'app.log'\n",
    "\n",
    "LOGGING_CONFIG = {\n",
    "    'version': 1,                                 # always 1; only valid value\n",
    "    'disable_existing_loggers': False,            # keep loggers created before now\n",
    "    'formatters': {\n",
    "        'brief':    {'format': '%(levelname)s: %(message)s'},\n",
    "        'detailed': {'format': '%(asctime)s - %(name)s - %(levelname)s - %(message)s'},\n",
    "    },\n",
    "    'handlers': {\n",
    "        'console': {'class': 'logging.StreamHandler',\n",
    "                    'level': 'WARNING', 'formatter': 'brief'},\n",
    "        'file':    {'class': 'logging.FileHandler',\n",
    "                    'level': 'DEBUG',   'formatter': 'detailed',\n",
    "                    'filename': str(log_path)},\n",
    "    },\n",
    "    'loggers': {\n",
    "        'my_project': {'level': 'DEBUG',\n",
    "                       'handlers': ['console', 'file'],\n",
    "                       'propagate': False},\n",
    "    },\n",
    "    'root': {'level': 'WARNING', 'handlers': ['console']},\n",
    "}\n",
    "\n",
    "logging.config.dictConfig(LOGGING_CONFIG)\n",
    "\n",
    "# In each module: logger = logging.getLogger(__name__)\n",
    "# Children of 'my_project' (e.g. 'my_project.database') inherit automatically.\n",
    "log = logging.getLogger('my_project.demo')\n",
    "log.debug('verbose trace — file only')\n",
    "log.info('routine progress — file only')\n",
    "log.warning('something to notice — file AND console')\n",
    "\n",
    "for handler in logging.getLogger('my_project').handlers:\n",
    "    handler.flush()\n",
    "\n",
    "print('--- file contents ---')\n",
    "print(log_path.read_text())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variant: load config from a JSON file\n",
    "\n",
    "Pull the config out of code so it can be tweaked by ops without a code deploy. JSON works out of the box; YAML is nicer to read if you're willing to take the `pyyaml` dependency.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import logging\n",
    "import logging.config\n",
    "import tempfile\n",
    "from pathlib import Path\n",
    "\n",
    "tmp = Path(tempfile.mkdtemp())\n",
    "config_file = tmp / 'logging_config.json'\n",
    "log_file = tmp / 'app.log'\n",
    "\n",
    "config = {\n",
    "    'version': 1,\n",
    "    'disable_existing_loggers': False,\n",
    "    'formatters': {\n",
    "        'standard': {'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'}\n",
    "    },\n",
    "    'handlers': {\n",
    "        'file': {'class': 'logging.FileHandler', 'level': 'DEBUG',\n",
    "                 'formatter': 'standard', 'filename': str(log_file)},\n",
    "    },\n",
    "    'root': {'level': 'DEBUG', 'handlers': ['file']},\n",
    "}\n",
    "config_file.write_text(json.dumps(config, indent=2))\n",
    "\n",
    "# At program start:\n",
    "logging.config.dictConfig(json.loads(config_file.read_text()))\n",
    "logging.getLogger('json_config_demo').info('loaded config from JSON')\n",
    "\n",
    "for handler in logging.getLogger().handlers:\n",
    "    handler.flush()\n",
    "print(log_file.read_text())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variant: hierarchy in action\n",
    "\n",
    "Children inherit from their dotted-name parent. You can set a child's level *higher* than its parent's to quieten a noisy module without losing the rest.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import logging.config\n",
    "\n",
    "logging.config.dictConfig({\n",
    "    'version': 1,\n",
    "    'disable_existing_loggers': False,\n",
    "    'formatters': {'standard': {'format': '%(name)s - %(levelname)s - %(message)s'}},\n",
    "    'handlers': {'console': {'class': 'logging.StreamHandler', 'formatter': 'standard'}},\n",
    "    'loggers': {\n",
    "        'my_project':          {'level': 'DEBUG',   'handlers': ['console'], 'propagate': False},\n",
    "        'my_project.database': {'level': 'WARNING'},   # quieter child\n",
    "    },\n",
    "})\n",
    "\n",
    "logging.getLogger('my_project.app').debug('inherits DEBUG from parent')\n",
    "logging.getLogger('my_project.database').debug('SUPPRESSED — child is WARNING')\n",
    "logging.getLogger('my_project.database').warning('slow database query')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why this works\n",
    "\n",
    "`dictConfig` is the declarative setup path: formatters, handlers, and loggers are all described in one dict rather than wired up with individual calls. You configure once at program start and every subsequent `getLogger(__name__)` call inherits the result.\n",
    "\n",
    "Logger names are dot-separated and form a hierarchy. `my_project.database` is a child of `my_project`, which is a child of the root logger. A child without its own handlers *propagates* records to its parent — that's how modules across your codebase all end up writing through the same handlers without each one configuring them.\n",
    "\n",
    "Each handler has its own level filter, which is how 'console is terse, file is verbose' works. A record passes the logger's level first, then the handler's; both gates have to open. `propagate: False` on the project logger stops records from bubbling up to the root (where you'd get duplicates if the root had its own handlers for the same destinations).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Trade-offs\n",
    "\n",
    "Configure logging *once*, at the application entry point (inside your `main()` or `if __name__ == '__main__':`). Library code must not call `dictConfig` or `basicConfig` — configuration is the application's decision, not the library's. A library should at most add a `logging.NullHandler` to its top-level logger to silence the 'no handlers found' warning.\n",
    "\n",
    "For more complex setups, pull the config dict out to a JSON or YAML file and load it at startup — that lets ops tweak levels and destinations without editing code. See the extra cells for the JSON variant.\n",
    "\n",
    "For dev-vs-prod differences, feed the environment into a small function that returns the right dict (`get_logging_config(env)`). Keeps the branching in one place rather than scattered across handler definitions.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Related reading\n",
    "\n",
    "- [Create custom log handlers](https://agilearn.co.uk/guides/logging-and-debugging/recipes/create-custom-log-handlers) — for destinations not covered by the built-ins.\n",
    "- [Avoid common logging mistakes](https://agilearn.co.uk/guides/logging-and-debugging/recipes/avoid-common-logging-mistakes) — duplicate handlers, f-strings in log calls, library-side configuration.\n",
    "- [Understanding log levels](https://agilearn.co.uk/guides/logging-and-debugging/concepts/understanding-log-levels) — what DEBUG/INFO/WARNING/ERROR actually mean.\n",
    "- [Logging module quick reference](https://agilearn.co.uk/guides/logging-and-debugging/reference/logging-module-quick-reference) — the dictConfig schema in full.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}