{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "title",
   "metadata": {},
   "source": [
    "# Log levels and formatting\n",
    "\n",
    "In this tutorial, you will explore the five standard log levels in depth and learn\n",
    "how to customise the format of your log messages. Understanding log levels is essential\n",
    "for controlling which messages appear in your output.\n",
    "\n",
    "**Time commitment:** 15–20 minutes\n",
    "\n",
    "**Prerequisites:**\n",
    "\n",
    "- [Your first log message](https://agilearn.co.uk/guides/logging-and-debugging/learn/01-your-first-log-message) tutorial completed\n",
    "- Basic understanding of `logging.basicConfig()` and `logging.getLogger()`\n",
    "\n",
    "## Learning objectives\n",
    "\n",
    "By the end of this tutorial, you will be able to:\n",
    "\n",
    "- Name the five standard log levels and their numeric values\n",
    "- Set the effective log level for a logger\n",
    "- Use format strings to customise log message appearance\n",
    "- Apply date formatting with the `datefmt` parameter\n",
    "- Explain why `%s` formatting is preferred over f-strings in log calls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "levels-overview",
   "metadata": {},
   "source": [
    "## The five standard log levels\n",
    "\n",
    "The `logging` module defines five standard severity levels, each with a name and a\n",
    "numeric value. The numeric values determine the ordering: higher values represent\n",
    "more severe events.\n",
    "\n",
    "| Level | Numeric value | Purpose |\n",
    "|-------|--------------|----------|\n",
    "| `DEBUG` | 10 | Detailed diagnostic information for developers |\n",
    "| `INFO` | 20 | Confirmation that things are working as expected |\n",
    "| `WARNING` | 30 | Something unexpected happened, but the program still works |\n",
    "| `ERROR` | 40 | A specific operation failed |\n",
    "| `CRITICAL` | 50 | The program itself may not be able to continue |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "numeric-values",
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "# Each level has a numeric value\n",
    "print(\"DEBUG:\", logging.DEBUG)\n",
    "print(\"INFO:\", logging.INFO)\n",
    "print(\"WARNING:\", logging.WARNING)\n",
    "print(\"ERROR:\", logging.ERROR)\n",
    "print(\"CRITICAL:\", logging.CRITICAL)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "how-levels-work",
   "metadata": {},
   "source": [
    "## How log levels work\n",
    "\n",
    "When you set a log level on a logger, only messages at that level **or above** are\n",
    "processed. Messages below the effective level are silently discarded.\n",
    "\n",
    "For example, if you set the level to `WARNING` (30), then:\n",
    "\n",
    "- `DEBUG` (10) messages are **discarded**\n",
    "- `INFO` (20) messages are **discarded**\n",
    "- `WARNING` (30) messages are **processed**\n",
    "- `ERROR` (40) messages are **processed**\n",
    "- `CRITICAL` (50) messages are **processed**\n",
    "\n",
    "Think of it as a threshold: only messages that meet or exceed the threshold pass through."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "level-filtering-demo",
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "# Set level to INFO – DEBUG messages will be suppressed\n",
    "logging.basicConfig(level=logging.INFO, format=\"%(levelname)s: %(message)s\")\n",
    "\n",
    "logging.debug(\"This will NOT appear (below INFO threshold)\")\n",
    "logging.info(\"This WILL appear\")\n",
    "logging.warning(\"This WILL appear\")\n",
    "logging.error(\"This WILL appear\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "setting-level-section",
   "metadata": {},
   "source": [
    "## Setting the log level\n",
    "\n",
    "There are two main ways to set the log level:\n",
    "\n",
    "1. **Using `logging.basicConfig(level=...)`** -- Sets the level on the root logger\n",
    "2. **Using `logger.setLevel()`** -- Sets the level on a specific named logger\n",
    "\n",
    "Let us see both approaches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "setlevel-demo",
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "logging.basicConfig(format=\"%(name)s - %(levelname)s: %(message)s\")\n",
    "\n",
    "# Create two loggers with different levels\n",
    "app_logger = logging.getLogger(\"app\")\n",
    "app_logger.setLevel(logging.DEBUG)\n",
    "\n",
    "db_logger = logging.getLogger(\"database\")\n",
    "db_logger.setLevel(logging.WARNING)\n",
    "\n",
    "# The app logger shows everything from DEBUG upwards\n",
    "app_logger.debug(\"Detailed app information\")\n",
    "app_logger.info(\"App is running\")\n",
    "\n",
    "# The database logger only shows WARNING and above\n",
    "db_logger.debug(\"This will NOT appear\")\n",
    "db_logger.info(\"This will NOT appear either\")\n",
    "db_logger.warning(\"Connection pool is almost full\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "formatting-section",
   "metadata": {},
   "source": [
    "## Formatting log messages\n",
    "\n",
    "The format of log messages is controlled by **format strings**. These strings use\n",
    "special placeholders that are replaced with information from the log record when a\n",
    "message is emitted.\n",
    "\n",
    "Format placeholders use the `%(name)s` syntax, where `name` is an attribute of the\n",
    "log record."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "common-attributes",
   "metadata": {},
   "source": [
    "### Common format attributes\n",
    "\n",
    "The following table lists the most commonly used format attributes:\n",
    "\n",
    "| Attribute | Format | Description |\n",
    "|-----------|--------|-------------|\n",
    "| `asctime` | `%(asctime)s` | Human-readable timestamp |\n",
    "| `name` | `%(name)s` | Name of the logger |\n",
    "| `levelname` | `%(levelname)s` | Severity level (DEBUG, INFO, and so on) |\n",
    "| `message` | `%(message)s` | The log message |\n",
    "| `filename` | `%(filename)s` | Name of the source file |\n",
    "| `lineno` | `%(lineno)d` | Line number in the source file |\n",
    "| `funcName` | `%(funcName)s` | Name of the function that logged the message |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "format-demo",
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "# A detailed format string with timestamp, logger name, level, and message\n",
    "logging.basicConfig(\n",
    "    level=logging.DEBUG,\n",
    "    format=\"%(asctime)s - %(name)s - %(levelname)s - %(message)s\"\n",
    ")\n",
    "\n",
    "logger = logging.getLogger(\"my_app\")\n",
    "logger.info(\"Application started\")\n",
    "logger.debug(\"Loading configuration\")\n",
    "logger.warning(\"Configuration file not found, using defaults\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "format-with-location",
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "# Include source file location in the format\n",
    "logging.basicConfig(\n",
    "    level=logging.DEBUG,\n",
    "    format=\"%(levelname)s [%(filename)s:%(lineno)d] %(message)s\"\n",
    ")\n",
    "\n",
    "logger = logging.getLogger(\"location_demo\")\n",
    "logger.info(\"This message includes source location\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "datefmt-section",
   "metadata": {},
   "source": [
    "## Date formatting\n",
    "\n",
    "By default, the `%(asctime)s` attribute produces timestamps in the format\n",
    "`2024-01-15 14:30:00,123`. You can customise this using the `datefmt` parameter,\n",
    "which accepts standard `strftime` directives.\n",
    "\n",
    "Some useful directives:\n",
    "\n",
    "| Directive | Meaning | Example |\n",
    "|-----------|---------|----------|\n",
    "| `%d` | Day of the month (zero-padded) | 09 |\n",
    "| `%m` | Month (zero-padded) | 02 |\n",
    "| `%Y` | Four-digit year | 2026 |\n",
    "| `%H` | Hour (24-hour, zero-padded) | 14 |\n",
    "| `%M` | Minute (zero-padded) | 30 |\n",
    "| `%S` | Second (zero-padded) | 05 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "datefmt-demo",
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "# Use a British date format: DD/MM/YYYY HH:MM:SS\n",
    "logging.basicConfig(\n",
    "    level=logging.DEBUG,\n",
    "    format=\"%(asctime)s - %(levelname)s - %(message)s\",\n",
    "    datefmt=\"%d/%m/%Y %H:%M:%S\"\n",
    ")\n",
    "\n",
    "logger = logging.getLogger(\"date_demo\")\n",
    "logger.info(\"This message has a custom date format\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "lazy-formatting-section",
   "metadata": {},
   "source": [
    "## Lazy string formatting\n",
    "\n",
    "An important best practice when using the `logging` module is to use `%s` style\n",
    "formatting in log calls instead of f-strings.\n",
    "\n",
    "**Why?** With `%s` formatting, the string is only formatted if the message will actually\n",
    "be emitted. If the log level means the message will be discarded, the formatting\n",
    "is skipped entirely. This is called **lazy evaluation**.\n",
    "\n",
    "With f-strings, the formatting always happens, even when the message is discarded.\n",
    "This wastes processing time unnecessarily."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "lazy-formatting-demo",
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "logging.basicConfig(level=logging.WARNING, format=\"%(levelname)s: %(message)s\")\n",
    "\n",
    "logger = logging.getLogger(\"formatting_demo\")\n",
    "\n",
    "username = \"alice\"\n",
    "action = \"logged in\"\n",
    "\n",
    "# Correct: %s formatting (lazy – only formatted if DEBUG is enabled)\n",
    "logger.debug(\"User %s has %s\", username, action)\n",
    "\n",
    "# Incorrect: f-string (always formatted, even though DEBUG is disabled)\n",
    "# logger.debug(f\"User {username} has {action}\")  # Do not do this\n",
    "\n",
    "print(\"The DEBUG message above was discarded without formatting the string.\")\n",
    "print(\"With an f-string, the formatting would have happened anyway.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exercises-section",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "Try these exercises to reinforce what you have learned."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exercise-1",
   "metadata": {},
   "source": [
    "### Exercise 1: Level filtering\n",
    "\n",
    "Configure logging at `ERROR` level. Then log a message at each of the five levels.\n",
    "Predict which messages will appear before running the cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "exercise-1-solution",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise 1: Solution\n",
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "logging.basicConfig(level=logging.ERROR, format=\"%(levelname)s: %(message)s\")\n",
    "\n",
    "logging.debug(\"Debug message\")       # Will NOT appear\n",
    "logging.info(\"Info message\")          # Will NOT appear\n",
    "logging.warning(\"Warning message\")    # Will NOT appear\n",
    "logging.error(\"Error message\")        # Will appear\n",
    "logging.critical(\"Critical message\")  # Will appear\n",
    "\n",
    "# Only ERROR (40) and CRITICAL (50) meet the ERROR (40) threshold"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exercise-2",
   "metadata": {},
   "source": [
    "### Exercise 2: Custom format\n",
    "\n",
    "Create a format string that produces output like the following:\n",
    "\n",
    "```\n",
    "[INFO] my_app :: Application started (line 5)\n",
    "```\n",
    "\n",
    "The format should include the level name in square brackets, the logger name,\n",
    "the message, and the line number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "exercise-2-solution",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise 2: Solution\n",
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "logging.basicConfig(\n",
    "    level=logging.DEBUG,\n",
    "    format=\"[%(levelname)s] %(name)s :: %(message)s (line %(lineno)d)\"\n",
    ")\n",
    "\n",
    "logger = logging.getLogger(\"my_app\")\n",
    "logger.info(\"Application started\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exercise-3",
   "metadata": {},
   "source": [
    "### Exercise 3: Date formatting\n",
    "\n",
    "Configure logging with a format that includes a British-style timestamp\n",
    "(`DD/MM/YYYY HH:MM:SS`), the level name, and the message. Log a few test messages\n",
    "to verify the output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "exercise-3-solution",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise 3: Solution\n",
    "import importlib\n",
    "import logging\n",
    "\n",
    "importlib.reload(logging)\n",
    "\n",
    "logging.basicConfig(\n",
    "    level=logging.DEBUG,\n",
    "    format=\"%(asctime)s [%(levelname)s] %(message)s\",\n",
    "    datefmt=\"%d/%m/%Y %H:%M:%S\"\n",
    ")\n",
    "\n",
    "logger = logging.getLogger(\"date_exercise\")\n",
    "logger.info(\"Server started\")\n",
    "logger.warning(\"High memory usage detected\")\n",
    "logger.error(\"Connection to database failed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "summary",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "In this tutorial, you learned the following:\n",
    "\n",
    "- The five standard log levels are `DEBUG` (10), `INFO` (20), `WARNING` (30),\n",
    "  `ERROR` (40), and `CRITICAL` (50)\n",
    "- Setting a log level creates a threshold: only messages at that level or above are processed\n",
    "- Format strings use `%(attribute)s` placeholders to include information such as\n",
    "  timestamps, logger names, level names, and source locations\n",
    "- The `datefmt` parameter customises the timestamp format using `strftime` directives\n",
    "- Always use `%s` formatting in log calls, not f-strings, to benefit from lazy evaluation\n",
    "\n",
    "**Next tutorial:** Continue to [Logging to files](https://agilearn.co.uk/guides/logging-and-debugging/learn/03-logging-to-files) to learn\n",
    "how to direct log output to files and combine multiple output destinations."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}