Logging to files¶

In this tutorial, you will learn how to direct log output to files using handlers. File-based logging is essential for production applications, where you need a persistent record of what happened.

Time commitment: 15–20 minutes

Prerequisites:

• Your first log message tutorial completed
• Log levels and formatting tutorial completed

Learning objectives¶

By the end of this tutorial, you will be able to:

• Use logging.FileHandler to write log messages to a file
• Use logging.handlers.RotatingFileHandler for size-limited log files
• Combine multiple handlers to send log output to different destinations
• Set different log levels for different handlers

Why log to files?¶

Console output disappears when the terminal is closed. For production applications, you need log messages stored persistently so that you can:

• Review what happened after the fact (post-mortem analysis)
• Track application behaviour over time
• Share logs with team members for debugging
• Analyse patterns and trends in application events

The logging module provides several handler classes for writing to files. A handler is an object that sends log records to a specific destination.

Using logging.FileHandler¶

The simplest way to log to a file is with logging.FileHandler. It writes all log messages to a single file.

Let us create a logger with a file handler. We will use a temporary file so the example cleans up after itself.

In [ ]:

import logging
import os
import tempfile

# Create a temporary file for our log output
log_dir = tempfile.mkdtemp()
log_file = os.path.join(log_dir, "app.log")

# Create a logger
logger = logging.getLogger("file_demo")
logger.setLevel(logging.DEBUG)

# Create a file handler
file_handler = logging.FileHandler(log_file)
file_handler.setLevel(logging.DEBUG)

# Create a formatter and attach it to the handler
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
file_handler.setFormatter(formatter)

# Add the handler to the logger
logger.addHandler(file_handler)

# Log some messages
logger.info("Application started")
logger.debug("Loading configuration")
logger.warning("Configuration file not found, using defaults")

# Flush and read the file to see what was written
file_handler.flush()

with open(log_file, "r", encoding="utf-8") as f:
    print("Contents of log file:")
    print(f.read())

# Clean up the handler
logger.removeHandler(file_handler)
file_handler.close()

import logging import os import tempfile # Create a temporary file for our log output log_dir = tempfile.mkdtemp() log_file = os.path.join(log_dir, "app.log") # Create a logger logger = logging.getLogger("file_demo") logger.setLevel(logging.DEBUG) # Create a file handler file_handler = logging.FileHandler(log_file) file_handler.setLevel(logging.DEBUG) # Create a formatter and attach it to the handler formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") file_handler.setFormatter(formatter) # Add the handler to the logger logger.addHandler(file_handler) # Log some messages logger.info("Application started") logger.debug("Loading configuration") logger.warning("Configuration file not found, using defaults") # Flush and read the file to see what was written file_handler.flush() with open(log_file, "r", encoding="utf-8") as f: print("Contents of log file:") print(f.read()) # Clean up the handler logger.removeHandler(file_handler) file_handler.close()

The key steps for setting up file logging are:

1. Create a logger with logging.getLogger()
2. Create a logging.FileHandler with the path to the log file
3. Create a logging.Formatter with your desired format string
4. Attach the formatter to the handler with handler.setFormatter()
5. Add the handler to the logger with logger.addHandler()

Combining console and file handlers¶

A logger can have multiple handlers attached at the same time. A common pattern is to send log output to both the console and a file. This way, you can see messages in real time on the console while also keeping a persistent record in a file.

In [ ]:

import logging
import os
import tempfile

log_dir = tempfile.mkdtemp()
log_file = os.path.join(log_dir, "dual.log")

# Create a logger
logger = logging.getLogger("dual_demo")
logger.setLevel(logging.DEBUG)

# Console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)
console_formatter = logging.Formatter("%(levelname)s: %(message)s")
console_handler.setFormatter(console_formatter)

# File handler (with more detail)
file_handler = logging.FileHandler(log_file)
file_handler.setLevel(logging.DEBUG)
file_formatter = logging.Formatter(
    "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)
file_handler.setFormatter(file_formatter)

# Add both handlers
logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Log some messages
logger.info("Starting the application")
logger.warning("Low disk space")

# Show the file contents
file_handler.flush()
print("\n--- File contents ---")
with open(log_file, "r", encoding="utf-8") as f:
    print(f.read())

# Clean up
logger.removeHandler(console_handler)
logger.removeHandler(file_handler)
console_handler.close()
file_handler.close()

import logging import os import tempfile log_dir = tempfile.mkdtemp() log_file = os.path.join(log_dir, "dual.log") # Create a logger logger = logging.getLogger("dual_demo") logger.setLevel(logging.DEBUG) # Console handler console_handler = logging.StreamHandler() console_handler.setLevel(logging.DEBUG) console_formatter = logging.Formatter("%(levelname)s: %(message)s") console_handler.setFormatter(console_formatter) # File handler (with more detail) file_handler = logging.FileHandler(log_file) file_handler.setLevel(logging.DEBUG) file_formatter = logging.Formatter( "%(asctime)s - %(name)s - %(levelname)s - %(message)s" ) file_handler.setFormatter(file_formatter) # Add both handlers logger.addHandler(console_handler) logger.addHandler(file_handler) # Log some messages logger.info("Starting the application") logger.warning("Low disk space") # Show the file contents file_handler.flush() print("\n--- File contents ---") with open(log_file, "r", encoding="utf-8") as f: print(f.read()) # Clean up logger.removeHandler(console_handler) logger.removeHandler(file_handler) console_handler.close() file_handler.close()

Notice that each handler can have its own formatter. In this example, the console shows a brief format, while the file stores a more detailed format including timestamps and the logger name.

Different levels for different handlers¶

Each handler can have its own log level, independent of the logger level. This is useful when you want detailed logging in files but only important messages on the console.

Important: The logger level acts as a first gate. If a message does not pass the logger level, it is not sent to any handler. Handler levels provide a second gate for each individual handler.

In [ ]:

import logging
import os
import tempfile

log_dir = tempfile.mkdtemp()
log_file = os.path.join(log_dir, "levels.log")

logger = logging.getLogger("level_demo")
logger.setLevel(logging.DEBUG)

# Console handler – only show WARNING and above
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)
console_handler.setFormatter(logging.Formatter("%(levelname)s: %(message)s"))

# File handler – capture everything from DEBUG upwards
file_handler = logging.FileHandler(log_file)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(
    logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
)

logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Log at all levels
logger.debug("Detailed diagnostic information")
logger.info("General information")
logger.warning("Something unexpected happened")
logger.error("An error occurred")

# Show what went to the file
file_handler.flush()
print("\n--- File contents (all levels) ---")
with open(log_file, "r", encoding="utf-8") as f:
    print(f.read())

# Clean up
logger.removeHandler(console_handler)
logger.removeHandler(file_handler)
console_handler.close()
file_handler.close()

import logging import os import tempfile log_dir = tempfile.mkdtemp() log_file = os.path.join(log_dir, "levels.log") logger = logging.getLogger("level_demo") logger.setLevel(logging.DEBUG) # Console handler – only show WARNING and above console_handler = logging.StreamHandler() console_handler.setLevel(logging.WARNING) console_handler.setFormatter(logging.Formatter("%(levelname)s: %(message)s")) # File handler – capture everything from DEBUG upwards file_handler = logging.FileHandler(log_file) file_handler.setLevel(logging.DEBUG) file_handler.setFormatter( logging.Formatter("%(asctime)s - %(levelname)s - %(message)s") ) logger.addHandler(console_handler) logger.addHandler(file_handler) # Log at all levels logger.debug("Detailed diagnostic information") logger.info("General information") logger.warning("Something unexpected happened") logger.error("An error occurred") # Show what went to the file file_handler.flush() print("\n--- File contents (all levels) ---") with open(log_file, "r", encoding="utf-8") as f: print(f.read()) # Clean up logger.removeHandler(console_handler) logger.removeHandler(file_handler) console_handler.close() file_handler.close()

In this example:

• The console only shows WARNING and ERROR (two messages)
• The file captures all four messages, including DEBUG and INFO

This is a common production pattern: keep detailed logs in files for later analysis, but only show important messages on the console to avoid clutter.

RotatingFileHandler¶

A plain FileHandler writes to a single file that grows without limit. For long-running applications, this can become a problem as the file grows very large.

logging.handlers.RotatingFileHandler solves this by automatically rotating log files when they reach a specified size. It keeps a configurable number of backup files.

Parameters:

• maxBytes -- Maximum size of each log file in bytes (0 means no limit)
• backupCount -- Number of backup files to keep

In [ ]:

import logging
import logging.handlers
import os
import tempfile

log_dir = tempfile.mkdtemp()
log_file = os.path.join(log_dir, "rotating.log")

logger = logging.getLogger("rotating_demo")
logger.setLevel(logging.DEBUG)

# Create a rotating file handler with a small max size for demonstration
rotating_handler = logging.handlers.RotatingFileHandler(
    log_file,
    maxBytes=200,
    backupCount=3
)
rotating_handler.setFormatter(
    logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
)
logger.addHandler(rotating_handler)

# Write enough messages to trigger rotation
for i in range(20):
    logger.info("Log message number %s", i)

# Show what files were created
print("Files in log directory:")
for filename in sorted(os.listdir(log_dir)):
    filepath = os.path.join(log_dir, filename)
    size = os.path.getsize(filepath)
    print("  %s (%s bytes)", filename, size)

# Clean up
logger.removeHandler(rotating_handler)
rotating_handler.close()

import logging import logging.handlers import os import tempfile log_dir = tempfile.mkdtemp() log_file = os.path.join(log_dir, "rotating.log") logger = logging.getLogger("rotating_demo") logger.setLevel(logging.DEBUG) # Create a rotating file handler with a small max size for demonstration rotating_handler = logging.handlers.RotatingFileHandler( log_file, maxBytes=200, backupCount=3 ) rotating_handler.setFormatter( logging.Formatter("%(asctime)s - %(levelname)s - %(message)s") ) logger.addHandler(rotating_handler) # Write enough messages to trigger rotation for i in range(20): logger.info("Log message number %s", i) # Show what files were created print("Files in log directory:") for filename in sorted(os.listdir(log_dir)): filepath = os.path.join(log_dir, filename) size = os.path.getsize(filepath) print(" %s (%s bytes)", filename, size) # Clean up logger.removeHandler(rotating_handler) rotating_handler.close()

When the log file reaches the maxBytes limit, it is renamed with a .1 suffix, and a new log file is started. The previous .1 file becomes .2, and so on. Files beyond backupCount are deleted automatically.

TimedRotatingFileHandler¶

If you prefer to rotate log files based on time rather than size, use logging.handlers.TimedRotatingFileHandler. It can rotate logs at regular intervals such as every hour, every day, or every week.

Common when values:

Value	Interval
"S"	Every second
"M"	Every minute
"H"	Every hour
"D"	Every day
"midnight"	At midnight

In [ ]:

import logging
import logging.handlers
import os
import tempfile

log_dir = tempfile.mkdtemp()
log_file = os.path.join(log_dir, "timed.log")

logger = logging.getLogger("timed_demo")
logger.setLevel(logging.DEBUG)

# Rotate every second (for demonstration purposes)
timed_handler = logging.handlers.TimedRotatingFileHandler(
    log_file,
    when="S",
    interval=1,
    backupCount=3
)
timed_handler.setFormatter(
    logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
)
logger.addHandler(timed_handler)

logger.info("This message goes to the timed rotating log")

print("Log file created at:", log_file)
print("In production, this handler would rotate files based on time.")

# Clean up
logger.removeHandler(timed_handler)
timed_handler.close()

import logging import logging.handlers import os import tempfile log_dir = tempfile.mkdtemp() log_file = os.path.join(log_dir, "timed.log") logger = logging.getLogger("timed_demo") logger.setLevel(logging.DEBUG) # Rotate every second (for demonstration purposes) timed_handler = logging.handlers.TimedRotatingFileHandler( log_file, when="S", interval=1, backupCount=3 ) timed_handler.setFormatter( logging.Formatter("%(asctime)s - %(levelname)s - %(message)s") ) logger.addHandler(timed_handler) logger.info("This message goes to the timed rotating log") print("Log file created at:", log_file) print("In production, this handler would rotate files based on time.") # Clean up logger.removeHandler(timed_handler) timed_handler.close()

Practical example¶

Let us combine what you have learned into a complete, practical function that sets up logging for a small application. This function creates a logger with both console and file output, with different levels for each.

In [ ]:

import logging
import os
import tempfile


def setup_application_logging(
    name: str, log_file: str, console_level: int = logging.WARNING,
    file_level: int = logging.DEBUG
) -> logging.Logger:
    """Set up a logger with console and file handlers.

    Args:
        name: The name for the logger.
        log_file: The path to the log file.
        console_level: Minimum level for console output.
        file_level: Minimum level for file output.

    Returns:
        A configured Logger instance.
    """
    logger = logging.getLogger(name)
    logger.setLevel(min(console_level, file_level))

    if not logger.handlers:
        console_handler = logging.StreamHandler()
        console_handler.setLevel(console_level)
        console_handler.setFormatter(
            logging.Formatter("%(levelname)s: %(message)s")
        )

        file_handler = logging.FileHandler(log_file)
        file_handler.setLevel(file_level)
        file_handler.setFormatter(
            logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
        )

        logger.addHandler(console_handler)
        logger.addHandler(file_handler)

    return logger


# Use the function
log_dir = tempfile.mkdtemp()
log_path = os.path.join(log_dir, "application.log")

app_logger = setup_application_logging("my_app", log_path)

app_logger.debug("Debug information (file only)")
app_logger.info("Info message (file only)")
app_logger.warning("Warning message (console and file)")
app_logger.error("Error message (console and file)")

# Show file contents
for handler in app_logger.handlers:
    handler.flush()

print("\n--- File contents ---")
with open(log_path, "r", encoding="utf-8") as f:
    print(f.read())

# Clean up
for handler in app_logger.handlers[:]:
    handler.close()
    app_logger.removeHandler(handler)

import logging import os import tempfile def setup_application_logging( name: str, log_file: str, console_level: int = logging.WARNING, file_level: int = logging.DEBUG ) -> logging.Logger: """Set up a logger with console and file handlers. Args: name: The name for the logger. log_file: The path to the log file. console_level: Minimum level for console output. file_level: Minimum level for file output. Returns: A configured Logger instance. """ logger = logging.getLogger(name) logger.setLevel(min(console_level, file_level)) if not logger.handlers: console_handler = logging.StreamHandler() console_handler.setLevel(console_level) console_handler.setFormatter( logging.Formatter("%(levelname)s: %(message)s") ) file_handler = logging.FileHandler(log_file) file_handler.setLevel(file_level) file_handler.setFormatter( logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") ) logger.addHandler(console_handler) logger.addHandler(file_handler) return logger # Use the function log_dir = tempfile.mkdtemp() log_path = os.path.join(log_dir, "application.log") app_logger = setup_application_logging("my_app", log_path) app_logger.debug("Debug information (file only)") app_logger.info("Info message (file only)") app_logger.warning("Warning message (console and file)") app_logger.error("Error message (console and file)") # Show file contents for handler in app_logger.handlers: handler.flush() print("\n--- File contents ---") with open(log_path, "r", encoding="utf-8") as f: print(f.read()) # Clean up for handler in app_logger.handlers[:]: handler.close() app_logger.removeHandler(handler)

Exercises¶

Try these exercises to reinforce what you have learned.

Exercise 1: Basic file logging¶

Create a logger called "exercise_logger" that writes INFO level and above to a temporary file. Log three messages, then read and print the file contents.

In [ ]:

# Exercise 1: Solution
import logging
import os
import tempfile

log_dir = tempfile.mkdtemp()
log_file = os.path.join(log_dir, "exercise1.log")

logger = logging.getLogger("exercise_logger")
logger.setLevel(logging.INFO)

handler = logging.FileHandler(log_file)
handler.setFormatter(logging.Formatter("%(levelname)s - %(message)s"))
logger.addHandler(handler)

logger.info("Application started")
logger.warning("Disk space is low")
logger.error("Failed to connect to server")

handler.flush()
with open(log_file, "r", encoding="utf-8") as f:
    print(f.read())

logger.removeHandler(handler)
handler.close()

# Exercise 1: Solution import logging import os import tempfile log_dir = tempfile.mkdtemp() log_file = os.path.join(log_dir, "exercise1.log") logger = logging.getLogger("exercise_logger") logger.setLevel(logging.INFO) handler = logging.FileHandler(log_file) handler.setFormatter(logging.Formatter("%(levelname)s - %(message)s")) logger.addHandler(handler) logger.info("Application started") logger.warning("Disk space is low") logger.error("Failed to connect to server") handler.flush() with open(log_file, "r", encoding="utf-8") as f: print(f.read()) logger.removeHandler(handler) handler.close()

Exercise 2: Dual output¶

Create a logger with two handlers: a StreamHandler at ERROR level and a FileHandler at DEBUG level. Log messages at all five levels. Verify that only ERROR and CRITICAL appear on the console, but all messages are in the file.

In [ ]:

# Exercise 2: Solution
import logging
import os
import tempfile

log_dir = tempfile.mkdtemp()
log_file = os.path.join(log_dir, "exercise2.log")

logger = logging.getLogger("exercise2_logger")
logger.setLevel(logging.DEBUG)

console_handler = logging.StreamHandler()
console_handler.setLevel(logging.ERROR)
console_handler.setFormatter(logging.Formatter("CONSOLE: %(levelname)s - %(message)s"))

file_handler = logging.FileHandler(log_file)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(logging.Formatter("FILE: %(levelname)s - %(message)s"))

logger.addHandler(console_handler)
logger.addHandler(file_handler)

logger.debug("Debug message")
logger.info("Info message")
logger.warning("Warning message")
logger.error("Error message")
logger.critical("Critical message")

file_handler.flush()
print("\n--- File contents (all five levels) ---")
with open(log_file, "r", encoding="utf-8") as f:
    print(f.read())

logger.removeHandler(console_handler)
logger.removeHandler(file_handler)
console_handler.close()
file_handler.close()

# Exercise 2: Solution import logging import os import tempfile log_dir = tempfile.mkdtemp() log_file = os.path.join(log_dir, "exercise2.log") logger = logging.getLogger("exercise2_logger") logger.setLevel(logging.DEBUG) console_handler = logging.StreamHandler() console_handler.setLevel(logging.ERROR) console_handler.setFormatter(logging.Formatter("CONSOLE: %(levelname)s - %(message)s")) file_handler = logging.FileHandler(log_file) file_handler.setLevel(logging.DEBUG) file_handler.setFormatter(logging.Formatter("FILE: %(levelname)s - %(message)s")) logger.addHandler(console_handler) logger.addHandler(file_handler) logger.debug("Debug message") logger.info("Info message") logger.warning("Warning message") logger.error("Error message") logger.critical("Critical message") file_handler.flush() print("\n--- File contents (all five levels) ---") with open(log_file, "r", encoding="utf-8") as f: print(f.read()) logger.removeHandler(console_handler) logger.removeHandler(file_handler) console_handler.close() file_handler.close()

Exercise 3: Rotating file handler¶

Create a logger with a RotatingFileHandler that limits each file to 300 bytes and keeps two backup files. Write 15 log messages and then list the files in the log directory to see the rotation in action.

In [ ]:

# Exercise 3: Solution
import logging
import logging.handlers
import os
import tempfile

log_dir = tempfile.mkdtemp()
log_file = os.path.join(log_dir, "rotating_exercise.log")

logger = logging.getLogger("rotating_exercise")
logger.setLevel(logging.DEBUG)

handler = logging.handlers.RotatingFileHandler(
    log_file,
    maxBytes=300,
    backupCount=2
)
handler.setFormatter(
    logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
)
logger.addHandler(handler)

for i in range(15):
    logger.info("Message number %s with some content", i)

print("Log files created:")
for filename in sorted(os.listdir(log_dir)):
    filepath = os.path.join(log_dir, filename)
    print("  %s (%s bytes)" % (filename, os.path.getsize(filepath)))

logger.removeHandler(handler)
handler.close()

# Exercise 3: Solution import logging import logging.handlers import os import tempfile log_dir = tempfile.mkdtemp() log_file = os.path.join(log_dir, "rotating_exercise.log") logger = logging.getLogger("rotating_exercise") logger.setLevel(logging.DEBUG) handler = logging.handlers.RotatingFileHandler( log_file, maxBytes=300, backupCount=2 ) handler.setFormatter( logging.Formatter("%(asctime)s - %(levelname)s - %(message)s") ) logger.addHandler(handler) for i in range(15): logger.info("Message number %s with some content", i) print("Log files created:") for filename in sorted(os.listdir(log_dir)): filepath = os.path.join(log_dir, filename) print(" %s (%s bytes)" % (filename, os.path.getsize(filepath))) logger.removeHandler(handler) handler.close()

Summary¶

In this tutorial, you learned the following:

• logging.FileHandler writes log messages to a file
• A logger can have multiple handlers, each with its own formatter and level
• A common pattern is to use a StreamHandler for the console (at a higher level) and a FileHandler for files (at a lower level)
• logging.handlers.RotatingFileHandler rotates log files when they reach a specified size
• logging.handlers.TimedRotatingFileHandler rotates log files based on time intervals
• Always clean up handlers when you are done (close and remove them from the logger)

Next tutorial: Continue to Debugging with pdb to learn how to use the built-in Python debugger for stepping through code and inspecting variables.